Model rankings

Best Local LLMs

The best local LLMs you can run on your own hardware in 2026 — Qwen 3.6, Gemma 4, gpt-oss-20b, Mistral Small and DeepSeek-R1 distills ranked by VRAM tier, with quantisation, hardware and tooling picks from an 8GB laptop to a 128GB unified-memory desktop.

Updated August 2026

Quick answer: The best local LLM for most people in 2026 is Qwen 3.6 27B, which runs on a single 24GB GPU at roughly 16GB in Q4_K_M quantisation and leaves room for context (MarkTechPost). On a laptop or a 12–16GB card, Gemma 4 12B is the strongest pick; for step-by-step reasoning, DeepSeek-R1-Distill-Qwen-32B (MIT-licensed) is the best fit on 24GB; and gpt-oss-20b loads in about 14GB and is the easiest capable reasoning model to run. New since 10 August: Meta’s Apache 2.0 Muse Glimmer 30B is now the best pick if you want a local agent rather than a chat model — it fits a 24GB card at ~17GB and holds its full 131K context there. The one caveat: your hardware chooses your model, not the other way around — the frontier open models (Kimi K3, DeepSeek V4, GLM-5.2) are server-class and do not fit a single consumer card.

This page is about running models on hardware you own — how much VRAM each model needs, which quantisation to pick, what a laptop versus a 24GB GPU versus a 128GB unified-memory desktop can actually run, and the tools that make it a one-command job. It is the deployment companion to our best open-source LLMs ranking, which covers licences, benchmarks and hosted inference; if you want the capability-and-licence view rather than the hardware view, start there. For the full field including proprietary frontier models, see best AI models. Every figure here is cited to a primary or independent source, and where a number is not verifiable we write “data not available” rather than guess.

How much VRAM you actually need

Whether a model fits comes down to three things competing for the same memory, and getting the split right is the whole game (MarkTechPost).

The first is the model weights, whose size is set by parameter count and quantisation. At Q4_K_M — the common home-inference default — each parameter costs roughly 0.58 bytes, so a 32B model needs about 18–20GB in weights alone, a 27B model about 16GB, a 13B model about 8GB, and a 7–8B model about 4–5GB. The second is the KV cache, which grows with context length: longer prompts and longer sessions eat more memory. The third is runtime overhead from the serving stack. A safe rule of thumb adds roughly 1–2GB for the KV cache and runtime at short context.

There is one trap worth stating plainly. Mixture-of-Experts (MoE) models are sized by their total parameters, never their active parameters (MarkTechPost). Every expert stays resident in memory even though only a few route per token, so a 35B-total, 3B-active model still occupies the memory of a 35B model. This is why GLM-5.2 (~753B total, 40B active), Kimi K2.6 (~1T total) and Kimi K3 (~2.8T total, 16 of 896 experts active per token) cannot run on a consumer card despite firing only a fraction of their weights per step.

The practical entry point is modest: 16GB of system RAM, a modern CPU, and either a GPU with 6GB or more of VRAM or an Apple Silicon Mac (Overchat AI). For most people, 8–12GB of VRAM is the sweet spot for a 7–8B model at Q4_K_M running at 40-plus tokens per second (LocalLLM.in).

The best local LLMs by hardware tier (August 2026)

The modern strategy is to run a right-sized 20B–35B model that fits cleanly with room for context, not to squeeze the biggest 70B quant onto the card (MarkTechPost). This table maps the hardware you have to the model you should run first.

Best on a laptop or 8GB GPU: a 7–9B model

At 8GB the realistic field is 7–9B models: a small Gemma 4 variant, an 8B-class Llama 4, or a compact Qwen 3.6. Each loads in roughly 4–5GB at Q4_K_M and runs at 40-plus tokens per second, which is comfortable for chat, drafting and light coding (LocalLLM.in). Apple Silicon laptops punch above this because unified memory acts as VRAM — an M-series Mac with 16GB or more can run models a same-priced Windows laptop cannot (Overchat AI).

Best on a 12–16GB GPU: Gemma 4 12B

Google DeepMind released Gemma 4 on 2 April 2026 under Apache 2.0 — the first Gemma generation with a fully open licence (MarkTechPost). Its 12B unified multimodal model is the natural pick for this tier, loading in roughly 8GB at Q4_K_M with headroom for vision input and long context. Phi-4 14B and DeepSeek-R1-Distill-Qwen-14B are the strong reasoning alternatives at the same size.

Best that needs almost nothing: gpt-oss-20b

OpenAI’s gpt-oss-20b is an Apache 2.0 open-weight reasoning model with a MoE design — 21B total parameters, about 3.6B active per token — that ships in a native MXFP4 4-bit format and loads in roughly 14GB with generous headroom (MarkTechPost). At the recommended quantisation with an 8K context it fits in about 12.7GB, so a 16GB card runs it comfortably (IntuitionLabs). It is strong on structured reasoning and tool calling and weaker on broad world knowledge — the easiest capable reasoning model to stand up locally.

Best on a 24GB GPU: Qwen 3.6 27B

A single 24GB card — an RTX 4090 or RTX 3090 — is the practical floor for serious local inference, and Qwen 3.6 27B is the strongest single default for it. It is a dense 27B model released in April 2026 under Apache 2.0, tuned for agentic coding, repository-level reasoning and frontend work, and it needs roughly 16GB at Q4_K_M with comfortable headroom for context (MarkTechPost). For visual and multilingual work on the same card, Gemma 4 26B (a MoE with ~4B active and 140-plus language coverage) is the alternative; for the deepest step-by-step reasoning, DeepSeek-R1-Distill-Qwen-32B (MIT) is the tightest fit at about 18–20GB; and Mistral Small 3.2 24B is the lightest at roughly 14GB, leaving the most room for context.

Best on a 32GB GPU: Qwen 3.6 35B-A3B

The RTX 5090 adds 8GB over the 4090 (32GB of GDDR7 on a 512-bit bus, about 1.79 TB/s of bandwidth). It does not unlock a different class of model so much as run the 24GB-tier models faster and with far more context — its bandwidth gives roughly a 1.9x speed advantage on small models. The Qwen 3.6 35B-A3B MoE (35B total, ~3B active) is the standout: it decodes far faster than a dense 35B because only a fraction of weights fire per step, while still occupying roughly 20GB of memory (MarkTechPost).

Best local agent on 24–32GB: Muse Glimmer 30B

If the job is an agent rather than a chatbot — MCP servers, tool chains, multi-step workflows that have to recover from failure — Meta’s Muse Glimmer (10 August 2026, Apache 2.0) is the strongest thing that fits a consumer card. Meta shipped two 4-bit builds sized deliberately at the two tiers above: K-Quant-17GB for 24GB cards, with about 1.0% degradation from full precision, and K-Quant-Dynamic at roughly 30GB for 32GB cards, with about 0.2% degradation — an unusually small loss that makes the 5090 build near-lossless in practice.

Two details matter for local use. Its 16:1 grouped-query attention ratio shrinks the KV cache far enough that the full 131K context fits on a 24GB card without quantising the cache — normally the first thing to break when you push context at home. And Meta ships a separate DFlash speculative-decoding drafter that proposes 16 tokens per pass: with it, an RTX 5090 goes from 74.9 to 233.4 tokens/second (3.1x), an M5 Max from 26.6 to 50.2, an M4 Max from 23.7 to 37.8. Running Glimmer without the drafter leaves most of its speed on the table.

The trade-off against Qwen 3.6 27B is clean: Glimmer leads on agentic orchestration and tool use (MCP Atlas 75.5 vs 62.5), Qwen leads on computer use and terminal work (OSWorld-Verified 75.6 vs 65.9). Both are Apache 2.0 and both fit 24GB. All Glimmer figures are Meta-reported.

Best on 48–64GB: 70B-class and near-flagship dense models

With two 24GB cards or a 64GB Mac you move into 70B-class territory at 4-bit, plus near-flagship dense coding models that do not fit a single consumer card. This is the point at which local quality gets genuinely close to hosted mid-tier APIs, at the cost of a more serious rig. Memory bandwidth starts to matter as much as capacity here, so a dual-GPU NVIDIA setup will out-token a same-capacity Mac on the models both can hold.

Best on 128GB unified memory: gpt-oss-120b and server-class MoE

A 128GB unified-memory machine is the first tier that runs the large MoE models. gpt-oss-120b is the headline fit: its MXFP4 checkpoint is about 60.8 GiB, and a clean run wants 80GB of memory, with roughly 60GB the constrained floor (YingTu). On a 128GB box it loads with room to spare. This is also the only single-device way to touch the frontier open models — see what won’t run locally below.

Quantisation: shrinking a model without breaking it

Quantisation is the lever that makes local inference possible: it trades a little quality for a large cut in memory. The naming comes from the llama.cpp GGUF ecosystem that Ollama, LM Studio and Jan all sit on top of.

The practical guidance is simple: Q4_K_M is the default, Q5_K_M and Q6_K raise quality when you have spare memory, and Q8_0 or BF16 rarely fit a 30B-class model on a single 24GB card (MarkTechPost). Some models, including gpt-oss, ship in a native 4-bit format (MXFP4) rather than a post-hoc GGUF quant, so you get the memory saving without choosing a quant yourself. One serving note that materially affects quality: prefer higher-precision serving where you can, because aggressive activation quantisation degrades output on capable models.

Hardware: what to run local LLMs on

There are two ways to buy memory for local models, and they trade off differently. A discrete GPU gives you very high bandwidth but limited capacity; a unified-memory system gives you huge capacity at lower bandwidth. Bandwidth sets your tokens-per-second on a model that fits; capacity sets whether it fits at all.

The RTX 5090 is the fastest single card for local models — 32GB of GDDR7 and about 1.79 TB/s of bandwidth make 7–13B assistants noticeably snappier — but for a large MoE that will not fit 32GB, unified memory is the only viable single-device option (Hardwarepedia). Apple’s approach shares one pool of memory across CPU, GPU and Neural Engine, so on a 128GB M-series Max there is no VRAM cliff and all 128GB is GPU-addressable at full bandwidth (XDA). AMD’s Ryzen AI Max+ 395 reaches the same 128GB unified ceiling in a thin-and-light, with up to 96GB convertible to VRAM through Variable Graphics Memory and 256 GB/s of bandwidth.

For a purpose-built desktop, NVIDIA’s DGX Spark puts a GB10 Grace Blackwell superchip and 128GB of unified LPDDR5x (273 GB/s) in a compact box, delivering about 1 petaFLOP of AI performance and roughly 200-billion-parameter inference capacity with full CUDA support. It launched at $3,999 in October 2025 and rose to $4,699 in February 2026 amid memory-supply constraints (explainx.ai). Memory bandwidth is the variable to watch across all of these: inference is bandwidth-sensitive, and a GPU’s ~1 TB/s far outpaces CPU system RAM’s tens of GB/s, which is why CPU-offloaded models feel slow (YingTu).

How to run a local LLM: the tools

Four runtimes cover almost every setup, and they share a foundation: llama.cpp is the inference library that Ollama, LM Studio and Jan all wrap, and GGUF is the common quantised format, so any GGUF model that runs in one runs in the others (Codersera). The table below is built to be affiliate-ready; we take no commission and the links are plain.

For one-developer prototyping on any operating system, start with Ollama — it is the de facto standard, with a clean CLI, an OpenAI-compatible REST API and automatic GPU offloading (Codersera). If you would rather browse and click, LM Studio is the GUI-first answer, with a model browser that recommends a quant based on your RAM and GPU. Reach for llama.cpp directly when you need a quant or architecture that has not landed in Ollama yet, and vLLM when you are serving concurrent users in production. On a Mac, MLX squeezes the most out of unified memory. If you want open-model economics without owning any hardware, hosted inference providers are covered on our best open-source LLMs page instead.

What won’t run locally (yet)

The strongest open models of 2026 are large sparse MoE systems that are excellent on benchmarks but do not run on one consumer card, because MoE memory tracks total parameters (MarkTechPost). The ceiling moved sharply in July: Moonshot’s Kimi K3 shipped open weights at 2.8 trillion parameters — roughly 594GB — making it the largest open model released and comfortably beyond any single-box setup, unified memory included.

These need multi-GPU rigs or high-memory unified systems, and for most people they are best used as an API rather than self-hosted (MarkTechPost). Their smaller distilled and dense siblings — the R1 distills, Gemma 4, Qwen 3.6 and Mistral Small 3.2 — are what bring most of the capability down to hardware you own. For how these models rank on capability and licence rather than hardware, see best open-source LLMs; for the whole field, see best AI models.

How we rank

This is a deployment ranking, so we weight what decides whether a model is usable on real hardware: memory footprint at a sensible quantisation, the quality-per-gigabyte trade-off, tokens-per-second on common cards, and how well the model is supported by the local tooling. We cross-check vendor figures against independent testing and mark a cell “data not available” rather than publish a number we cannot verify — several capable local models have well-established community performance but no single standardised figure, and we say so rather than invent one. Capability and licence rankings live on the best open-source LLMs page, which shares this plural, evidence-first method. This page is re-scored monthly, and the date at the top is the last full pass.

Frequently asked questions

What is the best local LLM in 2026?

For most people it is Qwen 3.6 27B, a dense Apache 2.0 model that runs on a single 24GB GPU at roughly 16GB in Q4_K_M with room for context, and is tuned for coding and repository-scale reasoning (MarkTechPost). On a laptop or a 12–16GB card, Gemma 4 12B is the strongest pick; for the easiest capable reasoning model, gpt-oss-20b loads in about 14GB.

How much VRAM do I need to run a local LLM?

At minimum, a GPU with 6GB of VRAM (or an Apple Silicon Mac) plus 16GB of system RAM will run a small model (Overchat AI). The comfortable sweet spot is 8–12GB for a 7–8B model at 40-plus tokens per second, 24GB for a 27–32B model, and 128GB of unified memory for the large MoE models. As a rule of thumb at Q4_K_M, a model needs roughly 0.58 bytes per parameter in weights, plus 1–2GB for the KV cache and runtime.

What is the best local LLM for a 24GB GPU?

Qwen 3.6 27B is the strongest single default at about 16GB in Q4_K_M (MarkTechPost). For visual and multilingual work choose Gemma 4 26B; for the deepest reasoning choose DeepSeek-R1-Distill-Qwen-32B (a tighter ~18–20GB fit); and for the lightest footprint with the most context headroom choose Mistral Small 3.2 24B at roughly 14GB.

Can I run a local LLM on a laptop?

Yes. A laptop with 16GB of RAM and a 6–8GB GPU runs a 7–9B model well, and an Apple Silicon MacBook does better still because unified memory acts as VRAM — a 16GB or larger M-series machine runs models a same-priced Windows laptop cannot (Overchat AI). Install Ollama or LM Studio, pull a small Gemma 4 or Qwen 3.6 build, and you are running offline in minutes.

What is the best local LLM for coding?

Qwen 3.6 27B is the best local coding model that fits a single 24GB card, built for agentic coding and repository-level reasoning (MarkTechPost). For a 48GB-plus rig, near-flagship dense coding models get closer to hosted quality. For the full coding picture including proprietary agents and cloud models, see our best AI for coding guide.

Is a Mac or an NVIDIA GPU better for local LLMs?

It depends on the model size. An RTX 5090 is the fastest single device for anything that fits its 32GB, thanks to ~1.79 TB/s of bandwidth (Hardwarepedia). But a 128GB Apple M-series Max can load large MoE models an NVIDIA consumer card cannot, because all its unified memory is GPU-addressable with no VRAM cliff (XDA). Buy the GPU for speed on models that fit; buy unified memory for capacity on models that do not.

Can I run DeepSeek V4, Kimi K3 or Kimi K2 locally?

Not on a single consumer card. Kimi K3 is the extreme case at about 2.8 trillion parameters (~594GB of weights), well beyond even a 128GB unified-memory box. DeepSeek V4’s V4-Pro checkpoint is near 1.6 trillion parameters and Kimi K2.6 is around 1 trillion, and because MoE memory tracks total parameters, all of them need multi-GPU rigs or high-memory unified systems (MarkTechPost). For local use, run their smaller distilled or dense siblings instead, and treat the full models as an API option.

What is the difference between a local LLM and an open-source LLM?

They overlap but answer different questions. “Open-source” (more precisely, open-weight) is about the licence and the model — whether you can download, inspect and legally ship it — which our best open-source LLMs page ranks. “Local” is about deployment — whether a given model actually fits and runs on your hardware, which is what this page ranks. Every local model is open-weight, but not every open-weight model is local: the biggest ones only run in a data centre.

Is Q4_K_M quantisation good enough?

For almost all local use, yes. Q4_K_M is the standard balance of quality and memory footprint and the default across the Qwen, Gemma and Llama families (MarkTechPost). If you have spare VRAM, Q5_K_M or Q6_K add a little quality; below Q4 (Q3 and Q2) degradation becomes noticeable, so only drop lower to fit a model that otherwise would not load at all.

This ranking is re-scored monthly and updated as new local-capable models ship and hardware evolves. Memory footprints vary by quantisation, context length and serving stack; figures are for Q4_K_M at short context unless stated, and we mark cells “data not available” rather than publish unverified numbers. Hardware, pricing and availability current as of 10 August 2026 and subject to change — always confirm a model’s licence with its maker before commercial use.