Hobbyist's Hideaway LogoHobbyist's Hideaway
Technology
•15 min read

Best Local LLM for Coding in 2026 (and the GPU You Actually Need)

Share
Best Local LLM for Coding in 2026 (and the GPU You Actually Need)
On this page(10 sections)

The best local LLM for coding in late 2026 depends mostly on how much memory you have. On a 16 GB card, run gpt-oss-20b or Devstral Small. On a 24 GB card, Qwen3-Coder 30B-A3B is the one to beat. With 64 GB or more of unified memory, Qwen3-Coder-Next or gpt-oss-120b get you closest to the cloud. For the GPU itself, a used RTX 3090 is still the best value, and an RTX 5090 is the fastest single consumer card.

Below: model picks by VRAM tier, how far you can push quantization for code, why coding agents need so much context, which GPU to buy, and how to plug a local model into VS Code, Cline and Aider. I'll also be honest about where local models still fall short.

Quick Answer: Best Local Coding LLM by VRAM

Approximate picks for 4-bit quantized models (Q4_K_M or the model's native format) with enough room left for a useful context window:

Your memoryBest pickAlso tryRealistic use
8 GB VRAMQwen3.5 9B (Q4)Qwen2.5-Coder 7B for autocompleteAutocomplete, explaining code, small scripts
12 GB VRAMQwen3.5 9B (Q6) with 32K contextgpt-oss-20b with light offloadChat in the editor, single-file edits
16 GB VRAMgpt-oss-20bDevstral Small 24B (Q4), Qwen3.6 35B-A3B (~3-bit)First tier where agents are usable
24 GB VRAMQwen3-Coder 30B-A3BQwen3.6 27B, GLM-4.7-Flash, Devstral Small at Q6Agentic edits on small and medium repos
32 GB VRAM (RTX 5090)Qwen3-Coder 30B-A3B with 64K+ contextQwen3.6 27B at Q6Same models, more context, much faster
64–128 GB unified memoryQwen3-Coder-Next 80B-A3Bgpt-oss-120b, GLM-4.5-AirClosest to cloud quality you can run at home

Model names move fast. This list reflects what the community recommends in late 2026. Before you pull anything, check the exact tag and size in the Ollama library or LM Studio's model browser.

New to sizing models? Our guide to the best local AI models for low VRAM and RAM explains the memory math.

What Makes a Local Model Good at Coding?

Coding is harder on a local model than general chat. Four things matter more than leaderboard scores.

1. Reliable tool calling

Coding agents like Cline, Roo Code and Aider ask the model to call tools: read a file, search the repo, apply a diff, run a command. A model that fumbles the tool-call format will loop, hallucinate file paths or give up. Models trained for agentic coding (Qwen3-Coder, Devstral, gpt-oss) are far better at this than older "chat about code" models.

2. Context length

An agent's system prompt and tool definitions alone often run to several thousand tokens, sometimes 10K or more. Add a few source files and some terminal output and you're at 30K quickly. A model with a 4K or 8K window simply can't do agentic work.

3. Fill-in-the-middle (FIM) for autocomplete

Tab completion is a different job. It needs a small, fast model trained to fill gaps between code before and after your cursor. Big chat models are too slow for this, and many don't support FIM at all.

4. Speed, including prompt processing

Agents spend much of their time reading: processing huge prompts before the first output token appears. GPUs are very fast at this. Macs and CPUs are much slower, which is why a Mac can feel great in chat and sluggish in an agent.

The Best Local Coding Models in 2026

Qwen3-Coder 30B-A3B: the 24 GB sweet spot

A mixture-of-experts (MoE) model with about 30B total parameters but only ~3B active per token. That means it needs the memory of a 30B model but runs at the speed of a 3B one. At Q4 it's roughly 18–19 GB, so it fits a 24 GB card with room for a decent context. It was trained specifically for agentic coding and tool use, and it's Apache 2.0 licensed. For most people with a 3090 or 4090, this is the default answer.

Qwen3-Coder-Next: the big-memory pick

The larger sibling, an 80B-total, ~3B-active MoE aimed squarely at coding agents, with a very long context window. At Q4 it needs roughly 45–50 GB, so it's a Mac with 64 GB+ (ideally 96 GB+), an AMD Strix Halo box with 128 GB, or a multi-GPU rig. Thanks to the small active parameter count, it stays quick even on unified memory. Note that Ollama's library also lists cloud variants of some models, so make sure the tag you pull is a local one.

gpt-oss-20b and gpt-oss-120b

OpenAI's open-weight reasoning models (Apache 2.0). gpt-oss-20b runs in about 16 GB of memory in its native MXFP4 format, has adjustable reasoning effort, and handles tool calls well. It's the best all-rounder for 16 GB cards and 24 GB Macs. gpt-oss-120b needs around 65 GB, so it's for 96 GB+ Macs or 128 GB unified-memory boxes. Both are general models rather than code specialists, so they shine at reasoning through a bug and are a little weaker at long multi-file edits.

Devstral Small (24B)

Mistral's agentic coding model, built with the All Hands team for software engineering agents. It's a dense 24B model, around 14 GB at Q4, so it fits a 16 GB card with a modest context. It's slower than the MoE options but very consistent with tool calls. Pull the newest Devstral Small release, as Mistral updates it regularly.

Qwen3.6 and GLM-4.7-Flash: strong 24 GB alternatives

Qwen3.6 27B (dense, slower but very sharp) and Qwen3.6 35B-A3B (MoE, fast) are general models that code very well, so they're ideal if you want one model for chat and code. GLM-4.7-Flash is a 30B-class MoE (about 3B active, MIT licensed, roughly 19 GB at Q4). Try them side by side with Qwen3-Coder, because each has different strengths across languages and frameworks.

What about DeepSeek, Kimi and the giant models?

The open-weight models that genuinely rival frontier cloud models, such as the big DeepSeek releases, Kimi K2, GLM-5 and Qwen3-Coder 480B, have hundreds of billions of parameters and need hundreds of GB of memory even when quantized. For most hobbyists, the practical way to use them is a cheap API.

Quantization: How Low Can You Go for Code?

Code is less forgiving than prose. A slightly worse word choice in an email doesn't matter. A slightly wrong token in code is a syntax error or a subtle bug.

QuantizationSize vs FP16For coding
Q8_0~53%Practically lossless. Use it if it fits
Q6_K~41%Excellent; hard to tell from Q8
Q5_K_M~36%Very good, a nice middle ground
Q4_K_M~30%The practical default; fine for most coding
Q3 / IQ3~23%Noticeably more mistakes, especially in long edits
Q2~17%Avoid for code

Rules of thumb:

  • Q4_K_M is the floor for agentic coding. Below that, tool calls and diffs start breaking more often.
  • If you have to choose, a bigger model at Q4 usually beats a smaller model at Q8.
  • Don't trade away context for a higher quant. An agent with Q6 weights and 8K context is less useful than Q4 with 32K.
  • Native formats are fine. gpt-oss ships in MXFP4, the format it was trained for.
  • "Dynamic" quants (such as Unsloth's UD versions) keep sensitive layers at higher precision, often a free quality boost at the same size.

Context Length: Why Coding Agents Eat Memory

Every token in the context window needs space in the KV cache. For a model like Qwen3-Coder 30B-A3B, that works out to roughly 3 GB per 32K tokens at full precision. Different architectures vary, but plan for several extra GB on top of the model weights.

TaskContext you want
Tab autocomplete4K–8K
Chat about a file or function16K–32K
Coding agents (Cline, Roo, Aider)32K minimum, 64K+ comfortable

Set the context window (the step everyone misses)

Depending on the version and how much VRAM you have, Ollama's default context window is often much smaller than an agent needs. If you skip this, the model silently forgets the start of the conversation, including the tool instructions, and the agent falls apart.

# Ollama: set a default for every model (systemd: add it with `sudo systemctl edit ollama.service`)
OLLAMA_CONTEXT_LENGTH=65536 ollama serve

# Halve KV-cache memory with almost no quality loss
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 OLLAMA_CONTEXT_LENGTH=65536 ollama serve

Or bake it into a named model with a Modelfile:

FROM qwen3-coder:30b
PARAMETER num_ctx 65536
ollama create qwen3-coder-64k -f Modelfile

In LM Studio, set Context Length in the model's load settings before loading it, and enable flash attention there too. Then confirm with ollama ps (or LM Studio's resource readout) that everything is still 100% on the GPU. If context pushes layers onto the CPU, speed drops dramatically.

Not sure which runner to use? Our Ollama vs LM Studio comparison covers the trade-offs.

Best GPU for Local LLM Coding

Open PC case with a large graphics card next to a monitor showing a code editor
For local coding models, VRAM capacity and memory bandwidth matter more than raw gaming performance.

For local AI, VRAM decides what you can run, and memory bandwidth decides how fast it runs. Gaming benchmarks are almost irrelevant.

GPU / systemMemoryBandwidthVerdict for coding
RTX 3060 12 GB (used)12 GB360 GB/sCheapest useful card; autocomplete and chat
Intel Arc B58012 GB456 GB/sGood value; software support improving
RTX 5060 Ti 16 GB16 GB448 GB/sBest new budget card for gpt-oss-20b
Radeon RX 9070 XT16 GB~640 GB/sFast for the money; best on Linux
RTX 3090 (used)24 GB936 GB/sBest value for local coding
Radeon RX 7900 XTX24 GB960 GB/s24 GB for less; more setup on AMD
RTX 409024 GB~1,000 GB/sFast, but rarely worth the premium over a 3090
RTX 509032 GB~1,790 GB/sFastest consumer card; room for long context
Mac with M4 Pro (48–64 GB)Unified273 GB/sBig models, quiet, slower prompt processing
Mac Studio (M4 Max / M3 Ultra)Up to 128–512 GB410–819 GB/sRuns the 80B–120B class comfortably
Ryzen AI Max+ 395 (128 GB)Unified~256 GB/sCheapest way to 100 GB+ of model memory

NVIDIA: the path of least resistance

Every tool supports CUDA on day one. If you want things to just work, buy NVIDIA. Aim for 16 GB as a minimum for coding, because 8 GB cards can only run the small models that struggle with agents.

The used RTX 3090: still the sweet spot

Six years after launch, the 3090 remains the community favourite. 24 GB of VRAM and nearly 1 TB/s of bandwidth means Qwen3-Coder 30B-A3B fits with a usable context and runs fast. The catches: it draws around 350 W, it's a big triple-slot card, and you're buying used, so check the fans and thermals. Two 3090s (48 GB) is a popular next step: enough for 70B-class models at Q4, or Qwen3-Coder-Next at a lower quant. Our budget PC build guide covers PSU and case sizing if you're building around one.

AMD: great value, a little more tinkering

Radeon cards offer more VRAM per dollar. On Linux with ROCm, or using the Vulkan backends in LM Studio and llama.cpp, they work well for inference. Expect occasional extra setup, and check that your exact card is supported before you buy.

Apple Silicon: capacity over speed

A Mac with 64 GB or more of unified memory can run models no consumer GPU can fit, silently and at a fraction of the power. MoE models like Qwen3-Coder-Next are a perfect match. The downside is prompt processing: big agent prompts take noticeably longer to ingest than on a high-end NVIDIA card. For heavy agent use, prefer a Max or Ultra chip over a base or Pro.

AMD's Ryzen AI Max+ 395 ("Strix Halo") boxes with 128 GB, and NVIDIA's DGX Spark, offer similar memory capacity on Linux, with bandwidth close to an M4 Pro. They're also best with MoE models.

Wiring a Local Model Into Your Editor and Agents

The trick that makes all of this work: Ollama and LM Studio both expose an OpenAI-compatible API. Almost every coding tool can talk to it.

RunnerBase URL
Ollamahttp://localhost:11434 (native) or http://localhost:11434/v1 (OpenAI-compatible)
LM Studiohttp://localhost:1234/v1

Continue (VS Code and JetBrains)

Continue is the easiest way to get local chat, edit and autocomplete in your editor. A minimal config.yaml using Ollama, with a big model for chat and a tiny one for tab completion:

name: Local Coding
version: 1.0.0
schema: v1
models:
  - name: Qwen3 Coder 30B
    provider: ollama
    model: qwen3-coder:30b
    roles:
      - chat
      - edit
      - apply
  - name: Qwen2.5 Coder 1.5B
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles:
      - autocomplete

For autocomplete, small FIM-trained models such as Qwen2.5-Coder 1.5B or 7B are still the go-to choice. Latency matters more than intelligence here.

Cline and Roo Code

Both are VS Code agents that read files, propose diffs and run terminal commands. In settings, choose Ollama or LM Studio as the API provider, enter the base URL, and pick your model.

Tips for local models:

  • Raise the context window first (see above). This fixes most "the agent keeps looping" complaints.
  • Use a model trained for agents: Qwen3-Coder, Devstral or gpt-oss.
  • Keep tasks small and specific ("add input validation to parse_config") rather than "refactor the app".
  • If your tool has a compact or local-model prompt option, turn it on to save context.

Aider

Aider is a terminal pair-programmer that edits files and makes git commits. It works well with local models:

# Ollama
export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/qwen3-coder:30b

# LM Studio
export LM_STUDIO_API_BASE=http://localhost:1234/v1
export LM_STUDIO_API_KEY=dummy
aider --model lm_studio/qwen3-coder-30b

Aider warns that Ollama silently truncates prompts that exceed the context window, so set OLLAMA_CONTEXT_LENGTH as shown above.

Everything else

Any tool with an "OpenAI-compatible" provider works the same way: point it at the base URL and use any string as the API key. Recent Ollama versions also include an ollama launch command that configures popular coding CLIs to use a local model.

To share one model across the household, run Ollama on a server and point every editor at it. Our Ollama and Open WebUI guide covers exposing it safely on your network.

Honest Reality Check: Local vs Cloud Coding Models

Local coding models in 2026 are genuinely good, much better than a year ago. But let's be straight about where they stand.

Where local models are great:

  • Autocomplete, boilerplate, docstrings and tests
  • Explaining unfamiliar code
  • Single-file edits, small features, scripts and config files
  • Code you can't send to a cloud service, with no rate limits or monthly bill

Where frontier cloud models still win clearly:

  • Large, multi-step agentic tasks across many files ("implement this feature end to end")
  • Long sessions where the agent must stay on track for dozens of tool calls
  • Tricky debugging across a big codebase
  • Speed: hosted models process huge prompts far faster than a home GPU

Even the best model that fits on a 24 GB card trails the top cloud models on big agentic jobs, and no amount of prompt tweaking fully closes that gap.

The setup most hobbyists land on is a hybrid: a local model for autocomplete, quick questions and private code, plus a cloud model for the occasional big job. Local is also a great fallback when you hit a rate limit or lose your connection.

FAQ

What is the best local LLM for coding on 16 GB of VRAM?

gpt-oss-20b is the best all-rounder, as it fits in about 16 GB in its native format and handles tool calls well. Devstral Small 24B at Q4 is the more code-focused alternative. Keep context to around 16–32K and enable KV-cache quantization to make everything fit.

Is a used RTX 3090 still worth it for local AI in 2026?

Yes. Its 24 GB of VRAM and ~936 GB/s bandwidth still make it the best value card for local LLMs, and it runs Qwen3-Coder 30B-A3B comfortably. Just budget for a strong PSU and check a used card's condition carefully.

Can a MacBook run a local coding model?

Yes, if it has enough unified memory. A 24 GB MacBook runs gpt-oss-20b nicely, and 36–48 GB opens up Qwen3-Coder 30B-A3B. Expect slower prompt processing than a desktop GPU, so agents feel slower than chat does.

Is Ollama or LM Studio better for coding?

Both work with Continue, Cline, Roo and Aider. Ollama is better if you want an always-on server for several machines. LM Studio is easier for testing models and tuning context. See our full Ollama vs LM Studio comparison.

Which local model is best for code autocomplete?

A small FIM-trained model, such as Qwen2.5-Coder 1.5B (or 7B if you have spare VRAM). Speed matters more than smarts for tab completion. Run it alongside your bigger chat model.

The Bottom Line

Choose the model your memory allows, then give it enough context:

  • 16 GB: gpt-oss-20b or Devstral Small
  • 24 GB (used RTX 3090): Qwen3-Coder 30B-A3B
  • 64 GB+ unified memory: Qwen3-Coder-Next or gpt-oss-120b

Set the context window to at least 32K, wire it into Continue, Cline or Aider through the OpenAI-compatible endpoint, and keep a cloud model on hand for the big jobs. Your homelab can do more than write code, too: the same Ollama server can power a fully local Home Assistant voice assistant, and if you only have a mini PC, see what to expect from Ollama on an N100.

Share this article
Share

Never Miss a Project

Join our community of makers. Get the latest guides on Homelab, Electronics, and Coding delivered to your inbox.