Local Coding Models: How Much VRAM Does Your GPU Need?

A local coding assistant's VRAM has two faces, since a small FIM model for cursor-side autocomplete and a big model for chat and agents use different budgets. Three measured essentials: (1) at Q4, roughly 0.5-0.6GB per billion, so Qwen2.5-Coder 7B is about 5GB (8GB card), 14B about 8.7GB (12GB), 32B 18-20GB (24GB). (2) Coding's real killer is long context - to hold files and repos, context grows and the KV cache explodes, and Ollama's default 2048 tokens cannot even hold a 500-line file. (3) Code is syntax-sensitive, so you must be more careful with quantization than general chat (one bracket or quote breaks the output). So coding VRAM is set not by "model size" alone but by use, context, and quant tier.

In one line: coding VRAM is set by use (FIM vs chat), context length, and quant tier - not model size alone; budget about 0.5-0.6GB per billion at Q4, and treat Q5+ as coding's safe line.

In plain terms: a coding assistant is two kinds of driving aid. One keeps you in the lane in real time (a small FIM model, always on); the other plans the whole route (a big model). They differ in when they are needed and what they consume.

Why does coding VRAM have two faces?#

Because autocomplete (FIM) and chat/agents are different jobs. FIM (fill-in-the-middle), which sees before and after the cursor and fills the gap, lives on low latency, so a small, fast model (7B class, resident) fits - Codestral, the FIM specialist, measures 95.3% FIM accuracy. Multi-file refactors and agents, conversely, want a big, long-context model (14-32B). So IDE integrations like Continue.dev assign separate autocomplete and chat models (two, two purposes). With spare VRAM you keep both resident.

Why is context the killer in coding?#

Because holding files and repos grows context and the KV cache scales linearly. Measured: even a 14B at Q4 with a 32K context plus q8 KV cache is about 14GB total, right at the 16GB ceiling; turning on 128K adds several GB more. Yet Ollama's default context is 2048 tokens, unable to hold a 500-line Python file, giving only the illusion of "knowing the repo" - you must raise it. If context is short, either (1) shrink the window, (2) quantize the KV to q4 (quality cost on long contexts), or (3) drop to a smaller model. Qwen2.5-Coder supports 128K, so open the context first.

Why be more careful with quantization for code?#

Because code is syntax-sensitive, so a small error breaks the output. General chat is fine at Q4, but coding breaks on a single bracket or quote. So the tiers differ: Q4_K_M is usable for chat and simple generation with occasional syntax slips on complex functions, Q5_K_M is the recommended minimum for real coding, and multi-file refactors and agents strongly prefer Q6/Q8. That is, where general LLMs default to Q4, coding's safe line is one step up (Q5+). Spend spare VRAM on context and that "one step up."

What fits on your GPU?#

The key is budgeting use, context, and quant tier together.

• 8GB: Qwen2.5-Coder 7B (autocomplete, simple chat). Keep context generous, aim for Q5.
• 12-16GB: 14B is the default (speed/quality balance); at 32K context, KV included approaches 16GB.
• 24GB: a 32B (chat) + Codestral (FIM) dual or large context. Measure exact values on your own file sizes and context.

Related reading: 에이전트 브라우저 제어, 직접 써보고 느낀 점과 한계, 오픈 음성 클로닝 파이프라인 — 우리는 이렇게 운영한다

Related reading: 로컬 음성합성(TTS) 오픈모델 — VRAM·RAM 요구량 실측, Ollama·LM Studio·llama.cpp 실행기 VRAM·RAM 실측

Reference links

• Qwen2.5-Coder (coding model, FIM)
• Continue.dev (IDE integration, autocomplete + chat)
• DeepSeek-Coder (multilingual coding)
• Ollama (local runtime, context settings)
• llama.cpp (GGUF, KV cache quantization)

Note: GB figures are public 2026 measurements, and total usage varies by context, KV precision, quantization, and workflow (not permanent numbers). Newer families like Qwen3-Coder and DeepSeek V3 have appeared, but for autocomplete and chat Qwen2.5-Coder and Codestral remain solid. Measure exact memory on your own files, context, and quantization (these numbers are only a start). Coding models move fast, so this is reviewed quarterly.