Local Coding Assistant Models: A 5-Minute Guide

A local coding assistant attaches to your editor in 5 minutes. The stack is four pieces - VS Code + Ollama + Continue (or Cline) + a coding model - and the thing beginners get most wrong is using two different models for autocomplete and chat. Autocomplete needs a small model that understands FIM (fill-in-the-middle) tokens (e.g. qwen2.5-coder 1.5B), while chat and edit use a larger reasoning model (7B-32B). Use one generic "instruct" chat model for both and autocomplete breaks. And your code runs entirely on your machine, so not a line leaves it.

In one line: a coding assistant is two kitchen knives. Autocomplete is a fast paring knife (a small, nimble FIM model), and refactor and explain are the big chef's knife (a large reasoning model) - try to do both with one and both come out clumsy.

Why use two models (and what is FIM)?#

Because autocomplete looks at "both sides of the cursor" and fills the middle. So only models trained on <fim_prefix>...<fim_suffix>...<fim_middle> tokens (qwen2.5-coder, starcoder2, deepseek-coder) complete cleanly. A generic chat model does not know these tokens and produces garbage. One common bug: Continue infers the FIM template from the model name, so pulling the model directly via Ollama (preserving the name) avoids a wrong-template accident. For chat, conversely, use an instruct model.

How far can your VRAM go?#

8GB runs a 7B chat plus a 1.5B autocomplete, 12GB a 14B, and 24GB a 32B or 30B-A3B. The 14B is the measured sweet spot (~89.9% HumanEval), and a 32B or Qwen3-Coder 30B-A3B (MoE, only 3B active) serves as a "real refactor" second model on a workstation. For speed, an Apple Silicon (M2 Pro+) 7B Q4 is a measured 30-50 tok/s with autocomplete latency under 350ms, while CPU-only 7B is 5-10 tok/s and sluggish, so 3B is realistic there. Rule of thumb: if ==disk + RAM + VRAM >= the quant size==, it fits on the device and you get 20+ tok/s.

Where do beginners get stuck?#

Three things: FIM, context, and expectations.

• FIM: using a generic chat model for autocomplete breaks it - use a FIM-capable coder model and pull it directly via Ollama.
• Context: turn on context providers so it sees the whole repo (one file only is like fixing an engine without the rest of the car).
• Expectations: hosted is still ahead at the very top, but ==the "local = toy" era is over== - a 32B is a real coding model and a 7B is a real autocomplete engine.

How do you do it in 5 minutes?#

Start with the easiest path.

• With Ollama, pull all three - autocomplete (1.5B), chat (7B), embeddings (nomic) - and install Continue.
• Split the roles: ==autocomplete = FIM model, chat = instruct model== (do not reuse one for both).
• Vary only the model size on the same task to see speed/quality, and switch to the big model only for heavy refactors.

Reference links

• Ollama (local runtime)
• Continue (editor integration, Apache)
• Cline (agentic coding extension)
• Qwen2.5-Coder (coding model)
• StarCoder2 (FIM coding model)

Note: score, speed, and VRAM figures are public 2026 measurements and guides and vary by model, quantization, hardware, and context. Measure exact speed on your own device with the method above (autocomplete latency drives the feel). Code stays local, which helps privacy, and since models and extensions update often, this is reviewed quarterly.