Local Multimodal (Vision-Language) Models: Speed and Quality

Local multimodal (image+text) models now show that small beats big. Qwen2.5-VL 7B fits an 8GB GPU at about 6GB yet beats the 90B Llama 3.2-Vision on document understanding (DocVQA 95.7 vs 90.1). The 72B scores MMMU 70.2, past GPT-4o (69.1), and if you need a commercially free MIT license, InternVL3-78B (MMMU about 72.2) is the answer. So speed and VRAM point to the 7B, while top quality and license point to the large models.

In plain terms: a VLM is an LLM with eyes. It "sees" images, tables, and documents and answers in text, the opposite direction from generators like SDXL or Flux that make images.

First, the terms. A VLM (Vision-Language Model) is an AI that understands images and text together. MMMU is a set of college-level visual-reasoning problems (higher is better); DocVQA is a benchmark for reading documents, receipts, and tables to answer questions (OCR and document understanding). VRAM is the dedicated memory on your graphics card, the space that holds the model. The numbers below compare across these three axes.

How does a small model beat a big one?#

Because training quality and efficiency have caught up with size. Qwen2.5-VL 7B's DocVQA 95.7 is the number that made people take local vision models seriously: a 7B reads documents better than a 90B. It does not win everywhere, though. On general visual reasoning (MMMU), Gemma 3 12B (59.6) edges Qwen2.5-VL 7B (58.6), and fine-grained shape recognition is still weak across all open models at around 30%.

First, see how a VLM "reads" an image. The image passes through a vision encoder into hundreds of visual tokens, and those tokens enter the LLM alongside your text to produce an answer.

Plotting the MMMU scores as bars makes the quality gap between the 7B and the 72B/78B class obvious.

Why does a VLM use more VRAM than a text model?#

Because of the vision encoder and image tokens. A VLM is heavier than a text model of the same parameter count: it processes the image through a vision encoder first, and that image enters the KV cache as hundreds of visual tokens. Inference is also slower than text (generation starts only after the encoder has seen the whole image). Quantization makes it practical: a 7B is about 14GB at FP16 but about 5-6GB at 4-bit, fitting an 8GB GPU. The 72B and 78B class is about 35-40GB even at 4-bit, needing two 24GB cards or a datacenter GPU.

Quantization lowers the bit-width of the model weights (FP16 16-bit to 4-bit), cutting memory sharply. The bars below show which GPU class each model fits at 4-bit.

How do license and use case differ?#

By job. For documents, tables, and OCR, Qwen2.5-VL leads the open field (72B DocVQA 96.4, OCRBench about 888), but its Tongyi Qianwen license needs checking. If you need MIT for free derivatives and commercial use, InternVL3 delivers MMMU about 72.2 at 78B. On an 8GB consumer GPU, the all-round default is Qwen2.5-VL 7B; for light general reasoning, Gemma 3.

How do you measure it yourself?#

Measure on your own images; benchmarks are only a starting point.

• Ask questions over 20-50 of your own documents, tables, and screenshots, and measure accuracy and seconds per image.
• Run the same model at 4-bit and 8-bit to compare VRAM and the speed-vs-accuracy curve.
• Pick tooling to fit your setup: llama.cpp (VRAM efficiency), vLLM (throughput), or Ollama (ease).

Reference links

• Qwen2.5-VL (repo)
• Qwen2.5-VL technical report (paper)
• InternVL (repo)
• MMMU benchmark (paper)
• Qwen2-VL (paper)

Note: MMMU and DocVQA figures are public 2025 measurements following each model's official methodology and vary with evaluation pipeline and quantization. Measure your own images with the method above. Newer models like Qwen3-VL and Llama 4 multimodal are reshaping the top, so this is reviewed quarterly.