Local Multimodal (VLM) VRAM: The OOM Culprit Is Image Tokens

A local multimodal (image+text, VLM) model's VRAM starts with model weights plus a vision tower, but the real killer is "image tokens." Three measured essentials: (1) 7B VL weights are about 14.5-17GB at fp16, 4-7GB at INT4 (a bit more than text-only because of the vision encoder). (2) The real bomb is the tokens one image creates - Qwen2.5-VL uses dynamic resolution, so an image takes 4-16384 tokens, and at max resolution (a full-page OCR scan) 16K tokens ≈ 2GB of KV cache attach to a single image. (3) So the cause of OOM is image size, not model size - even a 3B model OOMs a 16GB GPU on one big image. The lever is not model quantization but max_pixels (a resolution cap) and frame count.

In one line: VLM memory is a pile of photos on a desk. The brain (model) is a fixed size, but put a big photo (high resolution) or stack many images or video and the desk fills up instantly. Shrinking the photos before you place them (a resolution cap) is the strongest saving.

First, the terms. The KV cache is memory holding the intermediate state of tokens the model already processed (it grows with context length); image tokens are the units an image is chopped into for the model to "read"; and max_pixels is the cap on how many pixels of an image it sees (a token budget).

What sets multimodal VRAM?#

Weights + vision tower + the KV cache of image and text tokens. The language backbone (7B) is the fixed cost (fp16 14.5-17GB, INT4 4-7GB), and the vision encoder (ViT) has modest overhead thanks to window attention. The real variable is the KV cache, about 0.125GB per 1,000 tokens (fp16). But in a VLM, image tokens are added to that count - the decisive difference from an LLM that counted only text. So it is not "the 7B is 6GB, so 6GB is fine" but a calculation including image resolution, count, and video frames.

Why does one image blow up memory?#

Because dynamic resolution creates tokens in proportion to pixels. Qwen2.5-VL's default token range is 4-16384 per image, far wider than people expect, so without a cap one 4K screenshot or document scan eats 16K tokens and adds about 2GB of KV cache. In practice, even a 3B model OOM'd on a T4 (16GB) failing to allocate 12GB "because the image was huge" (fix: shrink the image). So capping the token budget with min_pixels/max_pixels is lever #1 (a 256-1280 range is recommended). Feed several document pages and it multiplies by ~2GB each, so count how many at once.

Why is video more dangerous?#

Because tokens multiply by the number of frames. Qwen2.5-VL views video with dynamic FPS sampling, so frames x tokens-per-frame pile into context, and high-res or long video quickly passes 32K and 128K context. At 128K, the KV cache exceeds the weights, pushing total VRAM past 31GB. So for video, lower the FPS and resolution to curb tokens, and pair with FP8 KV cache (half) and FlashAttention (essential above 32K). That is, for video VLMs, token-budget design, not "model choice," decides success.

How do you fit it on 8GB?#

The key is quantize the weights, cap the tokens.

• Weights: fit a 7B into 8GB with INT4 (4-7GB), vision tower included.
• Tokens: cap image resolution with max_pixels and shrink max-model-len (the 32K default is excessive). Limit document page count.
• KV: halve it with FP8 KV cache, and use FlashAttention for long context. Measure exact values on your own image resolution and count.

Related reading: 에이전트 브라우저 제어, 직접 써보고 느낀 점과 한계, ComfyUI로 이미지·영상 만들기: 우리가 직접 굴리며 잰 운영 회고

Related reading: 로컬 멀티모달(VLM) 모델, 직접 돌려본 속도·품질 비교, 로컬 음성합성(TTS) 오픈모델 — VRAM·RAM 요구량 실측

Reference links#

• Qwen2.5-VL (vision-language model)
• vLLM (VLM serving, KV quantization)
• Qwen2.5-VL-7B (model card, Hugging Face)
• llama.cpp (GGUF, vision, mmproj)
• Transformers (multimodal pipelines)

Note: GB and token figures are public 2026 measurements and vendor estimates and vary by resolution, batch, and attention backend (not permanent numbers; precise per-token figures come from vendor benchmarks). Image tokens depend heavily on max_pixels settings, so measure exact memory on your own resolution, count, and video length (these numbers are only a start). The VLM stack moves fast, so this is reviewed quarterly.