Local Image Generation (SDXL, Flux): VRAM and RAM, Measured

A local image model's VRAM is set not by the model alone but by the sum of VAE + text encoder + resolution-dependent working memory - unlike an LLM, it is resolution, not context (KV cache), that grows the activation memory. Three measured essentials: (1) SDXL is about 7-8GB at fp16, barely fitting an 8GB card but OOMing the moment you add ControlNet or a hires pass (12GB is comfortable). (2) Flux is 12B, so fp16 is 24-33GB, exceeding consumer GPUs - fp8 drops to 12-13GB, and GGUF Q4/NF4 to 6-8GB. (3) The hidden killer is Flux's T5-XXL text encoder (about 9GB at fp16) - used only in the conditioning pass, so offloading it to CPU saves the most. So the answer for image VRAM is not "model size" but encoder, resolution, and quantization.

In plain terms: an image model's memory is bench space. It is not just the tools (model) on the bench - the bigger the canvas (higher resolution) the more room you need, and a brush pot you rarely use (the T5 encoder) hogs space. Move that pot to the next room (CPU RAM) and the bench suddenly gets much wider.

First, the terms. The VAE is the decoder that turns the latent space back into actual image pixels. The text encoder turns the prompt into condition vectors the model understands; Flux uses the heavy T5-XXL. Activation memory is what the intermediate tensors computed during generation occupy, and it grows with resolution. Quantization (fp8, Q4, NF4) compresses weights to fewer bits, and offloading temporarily moves an unused part to CPU RAM to free VRAM.

What sets image-generation VRAM?#

Total usage of model (UNet/transformer) + VAE + text encoder + working memory. Published figures mean total VRAM during generation, not file size. And the decisive difference from LLMs: resolution grows the intermediate tensor (activation) memory (2048² is far heavier than 1024²). So it is not "the model is 6GB, so 6GB is fine" but a calculation including resolution, batch, and encoder. Press high resolution down with VAE tiling and big encoders down with offloading.

See how the four pieces fill total VRAM, and which piece resolution grows.

Why are SDXL and Flux so different?#

Because the parameter scale is 2.6B vs 12B, over 4x. SDXL (2.6B UNet) already fits consumer GPUs at fp16, so quantization barely helps (fp16 is already 7-8GB). Flux (12B), conversely, is the main beneficiary of quantization - fp8 halves its 24-33GB, and GGUF Q4/NF4 bring it to 6-8GB. On quality, diffusion models make fp8 practically indistinguishable from fp16, while Q4 slightly softens detail and text (Q4 is the usable floor; below Q3 anatomy and text break). So SDXL stays at fp16, and fp8 is Flux's sweet spot.

Plotting VRAM as you lower precision shows SDXL barely moves while Flux drops a lot - at 4x the scale, Flux is where quantization pays off.

How do you fit Flux on 8GB?#

GGUF Q4/NF4 plus T5-XXL offload. First, NF4 is much faster than fp8 on low VRAM - in one test an 8GB 3070Ti laptop hit fp8 8.3s/it vs NF4 2.15s/it (about 3.86x). Second, offload the T5 encoder (about 9GB) to CPU RAM so it leaves VRAM after conditioning and frees room for denoising (needs 16GB+ system RAM; use fp8 T5 under 32GB). When offloading, speed ranks NF4 > Q4_0 > fp8 > Q8 (heavy quant overhead can cost more than moving higher-precision weights CPU-to-GPU). In short: at 8GB, Q4/NF4 plus T5 offload.

See why offloading T5 rescues 8GB, on a timeline. The encoder is used only in the earlier conditioning pass, so once it is done you drop it from VRAM and hand the room to the heavy denoising.

What fits on your GPU?#

The key is budgeting with resolution and encoder included.

• 8GB: base SDXL (zero headroom, no ControlNet) or Flux Q4/NF4 + T5 offload (accept slower).
• 12GB (value): SDXL comfortable + Flux fp8/Q. 16GB: Flux fp8 single-file is exactly right (no low-VRAM gymnastics).
• 24GB: fp16, or stacking Flux + ControlNet + IP-Adapter. Measure exact values on your own resolution and workflow.

Related reading: 에이전트 브라우저 제어, 직접 써보고 느낀 점과 한계, ComfyUI란? 노드로 조립하는 이미지·영상 생성 파이프라인

Related reading: 로컬 이미지 생성(SDXL·Flux), 5분 시작 가이드, 로컬 이미지 생성 SDXL vs Flux, 직접 돌려본 속도·품질

Reference links

• FLUX (image generation model)
• ComfyUI (node-based generation engine)
• Diffusers (pipelines, CPU offload)
• SD WebUI Forge (NF4/GGUF low-VRAM Flux)
• ComfyUI-GGUF (Flux GGUF loader)

Note: GB figures are public 2026 measurements, and total usage varies by resolution, batch, encoder precision, and workflow (not permanent numbers). The Flux fp16 24 vs 33GB gap comes from whether the T5 encoder is loaded or offloaded. Measure exact memory on your own resolution, model, and quantization (these numbers are only a start). The image-generation stack moves fast, so this is reviewed quarterly.