GLM-5.2 744B MoE on Consumer Hardware: A Reality Check
In short: GLM-5.2 744B MoE on consumer hardware is technically possible only at the very top of the prosumer range (a single large unified-memory box or a rare multi-GPU rig at aggressive 4-bit quantization), and for the typical single-GPU home PC it is not feasible today.
GLM-5.2 744B MoE on consumer hardware is technically possible only at the very top of the prosumer range (a single large unified-memory box or a rare multi-GPU rig at aggressive 4-bit quantization), and for the typical single-GPU home PC it is not feasible today. The reason is simple: a Mixture-of-Experts model reduces the compute per token by activating only a slice of experts, but it does not reduce the memory you must hold. Every expert weight has to be resident somewhere, so the full 744B parameter set dictates your footprint.
What did Hax measure on its own stack?#
Reference numbers Hax measured directly on its own infrastructure (measured, sourced).
| Dataset item | Measured value | Date | Source |
|---|---|---|---|
| 최대 VRAM 상주(스냅샷) | 84.8 GB | 2026-07-04 | bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측) |
| 최소 여유 VRAM(풀 최저) | 10.2 GB | 2026-07-04 | bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측) |
| 카드당 총 VRAM | 95.6 GB | 2026-07-04 | bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측) |
- 표본
- 3 measured metrics (Hax /data curated)
- 측정 환경
- bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측)
- 수집일
- 2026-07-04
How can you reproduce these numbers?#
Follow the source column above and our open dataset at /data.
| Setup | Weight memory at 4-bit (estimated) | Fits? | Hax status |
|---|---|---|---|
| Single RTX 5090 32GB | ~372 GB | No | not measured / 측정대기 |
| Dual RTX 5090 64GB | ~372 GB | No | not measured / 측정대기 |
| Mac Studio 512GB unified | ~372 GB + KV | Tight yes (estimated) | not measured / 측정대기 |
| 512GB DDR5 CPU offload | ~372 GB | Yes but slow (estimated) | not measured / 측정대기 |
Note: All numbers above are estimated from parameter counts, not measured on Hax hardware. Hax status is 측정대기 (measurement pending) until we run GLM-5.2 in-house.
The memory math is the whole story. Weight size roughly equals parameters times bits divided by eight. At 4-bit that is about 744B * 0.5 = ~372 GB (estimated); at 8-bit about 744 GB (estimated); at FP16 close to 1.49 TB (estimated). Add 10-20% for KV cache, activations, and runtime overhead, so a working 4-bit deployment realistically wants 400 GB+ of addressable memory (estimated).
That rules out every single consumer GPU. An RTX 5090 tops out near 32 GB; even eight 24 GB cards give only ~192 GB, still short of the ~372 GB a 4-bit copy needs, so you would need roughly sixteen high-end GPUs (estimated) plus interconnect — no longer a home build.
The two paths that actually work at home are unified memory and CPU RAM offload. A 512 GB unified-memory workstation can hold a 4-bit copy with room for context, and because MoE activates only a fraction of experts per token, generation stays usable rather than crawling. Pure CPU offload into 256-512 GB of DDR5 also fits, but memory bandwidth for streaming expert weights becomes the bottleneck; expect low single-digit tokens per second (estimated).
Bottom line: GLM-5.2 744B MoE is a big-memory model, not a big-compute one for inference. If you have 400 GB+ of unified or system memory you can run it slowly; if you only have a gaming GPU, a smaller quantized model or an API is the practical choice. We will replace every estimate here with measured numbers once Hax benchmarks the model.
Related reading: 16GB GPU에서 Gemma 4 MoE 실행 가능 여부 및 업그레이드 판단, Gemma 4 MoE 저사양 실패 원인 분석
Responses
No responses yet. Be the first to respond.