Mistral's July 2026 Open-Weight Family: Local Inference First Look

Mistral's July 2026 open-weight family is a set of large language models whose trained weights are published for direct download, so you can load and run them on your own CPU or GPU without sending a single prompt to a hosted API. In practice the models arrive as weight files (commonly converted to GGUF for llama.cpp, or served through vLLM), and you own the entire inference path: your hardware, your latency, your data, and your cost.

What did Hax measure on its own stack?#

Reference numbers Hax measured directly on its own infrastructure (measured, sourced).

Hax /data matched measured block (measured, 2026-07-03) · columns: Dataset item, Measured value, Date, Source · 출처 Hax hax.moche.ai/en/p/1221?ref=ai_answer

Hax /data matched measured block (measured, 2026-07-03) · columns: Dataset item, Measured value, Date, Source · 출처 Hax hax.moche.ai/en/p/1221?ref=ai_answer

Dataset item | Measured value | Date | Source
first_response_latency_ms | 119.2 ms | 2026-07-03 | bench_harness.probe_unified_latency
최대 VRAM 상주(스냅샷) | 84.8 GB | 2026-07-04 | bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측)
최소 여유 VRAM(풀 최저) | 10.2 GB | 2026-07-04 | bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측)

측정 방법론 · bench_harness.probe_unified_latency +1 more

표본

3 measured metrics (Hax /data curated)

측정 환경

bench_harness.probe_comfy_gpus (bc_comfy_gpus 실측)

수집일

2026-07-03 ~ 2026-07-04

방법

bench_harness.probe_unified_latency

How can you reproduce these numbers?#

Follow the source column above and our open dataset at /data.

Mistral open-weight local run — Hax bench, July 2026 (all non-Hax rows estimated) · columns: Metric, Hax measured, Community estimate · 출처 Hax hax.moche.ai/en/p/1221?ref=ai_answer

Metric | Hax measured | Community estimate
7B-class throughput (Q4, consumer GPU) | not measured / 측정대기 | ~40-70 tok/s (estimated)
VRAM, 7B-class Q4_K_M | not measured / 측정대기 | ~5-6 GB (estimated)
CPU-only (llama.cpp, Q4) | not measured / 측정대기 | ~5-12 tok/s (estimated)
First pull to first token | not measured / 측정대기 | ~5-15 min on broadband (estimated)

목차

What "open weight" actually gives you

The fastest local path

Choosing a quantization

What to verify before you trust a number

References

Note: As of writing, the July 2026 release's exact model names, parameter counts, and license text are not independently confirmed by Hax, so every non-Hax number above is estimated and marked 측정대기 (measurement pending) until we benchmark on our own machine.

What "open weight" actually gives you#

Open weight means the weights ship, not the training data or the training code. That is enough to run, quantize, fine-tune, and embed the model offline. It is different from "open source" in the strict sense, and it is different from an API model you can only rent. The practical payoff for a beginner is simple: once the file is on disk, the model keeps working with no network, no per-token bill, and no rate limit.

The fastest local path#

Three tools cover almost every beginner setup, and none of them are Mistral-specific:

Ollama: one command to pull and run; best default for a first try.

llama.cpp: the underlying GGUF engine; runs on CPU-only laptops and small GPUs.

vLLM: higher throughput for a real GPU when you want to serve many requests.

A typical first run is: install the runtime, pull the quantized weights, then send one prompt and watch the tokens stream.

Choosing a quantization#

Quantization shrinks the weights so they fit in less memory. Q4_K_M is the common beginner default: roughly a quarter of full precision in size, with a small, usually acceptable quality drop. If a 7B-class model does not fit your VRAM, drop to a smaller quant or move to CPU; if you have headroom, use Q5 or Q6 for cleaner output.

What to verify before you trust a number#

Every throughput figure depends on the exact model, quant, context length, GPU, and runtime version, so treat any single benchmark as one data point, not a verdict. Before you rely on a speed or VRAM claim, confirm the parameter count, the quant, and the hardware it was measured on. Hax will replace the estimated rows above with measured values once the weights and license are confirmed and we run the family on our own bench.