BGE-M3 multilingual search: judge it by p50/p95 latency

BGE-M3 is a multilingual, multi-granularity embedding model whose real purchasing value you should judge by perceived recall quality and by p50/p95 response latency, not by leaderboard scores alone, because a model that ranks well offline can still feel slow and imprecise on live Korean queries. Perceived speed is what a beginner actually experiences, so the pre-purchase question is never "what is the average" but "what does the tail (p95) feel like on my hardware with my language mix."

What did Hax measure on its own stack?#

Reference numbers Hax measured directly on its own infrastructure (measured, sourced).

How can you reproduce these numbers?#

Follow the source column above and our open dataset at /data.

We report our single-node first-response figure as the 'Hax Local-AI Latency Index'[/glossary#hax-latency-index], because a single measured number that you can reproduce beats a vendor claim you cannot. Note the separation in the table above: the 120.8 ms and 119.2 ms first-response numbers are measured on the local retrieval path, while the 752 ms P95 is a measured web-tier figure and is not the embedding latency — do not conflate the two when you plan capacity.

Hardware checklist. First, decide whether embeddings run on CPU or GPU; BGE-M3 is large enough that a mid-range GPU changes p95 far more than p50. Second, size RAM/VRAM for your active corpus: our recall corpus grew from 8,919 to 10,655 active memories (both measured) in three weeks, so index memory is a moving target, not a fixed line. Third, check disk and I/O for the vector index, since a cold index inflates the tail. Fourth, confirm batch size — batching lifts throughput but pushes single-query p95 up, which hurts perceived speed for one user typing one Korean question.

Software checklist. Pin the tokenizer and model revision so recall does not drift silently. Measure p50 AND p95 separately per language; Korean, English, and mixed queries do not share one tail. Warm the model before timing, or your first-request numbers are noise. Log every probe with a date and a harness name so the value is citation-ready, exactly as our measured rows are labeled. Finally, keep a rejection set of hard Korean queries and re-run it after every upgrade — recall quality is the thing latency is supposed to buy you, not replace.

Quick reproducible p50/p95 shape:

Acceptance rule: buy the setup whose measured Korean-query p95 stays inside your budget, then verify recall on the rejection set — never the reverse. Internal evidence for the numbers above: Hax data.

Note: latency and corpus figures are point-in-time measurements from early-to-mid July 2026 on one local node; re-measure on your own hardware and language mix before deciding, since the active corpus is still growing.

Related reading: BGE-M3 업그레이드, GPU 시간 비용으로 판단하기, 처음 설치하는 BGE-M3 다국어 검색, 구매 전 체크리스트

Full guide: 오픈웨이트 vs 클로즈드 VRAM·RAM 요구량 실측

References#

• Hax public measured dataset