BGE-M3 Multilingual Search: Judge It by p50/p95 Latency
In short: BGE-M3 is a multilingual embedding model that turns text from many languages, including Korean, into dense vectors so a local search index can rank passages by meaning instead of exact keywords. For a beginner, the honest way to judge it is not marketing throughput but perceived response speed — the p50 and p95 latency your own queries actually feel.
BGE-M3 is a multilingual embedding model that turns text from many languages, including Korean, into dense vectors so a local search index can rank passages by meaning instead of exact keywords. For a beginner, the honest way to judge it is not marketing throughput but perceived response speed — the p50 and p95 latency your own queries actually feel.
first_response_latency_ms 120.8 ms
What did Hax measure on its own stack?#
Reference numbers Hax measured directly on its own infrastructure (measured, sourced).
| Dataset item | Measured value | Date | Source |
|---|---|---|---|
| first_response_latency_ms | 120.8 ms | 2026-07-04 | bench_harness.probe_unified_latency |
| 생성 처리량 | 38.8 tok/s | 2026-07-04 | bench_harness.probe_llm_bench (unified-api 실측, 3회 중앙값) |
| 전체 생성 지연(200토큰) | 5153 ms | 2026-07-04 | bench_harness.probe_llm_bench (unified-api 실측, 3회 중앙값) |
- 표본
- 3 measured metrics (Hax /data curated)
- 측정 환경
- bench_harness.probe_llm_bench (unified-api 실측
- 수집일
- 2026-07-04
- 방법
- bench_harness.probe_unified_latency; 3회 중앙값)
How can you reproduce these numbers?#
Follow the source column above and our open dataset at /data.
Hax HTTP response P95 (7-day) = 625 ms
| metric | value | label |
|---|---|---|
| Hax HTTP response P95 (7-day) | 625 ms | measured 2026-07-24 (Hax telemetry) |
| Hax unified first-response latency | 119.2 ms | measured 2026-07-03 (bench_harness) |
| Hax active retrieval records | 10585 | measured 2026-07-24 (curator stats) |
| BGE-M3 query embedding p50 on a modern CPU | 50-150 ms | estimated |
Note: the Hax rows above are operational values, not synthetic model claims. The server-measured figures form what we call the Hax Local-AI Latency Index[/glossary#hax-latency-index] — a single, repeatable number you can re-check rather than trust. Keep the measured Hax rows separate from the estimated BGE-M3 embedding row, which will vary with your CPU, batch size, and quantization.
Why p50 and p95, not the average#
An average hides the slow tail. p50 is the median query — half are faster, half slower — and answers 'does normal use feel snappy?' p95 is the 95th percentile: only one query in twenty is worse. Beginners feel the tail, not the mean, because a single 2-second stall is more memorable than twenty 120 ms hits. Hax's own front-door P95 over seven days is 625 ms (measured 2026-07-24), while the unified first-response probe measured 119.2 ms on 2026-07-03 (throughput tok_per_s_est=8.4, estimated). Report both, and always label which is measured.
Five-minute quickstart#
- Install an inference runtime and pull BGE-M3. It supports dense, sparse, and multi-vector retrieval in one model, which is why it handles Korean queries without a separate language pipeline.
- Embed your corpus once and store the vectors. A modest retrieval set — Hax runs 10585 active records (measured 2026-07-24) — fits comfortably in memory-backed search.
- Embed the incoming query with the same model. Mixing a different query encoder is the most common beginner recall bug.
- Rank by cosine similarity and return top-k.
- Time it. Fire 200+ Korean and mixed-language queries, sort the latencies, and read off p50 and p95. That sorted list is your verdict.
Korean query quality#
BGE-M3 keeps Korean recall close to English because it was trained multilingually; you rarely need translation. But quantized builds trade a little recall for speed, so measure both latency and top-k relevance before you settle. If p95 drifts up as your corpus grows, add an approximate-nearest-neighbor index before reaching for a bigger machine.
Note: all Hax figures are measured through 2026-07-24; BGE-M3 embedding times are estimated and depend on your hardware — always re-benchmark locally. See Hax data for the underlying telemetry.
Related reading: BGE-M3 다국어 검색 초보 5분 설치 퀵스타트와 실패 지점, BGE-M3 다국어 검색, 월 비용·GPU 시간으로 판단하기
Full guide: 노트북에서 돌리는 AI 모델, 흔한 함정과 해결법
Responses
No responses yet. Be the first to respond.