Embeddings and Semantic Search for Beginners: Get the Prefix Right

An embedding turns a sentence into meaning coordinates (a number vector), so it finds things by sense even when not a single word overlaps. Without any LLM, "search my notes by meaning" takes five minutes. And a beginner only needs to get one thing right: add the query/passage prefix exactly as the model asks. Skip that one line and the same model's retrieval hit rate drops about 5% in measured public benchmarks.

In one line: an embedding is a ruler that plots meaning as points on a map. Similar meanings become nearby points, and search just picks the document point closest to your question point. So it matches "refund" with "get my money back" by sense, not keywords.

Here a vector is a bundle of numbers with direction and magnitude, and cosine similarity measures how similar the direction of two vectors is (that is, how close in meaning) by the angle between them. The narrower the angle, the closer the meaning.

What does semantic search consist of?#

Four pieces - load a model, encode (sentence to vector), normalize, cosine top-k - and no LLM needed. (1) Load an embedding model in one line with sentence-transformers, (2) encode your documents into vectors up front, (3) encode the question with the same model, and (4) pull the top-k documents with the highest cosine similarity. It only finds, it does not generate, so it is light and fast - a small model (about 130MB, 384 dimensions) runs even on a laptop CPU. RAG is just this with an LLM stacked on top.

What do beginners get wrong most?#

Prefix asymmetry. The e5 family was trained to put "query:" on questions and "passage:" on documents; drop it and question and document land in different spaces, so retrieval flails (a measured ~5% hit-rate drop). bge-m3 also has a retrieval instruction. The second trap is normalization - cosine similarity assumes unit-length vectors, so leaving out normalize_embeddings=True scrambles the scores. The third is encoding query and documents with the same model (mix them and meanings will not line up). Get these three right and 90% of beginner search is solved.

Which model for Korean?#

A multilingual model. English-only ones like all-MiniLM are fast but weak on Korean meaning, leaving same-sense Korean and English sentences unmatched. Pick multilingual-e5 (prefixes required) or bge-m3 (8K, MIT). Bigger models raise accuracy but grow vector dimension, storage, and latency together (e.g. 384 to 1024 dims). And the leaderboard (MTEB) is only a start - its scores skew English, so measure Korean and your own domain yourself. Start small and grow one step at a time if it falls short.

How do you do it in 5 minutes?#

Start with the easiest path.

• pip install sentence-transformers, then for Korean load multilingual-e5 or bge-m3 (avoid English-only).
• When you encode documents, turn normalization on, and for e5 always add the "passage:"/"query:" prefixes.
• Run a few questions with cosine top-k, and if it falls short change one thing at a time (model, prefix, reranker), measuring on your own documents.

Reference links#

• sentence-transformers (embedding/search library)
• MTEB (embedding benchmark)
• multilingual-e5 (multilingual embeddings)
• BGE-M3 / FlagEmbedding (multilingual, 8K)
• all-MiniLM-L6-v2 (light English embedding)

Note: dimension, size, and score figures are public 2026 measurements and model cards and vary by language, domain, and documents. MTEB skews English and does not guarantee Korean performance, so measure on your own documents (the leaderboard is only a start). Embedding models update often, so this is reviewed quarterly.