Loading...
System Design and AI Engineering, by Roni Das. 972 interactive lessons.
Start LearningAI Engineering · Foundation · LLM and GenAI Ops
Search that matches meaning instead of exact words. Text is turned into embeddings, and the results are the documents whose embeddings sit closest to the query's.
In short
Semantic search is search that finds text by its meaning instead of by the exact words it shares with your question. It turns the question and every document into embeddings, lists of numbers, and returns the documents whose numbers point in the most similar direction.
In short
Semantic search is search that finds text by its meaning instead of by the exact words it shares with your question. It turns the question and every document into embeddings, lists of numbers, and returns the documents whose numbers point in the most similar direction.
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Semantic Search across a run of lessons, not one page. These 4 alone are about 213 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
Semantic search finds text by meaning instead of by shared words. This lesson builds it over this site's 878 lessons and asks 10 questions in six languages. In English the two models were close: 6 and 4 of 10 in the top 5. Across the five other languages, the multilingual bge-m3 found 21 of 50 and the English-trained nomic-embed-text 5 of 50. Four questions were never found. Rewriting one lesson's short description moved it from rank 34-50 to rank 1 in all six languages.
Should you embed a whole document as one list of numbers, or cut it into small pieces first? Measured on this course's 112 AI lessons with 470 of their own quiz questions and bge-m3 reading up to 8,192 tokens. With one embedding per lesson, the right lesson came first for 286 questions. With 200-word pieces, for 356. Pieces won on short lessons too, not only long ones, and whole lessons still won 31 questions.
An embedding turns a piece of text into a list of numbers, so that texts with similar meaning get similar numbers. This lesson measures two real embedding models on this laptop. With the English-trained model, sentences with the same meaning scored 0.89 to 0.96 and unrelated ones 0.52 to 0.60. Then the test that matters for a global audience: the same sentence in six other languages. The English-trained model scored five of the six translations below an unrelated English sentence. A multilingual model scored all six 0.92 to 0.98.
Every customer of this shop bought products with names like 'regency cakestand 3 tier'. I turned all 5,257 names into lists of numbers twice, once with a local neural model and once with word counts, averaged them per customer, and added them to lesson 1's six features. Over 20 seeds and 26,851 real test rows, no embedding gave a gain I could tell apart from luck. A control built from random lists, which knows which products a customer bought but nothing about their names, did as well as either embedding; how far ahead or behind it looks depends on which random table you draw. The vectors did group similar products; the model did not need that.
all part of
AI Engineering
203 lessons · about 209 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Searching by Meaning: Real Semantic Search, in Six Languagespay once,
yours for life
Think of a library. The card index finds books by the exact words on each card. Ask it "why does my website feel slow" and it finds nothing, because no card says that. A good librarian works differently. She hears what you mean and points you to the right shelf.
Semantic search is the librarian. "Semantic" means "about meaning". The search tries to match what you mean, even when your words and the document's words are different.
The older approach is keyword search. It finds documents by the words they share with the question, and it uses no model. It is fast and exact, but it cannot see that "slow website" and "latency" are about the same thing.
Why does this matter? Almost every AI product that answers questions about your own documents starts with a search step. A help bot first finds the right help page. A chat assistant over company files first finds the few files that matter. Only then does a language model write the answer. If search brings back the wrong text, the model writes a confident answer from the wrong text. This pattern is called retrieval-augmented generation, or RAG.
It rests on one idea. An embedding model turns a piece of text into a long list of numbers. In the lesson, bge-m3 gave 1,024 numbers per text and nomic-embed-text gave 768. The model was trained so that texts with a similar meaning get similar lists, even when they share no words.
To compare two lists, you use cosine similarity. It is a score for how closely two lists point in the same direction. Near 1 means very close in meaning. Lower means further apart.
Semantic search then has two steps, and they happen at different times:
The expensive part, embedding the documents, happens once. Each search embeds one short question. That is why semantic search can be fast.
One rule is easy to break. The question and the documents must be embedded by the same model. Each model builds its own space of numbers, so mixing two models gives scores that mean nothing. If you change the model, you must embed every document again.
Comparing against every document is fine for thousands of documents. For millions, real systems use special indexes that skip most comparisons. These are often kept in a vector database.
Real embeddings have hundreds of numbers, so here is the same arithmetic with three. These numbers are made up. Only the method is real. The question is (2, 1, 0). Document A is (4, 2, 1). Document B is (0, 1, 3).
First, the dot product: multiply numbers in the same position, then add. Question and A give 10. Question and B give 1. Next, the length of each list: the square root of the sum of its squares. That is about 2.236 for the question, 4.583 for A and 3.162 for B.
Then divide the dot product by the two lengths multiplied together. A scores about 0.976. B scores about 0.141. A points almost the same way as the question, so A ranks first.
That is all a semantic search does. It repeats these steps for every document and sorts. The lesson also ran a small real search over ten lesson titles. The question was "How can I stop one user from sending too many requests to my API?", asked in English, Spanish and Hindi. "Rate Limiting" came first every time, at 0.749, 0.729 and 0.736. The Hindi question was never translated. The model alone linked it to the English lesson.
One good example proves little. So the lesson measured semantic search properly. It searched all 878 lessons on this site, each as a title and a one-line description. It asked 10 questions in a learner's own words, in six languages. The right answer for each question was fixed before the run, so nobody could pick it after seeing the results.
The measure was simple: is the right lesson in the top 5 results?
Keyword search also failed in a different way. When it missed, it missed badly. The slow website question landed at rank 325 of 878. bge-m3 put the same question at rank 49. Still a miss, but much nearer the top, because the model could see the lesson was related without shared words.
Why were four questions never found? Look at one. "Why does my website feel slow even when the server is fast?" should find the Latency lesson. Its indexed text was a short definition: "Time delay in system response", then "most basic performance metric". It never says website, slow or feel.
The lesson tested whether the text was the cause. It changed only that one description, to say why a website or app feels slow. Nothing else changed. The Latency lesson moved from rank 34 to 50 up to rank 1, in all six languages.
The lesson is honest about a limit here. The new text was written knowing the question, so this is the best case. It proves the text was the cause. It does not prove every rewrite works as well.
Two more points follow. The rewrite was in English only, yet it fixed all six languages, because a multilingual model compares meaning, not words. And short labels alone are weak. With titles only, "Rate Limiting" won by just 0.005 in English and lost in Spanish. One added sentence made it win everywhere by about 0.2.
Neither one wins everywhere. Choose by what your users type.
Semantic search is not magic for badly written text. It cannot find meaning that the index text does not contain.
Most failures in the lesson came from the setup, not the model. These are the mistakes to avoid:
Start with a small test set: ten questions written before you look at any results, each with the document that should answer it. Run it again every time you change the model, the index text or the way documents are cut. The Searching by Meaning lesson gives you the runnable Python script, all ten questions and the full grid of ranks. It builds on what an embedding is, and a later lesson shows how to combine keyword and meaning search.
bge-m3 (BAAI)
A multilingual embedding model with 1,024 numbers per text. In the lesson's test it found 21 of 50 right answers outside English.
nomic-embed-text (Nomic AI)
An embedding model trained mostly on English. It expects a "search_query: " or "search_document: " label before each text.
Elasticsearch
A widely used search engine built around keyword search that can also store embeddings and search them by similarity, so one system can run both.
PostgreSQL with pgvector
The pgvector extension stores embeddings in a normal Postgres table and finds the nearest ones, a common first step before a dedicated vector database.
Semantic search finds text by meaning. In the lesson, the question "How can I stop one user from sending too many requests to my API?" found the Rate Limiting lesson first, even when asked in Hindi. The lesson text never contained the Hindi words; the embedding model matched the meaning.
Elasticsearch is a search engine, and semantic search is a method. Elasticsearch is best known for keyword search, which matches shared words. Semantic search matches meaning using embeddings. Elasticsearch can also store embeddings and search them by similarity, so it can do both, and many teams combine the two as hybrid search.
You need an embedding model, such as bge-m3 or nomic-embed-text, a way to run it, such as Ollama on your own computer, and somewhere to keep the embeddings. For a few thousand documents a plain list in Python is enough. For millions, use a vector database or a search engine with vector search.
Not always. In the lesson's English test, keyword search found 4 of 10 answers in the top 5, the same as nomic-embed-text, and bge-m3 found 6. Semantic search wins when people use different words or languages. Keyword search wins for exact codes, names and IDs.
Checked against these on .
Vector Database
Databases built for AI, storing and searching high-dimensional embeddings that power semantic search, RAG, and recommendation engines
Splitting Text Into Pieces: Why Search Works Better on Small Chunks
Should you embed a whole document as one list of numbers, or cut it into small pieces first? Measured on this course's 112 AI lessons with 470 of their own quiz questions and bge-m3 reading up to 8,192 tokens. With one embedding per lesson, the right lesson came first for 286 questions. With 200-word pieces, for 356. Pieces won on short lessons too, not only long ones, and whole lessons still won 31 questions.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
Cosine Similarity
A measure of how close two embeddings are, based on the angle between them rather than their length. The standard way to rank how related two pieces of text are in vector search.
Hybrid Retrieval
Running dense vector search and sparse keyword search together and fusing the results, so exact terms like an error code and paraphrased meaning both count.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
ANN (Approximate Nearest Neighbor)
Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.
HNSW
Hierarchical Navigable Small World, a graph-based nearest neighbor index that gives high recall at low latency by navigating layers of links, at the cost of more memory than IVF.
This page explains the idea. The "Searching by Meaning: Real Semantic Search, in Six Languages" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 203 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Searching by Meaning: Real Semantic Search, in Six Languages lessoncourse 2
Take models from notebook to production, with labs on real models.