We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
AI Engineering · Tokens And Embeddings
Free preview: the first 2 parts of this 17-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by PayPal
See the Generative AI and Agentic AI courseread the rest of this lesson in
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by PayPal
See the Generative AI and Agentic AI coursein Tokens And Embeddings
This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime AccessFoundation lessons of this course are free, no payment needed. Try them now
When you look for a book in a library, you do not read every book. The books are sorted by subject, so you walk to the right section and look there.

Every search in this chapter so far has been exact: the question is compared with every stored piece. That is fine for 1,823 pieces. With millions of pieces, comparing every one for every question is a lot of work, which is why large systems usually avoid it. This lesson measures the library trick: group the pieces first, then search only the nearest groups.
The trick has a price, and that price is the real subject of this lesson. When you skip shelves, you can miss a good book that was put on an unexpected shelf. So the question is never only "how much faster?". It is "how much faster, and how many good answers do I lose?". I measured both, on the same questions and pieces as the earlier lessons, so you can see the trade in real numbers before you meet it in a real database.
Before the method, it helps to see exactly what "a lot of work" means. In lesson 6 and after, every stored piece has an : a list of numbers that stands for its meaning. nomic-embed-text gives 768 numbers per piece. To score one piece against a question, the computer multiplies the question's 768 numbers with the piece's 768 numbers, one pair at a time, and adds the results. That is one comparison, and it is the cosine score from earlier lessons (the vectors have length 1, so this sum is the cosine).
Now count. One comparison is 768 multiplications. Exact search does one comparison per piece, so for our 1,823 pieces that is 1,823 × 768 = 1,400,064 multiplications for every question. A laptop does that in a tiny fraction of a second, which is why every earlier lesson could simply compare everything.
The count grows in a straight line with the number of pieces. Ten times more pieces means ten times more work, for every single question. This is general arithmetic, not something I measured: with one million pieces, the same sum becomes 1,000,000 × 768 = 768,000,000 multiplications per question. If many people ask questions at the same time, you pay that again for each of them.
So the goal of a faster search is simple to state: compare the question with fewer pieces, while still finding the pieces that exact search would have found. Clustered search is one way to do that. It spends some work once, in advance, so that each question can skip most of the pieces.