A Report, or a Box of Cards
Imagine a 40-page report. Someone asks you one small question about it. You can hand them the whole report, or you can hand them the one card from a card box that holds exactly that answer.
Search systems face the same choice. The last lesson showed two problems with long text: a model may cut it off, and even when it reads everything, one sentence counts for very little in one long .
The usual answer is to cut each document into small pieces before embedding, one card per idea. In this lesson I measure whether that really helps, on this course's own lessons.
Why does this matter? Nearly every system that answers questions from your own documents, the kind later chapters call , has to decide how to cut its documents. It is one of the first choices you make when you build one, and it is easy to make without thinking: many people embed each file as one piece because it is the simplest thing to do. This lesson shows, with real numbers, what that simple choice costs, and also where it is still the right choice.
There is a second reason. When the search returns a small piece instead of a whole document, you get the exact paragraph that answers the question. You can show that paragraph to the user, or pass it to a language model, instead of a whole lesson of several thousand words. So pieces can help twice: in finding the right document, and in pointing to the right part of it.
The Words You Need First
If a word below is new, read its line. Embedding, and search come from earlier lessons in this chapter.
Chunk (or piece). A short part of a document, embedded and searched on its own.
Chunking. Cutting documents into chunks before them.
Chunk size. How long each piece is. Here, 200 words.
Best piece. When a document is split, its score for a question is the score of its closest piece.
Ranked first. The right lesson was the top result.
Top 5. The right lesson was among the five highest-scoring lessons, as in lesson 4.
Median. The middle value when you sort a list. Half the values are below it and half above. Here, the median lesson length is used to split lessons into shorter and longer ones.
Quiz question. Every lesson in this course ends with a short quiz, like the one at the end of this page. The test below uses those questions as its search questions.