A Machine That Only Reads the Start
Picture a small machine that reads paper. You feed it a long scroll. It pulls in the first part, and the rest piles up on the floor behind it. The machine never says "that was too long". It just gives you an answer about the part it read.
models can behave exactly like this. Each one can read only so many tokens at once. Send more, and by default the extra is thrown away, with no error.
In this lesson I measure where two real models cut, what happens to a sentence placed after the cut, and the settings that move the cut.
Why should this worry you? In the last lesson, each document was one short line, so the whole of it was always read. Real documents are not like that. A policy, a manual, a long support thread or a lesson on this site can run to thousands of words. If you embed such a document as one piece, and the model reads only the first part, then every fact in the rest of it is invisible to search. A user can ask a question that matches the last page word for word, and the search will never return it.
The worst part is that nothing looks wrong. The call succeeds. You get an embedding of the right length. Your search returns results. Only the results are worse than they should be, and you have no error message to tell you why. This lesson gives you the one number that shows the problem, and the settings that fix part of it.