The Answer Was in the Document. Retrieval Could Not Return It.
Here is a failure that feels impossible the first time you meet it. A user asks a assistant a question the docs plainly answer, what is the maximum file size an upload accepts, and the assistant says it does not know. The sentence with the answer is right there in the upload guide, one clear line. Retrieval never brought it back, so the model never saw it, so it could not answer. Nothing was broken and no error was thrown. The fact simply fell on the wrong side of a chunk boundary.
They spend a day suspecting the model. They try a bigger one. Same result. They add a reranker to reorder the retrieved results. Same result. Nothing helps, because none of those things touch the actual problem. The sentence with the file-size limit had been cut in half when the document was chopped into pieces for the index. Half of it landed at the end of one chunk and half at the start of the next. Neither piece, on its own, reads like an answer to the question, so neither piece got retrieved.
This is the part people skip past on their way to the model and the prompt. Retrieval can only ever hand the model text that lives inside a chunk. The chunk boundaries decide what text can be retrieved together, which means they decide the ceiling on how good any answer can be. Choose them badly and there is no reranker, no prompt trick, and no frontier model that can recover a fact the boundary already split apart.
Chunking is unglamorous. It is also the first real lever you have on retrieval quality, and it is the one that quietly caps everything downstream. This lesson is about pulling it on purpose instead of by accident.