Only One Thing Moved
Take ten retrieved documents. One of them holds the answer, and it is sitting sixth in the list. Ask the model which document answers the question, and it gets it right four times out of fourteen.
Now move the right document from the middle of the list to the end. Same ten documents, same question, same model at temperature zero. It gets it right fourteen times out of fourteen.
Nothing about the retrieval changed. The same retriever returned the same ten documents for the same question. Nothing about the model changed. The only difference is which seat the right answer sat in.
That is what I first thought the lesson was. Two controls I ran later, on the correction slide after "The Row You Cannot Read", showed that most of this gap came from the answer format (how the documents were numbered, and the "10" printed in the prompt), not from where the right document sat. The rest is how it was measured, how a clean and repeatable result led me to the wrong explanation, and why the highest number in the run is the one you should trust least.
Think of a teacher marking a pile of ten exam papers late at night. She reads the first one carefully and the last one carefully, because the first is where she starts and the last is where she finishes. The ones in the middle get less attention. Nobody decided that. It is just how the pile works. This lesson measures whether a small language model does something similar with a pile of ten documents, and then finds something stranger than the teacher.