Bake the Same Loaf Twice Before You Change the Recipe
Imagine you bake bread at home and you want a better loaf. You could add more yeast, bake it longer, turn the oven up, or use a different flour. The sensible way is to change one thing at a time and see what happens. But there is a step most people skip. Before you change anything, bake the same recipe twice, exactly the same way. The two loaves will not come out identical. One will rise a little higher, one will brown a little more. That difference is not caused by anything you did. It is just how baking goes.
That small, natural difference is the most useful thing to know before you test anything. If a new flour gives you a loaf a little higher than usual, but two loaves from the old recipe already differ by that much, you have learned nothing about the flour. Only a change bigger than the natural difference tells you something.
Training a model works the same way. The last lesson trained one small model once, with one set of choices, and it got 93 of 131 tickets fully right. In this lesson I go back and change those choices, one at a time. How many examples does the model need? How many times should it read them? How big should each learning step be? Should it be scored on the whole example, or only on the answer?
And first, before any of that, I trained the exact same recipe two more times, to see how much the score moves on its own. Every other result in this lesson is judged against that number.