Look At This Line First
Look at this line. Each point on it is one read: one moment when a program asked a cache for a saved copy of some data. A cache is a store of saved copies kept close by, so answers come back fast. Each copy is saved under a name, called its key. When the copy is not there, the program has to ask the slow place where the real data lives, which I will call the origin. The height of each point is how long that read took, in milliseconds. A millisecond, written ms, is one thousandth of a second.
Almost every read took about one millisecond. Five reads took about 200. The tall spikes come back every four seconds, like a clock ticking. The two small bumps are most likely my laptop being busy with other jobs; those two reads never asked the slow source for anything.
I made this line myself, in the lab you will run later in this lesson. One program asked a cache for the same thing, the price of product number 42, stored under the key product:42. After each read it paused for 50 milliseconds, for 20 seconds. That made 337 reads, not 400, because each read takes a moment of its own before the pause starts, and the slow reads take a lot longer. The flat part of the line is the cache doing its job: the answer was already stored, so it came back in about a millisecond.
Now look at the spikes. They are tall, they are sharp, and they are evenly spaced. Every four seconds, one unlucky read took about 200 milliseconds instead of one. That is two hundred times slower. If this were a real shop page, that one visitor would feel the page stall.
Why every four seconds? Because I told the cache to forget the price after four seconds. That setting is called the time to live. When the stored copy is forgotten, the next reader finds nothing, so it has to go and fetch the price from the slow place where the real data lives, wait for it, and put a fresh copy back. Then the line is flat again for another four seconds, until the copy is forgotten again. The spikes are the forgetting.
Here is the same reading program, the same key, the same 20 seconds. I changed one thing in the cache code.
One spike at the very start, when the cache was empty, and then nothing. The cache still forgot the price on schedule. It simply fetched a new price a little before that moment, behind the readers' backs, so no reader ever found the shelf empty.
The change is this. When a reader comes along and the stored price is getting old, past the halfway point of its four seconds, the cache hands the reader the price it already has, and at the same moment starts fetching a fresh one in the background. The reader does not wait. By the time the old copy would have been forgotten, a new copy is already there.
That trick has a name. It is called refresh-ahead, and here is the short definition. Refresh-ahead is a pattern that reloads a cached value shortly before its time to live runs out, using a read as the signal, so that readers keep getting answers from the cache instead of waiting for the origin. A cache that does it is a refresh-ahead cache.
Notice the note under the chart in Figure 2. The origin was asked 10 times instead of 5. Refresh-ahead did not make the work disappear. It moved the work to a moment when nobody was waiting, and it did the work more often. That trade, fewer waits for more work, is the whole subject of this lesson.