A Cache Inside the Process
A local cache is the simplest one there is. You keep the answer in a variable, and next time somebody asks, you hand back what is already there. No server, no protocol, no configuration.
Both of these are caches. The difference between them is not speed. It is whether there is one copy of the answer or one copy per instance, and that single difference produces everything surprising in this lesson.
Nearly every discussion of local is about the first row of those keys. Nearly every incident caused by local caching is about the third.
What It Buys, Measured
The script is at scripts/labs/localcache/ladder.py. It measures three things on the machine it runs on: the speed, the memory, and the disagreement.
Three separate measurements, and the third one is the reason this lesson exists. The first two are what people expect. The third is what actually decides whether a local cache is a good idea for your data.
The dict lookup is a hash and a pointer dereference. The Redis GET is a syscall, a socket write, a context switch, a read and a parse. None of that gap is network , because Redis was running on the same laptop.
The gap is not caused by being slow. Redis is extremely fast. The gap is the cost of leaving your process at all, and that is the only thing a local cache removes.
The value was an integer and two short strings. Everything above about 40 bytes is the cost of holding it in a Python process, and it is measured with the interpreter's own allocation tracker instead of estimated.
Size a local cache by measuring it. Multiplying entries by the size of the value will be wrong by a factor of several, in the direction that runs your service out of memory.
The Measurement That Changes Your Mind
Here is the part the table hides. Every instance holds its own copy, and each one filled its copy at whatever moment it first needed the value. So when a value changes, the instances do not become correct together.
With one instance the expected wait is 30 seconds, half of the 60 second , which is what everybody assumes. With sixteen instances it is 56 seconds, which is 94 percent of the TTL. The TTL did not change. The instance count did.
The system is inconsistent until the last copy runs out. Adding instances adds more draws from the same distribution, and more draws means a higher expected maximum. It is the same mathematics as waiting for the slowest of N people to arrive.
Expiry Is Not a Gentle Event
A plain cache has no idea that another request is already fetching the same key, so every concurrent miss becomes its own query. This is worst on your hottest keys, which are the ones you cached first, and it appears as a periodic spike in database load with the period of your .
Single flight is one lock per key: the first miss fetches, and everybody else waits for its result. Most cache libraries call this single flight or cache loading, and it is the most valuable thing they provide. A hand-rolled map does not have it.
Cold Starts and Two Levels
At second zero every request is a miss, so a fresh instance is slower than having no cache at all. Hot keys fill within a few seconds, which is why the curve rises steeply, and the long tail takes minutes. During a rolling deploy every instance goes through this in turn.
Putting a shared cache behind the local one means a local miss costs a round trip instead of a database query, and a restarting instance refills from Redis. The divergence window is exactly where it was, because the local copies still expire independently.
Be clear about which problem that solves. It makes misses cheap. It does not make your instances agree.
When to Use One, and How to Size It
Asking whether something is slow leads to everything. Asking whether two instances may differ excludes exactly the things that cause incidents.
None of these can go stale, because none of them changes while the process is running. That single property removes the entire problem this lesson is about, and it covers most of the cases where in-process caching genuinely pays.
Quick Check
Knowledge Check
4 questions - Score 80% to pass
You run 16 instances, each with a 60 second local cache TTL. A value changes. How long until every instance is serving the new value?
A dict lookup measured 41 nanoseconds and a Redis GET on the same machine measured 16,122. What is that gap mostly made of?
Your cache holds 50,000 entries, each a small object with an id and two short strings. How much memory should you budget?
A hot key expires and 50 concurrent requests all miss. What happens with a plain TTL cache, and what fixes it?
