Cache Stampede
When many requests hit the database simultaneously because a popular cache entry expired. Solved with locking, probabilistic early expiration, or request coalescing.
What is Cache Stampede?
In short
A cache stampede is what happens when a popular cache entry expires and a flood of concurrent requests all miss the cache at the same instant, so they all hit the backend database together to recompute the same value. The fix is to let only one request rebuild the value while the others wait or serve stale data, using techniques like request coalescing, locking, or probabilistic early expiration.
What a cache stampede actually is
Caches work because one expensive computation gets stored and reused by thousands of cheap lookups. A cache stampede breaks that arrangement at the exact moment it matters most. When a hot key expires, every request that wanted it now finds nothing in the cache, so each one independently decides to go to the database and recompute the value. You get a thundering herd of identical, redundant work all landing on the backend in the same few milliseconds.
The danger scales with traffic. A key serving 50,000 requests per second might normally cause one database query every few minutes. The instant it expires, all 50,000 requests in that window become 50,000 database queries. The database, which was sized for the cached load, suddenly sees its real load spike by orders of magnitude.
Other names you will hear for the same problem are dog-piling, thundering herd, and cache miss storm. They all describe the same failure mode: a synchronized cache miss converting cheap reads into a burst of expensive recomputation.
Why it spirals into an outage
The first failure is recomputation cost. If rebuilding the value takes 200ms and 5,000 requests arrive during that window, the database now holds 5,000 concurrent queries that each do the same work. Connection pools fill up, queries queue, and latency climbs for every other query the database is trying to serve, not just the stampeded key.
The second failure is the feedback loop. As the database slows under the herd, each recompute takes longer, which means the window stays open longer, which lets even more requests pile in. Slow recomputes mean a wider stampede, and a wider stampede means slower recomputes. Without intervention this is how a single expired key takes down a whole service.
It gets worse when many keys expire together. If you set thousands of entries with the same TTL during a deploy or a bulk warm-up, they all expire at the same second and stampede simultaneously. This is why fixed TTLs are dangerous and why jitter matters.
How to prevent it
Request coalescing (also called single-flight) is the most common fix. When a request finds a cache miss, it tries to acquire a short-lived lock on that key. Only the winner recomputes the value and writes it back; everyone else waits briefly and then reads the freshly populated cache. Go's singleflight package and many HTTP caches implement exactly this. The backend sees one query instead of thousands.
Probabilistic early expiration spreads the work out in time instead of serializing it. Instead of every request treating the entry as valid until the hard TTL and then all missing at once, each request rolls a dice that gets more likely to trigger a recompute as the expiry approaches. One lucky request refreshes the value early, before it expires, while the cache still serves the old value to everyone else. The XFetch algorithm from a 2015 paper formalizes this.
Serving stale while revalidating is the other workhorse. Keep the old value available past its logical expiry, return it immediately to readers, and kick off an asynchronous refresh in the background. HTTP does this with the stale-while-revalidate cache-control directive, and CDNs like Cloudflare and Fastly support it directly. Readers never block on a cold backend.
Two supporting tactics help a lot: add random jitter to every TTL so keys never expire in lockstep, and warm hot keys proactively on a schedule so they are refreshed before they can ever expire under live traffic.
A concrete example
Picture a news site whose homepage trending-stories block is cached in Redis with a 60 second TTL. The page gets 30,000 requests per second. For 60 seconds Redis answers all of them and the database is idle. At second 61 the key expires.
Without protection, the next batch of requests, say 6,000 of them in the 200ms it takes to run the trending query, all miss Redis and all fire the same heavy aggregation query at the database. The query is expensive, the database chokes, latency on every endpoint spikes, and the homepage may error out entirely.
With single-flight coalescing, only the first request acquires the lock and runs the query. The other 5,999 requests see the lock, wait a few milliseconds, and read the value the winner wrote back. The database sees exactly one query. With stale-while-revalidate layered on top, even that one winner does not block readers: the old trending list is served for a few more milliseconds while the refresh completes in the background. The stampede never happens.
Where it is used in production
Redis
Most commonly stampeded layer; teams add SETNX-based locks or Lua scripts so one client holds a rebuild lock per key while others wait.
Cloudflare
Edge caching with stale-while-revalidate and origin request coalescing so a cache miss on a hot URL triggers a single origin fetch, not one per visitor.
Memcached at Facebook
Facebook's leases mechanism gives one client a token to recompute a missed key while concurrent readers wait or serve stale, described in their scaling-memcache paper.
Go singleflight
Standard library extension package used across Google and many services to collapse duplicate in-flight calls for the same key into one execution.
Frequently asked questions
- What is the difference between a cache stampede and a thundering herd?
- They describe the same thing. Thundering herd is the general term for many processes waking up to compete for one resource; cache stampede is that pattern applied specifically to a cache, where many requests miss the same expired key at once and all rush the backend.
- Does setting a longer TTL fix a cache stampede?
- No. A longer TTL just makes the stampede less frequent, not less severe. The entry still expires eventually, and when it does the same flood of concurrent misses hits the backend. You need coalescing, early expiration, or serving stale to actually fix it, not a bigger TTL.
- What is request coalescing or single-flight?
- It is the technique of letting only one request recompute a missing value while all other requests for the same key wait for that single result. The backend sees one query instead of thousands. Go's singleflight package is a well known implementation.
- How does probabilistic early expiration work?
- Each read computes a probability that grows as the cached value nears its TTL, and refreshes the value early if the dice roll triggers. One request refreshes ahead of expiry while everyone else keeps reading the still-valid value, so the key never expires under full concurrent load and no synchronized stampede occurs.
- Why does adding jitter to TTLs help?
- If many keys are written together with the same fixed TTL, they all expire at the same second and stampede at once. Adding a small random offset to each TTL spreads expirations across time so no two keys go cold simultaneously, smoothing backend load.
Learn Cache Stampede hands-on
This page explains the idea. The full lesson lets you step through the ring as servers join and leave, read the implementation, and check yourself with a quiz. It is one of 760+ lessons in the System Design Masterclass, from your first API call to distributed consensus. Eleven Foundation lessons are free, no signup. Lifetime access is ₹499 in India or $7.99 worldwide, one payment, no subscription.
Related lessons
Lessons that touch on Cache Stampede as part of a larger topic.
See also
Related glossary terms you might want to look up next.
Caching
Storing frequently accessed data in a faster storage layer so you don't have to fetch it from the original (slower) source every time.
TTL
Time To Live: how long a cached entry, DNS record, or packet is valid before it expires and must be refreshed or discarded.
Cache Eviction
The process of removing entries from a cache when it's full. Common policies: LRU (least recently used), LFU (least frequently used), FIFO (first in, first out).
CDN
A network of servers distributed globally that caches content close to users. Netflix uses CDNs to stream video from servers near you, not from one central location.
Cache-Aside
A caching pattern where the application checks the cache first; on a miss, it fetches from the database and populates the cache. Also called lazy loading.
Write-Through Cache
A caching pattern where every write goes to both the cache and the database simultaneously. Ensures consistency but adds write latency.