What Is a Cache Stampede?

Caches work because one expensive computation gets stored and reused by thousands of cheap lookups. A cache stampede breaks that arrangement at the exact moment it matters most. When a hot key expires, every request that wanted it now finds nothing in the cache, so each one independently decides to go to the database and recompute the value. You get a thundering herd of identical, redundant work all landing on the backend in the same few milliseconds.

The danger scales with traffic. A key serving 50,000 requests per second might normally cause one database query every few minutes. The instant it expires, all 50,000 requests in that window become 50,000 database queries. The database, which was sized for the cached load, suddenly sees its real load spike by orders of magnitude.

Other names you will hear for the same problem are dog-piling, thundering herd, and cache miss storm. They all describe the same failure mode: a synchronized cache miss converting cheap reads into a burst of expensive recomputation.

Request coalescing (also called single-flight) is the most common fix. When a request finds a cache miss, it tries to acquire a short-lived lock on that key. Only the winner recomputes the value and writes it back; everyone else waits briefly and then reads the freshly populated cache. Go's singleflight package and many HTTP caches implement exactly this. The backend sees one query instead of thousands.

Probabilistic early expiration spreads the work out in time instead of serializing it. Instead of every request treating the entry as valid until the hard TTL and then all missing at once, each request rolls a dice that gets more likely to trigger a recompute as the expiry approaches. One lucky request refreshes the value early, before it expires, while the cache still serves the old value to everyone else. The XFetch algorithm from a 2015 paper formalizes this.

Serving stale while revalidating is the other workhorse. Keep the old value available past its logical expiry, return it immediately to readers, and kick off an asynchronous refresh in the background. HTTP does this with the stale-while-revalidate cache-control directive, and CDNs like Cloudflare and Fastly support it directly. Readers never block on a cold backend.

Two supporting tactics help a lot: add random jitter to every TTL so keys never expire in lockstep, and warm hot keys proactively on a schedule so they are refreshed before they can ever expire under live traffic.

NGINX has a built-in single flight. With proxy_cache_lock on, only one request at a time may fetch a missing item from the upstream server. Other requests for that item wait until it lands in the cache, or until proxy_cache_lock_timeout passes (5 seconds by default). After the timeout a waiting request goes to the upstream, but its response is not cached. The lock is off by default, so you have to turn it on.

NGINX can also serve stale while one refresh runs. The updating option of proxy_cache_use_stale lets it return the expired copy while the item is being refreshed, and proxy_cache_background_update starts that refresh as a background subrequest. HTTP caches and CDNs follow a related standard, RFC 5861. A header such as Cache-Control: max-age=600, stale-while-revalidate=30 allows a cache to serve a stale response for up to 30 extra seconds while it revalidates in the background. stale-if-error=1200 lets it keep serving the stale copy for up to 1200 seconds when the origin returns server errors.

Match the fix to the symptom. Single flight removes duplicate loads of one hot key. TTL jitter helps when many keys were filled at one moment, for example after a deploy, and so expire together. Serving stale removes the wait for readers, but the reload still happens. The cache stampede prevention lesson measures each fix, and the thundering herd page covers the wider pattern outside caches.