Name: System Design Masterclass
Availability: InStock

We use analytics cookies.We use cookies to analyze site traffic and improve your experience.

Loading...

A Cache The Application Does Not Call

Every cache in this chapter so far has been something the application asked. It calls, it gets an answer, it decides what to do next. An cache sits in front, and answers before the application knows a request happened.

Two panels. Every cache so far in this chapter: the application asks, reads the answer, and decides what to do next, with the application holding the key and able to invalidate. An HTTP cache in front: the visitor asks the cache, which may never ask the app, and the app is not consulted, with the URL as the key and response headers as the only thing the app controls. — ComparisonFigure 1. A cache in front is the first thing the visitor reaches, and the application is behind it
The key is built from the URL, not from anything the application chose. Freshness comes from a header on a response the application sent earlier and is no longer being asked about.

Read the whole lesson through that inversion. Every measurement below is a consequence of the cache being in front, and none of them can happen to a cache the application calls itself.

Four Runs Against A Real Origin

The lab starts a real origin that logs every request it receives, then sends 20,000 real requests over 200 pages drawn from a Zipf distribution. The origin sends Cache-Control: public, max-age=60 on everything, which is exactly what an application is supposed to do.

A terminal capture of the Varnish lab. A first table of four runs over 20,000 requests: the origin as the framework ships it gives a 0.0 percent hit rate, 0 entries and 20,000 requests to the origin; with Set-Cookie removed it gives 99.0 percent, 200 entries and 200 to the origin; with cookies stripped but campaign tags kept it gives 95.1 percent, 982 entries and 982 to the origin; with tags stripped as well it gives 99.0 percent, 200 and 200. A second table sweeps the share of visitors carrying a Cookie header from 0 to 95 percent, with the hit rate falling from 99.0 percent to 4.3 percent and origin requests rising from 200 to 19,135. — terminal-figureFigure 2. The measurement. Four configurations, 20,000 real requests each
Every run served correct pages. Nothing errored, nothing was slow, and no application log contained anything unusual, and the origin did a hundred times more work in the first run than in the last.

A bar chart of the hit rate over 20,000 real requests. The framework as it ships gives 0.0 percent. The same with Set-Cookie removed gives 99.0 percent. — ChartFigure 3. The hit rate was zero, and removing one response header took it to ninety nine percent
Most frameworks create a session on the first response to every visitor, logged in or not, and send a Set-Cookie header with it. A shared cache is required to treat that response as belonging to one client.

The header is not wrong and the cache is not wrong. The application is telling a shared cache that this response is personal, and the cache is believing it.

A sequence diagram between a first-time visitor, the cache and the application. The visitor sends GET /article/41 with no cookie, the cache has nothing stored so it asks, the application creates a session for a visitor who did nothing, noted as the framework default, and returns 200 with Set-Cookie. The cache must not store a response with Set-Cookie, so the next visitor repeats all of it. — SequenceFigure 4. The session is created for a visitor who has not asked for one, and it kills the cache
This is a two-line change in most frameworks: do not start a session until something is put in it. Almost nobody makes it, because with no cache in front there is no visible consequence.

Look at the response headers of your own home page as a visitor with no cookies before doing anything else. If there is a Set-Cookie on it, no shared cache anywhere on the internet is storing that page, including every you are paying for.

Where this shows up in interviews

Where this shows up in interviews

Varnish Cache

The rest of this lesson

Continue this lesson

A Cache The Application Does Not Call

Four Runs Against A Real Origin