Name: System Design Masterclass
Availability: InStock

We use analytics cookies.We use cookies to analyze site traffic and improve your experience.

Loading...

Cacheable Is A Yes Or No. How Many Copies Is A Number

Whether a response can be cached at the edge is usually discussed as a single question, answered by Cache-Control. The more interesting question is how many copies the edge has to keep, because that is what decides the hit rate.

Two panels. The question people ask: is it cacheable, Cache-Control says public, so the edge will cache it, with keys reading decided by one header and answer yes or no. The question that matters: how many copies, Vary names the request headers, one entry per distinct value, with keys reading decided by your traffic and answer 200 or 18,404. — ComparisonFigure 1. Cache-Control decides whether it is stored. Vary decides how many times
Vary tells a cache that the response depends on a request header, so it keeps one entry per distinct value of that header. Adding a header to Vary does not stop a response being cacheable. It multiplies the key space.

The hit rate of an edge cache is a property of how many distinct keys your traffic produces, and one header in one response decides that number.

Two Hundred URLs, Four Answers

A sequence diagram between 20,000 requests, the edge cache and a real HTTP origin. 200 URLs, three encodings, 300 agents and 5,000 sessions. The key is the URL plus every header named in Vary. A miss is a real request over a real socket. The origin replies with Cache-Control public max-age 300, noted as the response never changing, only Vary. — SequenceFigure 2. One origin, one traffic pattern, and four values of one response header
The origin is a real server on a socket and every miss is a real request through it. Nothing about the response body or its Cache-Control changes between the four runs.

A terminal capture of the edge caching lab. A real HTTP origin on a socket and 20,000 real requests through a cache implementing the Vary rule, with 200 Zipf distributed URLs, three Accept-Encoding values, 300 user agents and 5,000 sessions, against a cache holding 10,000 entries. A table of four Vary configurations against distinct cache keys, hit rate and origin requests: none gives 200 keys and 99.0 percent; Accept-Encoding gives 501 and 97.5; User-Agent gives 7,628 and 61.9; Cookie gives 18,404 and 6.5 with 18,706 origin requests. — terminal-figureFigure 3. The measurement. Four values of Vary over the same twenty thousand requests
The same 200 URLs became 200, 501, 7,628 or 18,404 distinct cache entries, and the hit rate fell from 99.0 percent to 6.5, purely because of which request headers the response said it varies on.

A bar chart of distinct cache entries the edge must hold. No Vary gives 200, Accept-Encoding gives 501, User-Agent gives 7,628 and Cookie gives 18,404. — ChartFigure 4. Two hundred URLs became eighteen thousand four hundred cache entries
Nothing about the pages changed. The multiplier is set by how many distinct values your traffic has for the headers you named, which is three for encodings and one per session for cookies.

A bar chart of the percent of requests served by the edge. No Vary gives 99.0 percent, Accept-Encoding gives 97.5, User-Agent gives 61.9 and Cookie gives 6.5. — ChartFigure 5. Ninety-nine percent, ninety-seven, sixty-two, six
The origin load is the number to look at. It went from 200 requests to 18,706, which is ninety-three times, because of one header on one response.

Work out the cardinality of every header in your Vary before shipping it. It is a count of distinct values in a day of access logs, and it is the factor by which your edge cache is about to be multiplied.

Where this shows up in interviews

Where this shows up in interviews

Edge Caching

The rest of this lesson

Continue this lesson

Cacheable Is A Yes Or No. How Many Copies Is A Number

Two Hundred URLs, Four Answers