Name: System Design Masterclass
Availability: InStock

Every rule so far in this chapter has been a function of the request. Same path, same backend. Same hostname, same backend. Same header, same backend.

A weight is not a function. It is a coin, tossed once per request.

Four rows. By path: looks at what is being asked for, and the same request always goes the same way. By hostname: looks at which site was asked, same. By header: looks at something about the client, same. By weight: looks at nothing at all, and the same request may go either way. A closing line reads: every rule before this one is a function of the request, a weight is a coin toss, and the whole lesson is what that changes. — FigureFigure 1. The first rule in this chapter that does not look at the request at all
Send the same request twice under a matching rule and it goes to the same place twice. Under a weight it may not.

Everything else in this lesson follows from that one sentence. A user is not routed by a weight; each of their requests is. A percentage of traffic is not a percentage of users. And what you observe over a small number of requests is a sample, not the ratio.

Here is all of that, computed.

A recorded terminal session with four blocks. A: at a 50/50 split, 50 percent of users see both versions over two requests, 93.7 over five, 99.8 over ten and 100 over twenty; at a 5 percent canary the same figures are 9.6, 22.5, 40.0 and 64.0. B: seeing a 1 percent error rate double needs 4,636 requests at a 50 percent canary, 23,182 at 10 percent, 46,363 at 5 percent and 231,816 at 1 percent. C: weights A=5 B=1 C=1 produce different orderings under naive and smooth weighted round robin. D: a 10 percent weight observed over 20 requests lands between 0 and 25 percent, and over 10,000 between 9.4 and 10.6. — RecordedFigure 2. Four things a weight does that the configuration does not say
No network in this one. It is the arithmetic of a coin tossed once per request, seeded so it reproduces exactly.

Start with block A, because it is the number that changes how people talk about canaries.

A line chart. The x axis is requests in one user's session, zero to twenty. The y axis is percent of users who see both versions, zero to one hundred. A 50/50 split rises from 50 percent at two requests to 100 percent at twenty. A 5 percent canary rises from 9.6 percent at two to 64.0 percent at twenty. A note reads: 5 percent of traffic, 64 percent of users. — ChartFigure 3. A five percent canary is sixty-four percent of your users
The weight is applied to every request, so a user with a twenty-request session gets twenty chances to land on the new version.

That is the number to quote when somebody says a one percent rollout is safe. One percent of requests still means a noticeable fraction of users encountering the new version at least once, and a user who hits the new version once for a broken checkout has had a broken checkout.

A canary is only a small blast radius if a partial failure is harmless. A page that renders slightly wrong once, fine. A payment that fails once, not fine.

A hand-drawn diagram. On the left in red: a coin per request, which gives 10 percent of requests and most users see both, described as good for load and bad for people. On the right in green: hash the user id, which gives 10 percent of users and each one stays where it lands, described as good for people and sticky. Two notes read: the ratio is identical and the experiment is not, and one measures requests, the other measures users. — SketchFigure 4. Hash a stable key instead of tossing a coin

Weighted Routing

The rest of this lesson

Continue this lesson

The First Rule That Does Not Look at the Request

A Percentage of Traffic Is Not a Percentage of Users