Name: System Design Masterclass
Availability: InStock

The previous lesson ended on one sentence: stickiness is a routing answer to a storage question.

That is worth unpacking, because it is the most useful thing in either lesson. Every cost of sticky sessions came from the same place. The fleet went uneven because users were pinned. Adding servers did not help because existing users stayed pinned. A routine deploy logged people out because the machine holding their session went away. None of that is a property of load balancing. All of it is a property of the session living inside one process's memory.

So this lesson asks the storage question directly. Where does the session go?

There are only three answers, and every real system is one of them or a deliberate mix of two.

Three rows. In one instance's memory: costs nothing until it is not that instance, and needs stickiness, and loses everything on a restart. In a store all instances share: costs a network round trip per request, and adds a new dependency that is now on the critical path. In the client, signed: costs nothing per request at all, and you can no longer revoke it, and it can go stale. A closing line reads: there is no fourth option and no free one. — FigureFigure 1. Three places, and only three
Read the middle column first. Each row costs something on every single request, and the costs are completely different in kind: a lost machine, a network hop, or the ability to take a session back.

The first row is what you get by default. Nearly every web framework, out of the box, keeps the session in the memory of the process that created it. Nobody chooses that. It is simply what happens when nobody chooses anything, and stickiness is the routing configuration people reach for once it starts hurting.

The other two rows are choices. Making one of them makes the entire affinity question disappear instead of managing it forever.

Here is what that looks like when you actually run it.

A recorded terminal session. Three backends on ports 8831, 8832 and 8833 sit behind a proxy with stickiness switched off. Four requests carrying one cookie are served by 8831, 8832, 8833 and 8831, and every one reports user=roni and cart=3. A backend is then taken out of the pool, and the next three requests, served by 8832, 8833 and 8832, report exactly the same user and cart. — RecordedFigure 2. The same fleet as the last lesson, with the session moved out of the instances
Four requests with one cookie landed on three different machines and every machine knew who the user was. Then a backend was removed mid-run and nothing changed. Compare that with the previous lesson's capture, where removing the pinned backend lost the session immediately.

This is deliberately the same three backends, the same proxy, the same cookie and the same client as the sticky-sessions capture. Exactly one thing is different: the backends read and write the session through a store all three can see, instead of keeping it in their own memory. Stickiness is off. The is doing nothing clever whatsoever.

That is the whole idea of this lesson in one screen. The routing did not get smarter. The state stopped being somewhere only one machine could see.

Session Affinity

The rest of this lesson

Continue this lesson

The Question the Last Lesson Left Open