Name: System Design Masterclass
Availability: InStock

The last two lessons routed on the client. A cookie names a browser. An address names a network path. Both were ways of getting a request back to a machine that happened to be holding something.

This lesson routes on something else entirely: a key that names whose data the request is about. A tenant. A document. A device. A model. Something that has a home of its own, independent of who is asking for it.

Five routing keys with what each identifies and what it reaches. A stickiness cookie identifies one browser and reaches state stuck in one process. The client address identifies one network path and reaches very little, unevenly. A tenant id identifies whose data this is and reaches the shard it already lives on. A document or room id identifies which piece of state and reaches the process already holding it. A model or hardware class identifies what the work needs and reaches the machines that can do it. — FigureFigure 1. The mechanism is the same hash. What changes is what the key points at
The first two rows are the previous two lessons. The last three are this one, and the difference is that what they name genuinely lives somewhere.

Here is the concrete version. A multi-tenant product has three database shards. Each customer's rows live on exactly one of them. If a request for Acme lands on a server wired to the wrong shard, that server has to cross the network to fetch data another server had ready, so the request is slower for no reason at all.

Route on the tenant, and every Acme request reaches a server already connected to Acme's shard, with Acme's data in its local cache and Acme's connection pool already open.

That is the entire idea. Everything after this is about the ways it goes wrong.

A recorded terminal session. Three backends on ports 8841, 8842 and 8843 sit behind a proxy routing on the X-Tenant-Id header. Tenants acme, stark and contoso each land on 8841, 8842 and 8843 respectively, three times each. A request with no header goes round robin across all three. A client sending X-Tenant-Id contoso is routed to 8843. Backend 8842 is then removed and all three tenants move to different backends. — RecordedFigure 2. A real proxy routing on a header, and the three things that go wrong
Each tenant lands on its own backend and stays there. Then a request arrives with no header, a client claims to be a tenant it is not, and one backend leaving moves every tenant.

Every one of those three surprises is a section of this lesson. None of them is visible in the usual explanation, which is a code comment showing hash(tenant) % 3 and a sentence about data locality.

The has to get the key out of the request before it can choose anything. There are four normal places it can live, and they are not equally safe.

A sequence diagram with three lifelines: a client, the edge proxy and a backend group. The client sends GET slash v2 slash orders with Host acme.app.example, noted as also carrying an X-Tenant-Id header the client set itself. The edge derives the tenant from the hostname, noted as then deleting the client's header and writing its own. The request is routed to the group holding acme's shard, and the response returns. — SequenceFigure 3. Derive the key at the edge, then overwrite whatever the client sent
A subdomain, a path prefix, a verified token claim and a client-set header are four different levels of trust. The edge turns whichever one it uses into a single header the backends can rely on.

The step people skip is the delete. It is very common to write the rule as "set the tenant header if it is not already present", because that reads as defensive and it is the opposite. A client that sets the header first now controls its own routing, and every backend downstream sees a header that looks like it came from the proxy.

Overwrite unconditionally. No request should ever reach a backend carrying a tenant id the edge did not put there itself.

Beyond where it lives, a key has to have a few properties to be usable at all.

Five properties with the failure each is named after. Present on the very first request, because a key the application returns is too late to route on. Readable without parsing the body, because a key in the JSON forces the proxy to buffer every request. Stable for the life of the work, because a key that changes mid-flow reroutes somebody mid-flow. Coarse enough to be worth grouping, because one key per request is just random routing with extra steps. Not the authorisation boundary, because if the client can set it, it is a hint and never a permission.

Affinity Routing

The rest of this lesson

Continue this lesson

Routing by What the Request Is For

Where the Key Comes From, and What Makes One Workable