Name: System Design Masterclass
Availability: InStock

Everything in this chapter so far has balanced traffic inside one place. A picks a server. A gateway picks a service. Both make a choice and the choice takes effect immediately, because the thing making it sits directly in the path of the request.

Global server load balancing is the first mechanism in this chapter that does not sit in the path. It decides which data centre you should reach, and then it has to tell you, and the telling is the whole subject of this lesson.

Two panels side by side. The left panel, DNS-based GSLB, shows a user's resolver flowing into authoritative DNS which fans out to three regions. Its key reads: steers by answering differently, change takes effect when caches expire, you control the record only. The right panel, Anycast, shows a user's packet flowing into internet routing which fans out to the same three regions. Its key reads: steers by one address many sites, change takes effect on route convergence, you control the announcement. — ComparisonFigure 1. Both send a user to a region, and only one of them waits for a cache
There are two ways to steer a user to a region. One answers a question differently depending on who asked. The other publishes one address from many places and lets the internet work out which is nearest. The rest of this lesson is about the difference between them, because that difference decides how long a failover takes.

Most descriptions of GSLB stop at the left-hand column and present the as a dial. Turn it down, they say, and traffic moves faster. That sentence is the reason this lesson exists, because the TTL is not a dial. It is a request, sent to tens of thousands of caches that you do not own, cannot list, and have no way to flush.

Before measuring anything, it is worth being precise about what comes out of a GSLB. All the health checking and policy evaluation in the world eventually has to be expressed in a answer, and a DNS answer is a very small thing.

One DNS record shown in monospace at the top: api.example.com. 60 IN A 203.0.113.10, with the 60 highlighted in the accent colour. Below it five rows label each field. The name is the name that was asked for. The 60 is the TTL, described as a request and not a guarantee. IN is the class. A is the type, meaning an IPv4 address follows. The address is the answer this resolver got, which may differ from the next resolver's. A closing line reads: only one field is under your control after the answer leaves, and it is the one that stops applying the moment a cache copies it. — FigureFigure 2. The entire output of a GSLB, taken apart
Read the fields and notice what is missing. There is no field that means revoke this. There is no field that means check back if anything changes. There is no field that records why this address was chosen or who it was chosen for. A GSLB makes a rich decision and then expresses it in a format that can carry almost none of it.

The decision itself is genuinely sophisticated. A GSLB evaluates health, capacity and policy on every query, thousands of times a second.

A flowchart. A query arrives and asks whether the nearest region is healthy. If yes it checks whether that region is under its capacity cap, and if so answers with the nearest. If either check fails it moves to the next region by policy and repeats the same two checks, falling back to a last resort region. All paths converge on stamping the TTL, and then on a final box reading: hand it to a cache you will never hear from again. — FlowchartFigure 3. Every branch is careful, and the last box is where the care stops mattering
Follow the branches and they are all sensible. Health first, then capacity so that a failover does not flatten the region it fails over to, then policy for the order regions are tried in. Then the answer leaves, and from that moment the system has no way to correct it.

This is the shape of the whole problem. The control plane is fast. The delivery of its decisions is not.

Where this shows up in interviews

Where this shows up in interviews

Global Server Load Balancing (GSLB)

The rest of this lesson

Continue this lesson

A Fast Decision, Delivered Slowly

What a GSLB Actually Produces