Layer 7 Load Balancing

A load balancer sits in front of a group of backend servers and decides which server should handle each incoming request. A Layer 7 load balancer makes that decision by looking inside the request itself. It reads the HTTP method, the URL path, the Host header, cookies, query strings, and sometimes the body. This is the application layer, layer 7 of the seven-layer OSI model, which is why it is named that way.

Contrast this with Layer 4 load balancing, which only sees TCP and UDP information: source IP, destination IP, and ports. A Layer 4 balancer cannot tell the difference between a request for /login and a request for /checkout, because to it both are just bytes flowing over the same TCP connection. A Layer 7 balancer reads them as HTTP and can treat them differently.

Because it understands HTTP, a Layer 7 balancer can do things a Layer 4 one cannot: route /api/* to your API servers and /static/* to a cache pool, stick a user to the same backend using a session cookie, rewrite headers, terminate TLS, compress responses, and return a 503 page when every backend is down.

The Layer 7 balancer terminates the client connection. The client opens a TCP connection (and a TLS handshake for HTTPS) with the load balancer itself, not with the backend. The balancer decrypts the traffic, reads the full HTTP request, and only then decides where to send it.

It then opens its own connection to the chosen backend, usually reusing a pooled keep-alive connection so it does not pay the TCP and TLS setup cost on every request. The request is forwarded, the response comes back, and the balancer relays it to the client. Two separate connections exist: client to balancer, and balancer to backend.

Routing rules are evaluated in order. A typical config matches on host and path first, picks a backend pool, then applies a balancing algorithm within that pool such as round robin, least connections, or a hash of a cookie for stickiness. Health checks run continuously so a backend that fails an HTTP probe is pulled out of rotation.

The cost of all this is CPU and a small amount of latency. Parsing HTTP, decrypting TLS, and maintaining two connection sets per request is heavier than the near line-rate packet forwarding a Layer 4 balancer does. In practice the overhead is a fraction of a millisecond per request, which is fine for almost every web workload but matters at very high packet rates.

Reach for Layer 7 when routing depends on what is in the request. Path-based routing for microservices, host-based routing for multi-tenant apps, canary releases that send 5 percent of traffic to a new version, A/B tests keyed on a cookie, and sticky sessions for stateful apps all require Layer 7 visibility.

It is also the natural place to centralize TLS termination, gzip or Brotli compression, request and response header rewriting, rate limiting per path, and web application firewall rules. Putting these at the edge keeps your backend servers simpler.

The trade-off is throughput and complexity. Layer 7 balancers handle fewer raw connections per second than Layer 4 and add a hop of TLS decrypt and re-encrypt. For workloads that do not need request inspection, such as a database proxy or a raw TCP game server, Layer 4 is cheaper and faster. A common pattern is to combine them: Layer 4 at the very front for raw scale and DDoS absorption, Layer 7 behind it for smart HTTP routing.

Imagine an online store on three pools of servers: web frontends, an API service, and an image service. All traffic hits one hostname, shop.example.com, on a Layer 7 load balancer.

A request for GET /products/123 has path /products, so the balancer sends it to the web pool. A request for GET /api/cart matches /api and goes to the API pool. A request for GET /images/shoe.jpg matches /images and goes to the image pool, which may be backed by a cache. One public endpoint, three internal destinations, decided entirely by reading the path.

Now the team ships a new checkout service. They add a rule: route 5 percent of /api/checkout traffic to the new version and 95 percent to the old one. If error rates stay low, they raise it to 100 percent. None of this is possible at Layer 4 because the path is invisible there. This is exactly how an AWS Application Load Balancer or an NGINX or Envoy proxy is configured in production every day.