Load Balancer Algorithm

A load balancer sits in front of a pool of identical servers and spreads traffic across them so no single machine gets overwhelmed. The algorithm is the part that answers one question on every request: which server should handle this one? That choice is made in microseconds, millions of times per second on a busy system, so the logic has to be cheap to compute.

The algorithms split into two families. Static algorithms decide based on fixed rules and ignore live server state: round robin and IP hash are static. Dynamic algorithms look at what is happening right now, like how many connections each server is holding or how fast it is responding, and adapt. Least connections and least response time are dynamic.

Picking the right one matters because real backends are rarely identical in load. One request might be a 2 millisecond cache hit and the next might be a 5 second report generation. A naive algorithm can pile slow requests onto one server while others sit idle.

Round robin keeps a counter and hands request 1 to server A, request 2 to server B, request 3 to server C, then loops back to A. It is dead simple and needs no state about the backends. The weakness is that it treats a 10 second request the same as a 1 millisecond one, so load can drift uneven even though request counts look balanced.

Least connections tracks how many active connections each server currently holds and routes the next request to whichever has the fewest. This naturally handles uneven request durations because a server stuck on slow work shows a high connection count and stops receiving new traffic until it drains. It costs a little more because the balancer must keep a live count per server.

Weighted round robin and weighted least connections add a weight per server, so a box with 32 cores can be given weight 4 while an 8 core box gets weight 1, and it receives roughly four times the traffic. IP hash computes a hash of the client IP address and maps it to a fixed server, which gives sticky sessions without a shared session store, at the cost of uneven distribution if a few large clients dominate.

Use round robin when your servers are equally sized and your requests are short and uniform, like serving static assets or stateless API calls that finish fast. It is the cheapest option and the easiest to reason about.

Use least connections when request durations vary a lot, which is true for most real APIs and database-backed apps. It self-corrects against slow servers. Use the weighted variants when your fleet is a mix of machine sizes, common during a rolling migration to newer hardware. Reach for IP hash or another sticky method only when a request truly must land on the same server, for example an in-memory session or a websocket, and prefer a shared store like Redis instead if you can, because hashing makes scaling and draining servers harder.

A subtle trap with dynamic algorithms is the thundering herd: when a fresh server joins the pool with zero connections, least connections can dump a flood of new requests onto it all at once. Production balancers soften this with slow start, ramping a new server up over 30 to 60 seconds. Also remember the algorithm only routes; it does not fix a broken backend. You still need health checks to pull dead servers out of rotation.