We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Load Balancing & Proxies
The strategy a load balancer uses to distribute requests. Round-robin, least-connections, weighted, IP-hash, and random are common algorithms, each with different trade-offs.
In short
A load balancer algorithm is the rule a load balancer follows to decide which backend server gets each incoming request. Common ones are round robin (rotate through servers in order), least connections (pick the server with the fewest active requests), weighted variants (send more traffic to bigger servers), and IP hash (always route a given client to the same server).
In short
A load balancer algorithm is the rule a load balancer follows to decide which backend server gets each incoming request. Common ones are round robin (rotate through servers in order), least connections (pick the server with the fewest active requests), weighted variants (send more traffic to bigger servers), and IP hash (always route a given client to the same server).
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Load Balancer Algorithm across a run of lessons, not one page. These 4 alone are about 112 minutes of step-by-step reading, every one with a quiz.
How load balancers decide which server gets the next request. Round Robin, Weighted, Least Connections, IP Hash, and more.
How load balancer failover works with a shared virtual IP, VRRP and keepalived. Also covers ALB health checks and failover across zones.
How a GSLB decides which region a user reaches, and why the decision takes minutes to arrive even when it is made in microseconds.
Splitting traffic by proportion, why a percentage of requests is not a percentage of users, and what a small canary can actually tell you.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Load Balancer Algorithmspay once,
yours for life
A load balancer sits in front of a pool of identical servers and spreads traffic across them so no single machine gets overwhelmed. The algorithm is the part that answers one question on every request: which server should handle this one? That choice is made in microseconds, millions of times per second on a busy system, so the logic has to be cheap to compute.
The algorithms split into two families. Static algorithms decide based on fixed rules and ignore live server state: round robin and IP hash are static. Dynamic algorithms look at what is happening right now, like how many connections each server is holding or how fast it is responding, and adapt. Least connections and least response time are dynamic.
Picking the right one matters because real backends are rarely identical in load. One request might be a 2 millisecond cache hit and the next might be a 5 second report generation. A naive algorithm can pile slow requests onto one server while others sit idle.
Round robin keeps a counter and hands request 1 to server A, request 2 to server B, request 3 to server C, then loops back to A. It is dead simple and needs no state about the backends. The weakness is that it treats a 10 second request the same as a 1 millisecond one, so load can drift uneven even though request counts look balanced.
Least connections tracks how many active connections each server currently holds and routes the next request to whichever has the fewest. This naturally handles uneven request durations because a server stuck on slow work shows a high connection count and stops receiving new traffic until it drains. It costs a little more because the balancer must keep a live count per server.
Weighted round robin and weighted least connections add a weight per server, so a box with 32 cores can be given weight 4 while an 8 core box gets weight 1, and it receives roughly four times the traffic. IP hash computes a hash of the client IP address and maps it to a fixed server, which gives sticky sessions without a shared session store, at the cost of uneven distribution if a few large clients dominate.
Use round robin when your servers are equally sized and your requests are short and uniform, like serving static assets or stateless API calls that finish fast. It is the cheapest option and the easiest to reason about.
Use least connections when request durations vary a lot, which is true for most real APIs and database-backed apps. It self-corrects against slow servers. Use the weighted variants when your fleet is a mix of machine sizes, common during a rolling migration to newer hardware. Reach for IP hash or another sticky method only when a request truly must land on the same server, for example an in-memory session or a websocket, and prefer a shared store like Redis instead if you can, because hashing makes scaling and draining servers harder.
A subtle trap with dynamic algorithms is the thundering herd: when a fresh server joins the pool with zero connections, least connections can dump a flood of new requests onto it all at once. Production balancers soften this with slow start, ramping a new server up over 30 to 60 seconds. Also remember the algorithm only routes; it does not fix a broken backend. You still need health checks to pull dead servers out of rotation.
NGINX
Defaults to round robin and supports least_conn, ip_hash, and weighted variants with a single directive in the upstream block.
AWS Elastic Load Balancing
The Application Load Balancer uses round robin by default and offers least outstanding requests for uneven workloads.
HAProxy
Offers roundrobin, leastconn, source IP hashing, and the adaptive first algorithm, switchable per backend.
Google Cloud Load Balancing
Uses weighted least requests and rate or utilization based balancing modes to spread traffic across backend instance groups.
Round robin rotates through servers in fixed order and ignores how busy each one is, so it works best when requests are short and uniform. Least connections routes to whichever server currently holds the fewest active connections, which automatically avoids servers stuck on slow requests. Use least connections when request durations vary.
There is no single best one. Round robin is fine for equal servers and fast uniform requests. Least connections is the safer default for most real APIs where request times vary. Use weighted versions for mixed hardware sizes, and IP hash only when a client must stick to the same server.
IP hash routes every request from a given client IP to the same backend server by hashing the address. It gives session stickiness without a shared session store, which is useful for in-memory sessions or long-lived websocket connections. The downside is uneven load if a few large clients dominate, and harder server draining.
No. The algorithm decides which healthy server gets the next request. Health checks decide which servers are healthy in the first place by periodically probing them and removing failing ones from rotation. They work together: health checks filter the pool, then the algorithm distributes across what remains.
Slow start gradually ramps traffic to a newly added server over a window like 30 to 60 seconds instead of sending it full load immediately. It prevents dynamic algorithms such as least connections from dumping a burst of requests onto a fresh server that shows zero connections, which would otherwise overwhelm it before its caches warm up.
Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
Round Robin
A load balancing algorithm that distributes requests to servers in sequential order, cycling through the list. Simple but ignores server capacity differences.
Consistent Hashing
A hashing technique where adding or removing servers only moves a small fraction of keys. Used by Amazon DynamoDB and Cassandra for data distribution.
DNS
The phonebook of the internet. Translates human-readable domain names (google.com) into IP addresses that computers understand.
Proxy
An intermediary server that sits between the client and the destination server. Forward proxies act on behalf of clients; reverse proxies act on behalf of servers.
Reverse Proxy
A server in front of your web servers that receives every request first, passes it to one of them and returns the answer. It handles TLS, load balancing, caching and routing, and hides the servers behind it.
This page explains the idea. The "Load Balancer Algorithms" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Load Balancer Algorithms lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Load Balancer Algorithm in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
