We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Microservices Architecture
Gracefully finishing in-flight requests before removing a server from a load balancer's pool. Prevents dropping active connections during deployments or scale-ins.
In short
Connection draining is the practice of letting a server finish its in-flight requests before a load balancer stops sending it new traffic and removes it from the pool. It prevents dropped requests and broken responses when you deploy, scale in, or take a server out of rotation.
In short
Connection draining is the practice of letting a server finish its in-flight requests before a load balancer stops sending it new traffic and removes it from the pool. It prevents dropped requests and broken responses when you deploy, scale in, or take a server out of rotation.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Connection Draining across a run of lessons, not one page. These 4 alone are about 112 minutes of step-by-step reading, every one with a quiz.
Removing an instance on purpose, the race between the pool update and the shutdown, and what in-flight leaves out.
How load balancers decide which server gets the next request. Round Robin, Weighted, Least Connections, IP Hash, and more.
How load balancer failover works with a shared virtual IP, VRRP and keepalived. Also covers ALB health checks and failover across zones.
How a GSLB decides which region a user reaches, and why the decision takes minutes to arrive even when it is made in microseconds.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Connection Drainingpay once,
yours for life
When you remove a server from a load balancer, you have two problems happening at the same time. The load balancer needs to stop sending new requests to that server, and the server still has requests it is in the middle of handling. If you yank the server immediately, every one of those in-flight requests gets killed, and the user sees a 502 or a connection reset.
Connection draining solves the second problem. Instead of cutting the server off instantly, the load balancer marks it as draining, stops routing new requests to it, but keeps existing connections alive until they finish or a timeout expires. Once the server has no active requests left, or the timeout is hit, the load balancer removes it cleanly.
The setting is almost always controlled by a single value: a drain timeout, often called the deregistration delay. AWS Application Load Balancer defaults this to 300 seconds. NGINX and HAProxy call the equivalent state a server going into drain or maintenance mode.
A load balancer keeps a health and routing state for every backend. Connection draining adds a transitional state between healthy and removed. When you trigger a deregistration, the backend moves to draining. The load balancer stops adding it to the pool of candidates for new connections, but its connection table still tracks the open sockets it already has.
For each open connection, the load balancer waits for the response to be sent and the keep-alive connection to either close or go idle. If the application uses HTTP keep-alive, the draining logic usually sends a Connection: close header on the next response so the client does not reuse that socket. Once the connection count for that backend hits zero, removal is immediate.
The timeout is the safety valve. If a request is genuinely stuck, say a slow database query that never returns, the load balancer will not wait forever. After the drain timeout, it forcibly closes whatever is left. So picking the timeout is a balance: long enough to cover your real p99 request duration, short enough that a deploy or scale-in does not stall.
Use connection draining anywhere a backend leaves rotation while it might still be doing work: rolling deployments, autoscaling scale-in events, blue-green cutovers, draining a node before a Kubernetes pod eviction, or pulling an unhealthy instance for investigation. Without it, every deploy becomes a small outage for whoever happened to have a request in flight.
The main cost is time. A 300 second drain timeout means a scale-in or a rolling deploy can take that long per instance in the worst case, which slows down how fast you can ship or shrink a fleet. Teams usually tune it down to something like 30 to 60 seconds for normal web traffic, and only go higher for long-lived requests such as large file uploads or streaming responses.
Draining does not help with truly long-lived connections like WebSockets or gRPC streams that stay open for minutes or hours. For those you need application-level cooperation: the server has to notice it is shutting down and proactively ask clients to reconnect, often paired with a SIGTERM handler that stops accepting new streams and closes idle ones.
AWS Elastic Load Balancing
Calls it deregistration delay on target groups, default 300 seconds, used during Auto Scaling scale-in and ECS deployments.
NGINX
Marks an upstream server as drain so it serves only existing keep-alive and bound sessions while new requests skip it.
HAProxy
Supports a DRAIN state via the runtime API or stats socket, stopping new sessions while letting active ones finish.
Kubernetes
Sends SIGTERM and waits terminationGracePeriodSeconds while readiness probes pull the pod from Service endpoints, draining in-flight traffic before the container is killed.
Connection draining is the load balancer side: it stops sending new traffic to a backend and waits for existing requests to finish. Graceful shutdown is the server side: the process catches SIGTERM, stops accepting new work, finishes what it has, and exits. They work together, and you usually need both for a clean removal.
Set it slightly above your real p99 request duration. For typical web APIs that finish in under a second, 30 to 60 seconds is plenty. The AWS default of 300 seconds is conservative and often slows deploys unnecessarily. Only go higher if you have long uploads, batch endpoints, or streaming responses.
Not well on its own. Draining waits for connections to close, but a WebSocket can stay open indefinitely, so it will just hit the timeout and get force-closed. For WebSockets you need the application to detect shutdown and tell clients to reconnect to another server.
They are forcibly terminated. The load balancer closes the remaining connections, and any client mid-request gets an error or reset. That is why the timeout should cover your slowest realistic request, so this almost never happens in practice.
No. Health checks decide whether a backend is healthy enough to receive traffic. Connection draining decides how to remove a backend cleanly once you have already chosen to take it out. A failing health check can trigger removal, and draining controls how that removal happens.
Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
Rolling Deployment
Gradually replacing old instances with new ones, a few at a time. No downtime, but both versions run simultaneously during the rollout.
Health Check
An endpoint or mechanism that reports whether a service is running and healthy. Load balancers use health checks to route traffic away from unhealthy instances.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.
This page explains the idea. The "Connection Draining" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Connection Draining lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Connection Draining in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
