We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
This idea carries a full system design question on its own. Each walks through the full answer.
System Design Masterclass · Reliability Resilience
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Reliability Resilience
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
If you have ever run kubectl apply on a Deployment manifest, you have done a rolling deploy. It is the default strategy in , the default in AWS ECS, and the default in most container orchestrators. It got the default slot because it is the cheapest and simplest deployment pattern that still provides safety. No double infrastructure like blue-green. No percentage-based traffic splitting like canary. Just replace one pod at a time, check that it works, and move to the next.
The simplicity is the strength and the weakness. A rolling deploy with well-configured probes catches broken images, startup crashes, and health check failures automatically. A rolling deploy with no probes is blind. The controller marches through the fleet replacing healthy pods with broken ones, and by the time anyone notices the service is down.
Kubernetes chose rolling updates as the default after years of production operation at Google. The defaults of maxSurge 25% and maxUnavailable 25% were tuned to handle the common case: a stateless service where losing a quarter of capacity briefly is acceptable and over-provisioning by a quarter is cheap. Most teams never change these defaults, and for most workloads they are the right choice.
The places where rolling deploys fail are predictable. Stateful services that need . Services with long startup times that overwhelm dependencies when a wave of cold pods arrives. Low-traffic services where the probe window is too short to detect a broken version. Understanding these failure modes is what separates a rolling deploy that works from one that causes an outage.
Every rolling update is governed by two parameters. Getting them right is the difference between a smooth rollout and a capacity incident.
| Parameter | What It Controls | Default | Effect |
|---|---|---|---|
| maxSurge | Extra pods allowed above desired count during rollout | 25% | Higher = faster rollout but more temporary resource use |
| maxUnavailable | Pods allowed below ready count during rollout | 25% | Higher = faster rollout but less capacity guarantee |
For a 10-pod deployment with defaults, the controller can run up to 12 pods (10 + 25% surge) and can drop to 8 ready pods (10 - 25% unavailable). The safety envelope is 8 to 12 pods at any point during the rollout. Within that envelope the controller spawns new pods and drains old ones in batches.
A stateful service like a database cluster running 5 replicas should set maxUnavailable to 1 or even 0. Losing 25% of a 5-node cluster risks quorum failure. A high- stateless API can tolerate higher unavailability and benefits from the faster rollout speed.
Readiness probes are the mechanism that actually protects capacity. A pod that returns ready before its connection pools, caches, and JIT compilers are warm will take production traffic and fail. Airbnb published a post-mortem in 2021 about a Kubernetes rollout where new pods reported ready immediately. The cold pods took traffic, overwhelmed downstream dependencies, and caused a 20-minute error spike visible to users. The fix was a readiness probe that waited for 100 successful requests before reporting ready.