We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Microservices Architecture
A retry strategy that doubles the wait time between attempts (1s, 2s, 4s, 8s...) with random jitter. Prevents thundering herd problems when many clients retry simultaneously.
In short
Exponential backoff is a retry strategy where the wait between attempts grows multiplicatively, usually doubling each time (1s, 2s, 4s, 8s), instead of staying fixed. It is paired with random jitter so that many clients retrying after the same failure spread their attempts out over time instead of hammering the server in sync.
In short
Exponential backoff is a retry strategy where the wait between attempts grows multiplicatively, usually doubling each time (1s, 2s, 4s, 8s), instead of staying fixed. It is paired with random jitter so that many clients retrying after the same failure spread their attempts out over time instead of hammering the server in sync.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Exponential Backoff across a run of lessons, not one page. These 3 alone are about 63 minutes of step-by-step reading, every one with a quiz.
Instead of hammering a failing service, wait longer between each retry, giving the system time to recover.
Retry failed operations intelligently, exponential backoff, jitter, and retry budgets.
Add randomness to retry timing to prevent the thundering herd, the missing piece of exponential backoff.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Exponential Backoffpay once,
yours for life
When a network call fails, the simplest fix is to try again. But retrying immediately, or on a fixed timer, is dangerous: if a service is overloaded, a flood of instant retries makes the overload worse. Exponential backoff solves this by increasing the delay after each failed attempt, typically by a factor of 2.
A common schedule looks like: wait 1 second, then 2, then 4, then 8, then 16, often with a cap (a maximum delay) so it never grows past, say, 30 or 60 seconds. After a set number of attempts the client gives up and surfaces an error.
The point is to give the failing system room to recover. Each retry waits longer than the last, so a temporary blip gets one or two quick retries, while a longer outage triggers fewer, more spaced-out attempts instead of a relentless barrage.
The delay for attempt number n is base * factor^n. With a base of 1 second and a factor of 2, attempt 0 waits 1s, attempt 1 waits 2s, attempt 2 waits 4s, and so on. A max-delay cap clamps the result so it plateaus instead of growing forever.
The critical second half is jitter: a random component added to (or multiplying) the delay. Without jitter, a thousand clients that all failed at the same instant would all wait exactly 4 seconds and then retry at the same instant again, recreating the spike. This is the thundering herd problem.
A widely used variant is full jitter: instead of waiting exactly base * factor^n, the client waits a random amount between 0 and that value. AWS documented in their architecture blog that full jitter both reduces server load and finishes the overall work faster than no jitter or partial jitter, because it spreads retries evenly across the window.
Use exponential backoff for any retry against a remote dependency that can be transiently busy: HTTP calls, database connections, message queue publishes, calls to rate-limited third-party APIs. It is the default retry policy in most cloud SDKs precisely because the alternative (tight retry loops) causes cascading failures.
The main trade-off is latency. By design, later retries take a long time, so a request that retries five times might take 30+ seconds to either succeed or fail. For user-facing requests you usually keep the attempt count and max delay small; for background jobs you can afford a longer ceiling.
Always retry only idempotent operations, or operations protected by an idempotency key, otherwise a retry might charge a card or create a duplicate order. Pair backoff with a retry limit and ideally a circuit breaker so that a fully dead dependency stops getting hammered at all once it is clearly down.
Say your checkout service calls a payment provider and gets a 503 Service Unavailable. With exponential backoff and full jitter, the client picks a random delay between 0 and 1s, retries, fails again, then picks a random delay between 0 and 2s, then 0 and 4s.
If the provider had a two second hiccup, most clients land their successful retry within the first few seconds, and because each chose a different random delay, they arrive staggered rather than all at once. The provider sees a smooth trickle of retries instead of a wall of traffic.
If the outage lasts longer, the growing delays mean each client sends far fewer requests per minute than a fixed-interval retry would, so the recovering provider is not knocked over again the moment it comes back up.
AWS SDKs
Every AWS SDK retries throttled and 5xx responses with exponential backoff and jitter by default; the full-jitter approach was popularized by an AWS architecture blog post.
Google Cloud and gRPC
gRPC client retry configs and Google Cloud client libraries use exponential backoff with configurable initial, max, and multiplier values for retryable status codes.
Kubernetes
The kubelet and controllers use exponential backoff for failing pods (CrashLoopBackOff) and requeued work items, capping the delay so a broken pod is not restarted in a tight loop.
Stripe and other payment APIs
Stripe recommends exponential backoff plus idempotency keys when retrying API calls so a retried charge is never executed twice.
Linear backoff increases the wait by a fixed amount each time (2s, 4s, 6s, 8s), growing slowly. Exponential backoff multiplies the wait each time (2s, 4s, 8s, 16s), so it backs off much faster, which relieves an overloaded server quicker during longer outages.
Without jitter, all clients that failed at the same moment wait the same fixed delay and then retry at the same moment, recreating the traffic spike. Jitter adds randomness so retries spread out across the window, avoiding this thundering herd effect.
There is no universal number, but 3 to 5 attempts with a capped maximum delay (often 30 to 60 seconds) is typical. User-facing requests use fewer attempts to keep latency low; background jobs can afford more. Always set a hard limit so a dead dependency does not get retried forever.
No. Only retry idempotent operations, or operations guarded by an idempotency key, because a retry of a non-idempotent call (like charging a card) can duplicate the side effect. Also only retry transient failures such as timeouts, 429, and 5xx, not client errors like 400 or 404.
Full jitter waits a random amount between 0 and the computed backoff value. Equal jitter waits half the computed value plus a random half. AWS found full jitter spreads load most evenly and completes the work fastest, so it is the common recommendation.
Rate Limits and Provider Failure: The Number One Production Error
HTTP 429 is the first failure an LLM app hits at scale. Beat rate limits and provider outages with token buckets, backoff, degraded modes, and failover.
Retry Patterns
Retry failed operations intelligently, exponential backoff, jitter, and retry budgets
Jitter
Add randomness to retry timing to prevent the thundering herd, the missing piece of exponential backoff
Retry
Automatically re-attempting a failed operation, usually with exponential backoff. Essential for handling transient failures in distributed systems.
Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Idempotency
An operation that produces the same result whether you run it once or multiple times. Critical for safe retries in distributed systems.
Timeout
A maximum duration to wait for an operation to complete before giving up. Without timeouts, a stalled dependency can hang your entire system indefinitely.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
This page explains the idea. The "Exponential Backoff" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Exponential Backoff lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Exponential Backoff in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
