Exponential Backoff
A retry strategy that doubles the wait time between attempts (1s, 2s, 4s, 8s...) with random jitter. Prevents thundering herd problems when many clients retry simultaneously.
What is Exponential Backoff?
In short
Exponential backoff is a retry strategy where the wait between attempts grows multiplicatively, usually doubling each time (1s, 2s, 4s, 8s), instead of staying fixed. It is paired with random jitter so that many clients retrying after the same failure spread their attempts out over time instead of hammering the server in sync.
What it is
When a network call fails, the simplest fix is to try again. But retrying immediately, or on a fixed timer, is dangerous: if a service is overloaded, a flood of instant retries makes the overload worse. Exponential backoff solves this by increasing the delay after each failed attempt, typically by a factor of 2.
A common schedule looks like: wait 1 second, then 2, then 4, then 8, then 16, often with a cap (a maximum delay) so it never grows past, say, 30 or 60 seconds. After a set number of attempts the client gives up and surfaces an error.
The point is to give the failing system room to recover. Each retry waits longer than the last, so a temporary blip gets one or two quick retries, while a longer outage triggers fewer, more spaced-out attempts instead of a relentless barrage.
How it works under the hood
The delay for attempt number n is base * factor^n. With a base of 1 second and a factor of 2, attempt 0 waits 1s, attempt 1 waits 2s, attempt 2 waits 4s, and so on. A max-delay cap clamps the result so it plateaus instead of growing forever.
The critical second half is jitter: a random component added to (or multiplying) the delay. Without jitter, a thousand clients that all failed at the same instant would all wait exactly 4 seconds and then retry at the same instant again, recreating the spike. This is the thundering herd problem.
A widely used variant is full jitter: instead of waiting exactly base * factor^n, the client waits a random amount between 0 and that value. AWS documented in their architecture blog that full jitter both reduces server load and finishes the overall work faster than no jitter or partial jitter, because it spreads retries evenly across the window.
When to use it and the trade-offs
Use exponential backoff for any retry against a remote dependency that can be transiently busy: HTTP calls, database connections, message queue publishes, calls to rate-limited third-party APIs. It is the default retry policy in most cloud SDKs precisely because the alternative (tight retry loops) causes cascading failures.
The main trade-off is latency. By design, later retries take a long time, so a request that needs five attempts might take 30+ seconds to either succeed or fail. For user-facing requests you usually keep the attempt count and max delay small; for background jobs you can afford a longer ceiling.
Always retry only idempotent operations, or operations protected by an idempotency key, otherwise a retry might charge a card or create a duplicate order. Pair backoff with a retry limit and ideally a circuit breaker so that a fully dead dependency stops getting hammered at all once it is clearly down.
A concrete example
Say your checkout service calls a payment provider and gets a 503 Service Unavailable. With exponential backoff and full jitter, the client picks a random delay between 0 and 1s, retries, fails again, then picks a random delay between 0 and 2s, then 0 and 4s.
If the provider had a two second hiccup, most clients land their successful retry within the first few seconds, and because each chose a different random delay, they arrive staggered rather than all at once. The provider sees a smooth trickle of retries instead of a wall of traffic.
If the outage lasts longer, the growing delays mean each client sends far fewer requests per minute than a fixed-interval retry would, so the recovering provider is not knocked over again the moment it comes back up.
Where it is used in production
AWS SDKs
Every AWS SDK retries throttled and 5xx responses with exponential backoff and jitter by default; the full-jitter approach was popularized by an AWS architecture blog post.
Google Cloud and gRPC
gRPC client retry configs and Google Cloud client libraries use exponential backoff with configurable initial, max, and multiplier values for retryable status codes.
Kubernetes
The kubelet and controllers use exponential backoff for failing pods (CrashLoopBackOff) and requeued work items, capping the delay so a broken pod is not restarted in a tight loop.
Stripe and other payment APIs
Stripe recommends exponential backoff plus idempotency keys when retrying API calls so a retried charge is never executed twice.
Frequently asked questions
- What is the difference between exponential backoff and linear backoff?
- Linear backoff increases the wait by a fixed amount each time (2s, 4s, 6s, 8s), growing slowly. Exponential backoff multiplies the wait each time (2s, 4s, 8s, 16s), so it backs off much faster, which relieves an overloaded server quicker during longer outages.
- Why do you need jitter with exponential backoff?
- Without jitter, all clients that failed at the same moment wait the same fixed delay and then retry at the same moment, recreating the traffic spike. Jitter adds randomness so retries spread out across the window, avoiding this thundering herd effect.
- How many times should you retry?
- There is no universal number, but 3 to 5 attempts with a capped maximum delay (often 30 to 60 seconds) is typical. User-facing requests use fewer attempts to keep latency low; background jobs can afford more. Always set a hard limit so a dead dependency does not get retried forever.
- Is it safe to retry any failed request?
- No. Only retry idempotent operations, or operations guarded by an idempotency key, because a retry of a non-idempotent call (like charging a card) can duplicate the side effect. Also only retry transient failures such as timeouts, 429, and 5xx, not client errors like 400 or 404.
- What is full jitter versus equal jitter?
- Full jitter waits a random amount between 0 and the computed backoff value. Equal jitter waits half the computed value plus a random half. AWS found full jitter spreads load most evenly and completes the work fastest, so it is the common recommendation.
Learn Exponential Backoff hands-on
This page explains the idea. The full lesson lets you step through the ring as servers join and leave, read the implementation, and check yourself with a quiz. It is one of 760+ lessons in the System Design Masterclass, from your first API call to distributed consensus. Eleven Foundation lessons are free, no signup. Lifetime access is ₹499 in India or $7.99 worldwide, one payment, no subscription.
Related lessons
Lessons that touch on Exponential Backoff as part of a larger topic.
Rate Limits and Provider Failure: The Number One Production Error
HTTP 429 is the first failure an LLM app hits at scale. Beat rate limits and provider outages with token buckets, backoff, degraded modes, and failover.
ml-advanced · llm genai ops
Retry Patterns
Retry failed operations intelligently, exponential backoff, jitter, and retry budgets
advanced · reliability resilience
Jitter
Add randomness to retry timing to prevent the thundering herd, the missing piece of exponential backoff
intermediate · microservices architecture
See also
Related glossary terms you might want to look up next.
Retry
Automatically re-attempting a failed operation, usually with exponential backoff. Essential for handling transient failures in distributed systems.
Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Idempotency
An operation that produces the same result whether you run it once or multiple times. Critical for safe retries in distributed systems.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.