We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Microservices Architecture
A circuit breaker cycles through three states: Closed (requests flow normally), Open (requests are blocked after failures), Half-Open (a few test requests check if the service recovered).
In short
A circuit breaker is a fault-handling component that wraps calls to a remote service and moves between three states: Closed, where calls pass through and failures are counted; Open, where calls are rejected immediately for a cooldown period after too many failures; and Half-Open, where a small number of trial calls are allowed through to test whether the service has recovered, promoting the breaker back to Closed on success or back to Open on failure.
In short
A circuit breaker is a fault-handling component that wraps calls to a remote service and moves between three states: Closed, where calls pass through and failures are counted; Open, where calls are rejected immediately for a cooldown period after too many failures; and Half-Open, where a small number of trial calls are allowed through to test whether the service has recovered, promoting the breaker back to Closed on success or back to Open on failure.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Circuit Breaker States across a run of lessons, not one page. These 4 alone are about 167 minutes of step-by-step reading, 2 of them with code you run in the browser, all with a quiz.
Prevent cascading failures in distributed systems by failing fast when a downstream service is unhealthy.
Stop calling failing services to prevent cascade failures, open, closed, and half-open states.
Delta Lake, Apache Iceberg and Apache Hudi: what is actually in the metadata? I wrote the same 11,198,026 NYC taxi trips into all three, three commits each, and opened every metadata file. Delta kept 3 JSON commits, Iceberg 10 files in a four-level tree, Hudi a timeline and a metadata table. All three returned January's exact 3,475,226 rows when asked for the first version. After 120 small appends, an Iceberg planner opened 122 metadata files and a Delta reader 22; Hudi packed the rows into 1 file group instead of 120 files. Compaction fixed the count, and cleanup ended time travel.
Measured against a real Redis on loopback, a hit costs 0.056 ms against 0.00004 for a local one. Work cheaper than that is made slower by caching it, and at 232 KB the parse was 94 percent of the hit.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Circuit Breaker Patternpay once,
yours for life
A circuit breaker sits between a caller and a downstream dependency (an API, a database, a payment gateway). Instead of letting every request hammer a service that is already failing, it tracks the health of recent calls and changes behavior based on three states.
Closed is the normal, healthy state. Every request is allowed through to the downstream service. The breaker keeps a running tally of successes and failures. As long as failures stay under the threshold, it stays Closed.
Open is the tripped state. Once failures cross the configured threshold, the breaker flips to Open and stops sending traffic downstream. Calls fail instantly with an error or a fallback response, so the caller does not waste time waiting on timeouts. The breaker stays Open for a fixed cooldown window, often 5 to 60 seconds.
Half-Open is the recovery probe. When the cooldown expires, the breaker lets a small number of trial requests through. If those succeed, the service looks healthy again and the breaker returns to Closed. If any of them fail, it snaps back to Open and waits out another cooldown.
The Closed-to-Open transition is driven by a failure detector. Older designs trip after N consecutive failures (for example, 5 in a row). Modern libraries like Resilience4j use a sliding window: they look at the last 100 calls or the last 10 seconds and trip if the failure rate exceeds a percentage, such as 50 percent. Slow calls that exceed a latency limit can also count as failures.
The Open-to-Half-Open transition is purely time based. A timer starts when the breaker opens. When it expires, the breaker does not immediately resume full traffic. It moves to Half-Open and caps how many test calls it allows at once, often just one to ten.
The Half-Open transitions decide the verdict. A configured number of successful trial calls promotes the breaker to Closed and resets the failure counters. A single failure (or failures past a small threshold) demotes it back to Open. While in Half-Open, extra calls beyond the trial limit are usually rejected so a flood of traffic does not overwhelm a service that is still fragile.
State is held per dependency instance, not per request. The counters, the window, and the current state live in memory in the calling service, which is why each pod or process maintains its own breaker rather than a shared global one.
Use a circuit breaker on any synchronous network call to a dependency that can fail or slow down: a third-party API, a microservice, a cache, a database behind a connection pool. The core benefit is fail-fast behavior. Without it, a slow downstream service causes callers to pile up on timeouts, exhaust thread pools and connections, and drag down the whole system in a cascading failure.
The main trade-off is that an open breaker rejects requests that might have succeeded. During the Open state you are trading some availability for stability of the overall system. Pairing the breaker with a fallback (cached data, a default value, a degraded feature) softens this for the user.
Tuning is the hard part. Set the failure threshold too low and the breaker trips on normal blips, hurting availability. Set it too high and it never protects you. The cooldown also matters: too short and you keep slamming a recovering service, too long and you stay degraded after it is healthy. These values need to be tuned against real traffic and timeout settings.
One caveat: because state is per instance, a fleet of 50 pods has 50 independent breakers. They will not trip in lockstep, which is usually fine but means recovery probing is distributed rather than coordinated.
Netflix Hystrix
The library that popularized the pattern, wrapping every inter-service call at Netflix with per-dependency breakers and fallbacks before being put in maintenance mode.
Resilience4j
The modern Java standard that replaced Hystrix, using a count-based or time-based sliding window and exposing Closed, Open, and Half-Open as explicit configurable states.
Istio and Envoy
Implement breaker-style outlier detection at the service mesh layer, ejecting unhealthy upstream hosts so application code does not have to manage breakers itself.
Polly (.NET)
Provides circuit breaker policies for .NET services, with both consecutive-failure and advanced rate-based breakers that move through the same three states.
Open rejects every call immediately during a cooldown window and sends no traffic downstream. Half-Open begins after the cooldown and lets a small, limited number of trial calls through to test whether the service recovered, then decides to close or re-open based on the result.
It watches recent calls. Simple breakers trip after a set number of consecutive failures. Modern ones use a sliding window over the last N calls or last few seconds and trip when the failure rate (and sometimes the slow-call rate) crosses a configured percentage like 50 percent.
They fail fast. The breaker returns an error or a fallback response without contacting the downstream service, so callers do not block on timeouts and connection or thread pools are not exhausted.
No. State is held in memory per process or pod, so each instance maintains its own counters and breaker. A service running 20 replicas effectively has 20 independent breakers.
A retry re-attempts a single failed call hoping it succeeds. A circuit breaker tracks failure patterns over many calls and stops attempts entirely once a dependency looks unhealthy. They are often combined, but retrying through an open breaker would defeat its purpose, so retries should respect the breaker state.
Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Retry
Automatically re-attempting a failed operation, usually with exponential backoff. Essential for handling transient failures in distributed systems.
Timeout
A maximum duration to wait for an operation to complete before giving up. Without timeouts, a stalled dependency can hang your entire system indefinitely.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.
This page explains the idea. The "Circuit Breaker Pattern" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Circuit Breaker Pattern lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Circuit Breaker States in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
