We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Microservices Architecture
A pattern that isolates different parts of a system so a failure in one part doesn't sink the whole ship. Named after the compartments in a ship's hull.
In short
The bulkhead pattern isolates a system's resources into separate pools so that one overloaded or failing component cannot exhaust the resources every other component depends on. It is named after the watertight compartments in a ship's hull: flood one compartment and the ship stays afloat because the others are sealed off.
In short
The bulkhead pattern isolates a system's resources into separate pools so that one overloaded or failing component cannot exhaust the resources every other component depends on. It is named after the watertight compartments in a ship's hull: flood one compartment and the ship stays afloat because the others are sealed off.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Bulkhead across a run of lessons, not one page. These 4 alone are about 91 minutes of step-by-step reading, 1 of them with code you run in the browser, all with a quiz.
Isolate components into compartments so a failure in one doesn't sink the entire system, inspired by ship hull design.
Extending Kubernetes' API with your own resource types, the foundation of the operator pattern.
Migrate from monolith to microservices incrementally, replacing pieces one at a time while the old system stays alive.
Prevent cascading failures in distributed systems by failing fast when a downstream service is unhealthy.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Bulkhead Patternpay once,
yours for life
A bulkhead is a way of partitioning shared resources, usually threads, connection pool slots, or memory, so that each consumer gets its own fixed allocation. When one consumer uses up its allocation, it gets rejected or queued, but it cannot reach into and drain the slots reserved for everyone else.
The classic failure it prevents is resource exhaustion cascading across a service. Imagine a service that calls three downstream APIs from one shared pool of 100 threads. If one of those APIs slows down to 5 seconds per call, requests pile up waiting on it, and within seconds all 100 threads are blocked on the slow dependency. Now calls to the other two healthy APIs also fail, because there are no free threads left. The slow dependency took down everything.
With bulkheads you split that pool: 40 threads for API A, 30 for API B, 30 for API C. When API A goes slow, it can only ever block its own 40 threads. API B and C keep serving traffic. You traded some peak capacity for the guarantee that one bad dependency stays contained.
The most common implementation is the thread pool bulkhead. Each downstream call or each tenant runs through a dedicated thread pool with a bounded queue. Hystrix, the library Netflix built, did exactly this: every dependency got its own small thread pool, and a call that could not get a thread in time was rejected immediately instead of waiting.
A lighter weight version is the semaphore bulkhead. Instead of separate threads, you keep a counter of allowed concurrent calls per dependency. A call decrements the counter on entry and increments it on exit. If the counter hits zero, new calls fail fast. This avoids the overhead of extra threads but does not isolate slow calls onto their own execution context, so it is best when calls are cheap and you mainly want to cap concurrency.
Bulkheads also exist at coarser levels. You can give each customer tier its own set of server instances, give each service its own database connection pool, or run noisy workloads in a separate Kubernetes node pool. The principle is identical at every layer: bound the blast radius by not sharing the pool that can be exhausted.
Reach for a bulkhead when one slow or failing dependency can starve unrelated traffic, or when a single noisy tenant can degrade service for everyone else. It pairs naturally with the circuit breaker pattern: the bulkhead caps how many requests can be stuck on a dependency at once, and the circuit breaker stops sending requests altogether once that dependency is clearly down.
The main cost is lower peak utilization. If you carve 100 threads into four pools of 25, then a burst of traffic to one dependency can be rejected even while 75 threads sit idle in the other pools. You are deliberately leaving capacity on the table to protect isolation. Sizing each pool is the hard part: too small and you reject healthy traffic, too large and the isolation is meaningless.
Bulkheads also add operational complexity. More pools mean more metrics to watch, more tuning, and more thread or memory overhead from the pools themselves. For a small service with one dependency, a single bounded pool and a timeout is usually enough. Bulkheads earn their keep once you have several dependencies of differing reliability sharing the same process.
Netflix is the textbook case. Their API gateway fans out to hundreds of backend services to assemble a single home screen. Early on, a single struggling backend could consume all the gateway's threads and take down the entire member experience. Hystrix solved this by wrapping every backend dependency in its own thread pool bulkhead plus a circuit breaker, so a recommendation service melting down only removed the recommendations row, not the whole page.
You can see the same idea in resilience libraries today. Resilience4j ships a Bulkhead module where you annotate a method with a named bulkhead and configure maxConcurrentCalls, for example 25, and maxWaitDuration. Calls beyond the limit are rejected with a BulkheadFullException, which your code turns into a fallback or a fast error rather than a hung request.
Netflix Hystrix
Wrapped each backend dependency in its own thread pool so one failing service could not exhaust the API gateway's threads and crash the home screen.
Resilience4j
Provides Bulkhead and ThreadPoolBulkhead modules that cap concurrent calls per dependency and reject overflow with a BulkheadFullException.
Kubernetes
Separate node pools, resource requests, and limits isolate noisy workloads so one pod cannot starve CPU or memory from others on the cluster.
AWS Cell-Based Architecture
Amazon partitions services into independent cells, each serving a slice of customers, so a failure stays inside one cell instead of the whole region.
A bulkhead limits how many requests can be in flight to a dependency at once, capping the blast radius of a slowdown. A circuit breaker watches the failure rate and stops sending requests entirely once a dependency is clearly down. They solve different problems and are usually used together.
A thread pool bulkhead runs each dependency on its own dedicated threads, which isolates slow blocking calls onto a separate execution context but costs extra threads. A semaphore bulkhead just counts concurrent calls with no extra threads, which is cheaper but does not isolate slow calls, so it suits fast, cheap operations where you only need to cap concurrency.
Start from the dependency's expected concurrency: peak requests per second multiplied by average latency in seconds gives the concurrent calls you need. Add headroom for bursts, then watch rejection metrics in production. Frequent rejections during healthy traffic means the pool is too small; rejections only during a real incident means it is doing its job.
It lowers peak utilization on purpose, since reserved capacity in one pool cannot be borrowed by another. Thread pool bulkheads also add some context-switching and memory overhead. You accept that cost in exchange for the guarantee that one failing component stays contained instead of taking everything down with it.
It comes from ship building. A ship's hull is divided into watertight compartments by walls called bulkheads, so if the hull is breached, only one compartment floods and the ship stays afloat. The software pattern applies the same isolation idea to system resources.
Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Rate Limiting
Controlling how many requests a client can make in a given time window. Protects your API from abuse and ensures fair usage.
Timeout
A maximum duration to wait for an operation to complete before giving up. Without timeouts, a stalled dependency can hang your entire system indefinitely.
Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.
This page explains the idea. The "Bulkhead Pattern" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Bulkhead Pattern lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Bulkhead in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
