We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Core Fundamentals
A break in communication between nodes in a distributed system. Some nodes can't reach others. The 'P' in CAP theorem that forces the trade-off between consistency and availability.
In short
A network partition is a failure where nodes in a distributed system can no longer communicate with each other, splitting the cluster into two or more groups that each keep running but cannot see the others. It is the "P" in the CAP theorem, and it forces every distributed system to choose between staying consistent and staying available while the split lasts.
In short
A network partition is a failure where nodes in a distributed system can no longer communicate with each other, splitting the cluster into two or more groups that each keep running but cannot see the others. It is the "P" in the CAP theorem, and it forces every distributed system to choose between staying consistent and staying available while the split lasts.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Network Partition across a run of lessons, not one page. These 4 alone are about 101 minutes of step-by-step reading, every one with a quiz.
When nodes can't talk to each other, the failure mode that defines distributed systems.
Consistency, Availability, Partition Tolerance, you can only pick two, and you must pick partition tolerance.
Running two versions at once, everything they have to share, and why the weight change is the easiest part of it.
The algorithm that makes adding or removing nodes cheap, an interview favorite and production essential.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Network Partitioningpay once,
yours for life
Picture a cluster of five database servers spread across two data centers, three in one and two in the other. They normally talk over the network constantly: replicating writes, running health checks, electing a leader. Now imagine the link between the two data centers goes down. The three nodes on one side can still talk to each other, the two nodes on the other side can still talk to each other, but the two groups cannot reach across. That is a network partition. The cluster is split into islands.
The important and counterintuitive part is that none of the nodes have crashed. Every server is up, healthy, and serving requests. From inside each island the rest of the cluster simply looks dead, because a node that does not respond looks identical to a node that is unreachable. A server cannot tell the difference between a peer that has failed and a peer it just cannot reach.
Partitions are not exotic. They come from a switch reboot, a misconfigured firewall rule, a cut fiber line, a cloud availability zone losing connectivity, or even a garbage collection pause long enough that a node stops answering heartbeats. Anything that interrupts communication between nodes for long enough counts.
The CAP theorem says that during a partition a distributed system can guarantee at most two of three properties: consistency, availability, and partition tolerance. Since real networks always partition eventually, partition tolerance is not optional. That leaves a hard choice between consistency and availability, and you only have to make it while the partition is active.
Suppose a client writes to one island and another client reads from the other island. If the system insists on consistency, the island that cannot confirm the write with a majority must refuse to answer, returning an error or blocking. That keeps everyone seeing the same data but makes part of the cluster unavailable. This is the CP choice.
If instead the system insists on availability, both islands keep accepting reads and writes. Nobody gets an error, but the two sides now hold different data and will have to reconcile it later, which means clients can briefly read stale or conflicting values. This is the AP choice. Neither answer is wrong. It depends on whether a wrong answer or no answer is more dangerous for your use case.
The most common defense is quorum, usually a majority. With five nodes, any operation must be acknowledged by at least three. During a 3-versus-2 split, only the side with three nodes can form a majority, so it stays in charge and the minority side steps down and refuses writes. This guarantees that two islands can never both believe they are the authority, which prevents the dreaded split-brain where two leaders accept conflicting writes.
AP systems take the opposite approach. They let both sides accept writes and tag each version with metadata such as a vector clock or a last-write-wins timestamp. When the partition heals and the nodes can talk again, they exchange what they missed and merge it. Conflicts are resolved automatically by a rule, or surfaced to the application to resolve, as Amazon's Dynamo design did with shopping carts.
Detection itself is just timeouts. A node decides a peer is gone after it misses heartbeats for some window, often a few seconds. Tuning that window is a balance: too short and a brief network hiccup triggers a needless failover, too long and the system is slow to react to a real partition.
Consider a banking ledger running on a strongly consistent store like a Raft-based system. A partition cuts the cluster in half. A customer on the minority side tries to withdraw money. The minority cannot reach a majority, so it cannot safely commit the debit, and it rejects the request with an error. The customer is annoyed, but the bank never lets two partitions both approve a withdrawal of the same balance. Here, refusing service is the correct behavior because a double spend is worse than a delay.
Now consider a shopping cart on an AP store like Cassandra or Riak. During the same kind of partition, a customer adds an item on one side. The write succeeds locally even though the other replicas cannot see it yet. When the partition heals, the carts are merged so no item is lost. Here, availability wins because a temporarily incomplete cart is harmless and an error at checkout would cost a sale.
The lesson engineers internalize is that the same failure demands opposite responses depending on what the data means. The partition is unavoidable. The policy you pick for surviving it is the real design decision.
Apache Cassandra
An AP design that keeps accepting reads and writes on both sides of a partition using tunable consistency and hinted handoff, then reconciles afterward.
Amazon DynamoDB
Built on the Dynamo lineage that favors availability during partitions and resolves divergent versions on read or via last-write-wins.
etcd and Kubernetes
Uses Raft so only the majority partition keeps serving the control plane; the minority refuses writes to avoid split-brain.
MongoDB
A replica set holds an election when nodes lose contact, and only the side with a majority of voting members can hold a primary that accepts writes.
No, and that is exactly what makes it hard. A crashed node is down everywhere, but in a partition every node is alive and serving requests; they just cannot reach each other. From one node's point of view an unreachable peer looks identical to a dead one, which is why systems rely on quorum rather than trying to tell the difference.
Split-brain is when two sides of a partition both decide they are the authority and both accept writes, producing conflicting data. Majority quorum prevents it: only the side that holds more than half the nodes can act as leader, so the two islands can never both think they are in charge at the same time.
You can make them rarer but you cannot eliminate them. Cables get cut, switches reboot, cloud zones lose connectivity, and a long garbage collection pause can make a healthy node look unreachable. The CAP theorem assumes partitions will happen, so the realistic goal is to handle them gracefully, not to prevent them.
No. The trade-off only applies while a partition is active. When the network is healthy a well designed system can offer both strong consistency and full availability. CAP only forces the choice during the failure window, which is usually seconds to minutes.
Through timeouts on heartbeats. Nodes ping each other on an interval, and if a node misses several heartbeats within a window of typically a few seconds, peers treat it as unreachable and react. Setting that window too short causes false alarms from brief hiccups; too long makes the system slow to respond to a real split.
CAP Theorem
When a network failure splits a distributed data store, each node must either refuse requests to stay consistent or keep answering with possibly stale data. It cannot guarantee both during the partition.
Split Brain
A failure scenario where a network partition causes two halves of a cluster to operate independently, each believing it's the leader. Can cause data corruption if not handled.
Fault Tolerance
A system's ability to keep operating correctly even when some of its components fail. Achieved through redundancy, replication, and graceful degradation.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Throughput
The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.
Bandwidth
The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.
This page explains the idea. The "Network Partitioning" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Network Partitioning lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Network Partition in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
