Leader Election

Imagine a group of five friends trying to order pizza. No one is "in charge." Everyone shouts their preferences simultaneously. One wants pepperoni, another wants margherita, two want Hawaiian, and the last person is lactose intolerant. Without someone to make a final decision, they'll argue for 45 minutes and end up ordering nothing.

This is exactly the problem distributed systems face. When you have multiple nodes that need to coordinate, deciding which writes to commit, which replica is authoritative, which node handles a particular request, someone has to make the final call. That "someone" is the leader.

But here's the catch: there's no boss. No predefined hierarchy. The nodes are peers. They have to elect a leader from among themselves, using only network messages. And they have to do it correctly, if two nodes both think they're the leader, you get split-brain. If no node thinks it's the leader, the system freezes.

Leader election is a solved problem. Raft, ZooKeeper's ZAB protocol, and etcd all do it reliably. But understanding the mechanics matters because when your Kafka cluster stops processing messages or your Kubernetes API server becomes unresponsive, the root cause is often a leader election that failed, took too long, or resulted in the wrong outcome.

There are several approaches to leader election, each with different trade-offs:

The Bully Algorithm, the node with the highest ID wins. When a node detects the leader is down, it sends an election message to all nodes with higher IDs. If none respond, it declares itself the leader. If a higher-ID node responds, that node takes over the election. Simple, but the highest-ID node always wins, even if it's slow or poorly connected.

Ring-based Election, nodes are arranged in a logical ring. When the leader fails, the detecting node passes an election message around the ring. Each node adds its ID to the message. When the message returns to the initiator, the node with the highest ID in the message becomes the leader. More fair than bully, but slow, you have to wait for the message to traverse the entire ring.

Raft Election, nodes have randomized election timeouts. When a follower doesn't hear from the leader within its timeout, it increments its term, votes for itself, and asks others for votes. The first candidate to get a majority of votes wins. Randomized timeouts prevent election storms. This is what most modern systems use.