We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Consistency Models
Free preview: the first 2 parts of this 9-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Consistency Models
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Every gamer knows the pain of losing hours of progress because they forgot to save. Checkpointing is the save game for distributed systems. It captures state at a known-good point so recovery means replaying from the last checkpoint, not from the beginning.
Without checkpoints, recovering a Flink job from failure means replaying the entire input from the beginning. For a job running for days processing billions of events, that replay could take hours. With checkpoints every 30 seconds, you replay at most 30 seconds of input.
Apache Flink's documentation reports that a production job at Alibaba processing 10 billion events per day uses incremental checkpointing every 10 seconds. State size: 2 TB. Checkpoint size (incremental): ~50 MB. Recovery time from any failure: under 30 seconds. Without checkpointing, recovery would require replaying the entire day's input, taking 4-6 hours.
The Chandy-Lamport algorithm (1985) solved how to take consistent distributed snapshots without stopping the system. Flink, Spark Streaming, and Streams all build on this foundational algorithm. Understanding it is understanding how modern stream processing achieves exactly-once semantics.
Chandy-Lamport algorithm:
The marker divides the message stream into "before checkpoint" and "after checkpoint." This creates a consistent cut across the distributed system without stopping anything.
| Strategy | Overhead | Recovery Time | Consistency | Used By |
|---|---|---|---|---|
| Uncoordinated | Low | High (domino effect) | Weak | Rarely used |
| Chandy-Lamport | Moderate | Low | Strong (consistent cut) | Academic, adapted by Flink |
| Barrier-based | Low | Low | Strong | Apache Flink |
| Incremental | Lowest | Low | Strong | Flink + RocksDB |
Flink barrier checkpointing injects barriers into data streams. When an operator receives barriers on all inputs, it snapshots its state to distributed storage (S3, HDFS, GCS). On failure, restore from checkpoint and replay from the barrier's offset.
Unaligned checkpoints (Flink 1.11+) solve the barrier alignment problem. Instead of waiting for barriers on all inputs (which causes buffering and ), operators snapshot immediately and include the buffered in-flight data as part of the checkpoint. Faster checkpointing at the cost of slightly larger checkpoint size.
Exactly-once semantics depend on checkpointing. Flink guarantees exactly-once processing by atomically committing checkpoint, Kafka consumer offsets, and sink output. If a failure occurs mid-checkpoint, everything rolls back to the last completed checkpoint. No duplicates, no gaps.