We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
This idea carries a full system design question on its own. Each walks through the full answer.
System Design Masterclass · Consistency Models
Free preview: the first 2 parts of this 9-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Consistency Models
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
In 2016, a Discord engineering blog post went viral. Their Go service would freeze for 10+ milliseconds during garbage collection, causing visible lag in voice channels. At Discord's scale, millions of concurrent users, a 10ms GC pause meant dropped audio frames for thousands of users simultaneously.
Garbage collection is the automatic process of reclaiming memory that is no longer in use. It is a convenience that frees developers from manual memory management (and the bugs that come with it: use-after-free, double-free, memory leaks). But that convenience comes at a cost: periodic pauses where the GC stops your application to do its work.
In a monolith, a 50ms GC pause might be imperceptible. In a distributed system with strict SLOs, it is a disaster. If your service has a 100ms p99 latency target and the JVM does a 200ms GC pause, you have blown your SLO for every request in flight. Multiply that across dozens of services in a request chain, and GC pauses become the dominant source of tail latency.
LinkedIn published a study in 2021 showing that GC pauses accounted for 40% of their p99 latency across Java services. Switching from G1GC to ZGC on their highest- services cut p99 by 60%.
Understanding GC, how different algorithms work, how to tune them, and when to avoid them entirely, is table stakes for building low-latency distributed systems.
All GC algorithms answer two questions: (1) which objects are garbage? and (2) how do we reclaim the memory?
Identifying garbage via reachability analysis: Start from "roots" (stack variables, static fields, registers) and trace all references. Any object not reachable from a root is garbage. This is more reliable than reference counting (which cannot handle circular references).
Generational hypothesis: Most objects die young. A request creates temporary objects (parsed JSON, intermediate calculations), uses them for microseconds, and discards them. The GC exploits this by dividing the heap into generations:
| Generation | Contents | Collection Frequency | Pause Duration |
|---|---|---|---|
| Young (Eden + Survivor) | Newly allocated objects | Very frequent (ms) | Short (1-10ms) |
| Old (Tenured) | Long-lived objects (caches, connections) | Infrequent | Long (50-500ms+) |
Key algorithms:
G1GC (Garbage-First). JVM default since Java 9. Divides heap into regions, collects the regions with the most garbage first. Targets a configurable pause time (default 200ms). Good general-purpose choice.
ZGC. JVM low- collector. Concurrent, does almost all work while the application runs. Sub-millisecond pauses regardless of heap size. Ideal for latency-sensitive services.
Shenandoah. Similar goals to ZGC. Concurrent compaction using forwarding pointers. Sub-10ms pauses. Red Hat's contribution to OpenJDK.
Go's GC. Concurrent, tri-color mark-and-sweep. Optimized for low latency. Sub-millisecond pauses in recent versions. No generational separation (as of Go 1.22).
V8's GC (Node.js). Generational with concurrent marking. Old-space collection can pause for tens of milliseconds on large heaps.