We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Consistency Models
Free preview: the first 2 parts of this 9-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Consistency Models
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Every outage post-mortem you have ever read that mentions "OOM Killed" is a memory management failure. The Linux kernel decided your process was consuming too much memory and terminated it without warning. No graceful shutdown. No connection draining. Just dead.
Memory is finite, expensive, and unforgiving. A CPU-bound service degrades gradually, getting slower as load increases. A memory-bound service works perfectly until it does not, then it falls off a cliff. One memory leak, one unbounded cache, one careless allocation pattern, and your service goes from 100% healthy to 100% dead in seconds.
At scale, memory management becomes a core architectural concern. stores everything in RAM, so every byte counts. Kafka brokers manage page cache for millions of messages. Database engines use buffer pools to cache hot pages. The difference between a junior and senior engineer is often how they think about memory: juniors let the garbage collector handle it; seniors profile allocations, set memory budgets, and design for bounded memory usage.
GitHub shared a post-mortem in 2022 where a single unbounded in-memory cache in a Ruby service grew from 200 MB to 12 GB over 6 hours, then triggered an OOM kill cascade across the cluster. The fix was a 3-line change adding an LRU eviction policy with a 500 MB cap.
Stack memory is fast, automatic, and limited. Function call frames and local variables live here. Allocated on function entry, freed on function exit. Fixed size (typically 1-8 MB per thread). Stack overflow means your recursion went too deep.
Heap memory is flexible, managed manually or by GC, and unbounded (until OOM). Objects, buffers, and anything with dynamic lifetime live here. This is where memory leaks live.
| Memory Area | Speed | Management | Typical Size | Failure Mode |
|---|---|---|---|---|
| Stack | Fastest | Automatic (LIFO) | 1-8 MB/thread | Stack overflow |
| Heap | Fast | Manual or GC | Limited by OS/container | OOM kill |
| mmap | Moderate | OS page cache | Limited by address space | Memory pressure |
| Off-heap | Fast | Manual | Application-managed | Silent corruption if mismanaged |
Memory allocators matter more than you think. The default allocator (glibc's ptmalloc) works fine for most applications. But at scale, fragmentation becomes a killer. Small allocations interspersed with large ones leave "holes" in memory that cannot be reused. Jemalloc (used by , Firefox, Facebook) and tcmalloc (used by Google) reduce fragmentation significantly.
Common memory problems in distributed systems: