We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Caching Strategies
A cache spread across multiple nodes that acts as a shared layer between application servers and the database. Redis Cluster and Memcached are common implementations.
In short
A distributed cache is a fast in-memory key-value store spread across multiple servers (nodes) that sits between your application and your database, so reads that would hit the database are served from RAM across the cluster instead. By partitioning data across nodes and replicating it, it gives you far more cache capacity and throughput than a single machine, plus the ability to survive a node failure.
In short
A distributed cache is a fast in-memory key-value store spread across multiple servers (nodes) that sits between your application and your database, so reads that would hit the database are served from RAM across the cluster instead. By partitioning data across nodes and replicating it, it gives you far more cache capacity and throughput than a single machine, plus the ability to survive a node failure.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Distributed Cache across a run of lessons, not one page. These 4 alone are about 82 minutes of step-by-step reading, every one with a quiz.
How a distributed cache spreads keys across nodes with consistent hashing and virtual nodes. Measured tests show why a hot key still overloads a node.
A shared cache is a network service, so how you call it matters far more than which product you picked.
Split massive datasets across multiple nodes so no single machine drowns.
Databases that live entirely in RAM, trading durability for speed measured in microseconds.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Distributed Cachepay once,
yours for life
A cache is a place to keep a copy of data so you do not have to recompute it or fetch it from a slower store. A local in-process cache lives inside one application server and dies when that server restarts. A distributed cache instead runs as a separate fleet of cache servers that every application server talks to over the network, so the cached data is shared across the whole app.
The data is stored as key-value pairs in memory. A key might be user:8421 and the value the serialized user record. Because it lives in RAM, a lookup typically returns in well under a millisecond, against the 1 to 50 milliseconds a database query often takes.
The word distributed matters because the dataset is too big or too busy for one machine. Instead of one cache server holding everything, the keys are split across many nodes. Redis Cluster and Memcached are the two implementations almost everyone reaches for; Hazelcast and Apache Ignite cover the JVM world.
The core trick is deciding which node owns which key. Naively you could do hash(key) % N where N is the node count, but that breaks badly when you add or remove a node because almost every key suddenly maps somewhere new and the whole cache effectively empties. To avoid that, distributed caches use consistent hashing (Memcached clients) or hash slots (Redis Cluster splits the keyspace into 16384 fixed slots and assigns slots to nodes). Adding a node then only moves a small fraction of the keys.
Clients usually compute the target node themselves and connect directly, so there is no central router to bottleneck on. Reads and writes go straight to the owning node.
To survive failures, nodes are replicated. In Redis Cluster each primary (master) shard has one or more replicas that receive a copy of its writes; if the primary dies, a replica is promoted. Memcached has no built-in replication, so a lost node means that slice of cached data is simply gone and gets refilled from the database on the next request.
The application is responsible for keeping the cache in sync with the database. The common pattern is cache-aside: on a read, check the cache, and on a miss load from the database and write the value back into the cache with a TTL (time to live) so stale entries eventually expire. On a write, update the database and either delete or update the cached key.
Reach for a distributed cache when reads heavily outnumber writes, the same data is requested over and over, and your database is becoming the bottleneck. Session storage, user profiles, product catalogs, rendered fragments, rate-limit counters, and leaderboards are classic fits. It is also the natural way to share state across a horizontally scaled, stateless app tier.
The main cost is that you now have a second source of truth that can disagree with the database. Cache invalidation is genuinely hard: forget to invalidate after a write and users see stale data. TTLs bound how stale data can get but do not eliminate the problem.
Other trade-offs: a cache miss is slower than no cache at all because you pay a network round trip before falling back to the database, so a cold cache or a low hit rate can hurt. Memory is expensive, so you size the cache for the hot subset of data and rely on an eviction policy such as LRU (least recently used) to drop the rest. You also need to plan for the thundering herd, where a popular key expires and thousands of requests all miss and hammer the database at once.
Picture an e-commerce product page that gets 50,000 views a minute for a popular item. Without a cache, every view runs the same product query against PostgreSQL, and the database melts. With a distributed cache, the first request loads the product, writes it to the cache under key product:SKU123 with a 60 second TTL, and the next 49,999 requests in that minute all read from RAM in under a millisecond. The database sees roughly one query per item per minute instead of tens of thousands.
When the seller changes the price, the application updates PostgreSQL and deletes product:SKU123 from the cache. The next view misses, reloads the fresh row, and repopulates the cache. That is cache-aside in practice, and it is how most large sites keep their databases alive under load.
Redis
The dominant distributed cache; Redis Cluster shards data across nodes using 16384 hash slots with replica failover.
Memcached
A simpler, pure key-value cache that many clients shard with consistent hashing; used at scale by Facebook for its social graph.
Amazon ElastiCache
AWS managed Redis and Memcached, handling sharding, replication, and failover so teams do not run the cluster themselves.
Netflix EVCache
Netflix's Memcached-based distributed cache that replicates data across availability zones to keep latency low and survive zone loss.
A local (in-process) cache lives inside a single application server's memory and is not shared, so each server has its own copy and entries vanish on restart. A distributed cache runs as a separate cluster that all servers share, giving consistent data across the fleet and far more total capacity, at the cost of a network round trip per lookup.
Redis on a single node is just an in-memory store. It becomes a distributed cache when you run Redis Cluster, which shards the keyspace across multiple nodes using hash slots and adds replicas for failover. Many teams also run Redis as a single large cache without clustering, which works until one node can no longer hold the data or serve the traffic.
It depends on the implementation. Redis Cluster keeps replicas of each shard and promotes a replica to primary if the original dies, so the data survives. Memcached has no replication, so losing a node loses that slice of cached data, which then gets refilled from the database on the next miss. Consistent hashing or hash slots ensure only the failed node's share of keys is affected.
Invalidation is removing or updating a cached entry when the underlying data changes so clients do not read stale values. It is hard because the cache and the database are two separate stores that can drift, and a single missed delete can serve wrong data indefinitely. Most teams combine explicit deletes on write with TTLs that bound how stale any entry can become.
That is a cache miss. In the cache-aside pattern the application then reads the value from the database, writes it back into the cache with a TTL, and returns it. A miss is slower than having no cache for that request because you pay the cache lookup plus the database query, which is why a high hit rate is what makes a cache worthwhile.
Redis
An in-memory data store used as a cache, message broker, and database. Blazing fast because everything lives in RAM.
Memcached
A simple, high-performance distributed memory caching system. Stores key-value pairs in RAM. Simpler than Redis but less feature-rich.
Consistent Hashing
A hashing technique where adding or removing servers only moves a small fraction of keys. Used by Amazon DynamoDB and Cassandra for data distribution.
Cache Stampede
When many requests hit the database simultaneously because a popular cache entry expired. Solved with locking, probabilistic early expiration, or request coalescing.
CDN
A network of servers distributed globally that caches content close to users. Netflix uses CDNs to stream video from servers near you, not from one central location.
Cache Eviction
The process of removing entries from a cache when it's full. Common policies: LRU (least recently used), LFU (least frequently used), FIFO (first in, first out).
This page explains the idea. The "Distributed Cache" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Distributed Cache lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Distributed Cache in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
