Caching
Storing frequently accessed data in a faster storage layer so you don't have to fetch it from the original (slower) source every time.
What is Caching?
In short
Caching keeps a copy of frequently used data in a faster, closer storage layer so most requests are served without touching the slow original source. Done well it cuts latency from hundreds of milliseconds to single digits and removes most of the load from your database, at the cost of having to decide when the copy is allowed to be stale.
Why caching matters
Fetching data from its original source is often the slowest part of handling a request. A database query, a call to another service, or rendering a result from scratch can take tens or hundreds of milliseconds. If the same data is requested again and again, repeating that work every time is wasteful and slow.
A cache stores a copy of that data in a faster, closer place: memory instead of disk, a server in the same data center instead of one across the world, an edge location instead of your origin. When the next request can be answered from the cache, it skips the expensive work entirely. The two payoffs are lower latency for users and far less load on the systems behind the cache, which is often what lets a service survive a traffic spike at all.
Cache-aside: the most common pattern
The default caching pattern is cache-aside, also called lazy loading. The application checks the cache first. If the value is there, that is a cache hit and it is returned immediately. If it is not, that is a cache miss: the application reads from the database, writes the value into the cache for next time, and then returns it. The first request for a key pays the full cost once; every request after that is fast until the entry expires.
The appeal of cache-aside is that the cache only ever holds data that has actually been requested, and the application stays in control. The trade-off is that the very first read of each key is always a miss, and if the underlying data changes you have to invalidate or update the cached copy yourself, or it will serve stale data until it expires.
Cache-aside (lazy loading)
The app talks to the cache first. A hit returns instantly; a miss reads the database, fills the cache, then returns.
Write strategies and the staleness problem
Reads are only half the story. When data changes, you choose how the cache and the database stay in sync. Write-through updates the cache and the database together on every write, so the cache is always current but writes are a little slower. Write-back updates the cache immediately and writes to the database later in the background, which is fast but risks losing recent writes if the cache fails before they are flushed. Write-around writes only to the database and lets the cache fill on the next read, which avoids caching data that may never be read again.
Every cache lives with one hard truth: a cached copy can become stale the moment the source changes. The whole craft of caching is deciding how much staleness is acceptable and how you bound it, which is what time to live is for.
Time to live and eviction
Time to live, or TTL, is an expiry stamped on each cached entry. When the TTL passes, the entry is considered stale and is refetched on the next request. TTL is the simplest way to bound staleness: a one-minute TTL means a user might see data up to a minute old, which is fine for a product listing and unacceptable for an account balance. You pick the TTL per kind of data based on how fresh it has to be.
Caches are also bounded in size, so when they fill up they must evict something to make room. The common policy is LRU, least recently used, which throws out whatever has not been touched for the longest time, on the assumption that recently used data is most likely to be used again. Between TTL expiry and size-based eviction, an entry leaves the cache either because it got too old or because it was crowded out.
In a real system
A CDN is caching applied to the network: copies of content sit at edge locations near users, so a hit never travels back to the origin.
Where it is used in production
Redis
The most widely used in-memory cache, holding hot keys in RAM with per-key TTLs and a choice of eviction policies including LRU.
Memcached
A simple, fast in-memory key-value cache built specifically for caching, used heavily by large sites to take read load off databases.
Cloudflare
A global CDN that caches content at edge locations close to users, so most requests are served without ever reaching the origin.
Varnish
An HTTP accelerator that sits in front of web servers and caches whole responses in memory for very high read throughput.
Frequently asked questions
- What is caching?
- Caching stores a copy of frequently used data in a faster, closer storage layer so most requests are served without recomputing or refetching from the slow original source. It lowers latency for users and reduces load on the systems behind it, at the cost of managing staleness.
- What is the cache-aside pattern?
- In cache-aside (lazy loading) the application checks the cache first. On a hit it returns the cached value. On a miss it reads the database, writes the value into the cache, and returns it. The cache only holds data that has actually been requested, and the application is responsible for keeping it fresh.
- What is the difference between write-through and write-back caching?
- Write-through updates the cache and the database together on every write, so the cache is always current but writes are slightly slower. Write-back updates the cache immediately and writes to the database later in the background, which is faster but can lose recent writes if the cache fails before flushing.
- What is TTL in caching?
- TTL (time to live) is an expiry time on a cached entry. Once the TTL passes, the entry is treated as stale and refetched on the next request. It is the simplest way to bound how old cached data can be, and you set it per kind of data based on how fresh it must be.
- What is cache eviction and what is LRU?
- A cache has limited space, so when it fills up it evicts entries to make room. LRU (least recently used) is the common policy: it removes whatever has not been accessed for the longest time, on the assumption that recently used data is most likely to be needed again.
Learn Caching hands-on
This page explains the idea. The full lesson lets you step through the ring as servers join and leave, read the implementation, and check yourself with a quiz. It is one of 760+ lessons in the System Design Masterclass, from your first API call to distributed consensus. Eleven Foundation lessons are free, no signup. Lifetime access is ₹499 in India or $7.99 worldwide, one payment, no subscription.
Related lessons
Lessons that touch on Caching as part of a larger topic.
Client-Side Caching
Caching data in the client application, beyond browser HTTP cache
foundation · caching strategies
Application-Level Caching
Distributed caching with Redis and Memcached, the shared brain across your servers
foundation · caching strategies
Server-Side Caching
Caching at the server and infrastructure level, reverse proxies, CDNs, and full-page caching
foundation · caching strategies
Result Set Caching
Caching the final query result, not raw pages, but the exact rows your application needs
foundation · caching strategies
Browser Caching
How browsers store resources locally to avoid re-downloading them
foundation · caching strategies
See also
Related glossary terms you might want to look up next.
Redis
An in-memory data store used as a cache, message broker, and database. Blazing fast because everything lives in RAM.
Memcached
A simple, high-performance distributed memory caching system. Stores key-value pairs in RAM. Simpler than Redis but less feature-rich.
CDN
A network of servers distributed globally that caches content close to users. Netflix uses CDNs to stream video from servers near you, not from one central location.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Throughput
The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.
Bandwidth
The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.