We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Core Fundamentals
Storing frequently accessed data in a faster storage layer so you don't have to fetch it from the original (slower) source every time.
In short
Caching keeps a copy of frequently used data in a faster, closer storage layer so most requests are served without touching the slow original source. Done well it cuts latency from hundreds of milliseconds to single digits and removes most of the load from your database, at the cost of having to decide when the copy is allowed to be stale.
In short
Caching keeps a copy of frequently used data in a faster, closer storage layer so most requests are served without touching the slow original source. Done well it cuts latency from hundreds of milliseconds to single digits and removes most of the load from your database, at the cost of having to decide when the copy is allowed to be stale.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Caching across a run of lessons, not one page. These 4 alone are about 102 minutes of step-by-step reading, every one with a quiz.
Temporary data storage for faster access and reduced load.
Caching the finished response removes all the work, and one personalised element removes all the benefit.
Caching a result set freezes it, and the deep pagination it is usually bought to fix disappears when you change the question instead.
The only cache with no purge, the only one that can answer with no network, and why revalidation gives up the thing that makes it valuable.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Cachingpay once,
yours for life
Fetching data from its original source is often the slowest part of handling a request. A database query, a call to another service, or rendering a result from scratch can take tens or hundreds of milliseconds. If the same data is requested again and again, repeating that work every time is wasteful and slow.
A cache stores a copy of that data in a faster, closer place: memory instead of disk, a server in the same data center instead of one across the world, an edge location instead of your origin. When the next request can be answered from the cache, it skips the expensive work entirely. The two payoffs are lower latency for users and far less load on the systems behind the cache, which is often what lets a service survive a traffic spike at all.
The default caching pattern is cache-aside, also called lazy loading. The application checks the cache first. If the value is there, that is a cache hit and it is returned immediately. If it is not, that is a cache miss: the application reads from the database, writes the value into the cache for next time, and then returns it. The first request for a key pays the full cost once; every request after that is fast until the entry expires.
The appeal of cache-aside is that the cache only ever holds data that has actually been requested, and the application stays in control. The trade-off is that the very first read of each key is always a miss, and if the underlying data changes you have to invalidate or update the cached copy yourself, or it will serve stale data until it expires.
Cache-aside (lazy loading)
The app talks to the cache first. A hit returns instantly; a miss reads the database, fills the cache, then returns.
Reads are only half the story. When data changes, you choose how the cache and the database stay in sync. Write-through updates the cache and the database together on every write, so the cache is always current but writes are a little slower. Write-back updates the cache immediately and writes to the database later in the background, which is fast but risks losing recent writes if the cache fails before they are flushed. Write-around writes only to the database and lets the cache fill on the next read, which avoids caching data that may never be read again.
Every cache lives with one hard truth: a cached copy can become stale the moment the source changes. The whole craft of caching is deciding how much staleness is acceptable and how you bound it, which is what time to live is for.
Time to live, or TTL, is an expiry stamped on each cached entry. When the TTL passes, the entry is considered stale and is refetched on the next request. TTL is the simplest way to bound staleness: a one-minute TTL means a user might see data up to a minute old, which is fine for a product listing and unacceptable for an account balance. You pick the TTL per kind of data based on how fresh it has to be.
Caches are also bounded in size, so when they fill up they must evict something to make room. The common policy is LRU, least recently used, which throws out whatever has not been touched for the longest time, on the assumption that recently used data is most likely to be used again. Between TTL expiry and size-based eviction, an entry leaves the cache either because it got too old or because it was crowded out.
In a real system
A CDN is caching applied to the network: copies of content sit at edge locations near users, so a hit never travels back to the origin.
Redis
The most widely used in-memory cache, holding hot keys in RAM with per-key TTLs and a choice of eviction policies including LRU.
Memcached
A simple, fast in-memory key-value cache built specifically for caching, used heavily by large sites to take read load off databases.
Cloudflare
A global CDN that caches content at edge locations close to users, so most requests are served without ever reaching the origin.
Varnish
An HTTP accelerator that sits in front of web servers and caches whole responses in memory for very high read throughput.
Caching stores a copy of frequently used data in a faster, closer storage layer so most requests are served without recomputing or refetching from the slow original source. It lowers latency for users and reduces load on the systems behind it, at the cost of managing staleness.
In cache-aside (lazy loading) the application checks the cache first. On a hit it returns the cached value. On a miss it reads the database, writes the value into the cache, and returns it. The cache only holds data that has actually been requested, and the application is responsible for keeping it fresh.
Write-through updates the cache and the database together on every write, so the cache is always current but writes are slightly slower. Write-back updates the cache immediately and writes to the database later in the background, which is faster but can lose recent writes if the cache fails before flushing.
TTL (time to live) is an expiry time on a cached entry. Once the TTL passes, the entry is treated as stale and refetched on the next request. It is the simplest way to bound how old cached data can be, and you set it per kind of data based on how fresh it must be.
A cache has limited space, so when it fills up it evicts entries to make room. LRU (least recently used) is the common policy: it removes whatever has not been accessed for the longest time, on the assumption that recently used data is most likely to be needed again.
Server-Side Caching
Caching the finished response removes all the work, and one personalised element removes all the benefit
Result Set Caching
Caching a result set freezes it, and the deep pagination it is usually bought to fix disappears when you change the question instead
Browser Caching
The only cache with no purge, the only one that can answer with no network, and why revalidation gives up the thing that makes it valuable
Client-Side Caching
A cache your code keeps on the user's device, why the usual objection to it is measurably wrong, and what actually limits it
Distributed Caching with Redis and Memcached
A shared cache is a network service, so how you call it matters far more than which product you picked
Redis
An in-memory data store used as a cache, message broker, and database. Blazing fast because everything lives in RAM.
Memcached
A simple, high-performance distributed memory caching system. Stores key-value pairs in RAM. Simpler than Redis but less feature-rich.
CDN
A network of servers distributed globally that caches content close to users. Netflix uses CDNs to stream video from servers near you, not from one central location.
Service Worker
A script the browser runs in the background, separate from the web page. Enables offline caching, push notifications, and background sync for progressive web apps.
Cache-Aside
A caching pattern where the application checks the cache first; on a miss, it fetches from the database and populates the cache. Also called lazy loading.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
This page explains the idea. The "Caching" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Caching lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Caching in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
