We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningAdvanced · Distributed Systems Core
An uneven distribution of load where one node, shard, or partition receives disproportionately more traffic than others. Caused by poor shard keys or skewed access patterns.
In short
A hot spot is when one node, shard, or partition in a distributed system gets a disproportionate share of the traffic while its peers sit nearly idle. It usually comes from a bad partition key or a skewed access pattern, and it turns one machine into the bottleneck even though you added more machines.
In short
A hot spot is when one node, shard, or partition in a distributed system gets a disproportionate share of the traffic while its peers sit nearly idle. It usually comes from a bad partition key or a skewed access pattern, and it turns one machine into the bottleneck even though you added more machines.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Hot Spot across a run of lessons, not one page. These 4 alone are about 95 minutes of step-by-step reading, 1 of them with code you run in the browser, all with a quiz.
Back up while the system is running and serving traffic, zero downtime data protection.
How a distributed cache spreads keys across nodes with consistent hashing and virtual nodes. Measured tests show why a hot key still overloads a node.
Find the constraint limiting system performance, profiling, tracing, and load testing.
Model system behavior as a finite set of states and transitions, the foundation of distributed protocols.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Hot Backuppay once,
yours for life
You split data across many servers so no single machine has to do everything. The promise is that with 10 shards, each one handles about a tenth of the load. A hot spot breaks that promise. One shard ends up serving most of the reads and writes while the other nine barely sweat.
The symptom is easy to spot once you look: total throughput is fine on paper, but latency spikes and one node's CPU or disk is pinned at 100 percent while the rest report 10 to 20 percent. Adding more nodes does not help, because the extra capacity lands on the cool shards, not the hot one.
Hot spots come in two flavors. A data hot spot is when one partition holds far more rows than the others. A traffic hot spot is when the rows are spread evenly but one key gets hammered, like the row for a celebrity account or a flash-sale product.
Most systems decide where a key lives by hashing it or by range. With range partitioning, if you key on something monotonic like a timestamp or an auto-increment ID, every new write lands on the same partition, the one holding the newest range. That partition becomes a write hot spot while older partitions go read-only. This is exactly the trap people hit when they use a sequential row key in HBase or Bigtable.
With hash partitioning the data spreads evenly, but a single popular key still maps to a single partition. Hashing user_id does nothing to spread the load of one user_id that gets a million reads a second. The hash sent it all to one place on purpose, so it is reachable.
Low-cardinality keys are another source. If you partition by country and 60 percent of traffic is from one country, that shard carries 60 percent of the load no matter how many shards you have. The key simply does not have enough distinct values to spread work across the cluster.
The first defense is choosing a partition key with high cardinality and even access. Pick something that produces many distinct values and is queried uniformly. If you must key on time, prefix it with a hash or a shard number so writes scatter across partitions instead of stacking on the newest one. This is often called salting the key.
For a single scorching key, you split it. Salting appends a random suffix so one logical key becomes key_0 through key_9 across ten partitions, and a read fans out to all ten. That kills the write hot spot but makes reads more expensive because you now query ten places and merge. Caching the hot key in front of the database is usually cheaper, since a celebrity profile read a million times can be served from Redis instead of the shard.
Some systems rebalance automatically. They detect a partition that is too large or too busy and split it, then move half to another node. DynamoDB adaptive capacity does this by shifting throughput toward the hot partition within seconds. The trade-off is that detection takes time, so you still eat latency during the spike before the system reacts.
Twitter has the classic celebrity problem. A normal user has a few hundred followers, so writing a tweet to every follower's timeline is cheap. A user with 100 million followers would create a write hot spot on whatever shards hold those follower lists every time they post.
Twitter solved it with a hybrid fan-out. For ordinary accounts they push the tweet into each follower's precomputed timeline. For accounts above a follower threshold they skip the push and instead merge the celebrity's tweets in at read time when a follower loads their feed. This avoids the giant write burst on a single popular author while keeping reads fast for everyone else.
Amazon DynamoDB
Adaptive capacity and automatic partition splitting shift throughput to a hot partition so one popular key does not throttle the whole table.
Apache HBase / Google Bigtable
Both warn against sequential row keys because they create write hot spots on the newest region or tablet; salting or hashing the key prefix is the standard fix.
Redis
Used as a front cache for a single scorching key so the hot read is served from memory instead of pounding one database shard.
Hybrid timeline fan-out avoids a write hot spot from high-follower accounts by merging their tweets at read time instead of pushing to every follower.
Data skew means the data is unevenly distributed across partitions, so one holds far more rows. A hot spot is broader: it includes skew but also covers traffic that is uneven even when the data is balanced, like one popular key getting hammered. Skew often causes hot spots, but you can have a hot spot with perfectly even data.
Because the new servers only take on the cool partitions. The single overloaded partition still lives on one node, and the traffic for that key still routes there. You have to split or cache the hot key itself, not add capacity elsewhere.
Look for one node pinned near 100 percent CPU, disk, or network while peers are mostly idle, combined with rising tail latency. Per-partition metrics for request count and throughput will show one partition far above the rest. Most managed databases expose this directly.
Salting appends or prepends a value, often a small random number or hash, to spread one logical key across many partitions. It removes the hot spot on writes but makes reads more expensive, because a single logical read now has to query every salted variant and merge the results.
It prevents data skew from a sequential key, but not traffic hot spots. Hashing user_id spreads users evenly, yet one very popular user_id still maps to one partition and gets all its traffic there. You still need caching or key splitting for individual hot keys.
Shard Key
The column or field used to determine which shard a piece of data belongs to. A bad shard key creates hot spots; a good one distributes data and queries evenly.
Consistent Hashing
A hashing technique where adding or removing servers only moves a small fraction of keys. Used by Amazon DynamoDB and Cassandra for data distribution.
Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
CAP Theorem
When a network failure splits a distributed data store, each node must either refuse requests to stay consistent or keep answering with possibly stale data. It cannot guarantee both during the partition.
Consensus
The process of getting multiple nodes in a distributed system to agree on a single value. The foundation of distributed databases and coordination services.
Paxos
A family of protocols for solving consensus in unreliable networks. Famously difficult to understand but mathematically proven correct.
This page explains the idea. The course covers it step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
See the System Design Masterclass coursecourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Hot Spot in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
