We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
6 system design questions lean on this idea. Each walks through the full answer.
System Design Masterclass · Database Fundamentals
Free preview: the first 2 parts of this 16-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Database Fundamentals
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
You've done everything right. You've added read replicas to scale reads. You've partitioned your tables to keep them manageable. You've tuned your indexes, optimized your queries, maxed out the RAM and SSDs on your primary server.
And it's still not enough.
Your primary database server handles 15,000 writes per second. Your application needs 50,000. You can't add more replicas, replicas don't help with write . You can't buy a bigger server, you're already on the largest available instance. You can't partition more aggressively, all partitions are on the same server, sharing the same CPU, memory, and disk I/O.
You've hit the ceiling of vertical scaling. The only way forward is horizontal scaling for writes, splitting your data across multiple independent database servers, each handling a fraction of the total write load.
This is database . It's the most complex database architecture pattern you'll encounter. It introduces problems that don't exist in single-server systems: cross-shard queries, distributed transactions, rebalancing, and operational overhead that can consume an entire engineering team.
Every major company that has sharded their database. Instagram, Uber, Pinterest, Slack, Figma, describes it as one of the hardest engineering projects they've undertaken. Nobody shards for fun. You shard because you have no other choice.
is horizontal partitioning where each partition (called a shard) lives on a separate database server. Each shard is an independent database with its own CPU, memory, disk, and network connection.
The key differences between partitioning and sharding:
| Partitioning | Sharding | |
|---|---|---|
| Partitions on | One server | Multiple servers |
| Write scaling | No (shared resources) | Yes (independent resources) |
| Routing | Database engine (transparent) | Application or (explicit) |
| Transactions | Normal (same server) | Distributed (cross-server) |
| JOINs | Normal | Cross-shard: very expensive or impossible |
| Complexity | Moderate | Very high |
A sharded system needs a routing layer that knows which shard holds the data for a given query. This can be the application itself (application-level sharding), a proxy (like Vitess or ProxySQL), or the database's built-in sharding (like MongoDB's mongos or CockroachDB's automatic sharding).