We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Core Fundamentals
A system's ability to handle growing amounts of work by adding resources. A scalable system maintains performance as load increases.
In short
Scalability is a system's ability to handle a growing amount of work by adding resources, without performance falling apart. A scalable system keeps response times and throughput roughly steady as users, data, or requests increase, instead of slowing to a crawl or crashing.
In short
Scalability is a system's ability to handle a growing amount of work by adding resources, without performance falling apart. A scalable system keeps response times and throughput roughly steady as users, data, or requests increase, instead of slowing to a crawl or crashing.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Scalability across a run of lessons, not one page. These 4 alone are about 80 minutes of step-by-step reading, every one with a quiz.
How systems grow to handle more users, more data, and more traffic without falling apart.
How much work a system can handle in a given time period.
Two fundamental architecture patterns that shape how systems handle data, scale, and recover from failure.
Adding resources to a single node to handle more load.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Scalabilitypay once,
yours for life
Scalability is about how a system responds when you ask it to do more. More users sign up, more requests arrive per second, more data piles up. A scalable system absorbs that growth by adding hardware or capacity, and its performance stays close to flat. A system that is not scalable hits a wall: latency climbs, requests time out, and adding a bigger machine stops helping.
The word gets used loosely, so it helps to separate it from raw performance. Performance is how fast the system is for one request right now. Scalability is how that speed holds up as load multiplies. A service can be fast for 100 users and fall over at 100,000. That service is fast but not scalable.
There are two broad ways to add capacity. Vertical scaling means making one machine bigger: more CPU cores, more RAM, faster disks. Horizontal scaling means adding more machines and spreading work across them. Vertical scaling is simple but has a hard ceiling and a single failure point. Horizontal scaling has no real ceiling but forces you to deal with distributing state, balancing load, and keeping data consistent across nodes.
Horizontal scaling needs a way to route incoming requests across many identical servers. A load balancer like NGINX, HAProxy, or AWS Application Load Balancer sits in front and distributes traffic using round robin, least connections, or hashing. For this to work cleanly, the application servers should be stateless: any server can handle any request because session data lives somewhere shared, usually Redis or a database, not in local memory.
The database is almost always the part that scales last and breaks first, because it holds state. Read load is handled by adding read replicas and sending queries to them while writes go to a primary. Write load is harder. When one primary can no longer keep up, you shard: split the data across many databases by a key such as user ID, so each shard owns a slice of the rows. Discord shards by server ID, Instagram shards by user ID.
Caching is the cheapest scaling lever. Putting Redis or Memcached in front of the database, or a CDN like Cloudflare in front of static content, removes a huge fraction of work before it ever reaches the slow components. A cache hit ratio of 90 percent means the database only sees one tenth of the traffic.
The enemy of scaling is the shared bottleneck. Amdahl's law captures this: if even 5 percent of the work must happen on a single serialized resource, adding machines past a point gives almost nothing. Real scaling work is mostly hunting down and removing those single chokepoints.
Do not scale before you need to. A single well tuned PostgreSQL instance on a large machine comfortably serves tens of thousands of users and millions of rows. Sharding, multi-region replication, and microservices add real operational cost and new failure modes. Stack Overflow famously ran one of the busiest sites on the internet on a handful of servers with vertical scaling for years.
Horizontal scaling buys near unlimited headroom but at the price of complexity. Distributed systems force you to reason about partial failure, network partitions, and consistency. The CAP theorem says that during a network partition you must choose between consistency and availability. Sharding makes joins and transactions across shards painful. Every node you add is another thing that can break at 3 AM.
The practical path is to scale in the order that gives the most return per unit of pain: add caching first, then read replicas, add a CDN, scale the app tier horizontally behind a load balancer, and only shard the database when there is genuinely no other option. Auto scaling on AWS or Kubernetes lets the app tier grow and shrink with traffic so you pay for capacity only when load is high.
Picture a ticketing site that normally serves 500 requests per second, then a popular concert goes on sale and traffic jumps to 50,000 requests per second in minutes. Without scalability, the single app server saturates its CPU, the request queue backs up, latency goes from 80 milliseconds to 30 seconds, and users see errors.
A scalable design handles the spike. A CDN serves the static event pages so most reads never hit the origin. Kubernetes auto scaling spins the stateless app tier from 4 pods to 60 within a couple of minutes as CPU rises. Sessions and the seat inventory hot path live in Redis, so any pod can serve any user. A message queue like Kafka buffers the actual purchase writes so the database is fed a steady stream instead of a flood.
When the sale ends and traffic drops back to 500 requests per second, the pods scale back down and the bill returns to normal. Steady response times through a 100x swing in load, with cost that tracks demand, is what scalability looks like in practice.
Netflix
Runs thousands of stateless microservices on AWS that auto scale with viewing demand, peaking during evening prime time across regions.
Scaled to over a billion users by sharding its PostgreSQL data by user ID across many database servers and caching aggressively with Memcached.
Discord
Handles billions of messages by sharding storage by server ID and moving its message database from Cassandra to ScyllaDB for higher throughput per node.
Cloudflare
Scales read and static traffic for millions of sites by caching content at edge locations close to users, absorbing load before it reaches origin servers.
Performance is how fast the system handles one request right now. Scalability is whether that speed holds up as load grows. A system can be fast for a few users and still fail to scale to many. You can have one without the other.
Vertical scaling means making one machine bigger by adding CPU, RAM, or faster disks. It is simple but has a hard ceiling and one point of failure. Horizontal scaling means adding more machines and spreading the work across them. It has effectively no ceiling but requires load balancing, stateless servers, and handling distributed data.
Application servers can be made stateless and cloned freely, but the database holds state that every node depends on. You scale reads with replicas, but scaling writes forces sharding, which makes cross shard joins and transactions difficult. That is why caching and read replicas come before sharding.
Linear scaling means doubling the resources roughly doubles the capacity. It is the ideal. In reality shared bottlenecks and coordination overhead cause diminishing returns, which Amdahl's law describes: even a small serial fraction of the work caps how much extra machines can help.
Usually no. Premature scaling adds complexity, cost, and failure modes you may never need. A single tuned database and a small app tier serves most products well into tens of thousands of users. Add caching, replicas, and sharding only as real load demands it.
Throughput
How much work a system can handle in a given time period
Stateless vs Stateful Systems
Two fundamental architecture patterns that shape how systems handle data, scale, and recover from failure
Vertical Scaling
Adding resources to a single node to handle more load
Horizontal Scaling
Adding more nodes to a system to distribute load
Elasticity
Dynamic resource adjustment based on real-time demand
Horizontal Scaling
Adding more machines to handle increased load (scaling out). Like opening more checkout lanes instead of making one cashier faster.
Vertical Scaling
Making a single machine more powerful (more CPU, RAM, storage). Simpler but has physical limits. Also called 'scaling up.'
Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Throughput
The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.
Bandwidth
The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.
This page explains the idea. The "Scalability" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Scalability lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Scalability in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
