We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Core Fundamentals
Adding more machines to handle increased load (scaling out). Like opening more checkout lanes instead of making one cashier faster.
In short
Horizontal scaling means handling more load by adding more machines that work together, rather than making one machine bigger. Instead of upgrading a single server's CPU and memory, you run many identical servers behind a load balancer and split traffic across them.
In short
Horizontal scaling means handling more load by adding more machines that work together, rather than making one machine bigger. Instead of upgrading a single server's CPU and memory, you run many identical servers behind a load balancer and split traffic across them.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Horizontal Scaling across a run of lessons, not one page. These 4 alone are about 112 minutes of step-by-step reading, every one with a quiz.
Adding more nodes to a system to distribute load.
How systems grow to handle more users, more data, and more traffic without falling apart.
How load balancers decide which server gets the next request. Round Robin, Weighted, Least Connections, IP Hash, and more.
How load balancer failover works with a shared virtual IP, VRRP and keepalived. Also covers ALB health checks and failover across zones.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Horizontal Scalingpay once,
yours for life
Horizontal scaling, also called scaling out, is adding more servers to a system so the combined fleet handles more requests, more data, or more concurrent users. The opposite is vertical scaling, or scaling up, where you keep one machine and give it a faster CPU, more RAM, or more disk.
The checkout-lane analogy holds well. A single cashier can only ring up one customer at a time, and there is a hard ceiling on how fast that one person can move. Opening five more lanes lets six customers check out at once. Horizontal scaling opens more lanes; vertical scaling tries to make the one cashier superhuman.
The practical difference is the ceiling. The biggest single AWS EC2 instances top out at several hundred vCPUs and tens of terabytes of RAM, so vertical scaling eventually hits a wall and a single very large machine gets expensive fast. Adding more commodity machines has no such hard limit, which is why every large internet system scales out.
You run several identical copies of your application, each on its own machine or container, and put a load balancer in front of them. The load balancer spreads incoming requests across the instances using a policy like round robin, least connections, or a hash of the client IP. To the user it looks like one address; behind it, any of N servers can answer.
The hard part is state. If a server stores a user's session in its own memory, the next request might land on a different server that knows nothing about that user. The fix is to make application servers stateless and push shared state into something all of them reach, such as Redis for sessions or a shared Postgres database. Stateless app servers can then be added or removed freely.
Databases scale out differently because data cannot simply be duplicated and forgotten. Common techniques are read replicas to spread read traffic, and sharding, which splits rows across many database nodes by a key such as user_id. Systems like Cassandra and DynamoDB shard automatically with consistent hashing so adding nodes redistributes only a slice of the data instead of all of it.
Reach for horizontal scaling when load is variable or growing, when you need high availability, or when one machine simply is not big enough. Because there are many machines, the loss of one does not take the service down, and you can add capacity for a traffic spike then remove it afterward, which is exactly what autoscaling on Kubernetes or AWS Auto Scaling Groups does.
The cost is complexity. You now need a load balancer, health checks, service discovery, and a strategy for shared state and deployments across many nodes. Distributed systems also introduce failure modes a single box never has, such as network partitions and the consistency-versus-availability trade-offs described by the CAP theorem.
Vertical scaling is often the right first move precisely because it is simple. Doubling a server's RAM requires no code changes. A common pattern is to scale up until it gets costly or risky, then scale out once the architecture is ready, since stateless services and a shared data tier are what make scaling out cheap later.
Picture a single web server that comfortably serves 1,000 requests per second. A product launch pushes traffic to 8,000 requests per second and the server starts returning timeouts. You launch eight identical instances behind an Nginx or AWS Application Load Balancer, each handling roughly 1,000 requests per second, and the site responds normally again.
Because the app servers were made stateless, with sessions kept in Redis and data in Postgres, no user notices which of the eight machines served their request. When the launch traffic dies down that night, autoscaling terminates six of the instances so you only pay for what you use.
Netflix runs this pattern at enormous size. During peak evening hours it scales out to tens of thousands of stateless service instances across AWS regions, and scales back in overnight when demand drops, which would be impossible with one giant server.
Netflix
Runs thousands of stateless microservice instances on AWS Auto Scaling Groups, adding machines at peak viewing hours and removing them overnight.
Kubernetes
Horizontal Pod Autoscaler adds or removes identical pod replicas based on CPU, memory, or custom metrics so apps scale out automatically.
Amazon DynamoDB
Automatically shards data across many storage nodes using consistent hashing, so adding capacity redistributes only a portion of the keys.
Nginx
Commonly used as the load balancer that fans incoming requests out across a pool of identical backend servers.
Horizontal scaling adds more machines and splits work across them. Vertical scaling keeps one machine and makes it more powerful with more CPU, RAM, or disk. Vertical scaling is simpler but hits a hard ceiling and a single point of failure; horizontal scaling has effectively no ceiling but adds complexity.
Because any of the many servers can answer any request. If a server keeps a user's session only in its own memory, a follow-up request routed to a different server would lose that data. Pushing shared state into Redis or a database lets every server serve every user, so you can add or remove machines freely.
Yes, but it is harder than scaling app servers because data cannot just be duplicated. You use read replicas to spread reads, and sharding to split rows across nodes by a key such as user_id. Systems like Cassandra and DynamoDB do this automatically; relational databases usually need manual sharding.
No. Vertical scaling is often the right first step because it needs no code changes and no load balancer. Horizontal scaling wins when you need high availability, elastic capacity for traffic spikes, or scale beyond what one machine can give. A common path is to scale up first, then scale out once the architecture is stateless and ready.
Autoscaling is automated horizontal scaling. Tools like the Kubernetes Horizontal Pod Autoscaler or AWS Auto Scaling Groups watch metrics such as CPU usage and add machines when load rises, then remove them when load falls, so you only pay for the capacity you actually use.
Vertical Scaling
Making a single machine more powerful (more CPU, RAM, storage). Simpler but has physical limits. Also called 'scaling up.'
Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
Stateless
A system where each request contains all the information needed to process it. The server doesn't remember previous requests. Easier to scale horizontally.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Throughput
The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.
Bandwidth
The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.
This page explains the idea. The "Horizontal Scaling" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Horizontal Scaling lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Horizontal Scaling in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
