We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Kubernetes & Containers
A K8s controller that automatically scales the number of pods based on CPU utilization, memory usage, or custom metrics. The Kubernetes equivalent of auto-scaling groups.
In short
The Horizontal Pod Autoscaler (HPA) is a Kubernetes controller that automatically changes how many replica pods a workload runs, scaling the count up or down based on observed metrics like CPU utilization, memory usage, or custom application metrics. It adds pods when load rises and removes them when load falls, so a Deployment matches demand without a human editing the replica count.
In short
The Horizontal Pod Autoscaler (HPA) is a Kubernetes controller that automatically changes how many replica pods a workload runs, scaling the count up or down based on observed metrics like CPU utilization, memory usage, or custom application metrics. It adds pods when load rises and removes them when load falls, so a Deployment matches demand without a human editing the replica count.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Horizontal Pod Autoscaler across a run of lessons, not one page. These 4 alone are about 85 minutes of step-by-step reading, every one with a quiz.
Automatically adjusting container count and resources based on demand. HPA, VPA, KEDA, and cluster scaling.
Adding more nodes to a system to distribute load.
Splitting a table by rows, the partitioning strategy behind every large-scale database deployment.
Automatically deploying every change that passes the pipeline to production, no human gate, no release ceremony.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Auto-Scalingpay once,
yours for life
The HPA is one of the built-in controllers that runs inside the Kubernetes control plane, specifically inside the kube-controller-manager. You create an HorizontalPodAutoscaler object, point it at a workload (a Deployment, ReplicaSet, or StatefulSet), and tell it a target, for example keep average CPU at 60 percent across all pods, with a minimum of 2 replicas and a maximum of 20.
It is called horizontal scaling because it changes the number of pods, not the size of each pod. That is the opposite of the Vertical Pod Autoscaler (VPA), which changes the CPU and memory requests of a single pod. HPA gives you more identical copies; VPA gives you bigger copies.
It is often called the Kubernetes equivalent of an AWS Auto Scaling Group, but the level it works at is different. HPA scales pods inside a cluster. It does not add machines. When HPA wants more pods than the cluster has room for, a separate component (Cluster Autoscaler or Karpenter) adds nodes.
HPA runs a control loop on a fixed interval, every 15 seconds by default (set with --horizontal-pod-autoscaler-sync-period). On each tick it reads the current metric for the target pods, then computes a desired replica count with a simple ratio: desiredReplicas = ceil(currentReplicas * currentMetricValue / targetMetricValue). If average CPU is 90 percent and your target is 60 percent with 4 pods running, it wants ceil(4 * 90 / 60) = 6 pods.
To get those metrics, HPA queries the Kubernetes metrics APIs, not the pods directly. CPU and memory come from the metrics.k8s.io API, which is served by metrics-server (you must install it; it is not bundled). Custom and external metrics come from custom.metrics.k8s.io and external.metrics.k8s.io, usually backed by an adapter such as the Prometheus Adapter or KEDA.
Two safeguards stop it from thrashing. A tolerance (10 percent by default) means HPA ignores changes inside the dead band, so a target of 60 percent will not scale when CPU is 63 percent. Stabilization and scaling-policy rules slow down scale-down, with a 5 minute scale-down stabilization window by default, so a brief dip in traffic does not immediately kill pods you will need again a minute later.
HPA fits stateless, horizontally scalable workloads where load varies over time: web servers, API backends, queue consumers, anything where adding identical replicas spreads the work. It does not fit singleton workloads or anything that cannot run as multiple copies, and you should not point HPA and VPA at the same CPU or memory metric at once because they fight each other.
The biggest practical limit is reaction time. HPA reacts to metrics it can already see, so by the time CPU is high, users are already feeling it, and new pods still need time to schedule and warm up. For sharp traffic spikes, scale on a leading signal like queue depth or requests per second rather than CPU, and keep min replicas high enough to absorb the first burst.
Tuning matters. Set CPU and memory requests on your pods, because utilization is measured as a percentage of the request; with no request, percentage-based HPA cannot work. Pick a target that leaves headroom (60 to 70 percent CPU is common), set max high enough that you do not hit the ceiling during real peaks, and confirm Cluster Autoscaler or Karpenter can supply nodes, otherwise HPA will create pods that sit Pending forever.
Say you run an order API as a Deployment with 3 pods, each requesting 500m CPU. You apply an HPA with minReplicas 3, maxReplicas 30, and a target of 70 percent average CPU. During a normal afternoon the pods sit at 40 percent CPU, so HPA holds at 3 replicas.
A flash sale starts. Traffic triples, average CPU jumps to 95 percent. On its next 15 second tick HPA computes ceil(3 * 95 / 70) = 5, then on later ticks keeps climbing toward maxReplicas as load stays high, eventually settling around the count that holds CPU near 70 percent. When the sale ends and CPU drops, HPA waits out the 5 minute scale-down window and then gradually trims back toward 3.
In the v2 API you can combine signals: scale on whichever of average CPU, average memory, or a custom metric (like requests-per-second from Prometheus) demands the most replicas. HPA always picks the highest of the candidate counts, so the busiest dimension wins.
Kubernetes
HPA is a native part of upstream Kubernetes; the autoscaling/v2 API ships in every modern cluster.
Amazon EKS
Runs HPA alongside metrics-server and pairs it with Cluster Autoscaler or Karpenter to add EC2 nodes when pods cannot be scheduled.
Google GKE
Ships metrics-server enabled by default and adds Multidimensional Pod Autoscaling and a custom metrics Stackdriver adapter on top of standard HPA.
KEDA
A CNCF project that extends HPA with event-driven external metrics, letting you scale on Kafka lag, queue length, or HTTP rate, including scale-to-zero.
HPA changes the number of pods for a workload inside the cluster. The Cluster Autoscaler (or Karpenter) changes the number of nodes (machines) in the cluster. They work together: HPA asks for more pods, and if there is no room, the Cluster Autoscaler adds nodes so those pods can be scheduled.
HPA scales out by adding more identical pods (horizontal). VPA scales up by increasing the CPU and memory requests of each pod (vertical). Do not target the same CPU or memory metric with both at once, because they will fight over the same signal.
Almost always the metrics pipeline is missing. For CPU and memory you need metrics-server installed and your pods need resource requests set, since utilization is a percentage of the request. For custom metrics you need an adapter like the Prometheus Adapter or KEDA exposing the custom.metrics.k8s.io API.
HPA checks metrics every 15 seconds by default and only acts on what it already measures, so there is built-in lag, plus the time to schedule and warm up new pods. For sharp spikes, scale on a leading signal like queue depth or requests per second instead of CPU, and keep minReplicas high enough to absorb the first burst.
Standard HPA cannot; its minReplicas must be at least 1. Scale-to-zero needs an event-driven layer such as KEDA or Knative, which can bring a workload up from zero when an event arrives and back down to zero when idle.
Auto Scaling
Automatically adding or removing compute instances based on current demand. Scales out during traffic spikes and scales in during quiet periods to save cost.
Kubernetes Deployment
A K8s resource that manages rolling updates and rollbacks for a set of pods. You declare the desired state and K8s converges to it.
Kubernetes
An orchestration platform that automates deploying, scaling, and managing containerized applications. K8s is the operating system for your cloud.
Docker
A platform for packaging applications into lightweight, portable containers. 'Works on my machine' becomes 'works everywhere.'
Container
A lightweight, isolated environment that packages an application with its dependencies. Shares the host OS kernel, unlike VMs. Starts in milliseconds.
Kubernetes Pod
The smallest deployable unit in Kubernetes: one or more containers sharing network and storage. Pods are ephemeral; if one dies, K8s creates a replacement.
This page explains the idea. The "Auto-Scaling" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Auto-Scaling lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Horizontal Pod Autoscaler in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
