Horizontal Pod Autoscaler

The HPA is one of the built-in controllers that runs inside the Kubernetes control plane, specifically inside the kube-controller-manager. You create an HorizontalPodAutoscaler object, point it at a workload (a Deployment, ReplicaSet, or StatefulSet), and tell it a target, for example keep average CPU at 60 percent across all pods, with a minimum of 2 replicas and a maximum of 20.

It is called horizontal scaling because it changes the number of pods, not the size of each pod. That is the opposite of the Vertical Pod Autoscaler (VPA), which changes the CPU and memory requests of a single pod. HPA gives you more identical copies; VPA gives you bigger copies.

The shortDefinition is right that it is the Kubernetes equivalent of an AWS Auto Scaling Group, but the level it works at is different. HPA scales pods inside a cluster. It does not add machines. When HPA wants more pods than the cluster has room for, a separate component (Cluster Autoscaler or Karpenter) adds nodes.

HPA runs a control loop on a fixed interval, every 15 seconds by default (set with --horizontal-pod-autoscaler-sync-period). On each tick it reads the current metric for the target pods, then computes a desired replica count with a simple ratio: desiredReplicas = ceil(currentReplicas * currentMetricValue / targetMetricValue). If average CPU is 90 percent and your target is 60 percent with 4 pods running, it wants ceil(4 * 90 / 60) = 6 pods.

To get those metrics, HPA queries the Kubernetes metrics APIs, not the pods directly. CPU and memory come from the metrics.k8s.io API, which is served by metrics-server (you must install it; it is not bundled). Custom and external metrics come from custom.metrics.k8s.io and external.metrics.k8s.io, usually backed by an adapter such as the Prometheus Adapter or KEDA.

Two safeguards stop it from thrashing. A tolerance (10 percent by default) means HPA ignores changes inside the dead band, so a target of 60 percent will not scale when CPU is 63 percent. Stabilization and scaling-policy rules slow down scale-down, with a 5 minute scale-down stabilization window by default, so a brief dip in traffic does not immediately kill pods you will need again a minute later.

HPA fits stateless, horizontally scalable workloads where load varies over time: web servers, API backends, queue consumers, anything where adding identical replicas spreads the work. It does not fit singleton workloads or anything that cannot run as multiple copies, and you should not point HPA and VPA at the same CPU or memory metric at once because they fight each other.

The biggest practical limit is reaction time. HPA reacts to metrics it can already see, so by the time CPU is high, users are already feeling it, and new pods still need time to schedule and warm up. For sharp traffic spikes, scale on a leading signal like queue depth or requests per second rather than CPU, and keep min replicas high enough to absorb the first burst.

Tuning matters. Set CPU and memory requests on your pods, because utilization is measured as a percentage of the request; with no request, percentage-based HPA cannot work. Pick a target that leaves headroom (60 to 70 percent CPU is common), set max high enough that you do not hit the ceiling during real peaks, and confirm Cluster Autoscaler or Karpenter can supply nodes, otherwise HPA will create pods that sit Pending forever.

Say you run an order API as a Deployment with 3 pods, each requesting 500m CPU. You apply an HPA with minReplicas 3, maxReplicas 30, and a target of 70 percent average CPU. During a normal afternoon the pods sit at 40 percent CPU, so HPA holds at 3 replicas.

A flash sale starts. Traffic triples, average CPU jumps to 95 percent. On its next 15 second tick HPA computes ceil(3 * 95 / 70) = 5, then on later ticks keeps climbing toward maxReplicas as load stays high, eventually settling around the count that holds CPU near 70 percent. When the sale ends and CPU drops, HPA waits out the 5 minute scale-down window and then gradually trims back toward 3.

In the v2 API you can combine signals: scale on whichever of average CPU, average memory, or a custom metric (like requests-per-second from Prometheus) demands the most replicas. HPA always picks the highest of the candidate counts, so the busiest dimension wins.