Scaling — HPA, VPA, and Cluster Autoscaler

Kubernetes scaling operates on three independent axes: add more pods (horizontal), make pods bigger (vertical), and add more nodes (cluster). Most teams only use HPA and call it done, but real production scaling requires all three working together. The subtlety is that they interact — HPA adds pods, but if there's no node capacity, those pods sit Pending until the Cluster Autoscaler provisions a new machine. Get the chain wrong and your scaling is a bottleneck instead of a solution.

Autoscaling in Action: HPA (Pods) + Cluster Autoscaler (Nodes)

Horizontal Pod Autoscaler (HPA)

HPA is the workhorse. It watches metrics for your pods and adjusts the replica count up or down. The default metric is CPU utilization, but you can use memory, custom metrics (request latency, queue depth), or external metrics (SQS queue length, Pub/Sub backlog).

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Pods
          value: 4
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

The behavior section is critical and often overlooked. Without it, HPA can oscillate — scaling up aggressively on a spike, then scaling down too fast, then back up. The stabilization window prevents flapping: it holds the current replica count for the specified duration before scaling down. Scale up fast, scale down slowly — that's the principle.

How HPA calculates replicas: desiredReplicas = ceil(currentReplicas × (currentMetricValue / targetMetricValue)). If you have 3 pods at 90% CPU and the target is 70%, the formula says ceil(3 × 90/70) = ceil(3.86) = 4. Simple, but the math matters when you're debugging unexpected scaling behavior.

Vertical Pod Autoscaler (VPA)

VPA adjusts the CPU and memory requests for individual pods based on observed usage. If your pod requests 500m CPU but consistently uses 100m, VPA recommends lowering the request — saving resources and improving bin-packing. If it uses 450m, VPA recommends raising the request to prevent throttling.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  updatePolicy:
    updateMode: "Auto"  # or "Off" for recommendations only
  resourcePolicy:
    containerPolicies:
      - containerName: web
        minAllowed:
          cpu: "100m"
          memory: "128Mi"
        maxAllowed:
          cpu: "2"
          memory: "2Gi"

The catch: VPA currently restarts pods to apply new resource values. There's no in-place resizing yet (it's an alpha feature in newer Kubernetes versions). This means VPA in Auto mode will evict and recreate pods. For most production workloads, teams run VPA in Off mode to get recommendations, then apply them manually during maintenance windows.

Don't run HPA and VPA on the same metric. If HPA scales based on CPU and VPA adjusts CPU requests, they fight each other. Use HPA for horizontal scaling on CPU/memory, and VPA only for right-sizing containers where HPA isn't active, or use VPA in recommendation mode only.

Cluster Autoscaler

Cluster Autoscaler watches for pods that can't be scheduled (Pending state) because no node has enough resources. When it detects pending pods, it adds nodes from pre-configured node groups. When nodes are underutilized, it cordons and drains them, moving pods elsewhere and terminating the machine.

Key behaviors:

Scale up: triggered when pods are in Pending state due to insufficient resources
Scale down: triggered when a node's utilization falls below a threshold (~50%) for a sustained period (default 10 minutes)
Respects PodDisruptionBudgets: won't drain a node if it would violate a PDB
Node group selection: picks the node group that would best fit the pending pod's resource requests

The latency is real: detecting a pending pod, deciding which node group to use, requesting a VM from the cloud provider, waiting for it to boot, registering with the cluster — this can take 2-5 minutes. For workloads that need faster scaling, pre-provision warm nodes or use Karpenter.

Karpenter

Karpenter (now a CNCF project) replaces Cluster Autoscaler with a smarter approach. Instead of pre-defined node groups, Karpenter looks at the aggregate resource needs of pending pods and provisions the right-sized instance directly. If you have pending pods needing 4 vCPU total, Karpenter picks an instance type that fits — it might choose a single 4-vCPU machine instead of two 2-vCPU machines from a node group.

Karpenter is faster (30-60 seconds vs. 2-5 minutes), handles heterogeneous workloads better, and automatically uses spot instances when appropriate. It also consolidates — if running nodes are underutilized, it replaces them with smaller ones.

KEDA — Event-Driven Scaling

KEDA (Kubernetes Event-Driven Autoscaling) extends HPA to scale based on event sources: message queue depth, database connections, HTTP request rate, cron schedules. KEDA can even scale to zero pods and back up, which standard HPA can't do (HPA minimum is 1).

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0
  maxReplicaCount: 50
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.us-east-1.amazonaws.com/123/orders
        queueLength: "5"  # 1 pod per 5 messages

Resource Requests and Limits — The Foundation

None of these autoscalers work correctly without proper resource requests. HPA needs accurate CPU/memory utilization data. Cluster Autoscaler uses requests to determine if a pod fits on a node. VPA analyzes actual usage against requests.

The best practice: set requests based on the P95 observed usage, set memory limits to prevent runaway processes, and consider omitting CPU limits to allow bursting.

Key Takeaways

HPA scales pods horizontally — use the behavior section to prevent flapping (scale up fast, scale down slowly)
VPA right-sizes pod resources but currently requires pod restarts — use recommendation mode in production
Cluster Autoscaler adds nodes when pods are Pending, but takes 2-5 minutes — budget for that latency
Karpenter is faster and smarter than Cluster Autoscaler, provisioning right-sized nodes in 30-60 seconds
KEDA enables scale-to-zero and event-driven scaling from message queues, databases, and custom sources
Set accurate resource requests on every container — the entire autoscaling chain depends on them