Skip to content

Horizontal Pod Autoscaler case studies and implementations

The Kubernetes HPA scales replicas by comparing a current metric to a target, and the two case studies here (Deezer, Alibaba Cloud EDAS) both went beyond CPU and memory: Deezer with a custom Node.js metric, Alibaba with KEDA.

Summary

The Horizontal Pod Autoscaler (HPA) is the Kubernetes controller that adds or removes pod replicas based on a metric. Its core rule is a ratio between the desired and current metric value (kubernetes.io).

Two real-world implementations show where the defaults fall short. Deezer built a Prometheus-based pipeline around a custom load metric (infoq.com). Alibaba Cloud's EDAS platform standardized on KEDA (cncf.io).

Limits of coverage: the sources are one news write-up (InfoQ, 2024), one vendor blog (CNCF, 2021), the Kubernetes docs and one conference talk. They are not independent measurements.

Key ideas

CPU and memory can react too slowly. Relying on CPU and memory can delay scaling during sudden traffic spikes, which is why KEDA or custom metrics are suggested for faster response (infoq.com).

Pick a metric that tracks real load. Deezer found that event loop utilization (ELU, how busy a Node.js event loop is) represents server load better than CPU percentage (infoq.com).

Scale to zero is not native to the HPA. Alibaba's team listed missing scale-to-zero as a gap, since many users want it when a microservice is not being consumed (cncf.io). KEDA aims to cover this by supporting scale-to-zero (cncf.io).

How it works

The core calculation. The controller compares desired and current metric values. For example, a current value of 200m against a desired 100m doubles the replica count (kubernetes.io).

Metric sources. The HPA reads the resource metrics API for CPU and memory and a custom metrics API for others. Custom metrics are served through an adapter component (▶ 28:22). This is the 2018 design talk by Maciej Pytel and Solly Ross, so details may have changed since.

Damping flapping. When several recommendations arise, the controller can pick the highest within a configurable window, to avoid rapid downscaling (kubernetes.io). The behavior field sets this per HPA, for example a 300-second scaleDown stabilization window (kubernetes.io).

Case study: Deezer. Deezer collects metrics with Prometheus and uses the Prometheus Adapter to expose them to Kubernetes. Its custom metric is ELU (infoq.com). The team is now exploring KEDA for more flexibility (infoq.com).

Case study: Alibaba Cloud EDAS. EDAS required scale-to-zero out of the box. After evaluation, the team picked KEDA, which brings useful scalers (cncf.io). KEDA creates its own HPA objects, and the name length is limited to 63 characters (cncf.io).

Open questions

  • Neither case study publishes before/after numbers such as replica counts, cost or latency in the text ingested here.
  • Alibaba says it is working on AIOps-driven autoscaling decisions based on historical data, with no results reported (cncf.io).

Disagreements

Deezer reports that custom-metric HPA scaling has been beneficial, yet is still looking at KEDA (infoq.com). Alibaba, by contrast, went all the way to KEDA (cncf.io). Alibaba also lists KEDA drawbacks: triggers cannot be disabled, which may cause trouble in production (cncf.io).

References

Videos

  1. Horizontal Pod Autoscaler Reloaded - Scale on Custom Metrics - Maciej Pytel & Solly Ross (CNCF [Cloud Native Computing Foundation]), cited at 28:22

Web

  1. kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale
  2. Deezer Optimizes Kubernetes Autoscaling with Custom Metrics — infoq.com
  3. cncf.io/blog/2021/03/30/why-alibaba-cloud-uses-keda-for-application-autoscaling