Lesson 16 · Senior Kubernetes judgment

Autoscaling realism: KEDA, HPA, queueing, and scale-to-zero

How to explain autoscaling in terms of demand shape, queueing behavior, and operational trade-offs rather than treating every workload as a CPU-percentage problem.

Your win: explain why different workloads need different autoscaling signals, why KEDA changes the story materially, and how to talk honestly about scale-to-zero and event-driven behavior.

In plain English Plain English: good autoscaling chooses a signal that matches the real bottleneck, not just the easiest metric to graph.

Why this page matters more than it first appears

A lot of Kubernetes answers collapse autoscaling into “HPA scales on CPU.” That is not wrong, but it is much too small for many real systems. Backlog, queue depth, request rate, and wake-up latency often matter more than raw CPU. Once that is true, your autoscaling explanation has to become more honest and more workload-shaped.

This repo gives you that honesty directly. It does not use autoscaling only as a generic resource story. It uses KEDA heavily, scales notification consumers on Kafka lag, and treats VPA mostly as recommendation rather than automatic actuation. That is a better senior teaching setup because it shows how scaling policy follows workload shape, not just platform defaults.

This is also why strong autoscaling answers sound more nuanced than people expect. You are not just describing a controller. You are explaining which signal best represents pressure, which behavior triggers scaling, and what trade-offs show up when the system wakes up, cools down, or scales to zero.

The autoscaling rule Choose the scaling signal that best represents the real bottleneck or backlog for the workload, then be honest about the activation and cooldown trade-offs that follow.

What the repo teaches well

The notification consumer example is especially useful because it makes CPU look obviously incomplete. If messages are piling up, the urgent question is not “are cores busy enough?” It is “are we falling behind on queue work?” Kafka lag answers that question much more directly, which is why KEDA is such a natural fit here.

Scale-to-zero adds another useful trade-off. It can save cost dramatically, but it also introduces activation behavior and cold-start implications. So the senior answer is not just “KEDA can scale to zero.” It is “KEDA can scale to zero, which is powerful when idle capacity is wasteful, but you must still understand the wake-up and demand-shape consequences.”

Backend use case In this repo, KEDA dominates autoscaling, request-rate scaling is driven from Istio/Prometheus signals, notification consumers scale on Kafka lag, and VPA is commonly present in recommendation mode rather than full automatic mutation.

Common mistake Treating HPA on CPU as the default answer for every workload, even when backlog, event rate, or queue lag better describes the real scaling problem.

What stronger answers sound like

The strongest answers here sound like: “we scale on the signal that best represents the user-visible bottleneck or queueing pressure; in this repo that often means KEDA and event-driven triggers rather than CPU alone, with explicit trade-offs around cooldown, activation, and scale-to-zero behavior.”

That answer is stronger because it connects scaling policy to workload semantics, not just to the Kubernetes object name. It also sounds more like someone who has watched systems get behind and had to pick the least misleading signal.

How to get the most out of this page

Every time you see an autoscaler, ask two questions: “what pressure is this signal actually measuring?” and “what latency or cost trade-off follows from using it?” If you keep doing that, you will sound much less generic.

Compare HPA and KEDA with queueing in mind

Read the HPA and KEDA docs again, this time asking which signal best tracks pressure for request/queue-heavy systems.

→ Kubernetes — HPA
→ KEDA — Scaling deployments
→ Repo K8s map

Check yourself (from memory)

Q1. What makes an autoscaling policy operationally strong?

Good autoscaling follows workload semantics, not just default metric availability.

Why is queue lag often a better scaling signal than CPU?

recall, then click to reveal

Because queue lag measures whether work is accumulating faster than it is being processed, which often reflects user-visible pressure more directly than raw CPU utilization.

Want a “which signal would you scale on?” drill? Ask me.

Sources. Kubernetes HPA docs; KEDA docs; repo K8s map.