TL;DR: CPU may miss an I/O or downstream bottleneck, while a CPU-bound service can benefit from CPU-based scaling. Measure detection and startup delay, then select a signal tied to the actual bottleneck, such as queue age or in-flight requests, size the floor for the spike you can predict, and accept load shedding for the part you cannot.
How to approach it
Diagnose before prescribing. Name the two independent reasons CPU autoscaling can fail during a spike (the metric may miss saturation, and new capacity takes time), then give the alternative signal and the honest limit of what autoscaling can ever do.
A strong answer
There are two failures and they compound.
CPU can be the wrong signal. If a downstream call or connection pool limits throughput, requests accumulate while CPU stays low. For CPU-bound work, CPU saturation can itself cause queueing, so CPU remains a useful signal. Use a load test to relate per-replica utilization, concurrency and queue age to latency; there is no universal relationship between 70% CPU and a tripled p99.
Scaling is slow. The control loop evaluates on an interval, then a pod must be scheduled, an image pulled if it is not cached, the process started, dependencies connected, caches warmed, and a readiness probe passed. Measure this end-to-end delay for your application; it may be seconds or minutes. A spike that rises faster than capacity becomes ready needs prewarmed headroom, buffering or admission control. On a node-scaling event, add the time to provision a machine.
So the fix is in three parts. Change the signal to something leading: queue depth, in-flight request count, or requests per second per pod, chosen according to processing time and dependency capacity. Queue depth is accumulated waiting work, not a prediction of future arrivals. Kubernetes supports this through custom and external metrics, and validate the metric’s relation to replica count before adopting it.
Then change the floor. Autoscaling is for the slow shape of the day; it is not for the spike. If your traffic has a known peak, a scheduled scale-up before it costs less than the outage. Where the spike is genuinely unpredictable, keep enough headroom that you survive the measured detection and capacity-startup window, and be explicit that this is the price of not having a queue in front.
Finally, accept that some load cannot be served. Rate limiting and load shedding at the ingress, dropping the lowest-value traffic first, protects the requests that matter. A system that degrades on the analytics endpoint to keep checkout alive is behaving correctly; one that falls over completely is not.
The other trap worth naming: scaling the stateless tier is the easy half. Doubling application pods against a database with a fixed connection limit converts a slow service into a completely broken one, so the connection pool and the data tier have to be in the plan.
What interviewers probe next
"What if the bottleneck is downstream?" Then scaling out makes it worse by increasing concurrency against a saturated dependency. That is where circuit breaking and a concurrency limit belong.
"How do you pick the target value?" From a load test that finds the point where latency degrades, then set the target below it. A number chosen because it looked reasonable is the usual reason this fails.
"Would KEDA or predictive scaling help?" KEDA can expose event-source metrics, but still needs a suitable target and cannot eliminate startup delay. Predictive scaling helps for repeating daily shapes and does nothing for a genuine surprise.
Common mistakes
Answering with the HPA formula. The question is about why it did not work, and the formula is not the problem.
Ignoring warm-up entirely. Startup time is often the dominant term and it is the one candidates most often omit.
Scaling only the compute tier and leaving the database and connection pools out of the answer.