TL;DR: The scheduler places a pod once, based on requests, against the cluster as it looked at that moment. It never rebalances. So imbalance accumulates from inaccurate requests, from pods placed when the cluster was smaller, and from nodes added after the fact. Fix the requests first, add topology spread constraints, and use the descheduler for the drift that remains.
How to approach it
Explain that this is expected behaviour before treating it as a bug. Then separate the three causes, because they need different fixes and candidates usually name only one.
A strong answer
Scheduling is a one-time decision. When a pod is created, the scheduler filters nodes that cannot run it and scores the rest, then binds. From that moment the pod stays where it is until something evicts it. There is no background process moving pods to balance load, and that is deliberate: moving a running pod means killing it, and the default scheduler does not do that; separately configured consolidation or descheduling controllers can.
Three causes, in the order I would check.
Requests do not match reality. Scoring is based on requested resources, not usage. If pods request far more than they use, the scheduler spreads them thinly by its own arithmetic while actual utilisation is low and uneven. If they request far less, it packs them and the node is genuinely overloaded while the scheduler believes there is room. This is the root cause most of the time, and the fix is to derive requests from observed usage rather than from a number someone copied. A vertical pod autoscaler in recommendation mode is a fast way to get the data without letting it act.
The cluster changed after placement. Nodes added for unschedulable pods receive those pending workloads; already-bound pods do not move merely because new capacity appeared. A cluster that scaled up during a spike and back down leaves survivors concentrated on whichever nodes were not drained. This is drift rather than a scheduling error.
Placement was correct but is now wrong. A node cordoned for maintenance, a taint added, an affinity rule changed.
The fixes, in order of leverage.
Topology spread constraints are the most valuable and the least used. They express what you actually want, which is replicas distributed across nodes and zones rather than merely placed somewhere legal. The default scheduler has soft spreading preferences, but those are not a hard guarantee against placing replicas on one node, and the first thing you notice is a node failure taking the whole service down.
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels: { app: api }
ScheduleAnyway is the right default. DoNotSchedule turns an imbalance into unschedulable pods, which is usually a worse outcome.
Pod anti-affinity does something similar and is heavier to evaluate at scale; spread constraints largely superseded it.
The descheduler runs periodically and evicts pods that violate policies, so the scheduler places them again against the current cluster. It is one way to address drift; consolidation controllers can also evict and replace workloads under their own policies. It is also disruptive by definition, so it needs PodDisruptionBudgets to be respected and is best run on a schedule rather than continuously.
Node autoscaling distinguishes provisioning for pending pods from consolidation.
What interviewers probe next
"Why not rebalance automatically?" Because rebalancing means terminating running pods. Kubernetes leaves that decision to you, and the descheduler is opt-in for exactly that reason.
"Node shows 30% CPU but nothing schedules. Explain." Scheduling is on requests, monitoring on usage. The node's requests are committed even if the processes are idle.
"How does the cluster autoscaler interact?" It adds nodes when pods are Pending and removes underutilised ones, using requests. Inflated requests therefore cause it to add nodes you do not need, which is an imbalance problem and a cost problem at once.
Common mistakes
Treating it as a scheduler bug rather than as its documented behaviour.
Reaching for the descheduler first. If requests are wrong it will rebalance to another wrong placement.
Not using topology spread constraints, then being surprised when one node failure removes every replica.