DevOpsInterviewPrep logo
AI & GPU Infrastructure / 04
hardNewNVIDIAMetaMicrosoft

One node in your training fleet makes every job it touches 30 percent slower, but it passes health checks. How do you find and handle it?

GPUs fail gradually before they fail loudly. Xid errors, ECC retirement, thermal throttling and a degraded NVLink all produce a node that is up, schedulable, and slowing every gang it joins.

Updated Sep 2026 · Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Liveness checks answer whether the device responds, which is the wrong question. Run DCGM diagnostics plus a short collective benchmark on every node, compare each node against the fleet distribution rather than a fixed threshold, and automate cordon-and-drain on the signals that predict failure: Xid errors, rising ECC retirement, PCIe replay counts, sustained clock throttling and NVLink bandwidth below its peers.

How to approach it

Start from the observation that a slow node is worse than a dead node, because a dead node reschedules and a slow node caps every gang it joins at its own speed. Then structure the answer as detection, attribution and remediation, and say plainly that the detection signal has to be relative to the fleet, because absolute thresholds miss exactly this class of fault.

A strong answer

What actually degrades. Xid errors are the driver's fault codes and they are not all equal: some are application bugs (a bad memory access in a kernel), some are hardware (double-bit ECC, GPU falling off the bus). ECC memory retires pages over time, and a device retiring pages at a rising rate is on its way out. Thermal or power throttling shows up as clocks sitting below their target under load, often from a failing fan or a hot rack position rather than the GPU itself. On multi-GPU nodes the interconnect degrades independently: one NVLink lane down turns an all-reduce into a bottleneck while every device reports healthy. PCIe replay counts rising means the link is retrying, which shows up as slow host-to-device copies and a data loader that appears to be the problem.

Detection has to be comparative. DCGM exports the counters; the discipline is in how you read them. Run dcgmi diag -r 2 on drain and after every incident, and run a short NCCL all-reduce benchmark as part of node admission, recording achieved bandwidth. Then compare each node against the fleet: a node two standard deviations below the median on bus bandwidth is suspect even though every absolute number looks fine. This is why the answer is a distribution and not a threshold. The fleet tells you what healthy looks like this week, with this driver, at this ambient temperature.

rendering diagram…

Remediation is a policy question. Automate the cordon: a node crossing the rule set gets tainted immediately, because a human in the loop at 3am means the node keeps taking jobs. Draining is more delicate, since evicting one pod from a running gang kills the whole job. The workable pattern is to cordon on detection, let the current gang finish or hit its next checkpoint, then drain. Keep a quarantine pool where a suspect node runs one real job under observation before returning to service, because a node that passes diagnostics and fails under load is common enough to plan for. On cloud, the economics are simpler: terminate the instance and let the pool replace it. On owned hardware you are opening a support case, so the counters you captured are the evidence.

SignalWhat it usually meansAction
Xid 48Uncorrectable ECC eventQuarantine and follow architecture-specific recovery guidance
Xid 63 / 64Retirement or row-remapping recorded / failedInterpret with GPU generation, associated errors and diagnostic state
Xid 79GPU became inaccessible over PCIeCapture diagnostics; investigate power/link/driver causes before replacement
Rising retired pagesMemory wearing outSchedule replacement
Clocks below target under loadThermal or power limitCheck cooling, rack position
NCCL bandwidth below fleet p10Link or topology faultQuarantine, benchmark again

NVIDIA's Xid catalog distinguishes these codes and their recovery paths. Do not turn every retirement record or transient bus loss into an automatic RMA.

What interviewers probe next

"How do you avoid cordoning half the fleet after a driver upgrade?" Compare against the current fleet, not a historical baseline, and require a signal to persist across two checks. A regression that hits every node is a release problem, not a hardware problem, and your rule should be able to tell those apart.

"Who pays for the wasted GPU-hours?" Attribute them. Track GPU-hours lost to node faults separately from hours lost to job failures, because the two drive different fixes and the first one justifies replacement hardware.

"What about a node that only fails under a specific model?" That is usually not the node. Reproduce on a second node before opening a hardware case, or you will spend two weeks on an RMA for a memory-fragmentation bug.

Common mistakes

Treating GPU health as a liveness probe. The device answers, the job is slow, and nothing pages.

Ignoring the interconnect. A degraded link can limit a healthy GPU; include both link and compute diagnostics.

Draining a node out from under a running gang, which converts a 30 percent slowdown into a lost six-hour job.

That one was free, and so are 10 answers per topic without an account. Signing in doubles that to 20, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

Nothing here yet. Say how you would answer it.