DevOpsInterviewPrep logo
← AI & GPU Infrastructure / 13
expertNewMetaGoogleDatabricks

How is rolling out a new model different from rolling out new code, and what does a model canary actually measure?

Code rollouts ask 'does it work'; model rollouts ask 'is it better', and the second question has no HTTP status code. Shadow traffic, offline eval gates and quality canaries replace the 500-rate dashboards you cannot use.

Updated Sep 2026 · Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

Code rollouts ask 'does it work'; model rollouts ask 'is it better', and the second question has no HTTP status code. Shadow traffic, offline eval gates and quality canaries replace the 500-rate dashboards you cannot use.

an account raises the per-topic limit · no card
UP NEXT ON YOUR JOURNEY
Next in this trackYou have eight 80GB GPUs and a 70B model. Tensor parallel across all eight, or two replicas of four? Defend the choice.Next in this trackOne customer's batch job is making your interactive chat slow. Fix it at the serving layer.Next in this trackYour inference replica takes nine minutes from pod scheduled to first token served. Where does the time go and what do you fix?
DISCUSSION · 0

Nothing here yet. Say how you would answer it.