DevOpsInterviewPrep logo
DevOps System Design & Architecture / 01
hard★ EssentialNewGoogleAmazonSnowflake

Design a disaster recovery plan for a 5PB data platform with a four-hour recovery objective.

The classic napkin-math question. The number in the prompt is the question, and candidates who draw the architecture before doing the arithmetic design something the physics forbids.

Updated Sep 2026 · Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Do the arithmetic first. Five petabytes over a 10 Gbps link is roughly 46 days, so a full transfer cannot meet four hours at that bandwidth and this is a continuous replication problem, not a backup problem. The senior move after that is tiering the recovery objective by data class, because almost never does all 5PB need four hours.

How to approach it

Say the constraint out loud before you draw anything. Then let the arithmetic eliminate options, and spend the round on what survives. Ask which data classes exist and what the recovery point objective is, because RTO alone underspecifies the problem.

A strong answer

The arithmetic. 5PB is 5,000,000 GB. A 10 Gbps link moves about 1.25 GB/s at line rate and less in practice, so 5,000,000 divided by 1.25 is 4,000,000 seconds, about 46 days. At 100 Gbps it is still four to five days. Moving all 5PB in four hours requires roughly 2.78 Tbps payload throughput, before overhead and read/write limits. With the stated 10–100 Gbps paths, pre-position the data. That conclusion should arrive in the first two minutes.

What that forces. The data must already be at the recovery site, so the design is asynchronous continuous replication into a warm standby, and recovery is a promotion rather than a restore. Concretely: object storage with cross-region replication for the bulk, log shipping or change data capture for anything transactional, infrastructure defined as code so the compute layer can be stood up from a repository, and DNS or global load balancer failover to move traffic.

The recovery point objective is now the real design variable. The RPO is the acceptable loss objective; the recoverable durable replication position determines actual potential loss. Monitor acknowledged source writes against recoverable destination progress and distinguish received, flushed and replayed data. A replay-lag graph alone may overstate durable loss, while a stalled pipeline or missing objects can understate it. An RPO of five minutes and an RPO of zero are different systems: zero requires synchronous replication, which puts the inter-region round trip inside every write and is usually unacceptable for a data platform.

Moving 5PB during recovery 10 Gbps 46 days 100 Gbps 4.5 days Objective 4 hours So the data is already there before the incident. Recovery is a promotion, not a restore.

Then the cost conversation, which is the part that makes this a senior answer. A warm standby holding 5PB roughly doubles the storage bill before any compute. So the next question is whether all 5PB genuinely needs a four-hour objective. Validate that requirement with the owners rather than assuming it. Tier it: tier 1 is the operational data the business cannot run without, perhaps 5 percent of the volume, replicated continuously with four-hour recovery. Tier 2 is analytical data with a 24-hour objective, replicated to cheaper storage. Tier 3 is archival, restored from cold storage over days, and that is fine because nobody is blocked on it. Turning an impossible blanket requirement into a negotiated per-class one is the answer, and it is also what actually happens in real organisations.

Three things to say before the interviewer asks. Monitor recoverable replication progress against the RPO target and test consistency across stores. A promotion procedure you have never rehearsed is not a recovery plan, so it gets tested on a schedule with a documented result. And replication faithfully replicates corruption, so continuous replication does not remove the need for point-in-time backups: a bad write or a ransomware event needs a restore point, not a standby.

What interviewers probe next

"Cut the budget in half. What changes?" Tier more aggressively and accept a longer objective on tier 2, or move the standby storage to a colder class and accept a rehydration delay in the recovery path. Say which one you would choose and what you would tell the business.

"What if the primary region is degraded rather than gone?" That is the harder case, because a partial failure can cause split brain. Use a failover policy with quorum or authority checks, reliable fencing and protection against flapping. Depending on the recovery objective, the decision can be automated or require an operator; human approval alone does not prevent two active writers.

"How do you test this without a real disaster?" Scheduled game days that promote the standby and serve real traffic from it for a bounded window. An untested failover has an unknown success probability, which is the same as no plan.

Common mistakes

Drawing the architecture first. The interviewer added the number specifically to see whether you notice, and candidates who skip it end up defending an impossible restore.

Confusing RPO and RTO, or quoting an RTO with no RPO at all.

Proposing synchronous replication across regions for a 5PB platform without acknowledging what it does to write latency.

Treating replication as a backup. It is not, and the ransomware case makes that obvious in one sentence.

That one was free, and so are 10 answers per topic without an account. Signing in doubles that to 20, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

Nothing here yet. Say how you would answer it.