TL;DR: Token bucket per API key for the application layer, coarse fixed-window per IP at the edge for abuse, counters in Redis via Lua so check-and-decrement is atomic. Answer the failure question before they ask it: application quotas fail open, security limits fail closed, quota rejections return 429 with Retry-After; fail-open requests continue, while limiter-unavailable refusals need a documented 503 or security-denial contract.
How to approach it
Ask who the limit protects and from what, because there are two different problems hiding here: abuse (scrapers, credential stuffing) wants cheap edge rejection, while fairness between paying customers wants precise per-key accounting. Design both layers and say why one algorithm cannot serve both.
A strong answer
Algorithm. Token bucket is the industry default for good reason: it allows bursts (bucket depth) on top of a sustained rate (refill), which matches how real clients behave, and it needs two numbers rather than a log of every request.
| Algorithm | Memory | Burst behaviour | Verdict |
|---|---|---|---|
| Fixed window | O(1) | Up to 2x at window boundary | Fine at the edge, wrong for billing |
| Sliding window log | O(requests) | Exact | Precise but too expensive per request |
| Sliding window counter | O(1) | Bounded, approximate | Good middle ground |
| Token bucket | O(1) | Explicit, configurable | Default choice |
Distribution. Counters live in Redis, and the increment-and-check must be atomic, which means a small Lua script doing read-modify-write in one step; two separate commands race under concurrency and quietly oversell. Keys shard naturally by API key, so no single hot spot unless one customer genuinely out-traffics a shard. For very high scale, run local in-memory buckets that admit optimistically and reconcile against the shared store periodically: you get soft global accuracy at a fraction of the round trips, accepting brief oversell during sync intervals. Multi-region deployments make quotas eventually consistent by default; document that a customer can briefly exceed quota across regions rather than paying cross-region latency on every call.
Placement. Two layers, deliberately:
- Edge (WAF or L7 load balancer): coarse per-IP and per-ASN windows to absorb volumetric abuse before requests cost application resources. Imprecise on purpose; NAT makes IP identity fuzzy.
- Application: precise per-key token buckets with tier-based parameters, enforced after authentication so keys cannot be spoofed. Expensive endpoints get cost-weighted buckets (one bulk export costs fifty normal calls) rather than pretending all requests are equal.
The client contract. Every quota-exceeded rejection returns 429 with Retry-After, and success responses carry X-RateLimit-Limit, -Remaining and -Reset. This is not decoration: SDKs across your ecosystem implement exponential backoff against these headers, and their absence turns every burst into support tickets.
Failure semantics, where seniority shows. The shared limiter being down must not take the API down. My example policy: application fairness uses a bounded local fallback where availability warrants it, with concurrency/load-shed ceilings and monitoring, because losing revenue fairness for minutes beats rejecting all traffic. Abuse protection at the edge fails closed, because its absence invites scraping floods. Say the reversal condition: if abuse directly causes data loss or fraud exposure, flip the app layer toward fail-closed too.
What interviewers probe next
"Why not enforce everything at the edge?" An edge gateway can validate credentials and enforce cost-weighted tiers when it has the required trusted metadata. Place each limit where identity and cost are known, with origin enforcement or authenticated forwarding where needed.
"One customer's traffic hammers the same Redis key millions of times a second." That key becomes a hot shard problem: split the bucket across N sub-keys each with rate divided by N, chosen by hash of something per-request, trading exactness for throughput.
"How do you pick the numbers?" Sustained rate from the tier contract, burst from realistic client fan-out (a dashboard firing twenty parallel calls), validated against observed p99.9 demand, then published. Numbers nobody can discover are numbers everyone violates accidentally.
Common mistakes
One algorithm everywhere because one was asked about. Edge abuse control and billing-grade fairness have different precision budgets.
Checking and decrementing in two Redis round trips. The race is real and interviewers who have operated this will push on it.
Forgetting the limiter's own failure mode until asked. The question hands you that sub-question in the title; skipping it reads as never having run one.