DevOpsInterviewPrep logo
Incident Response & Production Debugging / 09
hardNewGoogleCloudflareMeta

TCP retransmissions are spiking but the network team says there is no packet loss. Who is right?

Both can be right, which is the whole point. Retransmissions mean a segment was not acknowledged, and the network is only one of the places that can happen.

Updated Sep 2026 · Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Both are right. A retransmission means an acknowledgement did not arrive in time, and the segment can be lost anywhere including inside the hosts. The usual causes are receive queue overflow at the endpoint, an accept backlog that is full, buffer exhaustion in a middlebox, and spurious loss detection caused by delayed or reordered packets. Look at the host counters before blaming the fabric.

How to approach it

Reframe the question in the first sentence: the network team is measuring their fabric, and the drop is probably not in it. Then name the host-side counters, which is what most candidates cannot do.

A strong answer

A retransmission happens when the sender does not receive an ACK within its retransmission timeout, or when duplicate ACKs indicate a gap. The segment may have been dropped in the fabric, but there are at least three places inside the endpoints where it can also be lost, and those are invisible to the network team's switch counters.

rendering diagram…

Receive queue overflow. If the application is not reading fast enough, the socket receive buffer fills, the advertised window shrinks toward zero, and the sender should pause ordinary data transmission and use window probes. That is flow control, not by itself evidence of retransmission-causing loss. This is an application throughput problem presenting as a network problem. ss -tin shows the receive queue and window; a persistently non-zero Recv-Q on the server side is the tell.

Accept backlog full. New connections arriving faster than the application accepts them fill the completed accept queue. Linux may ignore the final ACK and retransmit SYN-ACKs; pressure can also affect new SYN handling. The incomplete SYN queue is separate, and SYN cookies can change behavior. This looks exactly like SYN loss in the network. Check it directly:

netstat -s | grep -iE 'listen|overflow|pruned|collapse'
# "times the listen queue of a socket overflowed"
# "SYNs to LISTEN sockets dropped"

Those counters are the single most useful thing in this whole diagnosis and almost nobody looks at them.

Buffer exhaustion in a middlebox. A load balancer, firewall or NAT device under pressure drops segments without registering as fabric packet loss, because it is a device rather than a link. Conntrack table exhaustion on a busy NAT gateway is the specific version that bites in cloud environments, and it drops silently once the table is full.

Then the case where nothing is actually lost: sufficiently delayed packets or ACKs can trigger a spurious retransmission timeout, and reordering can trigger loss detection. This is distinct from BBR's bandwidth and RTT model. A microburst that fills a switch buffer without overflowing it adds queueing delay, the RTO fires early, and you see retransmissions with genuinely zero loss counted anywhere. Tuning tcp_rto_min is rarely the answer; reducing the burstiness or the buffer depth usually is.

The measurement that settles it: capture on both hosts simultaneously. If the sender transmits and the receiver's capture shows the segment arriving, the loss is not in the network and the ACK path or the receiver is the problem. If the receiver never sees it, walk the path.

What interviewers probe next

"How do you distinguish a retransmission from a duplicate ACK problem?" Duplicate ACKs can trigger fast retransmit because of loss or reordering; SACK and modern loss detection provide additional evidence. RTO-based retransmission with a backoff means the flow stalled, which is more serious.

"What is bufferbloat?" Oversized buffers that absorb bursts and add latency rather than dropping early. It converts a loss problem into a latency problem and makes congestion control behave badly.

"What would you check on a cloud instance specifically?" Instance-level network allowance and conntrack limits. Both throttle before any switch counter moves.

Common mistakes

Escalating to the network team and stopping. The counters that resolve this live on your hosts.

Naming TCP mechanisms without checking their counters with netstat -s or ss -tin.

Assuming retransmission always means loss. Latency-triggered timeouts are real and increasingly common.

References

Linux TCP semantics.

That one was free, and so are 10 answers per topic without an account. Signing in doubles that to 20, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

Nothing here yet. Say how you would answer it.