TL;DR: A layer 4 balancer forwards connections without reading them, so it is fast, protocol-agnostic and blind. A layer 7 balancer terminates the connection and reads the request, so it can route on path, header or cookie, retry operations whose semantics permit it, and do per-request balancing, at the cost of latency and of holding your certificates.
How to approach it
Frame it as what the balancer can see, because every difference follows from that. Then give a case where choosing the wrong layer causes a specific, visible failure.
A strong answer
At layer 4 the balancer makes its decision from the TCP or UDP header: source and destination address and port. It picks a backend when the connection opens and every packet on that connection goes to the same place. It never parses the payload, so it works for any protocol, adds very little latency, and scales extremely well. It also cannot do anything clever, because it has no idea what is inside.
At layer 7 the balancer terminates the connection, reads the HTTP request, and opens its own connection to a backend. Now it can route /api to one service and /static to another, balance per request rather than per connection, retry an idempotent request on a different backend when the first fails, inject or read headers, and enforce per-route rate limits. It also has to decrypt, which means it holds the certificate.
The failure that makes this concrete is gRPC and HTTP/2. Both multiplex many requests over one long-lived connection. Behind a layer 4 balancer, one connection means one backend, so a client that opens a single connection sends all its traffic to a single pod no matter how many replicas exist. The dashboard shows ten healthy backends and one of them at full CPU. The fix is not more replicas; it is a layer 7 balancer that balances per request.
TLS is the other decision point. Terminating at layer 7 lets you inspect and route, and it centralises certificate management, but the traffic behind it is plaintext unless you re-encrypt. Passing TLS through at layer 4 keeps it encrypted end to end and keeps certificates on the backends, which some compliance regimes require, and costs you every layer 7 feature. Re-encryption creates a separate backend TLS connection, whose handshake cost depends on connection reuse and session resumption. A transport-layer product such as AWS NLB can also terminate TLS without providing HTTP path routing.
My default: layer 7 at the edge for HTTP services, because routing and retries are worth the milliseconds, and layer 4 for anything that is not HTTP, for very high throughput where the per-request cost matters, and where an auditor requires end-to-end encryption.
What interviewers probe next
"Which one is a Kubernetes Service?" Layer 4. kube-proxy balances connections. An Ingress or Gateway controller is the layer 7 piece, which is why you need one to route by path.
"How does health checking differ?" Health checking is independent of forwarding: an L4 balancer may use HTTP/HTTPS health probes, while TCP checks only establish that a port responds. Choose a check that reflects the backend's ability to serve.
"What is direct server return, and why would you want it?" Responses bypass the balancer on the way back, which removes it as a bandwidth bottleneck. Layer 4 only, and it complicates the network path considerably.
Common mistakes
Saying layer 7 is "smarter" without naming a cost. Latency, certificate custody and a stateful component in the request path are all real.
Missing the gRPC and HTTP/2 case, which is the one that actually bites in production and the one interviewers most often use as the follow-up.
Assuming TLS must terminate at the balancer. Passthrough and re-encryption are both normal, and the choice is usually driven by compliance rather than engineering.
References