Why the thing that balances REST silently breaks gRPC.
Your win: explain why gRPC load balancing is different from HTTP/1.1,
name the two approaches, and connect it to streaming — a favourite "gotcha" interview
question.
In plain English
Plain English: gRPC load balancing is trickier than naive HTTP/1.1 balancing because one long-lived HTTP/2 connection can keep sending many calls to the same place.
The problem: one connection carries everything
gRPC runs on a long-lived HTTP/2 connection that multiplexes many
requests (Lesson 1). A naive connection-level (L4) load balancer picks a backend
once, at connect time — so every request on that connection pins to
that one backend. No per-request spreading.1
HTTP/1.1 (balances naturally) gRPC / HTTP/2 (pins)
──────────────────────────── ─────────────────────
req1 → new conn → LB → backend A ┌─ one conn ─┐→ LB → backend A
req2 → new conn → LB → backend B │ req1 req2 │ (A gets ALL of them)
req3 → new conn → LB → backend C │ req3 req4… │
(a conn per request → easy spread) └────────────┘
HTTP/1.1 opens a connection per request, so a plain LB balances by accident. gRPC's
efficiency (connection reuse) is exactly what defeats that LB. You need something
smarter.
Two approaches
Proxy LB (L7)
A gRPC-aware proxy (Envoy, a service mesh, a k8s L7 ingress) terminates HTTP/2,
inspects each request, and balances per-RPC across backends. The client
talks to the proxy only.
Client-side LB
The client discovers all backends (via service discovery — DNS,
a registry, a k8s headless service) and round-robins across them per RPC itself. No
extra hop.
Streaming pins harder
A long-lived stream (Part 2) lives on one backend for its entire life — so
even good per-RPC balancing can't rebalance an open stream, and a rolling deploy must
drain it. One more reason to prefer unary unless you truly need streaming (Lesson 7).
Anchor
Our services dial each other by name through a config-driven service map
(rsc.GRPCDial("notificationmgmt"),
bootstrap/resource.go:351) with insecure in-cluster
creds. The balancing is provided by the platform (Kubernetes service / mesh)
rather than explicit client-side LB policy in the code — a common and pragmatic setup.
"gRPC on Kubernetes needs an L7-aware LB or headless-service client-side balancing" is
the interview-ready summary.
Read this next
gRPC — "gRPC Load Balancing" + "gRPC on Kubernetes without tears"
The blog lays out proxy vs client-side LB precisely; the Kubernetes post shows the
real-world fix for the connection-pinning problem.
Common mistake
Common mistake: assuming a normal L4/L7 balancer automatically spreads every RPC evenly once the client has a long-lived connection.
Check yourself (from memory)
Q1. A naive connection-level LB balances gRPC poorly because…
The connection is picked once and reused, so all
multiplexed requests pin to one backend. HTTP/1.1's per-request connections hide this.
Q2. The two gRPC load-balancing approaches are…
An L7 gRPC-aware proxy balances per-RPC, or the client
discovers backends and balances itself.
Q3. A long-lived stream complicates LB because it…
The stream can't be rebalanced once open, and deploys
must drain it — another argument for unary by default.
Why is gRPC load balancing different from HTTP/1.1, and what are the two fixes?
recall, then click to reveal
gRPC runs on a long-lived HTTP/2 connection that MULTIPLEXES many
requests. A naive connection-level (L4) LB picks a backend once at connect time, so ALL
requests pin to that backend — no per-request spread (HTTP/1.1 balances naturally
because it opens a connection per request). Two fixes: (1) PROXY LB — an L7/gRPC-aware
proxy (Envoy/mesh) balances per-RPC; (2) CLIENT-SIDE LB — the client discovers backends
(DNS/registry/k8s headless) and round-robins per RPC. Streaming pins even harder. In
our repo, services dial by name and the k8s platform provides the balancing.
Want to see how DNS-based client-side round-robin is configured in grpc-go
(grpc.WithDefaultServiceConfig)? Ask me.