Lesson 15 · Architecture & production

Load balancing & service discovery

Why the thing that balances REST silently breaks gRPC.

Your win: explain why gRPC load balancing is different from HTTP/1.1, name the two approaches, and connect it to streaming — a favourite "gotcha" interview question.

The problem: one connection carries everything

gRPC runs on a long-lived HTTP/2 connection that multiplexes many requests (Lesson 1). A naive connection-level (L4) load balancer picks a backend once, at connect time — so every request on that connection pins to that one backend. No per-request spreading.¹

HTTP/1.1 (balances naturally) gRPC / HTTP/2 (pins) ──────────────────────────── ───────────────────── req1 → new conn → LB → backend A ┌─ one conn ─┐→ LB → backend A req2 → new conn → LB → backend B │ req1 req2 │ (A gets ALL of them) req3 → new conn → LB → backend C │ req3 req4… │ (a conn per request → easy spread) └────────────┘

HTTP/1.1 opens a connection per request, so a plain LB balances by accident. gRPC's efficiency (connection reuse) is exactly what defeats that LB. You need something smarter.

Two approaches

Proxy LB (L7)

A gRPC-aware proxy (Envoy, a service mesh, a k8s L7 ingress) terminates HTTP/2, inspects each request, and balances per-RPC across backends. The client talks to the proxy only.

Client-side LB

The client discovers all backends (via service discovery — DNS, a registry, a k8s headless service) and round-robins across them per RPC itself. No extra hop.

Streaming pins harder A long-lived stream (Part 2) lives on one backend for its entire life — so even good per-RPC balancing can't rebalance an open stream, and a rolling deploy must drain it. One more reason to prefer unary unless you truly need streaming (Lesson 7).

Anchor Our services dial each other by name through a config-driven service map (rsc.GRPCDial("notificationmgmt"), bootstrap/resource.go:351) with insecure in-cluster creds. The balancing is provided by the platform (Kubernetes service / mesh) rather than explicit client-side LB policy in the code — a common and pragmatic setup. "gRPC on Kubernetes needs an L7-aware LB or headless-service client-side balancing" is the interview-ready summary.

gRPC — "gRPC Load Balancing" + "gRPC on Kubernetes without tears"

The blog lays out proxy vs client-side LB precisely; the Kubernetes post shows the real-world fix for the connection-pinning problem.

→ grpc.io/blog/grpc-load-balancing
→ kubernetes.io — gRPC LB without tears

Check yourself (from memory)

Q1. A naive connection-level LB balances gRPC poorly because…

The connection is picked once and reused, so all multiplexed requests pin to one backend. HTTP/1.1's per-request connections hide this.

Q2. The two gRPC load-balancing approaches are…

An L7 gRPC-aware proxy balances per-RPC, or the client discovers backends and balances itself.

Q3. A long-lived stream complicates LB because it…

The stream can't be rebalanced once open, and deploys must drain it — another argument for unary by default.

Why is gRPC load balancing different from HTTP/1.1, and what are the two fixes?

recall, then click to reveal

gRPC runs on a long-lived HTTP/2 connection that MULTIPLEXES many requests. A naive connection-level (L4) LB picks a backend once at connect time, so ALL requests pin to that backend — no per-request spread (HTTP/1.1 balances naturally because it opens a connection per request). Two fixes: (1) PROXY LB — an L7/gRPC-aware proxy (Envoy/mesh) balances per-RPC; (2) CLIENT-SIDE LB — the client discovers backends (DNS/registry/k8s headless) and round-robins per RPC. Streaming pins even harder. In our repo, services dial by name and the k8s platform provides the balancing.

Want to see how DNS-based client-side round-robin is configured in grpc-go (grpc.WithDefaultServiceConfig)? Ask me.

1. gRPC — Load Balancing.