Lesson 3 · Foundations

Consumer groups & rebalancing

How Kafka scales reading out — and heals when a consumer dies.

Your win: explain how a group splits a topic's partitions, what happens when a consumer joins or crashes, and why two groups on the same topic each get their own copy of the events.

A group is how consumers cooperate

A consumer group is a set of consumers sharing one group id. The rule that defines everything else: each partition is read by exactly one member of the group.¹ Kafka hands out the partitions among the members — that's how the group divides the work.

topic (3 partitions) group "spike.consumer-group.email-sending" P0 ─────────────▶ consumer A (reads P0) P1 ─────────────▶ consumer B (reads P1) P2 ─────────────▶ consumer C (reads P2) Add a 4th consumer D? → it sits IDLE. Can't beat 1-per-partition.

So the number of partitions is the ceiling on useful consumers in a group. In our wrapper, Instances(n) is how a service says "run n members in this group" — notification's async consumer runs Instances(2) to read two partitions at once.²

Your group ids, decoded

Our wrapper builds a group id for every (service, topic) pair:

internal/golibs/kafka/kafka.go:255

GenNewConsumerGroupID(service, topic)
   → "{prefix}{service}.consumer-group.{topic}"
   → "prod.tokyo.spike.consumer-group.communication.email-sending"

That naming is doing real work. Because each service gets its own group id on a topic, groups are independent: they each track their own committed offset, so they each receive every event.

Fan-out in your code Both notification and conversationmgmt consume arch.student-course-class-with-academic-year — but with different group ids (one per service). So both services see every class-membership event and sync independently. Same topic, two groups, two copies. That's pub/sub fan-out.

Two rules, don't mix them up Within a group: partitions are divided (each event goes to one member) — that's scaling. Across groups: each group gets all events (a full copy) — that's fan-out.

Rebalancing: what happens when membership changes

When a consumer joins, leaves, or dies, the group rebalances — Kafka reassigns the partitions across the surviving members.¹ Consumption pauses briefly during the reshuffle, then resumes. A crashed member's partitions are picked up by the others, from the last committed offset — so nothing is silently skipped (though something may be re-processed; that's Lesson 5).

This is also why our services care so much about liveness: if a consumer goroutine dies, the /healthz endpoint returns 503 and Kubernetes restarts the pod (cmd/server/notificationmgmt/consumers.go:110), triggering a clean rebalance rather than leaving partitions unread.

The two-lane pattern, now it makes sense

spike runs the same handler on two topics — communication.email-sending and its .high-priority sibling — each with its own group. Because the groups are independent, a huge backlog on the normal lane can't delay the high-priority lane: different partitions, different group, same pod (internal/spike/.../kafka/send_email_consumer.go:39). We'll give this pattern a full lesson later.

Confluent — Consumers, Consumer Groups & Offsets

The authoritative walkthrough of group membership, the one-partition-per-member rule, and rebalancing. Pair it with the Kafka 101 "Consumers" video.

→ docs.confluent.io/kafka/design/consumer-design.html
→ Kafka 101 — Consumers

Check yourself (from memory)

Q1. Within one consumer group, each partition is read by…

One partition → one member (within the group). This is what caps useful group size at the partition count.

Q2. Two services consume one topic with different group ids. Each service…

Different group = independent offsets = a full copy of every event. That's exactly how notification and conversationmgmt both consume the class-membership topic.

Q3. A consumer in a group crashes. Kafka responds by…

The group rebalances — survivors pick up the orphaned partitions from the last committed offset. (Follower→leader promotion is about replication, a different mechanism — Lesson 7.)

A topic has 3 partitions and you run 5 consumers in one group. What happens?

recall, then click to reveal

3 consumers each take one partition; the other 2 sit idle — a group can't have more active consumers than partitions. To use all 5, the topic needs ≥5 partitions. In our repo, Instances(n) sets how many members the group runs.

Curious how rebalancing decides who gets which partition, or what a "sticky" assignor is? Ask me — great interview follow-up territory.

1. Confluent — Kafka Consumer Design (groups, one-partition-per-member, rebalancing).

2. Repo: repo-kafka-map.md — consumer options & wiring (Instances(n), GenNewConsumerGroupID).