Lesson 4 · Reliability core

Producers, keys & acks

How a write becomes durable — and the knob that decides how durable.

Your win: explain what happens when a producer sends an event, what acks = 0 / 1 / all really trade off, and why our producer chose RequireOne — including the risk that choice accepts.

A produce is a write to the partition leader

When a producer sends an event, three things happen in order: the partitioner uses the key to pick a partition (Lesson 2), the producer sends the batch to the broker that holds that partition's leader replica, and the leader appends it to the log.1 Producers batch and retry automatically — they're built to be reliable and fast.

Our wrapper's writer shows both the partitioner and the retry policy in one place:

internal/golibs/kafka/kafka.go:465 
writer := &kafka.Writer{
    Balancer:     &kafka.Hash{},        // key → partition (Lesson 2)
    MaxAttempts:  math.MaxInt32,          // retry sends ~forever
    RequiredAcks: int(kafka.RequireOne), // the durability knob ↓
}

acks: how many replicas must confirm

acks (in segmentio, RequiredAcks) is the durability setting. It says how many replicas must acknowledge the write before the producer treats it as done.2

acksWaits forIf the leader dies…Feel
0nothing (fire-and-forget)silently lostfastest, least safe
1 / RequireOneleader onlylost if it hadn't replicated yetour setting — middle ground
all / RequireAllthe full in-sync replica setsafe — a follower has itslowest, strongest
The trade-off in one line acks buys durability with latency. 0 can lose data, all guarantees a replica has it before you move on, 1 sits between — safe unless the leader dies in the brief window before a follower copied the write.

Why RequireOne for us — and the risk it accepts

Our producer uses acks=1: wait for the leader, not the whole replica set. That keeps produce latency low (the API returns fast — remember Lesson 1), at the cost of a narrow data-loss window if a leader crashes before its followers catch up. For emails and notifications that's an accepted trade: throughput matters, and the consumer side is designed to tolerate the occasional lost or duplicated event (Lessons 5–6). An interviewer loves this answer because it's a choice with a reason, not a default.

Interview trap acks is a producer setting about durability of a write. It is not the same as a consumer committing an offset (Lesson 5). Mixing these up is the most common Kafka interview stumble. Producers ack; consumers commit.

The key is a first-class reliability tool

You met the key in Lesson 2 as "chooses the partition." Reframe it now as a reliability lever: by keying on EmailID, we guarantee all events for one email are ordered and land on one leader — so ordering survives even under producer retries. Choosing a good key is a design decision, not an afterthought.

Read / watch this next

Apache Kafka 101 — Producers

Short, concrete, from Confluent. Then read the Kafka docs' producer/durability notes for the precise acks definition.

→ developer.confluent.io/courses/apache-kafka/producers
→ Kafka docs — acks producer config

Check yourself (from memory)

Q1. With acks=1 (our setting), a write is confirmed once…

acks=1 waits for the leader only — fast, but a leader crash before replication loses that write. acks=all would wait for the in-sync replicas.

Q2. acks is fundamentally a setting about…

Producers ack (durability of the write); consumers commit (progress). Don't conflate them — a classic interview trap.
Why does our producer accept acks=1 instead of the safer acks=all?
recall, then click to reveal
To keep produce latency low so the API returns fast. The cost is a small data-loss window if a leader dies before followers replicate. That's acceptable because the workload (emails/notifications) values throughput and the consumer side already tolerates rare loss/duplication via at-least-once + idempotency.
Want to see how acks=all interacts with min.insync.replicas (the combo that actually guarantees durability)? Ask me — it's the natural bridge into Lesson 7.

1. Confluent Developer — Producers.

2. Confluent — Message Delivery Guarantees; Kafka docs — acks.