# Apache Kafka Glossary

The canonical vocabulary for this workspace. Lessons use *these* words, not
synonyms. Definitions are tight (what the term **is**, not how to use it) and
opinionated — where the field uses several words for one idea, one is chosen and the
rest listed as `Avoid`. Definitions reuse other glossary terms on purpose; that is
what makes the hard terms easier later.

## Core data model

**Event / Record / Message**:
An immutable fact appended to a topic — a key, a value (bytes), a timestamp, and
optional headers. In our repo the value is JSON.
_Avoid_: row, item, entry

**Topic**:
A named, append-only log of events that producers write to and consumers read from.
The unit of pub/sub. Example in our repo: `communication.email-sending`.
_Avoid_: queue, channel, stream (stream means something specific in Kafka Streams)

**Partition**:
One ordered, append-only sub-log of a topic. A topic is split into partitions; a
partition is the true unit of ordering and of parallelism.
_Avoid_: shard, slice

**Offset**:
A monotonically increasing integer id of an event **within one partition**. Ordering
and "where a consumer is up to" are both expressed as offsets.
_Avoid_: index, position, sequence number

**Key**:
The part of an event that decides its partition (via a partitioner). All events with
the same key land in the same partition, so they keep their relative order. In our
repo the key is often a domain id (e.g. `EmailID`).
_Avoid_: id, partition key (just "key")

**Log**:
The append-only, ordered, durable sequence of events that a partition physically is.
The single primitive the whole system is built on.
_Avoid_: file, journal

## Clients

**Producer**:
A client that appends events to a topic. In our repo: `PublishContext(ctx, topic,
key, value)`.
_Avoid_: publisher, writer (segmentio's type is `Writer`, but the role is "producer")

**Consumer**:
A client that reads events from a topic's partitions in offset order by pulling from
the broker.
_Avoid_: subscriber, reader, listener

**Consumer group**:
A set of consumers sharing one group id that cooperatively read a topic — each
partition is read by **exactly one** member. The unit of horizontal scaling and of
offset tracking. In our repo the id is
`{prefix}{service}.consumer-group.{topic}`.
_Avoid_: consumer pool, worker group

**Rebalance**:
The reassignment of a topic's partitions across a consumer group's members when a
member joins, leaves, or dies. Consumption pauses briefly while it happens.
_Avoid_: reshuffle, redistribution

## Cluster

**Broker**:
One Kafka server. A cluster is several brokers; each hosts a subset of partitions.
Our config points at a comma-separated broker list (e.g. `kafka-0:9094,...`).
_Avoid_: node, server (say "broker")

**Leader / Follower (replica)**:
For each partition, one broker holds the **leader** replica (serves all reads and
writes) and others hold **follower** replicas that copy it for fault tolerance.
_Avoid_: master/slave, primary/secondary

**Replication factor**:
How many copies (replicas) of each partition exist across brokers. Higher = more
durable, more storage. Forced to 1 locally in our repo.
_Avoid_: copies count, redundancy level

**ISR (In-Sync Replicas)**:
The set of replicas currently caught up to the leader. Only an ISR member can become
leader without data loss; `acks=all` waits for the ISR.
_Avoid_: synced replicas, live replicas

## Reliability & semantics

**acks**:
The producer's durability setting — how many replicas must acknowledge a write before
it's considered done (`0`, `1`/`RequireOne`, or `all`). Our producer uses
`RequireOne`.
_Avoid_: acknowledgement level, confirms

**Commit (offset commit)**:
A consumer recording "I've processed up to offset N" for a partition, stored in the
internal `__consumer_offsets` topic. Determines where a restarted consumer resumes.
_Avoid_: checkpoint, ack (consumers don't "ack")

**At-least-once**:
Delivery where every event is processed one **or more** times (never lost, possibly
duplicated) — the result of committing the offset *after* processing. Our default
(strict) consumer mode.
_Avoid_: guaranteed delivery

**At-most-once**:
Delivery where every event is processed zero or one time (never duplicated, possibly
lost) — the result of committing *before* processing. Our `AlwaysCommit()` mode
behaves this way for a failed message.
_Avoid_: best-effort

**Exactly-once (EOS)**:
Delivery where every event has effect exactly one time, via Kafka transactions +
idempotent producer. We do **not** use it — we get the same outcome with app-level
**idempotency** instead.
_Avoid_: EOS (spell it out first), once-only

**Idempotency (application-level)**:
Designing a handler so processing the same event twice has the same effect as once
(e.g. dedupe by an id). How our services make at-least-once safe.
_Avoid_: dedup logic

**Retention**:
How long a topic keeps events before deletion, by time or size — independent of
whether anyone consumed them. Set per topic in our `fink` topic configs.
_Avoid_: expiry, TTL

**Log compaction**:
A retention mode that keeps only the **latest** event per key (instead of deleting by
age), turning a topic into a changelog of current state.
_Avoid_: dedup, cleanup

## Our wrapper (repo-specific)

**`KafkaManagement`**:
Our shared interface (`internal/golibs/kafka`) over `segmentio/kafka-go`. All three
services produce/consume through it. See [repo-kafka-map.md](./repo-kafka-map.md).

**Retryable (the handler bool)**:
The first return value of our `MsgHandler` — `true` means "transient, retry me",
`false` means "give up (drop or stall)". Our stand-in for retry policy since there's
no dead-letter topic.
_Avoid_: shouldRetry, recoverable

**Lane (priority lane)**:
Our pattern of a normal topic + a `.high-priority` topic consumed by separate groups
in the same pod, so urgent work isn't stuck behind a backlog. Used by spike email and
notification.
_Avoid_: priority queue