Lesson 13 · Senior CDC practice

Delivery semantics, replay, and idempotent sinks under pressure

How to reason about at-least-once delivery, replay, retries, and why idempotent sinks carry more of the safety story than most people admit.

Your win: explain why at-least-once delivery is useful but incomplete, and how replay, redelivery, and idempotent upsert combine into the real correctness story for this pipeline.

Why this page matters more than it first looks

At first glance, this topic can seem almost too small for a whole lesson. You might think: “At-least-once means duplicates can happen, so we use upsert. Got it.” But the strongest earlier CDC lessons never stop at the quick slogan. They slow down and ask what that slogan really means in a live system.

That is exactly what this page is for. The foundational lessons already taught the mechanics: Debezium captures changes, Kafka carries them, and the sink applies them. The senior step is seeing that the delivery guarantee and the sink design are not separate facts. They only make sense together.

In plain English Plain English: the transport can repeat work; the sink has to make repeating that work safe.

Start with the temptation to answer too quickly

In interviews, many people answer the replay question too fast. They say, “Our pipeline is at-least-once.” That is not wrong. It is just unfinished. It tells you the system may redeliver a change, but it does not yet explain why the final destination still ends up correct.

The stronger answer is more complete: the transport may repeat work, but the destination write is designed so repeating that work does not corrupt the final state. In other words, the sink is carrying much more of the safety story than people first realize.

The replay rule At-least-once is safe here because the destination write is designed to be idempotent under repeated delivery, not because the transport magically avoids repeats.

What the sink is really buying

insert.mode = upsert can look like a low-level connector detail. In the strongest reading, it is not a detail at all. It is the practical design choice that lets the same logical change be applied again without changing the final answer.

That is why the mature explanation always pairs delivery semantics with sink behavior. If you only name the transport guarantee, your answer sounds partial. If you connect it to idempotent upsert on the destination primary key, your answer starts sounding like system understanding.

Backend use case In this repo, generated JDBC sinks rely on primary-key upsert, dropped hard deletes, and schema-intersection whitelists so plain-sync paths like bob → auth.users stay safe under at-least-once delivery.

Common mistake Saying “we have at-least-once” as if that alone were the correctness story, without naming the idempotent upsert that makes replay safe.

Replay and JDBC upsert together

Revisit the sink docs and the repo map together so the semantics and the config stay attached to one real example instead of floating as separate facts.

→ JDBC sink connector
→ Repo CDC map

Check yourself (from memory)

Q1. Why is replay safe in this pipeline?

The transport can repeat work; the sink makes repeating it safe.

Why can’t “at-least-once” be the full answer by itself?

recall, then click to reveal

Because at-least-once allows repeated delivery. The pipeline stays correct only because the sink applies those repeats idempotently, usually with primary-key upsert.

Want a “would this CDC path still be safe under replay?” design drill? Ask me.

Sources. JDBC sink docs; repo CDC map.