Lesson 15 · Senior Kafka practice

Retries, retry topics, and DLQ design

How to reason about transient failure, poison messages, retry escalation, and what a sane DLQ design would look like in this repo.

Your win: explain when inline retries are enough, when retry topics help, and how to describe a DLQ design that preserves liveness without silently losing the original event.

In plain English Plain English: retries are only good when the failure might become safe to retry later; otherwise they can turn one bad message into a stuck partition or a quiet drop.

Why this page matters

The course already taught the repo’s retryable handler bool, strict vs loose commit behavior, and the fact that there is no true DLQ today. The senior step is not learning one more definition. It is learning how to turn that knowledge into design judgment.

When should a message retry in place? When should it move to a retry topic with delay semantics? When should it stop blocking the main lane and be parked for later inspection? Those are system-design questions wearing Kafka clothes.

The retry rule Retry only when the failure is plausibly transient; park or drop deliberately when retrying would only repeat the same bad outcome and block useful work.

What a DLQ is really buying you

A DLQ is not just a bin for bad messages. Its real job is to preserve evidence and restore liveness at the same time. That means the bad message is parked with enough context to investigate, and the original partition is allowed to keep moving.

That second sentence matters. If the design does not explain how healthy work behind the bad message continues to flow, then the design has not fully solved the problem.

Backend use case This repo currently has no true DLQ, which makes it a strong interview topic: you can explain the current trade-offs honestly and then describe the clean next-step design.
Common mistake Saying “just add a DLQ” without explaining who reads it, what metadata it stores, and how the main partition avoids staying blocked.
Read this next

Error-handling patterns

Use the Confluent error-handling material as the conceptual backdrop, then compare it to the repo’s current stricter and looser behaviors.

→ Confluent — Error Handling Patterns
→ Confluent — Kafka DLQ

Check yourself (from memory)

Q1. What is the most senior purpose of a Kafka DLQ?

A DLQ is about liveness plus evidence, not magic reliability.
Why isn’t “just retry it more” always a good Kafka failure strategy?
recall, then click to reveal
Because permanent or semantic failures do not become safe through repetition; extra retries can only waste time, stall partitions, or delay healthy work behind the bad message.
Want me to sketch a repo-specific DLQ design for spike or notification? Ask me.

Sources. Confluent DLQ/error-handling material; repo failure-mode lesson.