Lesson 15 · Senior CI/CD judgment

Delivery confidence, rollback judgment, and blast-radius thinking

How to reason about whether a deployment is safe enough to ship, reversible enough to recover, and bounded enough to keep failure from spreading too far.

Your win: explain how senior engineers judge delivery confidence, think about rollback strength honestly, and reason about blast radius rather than treating “deploy succeeded” as the end of the story.

In plain English Plain English: a strong deployment story is not just “we can ship.” It is “we know what we are shipping, how widely it lands, and how we back out if it goes badly.”

Why rollback stories matter more than happy-path stories

Every pipeline looks good on the happy path. The better question is what happens when the release is wrong, late, incomplete, or only bad in one slice of the rollout. That is where senior CI/CD judgment lives.

This is also why rollback belongs inside delivery design, not as an afterthought you only discuss during incident response. If the system cannot tell you what older artifact to trust, how to redeploy it, or how far the current rollout has already spread, then the pipeline story is weaker than it first appears.

The rollback rule A rollback story is strong when you can name the exact artifact to go back to, the exact mechanism to restore it, and the blast radius you are trying to shrink by doing so.

What that means in this repo

This repo’s delivery model gives you a concrete story to reason about: one backend image, fan-out deployment across many orgs, push-based rollout, and rollback by re-deploying an older release tag. That is a real rollback story. But the strong answer also names the cost: redeploying an old tag is operationally clean and auditable, yet still depends on knowing which old tag is safe and how widely the current rollout has already landed.

The org matrix makes this even more important. One deploy fans out across multiple organizations. That is efficient, but it also means a bad release can spread quickly if the rollout shape does not give you enough time to notice trouble. So blast radius is not a separate operations topic. It is part of deployment design.

Backend use case In this repo, delivery confidence is shaped by the env×org matrix, `cancel-in-progress: false` during deploy, rollout waits via Skaffold/Helm status checks, and rollback by manual redeploy of a known older release tag.

Common mistake Saying “we can always roll back” without naming which artifact you would redeploy, how quickly, and how much of the matrix may already have been affected.

What stronger answers sound like

A mature answer here usually sounds like: “our rollback is operationally clear because the image tag equals the release tag, deploy is push-based, and we can redeploy a known previous artifact — but the real question is whether we notice the issue early enough, and how much of the org matrix is already exposed before we act.”

That answer is stronger because it connects mechanism to operational consequence, not just to tooling.

Re-read deploy mechanics through the lens of reversibility

Use the repo map and Skaffold CI/CD material to ask what makes a rollout recoverable in practice.

→ Skaffold CI/CD docs
→ Repo CI/CD map

Check yourself (from memory)

Q1. What makes a rollback story operationally strong?

A rollback story is strong when it is concrete, reversible, and scoped — not just optimistic.

Why is blast radius part of deployment design?

recall, then click to reveal

Because rollout shape determines how widely a bad release can spread before detection and recovery. Deployment design is not only about shipping; it is also about limiting exposure when shipping goes wrong.

Want a “would you feel safe deploying this?” review drill? Ask me.

Sources. Skaffold docs; repo CI/CD map.