December is when I go back through the year's incident log. Not to relitigate who broke what, but to check whether we actually got cheaper lessons out of our failures than we paid for them. That is the whole point of an incident postmortem: an outage or a failed launch is expensive no matter what you do next, so the only question worth asking is whether you at least bought a lesson with that expense.
Most teams I meet do not run postmortems at all, or they run something that looks like one but is actually a blame session with a calendar invite. Neither gets you the lesson. If nobody writes anything down, the same root cause resurfaces in six months wearing a different symptom. If the postmortem turns into finding the engineer who pushed the bad config, people start hiding near-misses instead of reporting them, and you lose visibility into the failures that have not happened yet.
I run postmortems for every production incident and every launch that missed its date badly enough to hurt, blameless by design, using five questions that fit on one page and take under an hour with the right people in the room.
Why blameless is not a soft-skills nicety
Blameless does not mean nobody is accountable. It means the document's job is to find the conditions that let the failure happen, not the person who happened to be holding the pager. Punish the person and you get an org where incidents get reported late, quietly, or not at all, because nobody wants to be the one whose name is in the timeline. That is worse than any single outage: you lose your own early warning system.
The businesses that recover fastest from repeated failures are the ones where an engineer will say "I think I caused this" in the first ten minutes, because they know that sentence gets them help, not a writeup with their name in the header. That is a culture decision, made once, that pays off every incident after.
The five-question template
Skip the elaborate templates with twelve sections nobody reads. Five questions, one page, done within 48 hours while memory is fresh.
- What happened, in plain language, with a timeline. Detection time, first response, resolution time. No jargon, no blame language ("the deploy failed" not "someone deployed carelessly").
- What was the actual customer or business impact. Downtime minutes, transactions lost, refunds issued, tickets generated. Numbers, not adjectives.
- What was the root cause, and what were the contributing factors. Root cause is rarely one thing. A bad migration is the trigger; missing staging parity and no rollback plan are the contributing factors that turned a five-minute blip into a two-hour outage.
- What went well. Always include this. It tells you what part of your incident response to keep doing, and it stops the document from reading as pure indictment.
- What are we changing, with an owner and a date. Not "we will improve monitoring." Specific: "Add alert for queue depth over 500, owned by [name], done by [date]." A postmortem with no dated action item is a diary entry, not process improvement.
A case in point
A multifinance company I worked with had a recurring pattern: batch settlement jobs failing overnight, discovered by an operations person at 8am, escalated cold. Three incidents in two months, each postmortem blaming "the script." After we forced the five-question format, question three surfaced the actual contributing factor every time: no alerting on job failure, so the only detection mechanism was a human noticing missing data the next morning. One action item, alert on job failure, fixed all three "different" incidents because they shared the same real root cause. That is the value an incident postmortem delivers that a bug ticket never will: it looks at the system around the failure, not just the failure.
Running the retro without it turning into a trial
A few mechanics that keep the room blameless in practice, not just on paper:
- The facilitator is not the person who caused the incident, and ideally not their direct manager.
- Write the timeline from logs and timestamps first, before anyone talks, so the discussion starts from facts instead of memory and defensiveness.
- Ban the phrase "should have" in the room. It is always aimed at a person. Replace with "the system did not catch."
- Publish the postmortem somewhere the whole engineering team can read it, not just leadership. The lesson only compounds if more than three people learn it.
Where postmortems fit into your year-end review
If you run one thing before January, pull every incident from this year and check two things: did each one have a dated action item, and was that action item actually completed. Teams are decent at writing the first part and bad at closing the loop on the second. An incident postmortem with an unfinished action item is a lesson you paid for and didn't cash in, and closing that gap is often cheaper than any new tooling you're considering for next year, a topic worth its own look when you're prioritizing tech initiatives for the new year.
The takeaway
An incident postmortem is not paperwork, it is how you make sure an expensive failure only happens once. Keep it blameless, keep it to five questions, and never close one without a named owner and a date. If your postmortems currently end with "we'll be more careful," you are not running postmortems, you are running eulogies. Fix that before the next outage, not after.