Incident postmortems

At a glance

Section	What it should contain	The common failure
Summary	What broke, for whom, for how long, in a paragraph	Written for insiders, unreadable in six months
Timeline	Timestamped events from systems, not recollection	Starts when someone noticed, hiding detection delay
User impact	Who was affected and what they could not do	Expressed in error rates rather than in lost capability
Contributing factors	The several things that had to be true at once	A single cause, which is almost never the whole story
What went well	Detection, tooling or decisions that helped	Omitted, so the same postmortem reads as failure every time
Actions	Specific, owned, dated, in the real backlog	"Improve monitoring", assigned to a team, undated

How it works

A postmortem exists to convert one expensive experience into a change that stops it recurring. Everything in the format serves that, and any section that does not is ceremony.

The timeline is the foundation because it is the part that resists narrative. Reconstructed from system timestamps rather than memory, it typically reveals a gap nobody discussed during the incident, most often between when the failure began and when anything noticed. That gap is frequently the largest single component of impact and the least discussed, because the response only starts once someone is aware.

The analysis is where postmortems most often stop too early. A single root cause is a comforting output and rarely a true one. Real incidents need several things to be simultaneously true: a change that introduced a defect, a test that did not cover it, an alert that did not fire, a dependency that failed in an unexpected direction. Stopping at whichever of these is most visible produces an action that removes one link from a chain that has several.

Blamelessness is the widely understood principle and the easier one to adopt. It means asking why an action seemed correct to a competent person given what they could see, rather than establishing who was responsible. The practical test is whether the postmortem's actions change the system or change the human. "Add a confirmation step before this destructive command" is systemic. "Remind the team to be careful" is not.

The harder discipline is follow-through, and it is where the practice usually fails. Postmortem actions compete with roadmap work that has a stakeholder asking about it, and they lose by default. An organisation that writes excellent postmortems and completes a third of their actions is producing documentation.

How it is measured

The metric that matters is action completion rate, tracked over a quarter rather than an incident. Anything below about two thirds means the process is generating work nobody intends to do, and it is more honest to write fewer actions and finish them.

Measure repeat incidents. The same failure mode appearing twice is direct evidence that the first postmortem did not work, and repeats are the strongest available signal about whether the practice is functioning. Categorising incidents by failure mode rather than by affected service is what makes this visible.

Check timeline quality by asking one question of each document: does it start before anybody noticed? A timeline beginning at the page conceals detection time, which is frequently where the improvement is available. How root cause analysis works and how it is measured covers the diagnosis half in more depth.

Track time from incident to published postmortem. Past two weeks the detail is memory rather than record, and the document's usefulness drops sharply. A team consistently taking a month is signalling that the practice has low priority, whatever it says otherwise.

Read old postmortems occasionally. If the same contributing factor appears across several, that is a systemic finding no individual document contained, and it is usually more valuable than any single action any of them produced.

Common misconceptions

That blameless means anonymous. Naming who did what is fine and often necessary for an accurate timeline. What is not fine is treating the person as the explanation.

That every incident needs one. Postmortems have a cost, and writing them for routine, well-understood failures devalues the practice. Write them for surprises and for repeats.

That the goal is a root cause. The goal is a change. Some incidents have a clear cause and no useful action, and some have a murky cause and an obvious improvement. The second is more valuable.

That a postmortem is a report to management. Its audience is the engineers who will meet the next similar failure. Written for an executive summary it becomes reassuring and useless.

That more actions are better. Ten actions produce two completions. Two actions produce two completions. What a post-incident review should produce covers choosing which ones are worth committing to.