
Before the meeting: write the timeline
One person, usually whoever led the response, writes down what happened in order, with times: the first alert, who noticed, what was tried, what worked, when customers stopped being affected. Share it a day ahead. A meeting that builds the timeline from memory spends its hour arguing about the order of events.
A 45-minute agenda
| Minutes | What happens |
|---|---|
| 5 | Read the timeline together and correct it |
| 15 | Why did it happen? Follow each cause back at least two steps |
| 10 | Why did it take as long as it did to notice and to fix? |
| 10 | Choose the changes, each with one owner and a date |
| 5 | What went well, so it is kept |
Questions that find causes, not culprits
- What made this the reasonable thing to do at the time?
- What did the system allow that it should not have?
- Why did no test, review or alert catch it?
- What did we not know, and where could we have learned it?
- Has something like this happened before?
"Who deployed it?" ends the conversation. "Why could a deploy go out without the migration?" starts the useful one. People describe what really happened only when they are sure it will not be used against them, and without the real story the fixes are guesses.
A template you can copy
# Postmortem: double charges on retried payments
Date of incident: 18 November
Impact: 41 customers charged twice over 3 hours. All refunded.
Severity: high
## Timeline
14:05 Retry change deployed to the payment worker
14:40 First support ticket about a double charge
16:10 Alert on charges per order fires
16:25 Retry turned off with a feature flag
17:05 Refunds started
## Causes
- A retry ran after a timeout even when the first charge had succeeded.
- Charges carried no idempotency key, so the provider saw two requests.
- No alert on more than one charge per order until 16:10.
## What went well
- The feature flag stopped the damage in 15 minutes.
## Changes
- Send one idempotency key per order with every charge. Dana, 25 November.
- Alert when an order has two successful charges. Sam, 22 November.
- Add a retry test that times out after the charge succeeds. Dana, 25 November.Changes that actually get done
- Three changes that happen beat ten that do not. Choose the ones that remove a whole class of failure.
- Each change is a ticket in the normal backlog, not a list in a document nobody opens again.
- Look at the list again at the next retrospective. A postmortem whose actions are never checked teaches the team that postmortems do not matter.
What blameless does not mean
It does not mean nobody is responsible, and it does not mean pretending a mistake was not a mistake. It means the question is what the system allowed, because the next person in the same place would make the same mistake. Fixing the system protects everyone. Blaming the person protects nobody.