On January 31, 2017, an engineer at GitLab ran a delete command against the wrong database server. Of the four backup and replication procedures the company relied on, only one could actually restore data, and it was never designed for disaster recovery. GitLab.com was down for about 18 hours and lost roughly 5,000 projects, 5,000 comments and 700 new user accounts, according to the company’s own postmortem.
That should have been a reputation-ending week. It is instead remembered as a model of how to behave in an outage, because GitLab kept its working notes in a publicly visible Google Doc and streamed the recovery live on YouTube, where it peaked at around 5,000 viewers. The failure was bad. The communication was so good that it changed the story.
Most product teams treat outage communication as somebody else’s job: support writes the status page, engineering runs the bridge, and the product manager waits to hear when it’s fixed. That is a mistake. Customers experience an incident as a product event. What they read during those hours shapes their trust in the product more than any feature you ship that quarter. Here are seven things worth getting right before the next one, in the order they tend to go wrong.
1. The first post is late because someone wanted to be sure
The instinct during an incident is to wait until you understand the problem. Customers do not need you to understand it; they need to know that you know. Hosted Graphite’s engineering team wrote this down as a rule: for the first notification, “prioritise speed over accuracy”, and cap fact gathering at a few minutes before you post.
Atlassian’s own Statuspage guidance makes the same point in three parts: acknowledge the issue quickly, summarize the known impact, and promise further updates. That first message can be two sentences. “We are seeing errors when customers try to export reports. We’re investigating and will post again by 14:30.” Nothing in it is a guess, and it stops your support queue from filling with people asking whether the problem is on their end.
2. Nobody says when the next update is coming
This is the single most useful habit, and the one most often skipped. A status update without a next-update time leaves every reader refreshing the page and every account manager pinging the bridge. Atlassian suggests a default of every 30 minutes, or whatever cadence suits the situation. Hosted Graphite runs tighter: no more than 20 minutes between updates while an incident is still being investigated, with the option to stretch to hourly during a long, predictable recovery, provided the previous update said so.
The rule that matters is the promise, not the number. If you said 14:30, post at 14:30, even when the update is “no change, still working on it.” A missed promised update reads to a customer as a team that has stopped paying attention.
3. The update describes the system instead of the customer
Engineers on a bridge call talk in components: replication lag, a failing node pool, elevated 5xx rates. Customers want to know what they can and cannot do right now. “Elevated error rates on the API gateway” means nothing to a finance manager trying to run month-end. “Scheduled reports are not sending; reports you run manually still work” tells them exactly how to plan their afternoon.
This translation is product work. The PM knows which workflows matter to which customer segments and which workaround is safe to recommend. If nobody on the incident is translating, the status page will either say nothing useful or say something technically true and practically misleading.
4. Internal and external messages tell different stories
Running network operations at a large telecom, I sat on more outage bridges than I could count. The worst ones were never the longest. They were the ones where the executive update, the customer notice and the frontline script were written by three different people at three different moments, and each said something slightly different about scope and timing. Our customer service reps would read one estimate on the internal page while customers were quoting a different one back to them from the public notice. Every mismatch cost us more credibility than the outage itself.
The fix we eventually settled on was dull and effective: one person owned the facts, and every audience got a version written from the same source text within minutes of each other. Atlassian lists consistency across channels as a core tip for the same reason. Your status page, in-app banner, support macros and sales talking points should all derive from one current statement. If you already fight status reports that say green while the work is red, incidents are where that habit does real damage.
5. The update blames someone else
When a third-party provider causes your outage, the temptation is to say so loudly. Atlassian’s guidance is blunt: own the problem even if another provider caused it. Your customers chose you, not your cloud host.
July 2024 showed how this plays out at scale. A faulty CrowdStrike sensor configuration update released at 04:09 UTC on July 19 was remediated at 05:27 UTC, a window of about 78 minutes. Microsoft later estimated that the update affected 8.5 million Windows devices. Airlines, hospitals and retailers whose systems went dark did not get much sympathy from their own customers for pointing at a vendor. The companies that came through best explained what they were doing to restore service, not whose fault it was. Name the cause when it helps customers act; otherwise keep the focus on recovery.
6. Someone promises a recovery time engineering never gave
Under pressure from a large account, someone on the commercial side says “it will be back within the hour.” Now the team has two incidents: the outage and the broken promise. Only commit to times that the incident lead has actually committed to, and phrase estimates as estimates. “We expect the restore to finish around 18:00 UTC; we’ll confirm at 17:00” holds up if it slips. “Fixed by 18:00” does not.
This is the same pattern as every estimate becoming a deadline, compressed into a few hours with the whole customer base watching. The PM’s job is to protect the incident team from making promises it cannot keep, and to give account owners language they can use without overcommitting.
7. The postmortem never reaches the customers who were hurt
A widely repeated idea in customer service holds that a well-handled failure can leave customers happier than if nothing had gone wrong. The research is far less generous. Magnini and colleagues found this service recovery paradox shows up only under narrow conditions, mostly when the failure is not severe and the customer has had no prior failure with the company. A 20-month field study by Maxham and Netemeyer in the Journal of Marketing found that satisfactory recoveries could produce the effect after one failure, but not after a second one.
So a great recovery does not erase an outage. What it buys you is permission for the next one not to end the relationship. A clear postmortem is a large part of that recovery. Hosted Graphite considers a postmortem by the next business day acceptable and recommends telling users in the resolution message when to expect it. GitLab published its full account ten days later, including the uncomfortable details about its broken backups and what it was changing.
The customer version should say what happened in plain language, what customers lost, what you are doing so it doesn’t recur, and whether anything is owed to them. Then send it directly to the accounts that were affected. Posting it on a blog nobody reads is not the same as telling the people you let down.
What the PM actually owns here
None of this requires the product manager to run the incident. The incident commander runs the incident. What the PM can own is everything around it: a pre-written first message for the three or four failure modes most likely to hit your product, a plain-language glossary that maps components to customer workflows, an agreed cadence, and a named person who writes the external update. Those are decided in a calm week, not at 2 a.m.
If you already treat your uptime target as a product decision, outage communication is the other half of that choice. The error budget decides how often customers will be hurt. The communication plan decides how they will feel about you afterward.