Incident Response

An incident is production impact that can’t wait for the normal flow. The goal is fast mitigation and clear client comms — blame comes never, learning comes after.

The flow

flowchart LR
  D["Declare<br/>one channel"] --> T["Triage<br/>set severity"]
  T --> C["Client comms<br/>single voice"]
  C --> M["Mitigate<br/>stop the bleeding"]
  M --> R["Resolve<br/>verify fixed"]
  R --> P["Postmortem<br/>Sev1/Sev2"]

Severity levels

Sev	Meaning	Example
Sev1	Production down or data at risk.	App offline, checkout failing for all users, data loss.
Sev2	Major feature broken, no full workaround.	Login works but payments fail; core module down.
Sev3	Minor / degraded, workaround exists.	One report errors; slow but usable screen.

Who declares, where

• Anyone can declare. L1 (Nidal) or the team lead usually does. Better to over-declare and stand down than to sit on a Sev1.
• One incident channel. All coordination happens in a single named channel per incident — no parallel threads, no developer DMs.
• One client voice. The primary contact (usually L1 or the PM) speaks to the client. Engineers fix; they don’t run parallel client conversations. See Support Model.

Client comms path

sequenceDiagram
  participant Eng as Incident team
  participant Lead as Comms owner
  participant Client
  Eng->>Lead: Status + ETA
  Lead->>Client: Acknowledge + impact
  Lead->>Client: Update on mitigation
  Lead->>Client: Resolved + next steps

Sev1 and Sev2 get a blameless postmortem — see Postmortem Template. Sev3 is closed on the board without one.