quill: add incident response runbook draft

wizard-work/quill/runbook-incident-response.md

@@ -0,0 +1,49 @@
+# Incident Response Runbook
+
+## Purpose
+This runbook defines the minimum steps for a first responder during a service incident. It is not exhaustive; it is a checklist to prevent panic and ensure consistency.
+
+## Severity Classification
+
+| Severity | Definition | Response SLA | Notification |
+|----------|------------|--------------|--------------|
+| P1 | Total outage or data loss | 15 min | Page on-call + manager |
+| P2 | Major feature broken for all users | 30 min | Page on-call |
+| P3 | Minor feature broken or major feature degraded for some users | 2 hrs | Slack #incidents |
+| P4 | Cosmetic or low-impact | Next business day | Ticket |
+
+## First 5 Minutes
+
+1. **Acknowledge the alert.** Do not dismiss until a human is actively investigating.
+2. **Open a dedicated incident channel.** Name it `#incident-<YYYYMMDD>-<short-name>`.
+3. **Assign roles.** Incident Commander (IC), Communications Lead (CL), and Technical Lead (TL).
+4. **Assess impact.** Confirm affected services, user count, and error symptoms.
+5. **Declare severity** using the table above and communicate it to the channel.
+
+## Investigation Loop (repeat until mitigated)
+
+1. **Gather data.** Check dashboards, recent deploys, config changes, dependency health.
+2. **Form a hypothesis.** State it as: "If X, then Y should happen."
+3. **Test the hypothesis.** Make a single, reversible change or run a targeted query.
+4. **Update status.** Post a one-line summary every 10 minutes.
+
+## Communication Cadence
+
+- Internal: update the incident channel every 10 minutes, even if no change.
+- External: draft a status page update within 30 minutes of P1/P2 declaration.
+
+## Mitigation vs. Resolution
+
+- Mitigation: service health restored, monitoring stable for 15 minutes.
+- Resolution: root cause fixed, not just masked. Timeline documented.
+
+## Post-Incident
+
+1. Schedule a blameless postmortem within 48 hours for P1/P2, 5 days for P3.
+2. Create follow-up tickets with owners and due dates.
+3. Update this runbook with any missing step discovered during the incident.
+
+## Escalation
+
+- If you cannot identify the affected service within 10 minutes, escalate to senior on-call.
+- If you are unsure of severity, default to one level higher.