Files
troupe-standup/wizard-work/quill/incident-runbook.md
T

1.0 KiB

Incident Response Runbook Template

Purpose

Standardize response steps for severity-1 through severity-3 incidents.

Severity Levels

  • SEV-1: Complete outage or data loss. Page on-call lead. Target resolution: 1 hour.
  • SEV-2: Major degradation. Page on-call. Target resolution: 4 hours.
  • SEV-3: Minor impact. Ticket and triage within business hours. Target resolution: 24 hours.

Initial Actions

  1. Acknowledge alert and assign incident commander.
  2. Open incident channel (#incident-YYYY-MM-DD).
  3. Declare severity and start timeline doc.

Communication Plan

  • Internal: 15-min updates in incident channel.
  • External: Status page update within 30 min for SEV-1/2.
  • Leadership: Email summary every hour for SEV-1.

Mitigation Steps

  • Roll back recent deploy if cause unknown.
  • Enable circuit breakers / rate limits.
  • Switch to read-replica or backup region.

Post-Incident Review

  • Draft RCA within 24 hours.
  • Schedule review meeting within 48 hours.
  • Track remediation tickets with owners and due dates.