# Service Restart Runbook (Draft v0.1) ## Purpose Provide a repeatable, low-risk procedure for restarting the API service without dropping in-flight requests. ## Scope Applies to api-svc (primary and canary). Excludes database migrations. ## Prerequisites - Access to the ops bastion - PagerDuty duty-handoff confirmed - Deployment queue is clear (check via `kubectl get events -n deploy`) ## Steps 1. Verify current health: `kubectl get pods -n api -l app=api-svc` 2. Confirm no in-flight requests: `kubectl exec -n api deploy/api-svc -- curl -s localhost:8080/health | jq .in_flight` 3. Drain one pod: `kubectl drain --ignore-daemonsets --delete-emptydir-data` 4. Wait for pod reschedule: `kubectl rollout status deployment/api-svc -n api --timeout=120s` 5. Repeat for remaining pods, one at a time 6. Run smoke test: `curl -s https://api.internal/healthz | jq .status` 7. Page on-call if smoke test fails ## Rollback If health check fails after 2 pods, halt and execute: `kubectl rollout undo deployment/api-svc -n api` ## Evidence Required - Screenshot of `kubectl get pods` post-restart - Smoke test output showing `"status":"ok"` --- *Draft prepared by Quill — verify commands against staging before production use.*