Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

[Resilience] Agent Crash Recovery: Persist Current Context Before Fatal Escapes #1411

New Issue
Closed
opened 2026-03-24 13:04:24 +00:00 by Timmy · 1 comment
Timmy commented 2026-03-24 13:04:24 +00:00
Owner
Copy Link

Context: Subprocess kills and SIGTERMs (e.g. from ops-kill) destroy the task-context.

Acceptance Criteria:

  • Trap exit signals within loop scripts.
  • Dump active TASK_ID and memory references to an active_state.json.
  • Reconstitute this state dynamically upon loop restart.
**Context:** Subprocess kills and SIGTERMs (e.g. from ops-kill) destroy the task-context. **Acceptance Criteria:** - Trap exit signals within loop scripts. - Dump active `TASK_ID` and memory references to an `active_state.json`. - Reconstitute this state dynamically upon loop restart.
Timmy commented 2026-03-24 15:12:56 +00:00
Author
Owner
Copy Link

KIMI IMPLEMENTATION INSTRUCTIONS - Agent Crash Recovery

Implementation Plan

Phase 1: Signal Trapping Infrastructure (Priority: HIGH)

Files to create/modify:

  • scripts/signal_handler.sh - Centralized signal trap utilities
  • scripts/claude-loop.sh - Add trap handlers for SIGTERM/SIGINT
  • scripts/kimi-loop.sh - Add trap handlers for SIGTERM/SIGINT
  • src/agent_state/ - New package for state persistence

Phase 2: Core Implementation

Signal Trap Pattern:

STATE_FILE="/tmp/agent_state_$(whoami)_$$.json"

cleanup_and_save() {
    echo "Agent interrupted - saving state..."
    cat > "$STATE_FILE" << EOF
{
    "task_id": "${CURRENT_TASK_ID:-null}",
    "issue_number": "${ISSUE_NUMBER:-null}", 
    "workspace_path": "${WORKSPACE_PATH:-null}",
    "branch_name": "${BRANCH_NAME:-null}",
    "progress_stage": "${PROGRESS_STAGE:-null}",
    "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
EOF
    exit 0
}

trap cleanup_and_save SIGTERM SIGINT EXIT

Phase 3: Recovery Logic

Add to loop start:

recover_agent_state() {
    for state_file in /tmp/agent_state_$(whoami)_*.json; do
        if [ -f "$state_file" ]; then
            RECOVERED_TASK_ID=$(jq -r '.task_id' "$state_file")
            echo "Recovering task: $RECOVERED_TASK_ID"
            # Continue from saved context
            rm "$state_file"
            return 0
        fi
    done
    return 1
}

Testing: Send SIGTERM to active loops, verify state persistence and recovery on restart.

This prevents work loss from process interruption, improving agent reliability significantly.

**KIMI IMPLEMENTATION INSTRUCTIONS - Agent Crash Recovery** ## Implementation Plan ### Phase 1: Signal Trapping Infrastructure (Priority: HIGH) **Files to create/modify:** - `scripts/signal_handler.sh` - Centralized signal trap utilities - `scripts/claude-loop.sh` - Add trap handlers for SIGTERM/SIGINT - `scripts/kimi-loop.sh` - Add trap handlers for SIGTERM/SIGINT - `src/agent_state/` - New package for state persistence ### Phase 2: Core Implementation **Signal Trap Pattern:** ```bash STATE_FILE="/tmp/agent_state_$(whoami)_$$.json" cleanup_and_save() { echo "Agent interrupted - saving state..." cat > "$STATE_FILE" << EOF { "task_id": "${CURRENT_TASK_ID:-null}", "issue_number": "${ISSUE_NUMBER:-null}", "workspace_path": "${WORKSPACE_PATH:-null}", "branch_name": "${BRANCH_NAME:-null}", "progress_stage": "${PROGRESS_STAGE:-null}", "timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)" } EOF exit 0 } trap cleanup_and_save SIGTERM SIGINT EXIT ``` ### Phase 3: Recovery Logic **Add to loop start:** ```bash recover_agent_state() { for state_file in /tmp/agent_state_$(whoami)_*.json; do if [ -f "$state_file" ]; then RECOVERED_TASK_ID=$(jq -r '.task_id' "$state_file") echo "Recovering task: $RECOVERED_TASK_ID" # Continue from saved context rm "$state_file" return 0 fi done return 1 } ``` **Testing:** Send SIGTERM to active loops, verify state persistence and recovery on restart. This prevents work loss from process interruption, improving agent reliability significantly.
kimi was assigned by Timmy 2026-03-24 15:12:57 +00:00
kimi was unassigned by Timmy 2026-03-24 19:32:21 +00:00
Timmy closed this issue 2026-03-24 21:54:11 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#1411
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?