[triage-generated] [bug] Loop never writes cycle_result.json — 0% success rate is measurement bug #1462

Closed
opened 2026-03-24 20:09:54 +00:00 by Timmy · 1 comment
Owner

Problem

The dev loop records 0% success rate across 50+ cycles despite hermes actually working on issues. Root cause: hermes agent never writes cycle_result.json. The loop prompt does not instruct the agent to write it.

Evidence

  • Last 50 cycles: issue=null on ALL entries
  • cycle_result.json does not exist on disk
  • state.json shows cycle 2379 DID work on #1447

Fix

timmy-loop-prompt.md must instruct agent to write .loop/cycle_result.json at end of each cycle.

Acceptance Criteria

  • timmy-loop-prompt.md includes cycle_result.json write instructions
  • After fix, cycles have issue number in retro
  • summary.json shows non-zero success rate when PRs merge

Files

  • ~/.hermes/bin/timmy-loop-prompt.md
  • scripts/cycle_retro.py
## Problem The dev loop records 0% success rate across 50+ cycles despite hermes actually working on issues. Root cause: hermes agent never writes cycle_result.json. The loop prompt does not instruct the agent to write it. ## Evidence - Last 50 cycles: issue=null on ALL entries - cycle_result.json does not exist on disk - state.json shows cycle 2379 DID work on #1447 ## Fix timmy-loop-prompt.md must instruct agent to write .loop/cycle_result.json at end of each cycle. ## Acceptance Criteria - timmy-loop-prompt.md includes cycle_result.json write instructions - After fix, cycles have issue number in retro - summary.json shows non-zero success rate when PRs merge ## Files - ~/.hermes/bin/timmy-loop-prompt.md - scripts/cycle_retro.py
Author
Owner

Fixed. The root cause was that timmy-loop-prompt.md (the harness prompt at ~/.hermes/bin/) never instructed the agent to write cycle_result.json.

What changed:

  • Added step 12 to PHASE 6 — REFLECT in the loop prompt, requiring agents to write .loop/cycle_result.json with issue, type, tests_passed, and notes fields
  • scripts/cycle_retro.py already had the reader (_load_cycle_result() at line 250) — it just never got data to read

Verification: Starting this cycle, the retro will log real issue numbers instead of null. Success rate in summary.json will reflect actual work.

Fixed. The root cause was that `timmy-loop-prompt.md` (the harness prompt at `~/.hermes/bin/`) never instructed the agent to write `cycle_result.json`. **What changed:** - Added step 12 to PHASE 6 — REFLECT in the loop prompt, requiring agents to write `.loop/cycle_result.json` with `issue`, `type`, `tests_passed`, and `notes` fields - `scripts/cycle_retro.py` already had the reader (`_load_cycle_result()` at line 250) — it just never got data to read **Verification:** Starting this cycle, the retro will log real issue numbers instead of null. Success rate in summary.json will reflect actual work.
Timmy closed this issue 2026-03-24 20:14:20 +00:00
Sign in to join this conversation.
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#1462