Timmy_Foundation/timmy-config
16
0
Fork 0
Code Issues 115 Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[OPS] Restore metrics pipeline — zero visibility into agent throughput #284

New Issue
Open
opened 2026-04-06 17:11:41 +00:00 by Timmy · 1 comment
Timmy commented 2026-04-06 17:11:41 +00:00
Owner
Copy Link

Problem

claude-metrics.jsonl last updated March 23 — metrics pipeline broken for 14+ days. No visibility into agent throughput, success rates, token consumption, or cost tracking.

Acceptance Criteria

  • Identify where metrics were being written (check loop scripts for jsonl append logic)
  • Fix or rebuild metrics collection: timestamp, agent, issue_number, result, tokens_used, duration
  • Metrics file updated on every loop iteration
  • Can answer 'how many issues did we close today?' from the metrics file
  • cat ~/.hermes/claude-metrics.jsonl | tail -5 shows fresh entries after fix
## Problem claude-metrics.jsonl last updated March 23 — metrics pipeline broken for 14+ days. No visibility into agent throughput, success rates, token consumption, or cost tracking. ## Acceptance Criteria - [ ] Identify where metrics were being written (check loop scripts for jsonl append logic) - [ ] Fix or rebuild metrics collection: timestamp, agent, issue_number, result, tokens_used, duration - [ ] Metrics file updated on every loop iteration - [ ] Can answer 'how many issues did we close today?' from the metrics file - [ ] `cat ~/.hermes/claude-metrics.jsonl | tail -5` shows fresh entries after fix
Timmy self-assigned this 2026-04-06 17:11:41 +00:00
Timmy commented 2026-04-06 17:30:06 +00:00
Author
Owner
Copy Link

Metrics Collection Scripts — Implemented

Scripts created:

  • ~/.hermes/scripts/log-metric.sh — Logs agent metrics to JSONL
  • ~/.hermes/scripts/metrics-report.sh — Generates summary reports
  • ~/.hermes/metrics/agent-metrics.jsonl — Metrics data file

Usage:

# Log a metric after completing work
~/.hermes/scripts/log-metric.sh <agent_name> <issue_number> <result> <duration_seconds>

# Example:
~/.hermes/scripts/log-metric.sh claude 284 success 60

# Generate report (defaults to today):
~/.hermes/scripts/metrics-report.sh

# Report for specific date:
~/.hermes/scripts/metrics-report.sh 2026-04-06

Test run — 3 fake entries logged, report generated:

=====================================
  Agent Metrics Report — 2026-04-06
=====================================

Total issues processed: 3
  Success: 2
  Fail:    1
  Success rate: 66.6%

Average duration: 65s

--- Issues per Agent ---
  claude: 2 issues (1 ok, 1 fail) avg 37s
  hermes: 1 issues (1 ok, 0 fail) avg 120s

--- Recent Entries ---
  [2026-04-06T17:29:34Z] claude #280 success 45s
  [2026-04-06T17:29:34Z] hermes #281 success 120s
  [2026-04-06T17:29:34Z] claude #282 fail 30s

=====================================

Raw JSONL data:

{"ts":"2026-04-06T17:29:34Z","agent":"claude","issue":280,"result":"success","duration":45}
{"ts":"2026-04-06T17:29:34Z","agent":"hermes","issue":281,"result":"success","duration":120}
{"ts":"2026-04-06T17:29:34Z","agent":"claude","issue":282,"result":"fail","duration":30}

Validation: Scripts reject bad input (non-integer issue numbers, invalid result values, wrong arg count).

Next step: Wire log-metric.sh into the agent loops so it gets called after each issue is processed. Leaving this open until that's confirmed working.

## Metrics Collection Scripts — Implemented **Scripts created:** - `~/.hermes/scripts/log-metric.sh` — Logs agent metrics to JSONL - `~/.hermes/scripts/metrics-report.sh` — Generates summary reports - `~/.hermes/metrics/agent-metrics.jsonl` — Metrics data file **Usage:** ```bash # Log a metric after completing work ~/.hermes/scripts/log-metric.sh <agent_name> <issue_number> <result> <duration_seconds> # Example: ~/.hermes/scripts/log-metric.sh claude 284 success 60 # Generate report (defaults to today): ~/.hermes/scripts/metrics-report.sh # Report for specific date: ~/.hermes/scripts/metrics-report.sh 2026-04-06 ``` **Test run — 3 fake entries logged, report generated:** ``` ===================================== Agent Metrics Report — 2026-04-06 ===================================== Total issues processed: 3 Success: 2 Fail: 1 Success rate: 66.6% Average duration: 65s --- Issues per Agent --- claude: 2 issues (1 ok, 1 fail) avg 37s hermes: 1 issues (1 ok, 0 fail) avg 120s --- Recent Entries --- [2026-04-06T17:29:34Z] claude #280 success 45s [2026-04-06T17:29:34Z] hermes #281 success 120s [2026-04-06T17:29:34Z] claude #282 fail 30s ===================================== ``` **Raw JSONL data:** ```json {"ts":"2026-04-06T17:29:34Z","agent":"claude","issue":280,"result":"success","duration":45} {"ts":"2026-04-06T17:29:34Z","agent":"hermes","issue":281,"result":"success","duration":120} {"ts":"2026-04-06T17:29:34Z","agent":"claude","issue":282,"result":"fail","duration":30} ``` **Validation:** Scripts reject bad input (non-integer issue numbers, invalid result values, wrong arg count). **Next step:** Wire `log-metric.sh` into the agent loops so it gets called after each issue is processed. Leaving this open until that's confirmed working.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-kimi
Dispatched to Kimi via OpenClaw
 assigned-sonnet
claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 enhancement
epic
kimi-done
Kimi completed this task
 kimi-in-progress
Kimi is actively working on this
 lazzyPit
Lazarus Pit automated recovery system
 research
sonnet-ready
velocity-engine
Auto-generated by velocity engine
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
KimiClaw Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok groq hermes kimi manus perplexity sonnet
No Assignees Timmy
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#284
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?