Timmy_Foundation/timmy-home
16
0
Fork 0
Code Issues 176 Pull Requests Actions Packages Projects Releases Wiki Activity

[#79] JSONL Scorecard Generator - overnight loop analysis #102

Merged
Timmy merged 1 commits from feature/scorecard-generator into main 2026-03-30 15:58:12 +00:00
Conversation 0 Commits 1 Files Changed 2 +513
allegro commented 2026-03-30 15:50:33 +00:00
Member
Copy Link

Summary

Analyzes overnight loop JSONL data and produces comprehensive scorecards with statistics, trends, and recommendations.

Features

  • Reads ~/shared/overnight-loop/*.jsonl
  • Generates both JSON (structured) and Markdown (human-readable) reports
  • Pass/fail statistics with pass rates
  • Duration analysis (avg, median, p95, min, max)
  • Per-task breakdowns (sorted by pass rate)
  • Hourly timeline trends
  • Error pattern frequency analysis
  • Auto-generated actionable recommendations

Output Files

  • ~/timmy/reports/scorecard_YYYYMMDD.json - Structured data
  • ~/timmy/reports/scorecard_YYYYMMDD.md - Human-readable report

Usage

# Basic usage
python uni-wizard/scripts/generate_scorecard.py

# Custom directories
python uni-wizard/scripts/generate_scorecard.py --input ~/data --output ~/reports

# Cron (daily at 6 AM)
0 6 * * * /root/timmy/venv/bin/python /root/timmy/uni-wizard/scripts/generate_scorecard.py

Report Contents

  1. Summary - Total tasks, pass/fail counts, pass rate
  2. Duration Stats - Average, median, P95, min, max
  3. Per-Task Breakdown - Each task type with pass rates and durations
  4. Timeline - Hourly performance trends
  5. Error Analysis - Most common error patterns
  6. Recommendations - Actionable insights based on data

Error Handling

  • Gracefully handles malformed JSONL lines
  • Skips empty files with warning
  • Tolerates missing optional fields

Acceptance Criteria

  • Reads any JSONL file in expected format
  • Produces both markdown and JSON output
  • Handles empty/malformed lines gracefully
  • Can be run manually or via cron

Closes #79

## Summary Analyzes overnight loop JSONL data and produces comprehensive scorecards with statistics, trends, and recommendations. ## Features - Reads `~/shared/overnight-loop/*.jsonl` - Generates both JSON (structured) and Markdown (human-readable) reports - Pass/fail statistics with pass rates - Duration analysis (avg, median, p95, min, max) - Per-task breakdowns (sorted by pass rate) - Hourly timeline trends - Error pattern frequency analysis - Auto-generated actionable recommendations ## Output Files - `~/timmy/reports/scorecard_YYYYMMDD.json` - Structured data - `~/timmy/reports/scorecard_YYYYMMDD.md` - Human-readable report ## Usage ```bash # Basic usage python uni-wizard/scripts/generate_scorecard.py # Custom directories python uni-wizard/scripts/generate_scorecard.py --input ~/data --output ~/reports # Cron (daily at 6 AM) 0 6 * * * /root/timmy/venv/bin/python /root/timmy/uni-wizard/scripts/generate_scorecard.py ``` ## Report Contents 1. **Summary** - Total tasks, pass/fail counts, pass rate 2. **Duration Stats** - Average, median, P95, min, max 3. **Per-Task Breakdown** - Each task type with pass rates and durations 4. **Timeline** - Hourly performance trends 5. **Error Analysis** - Most common error patterns 6. **Recommendations** - Actionable insights based on data ## Error Handling - Gracefully handles malformed JSONL lines - Skips empty files with warning - Tolerates missing optional fields ## Acceptance Criteria - [x] Reads any JSONL file in expected format - [x] Produces both markdown and JSON output - [x] Handles empty/malformed lines gracefully - [x] Can be run manually or via cron Closes #79
allegro added 1 commit 2026-03-30 15:50:33 +00:00
[#79] JSONL Scorecard Generator for overnight loop analysis 5f549bf1f6 
Generates comprehensive reports from overnight loop JSONL data:

**Features:**
- Reads ~/shared/overnight-loop/*.jsonl
- Produces JSON and Markdown reports
- Pass/fail statistics with pass rates
- Duration analysis (avg, median, p95)
- Per-task breakdowns
- Hourly timeline trends
- Error pattern analysis
- Auto-generated recommendations

**Reports:**
- ~/timmy/reports/scorecard_YYYYMMDD.json (structured)
- ~/timmy/reports/scorecard_YYYYMMDD.md (human-readable)

**Usage:**
  python uni-wizard/scripts/generate_scorecard.py

Closes #79
Timmy merged commit 276f2c32dd into main 2026-03-30 15:58:12 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
alembic
The Alchemical Vessel - Kimi-powered transformation
 architecture
Auto-created architecture label
 assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-kimi
Dispatched to Kimi via OpenClaw
 auth-needed
auth-needed
 bizzalo
bizzalo
 blocker
blocker
 claw
Claw Code runtime (Rust)
 claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 deadline-7am
deadline-7am
 epic
Parent tracking issue
 golden_bilbo
Bilbo at peak performance - max churn, fast responses
 harness-engineering
Auto-created harness-engineering label
 intel
Competitive intelligence
 kimi-done
Kimi completed this task
 kimi-in-progress
Kimi is actively working on this
 morning-report
morning-report
 runtime
Runtime abstraction layer
 saiyah
Auto-created saiyah label
 study
Learning from external source code
 substratum
The dissolution engine - runtime layer
 substratum:invoke
Trigger Substratum execution
 urgent
urgent
 velocity-engine
Auto-generated by velocity engine
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
KimiClaw Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok groq hermes kimi manus perplexity
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-home#102
Delete Branch "feature/scorecard-generator"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?