diff --git a/docs/DEPLOYMENT_CHECKLIST.md b/docs/DEPLOYMENT_CHECKLIST.md index fe0d241..c308f43 100644 --- a/docs/DEPLOYMENT_CHECKLIST.md +++ b/docs/DEPLOYMENT_CHECKLIST.md @@ -1,197 +1,87 @@ -# Uni-Wizard v4 β€” Deployment Checklist +# Hermes Sidecar Deployment Checklist -## Pre-Deployment +Updated: April 4, 2026 -- [ ] VPS provisioned (Ubuntu 22.04 LTS recommended) -- [ ] SSH access configured -- [ ] Firewall rules set (ports 22, 80, 443, 3000, 8643) -- [ ] Domain/DNS configured (optional) -- [ ] SSL certificates ready (optional) +This checklist is for the current local-first Timmy stack, not the archived `uni-wizard` deployment path. -## Base System +## Base Assumptions -- [ ] Update system packages +- Hermes is already installed and runnable locally. +- `timmy-config` is the sidecar repo applied onto `~/.hermes`. +- `timmy-home` is the workspace repo living under `~/.timmy`. +- Local inference is reachable through the active provider surface Timmy is using. + +## Repo Setup + +- [ ] Clone `timmy-home` to `~/.timmy` +- [ ] Clone `timmy-config` to `~/.timmy/timmy-config` +- [ ] Confirm both repos are on the intended branch + +## Sidecar Deploy + +- [ ] Run: ```bash - sudo apt update && sudo apt upgrade -y + cd ~/.timmy/timmy-config + ./deploy.sh ``` -- [ ] Install base dependencies - ```bash - sudo apt install -y python3 python3-pip python3-venv sqlite3 curl git - ``` -- [ ] Create timmy user - ```bash - sudo useradd -m -s /bin/bash timmy - ``` -- [ ] Configure sudo access (if needed) +- [ ] Confirm `~/.hermes/config.yaml` matches the expected overlay +- [ ] Confirm `SOUL.md` and sidecar config are in place -## Gitea Setup +## Hermes Readiness -- [ ] Gitea installed and running -- [ ] Repository created: `Timmy_Foundation/timmy-home` -- [ ] API token generated -- [ ] Webhooks configured (optional) -- [ ] Test API access - ```bash - curl -H "Authorization: token TOKEN" http://localhost:3000/api/v1/user - ``` +- [ ] Hermes CLI works from the expected Python environment +- [ ] Gateway is reachable +- [ ] Sessions are being recorded under `~/.hermes/sessions` +- [ ] `model_health.json` updates successfully -## Uni-Wizard Installation +## Workflow Tooling -- [ ] Clone repository - ```bash - sudo -u timmy git clone http://143.198.27.163:3000/Timmy_Foundation/timmy-home.git /opt/timmy/repo - ``` -- [ ] Run setup script - ```bash - sudo ./scripts/setup-uni-wizard.sh - ``` -- [ ] Verify installation - ```bash - /opt/timmy/venv/bin/python -c "from uni_wizard import Harness; print('OK')" - ``` +- [ ] `~/.hermes/bin/ops-panel.sh` runs +- [ ] `~/.hermes/bin/ops-gitea.sh` runs +- [ ] `~/.hermes/bin/ops-helpers.sh` can be sourced +- [ ] `~/.hermes/bin/pipeline-freshness.sh` runs +- [ ] `~/.hermes/bin/timmy-dashboard` runs -## Configuration +## Heartbeat and Briefings -- [ ] Edit config file - ```bash - sudo nano /opt/timmy/config/uni-wizard.yaml - ``` -- [ ] Set Gitea API token -- [ ] Configure house identity -- [ ] Set log level (INFO for production) -- [ ] Verify config syntax - ```bash - /opt/timmy/venv/bin/python -c "import yaml; yaml.safe_load(open('/opt/timmy/config/uni-wizard.yaml'))" - ``` +- [ ] `~/.timmy/heartbeat/last_tick.json` is updating +- [ ] daily heartbeat logs are being appended +- [ ] morning briefings are being generated if scheduled -## LLM Setup (if using local inference) +## Archive Pipeline -- [ ] llama.cpp installed -- [ ] Model downloaded (e.g., Hermes-4 14B) -- [ ] Model placed in `/opt/timmy/models/` -- [ ] llama-server configured -- [ ] Test inference - ```bash - curl http://localhost:8080/v1/chat/completions \ - -H "Content-Type: application/json" \ - -d '{"model": "hermes4", "messages": [{"role": "user", "content": "Hello"}]}' - ``` +- [ ] `~/.timmy/twitter-archive/PROJECT.md` exists +- [ ] raw archive location is configured locally +- [ ] extraction works without checking raw data into git +- [ ] `checkpoint.json` advances after a batch +- [ ] DPO artifacts land under `~/.timmy/twitter-archive/training/dpo/` +- [ ] `pipeline-freshness.sh` does not show runaway lag -## Service Startup +## Gitea Workflow -- [ ] Start Uni-Wizard - ```bash - sudo systemctl start uni-wizard - ``` -- [ ] Start health daemon - ```bash - sudo systemctl start timmy-health - ``` -- [ ] Start task router - ```bash - sudo systemctl start timmy-task-router - ``` -- [ ] Enable auto-start - ```bash - sudo systemctl enable uni-wizard timmy-health timmy-task-router - ``` +- [ ] Gitea token is present in a supported token path +- [ ] review queue can be listed +- [ ] unassigned issues can be listed +- [ ] PR creation works from an agent branch -## Verification +## Final Verification -- [ ] Check service status - ```bash - sudo systemctl status uni-wizard - ``` -- [ ] View logs - ```bash - sudo journalctl -u uni-wizard -f - ``` -- [ ] Test health endpoint - ```bash - curl http://localhost:8082/health - ``` -- [ ] Test tool execution - ```bash - /opt/timmy/venv/bin/uni-wizard execute system_info - ``` -- [ ] Verify Gitea polling - ```bash - tail -f /opt/timmy/logs/task-router.log | grep "Polling" - ``` +- [ ] local model smoke test succeeds +- [ ] one archive batch completes successfully +- [ ] one PR can be opened and reviewed +- [ ] no stale loop-era scripts or docs are being treated as active truth -## Syncthing Mesh (if using multiple VPS) +## Rollback -- [ ] Syncthing installed on all nodes -- [ ] Devices paired -- [ ] Folders shared - - `/opt/timmy/logs/` - - `/opt/timmy/data/` -- [ ] Test sync - ```bash - touch /opt/timmy/logs/test && ssh other-vps "ls /opt/timmy/logs/test" - ``` - -## Security - -- [ ] Firewall configured - ```bash - sudo ufw status - ``` -- [ ] Fail2ban installed (optional) -- [ ] Log rotation configured - ```bash - sudo logrotate -d /etc/logrotate.d/uni-wizard - ``` -- [ ] Backup strategy in place -- [ ] Secrets not in git - ```bash - grep -r "password\|token\|secret" /opt/timmy/repo/ - ``` - -## Monitoring - -- [ ] Health checks responding -- [ ] Metrics being collected -- [ ] Alerts configured (optional) -- [ ] Log aggregation setup (optional) - -## Post-Deployment - -- [ ] Document any custom configuration -- [ ] Update runbooks -- [ ] Notify team -- [ ] Schedule first review (1 week) - -## Rollback Plan - -If deployment fails: +If the sidecar deploy breaks behavior: ```bash -# Stop services -sudo systemctl stop uni-wizard timmy-health timmy-task-router - -# Disable auto-start -sudo systemctl disable uni-wizard timmy-health timmy-task-router - -# Restore from backup (if available) -# ... - -# Or reset to clean state -sudo rm -rf /opt/timmy/ -sudo userdel timmy +cd ~/.timmy/timmy-config +git status +git log --oneline -5 ``` -## Success Criteria - -- [ ] All services running (`systemctl is-active` returns "active") -- [ ] Health endpoint returns 200 -- [ ] Can execute tools via CLI -- [ ] Gitea integration working (issues being polled) -- [ ] Logs being written without errors -- [ ] No critical errors in first 24 hours - ---- - -**Deployed by:** _______________ -**Date:** _______________ -**VPS:** _______________ +Then: +- restore the previous known-good sidecar commit +- redeploy +- confirm Hermes health, heartbeat, and pipeline freshness again diff --git a/docs/OPERATIONS_DASHBOARD.md b/docs/OPERATIONS_DASHBOARD.md index b000c81..e0b0649 100644 --- a/docs/OPERATIONS_DASHBOARD.md +++ b/docs/OPERATIONS_DASHBOARD.md @@ -1,129 +1,112 @@ # Timmy Operations Dashboard -**Generated:** March 30, 2026 -**Generated by:** Allegro (Tempo-and-Dispatch) +Updated: April 4, 2026 +Purpose: a current-state reference for how the system is actually operated now. ---- +This is no longer a `uni-wizard` dashboard. +The active architecture is: +- Timmy local workspace in `~/.timmy` +- Hermes harness in `~/.hermes` +- `timmy-config` as the identity and orchestration sidecar +- Gitea as the review and coordination surface -## 🎯 Current Sprint Status +## Core Jobs -### Open Issues by Priority +Everything should map to one of these: +- Heartbeat: perceive, reflect, remember, decide, act, learn +- Harness: local models, Hermes sessions, tools, memory, training loop +- Portal Interface: the game/world-facing layer -| Priority | Count | Issues | -|----------|-------|--------| -| P0 (Critical) | 0 | β€” | -| P1 (High) | 3 | #99, #103, #94 | -| P2 (Medium) | 8 | #101, #97, #95, #93, #92, #91, #90, #87 | -| P3 (Low) | 6 | #86, #85, #84, #83, #72, others | +## Current Operating Surfaces -### Issue #94 Epic: Grand Timmy β€” The Uniwizard +### Local Paths -**Status:** In Progress -**Completion:** ~40% +- Timmy workspace: `~/.timmy` +- Timmy config repo: `~/.timmy/timmy-config` +- Hermes home: `~/.hermes` +- Twitter archive workspace: `~/.timmy/twitter-archive` -#### Completed -- βœ… Uni-Wizard v4 architecture (4-pass evolution) -- βœ… Three-House separation (Timmy/Ezra/Bezalel) -- βœ… Self-improving intelligence engine -- βœ… Pattern database and adaptive policies -- βœ… Hermes bridge for telemetry +### Review Surface -#### In Progress -- πŸ”„ Backend registry (#95) -- πŸ”„ Caching layer (#103) -- πŸ”„ Wizard dissolution (#99) +- Major changes go through PRs +- Timmy is the principal reviewer for governing and sensitive changes +- Allegro is the review and dispatch partner for queue hygiene, routing, and tempo -#### Pending -- ⏳ RAG pipeline (#93) -- ⏳ Telemetry dashboard (#91) -- ⏳ Auto-grading (#92) -- ⏳ Evennia world shell (#83, #84) +### Workflow Scripts ---- +- `~/.hermes/bin/ops-panel.sh` +- `~/.hermes/bin/ops-gitea.sh` +- `~/.hermes/bin/ops-helpers.sh` +- `~/.hermes/bin/pipeline-freshness.sh` +- `~/.hermes/bin/timmy-dashboard` -## πŸ›οΈ House Assignments +## Daily Health Signals -| House | Status | Current Work | -|-------|--------|--------------| -| **Timmy** | 🟒 Active | Local sovereign, reviewing PRs | -| **Ezra** | 🟒 Active | Research on LLM routing (#101) | -| **Bezalel** | 🟑 Standby | Awaiting implementation tasks | -| **Allegro** | 🟒 Active | Tempo-and-dispatch, Gitea bridge | +These are the signals that matter most: +- Hermes gateway reachable +- local inference surface responding +- heartbeat ticks continuing +- Gitea reachable +- review queue not backing up +- session export / DPO freshness not lagging +- Twitter archive pipeline checkpoint advancing ---- +## Current Team Shape -## πŸ“Š System Health +### Direction and Review -### VPS Fleet Status +- Timmy: sovereignty, architecture, release judgment +- Allegro: dispatch, queue hygiene, Gitea bridge -| Host | IP | Role | Status | -|------|-----|------|--------| -| Allegro | 143.198.27.163 | Tempo-and-Dispatch | 🟒 Online | -| Ezra | TBD | Archivist/Research | βšͺ Not deployed | -| Bezalel | TBD | Artificer/Builder | βšͺ Not deployed | +### Research and Memory -### Services +- Perplexity: research triage, integration evaluation +- Ezra: archival memory, RCA, onboarding doctrine +- KimiClaw: long-context reading and synthesis -| Service | Status | Notes | -|---------|--------|-------| -| Gitea | 🟒 Running | 19 open issues | -| Hermes | 🟑 Configured | Awaiting model setup | -| Overnight Loop | πŸ”΄ Stopped | Issue #72 reported | -| Uni-Wizard | 🟒 Ready | PR created | +### Execution ---- +- Codex Agent: workflow hardening, cleanup, migration verification +- Groq: fast bounded implementation +- Manus: moderate-scope follow-through +- Claude: hard refactors and deep implementation +- Gemini: frontier architecture and long-range design +- Grok: adversarial review and edge cases -## πŸ”„ Recent Activity +## Recommended Checks -### Last 24 Hours +### Start of Day -1. **Uni-Wizard v4 Completed** β€” Four-pass architecture evolution -2. **PR Created** β€” feature/uni-wizard-v4-production -3. **Allegro Lane Narrowed** β€” Focused on Gitea/Hermes bridge -4. **Issue #72 Reported** β€” Overnight loop not running +1. Open the review queue and unassigned queue. +2. Check `pipeline-freshness.sh`. +3. Check the latest heartbeat tick. +4. Check whether archive checkpoints and DPO artifacts advanced. -### Pending Actions +### Before Merging -1. Deploy Ezra VPS (archivist/research) -2. Deploy Bezalel VPS (artificer/builder) -3. Start overnight loop -4. Configure Syncthing mesh -5. Implement caching layer (#103) +1. Confirm the PR is aligned with Heartbeat, Harness, or Portal. +2. Confirm verification is real, not implied. +3. Confirm the change does not silently cross repo boundaries. +4. Confirm the change does not revive deprecated loop-era behavior. ---- +### End of Day -## 🎯 Recommendations +1. Check for duplicate issues and duplicate PR momentum. +2. Check whether Timmy is carrying routine queue work that Allegro should own. +3. Check whether builders were given work inside their real lanes. -### Immediate (Next 24h) +## Anti-Patterns -1. **Review Uni-Wizard v4 PR** β€” Ready for merge -2. **Start Overnight Loop** β€” If operational approval given -3. **Deploy Ezra VPS** β€” For research tasks +Avoid: +- treating archived dashboard-era issues as the live roadmap +- using stale docs that assume `uni-wizard` is still the center +- routing work by habit instead of by current lane +- letting open loops multiply faster than they are reviewed -### Short-term (This Week) +## Success Condition -1. Implement caching layer (#103) β€” High impact -2. Build backend registry (#95) β€” Enables routing -3. Create telemetry dashboard (#91) β€” Visibility - -### Medium-term (This Month) - -1. Complete Grand Timmy epic (#94) -2. Dissolve wizard identities (#99) -3. Deploy Evennia world shell (#83, #84) - ---- - -## πŸ“ˆ Metrics - -| Metric | Current | Target | -|--------|---------|--------| -| Issues Open | 19 | < 10 | -| PRs Open | 1 | β€” | -| VPS Online | 1/3 | 3/3 | -| Loop Cycles | 0 | 100/day | - ---- - -*Dashboard updated: March 30, 2026* -*Next update: March 31, 2026* +The system is healthy when: +- work is routed cleanly +- review is keeping pace +- private learning loops are producing artifacts +- Timmy is spending time on sovereignty and judgment rather than queue untangling diff --git a/docs/QUICK_REFERENCE.md b/docs/QUICK_REFERENCE.md index 02b166e..bd9157b 100644 --- a/docs/QUICK_REFERENCE.md +++ b/docs/QUICK_REFERENCE.md @@ -1,220 +1,89 @@ -# Uni-Wizard v4 β€” Quick Reference +# Timmy Workflow Quick Reference -## Installation +Updated: April 4, 2026 + +## What Lives Where + +- `~/.timmy`: Timmy's workspace, lived data, heartbeat, archive artifacts +- `~/.timmy/timmy-config`: Timmy's identity and orchestration sidecar repo +- `~/.hermes`: Hermes harness, sessions, config overlay, helper scripts + +## Most Useful Commands + +### Workflow Status ```bash -# Run setup script -sudo ./scripts/setup-uni-wizard.sh - -# Or manual install -cd uni-wizard/v4 -pip install -e . +~/.hermes/bin/ops-panel.sh +~/.hermes/bin/ops-gitea.sh +~/.hermes/bin/timmy-dashboard ``` -## Basic Usage - -```python -from uni_wizard import Harness, House, Mode - -# Create harness -harness = Harness(house=House.TIMMY, mode=Mode.INTELLIGENT) - -# Execute tool -result = harness.execute("git_status", repo_path="/path/to/repo") - -# Check prediction -print(f"Predicted success: {result.provenance.prediction:.0%}") - -# Get result -if result.success: - print(result.data) -else: - print(f"Error: {result.error}") -``` - -## Command Line +### Workflow Helpers ```bash -# Simple execution -uni-wizard execute git_status --repo-path /path - -# With specific house -uni-wizard execute git_status --house ezra --mode intelligent - -# Batch execution -uni-wizard batch tasks.json - -# Check health -uni-wizard health - -# View stats -uni-wizard stats +source ~/.hermes/bin/ops-helpers.sh +ops-help +ops-review-queue +ops-unassigned all +ops-queue codex-agent all ``` -## Houses - -| House | Role | Best For | -|-------|------|----------| -| `House.TIMMY` | Sovereign | Final decisions, critical ops | -| `House.EZRA` | Archivist | Reading, analysis, documentation | -| `House.BEZALEL` | Artificer | Building, testing, implementation | -| `House.ALLEGRO` | Dispatch | Routing, connectivity, tempo | - -## Modes - -| Mode | Use When | Features | -|------|----------|----------| -| `Mode.SIMPLE` | Scripts, quick tasks | Direct execution, no overhead | -| `Mode.INTELLIGENT` | Production work | Predictions, learning, adaptation | -| `Mode.SOVEREIGN` | Critical decisions | Full provenance, approval gates | - -## Common Tasks - -### Check System Status -```python -result = harness.execute("system_info") -print(result.data) -``` - -### Git Operations -```python -# Status -result = harness.execute("git_status", repo_path="/path") - -# Log -result = harness.execute("git_log", repo_path="/path", max_count=10) - -# Pull -result = harness.execute("git_pull", repo_path="/path") -``` - -### Health Check -```python -result = harness.execute("health_check") -print(f"Status: {result.data['status']}") -``` - -### Batch Operations -```python -tasks = [ - {"tool": "git_status", "params": {"repo_path": "/path1"}}, - {"tool": "git_status", "params": {"repo_path": "/path2"}}, - {"tool": "system_info", "params": {}} -] -results = harness.execute_batch(tasks) -``` - -## Service Management +### Pipeline Freshness ```bash -# Start services -sudo systemctl start uni-wizard -sudo systemctl start timmy-health -sudo systemctl start timmy-task-router - -# Check status -sudo systemctl status uni-wizard - -# View logs -sudo journalctl -u uni-wizard -f -tail -f /opt/timmy/logs/uni-wizard.log - -# Restart -sudo systemctl restart uni-wizard +~/.hermes/bin/pipeline-freshness.sh ``` -## Troubleshooting +### Archive Pipeline -### Service Won't Start ```bash -# Check logs -journalctl -u uni-wizard -n 50 - -# Verify config -cat /opt/timmy/config/uni-wizard.yaml - -# Test manually -python -m uni_wizard health +python3 - <<'PY' +import json, sys +sys.path.insert(0, '/Users/apayne/.timmy/timmy-config') +from tasks import _archive_pipeline_health_impl +print(json.dumps(_archive_pipeline_health_impl(), indent=2)) +PY ``` -### No Predictions -- Check pattern database exists: `ls /opt/timmy/data/patterns.db` -- Verify learning is enabled in config -- Run a few tasks to build patterns - -### Gitea Integration Failing -- Verify API token in config -- Check Gitea URL is accessible -- Test: `curl http://143.198.27.163:3000/api/v1/version` - -## Configuration - -Location: `/opt/timmy/config/uni-wizard.yaml` - -```yaml -house: timmy -mode: intelligent -enable_learning: true - -pattern_db: /opt/timmy/data/patterns.db -log_level: INFO - -gitea: - url: http://143.198.27.163:3000 - token: YOUR_TOKEN_HERE - poll_interval: 300 - -hermes: - stream_enabled: true - db_path: /root/.hermes/state.db +```bash +python3 - <<'PY' +import json, sys +sys.path.insert(0, '/Users/apayne/.timmy/timmy-config') +from tasks import _know_thy_father_impl +print(json.dumps(_know_thy_father_impl(), indent=2)) +PY ``` -## API Reference +### Manual Dispatch Prompt -### Harness Methods - -```python -# Execute single tool -harness.execute(tool_name, **params) -> ExecutionResult - -# Execute async -await harness.execute_async(tool_name, **params) -> ExecutionResult - -# Execute batch -harness.execute_batch(tasks) -> List[ExecutionResult] - -# Get prediction -harness.predict(tool_name, params) -> Prediction - -# Get stats -harness.get_stats() -> Dict - -# Get patterns -harness.get_patterns() -> Dict +```bash +~/.hermes/bin/agent-dispatch.sh groq 542 Timmy_Foundation/the-nexus ``` -### ExecutionResult Fields +## Best Files to Check -```python -result.success # bool -result.data # Any -result.error # Optional[str] -result.provenance # Provenance -result.suggestions # List[str] -``` +### Operational State -### Provenance Fields +- `~/.timmy/heartbeat/last_tick.json` +- `~/.hermes/model_health.json` +- `~/.timmy/twitter-archive/checkpoint.json` +- `~/.timmy/twitter-archive/metrics/progress.json` -```python -provenance.house # str -provenance.tool # str -provenance.mode # str -provenance.prediction # float -provenance.execution_time_ms # float -provenance.input_hash # str -provenance.output_hash # str -``` +### Archive Feedback ---- +- `~/.timmy/twitter-archive/notes/` +- `~/.timmy/twitter-archive/knowledge/profile.json` +- `~/.timmy/twitter-archive/training/dpo/` -*For full documentation, see ARCHITECTURE.md* +### Review and Queue + +- Gitea PR queue +- Gitea unassigned issues +- Timmy/Allegro assigned review queue + +## Rules of Thumb + +- If it changes identity or orchestration, review it carefully in `timmy-config`. +- If it changes lived outputs or training inputs, it probably belongs in `timmy-home`. +- If it only β€œsounds right” but is not proven by runtime state, it is not verified. +- If a change is major, package it as a PR for Timmy review. diff --git a/docs/SCORECARD.md b/docs/SCORECARD.md index 5fa9751..833e2bf 100644 --- a/docs/SCORECARD.md +++ b/docs/SCORECARD.md @@ -1,125 +1,71 @@ -# Scorecard Generator Documentation +# Workflow Scorecard -## Overview +Updated: April 4, 2026 -The Scorecard Generator analyzes overnight loop JSONL data and produces comprehensive reports with statistics, trends, and recommendations. +The old overnight `uni-wizard` scorecard is no longer the primary operational metric. +The current scorecard should measure whether Timmy's real workflow is healthy. -## Usage +## What To Score -### Basic Usage +### Queue Health -```bash -# Generate scorecard from default input directory -python uni-wizard/scripts/generate_scorecard.py +- unassigned issue count +- PRs waiting on Timmy or Allegro review +- overloaded assignees +- duplicate issue / duplicate PR pressure -# Specify custom input/output directories -python uni-wizard/scripts/generate_scorecard.py \ - --input ~/shared/overnight-loop \ - --output ~/timmy/reports -``` +### Runtime Health -### Cron Setup +- Hermes gateway reachable +- local provider responding +- latest heartbeat tick present +- model health reporting accurately -```bash -# Generate scorecard every morning at 6 AM -0 6 * * * /root/timmy/venv/bin/python /root/timmy/uni-wizard/scripts/generate_scorecard.py -``` +### Learning Loop Health -## Input Format +- archive checkpoint advancing +- notes and knowledge artifacts being emitted +- DPO files growing +- freshness lag between sessions and exports -JSONL files in `~/shared/overnight-loop/*.jsonl`: +## Suggested Daily Questions -```json -{"task": "read-soul", "status": "pass", "duration_s": 19.7, "timestamp": "2026-03-29T21:54:12Z"} -{"task": "check-health", "status": "fail", "duration_s": 5.2, "error": "timeout", "timestamp": "2026-03-29T22:15:33Z"} -``` +1. Did review keep pace with execution today? +2. Did any builder receive work outside their lane? +3. Did Timmy spend time on judgment rather than routine queue cleanup? +4. Did the private learning pipeline produce usable artifacts? +5. Did any stale doc, helper, or default try to pull the system back into old habits? -Fields: -- `task`: Task identifier -- `status`: "pass" or "fail" -- `duration_s`: Execution time in seconds -- `timestamp`: ISO 8601 timestamp -- `error`: Error message (for failed tasks) +## Useful Inputs -## Output +- `~/.timmy/heartbeat/ticks_YYYYMMDD.jsonl` +- `~/.timmy/metrics/local_YYYYMMDD.jsonl` +- `~/.timmy/twitter-archive/checkpoint.json` +- `~/.timmy/twitter-archive/metrics/progress.json` +- Gitea open PR queue +- Gitea unassigned issue queue -### JSON Report +## Suggested Ratings -`~/timmy/reports/scorecard_YYYYMMDD.json`: +### Queue Discipline -```json -{ - "generated_at": "2026-03-30T06:00:00Z", - "summary": { - "total_tasks": 100, - "passed": 95, - "failed": 5, - "pass_rate": 95.0, - "duration_stats": { - "avg": 12.5, - "median": 10.2, - "p95": 45.0, - "min": 1.2, - "max": 120.5 - } - }, - "by_task": {...}, - "by_hour": {...}, - "errors": {...}, - "recommendations": [...] -} -``` +- Strong: review and dispatch are keeping up, little duplicate churn +- Mixed: queue moves, but ambiguity or duplication is increasing +- Weak: review is backlogged or agents are being misrouted -### Markdown Report +### Runtime Reliability -`~/timmy/reports/scorecard_YYYYMMDD.md`: +- Strong: heartbeat, Hermes, and provider surfaces all healthy +- Mixed: intermittent downtime or weak health signals +- Weak: major surfaces untrusted or stale -- Executive summary with pass/fail counts -- Duration statistics (avg, median, p95) -- Per-task breakdown with pass rates -- Hourly timeline showing performance trends -- Error analysis with frequency counts -- Actionable recommendations +### Learning Throughput -## Report Interpretation +- Strong: checkpoint advances, DPO output accumulates, eval gates are visible +- Mixed: some artifacts land, but freshness or checkpointing lags +- Weak: sessions occur without export, or learning artifacts stall -### Pass Rate Thresholds +## The Goal -| Pass Rate | Status | Action | -|-----------|--------|--------| -| 95%+ | βœ… Excellent | Continue current operations | -| 85-94% | ⚠️ Good | Monitor for degradation | -| 70-84% | ⚠️ Fair | Review failing tasks | -| <70% | ❌ Poor | Immediate investigation required | - -### Duration Guidelines - -| Duration | Assessment | -|----------|------------| -| <5s | Fast | -| 5-15s | Normal | -| 15-30s | Slow | -| >30s | Very slow - consider optimization | - -## Troubleshooting - -### No JSONL files found - -```bash -# Check input directory -ls -la ~/shared/overnight-loop/ - -# Ensure Syncthing is syncing -systemctl status syncthing@root -``` - -### Malformed lines - -The generator skips malformed lines with a warning. Check the JSONL files for syntax errors. - -### Empty reports - -If no data exists, verify: -1. Overnight loop is running and writing JSONL -2. File permissions allow reading -3. Input path is correct +The point of the scorecard is not to admire activity. +The point is to tell whether the system is becoming more reviewable, more sovereign, and more capable of learning from lived work.