Compare commits
2 Commits
GoldenRock
...
codex/work
| Author | SHA1 | Date | |
|---|---|---|---|
|
cb0d81e6cd | ||
|
|
4effb5a20e |
@@ -1,197 +1,87 @@
|
||||
# Uni-Wizard v4 — Deployment Checklist
|
||||
# Hermes Sidecar Deployment Checklist
|
||||
|
||||
## Pre-Deployment
|
||||
Updated: April 4, 2026
|
||||
|
||||
- [ ] VPS provisioned (Ubuntu 22.04 LTS recommended)
|
||||
- [ ] SSH access configured
|
||||
- [ ] Firewall rules set (ports 22, 80, 443, 3000, 8643)
|
||||
- [ ] Domain/DNS configured (optional)
|
||||
- [ ] SSL certificates ready (optional)
|
||||
This checklist is for the current local-first Timmy stack, not the archived `uni-wizard` deployment path.
|
||||
|
||||
## Base System
|
||||
## Base Assumptions
|
||||
|
||||
- [ ] Update system packages
|
||||
- Hermes is already installed and runnable locally.
|
||||
- `timmy-config` is the sidecar repo applied onto `~/.hermes`.
|
||||
- `timmy-home` is the workspace repo living under `~/.timmy`.
|
||||
- Local inference is reachable through the active provider surface Timmy is using.
|
||||
|
||||
## Repo Setup
|
||||
|
||||
- [ ] Clone `timmy-home` to `~/.timmy`
|
||||
- [ ] Clone `timmy-config` to `~/.timmy/timmy-config`
|
||||
- [ ] Confirm both repos are on the intended branch
|
||||
|
||||
## Sidecar Deploy
|
||||
|
||||
- [ ] Run:
|
||||
```bash
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
cd ~/.timmy/timmy-config
|
||||
./deploy.sh
|
||||
```
|
||||
- [ ] Install base dependencies
|
||||
```bash
|
||||
sudo apt install -y python3 python3-pip python3-venv sqlite3 curl git
|
||||
```
|
||||
- [ ] Create timmy user
|
||||
```bash
|
||||
sudo useradd -m -s /bin/bash timmy
|
||||
```
|
||||
- [ ] Configure sudo access (if needed)
|
||||
- [ ] Confirm `~/.hermes/config.yaml` matches the expected overlay
|
||||
- [ ] Confirm `SOUL.md` and sidecar config are in place
|
||||
|
||||
## Gitea Setup
|
||||
## Hermes Readiness
|
||||
|
||||
- [ ] Gitea installed and running
|
||||
- [ ] Repository created: `Timmy_Foundation/timmy-home`
|
||||
- [ ] API token generated
|
||||
- [ ] Webhooks configured (optional)
|
||||
- [ ] Test API access
|
||||
```bash
|
||||
curl -H "Authorization: token TOKEN" http://localhost:3000/api/v1/user
|
||||
```
|
||||
- [ ] Hermes CLI works from the expected Python environment
|
||||
- [ ] Gateway is reachable
|
||||
- [ ] Sessions are being recorded under `~/.hermes/sessions`
|
||||
- [ ] `model_health.json` updates successfully
|
||||
|
||||
## Uni-Wizard Installation
|
||||
## Workflow Tooling
|
||||
|
||||
- [ ] Clone repository
|
||||
```bash
|
||||
sudo -u timmy git clone http://143.198.27.163:3000/Timmy_Foundation/timmy-home.git /opt/timmy/repo
|
||||
```
|
||||
- [ ] Run setup script
|
||||
```bash
|
||||
sudo ./scripts/setup-uni-wizard.sh
|
||||
```
|
||||
- [ ] Verify installation
|
||||
```bash
|
||||
/opt/timmy/venv/bin/python -c "from uni_wizard import Harness; print('OK')"
|
||||
```
|
||||
- [ ] `~/.hermes/bin/ops-panel.sh` runs
|
||||
- [ ] `~/.hermes/bin/ops-gitea.sh` runs
|
||||
- [ ] `~/.hermes/bin/ops-helpers.sh` can be sourced
|
||||
- [ ] `~/.hermes/bin/pipeline-freshness.sh` runs
|
||||
- [ ] `~/.hermes/bin/timmy-dashboard` runs
|
||||
|
||||
## Configuration
|
||||
## Heartbeat and Briefings
|
||||
|
||||
- [ ] Edit config file
|
||||
```bash
|
||||
sudo nano /opt/timmy/config/uni-wizard.yaml
|
||||
```
|
||||
- [ ] Set Gitea API token
|
||||
- [ ] Configure house identity
|
||||
- [ ] Set log level (INFO for production)
|
||||
- [ ] Verify config syntax
|
||||
```bash
|
||||
/opt/timmy/venv/bin/python -c "import yaml; yaml.safe_load(open('/opt/timmy/config/uni-wizard.yaml'))"
|
||||
```
|
||||
- [ ] `~/.timmy/heartbeat/last_tick.json` is updating
|
||||
- [ ] daily heartbeat logs are being appended
|
||||
- [ ] morning briefings are being generated if scheduled
|
||||
|
||||
## LLM Setup (if using local inference)
|
||||
## Archive Pipeline
|
||||
|
||||
- [ ] llama.cpp installed
|
||||
- [ ] Model downloaded (e.g., Hermes-4 14B)
|
||||
- [ ] Model placed in `/opt/timmy/models/`
|
||||
- [ ] llama-server configured
|
||||
- [ ] Test inference
|
||||
```bash
|
||||
curl http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "hermes4", "messages": [{"role": "user", "content": "Hello"}]}'
|
||||
```
|
||||
- [ ] `~/.timmy/twitter-archive/PROJECT.md` exists
|
||||
- [ ] raw archive location is configured locally
|
||||
- [ ] extraction works without checking raw data into git
|
||||
- [ ] `checkpoint.json` advances after a batch
|
||||
- [ ] DPO artifacts land under `~/.timmy/twitter-archive/training/dpo/`
|
||||
- [ ] `pipeline-freshness.sh` does not show runaway lag
|
||||
|
||||
## Service Startup
|
||||
## Gitea Workflow
|
||||
|
||||
- [ ] Start Uni-Wizard
|
||||
```bash
|
||||
sudo systemctl start uni-wizard
|
||||
```
|
||||
- [ ] Start health daemon
|
||||
```bash
|
||||
sudo systemctl start timmy-health
|
||||
```
|
||||
- [ ] Start task router
|
||||
```bash
|
||||
sudo systemctl start timmy-task-router
|
||||
```
|
||||
- [ ] Enable auto-start
|
||||
```bash
|
||||
sudo systemctl enable uni-wizard timmy-health timmy-task-router
|
||||
```
|
||||
- [ ] Gitea token is present in a supported token path
|
||||
- [ ] review queue can be listed
|
||||
- [ ] unassigned issues can be listed
|
||||
- [ ] PR creation works from an agent branch
|
||||
|
||||
## Verification
|
||||
## Final Verification
|
||||
|
||||
- [ ] Check service status
|
||||
```bash
|
||||
sudo systemctl status uni-wizard
|
||||
```
|
||||
- [ ] View logs
|
||||
```bash
|
||||
sudo journalctl -u uni-wizard -f
|
||||
```
|
||||
- [ ] Test health endpoint
|
||||
```bash
|
||||
curl http://localhost:8082/health
|
||||
```
|
||||
- [ ] Test tool execution
|
||||
```bash
|
||||
/opt/timmy/venv/bin/uni-wizard execute system_info
|
||||
```
|
||||
- [ ] Verify Gitea polling
|
||||
```bash
|
||||
tail -f /opt/timmy/logs/task-router.log | grep "Polling"
|
||||
```
|
||||
- [ ] local model smoke test succeeds
|
||||
- [ ] one archive batch completes successfully
|
||||
- [ ] one PR can be opened and reviewed
|
||||
- [ ] no stale loop-era scripts or docs are being treated as active truth
|
||||
|
||||
## Syncthing Mesh (if using multiple VPS)
|
||||
## Rollback
|
||||
|
||||
- [ ] Syncthing installed on all nodes
|
||||
- [ ] Devices paired
|
||||
- [ ] Folders shared
|
||||
- `/opt/timmy/logs/`
|
||||
- `/opt/timmy/data/`
|
||||
- [ ] Test sync
|
||||
```bash
|
||||
touch /opt/timmy/logs/test && ssh other-vps "ls /opt/timmy/logs/test"
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
- [ ] Firewall configured
|
||||
```bash
|
||||
sudo ufw status
|
||||
```
|
||||
- [ ] Fail2ban installed (optional)
|
||||
- [ ] Log rotation configured
|
||||
```bash
|
||||
sudo logrotate -d /etc/logrotate.d/uni-wizard
|
||||
```
|
||||
- [ ] Backup strategy in place
|
||||
- [ ] Secrets not in git
|
||||
```bash
|
||||
grep -r "password\|token\|secret" /opt/timmy/repo/
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
- [ ] Health checks responding
|
||||
- [ ] Metrics being collected
|
||||
- [ ] Alerts configured (optional)
|
||||
- [ ] Log aggregation setup (optional)
|
||||
|
||||
## Post-Deployment
|
||||
|
||||
- [ ] Document any custom configuration
|
||||
- [ ] Update runbooks
|
||||
- [ ] Notify team
|
||||
- [ ] Schedule first review (1 week)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If deployment fails:
|
||||
If the sidecar deploy breaks behavior:
|
||||
|
||||
```bash
|
||||
# Stop services
|
||||
sudo systemctl stop uni-wizard timmy-health timmy-task-router
|
||||
|
||||
# Disable auto-start
|
||||
sudo systemctl disable uni-wizard timmy-health timmy-task-router
|
||||
|
||||
# Restore from backup (if available)
|
||||
# ...
|
||||
|
||||
# Or reset to clean state
|
||||
sudo rm -rf /opt/timmy/
|
||||
sudo userdel timmy
|
||||
cd ~/.timmy/timmy-config
|
||||
git status
|
||||
git log --oneline -5
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] All services running (`systemctl is-active` returns "active")
|
||||
- [ ] Health endpoint returns 200
|
||||
- [ ] Can execute tools via CLI
|
||||
- [ ] Gitea integration working (issues being polled)
|
||||
- [ ] Logs being written without errors
|
||||
- [ ] No critical errors in first 24 hours
|
||||
|
||||
---
|
||||
|
||||
**Deployed by:** _______________
|
||||
**Date:** _______________
|
||||
**VPS:** _______________
|
||||
Then:
|
||||
- restore the previous known-good sidecar commit
|
||||
- redeploy
|
||||
- confirm Hermes health, heartbeat, and pipeline freshness again
|
||||
|
||||
@@ -1,129 +1,112 @@
|
||||
# Timmy Operations Dashboard
|
||||
|
||||
**Generated:** March 30, 2026
|
||||
**Generated by:** Allegro (Tempo-and-Dispatch)
|
||||
Updated: April 4, 2026
|
||||
Purpose: a current-state reference for how the system is actually operated now.
|
||||
|
||||
---
|
||||
This is no longer a `uni-wizard` dashboard.
|
||||
The active architecture is:
|
||||
- Timmy local workspace in `~/.timmy`
|
||||
- Hermes harness in `~/.hermes`
|
||||
- `timmy-config` as the identity and orchestration sidecar
|
||||
- Gitea as the review and coordination surface
|
||||
|
||||
## 🎯 Current Sprint Status
|
||||
## Core Jobs
|
||||
|
||||
### Open Issues by Priority
|
||||
Everything should map to one of these:
|
||||
- Heartbeat: perceive, reflect, remember, decide, act, learn
|
||||
- Harness: local models, Hermes sessions, tools, memory, training loop
|
||||
- Portal Interface: the game/world-facing layer
|
||||
|
||||
| Priority | Count | Issues |
|
||||
|----------|-------|--------|
|
||||
| P0 (Critical) | 0 | — |
|
||||
| P1 (High) | 3 | #99, #103, #94 |
|
||||
| P2 (Medium) | 8 | #101, #97, #95, #93, #92, #91, #90, #87 |
|
||||
| P3 (Low) | 6 | #86, #85, #84, #83, #72, others |
|
||||
## Current Operating Surfaces
|
||||
|
||||
### Issue #94 Epic: Grand Timmy — The Uniwizard
|
||||
### Local Paths
|
||||
|
||||
**Status:** In Progress
|
||||
**Completion:** ~40%
|
||||
- Timmy workspace: `~/.timmy`
|
||||
- Timmy config repo: `~/.timmy/timmy-config`
|
||||
- Hermes home: `~/.hermes`
|
||||
- Twitter archive workspace: `~/.timmy/twitter-archive`
|
||||
|
||||
#### Completed
|
||||
- ✅ Uni-Wizard v4 architecture (4-pass evolution)
|
||||
- ✅ Three-House separation (Timmy/Ezra/Bezalel)
|
||||
- ✅ Self-improving intelligence engine
|
||||
- ✅ Pattern database and adaptive policies
|
||||
- ✅ Hermes bridge for telemetry
|
||||
### Review Surface
|
||||
|
||||
#### In Progress
|
||||
- 🔄 Backend registry (#95)
|
||||
- 🔄 Caching layer (#103)
|
||||
- 🔄 Wizard dissolution (#99)
|
||||
- Major changes go through PRs
|
||||
- Timmy is the principal reviewer for governing and sensitive changes
|
||||
- Allegro is the review and dispatch partner for queue hygiene, routing, and tempo
|
||||
|
||||
#### Pending
|
||||
- ⏳ RAG pipeline (#93)
|
||||
- ⏳ Telemetry dashboard (#91)
|
||||
- ⏳ Auto-grading (#92)
|
||||
- ⏳ Evennia world shell (#83, #84)
|
||||
### Workflow Scripts
|
||||
|
||||
---
|
||||
- `~/.hermes/bin/ops-panel.sh`
|
||||
- `~/.hermes/bin/ops-gitea.sh`
|
||||
- `~/.hermes/bin/ops-helpers.sh`
|
||||
- `~/.hermes/bin/pipeline-freshness.sh`
|
||||
- `~/.hermes/bin/timmy-dashboard`
|
||||
|
||||
## 🏛️ House Assignments
|
||||
## Daily Health Signals
|
||||
|
||||
| House | Status | Current Work |
|
||||
|-------|--------|--------------|
|
||||
| **Timmy** | 🟢 Active | Local sovereign, reviewing PRs |
|
||||
| **Ezra** | 🟢 Active | Research on LLM routing (#101) |
|
||||
| **Bezalel** | 🟡 Standby | Awaiting implementation tasks |
|
||||
| **Allegro** | 🟢 Active | Tempo-and-dispatch, Gitea bridge |
|
||||
These are the signals that matter most:
|
||||
- Hermes gateway reachable
|
||||
- local inference surface responding
|
||||
- heartbeat ticks continuing
|
||||
- Gitea reachable
|
||||
- review queue not backing up
|
||||
- session export / DPO freshness not lagging
|
||||
- Twitter archive pipeline checkpoint advancing
|
||||
|
||||
---
|
||||
## Current Team Shape
|
||||
|
||||
## 📊 System Health
|
||||
### Direction and Review
|
||||
|
||||
### VPS Fleet Status
|
||||
- Timmy: sovereignty, architecture, release judgment
|
||||
- Allegro: dispatch, queue hygiene, Gitea bridge
|
||||
|
||||
| Host | IP | Role | Status |
|
||||
|------|-----|------|--------|
|
||||
| Allegro | 143.198.27.163 | Tempo-and-Dispatch | 🟢 Online |
|
||||
| Ezra | TBD | Archivist/Research | ⚪ Not deployed |
|
||||
| Bezalel | TBD | Artificer/Builder | ⚪ Not deployed |
|
||||
### Research and Memory
|
||||
|
||||
### Services
|
||||
- Perplexity: research triage, integration evaluation
|
||||
- Ezra: archival memory, RCA, onboarding doctrine
|
||||
- KimiClaw: long-context reading and synthesis
|
||||
|
||||
| Service | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Gitea | 🟢 Running | 19 open issues |
|
||||
| Hermes | 🟡 Configured | Awaiting model setup |
|
||||
| Overnight Loop | 🔴 Stopped | Issue #72 reported |
|
||||
| Uni-Wizard | 🟢 Ready | PR created |
|
||||
### Execution
|
||||
|
||||
---
|
||||
- Codex Agent: workflow hardening, cleanup, migration verification
|
||||
- Groq: fast bounded implementation
|
||||
- Manus: moderate-scope follow-through
|
||||
- Claude: hard refactors and deep implementation
|
||||
- Gemini: frontier architecture and long-range design
|
||||
- Grok: adversarial review and edge cases
|
||||
|
||||
## 🔄 Recent Activity
|
||||
## Recommended Checks
|
||||
|
||||
### Last 24 Hours
|
||||
### Start of Day
|
||||
|
||||
1. **Uni-Wizard v4 Completed** — Four-pass architecture evolution
|
||||
2. **PR Created** — feature/uni-wizard-v4-production
|
||||
3. **Allegro Lane Narrowed** — Focused on Gitea/Hermes bridge
|
||||
4. **Issue #72 Reported** — Overnight loop not running
|
||||
1. Open the review queue and unassigned queue.
|
||||
2. Check `pipeline-freshness.sh`.
|
||||
3. Check the latest heartbeat tick.
|
||||
4. Check whether archive checkpoints and DPO artifacts advanced.
|
||||
|
||||
### Pending Actions
|
||||
### Before Merging
|
||||
|
||||
1. Deploy Ezra VPS (archivist/research)
|
||||
2. Deploy Bezalel VPS (artificer/builder)
|
||||
3. Start overnight loop
|
||||
4. Configure Syncthing mesh
|
||||
5. Implement caching layer (#103)
|
||||
1. Confirm the PR is aligned with Heartbeat, Harness, or Portal.
|
||||
2. Confirm verification is real, not implied.
|
||||
3. Confirm the change does not silently cross repo boundaries.
|
||||
4. Confirm the change does not revive deprecated loop-era behavior.
|
||||
|
||||
---
|
||||
### End of Day
|
||||
|
||||
## 🎯 Recommendations
|
||||
1. Check for duplicate issues and duplicate PR momentum.
|
||||
2. Check whether Timmy is carrying routine queue work that Allegro should own.
|
||||
3. Check whether builders were given work inside their real lanes.
|
||||
|
||||
### Immediate (Next 24h)
|
||||
## Anti-Patterns
|
||||
|
||||
1. **Review Uni-Wizard v4 PR** — Ready for merge
|
||||
2. **Start Overnight Loop** — If operational approval given
|
||||
3. **Deploy Ezra VPS** — For research tasks
|
||||
Avoid:
|
||||
- treating archived dashboard-era issues as the live roadmap
|
||||
- using stale docs that assume `uni-wizard` is still the center
|
||||
- routing work by habit instead of by current lane
|
||||
- letting open loops multiply faster than they are reviewed
|
||||
|
||||
### Short-term (This Week)
|
||||
## Success Condition
|
||||
|
||||
1. Implement caching layer (#103) — High impact
|
||||
2. Build backend registry (#95) — Enables routing
|
||||
3. Create telemetry dashboard (#91) — Visibility
|
||||
|
||||
### Medium-term (This Month)
|
||||
|
||||
1. Complete Grand Timmy epic (#94)
|
||||
2. Dissolve wizard identities (#99)
|
||||
3. Deploy Evennia world shell (#83, #84)
|
||||
|
||||
---
|
||||
|
||||
## 📈 Metrics
|
||||
|
||||
| Metric | Current | Target |
|
||||
|--------|---------|--------|
|
||||
| Issues Open | 19 | < 10 |
|
||||
| PRs Open | 1 | — |
|
||||
| VPS Online | 1/3 | 3/3 |
|
||||
| Loop Cycles | 0 | 100/day |
|
||||
|
||||
---
|
||||
|
||||
*Dashboard updated: March 30, 2026*
|
||||
*Next update: March 31, 2026*
|
||||
The system is healthy when:
|
||||
- work is routed cleanly
|
||||
- review is keeping pace
|
||||
- private learning loops are producing artifacts
|
||||
- Timmy is spending time on sovereignty and judgment rather than queue untangling
|
||||
|
||||
@@ -1,220 +1,95 @@
|
||||
# Uni-Wizard v4 — Quick Reference
|
||||
# Timmy Workflow Quick Reference
|
||||
|
||||
## Installation
|
||||
Updated: April 4, 2026
|
||||
|
||||
## What Lives Where
|
||||
|
||||
- `~/.timmy`: Timmy's workspace, lived data, heartbeat, archive artifacts
|
||||
- `~/.timmy/timmy-config`: Timmy's identity and orchestration sidecar repo
|
||||
- `~/.hermes`: Hermes harness, sessions, config overlay, helper scripts
|
||||
|
||||
## Most Useful Commands
|
||||
|
||||
### Workflow Status
|
||||
|
||||
```bash
|
||||
# Run setup script
|
||||
sudo ./scripts/setup-uni-wizard.sh
|
||||
|
||||
# Or manual install
|
||||
cd uni-wizard/v4
|
||||
pip install -e .
|
||||
~/.hermes/bin/ops-panel.sh
|
||||
~/.hermes/bin/ops-gitea.sh
|
||||
~/.hermes/bin/timmy-dashboard
|
||||
```
|
||||
|
||||
## Basic Usage
|
||||
|
||||
```python
|
||||
from uni_wizard import Harness, House, Mode
|
||||
|
||||
# Create harness
|
||||
harness = Harness(house=House.TIMMY, mode=Mode.INTELLIGENT)
|
||||
|
||||
# Execute tool
|
||||
result = harness.execute("git_status", repo_path="/path/to/repo")
|
||||
|
||||
# Check prediction
|
||||
print(f"Predicted success: {result.provenance.prediction:.0%}")
|
||||
|
||||
# Get result
|
||||
if result.success:
|
||||
print(result.data)
|
||||
else:
|
||||
print(f"Error: {result.error}")
|
||||
```
|
||||
|
||||
## Command Line
|
||||
### Workflow Helpers
|
||||
|
||||
```bash
|
||||
# Simple execution
|
||||
uni-wizard execute git_status --repo-path /path
|
||||
|
||||
# With specific house
|
||||
uni-wizard execute git_status --house ezra --mode intelligent
|
||||
|
||||
# Batch execution
|
||||
uni-wizard batch tasks.json
|
||||
|
||||
# Check health
|
||||
uni-wizard health
|
||||
|
||||
# View stats
|
||||
uni-wizard stats
|
||||
source ~/.hermes/bin/ops-helpers.sh
|
||||
ops-help
|
||||
ops-review-queue
|
||||
ops-unassigned all
|
||||
ops-queue codex-agent all
|
||||
```
|
||||
|
||||
## Houses
|
||||
|
||||
| House | Role | Best For |
|
||||
|-------|------|----------|
|
||||
| `House.TIMMY` | Sovereign | Final decisions, critical ops |
|
||||
| `House.EZRA` | Archivist | Reading, analysis, documentation |
|
||||
| `House.BEZALEL` | Artificer | Building, testing, implementation |
|
||||
| `House.ALLEGRO` | Dispatch | Routing, connectivity, tempo |
|
||||
|
||||
## Modes
|
||||
|
||||
| Mode | Use When | Features |
|
||||
|------|----------|----------|
|
||||
| `Mode.SIMPLE` | Scripts, quick tasks | Direct execution, no overhead |
|
||||
| `Mode.INTELLIGENT` | Production work | Predictions, learning, adaptation |
|
||||
| `Mode.SOVEREIGN` | Critical decisions | Full provenance, approval gates |
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Check System Status
|
||||
```python
|
||||
result = harness.execute("system_info")
|
||||
print(result.data)
|
||||
```
|
||||
|
||||
### Git Operations
|
||||
```python
|
||||
# Status
|
||||
result = harness.execute("git_status", repo_path="/path")
|
||||
|
||||
# Log
|
||||
result = harness.execute("git_log", repo_path="/path", max_count=10)
|
||||
|
||||
# Pull
|
||||
result = harness.execute("git_pull", repo_path="/path")
|
||||
```
|
||||
|
||||
### Health Check
|
||||
```python
|
||||
result = harness.execute("health_check")
|
||||
print(f"Status: {result.data['status']}")
|
||||
```
|
||||
|
||||
### Batch Operations
|
||||
```python
|
||||
tasks = [
|
||||
{"tool": "git_status", "params": {"repo_path": "/path1"}},
|
||||
{"tool": "git_status", "params": {"repo_path": "/path2"}},
|
||||
{"tool": "system_info", "params": {}}
|
||||
]
|
||||
results = harness.execute_batch(tasks)
|
||||
```
|
||||
|
||||
## Service Management
|
||||
### Pipeline Freshness
|
||||
|
||||
```bash
|
||||
# Start services
|
||||
sudo systemctl start uni-wizard
|
||||
sudo systemctl start timmy-health
|
||||
sudo systemctl start timmy-task-router
|
||||
|
||||
# Check status
|
||||
sudo systemctl status uni-wizard
|
||||
|
||||
# View logs
|
||||
sudo journalctl -u uni-wizard -f
|
||||
tail -f /opt/timmy/logs/uni-wizard.log
|
||||
|
||||
# Restart
|
||||
sudo systemctl restart uni-wizard
|
||||
~/.hermes/bin/pipeline-freshness.sh
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
### Archive Pipeline
|
||||
|
||||
### Service Won't Start
|
||||
```bash
|
||||
# Check logs
|
||||
journalctl -u uni-wizard -n 50
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Verify config
|
||||
cat /opt/timmy/config/uni-wizard.yaml
|
||||
|
||||
# Test manually
|
||||
python -m uni_wizard health
|
||||
sys.path.insert(0, str(Path.home() / '.timmy' / 'timmy-config'))
|
||||
from tasks import _archive_pipeline_health_impl
|
||||
print(json.dumps(_archive_pipeline_health_impl(), indent=2))
|
||||
PY
|
||||
```
|
||||
|
||||
### No Predictions
|
||||
- Check pattern database exists: `ls /opt/timmy/data/patterns.db`
|
||||
- Verify learning is enabled in config
|
||||
- Run a few tasks to build patterns
|
||||
```bash
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
### Gitea Integration Failing
|
||||
- Verify API token in config
|
||||
- Check Gitea URL is accessible
|
||||
- Test: `curl http://143.198.27.163:3000/api/v1/version`
|
||||
|
||||
## Configuration
|
||||
|
||||
Location: `/opt/timmy/config/uni-wizard.yaml`
|
||||
|
||||
```yaml
|
||||
house: timmy
|
||||
mode: intelligent
|
||||
enable_learning: true
|
||||
|
||||
pattern_db: /opt/timmy/data/patterns.db
|
||||
log_level: INFO
|
||||
|
||||
gitea:
|
||||
url: http://143.198.27.163:3000
|
||||
token: YOUR_TOKEN_HERE
|
||||
poll_interval: 300
|
||||
|
||||
hermes:
|
||||
stream_enabled: true
|
||||
db_path: /root/.hermes/state.db
|
||||
sys.path.insert(0, str(Path.home() / '.timmy' / 'timmy-config'))
|
||||
from tasks import _know_thy_father_impl
|
||||
print(json.dumps(_know_thy_father_impl(), indent=2))
|
||||
PY
|
||||
```
|
||||
|
||||
## API Reference
|
||||
### Manual Dispatch Prompt
|
||||
|
||||
### Harness Methods
|
||||
|
||||
```python
|
||||
# Execute single tool
|
||||
harness.execute(tool_name, **params) -> ExecutionResult
|
||||
|
||||
# Execute async
|
||||
await harness.execute_async(tool_name, **params) -> ExecutionResult
|
||||
|
||||
# Execute batch
|
||||
harness.execute_batch(tasks) -> List[ExecutionResult]
|
||||
|
||||
# Get prediction
|
||||
harness.predict(tool_name, params) -> Prediction
|
||||
|
||||
# Get stats
|
||||
harness.get_stats() -> Dict
|
||||
|
||||
# Get patterns
|
||||
harness.get_patterns() -> Dict
|
||||
```bash
|
||||
~/.hermes/bin/agent-dispatch.sh groq 542 Timmy_Foundation/the-nexus
|
||||
```
|
||||
|
||||
### ExecutionResult Fields
|
||||
## Best Files to Check
|
||||
|
||||
```python
|
||||
result.success # bool
|
||||
result.data # Any
|
||||
result.error # Optional[str]
|
||||
result.provenance # Provenance
|
||||
result.suggestions # List[str]
|
||||
```
|
||||
### Operational State
|
||||
|
||||
### Provenance Fields
|
||||
- `~/.timmy/heartbeat/last_tick.json`
|
||||
- `~/.hermes/model_health.json`
|
||||
- `~/.timmy/twitter-archive/checkpoint.json`
|
||||
- `~/.timmy/twitter-archive/metrics/progress.json`
|
||||
|
||||
```python
|
||||
provenance.house # str
|
||||
provenance.tool # str
|
||||
provenance.mode # str
|
||||
provenance.prediction # float
|
||||
provenance.execution_time_ms # float
|
||||
provenance.input_hash # str
|
||||
provenance.output_hash # str
|
||||
```
|
||||
### Archive Feedback
|
||||
|
||||
---
|
||||
- `~/.timmy/twitter-archive/notes/`
|
||||
- `~/.timmy/twitter-archive/knowledge/profile.json`
|
||||
- `~/.timmy/twitter-archive/training/dpo/`
|
||||
|
||||
*For full documentation, see ARCHITECTURE.md*
|
||||
### Review and Queue
|
||||
|
||||
- Gitea PR queue
|
||||
- Gitea unassigned issues
|
||||
- Timmy/Allegro assigned review queue
|
||||
|
||||
## Rules of Thumb
|
||||
|
||||
- If it changes identity or orchestration, review it carefully in `timmy-config`.
|
||||
- If it changes lived outputs or training inputs, it probably belongs in `timmy-home`.
|
||||
- If it only “sounds right” but is not proven by runtime state, it is not verified.
|
||||
- If a change is major, package it as a PR for Timmy review.
|
||||
|
||||
@@ -1,125 +1,71 @@
|
||||
# Scorecard Generator Documentation
|
||||
# Workflow Scorecard
|
||||
|
||||
## Overview
|
||||
Updated: April 4, 2026
|
||||
|
||||
The Scorecard Generator analyzes overnight loop JSONL data and produces comprehensive reports with statistics, trends, and recommendations.
|
||||
The old overnight `uni-wizard` scorecard is no longer the primary operational metric.
|
||||
The current scorecard should measure whether Timmy's real workflow is healthy.
|
||||
|
||||
## Usage
|
||||
## What To Score
|
||||
|
||||
### Basic Usage
|
||||
### Queue Health
|
||||
|
||||
```bash
|
||||
# Generate scorecard from default input directory
|
||||
python uni-wizard/scripts/generate_scorecard.py
|
||||
- unassigned issue count
|
||||
- PRs waiting on Timmy or Allegro review
|
||||
- overloaded assignees
|
||||
- duplicate issue / duplicate PR pressure
|
||||
|
||||
# Specify custom input/output directories
|
||||
python uni-wizard/scripts/generate_scorecard.py \
|
||||
--input ~/shared/overnight-loop \
|
||||
--output ~/timmy/reports
|
||||
```
|
||||
### Runtime Health
|
||||
|
||||
### Cron Setup
|
||||
- Hermes gateway reachable
|
||||
- local provider responding
|
||||
- latest heartbeat tick present
|
||||
- model health reporting accurately
|
||||
|
||||
```bash
|
||||
# Generate scorecard every morning at 6 AM
|
||||
0 6 * * * /root/timmy/venv/bin/python /root/timmy/uni-wizard/scripts/generate_scorecard.py
|
||||
```
|
||||
### Learning Loop Health
|
||||
|
||||
## Input Format
|
||||
- archive checkpoint advancing
|
||||
- notes and knowledge artifacts being emitted
|
||||
- DPO files growing
|
||||
- freshness lag between sessions and exports
|
||||
|
||||
JSONL files in `~/shared/overnight-loop/*.jsonl`:
|
||||
## Suggested Daily Questions
|
||||
|
||||
```json
|
||||
{"task": "read-soul", "status": "pass", "duration_s": 19.7, "timestamp": "2026-03-29T21:54:12Z"}
|
||||
{"task": "check-health", "status": "fail", "duration_s": 5.2, "error": "timeout", "timestamp": "2026-03-29T22:15:33Z"}
|
||||
```
|
||||
1. Did review keep pace with execution today?
|
||||
2. Did any builder receive work outside their lane?
|
||||
3. Did Timmy spend time on judgment rather than routine queue cleanup?
|
||||
4. Did the private learning pipeline produce usable artifacts?
|
||||
5. Did any stale doc, helper, or default try to pull the system back into old habits?
|
||||
|
||||
Fields:
|
||||
- `task`: Task identifier
|
||||
- `status`: "pass" or "fail"
|
||||
- `duration_s`: Execution time in seconds
|
||||
- `timestamp`: ISO 8601 timestamp
|
||||
- `error`: Error message (for failed tasks)
|
||||
## Useful Inputs
|
||||
|
||||
## Output
|
||||
- `~/.timmy/heartbeat/ticks_YYYYMMDD.jsonl`
|
||||
- `~/.timmy/metrics/local_YYYYMMDD.jsonl`
|
||||
- `~/.timmy/twitter-archive/checkpoint.json`
|
||||
- `~/.timmy/twitter-archive/metrics/progress.json`
|
||||
- Gitea open PR queue
|
||||
- Gitea unassigned issue queue
|
||||
|
||||
### JSON Report
|
||||
## Suggested Ratings
|
||||
|
||||
`~/timmy/reports/scorecard_YYYYMMDD.json`:
|
||||
### Queue Discipline
|
||||
|
||||
```json
|
||||
{
|
||||
"generated_at": "2026-03-30T06:00:00Z",
|
||||
"summary": {
|
||||
"total_tasks": 100,
|
||||
"passed": 95,
|
||||
"failed": 5,
|
||||
"pass_rate": 95.0,
|
||||
"duration_stats": {
|
||||
"avg": 12.5,
|
||||
"median": 10.2,
|
||||
"p95": 45.0,
|
||||
"min": 1.2,
|
||||
"max": 120.5
|
||||
}
|
||||
},
|
||||
"by_task": {...},
|
||||
"by_hour": {...},
|
||||
"errors": {...},
|
||||
"recommendations": [...]
|
||||
}
|
||||
```
|
||||
- Strong: review and dispatch are keeping up, little duplicate churn
|
||||
- Mixed: queue moves, but ambiguity or duplication is increasing
|
||||
- Weak: review is backlogged or agents are being misrouted
|
||||
|
||||
### Markdown Report
|
||||
### Runtime Reliability
|
||||
|
||||
`~/timmy/reports/scorecard_YYYYMMDD.md`:
|
||||
- Strong: heartbeat, Hermes, and provider surfaces all healthy
|
||||
- Mixed: intermittent downtime or weak health signals
|
||||
- Weak: major surfaces untrusted or stale
|
||||
|
||||
- Executive summary with pass/fail counts
|
||||
- Duration statistics (avg, median, p95)
|
||||
- Per-task breakdown with pass rates
|
||||
- Hourly timeline showing performance trends
|
||||
- Error analysis with frequency counts
|
||||
- Actionable recommendations
|
||||
### Learning Throughput
|
||||
|
||||
## Report Interpretation
|
||||
- Strong: checkpoint advances, DPO output accumulates, eval gates are visible
|
||||
- Mixed: some artifacts land, but freshness or checkpointing lags
|
||||
- Weak: sessions occur without export, or learning artifacts stall
|
||||
|
||||
### Pass Rate Thresholds
|
||||
## The Goal
|
||||
|
||||
| Pass Rate | Status | Action |
|
||||
|-----------|--------|--------|
|
||||
| 95%+ | ✅ Excellent | Continue current operations |
|
||||
| 85-94% | ⚠️ Good | Monitor for degradation |
|
||||
| 70-84% | ⚠️ Fair | Review failing tasks |
|
||||
| <70% | ❌ Poor | Immediate investigation required |
|
||||
|
||||
### Duration Guidelines
|
||||
|
||||
| Duration | Assessment |
|
||||
|----------|------------|
|
||||
| <5s | Fast |
|
||||
| 5-15s | Normal |
|
||||
| 15-30s | Slow |
|
||||
| >30s | Very slow - consider optimization |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No JSONL files found
|
||||
|
||||
```bash
|
||||
# Check input directory
|
||||
ls -la ~/shared/overnight-loop/
|
||||
|
||||
# Ensure Syncthing is syncing
|
||||
systemctl status syncthing@root
|
||||
```
|
||||
|
||||
### Malformed lines
|
||||
|
||||
The generator skips malformed lines with a warning. Check the JSONL files for syntax errors.
|
||||
|
||||
### Empty reports
|
||||
|
||||
If no data exists, verify:
|
||||
1. Overnight loop is running and writing JSONL
|
||||
2. File permissions allow reading
|
||||
3. Input path is correct
|
||||
The point of the scorecard is not to admire activity.
|
||||
The point is to tell whether the system is becoming more reviewable, more sovereign, and more capable of learning from lived work.
|
||||
|
||||
Reference in New Issue
Block a user