docs: add timmy-config genome analysis (refs #669)

GENOME.md

@@ -0,0 +1,304 @@
+# GENOME.md — timmy-config
+
+Auto-generated facts were derived from the local `~/.timmy/timmy-config` checkout on 2026-04-14 and then reviewed manually for architecture, API surface, and operational meaning.
+
+## Project Overview
+
+`timmy-config` is the sovereign sidecar for Timmy.
+It is not the model runtime and it is not the work archive. It is the configuration and orchestration layer that tells Hermes who Timmy is, how he is routed, what scripts are available, what memories and skins are installed, and how the fleet is operated.
+
+The repo exists because the Timmy Foundation made a hard architectural split:
+- `hermes-agent` is the engine
+- `timmy-config` is the driver's seat
+- `timmy-home` is the lived workspace and artifact archive
+
+This repo is therefore a mixed codebase with three major responsibilities:
+1. Sidecar deployment into `~/.hermes/`
+2. Operational automation for the fleet
+3. Thin training/orchestration support without becoming a monolith again
+
+Observed quick facts from the current local checkout:
+- Source files: 178
+- Test files: 36
+- Config files: 24
+- Total lines: 35,624
+- Last commit on analyzed source: `7630806f` (`sync: align repo with live system config`)
+- Total commits: 393
+
+## Architecture Diagram
+
+```mermaid
+flowchart TD
+    A[timmy-config repo] --> B[deploy.sh]
+    B --> C[~/.hermes/config.yaml]
+    B --> D[~/.hermes/bin/*]
+    B --> E[~/.hermes/playbooks/*]
+    B --> F[~/.hermes/cron/*]
+    B --> G[~/.hermes/memories/*]
+    B --> H[~/.timmy/SOUL.md]
+
+    C --> I[Hermes gateway/runtime]
+    D --> J[Operational scripts]
+    E --> K[Playbook-driven agent behavior]
+    F --> L[Cron-scheduled automation]
+
+    J --> M[Gitea forge]
+    J --> N[Telegram / platform gateways]
+    J --> O[tmux wizard fleet]
+    J --> P[metrics + local files]
+
+    Q[orchestration.py + tasks.py] --> R[Huey / SQLite queue]
+    R --> J
+    Q --> M
+    Q --> P
+
+    S[gitea_client.py] --> M
+    T[training/*] --> U[curated datasets + evaluation recipes]
+    U --> P
+
+    V[timmy-home] <-->|artifacts / logs / training outputs| J
+    V <-->|continuity / notes / metrics| Q
+```
+
+## Entry Points and Data Flow
+
+### Primary entry points
+
+1. `deploy.sh`
+   - canonical sidecar deployment entry point
+   - copies config, scripts, playbooks, cron definitions, skins, and memories into `~/.hermes/`
+   - writes `SOUL.md` into `~/.timmy/`
+
+2. `orchestration.py`
+   - defines the Huey SQLite queue
+   - the root scheduler primitive for queued work
+
+3. `tasks.py`
+   - the real orchestration surface
+   - contains scheduled jobs, local Hermes invocations, archive/training helpers, JSONL helpers, continuity flushing, and repo lists
+
+4. `gitea_client.py`
+   - typed stdlib-only interface to Gitea
+   - shared API layer replacing scattered raw curl logic
+
+5. `bin/*`
+   - operational executables for health checks, deadman switch, dispatch, watchdogs, scans, and status panels
+   - this is the hands-on operator layer
+
+### Data flow summary
+
+- Configuration starts in repo files like `config.yaml`, `fallback-portfolios.yaml`, `channel_directory.json`, `memories/`, `skins/`, `playbooks/`, and `cron/`
+- `deploy.sh` overlays these into the Hermes runtime directory
+- Hermes runtime reads the deployed config and scripts
+- `tasks.py` and `bin/*` then interact with:
+  - Gitea (`gitea_client.py`, issue/PR automation)
+  - local Hermes sessions (`run_hermes_local` paths in `tasks.py`)
+  - local files in `~/.timmy/` and `~/.hermes/`
+  - metrics JSONL outputs
+  - Telegram / gateway surfaces
+
+This is a classic sidecar pattern: the repo does not own the engine, but it owns almost every operational decision around the engine.
+
+## Key Abstractions
+
+### 1. Sidecar overlay
+Core idea: never fork Hermes if a sidecar can express the behavior.
+This abstraction appears in:
+- `deploy.sh`
+- repo layout under `bin/`, `cron/`, `playbooks/`, `memories/`, `skins/`
+- the explicit README boundary between `timmy-config` and `timmy-home`
+
+### 2. Typed forge client
+`gitea_client.py` turns the forge into a stable internal API surface.
+Important abstractions:
+- `Issue`
+- `PullRequest`
+- `Comment`
+- `Label`
+- `User`
+- `GiteaClient`
+
+This is important because many other scripts can depend on one client instead of shelling out to brittle curl commands.
+
+### 3. Queue-backed orchestration
+`orchestration.py` + `tasks.py` define the move from ad-hoc shell automation to queued work.
+The central abstraction is not just “a cron job” but “a schedulable task with local model execution, continuity, and metrics.”
+
+### 4. Continuity as files, not vibes
+`tasks.py` contains explicit file-backed continuity helpers:
+- `flush_continuity(...)`
+- JSON readers/writers
+- JSONL append/load helpers
+- archive checkpoint/state files
+
+This abstraction matters because Timmy continuity survives compaction or restart by being written to disk.
+
+### 5. Training as thin recipes
+The training directory is intentionally framed as transitional.
+It exposes recipes and helper scripts, but the README is explicit that lived data belongs elsewhere.
+This is an important abstraction boundary:
+- configs + generators here
+- real activity artifacts in `timmy-home`
+
+## API Surface
+
+### Shell / operator API
+
+Important user-facing commands implied by the repo:
+- `./deploy.sh`
+- Huey consumer startup via `huey_consumer.py tasks.huey -w 2 -k thread`
+- scripts in `bin/` such as:
+  - `deadman-switch.sh`
+  - `fleet-status.sh`
+  - `model-health-check.sh`
+  - `start-loops.sh`
+  - `agent-dispatch.sh`
+
+### Python API surface
+
+Most reusable programmatic interfaces:
+
+#### `gitea_client.GiteaClient`
+Key methods include:
+- `list_org_repos`
+- `list_issues`
+- `get_issue`
+- `create_issue`
+- `update_issue`
+- `close_issue`
+- `assign_issue`
+- `add_labels`
+- `list_comments`
+- `create_comment`
+- `list_pulls`
+- `get_pull`
+- `create_pull`
+- `merge_pull`
+- `update_pull_branch`
+- `close_pull`
+
+#### `tasks.py` helpers
+Notable reusable surfaces:
+- `run_hermes_local(...)`
+- `hermes_local(...)`
+- `run_reflex_task(...)`
+- `run_archive_hermes(...)`
+- `flush_continuity(...)`
+- JSON/JSONL primitives:
+  - `read_json`
+  - `write_json`
+  - `load_jsonl`
+  - `write_jsonl`
+  - `append_jsonl`
+  - `count_jsonl_rows`
+
+This API surface is broad enough that timmy-config is functionally an operations SDK as much as a config repo.
+
+## Test Coverage Gaps
+
+The codebase-genome pipeline estimated:
+- source modules: 86
+- test modules: 36
+- estimated coverage: 15%
+- untested modules: 73
+
+That estimate is crude, but directionally useful.
+
+### High-value gaps
+
+The most important untested or under-tested areas are not random scripts. They are the system boundary scripts and operator-critical surfaces:
+
+1. `deploy.sh`
+   - highest leverage file in the repo
+   - if deploy breaks, the sidecar stops being the source of truth in practice
+
+2. `orchestration.py` and large portions of `tasks.py`
+   - especially scheduling, local-Hermes execution wrappers, checkpoint/state flows, and failure modes
+
+3. `gitea_client.py`
+   - some behavior may be covered indirectly, but the client is important enough to deserve deeper contract tests for pagination, merge failures, 405 handling, and retry behavior
+
+4. `bin/*` operational scripts
+   - deadman, watchdog, model health, and dispatch paths are safety critical
+
+5. Ansible deployment surface
+   - `ansible/playbooks/*` and `ansible/roles/*` are configuration-heavy but still represent production behavior
+
+### Practical takeaway
+
+If the next wave of testing work is prioritized, the order should be:
+1. deploy + runtime overlay correctness
+2. Gitea client contracts
+3. queue/task execution behavior in `tasks.py`
+4. deadman / watchdog / fleet-health scripts
+5. training and archive helper edge cases
+
+## Security Considerations
+
+The pipeline already flags several categories, and manual review supports them.
+
+### 1. Subprocess-heavy code
+This repo uses shell and subprocess execution widely.
+That is expected for an ops repo, but it increases command-injection risk.
+Files like `tasks.py`, deployment helpers, and many `bin/*` scripts need careful boundary checking around interpolated inputs.
+
+### 2. Secret adjacency
+This repo references tokens, auth files, routing configs, and platform integrations.
+Even when secrets are not hardcoded, the repo lives close to sensitive paths.
+That means review discipline matters:
+- no accidental dumps of live tokens
+- no committing generated auth artifacts
+- no relaxed assumptions about local paths being safe to expose
+
+### 3. HTTP and webhook surfaces
+Multiple scripts make outbound HTTP calls or serve automation endpoints.
+This means input validation, response validation, and timeout/error handling matter more than in a static config repo.
+
+### 4. SQLite and file-backed state
+SQLite/Huey plus JSON/JSONL file state are simple and sovereign, but they create corruption and stale-state risks if writes are partial or multiple writers race.
+The repo already encodes a strong preference for explicit checkpoints and continuity flushes, which is the right direction.
+
+### 5. Sidecar privilege
+Because `deploy.sh` writes directly into `~/.hermes/`, this repo effectively has configuration authority over the live runtime.
+That is powerful and dangerous. A bad deploy can break routing, scripts, or identity. In security terms, this repo is a control plane.
+
+## Performance Characteristics
+
+`timmy-config` is not performance-sensitive in the same way a game loop or serving stack is. Its performance profile is operational:
+
+- Many small scripts, low-latency startup expectations
+- File-backed state and JSONL append patterns optimized for simplicity over throughput
+- Huey + SQLite chosen for low operational overhead, not horizontal scale
+- Deployment is copy-based and cheap; correctness matters more than speed
+- The repo is large in surface area but shallow in runtime residency — most code runs only when called
+
+Practical performance characteristics:
+- deploy path should remain fast because it is mostly file copies
+- Gitea automation cost is network-bound, not CPU-bound
+- training helpers are the heaviest local operations but intentionally limited in scope
+- orchestration latency is dominated by external model/API calls rather than Python compute
+
+The biggest performance risk is not CPU.
+It is operational drift: too many scripts, overlapping logic, or old paths that still appear to exist. That increases human debugging cost even when machine runtime is fine.
+
+## Final Assessment
+
+`timmy-config` is best understood as a sovereign control plane.
+It is part config repo, part operations toolkit, part lightweight SDK, and part deployment overlay.
+
+Its strongest architectural idea is the boundary:
+- keep Hermes upstream
+- keep Timmy sovereign through the sidecar
+- keep lived work in timmy-home
+
+Its biggest risk is sprawl.
+A repo with 178 source files and 393 commits can easily become the place where every operational idea goes to live forever.
+The right long-term direction is not to make it smaller by deleting its authority, but to keep sharpening the boundaries inside it:
+- sidecar deploy surface
+- queue/orchestration surface
+- Gitea client surface
+- training recipe surface
+- operator script surface
+
+That keeps the genome legible.
+Without that, the repo becomes powerful but opaque — and a control plane cannot afford opacity.

genomes/fleet-ops-GENOME.md

@@ -1,185 +0,0 @@
-# GENOME.md: fleet-ops
-
-**Generated:** 2026-04-14
-**Repo:** Timmy_Foundation/fleet-ops
-**Purpose:** Sovereign fleet operations -- Ansible playbooks, monitoring, dispatch, infrastructure-as-code
-**Size:** 284 files | 14 Ansible roles | 15 Python scripts | 11 test files
-
----
-
-## Project Overview
-
-fleet-ops is the infrastructure-as-code repository for the Timmy Foundation's sovereign wizard fleet. It manages three VPS-based AI agent workers (Bezalel, Ezra, Allegro) through Ansible playbooks, Docker Compose, tmux dispatch, and automated monitoring.
-
-The fleet runs 50+ AI agent sessions simultaneously, dispatches work through Gitea issues, and monitors health through cron-based watchdogs. fleet-ops is the control plane.
-
-## Architecture
-
-```mermaid
-graph TD
-    A[Local Machine] -->|tmux dispatch| B[BURN Session: 50+ panes]
-    A -->|Ansible| C[VPS: Bezalel]
-    A -->|Ansible| D[VPS: Ezra]
-    A -->|Ansible| E[VPS: Allegro]
-    A -->|Gitea API| F[Forge: Gitea]
-
-    C -->|hermes-agent| G[Agent Worker]
-    D -->|hermes-agent| G
-    E -->|hermes-agent| G
-
-    H[Cron Jobs] --> I[Monitoring]
-    H --> J[Burndown]
-    H --> K[Dispatch Consumer]
-    H --> L[Nightly Efficiency]
-
-    I -->|deadman switch| M[Alert Manager]
-    I -->|health check| N[Telegram Alerts]
-
-    J -->|scan issues| F
-    K -->|consume issues| B
-    L -->|token stats| O[Reports]
-```
-
-## Entry Points
-
-| Entry Point | Type | Purpose |
-|-------------|------|---------|
-| `playbooks/site.yml` | Ansible | Master playbook -- deploys entire fleet |
-| `playbooks/provision_and_deploy.yml` | Ansible | Full VPS provisioning + service deploy |
-| `playbooks/deploy_hermes.yml` | Ansible | Deploy hermes-agent to wizard VPSes |
-| `playbooks/deploy_ollama.yml` | Ansible | Deploy Ollama inference server |
-| `playbooks/deploy_gitea.yml` | Ansible | Deploy Gitea forge |
-| `docker-compose.yml` | Docker | Local multi-service stack (ollama, gitea, agent, monitor) |
-| `scripts/dispatch_consumer.py` | Python | Consume Gitea issues and dispatch to tmux panes |
-| `scripts/burndown_watcher.py` | Python | Monitor backlog velocity across repos |
-| `scripts/fleet-status.py` | Python | One-command fleet health report |
-| `scripts/tmux-dispatch.sh` | Shell | Route work to tmux pane windows |
-
-## Data Flow
-
-```
-Gitea Issue Created
-  |
-  v
-dispatch_consumer.py (cron: 5m)
-  |
-  v
-tmux-dispatch.sh -> Assign to pane window (CRUCIBLE/GNOMES/FOUNDRY)
-  |
-  v
-hermes-agent in tmux pane (agent worker)
-  |
-  v
-Agent creates branch -> commits -> pushes -> opens PR
-  |
-  v
-auto_merge.sh (cron: 10m) -> Safe PRs merge automatically
-  |
-  v
-nightly_efficiency_report.py -> Token usage, cost, throughput stats
-```
-
-## Key Abstractions
-
-| Abstraction | Description |
-|-------------|-------------|
-| **Wizard** | A VPS running hermes-agent. Three active: Bezalel, Ezra, Allegro. |
-| **Fleet** | The collective of all wizards + local orchestration. |
-| **Burn** | High-throughput execution mode. 50+ agents working in parallel. |
-| **Dispatch** | Routing Gitea issues to tmux panes for agent processing. |
-| **Deadman** | Watchdog that alerts when heartbeats stop. |
-| **Burndown** | Tracking backlog velocity. Issues closed vs created per day. |
-| **Sovereignty** | No cloud dependency for core operations. Local inference preferred. |
-
-## Ansible Roles
-
-| Role | Purpose | Key Files |
-|------|---------|-----------|
-| `common` | Base OS config, packages, users | tasks/main.yml |
-| `hermes-agent` | Deploy agent service, config, env | templates/config.yaml.j2, hermes.service.j2 |
-| `ollama` | Deploy Ollama inference server | templates/ollama.service.j2 |
-| `gitea` | Deploy Gitea forge (Docker) | templates/docker-compose.yml.j2 |
-| `nginx` | Reverse proxy for all services | templates/site.conf.j2 |
-| `backups` | Automated backups for gitea, evennia | templates/backup.cron.j2 |
-| `monitoring` | Health checks, deadman switch | templates/deadman-switch.sh.j2 |
-| `auto-merge` | PR auto-merge for safe changes | files/scripts/auto_merge.sh |
-| `conduit` | Matrix homeserver (Conduit) | templates/conduit.toml.j2 |
-| `nostr-relay` | Nostr relay for sovereign comms | templates/strfry.conf.j2 |
-| `docker` | Docker installation and config | tasks/main.yml |
-| `evennia` | MUD world server (The Tower) | templates/settings.py.j2 |
-| `message-bus` | Inter-agent message bus | templates/busd.service.j2 |
-| `knowledge-store` | Persistent knowledge store | templates/knowledged.service.j2 |
-
-## Inventory
-
-| Host | Wizard | Role | Model |
-|------|--------|------|-------|
-| bezal | Bezalel | Agent worker | gemma-4-31b-it |
-| hermes-vps | Ezra | Agent worker | gemma-4-31b-it |
-| allegro-vps | Allegro | Agent worker | gemma-4-31b-it |
-| gitea-forge | -- | Gitea, registry | -- |
-
-## Cron Jobs
-
-| Job | Schedule | Script | Purpose |
-|-----|----------|--------|---------|
-| dispatch-consumer | 5m | scripts/dispatch_consumer.py | Route issues to agents |
-| burndown-watcher | 15m | scripts/burndown_watcher.py | Track backlog velocity |
-| nightly-efficiency | Daily 03:00 | scripts/nightly_efficiency_report.py | Token/cost report |
-| auto-merge | 10m | scripts/auto_merge.sh | Merge safe PRs |
-| morning-report | Daily 07:00 | scripts/morning_report_compile.py | Fleet status digest |
-| sovereign-guard | 5m | sovereign_sentinel.py | Security monitoring |
-| sovereign-pulse | 5m | sovereign_pulse.py | Health heartbeat |
-
-## Test Coverage
-
-| Test File | Tests | Area |
-|-----------|-------|------|
-| test_dispatch_consumer.py | Y | Issue dispatch routing |
-| test_health_dashboard.py | Y | Health check aggregation |
-| test_knowledge_store.py | Y | Knowledge persistence |
-| test_message_bus.py | Y | Inter-agent messaging |
-| test_nightly_efficiency_report.py | Y | Token/cost calculation |
-| test_profile_isolation.py | Y | Agent profile separation |
-| test_skill_scorer.py | Y | Skill quality scoring |
-| test_synthesis.py | Y | Synthesis engine |
-| test_video_engine_client.py | Y | Video generation client |
-| test_federation_sync.py | Y | Cross-wizard state sync |
-| test_heart.py | Y | Heart/compassion layer |
-
-### Gaps
-
-- No integration tests for full dispatch-to-merge pipeline
-- Ansible roles lack molecule tests (only lint)
-- No test for deadman switch (shell script, not Python)
-- tmux-dispatch.sh is pure shell, no test coverage
-- Docker Compose tested manually only
-
-## Security Considerations
-
-- **Vault-encrypted secrets.** API keys, tokens in `playbooks/group_vars/vault.yml` (ansible-vault)
-- **SSH key auth only.** No password auth on VPSes.
-- **Registry auth.** Private container registry at forge.alexanderwhitestone.com
-- **Nostr relay.** Sovereign comms channel, no third-party dependency.
-- **Deadman switch.** Alerts on heartbeat loss. Prevents silent fleet death.
-- **Sovereign sentinel.** Monitors for unauthorized access patterns.
-
-## Docker Services
-
-| Service | Image | Port | Purpose |
-|---------|-------|------|---------|
-| ollama | ollama/ollama:latest | 11434 | Local LLM inference |
-| gitea | gitea/gitea:latest | 3000, 2222 | Git hosting, issues |
-| agent | hermes-agent:prod | 8080 | Agent worker loop |
-| monitor | custom | internal | Health reporter |
-
-## Key Dependencies
-
-| Dependency | Type | Purpose |
-|------------|------|---------|
-| Ansible | IaC | Fleet provisioning and deployment |
-| Docker | Container | Service isolation |
-| tmux | Process | Agent session management |
-| Gitea | Forge | Issue tracking, PR workflow |
-| Ollama | Inference | Local model serving |
-| Telegram | Alerts | Human notification channel |

tests/test_timmy_config_genome.py

@@ -0,0 +1,61 @@
+from pathlib import Path
+
+GENOME = Path('GENOME.md')
+
+
+def read_genome() -> str:
+    assert GENOME.exists(), 'GENOME.md must exist at repo root'
+    return GENOME.read_text(encoding='utf-8')
+
+
+def test_genome_exists():
+    assert GENOME.exists(), 'GENOME.md must exist at repo root'
+
+
+def test_genome_has_required_sections():
+    text = read_genome()
+    for heading in [
+        '# GENOME.md — timmy-config',
+        '## Project Overview',
+        '## Architecture Diagram',
+        '## Entry Points and Data Flow',
+        '## Key Abstractions',
+        '## API Surface',
+        '## Test Coverage Gaps',
+        '## Security Considerations',
+        '## Performance Characteristics',
+    ]:
+        assert heading in text
+
+
+def test_genome_contains_mermaid_diagram():
+    text = read_genome()
+    assert '```mermaid' in text
+    assert 'graph TD' in text or 'flowchart TD' in text
+
+
+def test_genome_mentions_core_timmy_config_files():
+    text = read_genome()
+    for token in [
+        'deploy.sh',
+        'config.yaml',
+        'gitea_client.py',
+        'orchestration.py',
+        'tasks.py',
+        'bin/',
+        'playbooks/',
+        'training/',
+    ]:
+        assert token in text
+
+
+def test_genome_explains_sidecar_boundary():
+    text = read_genome()
+    assert 'sidecar' in text.lower()
+    assert 'Hermes' in text
+    assert 'timmy-home' in text
+
+
+def test_genome_is_substantial():
+    text = read_genome()
+    assert len(text) >= 5000