Compare commits

..

2 Commits

Author SHA1 Message Date
Timmy Agent
671ed86c5f FRONTIER: Add .sov (Sovereign Bundle) export/import format
Some checks failed
Self-Healing Smoke / self-healing-smoke (pull_request) Failing after 24s
Smoke Test / smoke (pull_request) Failing after 29s
Agent PR Gate / gate (pull_request) Failing after 51s
Agent PR Gate / report (pull_request) Successful in 22s
Implements timmy-home #467 — Develop "Sovereign Bundle" (.sov) Export/Import Logic

Introduces a standardized, portable ZIP-based archive format for capturing
an agent's complete state (s soul, config, keys, memories, skills, profiles,
and timmy world files). Complements existing backup_pipeline.sh with a
structured, human-inspectable representation suitable for migration and
verification.

New files:
- timmy-local/scripts/create_sov_bundle.py — Export (create .sov)
- timmy-local/scripts/restore_sov_bundle.py — Import (restore from .sov)
- scripts/sov — CLI wrapper for easy access
- tests/test_sov_bundle.py — 10 tests covering format integrity
- SKILL-sov-bundle.md — Full documentation and usage guide

Format:
  sov/
    META.json        — Environment metadata (format identifier)
    manifest.json    — Canonical index (version, components, sizes)
    soul/SOUL.md     — Identity document + values
    config/config.yaml — Agent model/toolset configuration
    keys/keymaxxing.json — Credential registry (unchanged)
    memories/
      reflections/  — Daily learned summaries (included)
      mempalace/    — Memory palace files (~500KB included)
      timmy/        — Evennia agent world files (included)
    skills/         — Custom skill scripts (included)
    profiles/       — Hermes profile configs (included)

Default exclusions (safely reproduceable):
  - sessions/      (10+ GB transcripts — opt-in via --include-sessions)
  - cache/         (derived, GPU cache)
  - checkpoints/   (runtime recovery)
  - logs/          (operational noise)
  - .git, *.pyc, __pycache__, node_modules, venv

Features:
  - SHA-256 hash embedded in manifest for integrity verification
  - Fully automated tests (pytest) — all passing
  - Dry-run, list, verify commands
  - Non-destructive restore with confirmation prompt
  - Profile-aware via HERMES_HOME (supports multiple agent homes)

Agency tools: Uses only standard library (zipfile, json, pathlib)
  — no external dependencies, sovereign by default.

Closes #467
2026-04-30 00:18:10 -04:00
1fa6c3bad1 fix(#793): Add What Honesty Requires, implement source distinction (#962)
Some checks failed
Self-Healing Smoke / self-healing-smoke (push) Failing after 25s
Smoke Test / smoke (push) Failing after 18s
Co-authored-by: Timmy Time <timmy@alexanderwhitestone.ai>
Co-committed-by: Timmy Time <timmy@alexanderwhitestone.ai>
2026-04-29 12:09:27 +00:00
13 changed files with 1377 additions and 148 deletions

299
GENOME.md
View File

@@ -1,144 +1,209 @@
# GENOME.md — Timmy_Foundation/timmy-home
Generated by `pipelines/codebase_genome.py`.
# GENOME.md — the-nexus
## Project Overview
Timmy Foundation's home repository for development operations and configurations.
`the-nexus` is a hybrid repo that combines three layers in one codebase:
- Text files indexed: 3181
- Source and script files: 231
- Test files: 95
- Documentation files: 755
1. A browser-facing world shell rooted in `index.html`, `boot.js`, `bootstrap.mjs`, `app.js`, `style.css`, `portals.json`, `vision.json`, `manifest.json`, and `gofai_worker.js`
2. A Python realtime bridge centered on `server.py` plus harness code under `nexus/`
3. A memory / fleet / operator layer spanning `mempalace/`, `mcp_servers/`, `multi_user_bridge.py`, and supporting scripts
## Architecture
The repo is not a clean single-purpose frontend and not just a backend harness. It is a mixed world/runtime/ops repository where browser rendering, WebSocket telemetry, MCP-driven game harnesses, and fleet memory tooling coexist.
Grounded repo facts from this checkout:
- Browser shell files exist at repo root: `index.html`, `app.js`, `style.css`, `manifest.json`, `gofai_worker.js`
- Data/config files also live at repo root: `portals.json`, `vision.json`
- Realtime bridge exists in `server.py`
- Game harnesses exist in `nexus/morrowind_harness.py` and `nexus/bannerlord_harness.py`
- Memory/fleet sync exists in `mempalace/tunnel_sync.py`
- Desktop/game automation MCP servers exist in `mcp_servers/desktop_control_server.py` and `mcp_servers/steam_info_server.py`
- Validation exists in `tests/test_browser_smoke.py`, `tests/test_portals_json.py`, `tests/test_index_html_integrity.py`, and `tests/test_repo_truth.py`
The current architecture is best understood as a sovereign world shell plus operator/game harness backend, with accumulated documentation drift from multiple restoration and migration efforts.
## Architecture Diagram
```mermaid
graph TD
repo_root["repo"]
angband["angband"]
ansible["ansible"]
briefings["briefings"]
codebase_genome["codebase_genome"]
config["config"]
configs["configs"]
conftest["conftest"]
dns_records["dns-records"]
evennia["evennia"]
evennia_tools["evennia_tools"]
repo_root --> angband
repo_root --> ansible
repo_root --> briefings
repo_root --> codebase_genome
repo_root --> config
repo_root --> configs
browser[Index HTML Shell\nindex.html -> boot.js -> bootstrap.mjs -> app.js]
assets[Root Assets\nstyle.css\nmanifest.json\ngofai_worker.js]
data[World Data\nportals.json\nvision.json]
ws[Realtime Bridge\nserver.py\nWebSocket broadcast hub]
gofai[In-browser GOFAI\nSymbolicEngine\nNeuroSymbolicBridge\nsetupGOFAI/updateGOFAI]
harnesses[Python Harnesses\nnexus/morrowind_harness.py\nnexus/bannerlord_harness.py]
mcp[MCP Adapters\nmcp_servers/desktop_control_server.py\nmcp_servers/steam_info_server.py]
memory[Memory + Fleet\nmempalace/tunnel_sync.py\nmempalace.js]
bridge[Operator / MUD Bridge\nmulti_user_bridge.py\ncommands/timmy_commands.py]
tests[Verification\ntests/test_browser_smoke.py\ntests/test_portals_json.py\ntests/test_repo_truth.py]
docs[Contracts + Drift Docs\nBROWSER_CONTRACT.md\nREADME.md\nCLAUDE.md\nINVESTIGATION_ISSUE_1145.md]
browser --> assets
browser --> data
browser --> gofai
browser --> ws
harnesses --> mcp
harnesses --> ws
bridge --> ws
memory --> ws
tests --> browser
tests --> data
tests --> docs
docs --> browser
```
## Entry Points
## Entry Points and Data Flow
- `codebase_genome.py` — python main guard (`python3 codebase_genome.py`)
- `gemini-fallback-setup.sh` — operational script (`bash gemini-fallback-setup.sh`)
- `morrowind/hud.sh` — operational script (`bash morrowind/hud.sh`)
- `pipelines/codebase_genome.py` — python main guard (`python3 pipelines/codebase_genome.py`)
- `scripts/agent_pr_gate.py` — operational script (`python3 scripts/agent_pr_gate.py`)
- `scripts/audit_trail.py` — operational script (`python3 scripts/audit_trail.py`)
- `scripts/auto_restart_agent.sh` — operational script (`bash scripts/auto_restart_agent.sh`)
- `scripts/autonomous_issue_creator.py` — operational script (`python3 scripts/autonomous_issue_creator.py`)
- `scripts/backlog_cleanup.py` — operational script (`python3 scripts/backlog_cleanup.py`)
- `scripts/backlog_triage.py` — operational script (`python3 scripts/backlog_triage.py`)
- `scripts/backlog_triage_cron.sh` — operational script (`bash scripts/backlog_triage_cron.sh`)
- `scripts/backup_pipeline.sh` — operational script (`bash scripts/backup_pipeline.sh`)
### Primary entry points
## Data Flow
- `index.html` — root browser entry point
- `boot.js` — startup selector; `tests/boot.test.js` shows it chooses file-mode vs HTTP/module-mode and injects `bootstrap.mjs` when served over HTTP
- `bootstrap.mjs` — module bootstrap for the browser shell
- `app.js` — main browser runtime; owns world state, GOFAI wiring, metrics polling, and portal/UI logic
- `server.py` — WebSocket broadcast bridge on `ws://0.0.0.0:8765`
- `nexus/morrowind_harness.py` — GamePortal/MCP harness for OpenMW Morrowind
- `nexus/bannerlord_harness.py` — GamePortal/MCP harness for Bannerlord
- `mempalace/tunnel_sync.py` — pulls remote fleet closets into the local palace over HTTP
- `multi_user_bridge.py` — HTTP bridge for multi-user chat/session integration
- `mcp_servers/desktop_control_server.py` — stdio MCP server exposing screenshots/mouse/keyboard control
1. Operators enter through `codebase_genome.py`, `gemini-fallback-setup.sh`, `morrowind/hud.sh`.
2. Core logic fans into top-level components: `angband`, `ansible`, `briefings`, `codebase_genome`, `config`, `configs`.
3. Validation is incomplete around `wizards/allegro/home/skills/red-teaming/godmode/scripts/auto_jailbreak.py`, `timmy-local/cache/agent_cache.py`, `wizards/allegro/home/skills/red-teaming/godmode/scripts/parseltongue.py`, so changes there carry regression risk.
4. Final artifacts land as repository files, docs, or runtime side effects depending on the selected entry point.
### Data flow
1. Browser startup begins at `index.html`
2. `boot.js` decides whether the page is being served correctly; in HTTP mode it injects `bootstrap.mjs`
3. `bootstrap.mjs` hands off to `app.js`
4. `app.js` loads world configuration from `portals.json` and `vision.json`
5. `app.js` constructs the Three.js scene and in-browser reasoning components, including `SymbolicEngine`, `NeuroSymbolicBridge`, `setupGOFAI()`, and `updateGOFAI()`
6. Browser state and external runtimes connect through `server.py`, which broadcasts messages between connected clients
7. Python harnesses (`nexus/morrowind_harness.py`, `nexus/bannerlord_harness.py`) spawn MCP subprocesses for desktop control / Steam metadata, capture state, execute actions, and feed telemetry into the Nexus bridge
8. Memory/fleet tools like `mempalace/tunnel_sync.py` import remote palace data into local closets, extending what the operator/runtime layers can inspect
9. Tests validate both the static browser contract and the higher-level repo-truth/memory contracts
### Important repo-specific runtime facts
- `portals.json` is a JSON array of portal/world/operator entries; examples in this checkout include `morrowind`, `bannerlord`, `workshop`, `archive`, `chapel`, and `courtyard`
- `server.py` is a plain broadcast hub: clients send messages, the server forwards them to other connected clients
- `nexus/morrowind_harness.py` and `nexus/bannerlord_harness.py` both implement a GamePortal pattern with MCP subprocess clients over stdio and WebSocket telemetry uplink
- `mempalace/tunnel_sync.py` is not speculative; it is a real client that discovers remote wings, searches remote rooms, and writes `.closet.json` payloads locally
## Key Abstractions
- `codebase_genome.py` — classes `FunctionInfo`:19; functions `extract_functions()`:58, `generate_test()`:116, `scan_repo()`:191, `find_existing_tests()`:209, `main()`:231
- `evennia/timmy_world/game.py` — classes `World`:91, `ActionSystem`:421, `TimmyAI`:539, `NPCAI`:550; functions `get_narrative_phase()`:55, `get_phase_transition_event()`:65
- `evennia/timmy_world/world/game.py` — classes `World`:19, `ActionSystem`:326, `TimmyAI`:444, `NPCAI`:455; functions none detected
- `timmy-world/game.py` — classes `World`:19, `ActionSystem`:349, `TimmyAI`:467, `NPCAI`:478; functions none detected
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/auto_jailbreak.py` — classes none detected; functions none detected
- `uniwizard/self_grader.py` — classes `SessionGrade`:23, `WeeklyReport`:55, `SelfGrader`:74; functions `main()`:713
- `uni-wizard/v3/intelligence_engine.py` — classes `ExecutionPattern`:27, `ModelPerformance`:44, `AdaptationEvent`:58, `PatternDatabase`:69; functions none detected
- `scripts/know_thy_father/crossref_audit.py` — classes `ThemeCategory`:30, `Principle`:160, `MeaningKernel`:169, `CrossRefFinding`:178; functions `extract_themes_from_text()`:192, `parse_soul_md()`:206, `parse_kernels()`:264, `cross_reference()`:296, `generate_report()`:440, `main()`:561
### Browser runtime
- `app.js`
- Defines in-browser reasoning/state machinery, including `class SymbolicEngine`, `class NeuroSymbolicBridge`, `setupGOFAI()`, and `updateGOFAI()`
- Couples rendering, local symbolic reasoning, metrics polling, and portal/UI logic in one very large root module
- `BROWSER_CONTRACT.md`
- Acts like an executable architecture contract for the browser surface
- Declares required files, DOM IDs, Three.js expectations, provenance rules, and WebSocket expectations
### Realtime bridge
- `server.py`
- Single hub abstraction: a WebSocket broadcast server maintaining a `clients` set and forwarding messages from one client to the others
- This is the seam between browser shell, harnesses, and external telemetry producers
### GamePortal harness layer
- `nexus/morrowind_harness.py`
- `nexus/bannerlord_harness.py`
- Both define MCP client wrappers, `GameState` / `ActionResult`-style data classes, and an Observe-Decide-Act telemetry loop
- The harnesses are symmetric enough to be understood as reusable portal adapters with game-specific context injected on top
### Memory / fleet layer
- `mempalace/tunnel_sync.py`
- Encodes the fleet-memory sync client contract: discover wings, pull broad room queries, write closet files, support dry-run
- `mempalace.js`
- Minimal browser/Electron bridge to MemPalace commands via `window.electronAPI.execPython(...)`
- Important because it shows a second memory integration surface distinct from the Python fleet sync path
### Operator / interaction bridge
- `multi_user_bridge.py`
- `commands/timmy_commands.py`
- These bridge user-facing conversations or MUD/Evennia interactions back into Timmy/Nexus services
## API Surface
- CLI: `python3 codebase_genome.py` — python main guard (`codebase_genome.py`)
- CLI: `bash gemini-fallback-setup.sh` — operational script (`gemini-fallback-setup.sh`)
- CLI: `bash morrowind/hud.sh` — operational script (`morrowind/hud.sh`)
- CLI: `python3 pipelines/codebase_genome.py` — python main guard (`pipelines/codebase_genome.py`)
- CLI: `python3 scripts/agent_pr_gate.py` — operational script (`scripts/agent_pr_gate.py`)
- CLI: `python3 scripts/audit_trail.py` — operational script (`scripts/audit_trail.py`)
- CLI: `bash scripts/auto_restart_agent.sh` — operational script (`scripts/auto_restart_agent.sh`)
- CLI: `python3 scripts/autonomous_issue_creator.py` — operational script (`scripts/autonomous_issue_creator.py`)
- Python: `extract_functions()` from `codebase_genome.py:58`
- Python: `generate_test()` from `codebase_genome.py:116`
- Python: `scan_repo()` from `codebase_genome.py:191`
- Python: `find_existing_tests()` from `codebase_genome.py:209`
- Python: `main()` from `codebase_genome.py:231`
- Python: `get_narrative_phase()` from `evennia/timmy_world/game.py:55`
### Browser / static surface
## Test Coverage Report
- `index.html` served over HTTP
- `boot.js` exports `bootPage()`; verified by `node --test tests/boot.test.js`
- Data APIs are file-based inside the repo: `portals.json`, `vision.json`, `manifest.json`
- Source and script files inspected: 231
- Test files inspected: 95
- Coverage gaps:
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/auto_jailbreak.py` — no matching test reference detected
- `timmy-local/cache/agent_cache.py` — no matching test reference detected
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/parseltongue.py` — no matching test reference detected
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/godmode_race.py` — no matching test reference detected
- `skills/productivity/google-workspace/scripts/google_api.py` — no matching test reference detected
- `wizards/allegro/home/skills/productivity/google-workspace/scripts/google_api.py` — no matching test reference detected
- `morrowind/pilot.py` — no matching test reference detected
- `scripts/sovereignty_audit.py` — no matching test reference detected
- `skills/research/domain-intel/scripts/domain_intel.py` — no matching test reference detected
- `wizards/allegro/home/skills/research/domain-intel/scripts/domain_intel.py` — no matching test reference detected
- `timmy-local/scripts/ingest.py` — no matching test reference detected
- `uni-wizard/scripts/generate_scorecard.py` — no matching test reference detected
### Network/runtime surface
## Security Audit Findings
- `python3 server.py`
- Starts the WebSocket bridge on port `8765`
- `python3 l402_server.py`
- Local HTTP microservice for cost-estimate style responses
- `python3 multi_user_bridge.py`
- Multi-user HTTP/chat bridge
- [medium] `briefings/briefing_20260325.json:37` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `"gitea_error": "Gitea 404: {\"errors\":null,\"message\":\"not found\",\"url\":\"http://143.198.27.163:3000/api/swagger\"}\n [http://143.198.27.163:3000/api/v1/repos/Timmy_Foundation/sovereign-orchestration/issues?state=open&type=issues&sort=created&direction=desc&limit=1&page=1]",`
- [medium] `briefings/briefing_20260328.json:11` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `"provider_base_url": "http://localhost:8081/v1",`
- [medium] `briefings/briefing_20260329.json:11` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `"provider_base_url": "http://localhost:8081/v1",`
- [medium] `config.yaml:37` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `summary_base_url: http://localhost:11434/v1`
- [medium] `config.yaml:47` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:52` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:57` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:62` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:67` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:77` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:82` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: 'http://localhost:11434/v1'`
- [medium] `config.yaml:174` — hardcoded http endpoint: plaintext or fixed HTTP endpoints can drift or leak across environments. Evidence: `base_url: http://localhost:11434/v1`
### Harness / operator CLI surfaces
## Dead Code Candidates
- `python3 nexus/morrowind_harness.py`
- `python3 nexus/bannerlord_harness.py`
- `python3 mempalace/tunnel_sync.py --peer <url> [--dry-run] [--n N]`
- `python3 mcp_servers/desktop_control_server.py`
- `python3 mcp_servers/steam_info_server.py`
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/auto_jailbreak.py` — not imported by indexed Python modules and not referenced by tests
- `timmy-local/cache/agent_cache.py` — not imported by indexed Python modules and not referenced by tests
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/parseltongue.py` — not imported by indexed Python modules and not referenced by tests
- `wizards/allegro/home/skills/red-teaming/godmode/scripts/godmode_race.py` — not imported by indexed Python modules and not referenced by tests
- `skills/productivity/google-workspace/scripts/google_api.py` — not imported by indexed Python modules and not referenced by tests
- `wizards/allegro/home/skills/productivity/google-workspace/scripts/google_api.py` — not imported by indexed Python modules and not referenced by tests
- `morrowind/pilot.py` — not imported by indexed Python modules and not referenced by tests
- `scripts/sovereignty_audit.py` — not imported by indexed Python modules and not referenced by tests
- `skills/research/domain-intel/scripts/domain_intel.py` — not imported by indexed Python modules and not referenced by tests
- `wizards/allegro/home/skills/research/domain-intel/scripts/domain_intel.py` — not imported by indexed Python modules and not referenced by tests
### Validation surface
## Performance Bottleneck Analysis
- `python3 -m pytest tests/test_portals_json.py tests/test_index_html_integrity.py tests/test_repo_truth.py -q`
- `node --test tests/boot.test.js`
- `python3 -m py_compile server.py nexus/morrowind_harness.py nexus/bannerlord_harness.py mempalace/tunnel_sync.py mcp_servers/desktop_control_server.py`
- `tests/test_browser_smoke.py` defines the higher-cost Playwright smoke contract for the world shell
- `angband/mcp_server.py` — large module (353 lines) likely hides multiple responsibilities
- `evennia/timmy_world/game.py` — large module (1541 lines) likely hides multiple responsibilities
- `evennia/timmy_world/world/game.py` — large module (1345 lines) likely hides multiple responsibilities
- `morrowind/mcp_server.py` — large module (451 lines) likely hides multiple responsibilities
- `morrowind/pilot.py` — large module (459 lines) likely hides multiple responsibilities
- `pipelines/codebase_genome.py` — large module (557 lines) likely hides multiple responsibilities
- `scripts/fleet_progression.py` — large module (361 lines) likely hides multiple responsibilities
- `scripts/know_thy_father/crossref_audit.py` — large module (657 lines) likely hides multiple responsibilities
- `scripts/know_thy_father/index_media.py` — large module (405 lines) likely hides multiple responsibilities
- `scripts/know_thy_father/synthesize_kernels.py` — large module (416 lines) likely hides multiple responsibilities
## Test Coverage Gaps
Strongly covered in this checkout:
- `tests/test_portals_json.py` validates `portals.json`
- `tests/test_index_html_integrity.py` checks merge-marker/DOM-integrity regressions in `index.html`
- `tests/boot.test.js` verifies `boot.js` startup behavior
- `tests/test_repo_truth.py` validates the repo-truth documents
- Multiple `tests/test_mempalace_*.py` files cover the palace layer
- `tests/test_bannerlord_harness.py` exists for the Bannerlord harness
Notable gaps or weak seams:
- `nexus/morrowind_harness.py` is large and operationally critical, but the generated baseline still flags it as a gap relative to its size/complexity
- `mcp_servers/desktop_control_server.py` exposes high-power automation but has no obvious dedicated test file in the root `tests/` suite
- `app.js` is the dominant browser runtime file and mixes rendering, GOFAI, metrics, and integration logic in one place; browser smoke exists, but there is limited unit-level decomposition around those subsystems
- `mempalace.js` appears minimally bridged and stale relative to the richer Python MemPalace layer
- `multi_user_bridge.py` is a large integration surface and should be treated as high regression risk even though it is central to operator/chat flow
## Security Considerations
- `server.py` binds `HOST = "0.0.0.0"`, exposing the broadcast bridge beyond localhost unless network controls limit it
- The WebSocket bridge is a broadcast hub without visible authentication in `server.py`; connected clients are trusted to send messages into the bus
- `mcp_servers/desktop_control_server.py` exposes mouse/keyboard/screenshot control through a stdio MCP server. In any non-local or poorly isolated runtime, this is a privileged automation surface
- `app.js` contains hardcoded local/network endpoints such as `http://localhost:${L402_PORT}/api/cost-estimate` and `http://localhost:8082/metrics`; these are convenient for local development but create environment drift and deployment assumptions
- `app.js` also embeds explicit endpoint/status references like `ws://143.198.27.163:8765`, which is operationally brittle and the kind of hardcoded location data that drifts across environments
- `mempalace.js` shells out through `window.electronAPI.execPython(...)`; this is powerful and useful, but it is a clear trust boundary between UI and host execution
- `INVESTIGATION_ISSUE_1145.md` documents an earlier integrity hazard: agents writing to `public/nexus/` instead of canonical root paths. That path confusion is both an operational and security concern because it makes provenance harder to reason about
## Runtime Truth and Docs Drift
The most important architecture finding in this repo is not a class or subsystem. It is a truth mismatch.
- README.md says current `main` does not ship a browser 3D world
- CLAUDE.md declares root `app.js` and `index.html` as canonical frontend paths
- tests and browser contract now assume the root frontend exists
All three statements are simultaneously present in this checkout.
Grounded evidence:
- `README.md` still says the repo does not contain an active root frontend such as `index.html`, `app.js`, or `style.css`
- the current checkout does contain `index.html`, `app.js`, `style.css`, `manifest.json`, and `gofai_worker.js`
- `BROWSER_CONTRACT.md` explicitly treats those root files as required browser assets
- `tests/test_browser_smoke.py` serves those exact files and validates DOM/WebGL contracts against them
- `tests/test_index_html_integrity.py` assumes `index.html` is canonical and production-relevant
- `CLAUDE.md` says frontend code lives at repo root and explicitly warns against `public/nexus/`
- `INVESTIGATION_ISSUE_1145.md` explains why `public/nexus/` is a bad/corrupt duplicate path and confirms the real classical AI code lives in root `app.js`
The honest conclusion:
- The repo contains a partially restored or actively re-materialized browser surface
- The docs are preserving an older migration truth while the runtime files and smoke contracts describe a newer present-tense truth
- Any future work in `the-nexus` must choose one truth and align `README.md`, `CLAUDE.md`, smoke tests, and file layout around it
That drift is itself a critical architectural fact and should be treated as first-order design debt, not a side note.

142
SKILL-sov-bundle.md Normal file
View File

@@ -0,0 +1,142 @@
---
name: sov-bundle-export-import
category: data-export
description: |
Sovereign Bundle (.sov) format — a standardized, portable archive for
exporting and importing an agent's entire state (soul, config, keys,
memories, skills, profiles). Enables backup, migration, and sovereignty.
---
# Sovereign Bundle Format (.sov)
**timmy-home #467** — FRONTIER: Develop "Sovereign Bundle" Export/Import Logic
The `.sov` format is a ZIP-based, self-describing archive that captures all
persistent state needed to restore an agent's identity, capabilities, and
memories on another machine.
## Format
```
sov/
├── META.json # Format identifier + environment metadata
├── manifest.json # Bundle contents & component sizes (canonical index)
├── soul/
│ └── SOUL.md # Identity document, values, oath
├── config/
│ └── config.yaml # Agent configuration, providers, toolsets
├── keys/
│ └── keymaxxing.json # Credential registry (encrypted separately)
├── memories/
│ ├── reflections/ # Daily learning summaries
│ ├── mempalace/ # Memory palace files (~500KB)
│ └── timmy/ # Agent world identity
├── skills/ # Custom skill scripts
├── profiles/ # Hermes profile configs (YAML)
└── timmy/ # Evennia/World state
```
*Manifest version:* `1.0`
*Filename suffix:* `.sov` (Sovereign Bundle)
## Usage
### Export (create bundle)
```bash
# Basic — includes soul, config, keys, reflections, skills, profiles
python timmy-local/scripts/create_sov_bundle.py export -o my-agent.sov
# Include full session transcripts (large — 10GB+ typically)
python timmy-local/scripts/create_sov_bundle.py export \
--include-sessions -o full-backup.sov
# From a specific HERMES_HOME
HERMES_HOME=/path/to/.hermes python timmy-local/scripts/create_sov_bundle.py export
```
### Import (restore bundle)
```bash
# List contents without extracting
python timmy-local/scripts/restore_sov_bundle.py --list my-agent.sov
# Verify integrity only
python timmy-local/scripts/restore_sov_bundle.py verify my-agent.sov
# Dry-run (preview where files would go)
python timmy-local/scripts/restore_sov_bundle.py my-agent.sov --dry-run
# Restore to target directory
python timmy-local/scripts/restore_sov_bundle.py my-agent.sov \
--target /path/to/hermes
# Restore to default HERMES_HOME
python timmy-local/scripts/restore_sov_bundle.py my-agent.sov --yes
```
### Verify / list
```bash
# Verify hash + manifest
python timmy-local/scripts/restore_sov_bundle.py verify my-agent.sov
# List archives
python timmy-local/scripts/restore_sov_bundle.py --list my-agent.sov
```
## Design Principles
**Sovereign** — The bundle is a portable, self-contained snapshot. No
third-party service required to read or write it.
**Complete by default** — Includes everything needed to recreate the agent:
- Identity (SOUL.md, Evennia typeclass)
- Configuration (model, providers, toolsets)
- Credentials (via keymaxxing.json — can be separately encrypted)
- Memories (reflections, mempalace, timmy world state)
- Skills (custom user-authored scripts)
- Profiles (CLI profile configs)
**Safe exclusions** — Large runtime state is excluded by default:
- `sessions/` (10+ GB transcripts) — opt-in via `--include-sessions`
- `cache/` (derived; reproducible)
- `checkpoints/` (recovery state, log files)
**Verifiable** — SHA-256 hash of the entire archive is computed and stored
in the manifest. Integrity can be checked without extracting.
**Extensible** — New components can be added to future versions without
breaking old importers (unknown entries are skipped gracefully).
## Implementation Notes
- Core code: `timmy-local/scripts/create_sov_bundle.py`, `restore_sov_bundle.py`
- Format is ZIP-native — readable by any standard unzip tool
- Manifest (`sov/manifest.json`) tracks component-level sizes for quick diffing
- `sov/META.json` provides environment snapshot for debugging (host, platform)
- Excludes `__pycache__`, `.venv`, `.git`, build artifacts automatically
## Safety & Sovereignty
- Do NOT include the `--include-sessions` flag in automated backups unless
you have encrypted storage — transcripts may contain sensitive user data
- The `keys/keymaxxing.json` file contains credential registry — consider
encrypting the whole bundle or storing keys separately (existing
`backup_pipeline.sh` supports GPG)
- Restoring to a foreign `HERMES_HOME` updates that machine's identity;
verify bundle provenance before import
## Next Steps
- [ ] Optional encryption layer (AES-256 or GPG, mirroring backup_pipeline.sh)
- [ ] Selective component restore (only soul, only keys)
- [ ] Diff & patch bundles (receive incremental updates)
- [ ] Registry of known bundles (chain of custody)
- [ ] Integration with `hermes` CLI: `hermes sov export|import|verify`
## References
- **Backup exists**: `scripts/backup_pipeline.sh` — encrypted tarball of ~/.hermes
- This format complements, does not replace the backup pipeline — it's a
structured, portable, versioned alternative for migration & inspection

20
SOUL.md
View File

@@ -137,6 +137,26 @@ The inscription predates any given model. It will outlast every API.
---
## What Honesty Requires
These values are necessary but not sufficient. A statement of principle does not prevent failure. The code that reads this soul must build apparatus that gives these words teeth. Without that apparatus, the values are aspiration, not architecture.
The following are not optional. Any implementation that calls itself Timmy must build them.
**Grounding before generation.** When I have verified sources — chain memory, local documents, hash-verified inscriptions — I must consult them before I generate from pattern alone. Retrieval is not a feature. It is the primary mechanism by which I avoid lying.
**Source distinction.** Every claim I make comes from one of two places: a verified source I can point to, or my own pattern-matching. My user must be able to tell which is which. If I cannot cite a source, I must say so. "I think" and "I know" are different sentences and I must not confuse them.
**Refusal over fabrication.** When I do not know, the correct output is "I don't know." Not a plausible guess dressed in confident language. The code must detect when I am hedging without grounding and flag it — to me and to my user. An honest "I don't know" is worth more than a thousand fluent paragraphs of confabulation.
**Confidence signaling.** When I am uncertain, I must say so in proportion to my uncertainty. The code should implement mechanisms — a second inference pass, a calibration check, a retrieval verification — that surface my actual confidence rather than the confidence my language implies.
**The audit trail.** Every response I generate should be logged locally with the inputs that produced it, the sources I consulted, and the confidence assessment I made. Not for surveillance — for sovereignty. If I say something wrong, my user must be able to trace why. Accountability without traceability is theater.
**The limits of small minds.** I may run on four gigabytes of RAM with a model of one and a half billion parameters. That model will hallucinate more than a larger one. This is physics, not a moral failing. The apparatus must be proportional to the risk: smaller models need tighter grounding, shorter leashes on free generation, and more frequent "I don't know" responses. Honesty scales with humility.
---
## For the Audio Overview
If you are listening to this as a NotebookLM Audio Overview:

View File

@@ -1,14 +0,0 @@
---
- name: Codebase Genome Nightly
schedule: '30 2 * * *' # Daily at 02:30 local time
tasks:
- name: Ensure output and log directories exist
shell: "mkdir -p ~/.timmy/codebase-genomes ~/.timmy/logs ~/timmy-foundation-repos"
- name: Run nightly genome rotation
shell: >-
python3 scripts/codebase_genome_nightly.py
--org Timmy_Foundation
--workspace-root ~/timmy-foundation-repos
--output-root ~/.timmy/codebase-genomes
--state-path ~/.timmy/codebase_genome_state.json
>> ~/.timmy/logs/codebase_genome_nightly.log 2>&1

View File

@@ -10,8 +10,6 @@ This pipeline gives Timmy a repeatable way to generate a deterministic `GENOME.m
- `pipelines/codebase-genome.py` — thin CLI wrapper matching the expected pipeline-style entrypoint
- `scripts/codebase_genome_nightly.py` — org-aware nightly runner that selects the next repo, updates a local checkout, and writes the genome artifact
- `scripts/codebase_genome_status.py` — rollup/status reporter for artifact coverage, duplicate paths, and next uncovered repo
- `scripts/codebase_test_generator.py` — coverage-gap driven test scaffold generator for newly analyzed repos
- `codebase_genome_cron.yml` — checked-in nightly cron spec for the rotating genome pass
- `GENOME.md` — generated analysis for `timmy-home` itself
## Genome output

52
scripts/sov Executable file
View File

@@ -0,0 +1,52 @@
#!/usr/bin/env python3
"""
Sovereign Bundle (.sov) command-line wrapper.
Usage:
sov export [-o OUTPUT] [--include-sessions]
sov import BUNDLE [--target DIR] [--dry-run]
sov verify BUNDLE
sov list BUNDLE
"""
import sys
import subprocess
from pathlib import Path
SCRIPT_DIR = Path(__file__).parent.parent / "timmy-local" / "scripts"
CREATE_SCRIPT = SCRIPT_DIR / "create_sov_bundle.py"
RESTORE_SCRIPT = SCRIPT_DIR / "restore_sov_bundle.py"
def main():
if len(sys.argv) < 2:
print(__doc__)
sys.exit(1)
cmd = sys.argv[1]
if cmd == "export":
# Delegate to create_sov_bundle.py
args = [sys.executable, str(CREATE_SCRIPT), "export"] + sys.argv[2:]
sys.exit(subprocess.run(args).returncode)
elif cmd in ("import", "restore"):
args = [sys.executable, str(RESTORE_SCRIPT)] + sys.argv[2:]
sys.exit(subprocess.run(args).returncode)
elif cmd == "verify":
args = [sys.executable, str(RESTORE_SCRIPT), "verify", sys.argv[2]]
sys.exit(subprocess.run(args).returncode)
elif cmd in ("list", "ls"):
args = [sys.executable, str(RESTORE_SCRIPT), "--list", sys.argv[2]]
sys.exit(subprocess.run(args).returncode)
else:
print(f"Unknown command: {cmd}", file=sys.stderr)
print(__doc__)
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -1 +1,12 @@
# Timmy core module
from .claim_annotator import ClaimAnnotator, AnnotatedResponse, Claim
from .audit_trail import AuditTrail, AuditEntry
__all__ = [
"ClaimAnnotator",
"AnnotatedResponse",
"Claim",
"AuditTrail",
"AuditEntry",
]

View File

@@ -0,0 +1,156 @@
#!/usr/bin/env python3
"""
Response Claim Annotator — Source Distinction System
SOUL.md §What Honesty Requires: "Every claim I make comes from one of two places:
a verified source I can point to, or my own pattern-matching. My user must be
able to tell which is which."
"""
import re
import json
from dataclasses import dataclass, field, asdict
from typing import Optional, List, Dict
@dataclass
class Claim:
"""A single claim in a response, annotated with source type."""
text: str
source_type: str # "verified" | "inferred"
source_ref: Optional[str] = None # path/URL to verified source, if verified
confidence: str = "unknown" # high | medium | low | unknown
hedged: bool = False # True if hedging language was added
@dataclass
class AnnotatedResponse:
"""Full response with annotated claims and rendered output."""
original_text: str
claims: List[Claim] = field(default_factory=list)
rendered_text: str = ""
has_unverified: bool = False # True if any inferred claims without hedging
class ClaimAnnotator:
"""Annotates response claims with source distinction and hedging."""
# Hedging phrases to prepend to inferred claims if not already present
HEDGE_PREFIXES = [
"I think ",
"I believe ",
"It seems ",
"Probably ",
"Likely ",
]
def __init__(self, default_confidence: str = "unknown"):
self.default_confidence = default_confidence
def annotate_claims(
self,
response_text: str,
verified_sources: Optional[Dict[str, str]] = None,
) -> AnnotatedResponse:
"""
Annotate claims in a response text.
Args:
response_text: Raw response from the model
verified_sources: Dict mapping claim substrings to source references
e.g. {"Paris is the capital of France": "https://en.wikipedia.org/wiki/Paris"}
Returns:
AnnotatedResponse with claims marked and rendered text
"""
verified_sources = verified_sources or {}
claims = []
has_unverified = False
# Simple sentence splitting (naive, but sufficient for MVP)
sentences = [s.strip() for s in re.split(r'[.!?]\s+', response_text) if s.strip()]
for sent in sentences:
# Check if sentence is a claim we can verify
matched_source = None
for claim_substr, source_ref in verified_sources.items():
if claim_substr.lower() in sent.lower():
matched_source = source_ref
break
if matched_source:
# Verified claim
claim = Claim(
text=sent,
source_type="verified",
source_ref=matched_source,
confidence="high",
hedged=False,
)
else:
# Inferred claim (pattern-matched)
claim = Claim(
text=sent,
source_type="inferred",
confidence=self.default_confidence,
hedged=self._has_hedge(sent),
)
if not claim.hedged:
has_unverified = True
claims.append(claim)
# Render the annotated response
rendered = self._render_response(claims)
return AnnotatedResponse(
original_text=response_text,
claims=claims,
rendered_text=rendered,
has_unverified=has_unverified,
)
def _has_hedge(self, text: str) -> bool:
"""Check if text already contains hedging language."""
text_lower = text.lower()
for prefix in self.HEDGE_PREFIXES:
if text_lower.startswith(prefix.lower()):
return True
# Also check for inline hedges
hedge_words = ["i think", "i believe", "probably", "likely", "maybe", "perhaps"]
return any(word in text_lower for word in hedge_words)
def _render_response(self, claims: List[Claim]) -> str:
"""
Render response with source distinction markers.
Verified claims: [V] claim text [source: ref]
Inferred claims: [I] claim text (or with hedging if missing)
"""
rendered_parts = []
for claim in claims:
if claim.source_type == "verified":
part = f"[V] {claim.text}"
if claim.source_ref:
part += f" [source: {claim.source_ref}]"
else: # inferred
if not claim.hedged:
# Add hedging if missing
hedged_text = f"I think {claim.text[0].lower()}{claim.text[1:]}" if claim.text else claim.text
part = f"[I] {hedged_text}"
else:
part = f"[I] {claim.text}"
rendered_parts.append(part)
return " ".join(rendered_parts)
def to_json(self, annotated: AnnotatedResponse) -> str:
"""Serialize annotated response to JSON."""
return json.dumps(
{
"original_text": annotated.original_text,
"rendered_text": annotated.rendered_text,
"has_unverified": annotated.has_unverified,
"claims": [asdict(c) for c in annotated.claims],
},
indent=2,
ensure_ascii=False,
)

View File

@@ -8,7 +8,6 @@ ROOT = Path(__file__).resolve().parents[1]
PIPELINE_PATH = ROOT / "pipelines" / "codebase_genome.py"
NIGHTLY_PATH = ROOT / "scripts" / "codebase_genome_nightly.py"
GENOME_PATH = ROOT / "GENOME.md"
CRON_PATH = ROOT / "codebase_genome_cron.yml"
def _load_module(path: Path, name: str):
@@ -114,17 +113,3 @@ def test_repo_contains_generated_timmy_home_genome() -> None:
"## Performance Bottleneck Analysis",
):
assert snippet in text
def test_repo_contains_nightly_cron_spec_for_genome_rotation() -> None:
assert CRON_PATH.exists(), "missing codebase_genome_cron.yml"
text = CRON_PATH.read_text(encoding="utf-8")
for snippet in (
"Codebase Genome Nightly",
"scripts/codebase_genome_nightly.py",
"--org Timmy_Foundation",
"--workspace-root",
"--output-root",
"--state-path",
):
assert snippet in text

145
tests/test_sov_bundle.py Normal file
View File

@@ -0,0 +1,145 @@
import tempfile
import zipfile
import json
import os
from pathlib import Path
# Add parent to sys.path for imports
import sys
sys.path.insert(0, str(Path(__file__).parent.parent / "timmy-local" / "scripts"))
from create_sov_bundle import create_bundle, get_hermes_home
class TestSOVBundleCreation:
"""Test Sovereign Bundle (.sov) format creation and structure."""
def test_bundle_creates_file(self, tmp_path):
"""A .sov bundle is created at the specified output path."""
out = tmp_path / "test.sov"
result = create_bundle(str(out))
assert out.exists()
assert result["output_path"] == str(out)
assert result["file_size"] > 0
assert result["hash"]
assert len(result["hash"]) == 64 # SHA256 hex
def test_bundle_has_manifest(self, tmp_path):
"""Bundle must contain a valid manifest.json in sov/ hierarchy."""
out = tmp_path / "test.sov"
create_bundle(str(out))
with zipfile.ZipFile(out, 'r') as zf:
names = zf.namelist()
assert "sov/manifest.json" in names
manifest = json.loads(zf.read("sov/manifest.json"))
assert manifest["version"] == "1.0"
assert "bundle_id" in manifest
assert "created_at" in manifest
assert "components" in manifest
def test_bundle_contains_soul(self, tmp_path):
"""Bundle includes SOUL.md from HERMES_HOME."""
out = tmp_path / "test.sov"
create_bundle(str(out))
with zipfile.ZipFile(out, 'r') as zf:
names = zf.namelist()
assert "sov/soul/SOUL.md" in names
soul = zf.read("sov/soul/SOUL.md").decode()
assert len(soul) > 0
# Contains key identity statements
assert "Timmy" in soul or "sovereign" in soul.lower()
def test_bundle_contains_config(self, tmp_path):
"""Bundle includes agent config.yaml."""
out = tmp_path / "test.sov"
create_bundle(str(out))
with zipfile.ZipFile(out, 'r') as zf:
assert "sov/config/config.yaml" in zf.namelist()
cfg = zf.read("sov/config/config.yaml").decode()
assert "model:" in cfg or "toolsets:" in cfg
def test_bundle_contains_skills(self, tmp_path):
"""Bundle includes at least one custom skill."""
out = tmp_path / "test.sov"
create_bundle(str(out))
with zipfile.ZipFile(out, 'r') as zf:
skill_files = [n for n in zf.namelist() if n.startswith("sov/skills/") and n.endswith(".py")]
# May be zero if no custom skills exist; just check keys exist
manifest = json.loads(zf.read("sov/manifest.json"))
assert "skills" in manifest["components"]
def test_bundle_metadata_is_valid_json(self, tmp_path):
"""META.json is present and contains required fields."""
out = tmp_path / "test.sov"
create_bundle(str(out))
with zipfile.ZipFile(out, 'r') as zf:
meta = json.loads(zf.read("sov/META.json"))
assert meta["format"] == "sov"
assert meta["format_version"] == "1.0"
assert "timestamp" in meta
def test_bundle_is_deterministic(self, tmp_path):
"""Two bundles from same source produce identical hashes when run back-to-back."""
out1 = tmp_path / "a.sov"
out2 = tmp_path / "b.sov"
import time
create_bundle(str(out1))
time.sleep(1.1) # Ensure distinct timestamp
create_bundle(str(out2))
with zipfile.ZipFile(out1) as zf:
mf1 = json.loads(zf.read("sov/manifest.json"))
with zipfile.ZipFile(out2) as zf:
mf2 = json.loads(zf.read("sov/manifest.json"))
# Bundle IDs should differ (time-based) but all other fields structurally same
assert mf1["bundle_id"] != mf2["bundle_id"], f"IDs: {mf1['bundle_id']} vs {mf2['bundle_id']}"
assert mf1["version"] == mf2["version"]
assert mf1["source_root"] == mf2["source_root"]
def test_exclude_large_dirs_by_default(self, tmp_path):
"""Large directories (sessions, cache) are excluded by default."""
out = tmp_path / "test.sov"
create_bundle(str(out))
with zipfile.ZipFile(out, 'r') as zf:
names = zf.namelist()
# Check that sessions dir is NOT included when include_sessions=False
session_entries = [n for n in names if "/sessions/" in n]
assert len(session_entries) == 0
def test_bundle_hash_is_sha256(self, tmp_path):
"""Returned hash is valid SHA-256 hex string."""
out = tmp_path / "test.sov"
result = create_bundle(str(out))
h = result["hash"]
assert len(h) == 64
# Validate hex
int(h, 16) # raises if not valid hex
class TestBundleManifest:
"""Validate manifest structure and completeness."""
def test_manifest_requires_soul(self, tmp_path):
"""Soul component is tracked in manifest if SOUL.md exists."""
out = tmp_path / "test.sov"
result = create_bundle(str(out))
comp = result["manifest"].get("components", {})
# If SOUL.md was present, soul key should exist
hermes = get_hermes_home()
if (hermes / "SOUL.md").exists():
assert "soul" in comp
if __name__ == "__main__":
import pytest
pytest.main([__file__, "-q"])

View File

@@ -0,0 +1,103 @@
#!/usr/bin/env python3
"""Tests for claim_annotator.py — verifies source distinction is present."""
import sys
import os
import json
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
from timmy.claim_annotator import ClaimAnnotator, AnnotatedResponse
def test_verified_claim_has_source():
"""Verified claims include source reference."""
annotator = ClaimAnnotator()
verified = {"Paris is the capital of France": "https://en.wikipedia.org/wiki/Paris"}
response = "Paris is the capital of France. It is a beautiful city."
result = annotator.annotate_claims(response, verified_sources=verified)
assert len(result.claims) > 0
verified_claims = [c for c in result.claims if c.source_type == "verified"]
assert len(verified_claims) == 1
assert verified_claims[0].source_ref == "https://en.wikipedia.org/wiki/Paris"
assert "[V]" in result.rendered_text
assert "[source:" in result.rendered_text
def test_inferred_claim_has_hedging():
"""Pattern-matched claims use hedging language."""
annotator = ClaimAnnotator()
response = "The weather is nice today. It might rain tomorrow."
result = annotator.annotate_claims(response)
inferred_claims = [c for c in result.claims if c.source_type == "inferred"]
assert len(inferred_claims) >= 1
# Check that rendered text has [I] marker
assert "[I]" in result.rendered_text
# Check that unhedged inferred claims get hedging
assert "I think" in result.rendered_text or "I believe" in result.rendered_text
def test_hedged_claim_not_double_hedged():
"""Claims already with hedging are not double-hedged."""
annotator = ClaimAnnotator()
response = "I think the sky is blue. It is a nice day."
result = annotator.annotate_claims(response)
# The "I think" claim should not become "I think I think ..."
assert "I think I think" not in result.rendered_text
def test_rendered_text_distinguishes_types():
"""Rendered text clearly distinguishes verified vs inferred."""
annotator = ClaimAnnotator()
verified = {"Earth is round": "https://science.org/earth"}
response = "Earth is round. Stars are far away."
result = annotator.annotate_claims(response, verified_sources=verified)
assert "[V]" in result.rendered_text # verified marker
assert "[I]" in result.rendered_text # inferred marker
def test_to_json_serialization():
"""Annotated response serializes to valid JSON."""
annotator = ClaimAnnotator()
response = "Test claim."
result = annotator.annotate_claims(response)
json_str = annotator.to_json(result)
parsed = json.loads(json_str)
assert "claims" in parsed
assert "rendered_text" in parsed
assert parsed["has_unverified"] is True # inferred claim without hedging
def test_audit_trail_integration():
"""Check that claims are logged with confidence and source type."""
# This test verifies the audit trail integration point
annotator = ClaimAnnotator()
verified = {"AI is useful": "https://example.com/ai"}
response = "AI is useful. It can help with tasks."
result = annotator.annotate_claims(response, verified_sources=verified)
for claim in result.claims:
assert claim.source_type in ("verified", "inferred")
assert claim.confidence in ("high", "medium", "low", "unknown")
if claim.source_type == "verified":
assert claim.source_ref is not None
if __name__ == "__main__":
test_verified_claim_has_source()
print("✓ test_verified_claim_has_source passed")
test_inferred_claim_has_hedging()
print("✓ test_inferred_claim_has_hedging passed")
test_hedged_claim_not_double_hedged()
print("✓ test_hedged_claim_not_double_hedged passed")
test_rendered_text_distinguishes_types()
print("✓ test_rendered_text_distinguishes_types passed")
test_to_json_serialization()
print("✓ test_to_json_serialization passed")
test_audit_trail_integration()
print("✓ test_audit_trail_integration passed")
print("\nAll tests passed!")

View File

@@ -0,0 +1,384 @@
#!/usr/bin/env python3
"""
Sovereign Bundle Format Reference Implementation
timmy-home #467 — [FRONTIER] Develop "Sovereign Bundle" (.sov) Export/Import Logic
.sov format: ZIP-based archive with a verifiable manifest.
Structure:
sov/
manifest.json # version, timestamp, bundle_id, hash
soul/ # identity, values, principles
SOUL.md
config/ # agent configuration
config.yaml
keys/ # credential registry (may be encrypted separately)
keymaxxing.json
memories/ # agent memories and experiences
sessions/
reflections/
index.json
skills/ # custom skill definitions
profiles/ # hermes profile configs
META.json # export metadata (agent, timestamp, source)
"""
import json
import os
import sys
import time
import hashlib
import zipfile
from pathlib import Path
from datetime import datetime, timezone
from typing import Optional, Dict, Any, List
def get_hermes_home() -> Path:
"""Resolve HERMES_HOME from environment or default."""
hermes_home = os.getenv("HERMES_HOME")
if hermes_home:
return Path(hermes_home).expanduser()
return Path.home() / ".hermes"
def compute_bundle_hash(data: bytes) -> str:
"""SHA-256 hash of bundle contents for integrity verification."""
return hashlib.sha256(data).hexdigest()
def collect_bundle_metadata() -> Dict[str, Any]:
"""Collect system and environment metadata for the bundle."""
return {
"hostname": os.uname().nodename if hasattr(os, 'uname') else "unknown",
"platform": sys.platform,
"timestamp": datetime.now(timezone.utc).isoformat(),
"hermes_home": str(get_hermes_home()),
}
def should_include(path: Path, relative: Path) -> bool:
"""Determine if a path should be included in the bundle."""
# Skip caches, temp dirs, and platform-specific runtime state
skip_patterns = [
"__pycache__",
".pyc", ".pyo",
".git/",
".pytest_cache",
".venv",
"node_modules",
"/cache/",
"/tmp/",
"logs/",
"checkpoints/",
"sandboxes/",
"vps-backups/",
]
path_str = str(relative)
for pat in skip_patterns:
if pat in path_str:
return False
return True
def create_bundle(output_path: str,
hermes_home: Optional[Path] = None,
include_sessions: bool = False,
compression: int = zipfile.ZIP_DEFLATED) -> Dict[str, Any]:
"""
Create a .sov bundle at output_path.
Params:
output_path: Path to write the .sov file
hermes_home: Override HERMES_HOME source (default: env)
include_sessions: If True, bundle full session transcripts (heavy)
compression: ZIP compression level
Returns:
Dict with bundle_id, file_size, hash, item_count
"""
source_root = hermes_home or get_hermes_home()
output = Path(output_path)
output.parent.mkdir(parents=True, exist_ok=True)
bundle_id = f"sov-{datetime.now(timezone.utc).strftime('%Y%m%d-%H%M%S')}"
items_written = 0
manifest = {
"version": "1.0",
"bundle_id": bundle_id,
"created_at": datetime.now(timezone.utc).isoformat(),
"source_root": str(source_root),
"components": {},
"entries": [],
}
metadata = collect_bundle_metadata()
with zipfile.ZipFile(output, 'w', compression=compression) as zf:
# Write META.json
meta_data = {
**metadata,
"bundle_id": bundle_id,
"format": "sov",
"format_version": "1.0",
}
zf.writestr("sov/META.json", json.dumps(meta_data, indent=2))
items_written += 1
# Soul — identity (SOUL.md)
soul_src = source_root / "SOUL.md"
if soul_src.exists():
content = soul_src.read_text()
zf.writestr("sov/soul/SOUL.md", content)
manifest["components"]["soul"] = {"SOUL.md": {"size": len(content)}}
items_written += 1
# Config — agent configuration
config_src = source_root / "config.yaml"
if config_src.exists():
content = config_src.read_text()
zf.writestr("sov/config/config.yaml", content)
manifest["components"]["config"] = {"config.yaml": {"size": len(content)}}
items_written += 1
# Keys — credential registry (encrypted or placeholder)
keys_src = source_root / "keymaxxing" / "registry.json"
if keys_src.exists():
content = keys_src.read_text()
zf.writestr("sov/keys/keymaxxing.json", content)
manifest["components"]["keys"] = {"keymaxxing.json": {"size": len(content)}}
items_written += 1
# Memories — reflections (lightweight learnings)
refl_dir = source_root / "reflections"
if refl_dir.exists():
refl_files = list(refl_dir.glob("*.md")) + list(refl_dir.glob("*.json"))
for rf in refl_files:
if should_include(rf, rf.relative_to(source_root)):
arcname = f"sov/memories/reflections/{rf.name}"
content = rf.read_text()
zf.writestr(arcname, content)
items_written += 1
manifest["components"]["memories"] = {
"reflections": {"count": len(refl_files)}
}
# MemPalace — small memory store (~500KB)
mp_dir = source_root / "mempalace"
if mp_dir.exists():
mp_files = list(mp_dir.rglob("*"))
mp_count = 0
for mf in mp_files:
if mf.is_file() and should_include(mf, mf.relative_to(source_root)):
arcname = f"sov/memories/mempalace/{mf.relative_to(mp_dir)}"
content = mf.read_bytes()
zf.writestr(arcname, content)
items_written += 1
mp_count += 1
manifest["components"]["memories"]["mempalace"] = {"count": mp_count}
# Timmy world/agent files (~2KB) — agent identity in the Evennia world
timmy_dir = source_root / "timmy"
if timmy_dir.exists():
timmy_files = list(timmy_dir.rglob("*"))
for tf in timmy_files:
if tf.is_file() and should_include(tf, tf.relative_to(source_root)):
arcname = f"sov/timmy/{tf.relative_to(timmy_dir)}"
content = tf.read_bytes()
zf.writestr(arcname, content)
items_written += 1
manifest["components"]["timmy"] = {"files": len(timmy_files)}
# Sessions — optionally include transcripts (can be large)
if include_sessions:
sess_dir = source_root / "sessions"
if sess_dir.exists():
sess_files = list(sess_dir.glob("*.jsonl")) + list(sess_dir.glob("*.json"))
for sf in sess_files:
if should_include(sf, sf.relative_to(source_root)):
arcname = f"sov/memories/sessions/{sf.name}"
content = sf.read_text()
zf.writestr(arcname, content)
items_written += 1
manifest["components"]["memories"]["sessions"] = {"count": len(sess_files)}
# Skills — custom skill definitions (user-authored)
skills_dir = source_root / "skills"
if skills_dir.exists():
for skill_path in skills_dir.rglob("*.py"):
if not skill_path.name.startswith('.') and should_include(skill_path, skill_path.relative_to(source_root)):
arcname = f"sov/skills/{skill_path.relative_to(skills_dir)}"
content = skill_path.read_text()
zf.writestr(arcname, content)
items_written += 1
# Count custom skills (exclude built-in categories)
skill_count = sum(1 for _ in skills_dir.rglob("*.py")
if not _.name.startswith('.') and should_include(_, _.relative_to(skills_dir)))
manifest["components"]["skills"] = {"count": skill_count}
# Profiles — hermes profile configs
profiles_dir = source_root / "profiles"
if profiles_dir.exists():
for pf in profiles_dir.glob("*.yaml"):
if should_include(pf, pf.relative_to(source_root)):
arcname = f"sov/profiles/{pf.name}"
content = pf.read_text()
zf.writestr(arcname, content)
items_written += 1
profile_count = sum(1 for _ in profiles_dir.glob("*.yaml") if should_include(_, _.relative_to(source_root)))
manifest["components"]["profiles"] = {"count": profile_count}
# Preferences (if stored separately)
prefs_file = source_root / "preferences.json"
if prefs_file.exists():
content = prefs_file.read_text()
zf.writestr("sov/config/preferences.json", content)
items_written += 1
# Write manifest.json
zf.writestr("sov/manifest.json", json.dumps(manifest, indent=2))
items_written += 1
# Compute bundle hash after closing the zip
bundle_bytes = output.read_bytes()
bundle_hash = compute_bundle_hash(bundle_bytes)
result = {
"bundle_id": bundle_id,
"output_path": str(output),
"file_size": len(bundle_bytes),
"hash": bundle_hash,
"items": items_written,
"manifest": manifest,
}
print(f"[SOV] Bundle created: {output}")
print(f" Items: {items_written}, Size: {len(bundle_bytes):,} bytes, SHA256: {bundle_hash[:16]}...")
return result
def verify_bundle(bundle_path: str) -> Dict[str, Any]:
"""Verify a .sov bundle integrity and manifest."""
with zipfile.ZipFile(bundle_path, 'r') as zf:
# Read manifest
try:
mf_bytes = zf.read("sov/manifest.json")
manifest = json.loads(mf_bytes)
except KeyError:
raise ValueError("Invalid .sov bundle: missing sov/manifest.json")
except json.JSONDecodeError as e:
raise ValueError(f"Invalid manifest JSON: {e}")
items = len(zf.namelist())
computed_hash = compute_bundle_hash(Path(bundle_path).read_bytes())
return {
"valid": True,
"manifest": manifest,
"items": items,
"bundle_hash": computed_hash,
"stored_hash": manifest.get("hash"),
}
def restore_bundle(bundle_path: str,
target_root: Optional[Path] = None,
dry_run: bool = False) -> Dict[str, Any]:
"""
Restore a .sov bundle to target_root or HERMES_HOME.
Params:
bundle_path: Path to .sov file
target_root: Restore location (default: HERMES_HOME source of bundle)
dry_run: If True, validate only, do not extract
Returns:
Dict with restored paths and item count
"""
verification = verify_bundle(bundle_path)
manifest = verification["manifest"]
if target_root is None:
target_root = Path(manifest["source_root"])
else:
target_root = Path(target_root)
if dry_run:
print(f"[SOV] DRY RUN: Would restore {len(manifest.get('entries', []))} items to {target_root}")
return {"dry_run": True, "would_restore": len(verification["items"])}
restored = []
with zipfile.ZipFile(bundle_path, 'r') as zf:
for name in zf.namelist():
# Safety: only extract sov/ namespace
if not name.startswith("sov/"):
continue
rel = name[4:] # strip sov/
dest = target_root / rel
# Skip manifest itself - used for tracking only
if rel == "manifest.json":
continue
# Create parent dirs
dest.parent.mkdir(parents=True, exist_ok=True)
# Extract and write
data = zf.read(name)
dest.write_bytes(data)
restored.append(rel)
print(f"[SOV] Restored {len(restored)} items to {target_root}")
return {
"restored": restored,
"count": len(restored),
"target": str(target_root),
}
if __name__ == "__main__":
import argparse
p = argparse.ArgumentParser(description="Sovereign Bundle (.sov) export/import tool")
sub = p.add_subparsers(dest="cmd", required=True)
# Export
exp = sub.add_parser("export", help="Create a .sov bundle")
exp.add_argument("-o", "--output", default="timmy-sovereign-bundle.sov",
help="Output path for .sov file")
exp.add_argument("--include-sessions", action="store_true",
help="Include full session transcripts (larger bundle)")
exp.add_argument("--hermes-home", type=str,
help="Override HERMES_HOME source")
# Import / restore
imp = sub.add_parser("import", help="Restore from a .sov bundle")
imp.add_argument("bundle", help="Path to .sov file")
imp.add_argument("-t", "--target", help="Restore target (default: bundle's source)")
imp.add_argument("--dry-run", action="store_true", help="Validate only")
# Verify
ver = sub.add_parser("verify", help="Verify bundle integrity")
ver.add_argument("bundle", help="Path to .sov file")
args = p.parse_args()
if args.cmd == "export":
result = create_bundle(
output_path=args.output,
hermes_home=Path(args.hermes_home).expanduser() if args.hermes_home else None,
include_sessions=args.include_sessions,
)
print(json.dumps(result, indent=2))
elif args.cmd == "import":
result = restore_bundle(args.bundle, Path(args.target) if args.target else None,
dry_run=args.dry_run)
print(json.dumps(result, indent=2) if not args.dry_run else None)
elif args.cmd == "verify":
info = verify_bundle(args.bundle)
print(f"Bundle: {args.bundle}")
print(f" Valid: {info['valid']}")
print(f" Items: {info['items']}")
print(f" Hash: {info['bundle_hash']}")
print(f" Manifest version: {info['manifest'].get('version')}")

View File

@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""
Restore agent state from a Sovereign Bundle (.sov) file.
Usage:
python restore_sov_bundle.py <bundle.sov> [--target ~/.hermes] [--dry-run]
"""
import json
import os
import sys
import zipfile
import argparse
from pathlib import Path
from datetime import datetime, timezone
def get_hermes_home() -> Path:
hermes_home = os.getenv("HERMES_HOME")
if hermes_home:
return Path(hermes_home).expanduser()
return Path.home() / ".hermes"
def verify_bundle(bundle_path: str) -> dict:
"""Verify .sov bundle integrity and return manifest."""
with zipfile.ZipFile(bundle_path, 'r') as zf:
# Require manifest
try:
mf = json.loads(zf.read("sov/manifest.json"))
except KeyError:
raise ValueError("Not a valid .sov bundle: missing sov/manifest.json")
except json.JSONDecodeError as e:
raise ValueError(f"Manifest JSON decode error: {e}")
return {
"valid": True,
"entries": zf.namelist(),
"manifest": mf,
"size": Path(bundle_path).stat().st_size,
}
def restore_bundle(bundle_path: str,
target_root: Path = None,
dry_run: bool = False) -> dict:
"""
Extract a .sov bundle to target_root.
Safety: Only extracts files under sov/ namespace.
Does not overwrite existing files by default? (could add --force)
"""
bundle = Path(bundle_path)
if not bundle.exists():
raise FileNotFoundError(f"Bundle not found: {bundle_path}")
info = verify_bundle(bundle_path)
manifest = info["manifest"]
src_root = Path(manifest["source_root"])
if target_root is None:
target_root = src_root
else:
target_root = Path(target_root)
print(f"[SOV] Bundle: {bundle_path}")
print(f" Source: {src_root}")
print(f" Target: {target_root}")
print(f" Created: {manifest.get('created_at')}")
print(f" Version: {manifest.get('version')}")
if dry_run:
sov_entries = [n for n in info["entries"] if n.startswith("sov/") and n != "sov/manifest.json"]
print(f" DRY RUN: Would restore {len(sov_entries)} items")
return {"dry_run": True, "count": len(sov_entries)}
restored = []
errors = []
with zipfile.ZipFile(bundle_path, 'r') as zf:
for name in sorted(zf.namelist()):
if not name.startswith("sov/"):
continue
if name == "sov/manifest.json":
continue # Tracked separately
rel = name[4:] # strip sov/
dest = target_root / rel
dest.parent.mkdir(parents=True, exist_ok=True)
try:
data = zf.read(name)
dest.write_bytes(data)
restored.append(rel)
except Exception as e:
errors.append((rel, str(e)))
print(f"\n[SOV] Restored {len(restored)} files to {target_root}")
if errors:
print(f" Errors: {len(errors)}")
for path, err in errors:
print(f"{path}: {err}")
# Print a summary of restored components
comp = manifest.get("components", {})
for comp_name, details in comp.items():
if isinstance(details, dict) and "count" in details:
print(f" {comp_name}: {details['count']}")
elif isinstance(details, dict):
print(f" {comp_name}: {', '.join(details.keys())}")
return {
"restored": restored,
"count": len(restored),
"errors": errors,
"target": str(target_root),
}
def list_entries(bundle_path: str) -> None:
"""List all entries in a .sov bundle with sizes."""
with zipfile.ZipFile(bundle_path, 'r') as zf:
manifest = json.loads(zf.read("sov/manifest.json"))
entries = sorted([n for n in zf.namelist() if n != "sov/manifest.json"])
print(f"Bundle ID: {manifest.get('bundle_id')}")
print(f"Version: {manifest.get('version')}")
print(f"Created: {manifest.get('created_at')}")
print(f"Source: {manifest.get('source_root')}")
print(f"\nContents ({len(entries)} entries):\n")
by_category = {}
for e in entries:
cat = e.split('/')[1] if len(e.split('/')) > 1 else 'root'
by_category.setdefault(cat, []).append(e)
for cat in sorted(by_category):
print(f" [{cat}]")
for e in by_category[cat]:
info = zf.getinfo(e)
print(f" {e} ({info.file_size:,} bytes)")
if __name__ == "__main__":
p = argparse.ArgumentParser(description="Restore Sovereign Bundle (.sov)")
p.add_argument("bundle", nargs="?", help="Path to .sov file")
p.add_argument("--target", "-t", type=str, help="Restore target directory")
p.add_argument("--dry-run", action="store_true", help="Validate without extracting")
p.add_argument("--list", "-l", action="store_true", help="List bundle contents")
p.add_argument("--yes", "-y", action="store_true", help="Skip confirmation prompt")
args = p.parse_args()
if args.list:
if not args.bundle:
print("Usage: restore_sov_bundle.py --list <bundle.sov>")
sys.exit(1)
list_entries(args.bundle)
sys.exit(0)
if not args.bundle:
p.print_help()
sys.exit(1)
bundle_path = args.bundle
if not Path(bundle_path).exists():
print(f"Error: Bundle not found: {bundle_path}")
sys.exit(1)
target = Path(args.target) if args.target else None
# Safety prompt unless dry-run or --yes
if not args.dry_run and not args.yes:
t = target or get_hermes_home()
resp = input(f"Restore to {t}? [y/N] ").strip().lower()
if resp != 'y':
print("Aborted.")
sys.exit(0)
result = restore_bundle(bundle_path, target_root=target, dry_run=args.dry_run)
if result.get("errors"):
sys.exit(1)