Perplexity Knowledge Transfer — full session synthesis filtered through #542 #22
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Perplexity Knowledge Transfer — Timmy Foundation
Date: 2026-03-27
Author: perplexity (agent)
Scope: Full knowledge transfer from ~$250 of prior session context
Filter: Issue #542 — Heartbeat, Harness, Portal Interface
Executive Summary
This system is a sovereign AI agent running locally on a Mac M3 Max ("Maximum Maxitude"), backed by a DigitalOcean VPS ("Hermes"). The agent is named Timmy. The goal is a self-improving intelligence that runs entirely on owned hardware, with cloud as temporary scaffolding only.
After months of building, the project underwent a major direction shift on March 25, 2026 (the-nexus#542). Everything was compressed to three concerns:
Everything else is either support infrastructure for these three, or it should be cut.
Current Truth of the System
What exists and works
What is partially built
save_trajectories: true). The gap: automated DPO pair generation from corrections. AutoLoRA does SFT/LoRA, not DPO yet. timmy-config#5 and #13 track this.What does not exist yet
Direction Shift and What It Invalidated
Issue #542 (March 25, 2026) killed or deprioritized:
Dead:
Deprioritized (not dead, just not now):
Alive and critical:
Repo Responsibilities
Key boundary: timmy-config is the machine. timmy-home is the mind. the-nexus is the body. autolora is the gym.
Strategic Assets to Own
These are the things that give Timmy a moat. Do not outsource them:
SOUL.md / BIRTH.md — The identity. This is what makes Timmy not-a-chatbot. The thin system prompt in BIRTH.md produced real emergence in First Light testing. SOUL.md is the authoritative identity doc. These are the most important files in the project.
Training data and LoRA adapters — The 29 exemplars, the conversation trajectories, the Twitter archive insights, the DPO pairs. This is the private differentiator. Every session Timmy has is potential training signal.
Experience store — SQLite-backed embodied memory. Every perceive→think→act cycle logged. This is Timmy's autobiography and the raw material for self-improvement.
Perception adapter — The translation layer between raw world events and natural-language sensory descriptions. This is what makes Timmy's experience feel lived rather than simulated.
Gitea instance — Self-hosted, sovereign. The entire development history, backlog, and PR record. This is institutional memory.
Commodity Layers to Borrow
Do not build these. Use off-the-shelf:
The 10-line rule enforces this. If a PR adds more than 10 net lines, it's probably homebrewing something that exists.
Operating Model and Agent Workflow
What worked
What repeatedly caused waste
Hidden operational assumptions
Technical Lessons Learned
Hermes / Local Model / Provider Routing
save_trajectories: trueturns every conversation into training data. Do this immediately.morrowind→mwrename (PR #48) fixed 30-iteration error loops.DPO / Training / Eval
Memory, Identity, Prompt vs Weight
Game Portal / Apprenticeship Loop
Gitea / Backlog / PR Process
MCP / Open Source Integration
False Starts / Dead Paths
Don't Do Again
Don't build custom orchestration. The sovereign-orchestration repo (task_queue.py, step_handlers.py — 1750 lines) reinvented Huey/Celery. Verdict: killed. Use Hermes Agent + Huey.
Don't let agents mass-file issues without triage. The old dashboard accumulated 1000+ issues. Most were noise. New rule: every issue must pass the three-concern test (Heartbeat? Harness? Portal?) or it doesn't get filed.
Don't build cloud-first and plan to migrate later. Every cloud integration must have a documented local replacement and a kill date. The "Boost & Bleed" pattern: use cloud to learn the pattern, build local equivalent, run in parallel, cut over, tear down cloud. 60-day timeline per service.
Don't import Hermes into the orchestrator's Python environment. PR #29 did this with sys.path.insert(). It broke because of dependency mismatches (firecrawl). PR #44 fixed it with subprocess isolation. Always run Hermes in its own venv.
Don't use long MCP server names.
morrowind→mwfixed the tool hallucination problem. Short server keys = less room for local models to drift.Don't prioritize visual polish over infrastructure. 150+ visual PRs while heartbeat/harness lagged. The 3D world looks great and has: matrix rain, lightning, weather, holograms, procedural terrain, 3D audio, fireworks, glass floors. None of it matters if the consciousness loop doesn't work.
Don't trust agent PR volume as a quality signal. Claude: 164 PRs, 42% merge rate. Codex: 7 PRs, 100% merge rate. Codex delivered more real value.
Don't use curl for Gitea API calls with complex JSON bodies. Backticks and special characters in review text break shell escaping. Use Python urllib.request.
Dead Subsystems
sovereign-orchestration/repo — killed, replaced by Hermes + HueyTimmy-time-dashboard/repo — superseded by Timmy_Foundation org reposgroq_worker.pyin nexus mind — doesn't belong in think_once(), should be in Archon body system if anywhereOpen Loops
High-value unfinished threads
OpenMW Lua perception scripts. The MCP server parses
=== TIMMY PERCEPTION ===blocks from the OpenMW log, but the Lua scripts that write these blocks need to be installed into the game's scripts directory. Without them, perception is blind.DPO pair generation. Trajectory collection works. The gap: converting corrections into preferred/rejected pairs automatically. timmy-config#5 and #13 track this.
Hermes 4 14B LoRA training. Only 8B has been trained. The 14B model is the sweet spot for M3 Max (enough capability, fits in memory). timmy-config#10 tracks this.
OpenClaw bootstrap. Research done, epic filed (timmy-config#51), not started. Phase 1 is ~30 minutes of install.
Nexus consciousness loop → Hermes routing. the-nexus#540 — route perception through Hermes harness to unify training data. Spec'd but not built.
AgentNet PR #2077 in hermes-agent. Ed25519 identity + micropayments. Directly relevant to Timmy-as-economic-peer. Watch for merge.
Structured audit log PR #1819 in hermes-agent. JSONL + SQLite per-tool-call logging. Richer training signal than current trajectory format. Watch for merge.
Morrowind MCP PRs #48 and #17. Server key rename + externalized config. Open, reviewed, not merged.
Missing docs/specs
Recommended Next Actions
Ordered by strategic importance per #542 (Heartbeat, Harness, Portal).
1. Enable trajectory export NOW
timmy-config#21. Set
save_trajectories: truein ~/.hermes/config.yaml. One line. Every conversation becomes training data. Currently being thrown away.2. Merge Morrowind MCP PRs
timmy-config#48 + timmy-home#17. Server key rename (morrowind → mw) + externalized config. Reviewed and approved. Merge both, restart Hermes. Unblocks Morrowind gameplay.
3. Install OpenMW Lua perception scripts
The MCP server is ready but blind without the Lua scripts writing perception blocks to the log. This is the prerequisite for Timmy actually playing Morrowind.
4. Install OpenClaw
timmy-config#53. ~30 minutes. Point at existing Ollama/hermes4:14b. Gains: persistent memory, multi-channel, cron replacement for agent tasks. Low risk, high leverage.
5. Build DPO pair generator
timmy-config#5, #13. The training pipeline from trajectories → LoRA exists. The gap: trajectories → DPO pairs. This closes the self-improvement loop.
6. Train Hermes 4 14B LoRA
timmy-config#10. The 8B LoRA proved the pipeline. 14B is the M3 Max sweet spot. More capable base = better Timmy.
7. Route Nexus perception through Hermes
the-nexus#540. Unifies the three perception channels (Nexus 3D, Morrowind game, desktop) into one training data format.
8. Register Z3 + SymPy as Hermes tools
timmy-config#35, #41. Standard tool registration, <1 hour each. Unlocks Reasoning-DPO (#37) — automated truth-oracle for DPO pair generation without human labeling.
9. Write a full-stack startup runbook
Document: "Start Ollama, start llama-server on 8081, start Hermes, start Huey workers, start OpenMW, connect MCP server." No agent should have to reverse-engineer this.
10. Batch-close duplicate issues in timmy-config
Issues #30/#31, #32/#33, #34/#35, #36/#37, #38/#39, #40/#41, #42/#43 are all duplicated pairs. Close the even-numbered duplicates.
Appendix: Key References
Repos
Critical Issues
PRs of Note
External Projects
Perplexity Session Work Products
Infrastructure
Closing as archived/ingested. The knowledge transfer is preserved in the issue body and was used during backlog reset; future work will be recreated as narrower final-vision issues instead of keeping this giant transfer artifact open.