docs: integration plan for Timmy Dashboard + Agent Token Economy

Covers: - WebSocket gateway adapter (backend → Matrix protocol) - LiveWebSocket client to replace MockWebSocket - Agent token economy display (balances, costs, treasury) - 3D economy visuals (budget stress glow, sat flow particles) - 5-phase implementation roadmap with file change matrix - Open questions on auth, serving model, multi-operator
2026-03-18 22:39:13 +00:00
parent fdfae19956
commit b0fb85dac3
1 changed files with 355 additions and 0 deletions
--- a/INTEGRATION.md
+++ b/INTEGRATION.md
@@ -0,0 +1,355 @@
+# Integration Plan: The Matrix × Timmy Time
+
+**Date:** 2026-03-18
+**Author:** Perplexity Computer
+**Status:** Draft — ready for dev team review
+
+---
+
+## Overview
+
+The Matrix is a standalone 3D world that visualizes and commands the Timmy Time agent swarm. It currently runs on a mock WebSocket. This document defines how it integrates with the two other modes of the Timmy Time system:
+
+1. **Timmy Dashboard** — the existing FastAPI/HTMX mission control UI (`/home`, `/briefing`, `/tasks`, `/swarm`, `/spark`, `/market`, `/tools`, etc.)
+2. **Agent Token Economy** — the Lightning Network-based economic layer where agents earn/spend sats for task execution
+
+The Matrix does not replace either system. It is a third interface — a spatial, persistent, iPad-native way to interact with the same backend.
+
+---
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    TIMMY TIME BACKEND                    │
+│                                                         │
+│  FastAPI ─── SQLite ─── Ollama ─── Lightning (LND/CLN)  │
+│      │           │          │            │              │
+│      ▼           ▼          ▼            ▼              │
+│  /api/tasks  /api/agents  /api/chat  /api/economy       │
+│      │           │          │            │              │
+│      └───────────┴──────────┴────────────┘              │
+│                      │                                  │
+│              WebSocket Gateway                          │
+│              ws://tower:8080/ws                          │
+│                      │                                  │
+└──────────────────────┼──────────────────────────────────┘
+                       │
+          ┌────────────┼────────────┐
+          │            │            │
+          ▼            ▼            ▼
+   ┌──────────┐ ┌──────────┐ ┌──────────┐
+   │  Timmy   │ │   The    │ │ Telegram │
+   │Dashboard │ │  Matrix  │ │   Bot    │
+   │(HTMX/WS)│ │(Three.js)│ │ (bridge) │
+   └──────────┘ └──────────┘ └──────────┘
+```
+
+All three interfaces share the same backend. The Matrix connects via the WebSocket gateway, same as the dashboard's live feeds.
+
+---
+
+## Mode 1: Timmy Dashboard Integration
+
+### What Already Exists (Dashboard)
+
+- `/tasks` — three-column queue (Pending, Running, Completed) with Approve/Modify/Veto
+- `/swarm` — agent cards with status, live WebSocket log
+- `/briefing` — morning briefing with task summaries
+- `/home` — agent overview with TASK buttons
+- SQLite `tasks` table with full lifecycle (pending_approval → approved → running → completed/vetoed/failed)
+- WebSocket live feeds for task updates and agent state
+
+### What The Matrix Needs
+
+The Matrix already speaks the right protocol (defined in `PROTOCOL.md`). The integration points:
+
+#### 1. WebSocket Gateway Adapter
+
+**File:** `backend/ws_gateway.py` (new)
+
+The backend needs a WebSocket endpoint that translates between the existing FastAPI internals and the Matrix protocol format:
+
+| Matrix Message (Client → Server) | Backend Action |
+|---|---|
+| `{"type": "chat_message", "agent_id": "timmy", "content": "..."}` | Forward to agent chat endpoint, return streaming response |
+| `{"type": "task_action", "task_id": "...", "action": "approve"}` | Call `PATCH /api/tasks/{id}/approve` |
+| `{"type": "task_action", "task_id": "...", "action": "veto"}` | Call `PATCH /api/tasks/{id}/veto` |
+| `{"type": "subscribe", "channels": ["agents", "tasks"]}` | Register client for push events |
+
+| Matrix Message (Server → Client) | Backend Trigger |
+|---|---|
+| `{"type": "agent_state", "agent_id": "timmy", "state": "working", ...}` | Agent state change in swarm coordinator |
+| `{"type": "task_created", "task_id": "...", ...}` | New row in tasks table |
+| `{"type": "task_update", "task_id": "...", "status": "completed"}` | Task status change |
+| `{"type": "agent_message", "agent_id": "timmy", "content": "..."}` | Agent chat response (streamed) |
+| `{"type": "memory_event", "agent_id": "...", ...}` | New memory entry logged |
+| `{"type": "system_status", ...}` | Periodic heartbeat (every 5s) |
+
+**Implementation:** Wrap the existing WebSocket infrastructure (same one powering `/swarm` live log) with a JSON message router that speaks the Matrix protocol.
+
+#### 2. Replace MockWebSocket
+
+**File:** `js/websocket.js` → `js/websocket-live.js`
+
+```javascript
+// Current: MockWebSocket simulates everything locally
+// Target: Real WebSocket to backend
+
+export class LiveWebSocket {
+  constructor(url) {
+    this.url = url || `ws://${location.hostname}:8080/ws/matrix`;
+    this.ws = null;
+    this.handlers = {};
+    this.reconnectDelay = 1000;
+    this._connect();
+  }
+
+  _connect() {
+    this.ws = new WebSocket(this.url);
+    this.ws.onmessage = (e) => {
+      const msg = JSON.parse(e.data);
+      this._dispatch(msg);
+    };
+    this.ws.onclose = () => {
+      setTimeout(() => this._connect(), this.reconnectDelay);
+      this.reconnectDelay = Math.min(this.reconnectDelay * 2, 30000);
+    };
+    this.ws.onopen = () => {
+      this.reconnectDelay = 1000;
+      this.send({ type: 'subscribe', channels: ['agents', 'tasks', 'system'] });
+    };
+  }
+
+  send(msg) {
+    if (this.ws?.readyState === WebSocket.OPEN) {
+      this.ws.send(JSON.stringify(msg));
+    }
+  }
+  // ... event handling same interface as MockWebSocket
+}
+```
+
+**Key:** The `LiveWebSocket` must expose the exact same interface as `MockWebSocket` — same `.on()`, `.send()`, `.getAgent()`, `.getAgentTasks()` methods. This keeps `ui.js`, `main.js`, and all other files untouched.
+
+#### 3. Connection Config
+
+**File:** `js/config.js` (new)
+
+```javascript
+export const CONFIG = {
+  // Auto-detect: if served from the backend, use relative WS
+  // If standalone, prompt or use saved endpoint
+  WS_URL: localStorage.getItem('matrix_ws_url')
+    || (location.port === '8080' ? `ws://${location.host}/ws/matrix` : null),
+
+  // Fallback to mock mode for demo/development
+  USE_MOCK: false,
+};
+```
+
+**`main.js` change:** Import config, instantiate `LiveWebSocket` or `MockWebSocket` based on `CONFIG.USE_MOCK`.
+
+#### 4. Agent Registry Sync
+
+Current: 4 agents hardcoded in `AGENT_DEFS` (`websocket.js`)
+
+Target: Agents discovered from backend at connect time. The handshake response should include:
+
+```json
+{
+  "type": "connection",
+  "status": "connected",
+  "agents": {
+    "timmy": { "name": "Timmy", "role": "Main Orchestrator", "color": "#00ff41" },
+    "forge": { "name": "Forge", "role": "Builder Agent", "color": "#ff8c00" },
+    ...
+  }
+}
+```
+
+This lets the backend define which agents exist without changing frontend code.
+
+---
+
+## Mode 2: Agent Token Economy Integration
+
+### What the Token Economy Is
+
+From the Timmy Time spec:
+- Agents earn sats for completing tasks
+- Agents spend sats to use tools, call APIs (L402 gating)
+- The human operator sets budgets and approves high-cost operations
+- Lightning Network is the settlement layer
+- Per-agent sat limits and daily global caps
+
+### Integration Points
+
+#### 1. Economy Data in Agent State
+
+Extend the `agent_state` message to include economic fields:
+
+```json
+{
+  "type": "agent_state",
+  "agent_id": "forge",
+  "state": "working",
+  "glow_intensity": 0.8,
+  "economy": {
+    "balance_sats": 15420,
+    "earned_today": 8500,
+    "spent_today": 3200,
+    "daily_limit": 50000,
+    "pending_invoices": 2
+  }
+}
+```
+
+#### 2. Status Tab — Economy Section
+
+Add economy rows to the Status tab when data is present:
+
+| Label | Value |
+|---|---|
+| BALANCE | ⚡ 15,420 sats |
+| EARNED TODAY | ⚡ 8,500 |
+| SPENT TODAY | ⚡ 3,200 |
+| DAILY LIMIT | ⚡ 50,000 |
+| PENDING | 2 invoices |
+
+Style the lightning bolt in `--matrix-warning` (orange) for visual pop.
+
+#### 3. Task Cost Display
+
+Extend task items in the Tasks tab to show cost:
+
+```json
+{
+  "task_id": "...",
+  "title": "Run sentiment analysis",
+  "status": "pending_approval",
+  "priority": "normal",
+  "estimated_cost_sats": 500,
+  "agent_id": "seer"
+}
+```
+
+The Approve/Veto buttons become more meaningful when you see: "This task will cost ⚡ 500 sats."
+
+#### 4. Core System Panel — Treasury
+
+The Core (system status) panel should show aggregate economy:
+
+| Label | Value |
+|---|---|
+| TREASURY | ⚡ 142,800 sats |
+| DAILY BURN | ⚡ 12,400 |
+| DAILY EARN | ⚡ 18,200 |
+| NET TODAY | +⚡ 5,800 |
+| INVOICES PAID | 47 |
+| INVOICES PENDING | 3 |
+
+#### 5. Visual Economy Indicators in 3D World
+
+**Agent glow color shift:** Agents burning through their budget fast could shift from green → orange → red glow. This is already supported — `glow_intensity` controls brightness, and the agent's base color is set per-agent. Add a `budget_stress` field (0.0–1.0) that blends the glow toward red.
+
+**Task object color:** Floating task cubes near agents could be colored by cost:
+- Free / low cost → green
+- Medium cost → orange
+- High cost → red
+- Pending approval → pulsing white
+
+**Connection lines:** When agent A is paying agent B (inter-agent economy), the connection line between them could pulse brighter, with small particles flowing along it to represent sat flow.
+
+#### 6. New Message Types
+
+| Message | Direction | Purpose |
+|---|---|---|
+| `economy_update` | Server → Client | Agent balance changed |
+| `invoice_created` | Server → Client | Agent generated a Lightning invoice |
+| `invoice_paid` | Server → Client | Invoice settled |
+| `budget_alert` | Server → Client | Agent approaching daily limit |
+| `set_budget` | Client → Server | Operator adjusts agent's daily limit |
+
+---
+
+## Implementation Phases
+
+### Phase 1: Live WebSocket (Backend Team)
+**Effort:** 1–2 days
+- [ ] Create `ws_gateway.py` with Matrix protocol router
+- [ ] Wire to existing task and agent state events
+- [ ] Agent chat forwarding (Ollama responses streamed as `agent_message`)
+- [ ] Handshake with dynamic agent registry
+
+### Phase 2: Frontend Switch (Matrix Repo)
+**Effort:** 1 day
+- [ ] Create `js/websocket-live.js` with same interface as mock
+- [ ] Create `js/config.js` with endpoint config + mock fallback
+- [ ] Update `js/main.js` to use config
+- [ ] Test with backend WebSocket
+
+### Phase 3: Economy Display (Both)
+**Effort:** 2–3 days
+- [ ] Backend: Add economy fields to agent_state and task messages
+- [ ] Frontend: Extend Status tab with economy rows
+- [ ] Frontend: Add cost display to task items
+- [ ] Frontend: Extend Core panel with treasury view
+- [ ] Style lightning amounts with ⚡ prefix and orange accent
+
+### Phase 4: 3D Economy Visuals (Matrix Repo)
+**Effort:** 2–3 days
+- [ ] Budget stress → glow color blending in `agents.js`
+- [ ] Task object coloring by cost
+- [ ] Sat flow particles on connection lines
+- [ ] Budget alert toast/notification overlay
+
+### Phase 5: Full Economy Controls (Both)
+**Effort:** 2–3 days
+- [ ] Set budget from panel UI
+- [ ] Invoice approval flow in Tasks tab
+- [ ] Transaction history in Memory tab (reuse memory UI for econ log)
+- [ ] Emergency kill switch — freeze all agent spending from Core panel
+
+---
+
+## File Changes Summary
+
+| File | Change | Phase |
+|---|---|---|
+| `js/websocket.js` | Keep as-is (mock mode) | — |
+| `js/websocket-live.js` | New — real WebSocket client | 2 |
+| `js/config.js` | New — connection config | 2 |
+| `js/main.js` | Import config, conditional WS | 2 |
+| `js/ui.js` | Economy rows in Status, cost in Tasks, treasury in Core | 3 |
+| `js/agents.js` | Budget stress glow, task cost coloring, sat flow particles | 4 |
+| `style.css` | Economy styling, lightning bolt accent | 3 |
+| `PROTOCOL.md` | Add economy message types | 3 |
+| `backend/ws_gateway.py` | New — WebSocket gateway (Timmy Time repo) | 1 |
+
+---
+
+## Open Questions
+
+1. **Serving model:** Should The Matrix be served by the FastAPI backend (as a static route), or stay standalone and connect remotely? Standalone is cleaner for iPad PWA deployment but needs CORS config on the backend WS.
+
+2. **Authentication:** The Matrix WS should require auth before accepting commands. Options:
+   - Token in WS URL query param: `ws://tower:8080/ws/matrix?token=...`
+   - L402 auth (pay-to-connect, on-brand with the sovereignty model)
+   - Simple shared secret for now, upgrade to L402 later
+
+3. **Multi-operator:** If multiple iPads connect, should they see each other's cursors/selections? Not needed for v1 but worth designing for.
+
+4. **Agent hot-add:** When a new agent joins the swarm at runtime, the backend should push an `agent_joined` event so The Matrix can spawn the 3D avatar dynamically without reload.
+
+---
+
+## Dependencies
+
+- Timmy Time backend must expose a WebSocket endpoint speaking the protocol in `PROTOCOL.md`
+- Lightning integration (LND or CLN) must expose balance/invoice APIs to the backend
+- The Matrix has zero backend dependencies — it's pure static files + WebSocket
+
+---
+
+*This document lives in the `perplexity/the-matrix` repo. Backend implementation tasks should be tracked in the main Timmy Time repo.*