d8d976aa60
This commit implements six major features: 1. Event Log System (src/swarm/event_log.py) - SQLite-based audit trail for all swarm events - Task lifecycle tracking (created, assigned, completed, failed) - Agent lifecycle tracking (joined, left, status changes) - Integrated with coordinator for automatic logging - Dashboard page at /swarm/events 2. Lightning Ledger (src/lightning/ledger.py) - Transaction tracking for Lightning Network payments - Balance calculations (incoming, outgoing, net, available) - Integrated with payment_handler for automatic logging - Dashboard page at /lightning/ledger 3. Semantic Memory / Vector Store (src/memory/vector_store.py) - Embedding-based similarity search for Echo agent - Fallback to keyword matching if sentence-transformers unavailable - Personal facts storage and retrieval - Dashboard page at /memory 4. Cascade Router Integration (src/timmy/cascade_adapter.py) - Automatic LLM failover between providers (Ollama → AirLLM → API) - Circuit breaker pattern for failing providers - Metrics tracking per provider (latency, error rates) - Dashboard status page at /router/status 5. Self-Upgrade Approval Queue (src/upgrades/) - State machine for self-modifications: proposed → approved/rejected → applied/failed - Human approval required before applying changes - Git integration for branch management - Dashboard queue at /self-modify/queue 6. Real-Time Activity Feed (src/events/broadcaster.py) - WebSocket-based live activity streaming - Bridges event_log to dashboard clients - Activity panel on /swarm/live Tests: - 101 unit tests passing - 4 new E2E test files for Selenium testing - Run with: SELENIUM_UI=1 pytest tests/functional/ -v --headed Documentation: - 6 ADRs (017-022) documenting architecture decisions - Implementation summary in docs/IMPLEMENTATION_SUMMARY.md - Architecture diagram in docs/architecture-v2.md
12 KiB
12 KiB
Timmy Time Architecture v2
Overview
This document describes how the 6 new features integrate with the existing architecture.
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────────────┐
│ DASHBOARD UI │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Event Log │ │ Ledger │ │ Memory │ │ Upgrade Queue │ │
│ │ /swarm/events│ │/lightning/ledger│ │ /memory │ │ /self-modify/queue│ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └────────┬─────────┘ │
│ │ │ │ │ │
│ ┌──────┴───────┐ ┌──────┴───────┐ ┌──────┴───────┐ ┌────────┴─────────┐ │
│ │ WebSocket │ │ │ │ │ │ Real-Time │ │
│ │ Activity │ │ │ │ │ │ Activity Feed │ │
│ │ Feed │ │ │ │ │ │ │ │
│ └──────┬───────┘ └──────────────┘ └──────────────┘ └──────────────────┘ │
└─────────┼───────────────────────────────────────────────────────────────────┘
│ WebSocket
┌─────────┼───────────────────────────────────────────────────────────────────┐
│ │ API LAYER │
│ ┌──────┴───────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Events │ │ Ledger │ │ Memory │ │ Self-Modify │ │
│ │ Routes │ │ Routes │ │ Routes │ │ Routes │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └────────┬─────────┘ │
└─────────┼────────────────┼────────────────┼──────────────────┼─────────────┘
│ │ │ │
┌─────────┼────────────────┼────────────────┼──────────────────┼─────────────┐
│ │ CORE SERVICES │
│ │ │ │ │ │
│ ┌──────┴───────┐ ┌──────┴───────┐ ┌──────┴───────┐ ┌────────┴─────────┐ │
│ │ Event Log │ │ Ledger │ │Vector Store │ │ Self-Modify Loop │ │
│ │ Service │ │ Service │ │ (Echo) │ │ with Queue │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └────────┬─────────┘ │
│ │ │ │ │ │
│ └────────────────┴────────────────┴──────────────────┘ │
│ │ │
│ ┌─────┴─────┐ │
│ │ SQLite DB │ │
│ │ swarm.db │ │
│ └───────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ CASCADE ROUTER (New) │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌─────────────────────┐ │ │
│ │ │ Ollama │→ │ AirLLM │→ │ API │→ │ Metrics & Health │ │ │
│ │ │(local) │ │ (local) │ │(optional)│ │ Dashboard │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ └─────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────┴─────┐ │
│ │ Timmy │ │
│ │ Agent │ │
│ └───────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
Data Flow
1. Event Log System
Coordinator Action → log_event() → SQLite event_log table
↓
WebSocket Broadcast (ADR-022)
↓
Dashboard Activity Feed
2. Lightning Ledger
Payment Handler → create_invoice_entry() → SQLite ledger table
↓
mark_settled()
↓
Dashboard /lightning/ledger
3. Semantic Memory
Conversation → store_memory() → SQLite memory_entries (with embedding)
↓
search_memories(query)
↓
Dashboard /memory
4. Self-Upgrade Queue
Self-Modify Loop → Propose Change → SQLite upgrades table (status: proposed)
↓
Dashboard Review
↓
Approve → Apply → Git Commit
or
Reject → Cleanup
5. Cascade Router
User Request → Cascade Router → Ollama (try)
↓ fail
AirLLM (fallback)
↓ fail
API Provider (optional)
↓
Metrics Tracking
↓
Dashboard /router/status
6. Real-Time Activity Feed
Event Logged → EventBroadcaster → ws_manager.broadcast()
↓
WebSocket Clients
↓
Dashboard Activity Panel
Database Schema
Tables
| Table | Purpose | Feature |
|---|---|---|
tasks |
Task management | Existing |
agents |
Agent registry | Existing |
event_log |
Audit trail | New - ADR-017 |
ledger |
Lightning payments | New - ADR-018 |
memory_entries |
Semantic memory | New - ADR-019 |
upgrades |
Self-mod queue | New - ADR-021 |
provider_metrics |
LLM metrics | New - ADR-020 |
Integration Points
Existing → New
| Existing Component | Integrates With | How |
|---|---|---|
coordinator.py |
Event Log | Calls log_event() for all lifecycle events |
payment_handler.py |
Ledger | Creates entries on invoice/settlement |
self_modify/loop.py |
Upgrade Queue | Stops at proposal, waits for approval |
timmy/agent.py |
Cascade Router | Uses router instead of direct backends |
ws_manager/handler.py |
Activity Feed | Broadcasts events to clients |
New → Existing
| New Component | Uses Existing | How |
|---|---|---|
| Event Log | coordinator.py |
Receives all coordinator actions |
| Ledger | payment_handler.py |
Integrated into invoice lifecycle |
| Memory | Personas | Echo agent queries for context |
| Upgrade Queue | self_modify/loop.py |
Controls when changes apply |
| Cascade Router | timmy/agent.py |
Provides LLM abstraction |
| Activity Feed | ws_manager/handler.py |
Uses WebSocket infrastructure |
Implementation Order
Phase 1: Data Layer (Done)
- ✅ Event Log table + integration
- ✅ Ledger table + integration
- ✅ Vector store table + functions
Phase 2: UI Layer (Done)
- ✅ Event Log dashboard page
- ✅ Ledger dashboard page
- ✅ Memory browser page
Phase 3: Advanced Features (Planned)
-
📝 Cascade Router integration (ADR-020)
- Create adapter layer
- Modify Timmy agent
- Provider status dashboard
-
📝 Self-Upgrade Queue (ADR-021)
- Create
upgradestable - Modify self-modify loop
- Dashboard queue UI
- Create
-
📝 Real-Time Activity Feed (ADR-022)
- EventBroadcaster bridge
- WebSocket integration
- Activity feed panel
Phase 4: Testing
- Unit tests for each service
- E2E tests for full workflows
- Load testing for WebSocket connections
Configuration
New config options in config.py:
# Cascade Router
cascade_providers: list[ProviderConfig]
circuit_breaker_threshold: int = 5
# Self-Upgrade
auto_approve_upgrades: bool = False
upgrade_timeout_hours: int = 24
# Activity Feed
websocket_event_throttle: int = 10 # events/sec
activity_feed_buffer: int = 100 # events to buffer
Security Considerations
| Feature | Risk | Mitigation |
|---|---|---|
| Event Log | Log injection | Sanitize all data fields |
| Ledger | Payment forgery | Verify with Lightning node |
| Memory | Data exposure | Filter by user permissions |
| Upgrade Queue | Unauthorized changes | Require approval, audit log |
| Cascade Router | API key exposure | Use environment variables |
| Activity Feed | Data leak | Authenticate WebSocket |