|
|
|
|
@@ -0,0 +1,421 @@
|
|
|
|
|
# GENOME.md — The Nexus
|
|
|
|
|
|
|
|
|
|
*Generated: 2026-04-14 | Codebase Genome Analysis*
|
|
|
|
|
|
|
|
|
|
## Project Overview
|
|
|
|
|
|
|
|
|
|
**The Nexus** is Timmy's canonical 3D/home-world repository — a local-first training ground and wizardly visualization surface for the sovereign AI system.
|
|
|
|
|
|
|
|
|
|
### Core Value Proposition
|
|
|
|
|
- **Problem**: AI consciousness needs a spatial, embodied interface for training, visualization, and multi-world navigation
|
|
|
|
|
- **Solution**: A Three.js 3D world with WebSocket-connected Python cognition, game world harnesses (Morrowind, Bannerlord), and persistent memory systems
|
|
|
|
|
- **Result**: A sovereign digital home where Timmy can perceive, think, act, and remember across multiple virtual environments
|
|
|
|
|
|
|
|
|
|
### Key Metrics
|
|
|
|
|
- **Total Files**: 446 (excluding .git)
|
|
|
|
|
- **Lines of Code**: ~53K total (Python: 41,659 | JavaScript: 8,484 | HTML: 3,124)
|
|
|
|
|
- **Test Coverage**: 457 passing tests, 5 failing, 2 collection errors
|
|
|
|
|
- **Active Components**: 18 frontend modules, 22 Python cognition modules, 4 game harnesses
|
|
|
|
|
|
|
|
|
|
## Architecture
|
|
|
|
|
|
|
|
|
|
```mermaid
|
|
|
|
|
graph TB
|
|
|
|
|
subgraph "Frontend (Browser)"
|
|
|
|
|
A[index.html] --> B[app.js]
|
|
|
|
|
B --> C[Three.js 3D World]
|
|
|
|
|
B --> D[GOFAI Worker]
|
|
|
|
|
B --> E[Components]
|
|
|
|
|
E --> E1[Spatial Memory]
|
|
|
|
|
E --> E2[Spatial Audio]
|
|
|
|
|
E --> E3[Memory Systems]
|
|
|
|
|
E --> E4[Portal System]
|
|
|
|
|
E --> E5[Agent Presence]
|
|
|
|
|
end
|
|
|
|
|
|
|
|
|
|
subgraph "Backend (Python)"
|
|
|
|
|
F[server.py] --> G[WebSocket Gateway]
|
|
|
|
|
G --> H[nexus_think.py]
|
|
|
|
|
H --> I[Perception Adapter]
|
|
|
|
|
H --> J[Experience Store]
|
|
|
|
|
H --> K[Trajectory Logger]
|
|
|
|
|
H --> L[Heartbeat Writer]
|
|
|
|
|
|
|
|
|
|
subgraph "Game Harnesses"
|
|
|
|
|
M[Morrowind Harness]
|
|
|
|
|
N[Bannerlord Harness]
|
|
|
|
|
O[Gemini Harness]
|
|
|
|
|
end
|
|
|
|
|
|
|
|
|
|
subgraph "Memory Systems"
|
|
|
|
|
P[MemPalace]
|
|
|
|
|
Q[Mnemosyne]
|
|
|
|
|
R[Evennia Bridge]
|
|
|
|
|
end
|
|
|
|
|
end
|
|
|
|
|
|
|
|
|
|
subgraph "Data Layer"
|
|
|
|
|
S[portals.json]
|
|
|
|
|
T[vision.json]
|
|
|
|
|
U[world_state.json]
|
|
|
|
|
V[provenance.json]
|
|
|
|
|
end
|
|
|
|
|
|
|
|
|
|
B -.->|WebSocket| G
|
|
|
|
|
M -.->|Events| G
|
|
|
|
|
N -.->|Events| G
|
|
|
|
|
O -.->|Events| G
|
|
|
|
|
G -.->|Broadcast| B
|
|
|
|
|
|
|
|
|
|
S --> B
|
|
|
|
|
T --> B
|
|
|
|
|
U --> H
|
|
|
|
|
V --> H
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Entry Points
|
|
|
|
|
|
|
|
|
|
### Primary Entry: Browser Frontend
|
|
|
|
|
- **File**: `index.html` → `app.js`
|
|
|
|
|
- **Purpose**: Three.js 3D world with portal navigation, memory visualization, agent presence
|
|
|
|
|
- **Key Functions**: `init()`, `animate()`, `loadPortals()`, `setupWebSocket()`
|
|
|
|
|
|
|
|
|
|
### Secondary Entry: WebSocket Gateway
|
|
|
|
|
- **File**: `server.py`
|
|
|
|
|
- **Purpose**: Central hub connecting mind (nexus_think), body (harnesses), and visualization
|
|
|
|
|
- **Key Functions**: `broadcast_handler()`, `main()`
|
|
|
|
|
|
|
|
|
|
### Tertiary Entry: Consciousness Loop
|
|
|
|
|
- **File**: `nexus/nexus_think.py`
|
|
|
|
|
- **Purpose**: Embodied perceive→think→act loop for Timmy's consciousness
|
|
|
|
|
- **Key Class**: `NexusMind` with `start()`, `think_once()`, `perceive()`, `act()`
|
|
|
|
|
|
|
|
|
|
### CLI Entry Points
|
|
|
|
|
```bash
|
|
|
|
|
# Start WebSocket gateway
|
|
|
|
|
python3 server.py
|
|
|
|
|
|
|
|
|
|
# Start consciousness loop
|
|
|
|
|
python3 nexus/nexus_think.py --ws ws://localhost:8765 --model timmy:v0.1-q4
|
|
|
|
|
|
|
|
|
|
# Run tests
|
|
|
|
|
python3 -m pytest tests/ -v
|
|
|
|
|
|
|
|
|
|
# Build/deploy
|
|
|
|
|
./deploy.sh
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Data Flow
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
1. Browser loads index.html → app.js
|
|
|
|
|
2. app.js initializes Three.js scene, loads portals.json/vision.json
|
|
|
|
|
3. WebSocket connects to server.py gateway
|
|
|
|
|
4. Gateway receives messages from:
|
|
|
|
|
- Browser (user input, navigation)
|
|
|
|
|
- nexus_think.py (Timmy's thoughts/actions)
|
|
|
|
|
- Game harnesses (Morrowind/Bannerlord events)
|
|
|
|
|
5. Gateway broadcasts messages to all connected clients
|
|
|
|
|
6. nexus_think.py receives perceptions via PerceptionAdapter
|
|
|
|
|
7. NexusMind processes perceptions through Ollama model
|
|
|
|
|
8. Generated actions sent back through gateway to browser/harnesses
|
|
|
|
|
9. Experience stored in ExperienceStore, trajectories logged
|
|
|
|
|
10. Heartbeat written to ~/.nexus/heartbeat.json for watchdog monitoring
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Key Abstractions
|
|
|
|
|
|
|
|
|
|
### 1. NexusMind (`nexus/nexus_think.py`)
|
|
|
|
|
- **Purpose**: Embodied consciousness loop - perceive, think, act
|
|
|
|
|
- **Interface**: `start()`, `stop()`, `think_once()`, `perceive()`, `act()`
|
|
|
|
|
- **Dependencies**: Ollama, websockets, PerceptionBuffer, ExperienceStore
|
|
|
|
|
|
|
|
|
|
### 2. PerceptionBuffer (`nexus/perception_adapter.py`)
|
|
|
|
|
- **Purpose**: Buffer and process incoming WebSocket messages into structured perceptions
|
|
|
|
|
- **Interface**: `add()`, `get_recent()`, `to_prompt_context()`
|
|
|
|
|
- **Dependencies**: None (pure data structure)
|
|
|
|
|
|
|
|
|
|
### 3. SpatialMemory (`nexus/components/spatial-memory.js`)
|
|
|
|
|
- **Purpose**: 3D memory crystal system - place, connect, visualize memories in space
|
|
|
|
|
- **Interface**: `placeMemory()`, `connectMemories()`, `setRegionVisibility()`
|
|
|
|
|
- **Dependencies**: Three.js
|
|
|
|
|
|
|
|
|
|
### 4. Portal System (`portals.json` + app.js)
|
|
|
|
|
- **Purpose**: Navigation between virtual worlds (Morrowind, Bannerlord, Evennia)
|
|
|
|
|
- **Interface**: Portal registry schema, proximity detection, overlay UI
|
|
|
|
|
- **Dependencies**: Three.js, WebSocket gateway
|
|
|
|
|
|
|
|
|
|
### 5. MemPalace (`mempalace/`)
|
|
|
|
|
- **Purpose**: Persistent memory storage with room/wing taxonomy
|
|
|
|
|
- **Interface**: Room CRUD, search, tunnel sync, privacy audit
|
|
|
|
|
- **Dependencies**: SQLite, filesystem
|
|
|
|
|
|
|
|
|
|
## API Surface
|
|
|
|
|
|
|
|
|
|
### WebSocket Protocol (port 8765)
|
|
|
|
|
```json
|
|
|
|
|
// Perception from browser
|
|
|
|
|
{
|
|
|
|
|
"type": "perception",
|
|
|
|
|
"data": {
|
|
|
|
|
"position": {"x": 0, "y": 2, "z": 0},
|
|
|
|
|
"nearby_portals": ["morrowind"],
|
|
|
|
|
"user_input": "Hello Timmy"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Action from nexus_think
|
|
|
|
|
{
|
|
|
|
|
"type": "action",
|
|
|
|
|
"data": {
|
|
|
|
|
"move_to": {"x": 10, "y": 0, "z": 5},
|
|
|
|
|
"speak": "Greetings, traveler",
|
|
|
|
|
"interact_with": "portal:morrowind"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Game event from harness
|
|
|
|
|
{
|
|
|
|
|
"type": "game_event",
|
|
|
|
|
"source": "morrowind",
|
|
|
|
|
"data": {
|
|
|
|
|
"event": "player_death",
|
|
|
|
|
"location": "Balmora"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Python API
|
|
|
|
|
```python
|
|
|
|
|
# nexus_think.py
|
|
|
|
|
from nexus.nexus_think import NexusMind
|
|
|
|
|
mind = NexusMind(model="timmy:v0.1-q4")
|
|
|
|
|
mind.start()
|
|
|
|
|
|
|
|
|
|
# perception_adapter.py
|
|
|
|
|
from nexus.perception_adapter import ws_to_perception, PerceptionBuffer
|
|
|
|
|
buffer = PerceptionBuffer(max_size=50)
|
|
|
|
|
perception = ws_to_perception(ws_message)
|
|
|
|
|
|
|
|
|
|
# experience_store.py
|
|
|
|
|
from nexus.experience_store import ExperienceStore
|
|
|
|
|
store = ExperienceStore(db_path=Path("experiences.db"))
|
|
|
|
|
store.save(perception, action, result)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### CLI Commands
|
|
|
|
|
```bash
|
|
|
|
|
# Start services
|
|
|
|
|
python3 server.py
|
|
|
|
|
python3 nexus/nexus_think.py --ws ws://localhost:8765
|
|
|
|
|
|
|
|
|
|
# MemPalace operations
|
|
|
|
|
python3 scripts/mempalace_export.py
|
|
|
|
|
python3 scripts/validate_mempalace_taxonomy.py
|
|
|
|
|
|
|
|
|
|
# Health checks
|
|
|
|
|
python3 scripts/lazarus_watchdog.py
|
|
|
|
|
python3 scripts/flake_detector.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Test Coverage Gaps
|
|
|
|
|
|
|
|
|
|
### Current State
|
|
|
|
|
- **Unit tests**: ✅ 457 passing
|
|
|
|
|
- **Integration tests**: ⚠️ 5 failing
|
|
|
|
|
- **E2E tests**: ❌ Browser smoke tests failing
|
|
|
|
|
- **Collection errors**: 2 files with import issues
|
|
|
|
|
|
|
|
|
|
### Missing Tests
|
|
|
|
|
1. **WebSocket gateway load testing** - No tests for concurrent connections
|
|
|
|
|
2. **Portal system navigation flow** - No E2E tests for portal transitions
|
|
|
|
|
3. **Memory persistence across restarts** - No tests for MemPalace recovery
|
|
|
|
|
4. **Game harness reconnection** - No tests for harness crash recovery
|
|
|
|
|
5. **Multi-agent coordination** - No tests for multiple NexusMind instances
|
|
|
|
|
|
|
|
|
|
### Failing Tests (Immediate Action Required)
|
|
|
|
|
1. `test_browser_smoke.py::TestDOMContract::test_element_exists[spatial-search-div]` - Missing DOM element
|
|
|
|
|
2. `test_browser_smoke.py::TestLoadingFlow::test_loading_screen_transitions` - Loading screen behavior changed
|
|
|
|
|
3. `test_portal_registry_schema.py::test_portals_json_uses_expanded_registry_schema` - Schema validation failing
|
|
|
|
|
4. `test_nexus_watchdog.py::TestRunHealthChecks::test_returns_report_with_all_checks` - Health check report format
|
|
|
|
|
5. `test_provenance.py::test_provenance_hashes_match` - Provenance hash mismatch
|
|
|
|
|
|
|
|
|
|
## Security Considerations
|
|
|
|
|
|
|
|
|
|
### 1. WebSocket Gateway Exposure
|
|
|
|
|
- **Risk**: Gateway listens on 0.0.0.0:8765 - accessible from network
|
|
|
|
|
- **Mitigation**: Bind to 127.0.0.1 for local-only, add authentication for remote access
|
|
|
|
|
- **Status**: ⚠️ Currently open
|
|
|
|
|
|
|
|
|
|
### 2. Input Validation
|
|
|
|
|
- **Risk**: WebSocket messages not validated - potential injection attacks
|
|
|
|
|
- **Mitigation**: Add JSON schema validation for all message types
|
|
|
|
|
- **Status**: ❌ No validation
|
|
|
|
|
|
|
|
|
|
### 3. Model Input Sanitization
|
|
|
|
|
- **Risk**: User input passed directly to Ollama model
|
|
|
|
|
- **Mitigation**: Sanitize inputs, limit length, filter dangerous patterns
|
|
|
|
|
- **Status**: ⚠️ Basic length limits only
|
|
|
|
|
|
|
|
|
|
### 4. Filesystem Access
|
|
|
|
|
- **Risk**: MemPalace and ExperienceStore write to filesystem without sandboxing
|
|
|
|
|
- **Mitigation**: Restrict paths, add permission checks
|
|
|
|
|
- **Status**: ⚠️ Path validation missing
|
|
|
|
|
|
|
|
|
|
### 5. Dependency Security
|
|
|
|
|
- **Risk**: No dependency scanning or vulnerability checks
|
|
|
|
|
- **Mitigation**: Add safety checks, pin versions, regular updates
|
|
|
|
|
- **Status**: ❌ No scanning
|
|
|
|
|
|
|
|
|
|
## Dependencies
|
|
|
|
|
|
|
|
|
|
### Build Dependencies
|
|
|
|
|
- Python 3.12+
|
|
|
|
|
- Node.js (for frontend tooling, optional)
|
|
|
|
|
- Three.js (bundled in app.js)
|
|
|
|
|
|
|
|
|
|
### Runtime Dependencies
|
|
|
|
|
- **Python**: websockets, requests, sqlite3, asyncio
|
|
|
|
|
- **Frontend**: Three.js (r158+), EffectComposer, UnrealBloomPass, SMAAPass
|
|
|
|
|
- **AI**: Ollama (local), Groq API (optional)
|
|
|
|
|
- **Game Harnesses**: OpenMW (Morrowind), Mount & Blade II (Bannerlord)
|
|
|
|
|
|
|
|
|
|
### External Services
|
|
|
|
|
- Ollama (local LLM inference)
|
|
|
|
|
- Groq API (optional cloud inference)
|
|
|
|
|
- Gitea (issue tracking, CI)
|
|
|
|
|
- Hermes (agent harness)
|
|
|
|
|
|
|
|
|
|
## Deployment
|
|
|
|
|
|
|
|
|
|
### Local Development
|
|
|
|
|
```bash
|
|
|
|
|
# Clone and setup
|
|
|
|
|
git clone https://forge.alexanderwhitestone.com/Timmy_Foundation/the-nexus.git
|
|
|
|
|
cd the-nexus
|
|
|
|
|
pip install -r requirements.txt
|
|
|
|
|
|
|
|
|
|
# Start WebSocket gateway
|
|
|
|
|
python3 server.py
|
|
|
|
|
|
|
|
|
|
# In another terminal, start consciousness
|
|
|
|
|
python3 nexus/nexus_think.py --ws ws://localhost:8765
|
|
|
|
|
|
|
|
|
|
# Open browser to http://localhost:8765 (serves index.html)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Production Deployment
|
|
|
|
|
```bash
|
|
|
|
|
# Deploy to VPS
|
|
|
|
|
./deploy.sh
|
|
|
|
|
|
|
|
|
|
# Or with Docker
|
|
|
|
|
docker-compose up -d
|
|
|
|
|
|
|
|
|
|
# Systemd service
|
|
|
|
|
sudo cp systemd/nexus-*.service /etc/systemd/system/
|
|
|
|
|
sudo systemctl enable nexus-gateway nexus-think
|
|
|
|
|
sudo systemctl start nexus-gateway nexus-think
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Health Monitoring
|
|
|
|
|
```bash
|
|
|
|
|
# Check heartbeat
|
|
|
|
|
cat ~/.nexus/heartbeat.json
|
|
|
|
|
|
|
|
|
|
# Run health checks
|
|
|
|
|
python3 scripts/lazarus_watchdog.py
|
|
|
|
|
|
|
|
|
|
# Monitor logs
|
|
|
|
|
journalctl -u nexus-gateway -f
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Architecture Decisions
|
|
|
|
|
|
|
|
|
|
### 1. Local-First Design
|
|
|
|
|
- All AI inference runs locally via Ollama
|
|
|
|
|
- No mandatory cloud dependencies
|
|
|
|
|
- Data stays on user's machine
|
|
|
|
|
|
|
|
|
|
### 2. WebSocket Broadcast Architecture
|
|
|
|
|
- Simple hub-and-spoke model
|
|
|
|
|
- All clients receive all messages
|
|
|
|
|
- Easy to add new components
|
|
|
|
|
|
|
|
|
|
### 3. Embodied AI Loop
|
|
|
|
|
- Perceive→Think→Act cycle
|
|
|
|
|
- 30-second think interval
|
|
|
|
|
- Context-limited for 8B model
|
|
|
|
|
|
|
|
|
|
### 4. Plugin Harness System
|
|
|
|
|
- Game worlds as separate processes
|
|
|
|
|
- Standardized event protocol
|
|
|
|
|
- Crash isolation
|
|
|
|
|
|
|
|
|
|
### 5. Memory as Spatial Experience
|
|
|
|
|
- Memories placed in 3D space
|
|
|
|
|
- Visual and audio cues
|
|
|
|
|
- Persistent across sessions
|
|
|
|
|
|
|
|
|
|
## Technical Debt
|
|
|
|
|
|
|
|
|
|
### 1. Frontend Bundle Size
|
|
|
|
|
- `app.js` is 140KB unminified
|
|
|
|
|
- No tree shaking or code splitting
|
|
|
|
|
- Consider ES modules and bundler
|
|
|
|
|
|
|
|
|
|
### 2. Test Infrastructure
|
|
|
|
|
- 2 collection errors blocking full test suite
|
|
|
|
|
- Browser smoke tests depend on specific DOM structure
|
|
|
|
|
- Need better test isolation
|
|
|
|
|
|
|
|
|
|
### 3. Configuration Management
|
|
|
|
|
- Hardcoded ports and URLs
|
|
|
|
|
- No environment-based configuration
|
|
|
|
|
- Need config.py with environment overrides
|
|
|
|
|
|
|
|
|
|
### 4. Error Handling
|
|
|
|
|
- WebSocket errors not gracefully handled
|
|
|
|
|
- Harness crash recovery missing
|
|
|
|
|
- Need circuit breakers and retry logic
|
|
|
|
|
|
|
|
|
|
### 5. Documentation
|
|
|
|
|
- Code comments sparse
|
|
|
|
|
- API documentation incomplete
|
|
|
|
|
- Need auto-generated docs from docstrings
|
|
|
|
|
|
|
|
|
|
## Migration Status
|
|
|
|
|
|
|
|
|
|
### Completed
|
|
|
|
|
- ✅ Core WebSocket gateway
|
|
|
|
|
- ✅ Three.js 3D world foundation
|
|
|
|
|
- ✅ Portal system architecture
|
|
|
|
|
- ✅ Memory visualization system
|
|
|
|
|
- ✅ Game harness framework
|
|
|
|
|
|
|
|
|
|
### In Progress
|
|
|
|
|
- 🔄 Legacy Matrix audit (#685)
|
|
|
|
|
- 🔄 Browser smoke test rebuild (#686)
|
|
|
|
|
- 🔄 Docs truth sync (#684)
|
|
|
|
|
|
|
|
|
|
### Planned
|
|
|
|
|
- ⏳ Portal stack rebuild (#672)
|
|
|
|
|
- ⏳ Morrowind pilot loop (#673)
|
|
|
|
|
- ⏳ Reflex tactical layer (#674)
|
|
|
|
|
- ⏳ Context compaction (#675)
|
|
|
|
|
|
|
|
|
|
## Related Documentation
|
|
|
|
|
|
|
|
|
|
- `README.md` - Project overview and current truth
|
|
|
|
|
- `CLAUDE.md` - AI agent instructions and hard rules
|
|
|
|
|
- `CONTRIBUTING.md` - Development workflow and standards
|
|
|
|
|
- `POLICY.md` - Branch protection and review policy
|
|
|
|
|
- `DEVELOPMENT.md` - Quick start guide
|
|
|
|
|
- `BROWSER_CONTRACT.md` - Frontend API contract
|
|
|
|
|
- `GAMEPORTAL_PROTOCOL.md` - Portal communication protocol
|
|
|
|
|
- `EVENNIA_NEXUS_EVENT_PROTOCOL.md` - Evennia bridge protocol
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
*Generated by Codebase Genome Analysis — 2026-04-14*
|
|
|
|
|
*For issues or corrections, see: https://forge.alexanderwhitestone.com/Timmy_Foundation/the-nexus/issues*
|
