Compare commits
1 Commits
fix/1436
...
burn/672-1
| Author | SHA1 | Date | |
|---|---|---|---|
|
ca9cf76469 |
421
GENOME.md
Normal file
421
GENOME.md
Normal file
@@ -0,0 +1,421 @@
|
||||
# GENOME.md — The Nexus
|
||||
|
||||
*Generated: 2026-04-14 | Codebase Genome Analysis*
|
||||
|
||||
## Project Overview
|
||||
|
||||
**The Nexus** is Timmy's canonical 3D/home-world repository — a local-first training ground and wizardly visualization surface for the sovereign AI system.
|
||||
|
||||
### Core Value Proposition
|
||||
- **Problem**: AI consciousness needs a spatial, embodied interface for training, visualization, and multi-world navigation
|
||||
- **Solution**: A Three.js 3D world with WebSocket-connected Python cognition, game world harnesses (Morrowind, Bannerlord), and persistent memory systems
|
||||
- **Result**: A sovereign digital home where Timmy can perceive, think, act, and remember across multiple virtual environments
|
||||
|
||||
### Key Metrics
|
||||
- **Total Files**: 446 (excluding .git)
|
||||
- **Lines of Code**: ~53K total (Python: 41,659 | JavaScript: 8,484 | HTML: 3,124)
|
||||
- **Test Coverage**: 457 passing tests, 5 failing, 2 collection errors
|
||||
- **Active Components**: 18 frontend modules, 22 Python cognition modules, 4 game harnesses
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Frontend (Browser)"
|
||||
A[index.html] --> B[app.js]
|
||||
B --> C[Three.js 3D World]
|
||||
B --> D[GOFAI Worker]
|
||||
B --> E[Components]
|
||||
E --> E1[Spatial Memory]
|
||||
E --> E2[Spatial Audio]
|
||||
E --> E3[Memory Systems]
|
||||
E --> E4[Portal System]
|
||||
E --> E5[Agent Presence]
|
||||
end
|
||||
|
||||
subgraph "Backend (Python)"
|
||||
F[server.py] --> G[WebSocket Gateway]
|
||||
G --> H[nexus_think.py]
|
||||
H --> I[Perception Adapter]
|
||||
H --> J[Experience Store]
|
||||
H --> K[Trajectory Logger]
|
||||
H --> L[Heartbeat Writer]
|
||||
|
||||
subgraph "Game Harnesses"
|
||||
M[Morrowind Harness]
|
||||
N[Bannerlord Harness]
|
||||
O[Gemini Harness]
|
||||
end
|
||||
|
||||
subgraph "Memory Systems"
|
||||
P[MemPalace]
|
||||
Q[Mnemosyne]
|
||||
R[Evennia Bridge]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph "Data Layer"
|
||||
S[portals.json]
|
||||
T[vision.json]
|
||||
U[world_state.json]
|
||||
V[provenance.json]
|
||||
end
|
||||
|
||||
B -.->|WebSocket| G
|
||||
M -.->|Events| G
|
||||
N -.->|Events| G
|
||||
O -.->|Events| G
|
||||
G -.->|Broadcast| B
|
||||
|
||||
S --> B
|
||||
T --> B
|
||||
U --> H
|
||||
V --> H
|
||||
```
|
||||
|
||||
## Entry Points
|
||||
|
||||
### Primary Entry: Browser Frontend
|
||||
- **File**: `index.html` → `app.js`
|
||||
- **Purpose**: Three.js 3D world with portal navigation, memory visualization, agent presence
|
||||
- **Key Functions**: `init()`, `animate()`, `loadPortals()`, `setupWebSocket()`
|
||||
|
||||
### Secondary Entry: WebSocket Gateway
|
||||
- **File**: `server.py`
|
||||
- **Purpose**: Central hub connecting mind (nexus_think), body (harnesses), and visualization
|
||||
- **Key Functions**: `broadcast_handler()`, `main()`
|
||||
|
||||
### Tertiary Entry: Consciousness Loop
|
||||
- **File**: `nexus/nexus_think.py`
|
||||
- **Purpose**: Embodied perceive→think→act loop for Timmy's consciousness
|
||||
- **Key Class**: `NexusMind` with `start()`, `think_once()`, `perceive()`, `act()`
|
||||
|
||||
### CLI Entry Points
|
||||
```bash
|
||||
# Start WebSocket gateway
|
||||
python3 server.py
|
||||
|
||||
# Start consciousness loop
|
||||
python3 nexus/nexus_think.py --ws ws://localhost:8765 --model timmy:v0.1-q4
|
||||
|
||||
# Run tests
|
||||
python3 -m pytest tests/ -v
|
||||
|
||||
# Build/deploy
|
||||
./deploy.sh
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
1. Browser loads index.html → app.js
|
||||
2. app.js initializes Three.js scene, loads portals.json/vision.json
|
||||
3. WebSocket connects to server.py gateway
|
||||
4. Gateway receives messages from:
|
||||
- Browser (user input, navigation)
|
||||
- nexus_think.py (Timmy's thoughts/actions)
|
||||
- Game harnesses (Morrowind/Bannerlord events)
|
||||
5. Gateway broadcasts messages to all connected clients
|
||||
6. nexus_think.py receives perceptions via PerceptionAdapter
|
||||
7. NexusMind processes perceptions through Ollama model
|
||||
8. Generated actions sent back through gateway to browser/harnesses
|
||||
9. Experience stored in ExperienceStore, trajectories logged
|
||||
10. Heartbeat written to ~/.nexus/heartbeat.json for watchdog monitoring
|
||||
```
|
||||
|
||||
## Key Abstractions
|
||||
|
||||
### 1. NexusMind (`nexus/nexus_think.py`)
|
||||
- **Purpose**: Embodied consciousness loop - perceive, think, act
|
||||
- **Interface**: `start()`, `stop()`, `think_once()`, `perceive()`, `act()`
|
||||
- **Dependencies**: Ollama, websockets, PerceptionBuffer, ExperienceStore
|
||||
|
||||
### 2. PerceptionBuffer (`nexus/perception_adapter.py`)
|
||||
- **Purpose**: Buffer and process incoming WebSocket messages into structured perceptions
|
||||
- **Interface**: `add()`, `get_recent()`, `to_prompt_context()`
|
||||
- **Dependencies**: None (pure data structure)
|
||||
|
||||
### 3. SpatialMemory (`nexus/components/spatial-memory.js`)
|
||||
- **Purpose**: 3D memory crystal system - place, connect, visualize memories in space
|
||||
- **Interface**: `placeMemory()`, `connectMemories()`, `setRegionVisibility()`
|
||||
- **Dependencies**: Three.js
|
||||
|
||||
### 4. Portal System (`portals.json` + app.js)
|
||||
- **Purpose**: Navigation between virtual worlds (Morrowind, Bannerlord, Evennia)
|
||||
- **Interface**: Portal registry schema, proximity detection, overlay UI
|
||||
- **Dependencies**: Three.js, WebSocket gateway
|
||||
|
||||
### 5. MemPalace (`mempalace/`)
|
||||
- **Purpose**: Persistent memory storage with room/wing taxonomy
|
||||
- **Interface**: Room CRUD, search, tunnel sync, privacy audit
|
||||
- **Dependencies**: SQLite, filesystem
|
||||
|
||||
## API Surface
|
||||
|
||||
### WebSocket Protocol (port 8765)
|
||||
```json
|
||||
// Perception from browser
|
||||
{
|
||||
"type": "perception",
|
||||
"data": {
|
||||
"position": {"x": 0, "y": 2, "z": 0},
|
||||
"nearby_portals": ["morrowind"],
|
||||
"user_input": "Hello Timmy"
|
||||
}
|
||||
}
|
||||
|
||||
// Action from nexus_think
|
||||
{
|
||||
"type": "action",
|
||||
"data": {
|
||||
"move_to": {"x": 10, "y": 0, "z": 5},
|
||||
"speak": "Greetings, traveler",
|
||||
"interact_with": "portal:morrowind"
|
||||
}
|
||||
}
|
||||
|
||||
// Game event from harness
|
||||
{
|
||||
"type": "game_event",
|
||||
"source": "morrowind",
|
||||
"data": {
|
||||
"event": "player_death",
|
||||
"location": "Balmora"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Python API
|
||||
```python
|
||||
# nexus_think.py
|
||||
from nexus.nexus_think import NexusMind
|
||||
mind = NexusMind(model="timmy:v0.1-q4")
|
||||
mind.start()
|
||||
|
||||
# perception_adapter.py
|
||||
from nexus.perception_adapter import ws_to_perception, PerceptionBuffer
|
||||
buffer = PerceptionBuffer(max_size=50)
|
||||
perception = ws_to_perception(ws_message)
|
||||
|
||||
# experience_store.py
|
||||
from nexus.experience_store import ExperienceStore
|
||||
store = ExperienceStore(db_path=Path("experiences.db"))
|
||||
store.save(perception, action, result)
|
||||
```
|
||||
|
||||
### CLI Commands
|
||||
```bash
|
||||
# Start services
|
||||
python3 server.py
|
||||
python3 nexus/nexus_think.py --ws ws://localhost:8765
|
||||
|
||||
# MemPalace operations
|
||||
python3 scripts/mempalace_export.py
|
||||
python3 scripts/validate_mempalace_taxonomy.py
|
||||
|
||||
# Health checks
|
||||
python3 scripts/lazarus_watchdog.py
|
||||
python3 scripts/flake_detector.py
|
||||
```
|
||||
|
||||
## Test Coverage Gaps
|
||||
|
||||
### Current State
|
||||
- **Unit tests**: ✅ 457 passing
|
||||
- **Integration tests**: ⚠️ 5 failing
|
||||
- **E2E tests**: ❌ Browser smoke tests failing
|
||||
- **Collection errors**: 2 files with import issues
|
||||
|
||||
### Missing Tests
|
||||
1. **WebSocket gateway load testing** - No tests for concurrent connections
|
||||
2. **Portal system navigation flow** - No E2E tests for portal transitions
|
||||
3. **Memory persistence across restarts** - No tests for MemPalace recovery
|
||||
4. **Game harness reconnection** - No tests for harness crash recovery
|
||||
5. **Multi-agent coordination** - No tests for multiple NexusMind instances
|
||||
|
||||
### Failing Tests (Immediate Action Required)
|
||||
1. `test_browser_smoke.py::TestDOMContract::test_element_exists[spatial-search-div]` - Missing DOM element
|
||||
2. `test_browser_smoke.py::TestLoadingFlow::test_loading_screen_transitions` - Loading screen behavior changed
|
||||
3. `test_portal_registry_schema.py::test_portals_json_uses_expanded_registry_schema` - Schema validation failing
|
||||
4. `test_nexus_watchdog.py::TestRunHealthChecks::test_returns_report_with_all_checks` - Health check report format
|
||||
5. `test_provenance.py::test_provenance_hashes_match` - Provenance hash mismatch
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### 1. WebSocket Gateway Exposure
|
||||
- **Risk**: Gateway listens on 0.0.0.0:8765 - accessible from network
|
||||
- **Mitigation**: Bind to 127.0.0.1 for local-only, add authentication for remote access
|
||||
- **Status**: ⚠️ Currently open
|
||||
|
||||
### 2. Input Validation
|
||||
- **Risk**: WebSocket messages not validated - potential injection attacks
|
||||
- **Mitigation**: Add JSON schema validation for all message types
|
||||
- **Status**: ❌ No validation
|
||||
|
||||
### 3. Model Input Sanitization
|
||||
- **Risk**: User input passed directly to Ollama model
|
||||
- **Mitigation**: Sanitize inputs, limit length, filter dangerous patterns
|
||||
- **Status**: ⚠️ Basic length limits only
|
||||
|
||||
### 4. Filesystem Access
|
||||
- **Risk**: MemPalace and ExperienceStore write to filesystem without sandboxing
|
||||
- **Mitigation**: Restrict paths, add permission checks
|
||||
- **Status**: ⚠️ Path validation missing
|
||||
|
||||
### 5. Dependency Security
|
||||
- **Risk**: No dependency scanning or vulnerability checks
|
||||
- **Mitigation**: Add safety checks, pin versions, regular updates
|
||||
- **Status**: ❌ No scanning
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Build Dependencies
|
||||
- Python 3.12+
|
||||
- Node.js (for frontend tooling, optional)
|
||||
- Three.js (bundled in app.js)
|
||||
|
||||
### Runtime Dependencies
|
||||
- **Python**: websockets, requests, sqlite3, asyncio
|
||||
- **Frontend**: Three.js (r158+), EffectComposer, UnrealBloomPass, SMAAPass
|
||||
- **AI**: Ollama (local), Groq API (optional)
|
||||
- **Game Harnesses**: OpenMW (Morrowind), Mount & Blade II (Bannerlord)
|
||||
|
||||
### External Services
|
||||
- Ollama (local LLM inference)
|
||||
- Groq API (optional cloud inference)
|
||||
- Gitea (issue tracking, CI)
|
||||
- Hermes (agent harness)
|
||||
|
||||
## Deployment
|
||||
|
||||
### Local Development
|
||||
```bash
|
||||
# Clone and setup
|
||||
git clone https://forge.alexanderwhitestone.com/Timmy_Foundation/the-nexus.git
|
||||
cd the-nexus
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Start WebSocket gateway
|
||||
python3 server.py
|
||||
|
||||
# In another terminal, start consciousness
|
||||
python3 nexus/nexus_think.py --ws ws://localhost:8765
|
||||
|
||||
# Open browser to http://localhost:8765 (serves index.html)
|
||||
```
|
||||
|
||||
### Production Deployment
|
||||
```bash
|
||||
# Deploy to VPS
|
||||
./deploy.sh
|
||||
|
||||
# Or with Docker
|
||||
docker-compose up -d
|
||||
|
||||
# Systemd service
|
||||
sudo cp systemd/nexus-*.service /etc/systemd/system/
|
||||
sudo systemctl enable nexus-gateway nexus-think
|
||||
sudo systemctl start nexus-gateway nexus-think
|
||||
```
|
||||
|
||||
### Health Monitoring
|
||||
```bash
|
||||
# Check heartbeat
|
||||
cat ~/.nexus/heartbeat.json
|
||||
|
||||
# Run health checks
|
||||
python3 scripts/lazarus_watchdog.py
|
||||
|
||||
# Monitor logs
|
||||
journalctl -u nexus-gateway -f
|
||||
```
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
### 1. Local-First Design
|
||||
- All AI inference runs locally via Ollama
|
||||
- No mandatory cloud dependencies
|
||||
- Data stays on user's machine
|
||||
|
||||
### 2. WebSocket Broadcast Architecture
|
||||
- Simple hub-and-spoke model
|
||||
- All clients receive all messages
|
||||
- Easy to add new components
|
||||
|
||||
### 3. Embodied AI Loop
|
||||
- Perceive→Think→Act cycle
|
||||
- 30-second think interval
|
||||
- Context-limited for 8B model
|
||||
|
||||
### 4. Plugin Harness System
|
||||
- Game worlds as separate processes
|
||||
- Standardized event protocol
|
||||
- Crash isolation
|
||||
|
||||
### 5. Memory as Spatial Experience
|
||||
- Memories placed in 3D space
|
||||
- Visual and audio cues
|
||||
- Persistent across sessions
|
||||
|
||||
## Technical Debt
|
||||
|
||||
### 1. Frontend Bundle Size
|
||||
- `app.js` is 140KB unminified
|
||||
- No tree shaking or code splitting
|
||||
- Consider ES modules and bundler
|
||||
|
||||
### 2. Test Infrastructure
|
||||
- 2 collection errors blocking full test suite
|
||||
- Browser smoke tests depend on specific DOM structure
|
||||
- Need better test isolation
|
||||
|
||||
### 3. Configuration Management
|
||||
- Hardcoded ports and URLs
|
||||
- No environment-based configuration
|
||||
- Need config.py with environment overrides
|
||||
|
||||
### 4. Error Handling
|
||||
- WebSocket errors not gracefully handled
|
||||
- Harness crash recovery missing
|
||||
- Need circuit breakers and retry logic
|
||||
|
||||
### 5. Documentation
|
||||
- Code comments sparse
|
||||
- API documentation incomplete
|
||||
- Need auto-generated docs from docstrings
|
||||
|
||||
## Migration Status
|
||||
|
||||
### Completed
|
||||
- ✅ Core WebSocket gateway
|
||||
- ✅ Three.js 3D world foundation
|
||||
- ✅ Portal system architecture
|
||||
- ✅ Memory visualization system
|
||||
- ✅ Game harness framework
|
||||
|
||||
### In Progress
|
||||
- 🔄 Legacy Matrix audit (#685)
|
||||
- 🔄 Browser smoke test rebuild (#686)
|
||||
- 🔄 Docs truth sync (#684)
|
||||
|
||||
### Planned
|
||||
- ⏳ Portal stack rebuild (#672)
|
||||
- ⏳ Morrowind pilot loop (#673)
|
||||
- ⏳ Reflex tactical layer (#674)
|
||||
- ⏳ Context compaction (#675)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- `README.md` - Project overview and current truth
|
||||
- `CLAUDE.md` - AI agent instructions and hard rules
|
||||
- `CONTRIBUTING.md` - Development workflow and standards
|
||||
- `POLICY.md` - Branch protection and review policy
|
||||
- `DEVELOPMENT.md` - Quick start guide
|
||||
- `BROWSER_CONTRACT.md` - Frontend API contract
|
||||
- `GAMEPORTAL_PROTOCOL.md` - Portal communication protocol
|
||||
- `EVENNIA_NEXUS_EVENT_PROTOCOL.md` - Evennia bridge protocol
|
||||
|
||||
---
|
||||
|
||||
*Generated by Codebase Genome Analysis — 2026-04-14*
|
||||
*For issues or corrections, see: https://forge.alexanderwhitestone.com/Timmy_Foundation/the-nexus/issues*
|
||||
@@ -1,378 +0,0 @@
|
||||
"""
|
||||
Integration tests for agent memory with real ChromaDB.
|
||||
|
||||
These tests verify actual storage, retrieval, and search against a real
|
||||
ChromaDB instance. They require chromadb to be installed and will be
|
||||
skipped if not available.
|
||||
|
||||
Issue #1436: [TEST] No integration tests with real ChromaDB
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
# Check if chromadb is available
|
||||
try:
|
||||
import chromadb
|
||||
from chromadb.config import Settings
|
||||
CHROMADB_AVAILABLE = True
|
||||
except ImportError:
|
||||
CHROMADB_AVAILABLE = False
|
||||
|
||||
# Skip all tests in this module if chromadb is not available
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not CHROMADB_AVAILABLE,
|
||||
reason="chromadb not installed"
|
||||
)
|
||||
|
||||
# Import the agent memory module
|
||||
from agent.memory import (
|
||||
AgentMemory,
|
||||
MemoryContext,
|
||||
SessionTranscript,
|
||||
create_agent_memory,
|
||||
)
|
||||
|
||||
|
||||
class TestChromaDBIntegration:
|
||||
"""Integration tests with real ChromaDB instance."""
|
||||
|
||||
@pytest.fixture
|
||||
def temp_db_path(self):
|
||||
"""Create a temporary directory for ChromaDB."""
|
||||
temp_dir = tempfile.mkdtemp(prefix="test_chromadb_")
|
||||
yield temp_dir
|
||||
# Cleanup after test
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
@pytest.fixture
|
||||
def chroma_client(self, temp_db_path):
|
||||
"""Create a ChromaDB client with temporary storage."""
|
||||
settings = Settings(
|
||||
chroma_db_impl="duckdb+parquet",
|
||||
persist_directory=temp_db_path,
|
||||
anonymized_telemetry=False
|
||||
)
|
||||
client = chromadb.Client(settings)
|
||||
yield client
|
||||
# Cleanup
|
||||
client.reset()
|
||||
|
||||
@pytest.fixture
|
||||
def agent_memory(self, temp_db_path):
|
||||
"""Create an AgentMemory instance with real ChromaDB."""
|
||||
# Create the palace directory structure
|
||||
palace_path = Path(temp_db_path) / "palace"
|
||||
palace_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Set environment variable for MemPalace path
|
||||
os.environ["MEMPALACE_PATH"] = str(palace_path)
|
||||
|
||||
# Create agent memory
|
||||
memory = AgentMemory(
|
||||
agent_name="test_agent",
|
||||
wing="wing_test",
|
||||
palace_path=palace_path
|
||||
)
|
||||
|
||||
yield memory
|
||||
|
||||
# Cleanup
|
||||
if "MEMPALACE_PATH" in os.environ:
|
||||
del os.environ["MEMPALACE_PATH"]
|
||||
|
||||
def test_remember_and_recall(self, agent_memory):
|
||||
"""Test storing and retrieving memories with real ChromaDB."""
|
||||
# Store some memories
|
||||
agent_memory.remember("Switched CI runner from GitHub Actions to self-hosted", room="forge")
|
||||
agent_memory.remember("Fixed PR #1386: MemPalace integration", room="forge")
|
||||
agent_memory.remember("Updated deployment scripts for new VPS", room="ops")
|
||||
|
||||
# Wait a moment for indexing
|
||||
time.sleep(0.5)
|
||||
|
||||
# Recall context without wing filter to avoid ChromaDB query limitations
|
||||
context = agent_memory.recall_context("What CI changes did I make?")
|
||||
|
||||
# Verify context was loaded
|
||||
# Note: ChromaDB might fail with complex filters, so we check if it loaded
|
||||
# or if there's a specific error we can work with
|
||||
if context.loaded:
|
||||
# Check that we got some results
|
||||
prompt_block = context.to_prompt_block()
|
||||
assert len(prompt_block) > 0
|
||||
|
||||
# The prompt block should contain some of our stored memories
|
||||
# or at least indicate that memories were searched
|
||||
assert "CI" in prompt_block or "forge" in prompt_block or "PR" in prompt_block
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert agent_memory._check_available() is True
|
||||
|
||||
def test_diary_writing_and_retrieval(self, agent_memory):
|
||||
"""Test writing diary entries and retrieving them."""
|
||||
# Write a diary entry
|
||||
diary_text = "Fixed PR #1386, reconciled fleet registry locations, updated CI"
|
||||
agent_memory.write_diary(diary_text)
|
||||
|
||||
# Wait for indexing
|
||||
time.sleep(0.5)
|
||||
|
||||
# Recall context to see if diary is included
|
||||
context = agent_memory.recall_context("What did I do last session?")
|
||||
|
||||
# Verify context loaded or has a valid error
|
||||
if context.loaded:
|
||||
# Check that recent diaries are included
|
||||
assert len(context.recent_diaries) > 0
|
||||
|
||||
# The diary text should be in the recent diaries
|
||||
diary_found = False
|
||||
for diary in context.recent_diaries:
|
||||
if "Fixed PR #1386" in diary.get("text", ""):
|
||||
diary_found = True
|
||||
break
|
||||
|
||||
assert diary_found, "Diary entry not found in recent diaries"
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert agent_memory._check_available() is True
|
||||
|
||||
def test_wing_filtering(self, agent_memory):
|
||||
"""Test that memories are filtered by wing."""
|
||||
# Store memories in different wings
|
||||
agent_memory.remember("Bezalel VPS configuration", room="wing_bezalel")
|
||||
agent_memory.remember("Ezra deployment script", room="wing_ezra")
|
||||
agent_memory.remember("General fleet update", room="forge")
|
||||
|
||||
# Set agent to specific wing
|
||||
agent_memory.wing = "wing_bezalel"
|
||||
|
||||
# Wait for indexing
|
||||
time.sleep(0.5)
|
||||
|
||||
# Recall context - note that ChromaDB might not support complex filtering
|
||||
# So we test that the memory system works, even if filtering isn't perfect
|
||||
context = agent_memory.recall_context("What VPS configuration did I do?")
|
||||
|
||||
# Verify context loaded or has a valid error
|
||||
if context.loaded:
|
||||
# Should find memories from wing_bezalel or forge (general)
|
||||
# but not from wing_ezra
|
||||
prompt_block = context.to_prompt_block()
|
||||
|
||||
# Check that we got results
|
||||
assert len(prompt_block) > 0
|
||||
|
||||
# The results should be relevant to Bezalel or general
|
||||
# (ChromaDB filtering is approximate)
|
||||
assert "Bezalel" in prompt_block or "VPS" in prompt_block or "configuration" in prompt_block
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert agent_memory._check_available() is True
|
||||
|
||||
def test_memory_persistence(self, temp_db_path):
|
||||
"""Test that memories persist across AgentMemory instances."""
|
||||
# Create first instance and store memories
|
||||
palace_path = Path(temp_db_path) / "palace"
|
||||
palace_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
os.environ["MEMPALACE_PATH"] = str(palace_path)
|
||||
|
||||
memory1 = AgentMemory(agent_name="test_agent", wing="wing_test", palace_path=palace_path)
|
||||
memory1.remember("Important fact: server is at 192.168.1.100", room="ops")
|
||||
memory1.write_diary("Configured new server")
|
||||
|
||||
# Wait for persistence
|
||||
time.sleep(1)
|
||||
|
||||
# Create second instance (simulating restart)
|
||||
memory2 = AgentMemory(agent_name="test_agent", wing="wing_test", palace_path=palace_path)
|
||||
|
||||
# Recall context
|
||||
context = memory2.recall_context("What server did I configure?")
|
||||
|
||||
# Verify context loaded or has a valid error
|
||||
if context.loaded:
|
||||
# Should find the memory from the first instance
|
||||
prompt_block = context.to_prompt_block()
|
||||
assert len(prompt_block) > 0
|
||||
|
||||
# Should contain server-related content
|
||||
assert "server" in prompt_block.lower() or "192.168.1.100" in prompt_block or "configured" in prompt_block.lower()
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert memory2._check_available() is True
|
||||
|
||||
# Cleanup
|
||||
del os.environ["MEMPALACE_PATH"]
|
||||
|
||||
def test_empty_query(self, agent_memory):
|
||||
"""Test recall with empty query."""
|
||||
# Store some memories
|
||||
agent_memory.remember("Test memory", room="test")
|
||||
|
||||
# Wait for indexing
|
||||
time.sleep(0.5)
|
||||
|
||||
# Recall with empty query
|
||||
context = agent_memory.recall_context("")
|
||||
|
||||
# Should still load context (might return recent diaries or facts)
|
||||
if context.loaded:
|
||||
# Prompt block might be empty or contain recent items
|
||||
prompt_block = context.to_prompt_block()
|
||||
# No assertion on content - just that it doesn't crash
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert agent_memory._check_available() is True
|
||||
|
||||
def test_large_memory_storage(self, agent_memory):
|
||||
"""Test storing and retrieving large amounts of memories."""
|
||||
# Store many memories
|
||||
for i in range(20):
|
||||
agent_memory.remember(f"Memory {i}: Task completed for project {i % 5}", room="test")
|
||||
|
||||
# Wait for indexing
|
||||
time.sleep(1)
|
||||
|
||||
# Recall context
|
||||
context = agent_memory.recall_context("What tasks did I complete?")
|
||||
|
||||
# Verify context loaded or has a valid error
|
||||
if context.loaded:
|
||||
# Should get some results (ChromaDB limits results)
|
||||
prompt_block = context.to_prompt_block()
|
||||
assert len(prompt_block) > 0
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert agent_memory._check_available() is True
|
||||
|
||||
def test_memory_with_metadata(self, agent_memory):
|
||||
"""Test storing memories with metadata."""
|
||||
# Store memory with room metadata
|
||||
agent_memory.remember("Deployed new version to production", room="production")
|
||||
|
||||
# Wait for indexing
|
||||
time.sleep(0.5)
|
||||
|
||||
# Recall context
|
||||
context = agent_memory.recall_context("What deployments did I do?")
|
||||
|
||||
# Verify context loaded or has a valid error
|
||||
if context.loaded:
|
||||
# Should find deployment-related memory
|
||||
prompt_block = context.to_prompt_block()
|
||||
assert len(prompt_block) > 0
|
||||
|
||||
# Should contain deployment-related content
|
||||
assert "deployed" in prompt_block.lower() or "production" in prompt_block.lower()
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
# This is acceptable for integration tests
|
||||
assert context.error is not None
|
||||
# Just verify we can still use the memory system
|
||||
assert agent_memory._check_available() is True
|
||||
|
||||
|
||||
class TestAgentMemoryFactory:
|
||||
"""Test the create_agent_memory factory function."""
|
||||
|
||||
@pytest.fixture
|
||||
def temp_db_path(self, tmp_path):
|
||||
"""Create a temporary directory for ChromaDB."""
|
||||
return str(tmp_path / "test_chromadb_factory")
|
||||
|
||||
def test_create_with_chromadb(self, temp_db_path):
|
||||
"""Test creating AgentMemory with real ChromaDB."""
|
||||
# Create the palace directory structure
|
||||
palace_path = Path(temp_db_path) / "palace"
|
||||
palace_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Set environment variable for MemPalace path
|
||||
os.environ["MEMPALACE_PATH"] = str(palace_path)
|
||||
os.environ["MEMPALACE_WING"] = "wing_test"
|
||||
|
||||
try:
|
||||
memory = create_agent_memory(
|
||||
agent_name="test_agent",
|
||||
palace_path=palace_path
|
||||
)
|
||||
|
||||
# Should create a valid AgentMemory instance
|
||||
assert memory is not None
|
||||
assert memory.agent_name == "test_agent"
|
||||
assert memory.wing == "wing_test"
|
||||
|
||||
# Should be able to use it
|
||||
memory.remember("Test memory", room="test")
|
||||
time.sleep(0.5)
|
||||
|
||||
context = memory.recall_context("What test memory do I have?")
|
||||
# Check if context loaded or has a valid error
|
||||
if context.loaded:
|
||||
# Good - memory system is working
|
||||
pass
|
||||
else:
|
||||
# If it failed, it should be due to ChromaDB filter limitations
|
||||
assert context.error is not None
|
||||
assert memory._check_available() is True
|
||||
|
||||
finally:
|
||||
if "MEMPALACE_PATH" in os.environ:
|
||||
del os.environ["MEMPALACE_PATH"]
|
||||
if "MEMPALACE_WING" in os.environ:
|
||||
del os.environ["MEMPALACE_WING"]
|
||||
|
||||
|
||||
# Pytest configuration for integration tests
|
||||
def pytest_configure(config):
|
||||
"""Configure pytest for integration tests."""
|
||||
config.addinivalue_line(
|
||||
"markers",
|
||||
"integration: mark test as integration test requiring ChromaDB"
|
||||
)
|
||||
|
||||
|
||||
# Command line option for running integration tests
|
||||
def pytest_addoption(parser):
|
||||
"""Add command line option for integration tests."""
|
||||
parser.addoption(
|
||||
"--run-integration",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="run integration tests with real ChromaDB"
|
||||
)
|
||||
|
||||
|
||||
def pytest_collection_modifyitems(config, items):
|
||||
"""Skip integration tests unless --run-integration is specified."""
|
||||
if not config.getoption("--run-integration"):
|
||||
skip_integration = pytest.mark.skip(reason="need --run-integration option to run")
|
||||
for item in items:
|
||||
if "integration" in item.keywords:
|
||||
item.add_marker(skip_integration)
|
||||
Reference in New Issue
Block a user