148 lines
3.9 KiB
Markdown
148 lines
3.9 KiB
Markdown
# Hermes-Agent Cutover Test Plan
|
|
|
|
## Date: 2026-03-30
|
|
## Author: Timmy (Opus)
|
|
|
|
## What's Happening
|
|
Merging gitea/main (gemini's 12 new files + allegro's merges) into our local working copy,
|
|
then rebasing timmy-custom (our +410 lines) on top.
|
|
|
|
## Pre-Existing Issues (BEFORE cutover)
|
|
- `firecrawl` module not installed → all tests that import `model_tools` fail
|
|
- Test suite cannot run cleanly even on current main
|
|
- 583 pip packages installed
|
|
- google-genai NOT installed (will be added by cutover)
|
|
|
|
---
|
|
|
|
## BEFORE Baseline (captured 2026-03-30 18:30 ET)
|
|
|
|
| Metric | Value |
|
|
|:-------|:------|
|
|
| Commit | fb634068 (NousResearch upstream) |
|
|
| Hermes Version | v0.5.0 (2026.3.28) |
|
|
| CLI cold start (`hermes status`) | 0.195s |
|
|
| Import time (`from run_agent import AIAgent`) | FAILS (missing firecrawl) |
|
|
| Disk usage | 909M |
|
|
| Installed packages | 583 |
|
|
| google-genai | NOT INSTALLED |
|
|
| Tests passing | 0 (firecrawl blocks everything) |
|
|
| Local modifications | 0 files (clean main) |
|
|
| Model | claude-opus-4-6 via Anthropic |
|
|
| Fallback chain | codex → gemini → groq → grok → kimi → openrouter |
|
|
|
|
---
|
|
|
|
## Cutover Steps
|
|
|
|
### Step 1: Update local main from gitea
|
|
```bash
|
|
cd ~/.hermes/hermes-agent
|
|
git checkout main
|
|
git pull gitea main
|
|
```
|
|
Expected: 17 new commits, 12 new files, pyproject.toml change.
|
|
|
|
### Step 2: Install new dependency
|
|
```bash
|
|
pip install google-genai
|
|
```
|
|
Expected: google-genai + deps installed.
|
|
|
|
### Step 3: Rebase timmy-custom onto new main
|
|
```bash
|
|
git checkout timmy-custom
|
|
git rebase main
|
|
```
|
|
Expected: possible conflict in pyproject.toml (the only shared file).
|
|
|
|
### Step 4: Verify
|
|
Run the AFTER checks below.
|
|
|
|
---
|
|
|
|
## AFTER Checks (run after cutover)
|
|
|
|
### A. Basic health
|
|
```bash
|
|
hermes status # Should show same providers + version
|
|
hermes --version # Should still be v0.5.0
|
|
```
|
|
|
|
### B. CLI cold start time
|
|
```bash
|
|
time hermes status # Compare to 0.195s baseline
|
|
```
|
|
|
|
### C. Import time
|
|
```bash
|
|
cd ~/.hermes/hermes-agent
|
|
time python3 -c "from run_agent import AIAgent"
|
|
# Should work now if firecrawl is installed, or still fail on firecrawl (pre-existing)
|
|
```
|
|
|
|
### D. New files present
|
|
```bash
|
|
ls agent/gemini_adapter.py agent/knowledge_ingester.py agent/meta_reasoning.py agent/symbolic_memory.py
|
|
ls skills/creative/sovereign_thinking.py skills/memory/intersymbolic_graph.py skills/research/realtime_learning.py
|
|
ls tools/gitea_client.py tools/graph_store.py
|
|
ls tests/agent/test_symbolic_memory.py tests/tools/test_graph_store.py
|
|
```
|
|
|
|
### E. Our customizations intact
|
|
```bash
|
|
git log --oneline -3 # Should show timmy-custom commit on top
|
|
git diff HEAD~1 --stat # Should show our 6 files (+410 lines)
|
|
```
|
|
|
|
### F. Disk usage
|
|
```bash
|
|
du -sh ~/.hermes/hermes-agent/
|
|
pip list | wc -l
|
|
```
|
|
|
|
### G. google-genai transparent fallback
|
|
```bash
|
|
python3 -c "
|
|
try:
|
|
from agent.gemini_adapter import GeminiAdapter
|
|
a = GeminiAdapter()
|
|
print('GeminiAdapter loaded (GOOGLE_API_KEY needed for actual calls)')
|
|
except ImportError as e:
|
|
print(f'Import failed: {e}')
|
|
except Exception as e:
|
|
print(f'Loaded but init failed (expected without key): {e}')
|
|
"
|
|
```
|
|
|
|
### H. Test suite
|
|
```bash
|
|
python3 -m pytest tests/ -x --tb=line -q 2>&1 | tail -10
|
|
# Compare to BEFORE (which also fails on firecrawl)
|
|
```
|
|
|
|
### I. Actual agent session
|
|
```bash
|
|
hermes -m "Say hello in 5 words"
|
|
# Verify the agent still works end-to-end
|
|
```
|
|
|
|
---
|
|
|
|
## Rollback Plan
|
|
If anything breaks:
|
|
```bash
|
|
cd ~/.hermes/hermes-agent
|
|
git checkout main
|
|
git reset --hard fb634068 # Original upstream commit
|
|
pip uninstall google-genai # Remove new dep
|
|
```
|
|
|
|
## Success Criteria
|
|
1. `hermes status` shows same providers, no errors
|
|
2. CLI cold start within 50% of baseline (< 0.3s)
|
|
3. Agent sessions work (`hermes -m` responds)
|
|
4. Our timmy-custom changes present (refusal detection, kimi routing, usage pricing, auth)
|
|
5. New gemini files present but don't interfere when GOOGLE_API_KEY is unset
|
|
6. No new test failures beyond the pre-existing firecrawl issue
|