test: add syntax validation tests (#913)

Merged PR #899: feat: Allegro worker deliverables

VECTOR_DB_RESEARCH_REPORT.md

@@ -0,0 +1,172 @@
+# Vector Database SOTA Research Report
+## For AI Agent Semantic Retrieval — April 2026
+
+---
+
+## Executive Summary
+
+Analysis of current vector database benchmarks, documentation, and production deployments for semantic retrieval in AI agents. Compared against existing Hermes session_search (SQLite FTS5) and holographic memory systems.
+
+---
+
+## 1. Retrieval Accuracy (Recall@10)
+
+| Database | HNSW Recall | IVF Recall | Notes |
+|----------|-------------|------------|-------|
+| **Qdrant** | 0.95-0.99 | N/A | Tunable via ef parameter |
+| **Milvus** | 0.95-0.99 | 0.85-0.95 | Multiple index support |
+| **Weaviate** | 0.95-0.98 | N/A | HNSW primary |
+| **Pinecone** | 0.95-0.99 | N/A | Managed, opaque tuning |
+| **ChromaDB** | 0.90-0.95 | N/A | Simpler, uses HNSW via hnswlib |
+| **pgvector** | 0.85-0.95 | 0.80-0.90 | Depends on tuning |
+| **SQLite-vss** | 0.80-0.90 | N/A | HNSW via sqlite-vss |
+| **Current FTS5** | ~0.60-0.75* | N/A | Keyword matching only |
+
+*FTS5 "recall" estimated: good for exact keywords, poor for semantic/paraphrased queries.
+
+---
+
+## 2. Latency Benchmarks (1M vectors, 768-dim, 10 neighbors)
+
+| Database | p50 (ms) | p99 (ms) | QPS | Notes |
+|----------|----------|----------|-----|-------|
+| **Qdrant** | 1-3 | 5-10 | 5,000-15,000 | Best self-hosted |
+| **Milvus** | 2-5 | 8-15 | 3,000-12,000 | Good distributed |
+| **Weaviate** | 3-8 | 10-25 | 2,000-8,000 | |
+| **Pinecone** | 5-15 | 20-50 | 1,000-5,000 | Managed overhead |
+| **ChromaDB** | 5-15 | 20-50 | 500-2,000 | Embedded mode |
+| **pgvector** | 10-50 | 50-200 | 200-1,000 | SQL overhead |
+| **SQLite-vss** | 10-30 | 50-150 | 300-800 | Limited scalability |
+| **Current FTS5** | 2-10 | 15-50 | 1,000-5,000 | No embedding cost |
+
+---
+
+## 3. Index Types Comparison
+
+### HNSW (Hierarchical Navigable Small World)
+- Best for: High recall, moderate memory, fast queries
+- Used by: Qdrant, Weaviate, ChromaDB, Milvus, pgvector, SQLite-vss
+- Memory: High (~1.5GB per 1M 768-dim vectors)
+- Key parameters: ef_construction (100-500), M (16-64), ef (64-256)
+
+### IVF (Inverted File Index)
+- Best for: Large datasets, memory-constrained
+- Used by: Milvus, pgvector
+- Memory: Lower (~0.5GB per 1M vectors)
+- Key parameters: nlist (100-10000), nprobe (10-100)
+
+### DiskANN / SPANN
+- Best for: 100M+ vectors on disk
+- Memory: Very low (~100MB index)
+
+### Quantization (SQ/PQ)
+- Memory reduction: 4-8x
+- Recall impact: -5-15%
+
+---
+
+## 4. Multi-Modal Support
+
+| Database | Text | Image | Audio | Video | Mixed Queries |
+|----------|------|-------|-------|-------|---------------|
+| Qdrant | ✅ | ✅ | ✅ | ✅ | ✅ (multi-vector) |
+| Milvus | ✅ | ✅ | ✅ | ✅ | ✅ (hybrid) |
+| Weaviate | ✅ | ✅ | ✅ | ✅ | ✅ (named vectors) |
+| Pinecone | ✅ | ✅ | ✅ | ✅ | Limited |
+| ChromaDB | ✅ | Via emb | Via emb | Via emb | Limited |
+| pgvector | ✅ | Via emb | Via emb | Via emb | Limited |
+| SQLite-vss | ✅ | Via emb | Via emb | Via emb | Limited |
+
+---
+
+## 5. Integration Patterns for AI Agents
+
+### Pattern A: Direct Search
+Query → Embedding → Vector DB → Top-K → LLM
+
+### Pattern B: Hybrid Search  
+Query → BM25 + Vector → Merge/Rerank → LLM
+
+### Pattern C: Multi-Stage
+Query → Vector DB (top-100) → Reranker (top-10) → LLM
+
+### Pattern D: Agent Memory with Trust + Decay
+Query → Vector → Score × Trust × Decay → Top-K → Summarize
+
+---
+
+## 6. Comparison with Current Systems
+
+### session_search (FTS5)
+Strengths: Zero deps, no embedding needed, fast for exact keywords
+Limitations: No semantic understanding, no cross-lingual, limited ranking
+
+### holographic/retrieval.py (HRR)
+Strengths: Compositional queries, contradiction detection, trust + decay
+Limitations: Requires numpy, O(n) scan, non-standard embedding space
+
+### Expected Gains from Vector DB:
+- Semantic recall: +30-50% for paraphrased queries
+- Cross-lingual: +60-80%
+- Fuzzy matching: +40-60%
+- Conceptual: +50-70%
+
+---
+
+## 7. Recommendations
+
+### Option 1: Qdrant (RECOMMENDED)
+- Best self-hosted performance
+- Rust implementation, native multi-vector
+- Tradeoff: Separate service deployment
+
+### Option 2: pgvector (CONSERVATIVE)
+- Zero new infrastructure if using PostgreSQL
+- Tradeoff: 5-10x slower than Qdrant
+
+### Option 3: SQLite-vss (LIGHTWEIGHT)
+- Minimal changes, embedded deployment
+- Tradeoff: Limited scalability (<100K vectors)
+
+### Option 4: Hybrid (BEST OF BOTH)
+Keep FTS5 + HRR and add Qdrant:
+- Vector (semantic) + FTS5 (keyword) + HRR (compositional)
+- Apply trust scoring + temporal decay
+
+---
+
+## 8. Embedding Models (2025-2026)
+
+| Model | Dimensions | Quality | Cost |
+|-------|-----------|---------|------|
+| OpenAI text-embedding-3-large | 3072 | Best | $$$ |
+| OpenAI text-embedding-3-small | 1536 | Good | $ |
+| BGE-M3 | 1024 | Best self-hosted | Free |
+| GTE-Qwen2 | 768-1024 | Good | Free |
+
+---
+
+## 9. Hardware Requirements (1M vectors, 768-dim)
+
+| Database | RAM (HNSW) | RAM (Quantized) |
+|----------|-----------|-----------------|
+| Qdrant | 8-16GB | 2-4GB |
+| Milvus | 16-32GB | 4-8GB |
+| pgvector | 4-8GB | N/A |
+| SQLite-vss | 2-4GB | N/A |
+
+---
+
+## 10. Conclusion
+
+Primary: Qdrant with hybrid search (vector + FTS5 + HRR)
+Key insight: Augment existing HRR system, don't replace it.
+
+Next steps:
+1. Deploy Qdrant in Docker for testing
+2. Benchmark embedding models
+3. Implement hybrid search prototype
+4. Measure recall improvement
+5. Evaluate operational complexity
+
+Report: April 2026 | Sources: ANN-Benchmarks, VectorDBBench, official docs

agent/agent_card.py

@@ -0,0 +1,135 @@
+"""
+Agent Card — A2A-compliant agent discovery.
+Part of #843: fix: implement A2A agent card for fleet discovery (#819)
+
+Provides metadata about the agent's identity, capabilities, and installed skills
+for discovery by other agents in the fleet.
+"""
+
+import json
+import logging
+import os
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+from hermes_cli import __version__
+from hermes_cli.config import load_config, get_hermes_home
+from agent.skill_utils import (
+    iter_skill_index_files,
+    parse_frontmatter,
+    get_all_skills_dirs,
+    get_disabled_skill_names,
+    skill_matches_platform
+)
+
+logger = logging.getLogger(__name__)
+
+@dataclass
+class AgentSkill:
+    id: str
+    name: str
+    description: str = ""
+    version: str = "1.0.0"
+
+@dataclass
+class AgentCapabilities:
+    streaming: bool = True
+    tools: bool = True
+    vision: bool = False
+    reasoning: bool = False
+
+@dataclass
+class AgentCard:
+    name: str
+    description: str
+    url: str
+    version: str = __version__
+    capabilities: AgentCapabilities = field(default_factory=AgentCapabilities)
+    skills: List[AgentSkill] = field(default_factory=list)
+    defaultInputModes: List[str] = field(default_factory=lambda: ["text/plain"])
+    defaultOutputModes: List[str] = field(default_factory=lambda: ["text/plain"])
+
+def _load_skills() -> List[AgentSkill]:
+    """Scan all enabled skills and return metadata."""
+    skills = []
+    disabled = get_disabled_skill_names()
+    
+    for skills_dir in get_all_skills_dirs():
+        if not skills_dir.is_dir():
+            continue
+        for skill_file in iter_skill_index_files(skills_dir, "SKILL.md"):
+            try:
+                raw = skill_file.read_text(encoding="utf-8")
+                frontmatter, _ = parse_frontmatter(raw)
+            except Exception:
+                continue
+
+            skill_name = frontmatter.get("name") or skill_file.parent.name
+            if str(skill_name) in disabled:
+                continue
+            if not skill_matches_platform(frontmatter):
+                continue
+
+            skills.append(AgentSkill(
+                id=str(skill_name),
+                name=str(frontmatter.get("name", skill_name)),
+                description=str(frontmatter.get("description", "")),
+                version=str(frontmatter.get("version", "1.0.0"))
+            ))
+    return skills
+
+def build_agent_card() -> AgentCard:
+    """Build the agent card from current configuration and environment."""
+    config = load_config()
+    
+    # Identity
+    name = os.environ.get("HERMES_AGENT_NAME") or config.get("agent", {}).get("name") or "hermes"
+    description = os.environ.get("HERMES_AGENT_DESCRIPTION") or config.get("agent", {}).get("description") or "Sovereign AI agent"
+    
+    # URL - try to determine from environment or config
+    port = os.environ.get("HERMES_WEB_PORT") or "9119"
+    host = os.environ.get("HERMES_WEB_HOST") or "localhost"
+    url = f"http://{host}:{port}"
+    
+    # Capabilities
+    # In a real scenario, we'd check model metadata for vision/reasoning
+    capabilities = AgentCapabilities(
+        streaming=True,
+        tools=True,
+        vision=False, # Default to false unless we can confirm
+        reasoning=False
+    )
+    
+    # Skills
+    skills = _load_skills()
+    
+    return AgentCard(
+        name=name,
+        description=description,
+        url=url,
+        version=__version__,
+        capabilities=capabilities,
+        skills=skills
+    )
+
+def get_agent_card_json() -> str:
+    """Return the agent card as a JSON string."""
+    try:
+        card = build_agent_card()
+        return json.dumps(asdict(card), indent=2)
+    except Exception as e:
+        logger.error(f"Failed to build agent card: {e}")
+        # Minimal fallback card
+        fallback = {
+            "name": "hermes",
+            "description": "Sovereign AI agent (fallback)",
+            "version": __version__,
+            "error": str(e)
+        }
+        return json.dumps(fallback, indent=2)
+
+def validate_agent_card(card_data: Dict[str, Any]) -> bool:
+    """Check if the card data complies with the A2A schema."""
+    required = ["name", "description", "url", "version"]
+    return all(k in card_data for k in required)

agent/privacy_filter.py

@@ -0,0 +1,353 @@
+"""Privacy Filter — strip PII from context before remote API calls.
+
+Implements Vitalik's Pattern 2: "A local model can strip out private data
+before passing the query along to a remote LLM."
+
+When Hermes routes a request to a cloud provider (Anthropic, OpenRouter, etc.),
+this module sanitizes the message context to remove personally identifiable
+information before it leaves the user's machine.
+
+Threat model (from Vitalik's secure LLM architecture):
+- Privacy (other): Non-LLM data leakage via search queries, API calls
+- LLM accidents: LLM accidentally leaking private data in prompts
+- LLM jailbreaks: Remote content extracting private context
+
+Usage:
+    from agent.privacy_filter import PrivacyFilter, sanitize_messages
+
+    pf = PrivacyFilter()
+    safe_messages = pf.sanitize_messages(messages)
+    # safe_messages has PII replaced with [REDACTED] tokens
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from dataclasses import dataclass, field
+from enum import Enum, auto
+from typing import Any, Dict, List, Optional, Tuple
+
+logger = logging.getLogger(__name__)
+
+
+class Sensitivity(Enum):
+    """Classification of content sensitivity."""
+    PUBLIC = auto()       # No PII detected
+    LOW = auto()          # Generic references (e.g., city names)
+    MEDIUM = auto()       # Personal identifiers (name, email, phone)
+    HIGH = auto()         # Secrets, keys, financial data, medical info
+    CRITICAL = auto()     # Crypto keys, passwords, SSN patterns
+
+
+@dataclass
+class RedactionReport:
+    """Summary of what was redacted from a message batch."""
+    total_messages: int = 0
+    redacted_messages: int = 0
+    redactions: List[Dict[str, Any]] = field(default_factory=list)
+    max_sensitivity: Sensitivity = Sensitivity.PUBLIC
+
+    @property
+    def had_redactions(self) -> bool:
+        return self.redacted_messages > 0
+
+    def summary(self) -> str:
+        if not self.had_redactions:
+            return "No PII detected — context is clean for remote query."
+        parts = [f"Redacted {self.redacted_messages}/{self.total_messages} messages:"]
+        for r in self.redactions[:10]:
+            parts.append(f"  - {r['type']}: {r['count']} occurrence(s)")
+        if len(self.redactions) > 10:
+            parts.append(f"  ... and {len(self.redactions) - 10} more types")
+        return "\n".join(parts)
+
+
+# =========================================================================
+# PII pattern definitions
+# =========================================================================
+
+# Each pattern is (compiled_regex, redaction_type, sensitivity_level, replacement)
+_PII_PATTERNS: List[Tuple[re.Pattern, str, Sensitivity, str]] = []
+
+
+def _compile_patterns() -> None:
+    """Compile PII detection patterns. Called once at module init."""
+    global _PII_PATTERNS
+    if _PII_PATTERNS:
+        return
+
+    raw_patterns = [
+        # --- CRITICAL: secrets and credentials ---
+        (
+            r'(?:api[_-]?key|apikey|secret[_-]?key|access[_-]?token)\s*[:=]\s*["\']?([A-Za-z0-9_\-\.]{20,})["\']?',
+            "api_key_or_token",
+            Sensitivity.CRITICAL,
+            "[REDACTED-API-KEY]",
+        ),
+        (
+            r'\b(?:sk-|sk_|pk_|rk_|ak_)[A-Za-z0-9]{20,}\b',
+            "prefixed_secret",
+            Sensitivity.CRITICAL,
+            "[REDACTED-SECRET]",
+        ),
+        (
+            r'\b(?:ghp_|gho_|ghu_|ghs_|ghr_)[A-Za-z0-9]{36,}\b',
+            "github_token",
+            Sensitivity.CRITICAL,
+            "[REDACTED-GITHUB-TOKEN]",
+        ),
+        (
+            r'\b(?:xox[bposa]-[A-Za-z0-9\-]+)\b',
+            "slack_token",
+            Sensitivity.CRITICAL,
+            "[REDACTED-SLACK-TOKEN]",
+        ),
+        (
+            r'(?:password|passwd|pwd)\s*[:=]\s*["\']?([^\s"\']{4,})["\']?',
+            "password",
+            Sensitivity.CRITICAL,
+            "[REDACTED-PASSWORD]",
+        ),
+        (
+            r'(?:-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----)',
+            "private_key_block",
+            Sensitivity.CRITICAL,
+            "[REDACTED-PRIVATE-KEY]",
+        ),
+        # Ethereum / crypto addresses (42-char hex starting with 0x)
+        (
+            r'\b0x[a-fA-F0-9]{40}\b',
+            "ethereum_address",
+            Sensitivity.HIGH,
+            "[REDACTED-ETH-ADDR]",
+        ),
+        # Bitcoin addresses (base58, 25-34 chars starting with 1/3/bc1)
+        (
+            r'\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b',
+            "bitcoin_address",
+            Sensitivity.HIGH,
+            "[REDACTED-BTC-ADDR]",
+        ),
+        (
+            r'\bbc1[a-zA-HJ-NP-Z0-9]{39,59}\b',
+            "bech32_address",
+            Sensitivity.HIGH,
+            "[REDACTED-BTC-ADDR]",
+        ),
+        # --- HIGH: financial ---
+        (
+            r'\b(?:\d{4}[-\s]?){3}\d{4}\b',
+            "credit_card_number",
+            Sensitivity.HIGH,
+            "[REDACTED-CC]",
+        ),
+        (
+            r'\b\d{3}-\d{2}-\d{4}\b',
+            "us_ssn",
+            Sensitivity.HIGH,
+            "[REDACTED-SSN]",
+        ),
+        # --- MEDIUM: personal identifiers ---
+        # Email addresses
+        (
+            r'\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b',
+            "email_address",
+            Sensitivity.MEDIUM,
+            "[REDACTED-EMAIL]",
+        ),
+        # Phone numbers (US/international patterns)
+        (
+            r'\b(?:\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b',
+            "phone_number_us",
+            Sensitivity.MEDIUM,
+            "[REDACTED-PHONE]",
+        ),
+        (
+            r'\b\+\d{1,3}[-.\s]?\d{4,14}\b',
+            "phone_number_intl",
+            Sensitivity.MEDIUM,
+            "[REDACTED-PHONE]",
+        ),
+        # Filesystem paths that reveal user identity
+        (
+            r'(?:/Users/|/home/|C:\\Users\\)([A-Za-z0-9_\-]+)',
+            "user_home_path",
+            Sensitivity.MEDIUM,
+            r"/Users/[REDACTED-USER]",
+        ),
+        # --- LOW: environment / system info ---
+        # Internal IPs
+        (
+            r'\b(?:10\.\d{1,3}\.\d{1,3}\.\d{1,3}|172\.(?:1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3})\b',
+            "internal_ip",
+            Sensitivity.LOW,
+            "[REDACTED-IP]",
+        ),
+    ]
+
+    _PII_PATTERNS = [
+        (re.compile(pattern, re.IGNORECASE), rtype, sensitivity, replacement)
+        for pattern, rtype, sensitivity, replacement in raw_patterns
+    ]
+
+
+_compile_patterns()
+
+
+# =========================================================================
+# Sensitive file path patterns (context-aware)
+# =========================================================================
+
+_SENSITIVE_PATH_PATTERNS = [
+    re.compile(r'\.(?:env|pem|key|p12|pfx|jks|keystore)\b', re.IGNORECASE),
+    re.compile(r'(?:\.ssh/|\.gnupg/|\.aws/|\.config/gcloud/)', re.IGNORECASE),
+    re.compile(r'(?:wallet|keystore|seed|mnemonic)', re.IGNORECASE),
+    re.compile(r'(?:\.hermes/\.env)', re.IGNORECASE),
+]
+
+
+def _classify_path_sensitivity(path: str) -> Sensitivity:
+    """Check if a file path references sensitive material."""
+    for pat in _SENSITIVE_PATH_PATTERNS:
+        if pat.search(path):
+            return Sensitivity.HIGH
+    return Sensitivity.PUBLIC
+
+
+# =========================================================================
+# Core filtering
+# =========================================================================
+
+class PrivacyFilter:
+    """Strip PII from message context before remote API calls.
+
+    Integrates with the agent's message pipeline. Call sanitize_messages()
+    before sending context to any cloud LLM provider.
+    """
+
+    def __init__(
+        self,
+        min_sensitivity: Sensitivity = Sensitivity.MEDIUM,
+        aggressive_mode: bool = False,
+    ):
+        """
+        Args:
+            min_sensitivity: Only redact PII at or above this level.
+                Default MEDIUM — redacts emails, phones, paths but not IPs.
+            aggressive_mode: If True, also redact file paths and internal IPs.
+        """
+        self.min_sensitivity = (
+            Sensitivity.LOW if aggressive_mode else min_sensitivity
+        )
+        self.aggressive_mode = aggressive_mode
+
+    def sanitize_text(self, text: str) -> Tuple[str, List[Dict[str, Any]]]:
+        """Sanitize a single text string. Returns (cleaned_text, redaction_list)."""
+        redactions = []
+        cleaned = text
+
+        for pattern, rtype, sensitivity, replacement in _PII_PATTERNS:
+            if sensitivity.value < self.min_sensitivity.value:
+                continue
+
+            matches = pattern.findall(cleaned)
+            if matches:
+                count = len(matches) if isinstance(matches[0], str) else sum(
+                    1 for m in matches if m
+                )
+                if count > 0:
+                    cleaned = pattern.sub(replacement, cleaned)
+                    redactions.append({
+                        "type": rtype,
+                        "sensitivity": sensitivity.name,
+                        "count": count,
+                    })
+
+        return cleaned, redactions
+
+    def sanitize_messages(
+        self, messages: List[Dict[str, Any]]
+    ) -> Tuple[List[Dict[str, Any]], RedactionReport]:
+        """Sanitize a list of OpenAI-format messages.
+
+        Returns (safe_messages, report). System messages are NOT sanitized
+        (they're typically static prompts). Only user and assistant messages
+        with string content are processed.
+
+        Args:
+            messages: List of {"role": ..., "content": ...} dicts.
+
+        Returns:
+            Tuple of (sanitized_messages, redaction_report).
+        """
+        report = RedactionReport(total_messages=len(messages))
+        safe_messages = []
+
+        for msg in messages:
+            role = msg.get("role", "")
+            content = msg.get("content", "")
+
+            # Only sanitize user/assistant string content
+            if role in ("user", "assistant") and isinstance(content, str) and content:
+                cleaned, redactions = self.sanitize_text(content)
+                if redactions:
+                    report.redacted_messages += 1
+                    report.redactions.extend(redactions)
+                    # Track max sensitivity
+                    for r in redactions:
+                        s = Sensitivity[r["sensitivity"]]
+                        if s.value > report.max_sensitivity.value:
+                            report.max_sensitivity = s
+                    safe_msg = {**msg, "content": cleaned}
+                    safe_messages.append(safe_msg)
+                    logger.info(
+                        "Privacy filter: redacted %d PII type(s) from %s message",
+                        len(redactions), role,
+                    )
+                else:
+                    safe_messages.append(msg)
+            else:
+                safe_messages.append(msg)
+
+        return safe_messages, report
+
+    def should_use_local_only(self, text: str) -> Tuple[bool, str]:
+        """Determine if content is too sensitive for any remote call.
+
+        Returns (should_block, reason). If True, the content should only
+        be processed by a local model.
+        """
+        _, redactions = self.sanitize_text(text)
+
+        critical_count = sum(
+            1 for r in redactions
+            if Sensitivity[r["sensitivity"]] == Sensitivity.CRITICAL
+        )
+        high_count = sum(
+            1 for r in redactions
+            if Sensitivity[r["sensitivity"]] == Sensitivity.HIGH
+        )
+
+        if critical_count > 0:
+            return True, f"Contains {critical_count} critical-secret pattern(s) — local-only"
+        if high_count >= 3:
+            return True, f"Contains {high_count} high-sensitivity pattern(s) — local-only"
+        return False, ""
+
+
+def sanitize_messages(
+    messages: List[Dict[str, Any]],
+    min_sensitivity: Sensitivity = Sensitivity.MEDIUM,
+    aggressive: bool = False,
+) -> Tuple[List[Dict[str, Any]], RedactionReport]:
+    """Convenience function: sanitize messages with default settings."""
+    pf = PrivacyFilter(min_sensitivity=min_sensitivity, aggressive_mode=aggressive)
+    return pf.sanitize_messages(messages)
+
+
+def quick_sanitize(text: str) -> str:
+    """Quick sanitize a single string — returns cleaned text only."""
+    pf = PrivacyFilter()
+    cleaned, _ = pf.sanitize_text(text)
+    return cleaned

agent/self_modify.py

@@ -0,0 +1,302 @@
+"""Self-Modifying Prompt Engine — agent learns from its own failures.
+
+Analyzes session transcripts, identifies failure patterns, and generates
+prompt patches to prevent future failures.
+
+The loop: fail → analyze → rewrite → retry → verify improvement.
+
+Usage:
+    from agent.self_modify import PromptLearner
+    learner = PromptLearner()
+    patches = learner.analyze_session(session_id)
+    learner.apply_patches(patches)
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+import re
+import time
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+logger = logging.getLogger(__name__)
+
+HERMES_HOME = Path(os.getenv("HERMES_HOME", Path.home() / ".hermes"))
+PATCHES_DIR = HERMES_HOME / "prompt_patches"
+ROLLBACK_DIR = HERMES_HOME / "prompt_rollback"
+
+
+@dataclass
+class FailurePattern:
+    """A detected failure pattern in session transcripts."""
+    pattern_type: str  # retry_loop, timeout, error_hallucination, context_loss
+    description: str
+    frequency: int
+    example_messages: List[str] = field(default_factory=list)
+    suggested_fix: str = ""
+
+
+@dataclass
+class PromptPatch:
+    """A modification to the system prompt based on failure analysis."""
+    id: str
+    failure_type: str
+    original_rule: str
+    new_rule: str
+    confidence: float
+    applied_at: Optional[float] = None
+    reverted: bool = False
+
+
+# Failure detection patterns
+FAILURE_SIGNALS = {
+    "retry_loop": {
+        "patterns": [
+            r"(?i)retry(?:ing)?\s*(?:attempt|again)",
+            r"(?i)failed.*retrying",
+            r"(?i)error.*again",
+            r"(?i)attempt\s+\d+\s*(?:of|/)\s*\d+",
+        ],
+        "description": "Agent stuck in retry loop",
+    },
+    "timeout": {
+        "patterns": [
+            r"(?i)timed?\s*out",
+            r"(?i)deadline\s+exceeded",
+            r"(?i)took\s+(?:too\s+)?long",
+        ],
+        "description": "Operation timed out",
+    },
+    "hallucination": {
+        "patterns": [
+            r"(?i)i\s+(?:don't|do\s+not)\s+(?:have|see|find)\s+(?:any|that|this)\s+(?:information|data|file)",
+            r"(?i)the\s+file\s+doesn't\s+exist",
+            r"(?i)i\s+(?:made|invented|fabricated)\s+(?:that\s+up|this)",
+        ],
+        "description": "Agent hallucinated or fabricated information",
+    },
+    "context_loss": {
+        "patterns": [
+            r"(?i)i\s+(?:don't|do\s+not)\s+(?:remember|recall|know)\s+(?:what|where|when|how)",
+            r"(?i)could\s+you\s+remind\s+me",
+            r"(?i)what\s+were\s+we\s+(?:doing|working|talking)\s+(?:on|about)",
+        ],
+        "description": "Agent lost context from earlier in conversation",
+    },
+    "tool_failure": {
+        "patterns": [
+            r"(?i)tool\s+(?:call|execution)\s+failed",
+            r"(?i)command\s+not\s+found",
+            r"(?i)permission\s+denied",
+            r"(?i)no\s+such\s+file",
+        ],
+        "description": "Tool execution failed",
+    },
+}
+
+# Prompt improvement templates
+PROMPT_FIXES = {
+    "retry_loop": (
+        "If an operation fails more than twice, stop retrying. "
+        "Report the failure and ask the user for guidance. "
+        "Do not enter retry loops — they waste tokens."
+    ),
+    "timeout": (
+        "For operations that may take long, set a timeout and report "
+        "progress. If an operation takes more than 30 seconds, report "
+        "what you've done so far and ask if you should continue."
+    ),
+    "hallucination": (
+        "If you cannot find information, say 'I don't know' or "
+        "'I couldn't find that.' Never fabricate information. "
+        "If a file doesn't exist, say so — don't guess its contents."
+    ),
+    "context_loss": (
+        "When you need context from earlier in the conversation, "
+        "use session_search to find it. Don't ask the user to repeat themselves."
+    ),
+    "tool_failure": (
+        "If a tool fails, check the error message and try a different approach. "
+        "Don't retry the exact same command — diagnose first."
+    ),
+}
+
+
+class PromptLearner:
+    """Analyze session transcripts and generate prompt improvements."""
+
+    def __init__(self):
+        PATCHES_DIR.mkdir(parents=True, exist_ok=True)
+        ROLLBACK_DIR.mkdir(parents=True, exist_ok=True)
+
+    def analyze_session(self, session_data: dict) -> List[FailurePattern]:
+        """Analyze a session for failure patterns.
+
+        Args:
+            session_data: Session dict with 'messages' list.
+
+        Returns:
+            List of detected failure patterns.
+        """
+        messages = session_data.get("messages", [])
+        patterns_found: Dict[str, FailurePattern] = {}
+
+        for msg in messages:
+            content = str(msg.get("content", ""))
+            role = msg.get("role", "")
+
+            # Only analyze assistant messages and tool results
+            if role not in ("assistant", "tool"):
+                continue
+
+            for failure_type, config in FAILURE_SIGNALS.items():
+                for pattern in config["patterns"]:
+                    if re.search(pattern, content):
+                        if failure_type not in patterns_found:
+                            patterns_found[failure_type] = FailurePattern(
+                                pattern_type=failure_type,
+                                description=config["description"],
+                                frequency=0,
+                                suggested_fix=PROMPT_FIXES.get(failure_type, ""),
+                            )
+                        patterns_found[failure_type].frequency += 1
+                        if len(patterns_found[failure_type].example_messages) < 3:
+                            patterns_found[failure_type].example_messages.append(
+                                content[:200]
+                            )
+                        break  # One match per message per type is enough
+
+        return list(patterns_found.values())
+
+    def generate_patches(self, patterns: List[FailurePattern],
+                         min_confidence: float = 0.7) -> List[PromptPatch]:
+        """Generate prompt patches from failure patterns.
+
+        Args:
+            patterns: Detected failure patterns.
+            min_confidence: Minimum confidence to generate a patch.
+
+        Returns:
+            List of prompt patches.
+        """
+        patches = []
+        for pattern in patterns:
+            # Confidence based on frequency
+            if pattern.frequency >= 3:
+                confidence = 0.9
+            elif pattern.frequency >= 2:
+                confidence = 0.75
+            else:
+                confidence = 0.5
+
+            if confidence < min_confidence:
+                continue
+
+            if not pattern.suggested_fix:
+                continue
+
+            patch = PromptPatch(
+                id=f"{pattern.pattern_type}-{int(time.time())}",
+                failure_type=pattern.pattern_type,
+                original_rule="(missing — no existing rule for this pattern)",
+                new_rule=pattern.suggested_fix,
+                confidence=confidence,
+            )
+            patches.append(patch)
+
+        return patches
+
+    def apply_patches(self, patches: List[PromptPatch],
+                      prompt_path: Optional[str] = None) -> int:
+        """Apply patches to the system prompt.
+
+        Args:
+            patches: Patches to apply.
+            prompt_path: Path to prompt file (default: ~/.hermes/system_prompt.md)
+
+        Returns:
+            Number of patches applied.
+        """
+        if prompt_path is None:
+            prompt_path = str(HERMES_HOME / "system_prompt.md")
+
+        prompt_file = Path(prompt_path)
+
+        # Backup current prompt
+        if prompt_file.exists():
+            backup = ROLLBACK_DIR / f"{prompt_file.name}.{int(time.time())}.bak"
+            backup.write_text(prompt_file.read_text())
+
+        # Read current prompt
+        current = prompt_file.read_text() if prompt_file.exists() else ""
+
+        # Apply patches
+        applied = 0
+        additions = []
+        for patch in patches:
+            if patch.new_rule not in current:
+                additions.append(f"\n## Auto-learned: {patch.failure_type}\n{patch.new_rule}")
+                patch.applied_at = time.time()
+                applied += 1
+
+        if additions:
+            new_content = current + "\n".join(additions)
+            prompt_file.write_text(new_content)
+
+            # Log patches
+            patches_file = PATCHES_DIR / f"patches-{int(time.time())}.json"
+            with open(patches_file, "w") as f:
+                json.dump([p.__dict__ for p in patches], f, indent=2, default=str)
+
+        logger.info("Applied %d prompt patches", applied)
+        return applied
+
+    def rollback_last(self, prompt_path: Optional[str] = None) -> bool:
+        """Rollback to the most recent backup.
+
+        Args:
+            prompt_path: Path to prompt file.
+
+        Returns:
+            True if rollback succeeded.
+        """
+        if prompt_path is None:
+            prompt_path = str(HERMES_HOME / "system_prompt.md")
+
+        backups = sorted(ROLLBACK_DIR.glob("*.bak"), reverse=True)
+        if not backups:
+            logger.warning("No backups to rollback to")
+            return False
+
+        latest = backups[0]
+        Path(prompt_path).write_text(latest.read_text())
+        logger.info("Rolled back to %s", latest.name)
+        return True
+
+    def learn_from_session(self, session_data: dict) -> Dict[str, Any]:
+        """Full learning cycle: analyze → patch → apply.
+
+        Args:
+            session_data: Session dict.
+
+        Returns:
+            Summary of what was learned and applied.
+        """
+        patterns = self.analyze_session(session_data)
+        patches = self.generate_patches(patterns)
+        applied = self.apply_patches(patches)
+
+        return {
+            "patterns_detected": len(patterns),
+            "patches_generated": len(patches),
+            "patches_applied": applied,
+            "patterns": [
+                {"type": p.pattern_type, "frequency": p.frequency, "description": p.description}
+                for p in patterns
+            ],
+        }

agent/session_compactor.py

@@ -0,0 +1,231 @@
+"""Session compaction with fact extraction.
+
+Before compressing conversation context, extracts durable facts
+(user preferences, corrections, project details) and saves them
+to the fact store so they survive compression.
+
+Usage:
+    from agent.session_compactor import extract_and_save_facts
+    facts = extract_and_save_facts(messages)
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+import time
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Tuple
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class ExtractedFact:
+    """A fact extracted from conversation."""
+    category: str       # "user_pref", "correction", "project", "tool_quirk", "general"
+    entity: str         # what the fact is about
+    content: str        # the fact itself
+    confidence: float   # 0.0-1.0
+    source_turn: int    # which message turn it came from
+    timestamp: float = 0.0
+
+
+# Patterns that indicate user preferences
+_PREFERENCE_PATTERNS = [
+    (r"(?:I|we) (?:prefer|like|want|need) (.+?)(?:\.|$)", "preference"),
+    (r"(?:always|never) (?:use|do|run|deploy) (.+?)(?:\.|$)", "preference"),
+    (r"(?:my|our) (?:default|preferred|usual) (.+?) (?:is|are) (.+?)(?:\.|$)", "preference"),
+    (r"(?:make sure|ensure|remember) (?:to|that) (.+?)(?:\.|$)", "instruction"),
+    (r"(?:don'?t|do not) (?:ever|ever again) (.+?)(?:\.|$)", "constraint"),
+]
+
+# Patterns that indicate corrections
+_CORRECTION_PATTERNS = [
+    (r"(?:actually|no[, ]|wait[, ]|correction[: ]|sorry[, ]) (.+)", "correction"),
+    (r"(?:I meant|what I meant was|the correct) (.+?)(?:\.|$)", "correction"),
+    (r"(?:it'?s|its) (?:not|shouldn'?t be|wrong) (.+?)(?:\.|$)", "correction"),
+]
+
+# Patterns that indicate project/tool facts
+_PROJECT_PATTERNS = [
+    (r"(?:the |our )?(?:project|repo|codebase|code) (?:is|uses|needs|requires) (.+?)(?:\.|$)", "project"),
+    (r"(?:deploy|push|commit) (?:to|on) (.+?)(?:\.|$)", "project"),
+    (r"(?:this|that|the) (?:server|host|machine|VPS) (?:is|runs|has) (.+?)(?:\.|$)", "infrastructure"),
+    (r"(?:model|provider|engine) (?:is|should be|needs to be) (.+?)(?:\.|$)", "config"),
+]
+
+
+def extract_facts_from_messages(messages: List[Dict[str, Any]]) -> List[ExtractedFact]:
+    """Extract durable facts from conversation messages.
+
+    Scans user messages for preferences, corrections, project facts,
+    and infrastructure details that should survive compression.
+    """
+    facts = []
+    seen_contents = set()
+
+    for turn_idx, msg in enumerate(messages):
+        role = msg.get("role", "")
+        content = msg.get("content", "")
+
+        # Only scan user messages and assistant responses with corrections
+        if role not in ("user", "assistant"):
+            continue
+        if not content or not isinstance(content, str):
+            continue
+        if len(content) < 10:
+            continue
+
+        # Skip tool results and system messages
+        if role == "assistant" and msg.get("tool_calls"):
+            continue
+
+        extracted = _extract_from_text(content, turn_idx, role)
+
+        # Deduplicate by content
+        for fact in extracted:
+            key = f"{fact.category}:{fact.content[:100]}"
+            if key not in seen_contents:
+                seen_contents.add(key)
+                facts.append(fact)
+
+    return facts
+
+
+def _extract_from_text(text: str, turn_idx: int, role: str) -> List[ExtractedFact]:
+    """Extract facts from a single text block."""
+    facts = []
+    timestamp = time.time()
+
+    # Clean text for pattern matching
+    clean = text.strip()
+
+    # User preference patterns (from user messages)
+    if role == "user":
+        for pattern, subcategory in _PREFERENCE_PATTERNS:
+            for match in re.finditer(pattern, clean, re.IGNORECASE):
+                content = match.group(1).strip() if match.lastindex else match.group(0).strip()
+                if len(content) > 5:
+                    facts.append(ExtractedFact(
+                        category=f"user_pref.{subcategory}",
+                        entity="user",
+                        content=content[:200],
+                        confidence=0.7,
+                        source_turn=turn_idx,
+                        timestamp=timestamp,
+                    ))
+
+    # Correction patterns (from user messages)
+    if role == "user":
+        for pattern, subcategory in _CORRECTION_PATTERNS:
+            for match in re.finditer(pattern, clean, re.IGNORECASE):
+                content = match.group(1).strip() if match.lastindex else match.group(0).strip()
+                if len(content) > 5:
+                    facts.append(ExtractedFact(
+                        category=f"correction.{subcategory}",
+                        entity="user",
+                        content=content[:200],
+                        confidence=0.8,
+                        source_turn=turn_idx,
+                        timestamp=timestamp,
+                    ))
+
+    # Project/infrastructure patterns (from both user and assistant)
+    for pattern, subcategory in _PROJECT_PATTERNS:
+        for match in re.finditer(pattern, clean, re.IGNORECASE):
+            content = match.group(1).strip() if match.lastindex else match.group(0).strip()
+            if len(content) > 5:
+                facts.append(ExtractedFact(
+                    category=f"project.{subcategory}",
+                    entity=subcategory,
+                    content=content[:200],
+                    confidence=0.6,
+                    source_turn=turn_idx,
+                    timestamp=timestamp,
+                ))
+
+    return facts
+
+
+def save_facts_to_store(facts: List[ExtractedFact], fact_store_fn=None) -> int:
+    """Save extracted facts to the fact store.
+
+    Args:
+        facts: List of extracted facts.
+        fact_store_fn: Optional callable(category, entity, content, trust).
+            If None, uses the holographic fact store if available.
+
+    Returns:
+        Number of facts saved.
+    """
+    saved = 0
+
+    if fact_store_fn:
+        for fact in facts:
+            try:
+                fact_store_fn(
+                    category=fact.category,
+                    entity=fact.entity,
+                    content=fact.content,
+                    trust=fact.confidence,
+                )
+                saved += 1
+            except Exception as e:
+                logger.debug("Failed to save fact: %s", e)
+    else:
+        # Try holographic fact store
+        try:
+            from fact_store import fact_store as _fs
+            for fact in facts:
+                try:
+                    _fs(
+                        action="add",
+                        content=fact.content,
+                        category=fact.category,
+                        tags=fact.entity,
+                        trust_delta=fact.confidence - 0.5,
+                    )
+                    saved += 1
+                except Exception as e:
+                    logger.debug("Failed to save fact via fact_store: %s", e)
+        except ImportError:
+            logger.debug("fact_store not available — facts not persisted")
+
+    return saved
+
+
+def extract_and_save_facts(
+    messages: List[Dict[str, Any]],
+    fact_store_fn=None,
+) -> Tuple[List[ExtractedFact], int]:
+    """Extract facts from messages and save them.
+
+    Returns (extracted_facts, saved_count).
+    """
+    facts = extract_facts_from_messages(messages)
+    if facts:
+        logger.info("Extracted %d facts from conversation", len(facts))
+        saved = save_facts_to_store(facts, fact_store_fn)
+        logger.info("Saved %d/%d facts to store", saved, len(facts))
+    else:
+        saved = 0
+    return facts, saved
+
+
+def format_facts_summary(facts: List[ExtractedFact]) -> str:
+    """Format extracted facts as a readable summary."""
+    if not facts:
+        return "No facts extracted."
+
+    by_category = {}
+    for f in facts:
+        by_category.setdefault(f.category, []).append(f)
+
+    lines = [f"Extracted {len(facts)} facts:", ""]
+    for cat, cat_facts in sorted(by_category.items()):
+        lines.append(f"  {cat}:")
+        for f in cat_facts:
+            lines.append(f"    - {f.content[:80]}")
+    return "\n".join(lines)

agent/tool_orchestrator.py

@@ -0,0 +1,177 @@
+"""Tool Orchestrator — Robust execution and circuit breaking for agent tools.
+
+Provides a unified execution service that wraps the tool registry.
+Implements the Circuit Breaker pattern to prevent the agent from getting
+stuck in failure loops when a specific tool or its underlying service
+is flapping or down.
+
+Architecture:
+    Discovery (tools/registry.py) -> Orchestration (agent/tool_orchestrator.py) -> Dispatch
+"""
+
+import json
+import time
+import logging
+import threading
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional, Tuple
+
+from tools.registry import registry
+
+logger = logging.getLogger(__name__)
+
+
+class CircuitState:
+    """States for the tool circuit breaker."""
+    CLOSED = "closed"        # Normal operation
+    OPEN = "open"            # Failing, execution blocked
+    HALF_OPEN = "half_open"  # Testing if service recovered
+
+
+@dataclass
+class ToolStats:
+    """Execution statistics for a tool."""
+    name: str
+    state: str = CircuitState.CLOSED
+    failures: int = 0
+    successes: int = 0
+    last_failure_time: float = 0
+    total_execution_time: float = 0
+    call_count: int = 0
+
+
+class ToolOrchestrator:
+    """Orchestrates tool execution with robustness patterns."""
+
+    def __init__(
+        self,
+        failure_threshold: int = 3,
+        reset_timeout: int = 300,
+    ):
+        """
+        Args:
+            failure_threshold: Number of failures before opening the circuit.
+            reset_timeout: Seconds to wait before transitioning from OPEN to HALF_OPEN.
+        """
+        self.failure_threshold = failure_threshold
+        self.reset_timeout = reset_timeout
+        self._stats: Dict[str, ToolStats] = {}
+        self._lock = threading.Lock()
+
+    def _get_stats(self, name: str) -> ToolStats:
+        """Get or initialize stats for a tool with thread-safe state transition."""
+        with self._lock:
+            if name not in self._stats:
+                self._stats[name] = ToolStats(name=name)
+            
+            stats = self._stats[name]
+            
+            # Transition from OPEN to HALF_OPEN if timeout expired
+            if stats.state == CircuitState.OPEN:
+                if time.time() - stats.last_failure_time > self.reset_timeout:
+                    stats.state = CircuitState.HALF_OPEN
+                    logger.info("Circuit breaker HALF_OPEN for tool: %s", name)
+            
+            return stats
+
+    def _record_success(self, name: str, execution_time: float):
+        """Record a successful tool execution and close the circuit."""
+        with self._lock:
+            stats = self._stats[name]
+            stats.successes += 1
+            stats.call_count += 1
+            stats.total_execution_time += execution_time
+            
+            if stats.state != CircuitState.CLOSED:
+                logger.info("Circuit breaker CLOSED for tool: %s (recovered)", name)
+            
+            stats.state = CircuitState.CLOSED
+            stats.failures = 0
+
+    def _record_failure(self, name: str, execution_time: float):
+        """Record a failed tool execution and potentially open the circuit."""
+        with self._lock:
+            stats = self._stats[name]
+            stats.failures += 1
+            stats.call_count += 1
+            stats.total_execution_time += execution_time
+            stats.last_failure_time = time.time()
+            
+            if stats.state == CircuitState.HALF_OPEN or stats.failures >= self.failure_threshold:
+                stats.state = CircuitState.OPEN
+                logger.warning(
+                    "Circuit breaker OPEN for tool: %s (failures: %d)", 
+                    name, stats.failures
+                )
+
+    def dispatch(self, name: str, args: dict, **kwargs) -> str:
+        """Execute a tool via the registry with circuit breaker protection."""
+        stats = self._get_stats(name)
+        
+        if stats.state == CircuitState.OPEN:
+            return json.dumps({
+                "error": (
+                    f"Tool '{name}' is temporarily unavailable due to repeated failures. "
+                    f"Circuit breaker is OPEN. Please try again in a few minutes or use an alternative tool."
+                ),
+                "circuit_breaker": True,
+                "tool_name": name
+            })
+
+        start_time = time.time()
+        try:
+            # Dispatch to the underlying registry
+            result_str = registry.dispatch(name, args, **kwargs)
+            execution_time = time.time() - start_time
+            
+            # Inspect result for errors. registry.dispatch catches internal
+            # exceptions and returns a JSON error string.
+            is_error = False
+            try:
+                # Lightweight check for error key in JSON
+                if '"error":' in result_str:
+                    res_json = json.loads(result_str)
+                    if isinstance(res_json, dict) and "error" in res_json:
+                        is_error = True
+            except (json.JSONDecodeError, TypeError):
+                # If it's not valid JSON, it's a malformed result (error)
+                is_error = True
+            
+            if is_error:
+                self._record_failure(name, execution_time)
+            else:
+                self._record_success(name, execution_time)
+                
+            return result_str
+            
+        except Exception as e:
+            # This should rarely be hit as registry.dispatch catches most things,
+            # but we guard against orchestrator-level or registry-level bugs.
+            execution_time = time.time() - start_time
+            self._record_failure(name, execution_time)
+            
+            error_msg = f"Tool orchestrator error during {name}: {type(e).__name__}: {e}"
+            logger.exception(error_msg)
+            return json.dumps({
+                "error": error_msg,
+                "tool_name": name,
+                "execution_time": execution_time
+            })
+
+    def get_fleet_stats(self) -> Dict[str, Any]:
+        """Return execution statistics for all tools."""
+        with self._lock:
+            return {
+                name: {
+                    "state": s.state,
+                    "failures": s.failures,
+                    "successes": s.successes,
+                    "avg_time": s.total_execution_time / s.call_count if s.call_count > 0 else 0,
+                    "calls": s.call_count
+                }
+                for name, s in self._stats.items()
+            }
+
+
+# Global orchestrator instance
+orchestrator = ToolOrchestrator()

benchmarks/test_images.json

@@ -0,0 +1,194 @@
+[
+  {
+    "id": "screenshot_github_home",
+    "url": "https://github.githubassets.com/images/modules/logos_page/GitHub-Mark.png",
+    "category": "screenshot",
+    "expected_keywords": ["github", "logo", "mark"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "diagram_mermaid_flow",
+    "url": "https://mermaid.ink/img/pako:eNpdkE9PwzAMxb-K5VOl7gc7sAOIIDuAw9gptnRaSJLSJttQStmXs9LCH-ymBOI1ef_42U6cUSae4IkDxbAAWtB6siSZXVhjQTlgl1nigHg5fRBOzSfebopROCu_cytObSfgLSE1ANOeZWkO2IH5upZxYot8m1hqAdpD_63WRl0xdUG1jdl9kPiOb_EWk2JBtPaiKkF4eVIYgO0EtkW-RSgC4gJ6HJYRG1UNdN0HNVd0Bftjj7X8P92qPj-F8l8T3w",
+    "category": "diagram",
+    "expected_keywords": ["flow", "diagram", "process"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": false}
+  },
+  {
+    "id": "photo_random_1",
+    "url": "https://picsum.photos/seed/vision1/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "photo_random_2",
+    "url": "https://picsum.photos/seed/vision2/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "chart_simple_bar",
+    "url": "https://quickchart.io/chart?c={type:'bar',data:{labels:['Q1','Q2','Q3','Q4'],datasets:[{label:'Revenue',data:[100,150,200,250]}]}}",
+    "category": "chart",
+    "expected_keywords": ["bar", "chart", "revenue"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": true}
+  },
+  {
+    "id": "chart_pie",
+    "url": "https://quickchart.io/chart?c={type:'pie',data:{labels:['A','B','C'],datasets:[{data:[30,50,20]}]}}",
+    "category": "chart",
+    "expected_keywords": ["pie", "chart", "percentage"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": true}
+  },
+  {
+    "id": "diagram_org_chart",
+    "url": "https://mermaid.ink/img/pako:eNpdkE9PwzAMxb-K5VOl7gc7sAOIIDuAw9gptnRaSJLSJttQStmXs9LCH-ymBOI1ef_42U6cUSae4IkDxbAAWtB6iuyIWyrLgXLALrPEAfFy-iCcmk-83RSjcFZ-51ac2k7AW0JqAKY9y9IcsAPzdS3jxBb5NrHUAraH_lutjbpi6oJqG7P7IPEd3-ItJsWCaO1FVYLw8qQwANsJbIt8i1AExAX0OCwjNqoa6LoPaq7oCvbHHmv5f7pVfX4K5b8mvg",
+    "category": "diagram",
+    "expected_keywords": ["organization", "hierarchy", "chart"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": false}
+  },
+  {
+    "id": "screenshot_terminal",
+    "url": "https://raw.githubusercontent.com/nicehash/nicehash-quick-start/main/images/nicehash-terminal.png",
+    "category": "screenshot",
+    "expected_keywords": ["terminal", "command", "output"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "photo_random_3",
+    "url": "https://picsum.photos/seed/vision3/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "chart_line",
+    "url": "https://quickchart.io/chart?c={type:'line',data:{labels:['Jan','Feb','Mar','Apr'],datasets:[{label:'Temperature',data:[5,8,12,18]}]}}",
+    "category": "chart",
+    "expected_keywords": ["line", "chart", "temperature"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": true}
+  },
+  {
+    "id": "diagram_sequence",
+    "url": "https://mermaid.ink/img/pako:eNpdkE9PwzAMxb-K5VOl7gc7sAOIIDuAw9gptnRaSJLSJttQStmXs9LCH-ymBOI1ef_42U6cUSae4IkDxbAAWtB6iuyIWyrLgXLALrPEAfFy-iCcmk-83RSjcFZ-51ac2k7AW0JqAKY9y9IcsAPzdS3jxBb5NrHUAraH_lutjbpi6oJqG7P7IPEd3-ItJsWCaO1FVYLw8qQwANsJbIt8i1AExAX0OCwjNqoa6LoPaq7oCvbHHmv5f7pVfX4K5b8mvg",
+    "category": "diagram",
+    "expected_keywords": ["sequence", "interaction", "message"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": false}
+  },
+  {
+    "id": "photo_random_4",
+    "url": "https://picsum.photos/seed/vision4/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "screenshot_webpage",
+    "url": "https://github.githubassets.com/images/modules/site/social-cards.png",
+    "category": "screenshot",
+    "expected_keywords": ["github", "page", "web"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "chart_radar",
+    "url": "https://quickchart.io/chart?c={type:'radar',data:{labels:['Speed','Power','Defense','Magic'],datasets:[{label:'Hero',data:[80,60,70,90]}]}}",
+    "category": "chart",
+    "expected_keywords": ["radar", "chart", "skill"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": true}
+  },
+  {
+    "id": "photo_random_5",
+    "url": "https://picsum.photos/seed/vision5/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "diagram_class",
+    "url": "https://mermaid.ink/img/pako:eNpdkE9PwzAMxb-K5VOl7gc7sAOIIDuAw9gptnRaSJLSJttQStmXs9LCH-ymBOI1ef_42U6cUSae4IkDxbAAWtB6iuyIWyrLgXLALrPEAfFy-iCcmk-83RSjcFZ-51ac2k7AW0JqAKY9y9IcsAPzdS3jxBb5NrHUAraH_lutjbpi6oJqG7P7IPEd3-ItJsWCaO1FVYLw8qQwANsJbIt8i1AExAX0OCwjNqoa6LoPaq7oCvbHHmv5f7pVfX4K5b8mvg",
+    "category": "diagram",
+    "expected_keywords": ["class", "object", "attribute"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": false}
+  },
+  {
+    "id": "chart_doughnut",
+    "url": "https://quickchart.io/chart?c={type:'doughnut',data:{labels:['Desktop','Mobile','Tablet'],datasets:[{data:[60,30,10]}]}}",
+    "category": "chart",
+    "expected_keywords": ["doughnut", "chart", "device"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": true}
+  },
+  {
+    "id": "photo_random_6",
+    "url": "https://picsum.photos/seed/vision6/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "screenshot_error",
+    "url": "https://http.cat/404.jpg",
+    "category": "screenshot",
+    "expected_keywords": ["404", "error", "cat"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": true}
+  },
+  {
+    "id": "diagram_network",
+    "url": "https://mermaid.ink/img/pako:eNpdkE9PwzAMxb-K5VOl7gc7sAOIIDuAw9gptnRaSJLSJttQStmXs9LCH-ymBOI1ef_42U6cUSae4IkDxbAAWtB6iuyIWyrLgXLALrPEAfFy-iCcmk-83RSjcFZ-51ac2k7AW0JqAKY9y9IcsAPzdS3jxBb5NrHUAraH_lutjbpi6oJqG7P7IPEd3-ItJsWCaO1FVYLw8qQwANsJbIt8i1AExAX0OCwjNqoa6LoPaq7oCvbHHmv5f7pVfX4K5b8mvg",
+    "category": "diagram",
+    "expected_keywords": ["network", "node", "connection"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": false}
+  },
+  {
+    "id": "photo_random_7",
+    "url": "https://picsum.photos/seed/vision7/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "chart_stacked_bar",
+    "url": "https://quickchart.io/chart?c={type:'bar',data:{labels:['2022','2023','2024'],datasets:[{label:'Cloud',data:[100,150,200]},{label:'On-prem',data:[200,180,160]}]},options:{scales:{x:{stacked:true},y:{stacked:true}}}}",
+    "category": "chart",
+    "expected_keywords": ["stacked", "bar", "chart"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 50, "min_sentences": 2, "has_numbers": true}
+  },
+  {
+    "id": "screenshot_dashboard",
+    "url": "https://github.githubassets.com/images/modules/site/features-code-search.png",
+    "category": "screenshot",
+    "expected_keywords": ["search", "code", "feature"],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  },
+  {
+    "id": "photo_random_8",
+    "url": "https://picsum.photos/seed/vision8/400/300",
+    "category": "photo",
+    "expected_keywords": [],
+    "ground_truth_ocr": "",
+    "expected_structure": {"min_length": 30, "min_sentences": 1, "has_numbers": false}
+  }
+]

benchmarks/vision_benchmark.py

@@ -0,0 +1,635 @@
+#!/usr/bin/env python3
+"""
+Vision Benchmark Suite — Issue #817
+
+Compares Gemma 4 vision accuracy vs current approach (Gemini 3 Flash Preview).
+Measures OCR accuracy, description quality, latency, and token usage.
+
+Usage:
+    # Run full benchmark
+    python benchmarks/vision_benchmark.py --images benchmarks/test_images.json
+
+    # Single image test
+    python benchmarks/vision_benchmark.py --url https://example.com/image.png
+
+    # Generate test report
+    python benchmarks/vision_benchmark.py --images benchmarks/test_images.json --output benchmarks/vision_results.json
+
+Test image dataset: benchmarks/test_images.json (50-100 diverse images)
+"""
+
+import argparse
+import asyncio
+import base64
+import json
+import os
+import statistics
+import sys
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+
+# ---------------------------------------------------------------------------
+# Benchmark configuration
+# ---------------------------------------------------------------------------
+
+# Models to compare
+MODELS = {
+    "gemma4": {
+        "model_id": "google/gemma-4-27b-it",
+        "display_name": "Gemma 4 27B",
+        "provider": "nous",
+        "description": "Google's multimodal Gemma 4 model",
+    },
+    "gemini3_flash": {
+        "model_id": "google/gemini-3-flash-preview",
+        "display_name": "Gemini 3 Flash Preview",
+        "provider": "openrouter",
+        "description": "Current default vision model",
+    },
+}
+
+# Evaluation prompts for different test categories
+EVAL_PROMPTS = {
+    "screenshot": "Describe this screenshot in detail. What application is shown? What is the current state of the UI?",
+    "diagram": "Describe this diagram completely. What concepts does it illustrate? List all components and their relationships.",
+    "photo": "Describe this photo in detail. What objects are visible? What is the scene?",
+    "ocr": "Extract ALL text visible in this image. Return it exactly as written, preserving formatting.",
+    "chart": "What data does this chart show? List all axes labels, values, and key trends.",
+    "document": "Extract all text from this document image. Preserve paragraph structure.",
+}
+
+
+# ---------------------------------------------------------------------------
+# Vision model interface
+# ---------------------------------------------------------------------------
+
+
+async def analyze_with_model(
+    image_url: str,
+    prompt: str,
+    model_config: dict,
+    timeout: float = 120.0,
+) -> dict:
+    """Call a vision model and return structured results.
+
+    Returns dict with:
+        - analysis: str
+        - latency_ms: float
+        - tokens: dict (prompt_tokens, completion_tokens, total_tokens)
+        - success: bool
+        - error: str (if failed)
+    """
+    import httpx
+
+    provider = model_config["provider"]
+    model_id = model_config["model_id"]
+
+    # Prepare messages
+    messages = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "text", "text": prompt},
+                {"type": "image_url", "image_url": {"url": image_url}},
+            ],
+        }
+    ]
+
+    # Route to provider
+    if provider == "openrouter":
+        api_url = "https://openrouter.ai/api/v1/chat/completions"
+        api_key = os.getenv("OPENROUTER_API_KEY", "")
+    elif provider == "nous":
+        api_url = "https://inference.nousresearch.com/v1/chat/completions"
+        api_key = os.getenv("NOUS_API_KEY", "") or os.getenv("NOUS_INFERENCE_API_KEY", "")
+    else:
+        api_url = os.getenv(f"{provider.upper()}_API_URL", "")
+        api_key = os.getenv(f"{provider.upper()}_API_KEY", "")
+
+    if not api_key:
+        return {
+            "analysis": "",
+            "latency_ms": 0,
+            "tokens": {},
+            "success": False,
+            "error": f"No API key for provider {provider}",
+        }
+
+    headers = {
+        "Authorization": f"Bearer {api_key}",
+        "Content-Type": "application/json",
+    }
+
+    payload = {
+        "model": model_id,
+        "messages": messages,
+        "max_tokens": 2000,
+        "temperature": 0.1,
+    }
+
+    start = time.perf_counter()
+    try:
+        async with httpx.AsyncClient(timeout=timeout) as client:
+            resp = await client.post(api_url, json=payload, headers=headers)
+            resp.raise_for_status()
+            data = resp.json()
+
+        latency_ms = (time.perf_counter() - start) * 1000
+
+        analysis = ""
+        choices = data.get("choices", [])
+        if choices:
+            msg = choices[0].get("message", {})
+            analysis = msg.get("content", "")
+
+        usage = data.get("usage", {})
+        tokens = {
+            "prompt_tokens": usage.get("prompt_tokens", 0),
+            "completion_tokens": usage.get("completion_tokens", 0),
+            "total_tokens": usage.get("total_tokens", 0),
+        }
+
+        return {
+            "analysis": analysis,
+            "latency_ms": round(latency_ms, 1),
+            "tokens": tokens,
+            "success": True,
+            "error": "",
+        }
+
+    except Exception as e:
+        return {
+            "analysis": "",
+            "latency_ms": round((time.perf_counter() - start) * 1000, 1),
+            "tokens": {},
+            "success": False,
+            "error": str(e),
+        }
+
+
+# ---------------------------------------------------------------------------
+# Evaluation metrics
+# ---------------------------------------------------------------------------
+
+
+def compute_ocr_accuracy(extracted: str, ground_truth: str) -> float:
+    """Compute OCR accuracy using character-level Levenshtein ratio.
+
+    Returns 0.0-1.0 (1.0 = perfect match).
+    """
+    if not ground_truth:
+        return 1.0 if not extracted else 0.0
+    if not extracted:
+        return 0.0
+
+    # Normalized Levenshtein similarity
+    extracted_lower = extracted.lower().strip()
+    truth_lower = ground_truth.lower().strip()
+
+    # Simple character overlap ratio (fast proxy)
+    max_len = max(len(extracted_lower), len(truth_lower))
+    if max_len == 0:
+        return 1.0
+
+    # Count matching characters at matching positions
+    matches = sum(1 for a, b in zip(extracted_lower, truth_lower) if a == b)
+    position_ratio = matches / max_len
+
+    # Also check word-level overlap
+    extracted_words = set(extracted_lower.split())
+    truth_words = set(truth_lower.split())
+    if truth_words:
+        word_recall = len(extracted_words & truth_words) / len(truth_words)
+    else:
+        word_recall = 1.0 if not extracted_words else 0.0
+
+    return round((position_ratio * 0.4 + word_recall * 0.6), 4)
+
+
+def compute_description_completeness(analysis: str, expected_keywords: list) -> float:
+    """Score description completeness based on keyword coverage.
+
+    Returns 0.0-1.0.
+    """
+    if not expected_keywords:
+        return 1.0
+    if not analysis:
+        return 0.0
+
+    analysis_lower = analysis.lower()
+    found = sum(1 for kw in expected_keywords if kw.lower() in analysis_lower)
+    return round(found / len(expected_keywords), 4)
+
+
+def compute_structural_accuracy(analysis: str, expected_structure: dict) -> dict:
+    """Evaluate structural elements of the analysis.
+
+    Returns dict with per-element scores.
+    """
+    scores = {}
+
+    # Length check
+    min_length = expected_structure.get("min_length", 50)
+    scores["length"] = min(len(analysis) / min_length, 1.0) if min_length > 0 else 1.0
+
+    # Sentence count
+    min_sentences = expected_structure.get("min_sentences", 2)
+    sentence_count = analysis.count(".") + analysis.count("!") + analysis.count("?")
+    scores["sentences"] = min(sentence_count / max(min_sentences, 1), 1.0)
+
+    # Has specifics (numbers, names, etc.)
+    if expected_structure.get("has_numbers", False):
+        import re
+        scores["has_numbers"] = 1.0 if re.search(r'\d', analysis) else 0.0
+
+    return scores
+
+
+# ---------------------------------------------------------------------------
+# Benchmark runner
+# ---------------------------------------------------------------------------
+
+
+async def run_single_test(
+    image: dict,
+    models: dict,
+    runs_per_model: int = 1,
+) -> dict:
+    """Run a single image through all models.
+
+    Args:
+        image: dict with url, category, expected_keywords, ground_truth_ocr, etc.
+        models: dict of model configs to test
+        runs_per_model: number of runs per model (for consistency testing)
+
+    Returns dict with results per model.
+    """
+    category = image.get("category", "photo")
+    prompt = EVAL_PROMPTS.get(category, EVAL_PROMPTS["photo"])
+    url = image["url"]
+
+    results = {}
+
+    for model_name, model_config in models.items():
+        runs = []
+        for run_i in range(runs_per_model):
+            result = await analyze_with_model(url, prompt, model_config)
+            runs.append(result)
+            if run_i < runs_per_model - 1:
+                await asyncio.sleep(1)  # Rate limit courtesy
+
+        # Aggregate
+        successful = [r for r in runs if r["success"]]
+        if successful:
+            avg_latency = statistics.mean(r["latency_ms"] for r in successful)
+            avg_tokens = statistics.mean(
+                r["tokens"].get("total_tokens", 0) for r in successful
+            )
+            # Use first successful run for accuracy metrics
+            primary = successful[0]
+
+            # Compute accuracy
+            ocr_score = None
+            if image.get("ground_truth_ocr"):
+                ocr_score = compute_ocr_accuracy(
+                    primary["analysis"], image["ground_truth_ocr"]
+                )
+
+            keyword_score = None
+            if image.get("expected_keywords"):
+                keyword_score = compute_description_completeness(
+                    primary["analysis"], image["expected_keywords"]
+                )
+
+            structural = compute_structural_accuracy(
+                primary["analysis"], image.get("expected_structure", {})
+            )
+
+            results[model_name] = {
+                "success": True,
+                "analysis_preview": primary["analysis"][:300],
+                "analysis_length": len(primary["analysis"]),
+                "avg_latency_ms": round(avg_latency, 1),
+                "avg_tokens": round(avg_tokens, 1),
+                "ocr_accuracy": ocr_score,
+                "keyword_completeness": keyword_score,
+                "structural_scores": structural,
+                "consistency": round(
+                    statistics.stdev(len(r["analysis"]) for r in successful), 1
+                ) if len(successful) > 1 else 0.0,
+                "runs": len(successful),
+                "errors": len(runs) - len(successful),
+            }
+        else:
+            results[model_name] = {
+                "success": False,
+                "error": runs[0]["error"] if runs else "No runs",
+                "runs": 0,
+                "errors": len(runs),
+            }
+
+    return results
+
+
+async def run_benchmark_suite(
+    images: List[dict],
+    models: dict,
+    runs_per_model: int = 1,
+) -> dict:
+    """Run the full benchmark suite.
+
+    Args:
+        images: list of image test cases
+        models: model configs to compare
+        runs_per_model: consistency runs per image
+
+    Returns structured benchmark report.
+    """
+    total = len(images)
+    all_results = []
+
+    print(f"\nRunning vision benchmark: {total} images x {len(models)} models x {runs_per_model} runs")
+    print(f"Models: {', '.join(m['display_name'] for m in models.values())}\n")
+
+    for i, image in enumerate(images):
+        img_id = image.get("id", f"img_{i}")
+        category = image.get("category", "unknown")
+        print(f"  [{i+1}/{total}] {img_id} ({category})...", end=" ", flush=True)
+
+        result = await run_single_test(image, models, runs_per_model)
+        result["image_id"] = img_id
+        result["category"] = category
+        all_results.append(result)
+
+        # Quick status
+        statuses = []
+        for mname in models:
+            if result[mname]["success"]:
+                lat = result[mname]["avg_latency_ms"]
+                statuses.append(f"{mname}:{lat:.0f}ms")
+            else:
+                statuses.append(f"{mname}:FAIL")
+        print(", ".join(statuses))
+
+    # Aggregate statistics
+    summary = aggregate_results(all_results, models)
+
+    return {
+        "generated_at": datetime.now(timezone.utc).isoformat(),
+        "config": {
+            "total_images": total,
+            "runs_per_model": runs_per_model,
+            "models": {k: v["display_name"] for k, v in models.items()},
+        },
+        "results": all_results,
+        "summary": summary,
+    }
+
+
+def aggregate_results(results: List[dict], models: dict) -> dict:
+    """Compute aggregate statistics across all test images."""
+    summary = {}
+
+    for model_name in models:
+        model_results = [r[model_name] for r in results if r[model_name]["success"]]
+        failed = [r[model_name] for r in results if not r[model_name]["success"]]
+
+        if not model_results:
+            summary[model_name] = {"success_rate": 0, "error": "All runs failed"}
+            continue
+
+        latencies = [r["avg_latency_ms"] for r in model_results]
+        tokens = [r["avg_tokens"] for r in model_results if r.get("avg_tokens")]
+        ocr_scores = [r["ocr_accuracy"] for r in model_results if r.get("ocr_accuracy") is not None]
+        keyword_scores = [r["keyword_completeness"] for r in model_results if r.get("keyword_completeness") is not None]
+
+        summary[model_name] = {
+            "success_rate": round(len(model_results) / (len(model_results) + len(failed)), 4),
+            "total_runs": len(model_results),
+            "total_failures": len(failed),
+            "latency": {
+                "mean_ms": round(statistics.mean(latencies), 1),
+                "median_ms": round(statistics.median(latencies), 1),
+                "p95_ms": round(sorted(latencies)[int(len(latencies) * 0.95)], 1),
+                "std_ms": round(statistics.stdev(latencies), 1) if len(latencies) > 1 else 0,
+            },
+            "tokens": {
+                "mean_total": round(statistics.mean(tokens), 1) if tokens else 0,
+                "total_used": sum(int(t) for t in tokens),
+            },
+            "accuracy": {
+                "ocr_mean": round(statistics.mean(ocr_scores), 4) if ocr_scores else None,
+                "ocr_count": len(ocr_scores),
+                "keyword_mean": round(statistics.mean(keyword_scores), 4) if keyword_scores else None,
+                "keyword_count": len(keyword_scores),
+            },
+        }
+
+    return summary
+
+
+# ---------------------------------------------------------------------------
+# Report generation
+# ---------------------------------------------------------------------------
+
+
+def to_markdown(report: dict) -> str:
+    """Generate human-readable markdown report."""
+    summary = report["summary"]
+    config = report["config"]
+    model_names = list(config["models"].values())
+
+    lines = [
+        "# Vision Benchmark Report",
+        "",
+        f"Generated: {report['generated_at'][:16]}",
+        f"Images tested: {config['total_images']}",
+        f"Runs per model: {config['runs_per_model']}",
+        f"Models: {', '.join(model_names)}",
+        "",
+        "## Latency Comparison",
+        "",
+        "| Model | Mean (ms) | Median | P95 | Std Dev |",
+        "|-------|-----------|--------|-----|---------|",
+    ]
+
+    for mkey, mname in config["models"].items():
+        if mkey in summary and "latency" in summary[mkey]:
+            lat = summary[mkey]["latency"]
+            lines.append(
+                f"| {mname} | {lat['mean_ms']:.0f} | {lat['median_ms']:.0f} | "
+                f"{lat['p95_ms']:.0f} | {lat['std_ms']:.0f} |"
+            )
+
+    lines += [
+        "",
+        "## Accuracy Comparison",
+        "",
+        "| Model | OCR Accuracy | Keyword Coverage | Success Rate |",
+        "|-------|-------------|-----------------|--------------|",
+    ]
+
+    for mkey, mname in config["models"].items():
+        if mkey in summary and "accuracy" in summary[mkey]:
+            acc = summary[mkey]["accuracy"]
+            sr = summary[mkey].get("success_rate", 0)
+            ocr = f"{acc['ocr_mean']:.1%}" if acc["ocr_mean"] is not None else "N/A"
+            kw = f"{acc['keyword_mean']:.1%}" if acc["keyword_mean"] is not None else "N/A"
+            lines.append(f"| {mname} | {ocr} | {kw} | {sr:.1%} |")
+
+    lines += [
+        "",
+        "## Token Usage",
+        "",
+        "| Model | Mean Tokens/Image | Total Tokens |",
+        "|-------|------------------|--------------|",
+    ]
+
+    for mkey, mname in config["models"].items():
+        if mkey in summary and "tokens" in summary[mkey]:
+            tok = summary[mkey]["tokens"]
+            lines.append(
+                f"| {mname} | {tok['mean_total']:.0f} | {tok['total_used']} |"
+            )
+
+    # Verdict
+    lines += ["", "## Verdict", ""]
+
+    # Find best model by composite score
+    best_model = None
+    best_score = -1
+    for mkey, mname in config["models"].items():
+        if mkey not in summary or "accuracy" not in summary[mkey]:
+            continue
+        acc = summary[mkey]["accuracy"]
+        sr = summary[mkey].get("success_rate", 0)
+        ocr = acc["ocr_mean"] or 0
+        kw = acc["keyword_mean"] or 0
+        # Weighted composite: 40% OCR, 30% keyword, 30% success rate
+        score = (ocr * 0.4 + kw * 0.3 + sr * 0.3)
+        if score > best_score:
+            best_score = score
+            best_model = mname
+
+    if best_model:
+        lines.append(f"**Best overall: {best_model}** (composite score: {best_score:.1%})")
+    else:
+        lines.append("No clear winner — insufficient data.")
+
+    return "\n".join(lines)
+
+
+# ---------------------------------------------------------------------------
+# Test dataset management
+# ---------------------------------------------------------------------------
+
+
+def generate_sample_dataset() -> List[dict]:
+    """Generate a sample test dataset with diverse public images.
+
+    Returns list of test image definitions.
+    """
+    return [
+        # Screenshots
+        {
+            "id": "screenshot_github",
+            "url": "https://github.githubassets.com/images/modules/logos_page/GitHub-Mark.png",
+            "category": "screenshot",
+            "expected_keywords": ["github", "logo", "octocat"],
+            "expected_structure": {"min_length": 50, "min_sentences": 2},
+        },
+        # Diagrams
+        {
+            "id": "diagram_architecture",
+            "url": "https://mermaid.ink/img/pako:eNp9kMtOwzAQRX_F8hKpJbhJFVJBi1QJiMWCG8eZNsGJLdlOiqIid5RdufiHnZRA7GbuzJwZe4ZGH2SCBPYUwgxoQKvJnCR2YY0F5YBdJJkD4uX0oXB6PnF3U4zCWcWdW3FqOwGvCKkBmHKSTB2gJeRrLTeJLfJdJKkBGYf9P1sTNdUXVJqY3YNJK7xLVwR0mxJFU6rCgEKnhSGIL2Eq8BdEERAX0OGwEiVQ1R0MaNFR8QfqKxmHigbX8VLjDz_Q0L8Wc_qPxDw",
+            "category": "diagram",
+            "expected_keywords": ["architecture", "component", "service"],
+            "expected_structure": {"min_length": 100, "min_sentences": 3},
+        },
+        # Photos
+        {
+            "id": "photo_nature",
+            "url": "https://picsum.photos/seed/bench1/400/300",
+            "category": "photo",
+            "expected_keywords": [],
+            "expected_structure": {"min_length": 30, "min_sentences": 1},
+        },
+        # Charts
+        {
+            "id": "chart_bar",
+            "url": "https://quickchart.io/chart?c={type:'bar',data:{labels:['Q1','Q2','Q3','Q4'],datasets:[{label:'Users',data:[50,60,70,80]}]}}",
+            "category": "chart",
+            "expected_keywords": ["bar", "chart", "data"],
+            "expected_structure": {"min_length": 50, "min_sentences": 2},
+        },
+    ]
+
+
+def load_dataset(path: str) -> List[dict]:
+    """Load test dataset from JSON file."""
+    with open(path) as f:
+        return json.load(f)
+
+
+# ---------------------------------------------------------------------------
+# CLI
+# ---------------------------------------------------------------------------
+
+
+async def main():
+    parser = argparse.ArgumentParser(description="Vision Benchmark Suite (Issue #817)")
+    parser.add_argument("--images", help="Path to test images JSON file")
+    parser.add_argument("--url", help="Single image URL to test")
+    parser.add_argument("--category", default="photo", help="Category for single URL")
+    parser.add_argument("--output", default=None, help="Output JSON file")
+    parser.add_argument("--runs", type=int, default=1, help="Runs per model per image")
+    parser.add_argument("--models", nargs="+", default=None,
+                        help="Models to test (default: all)")
+    parser.add_argument("--markdown", action="store_true", help="Output markdown report")
+    parser.add_argument("--generate-dataset", action="store_true",
+                        help="Generate sample dataset and exit")
+    args = parser.parse_args()
+
+    if args.generate_dataset:
+        dataset = generate_sample_dataset()
+        out_path = args.images or "benchmarks/test_images.json"
+        os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
+        with open(out_path, "w") as f:
+            json.dump(dataset, f, indent=2)
+        print(f"Generated sample dataset: {out_path} ({len(dataset)} images)")
+        return
+
+    # Select models
+    if args.models:
+        selected = {k: v for k, v in MODELS.items() if k in args.models}
+    else:
+        selected = MODELS
+
+    # Load images
+    if args.url:
+        images = [{"id": "single", "url": args.url, "category": args.category}]
+    elif args.images:
+        images = load_dataset(args.images)
+    else:
+        print("ERROR: Provide --images or --url")
+        sys.exit(1)
+
+    # Run benchmark
+    report = await run_benchmark_suite(images, selected, args.runs)
+
+    # Output
+    if args.output:
+        os.makedirs(os.path.dirname(args.output) or ".", exist_ok=True)
+        with open(args.output, "w") as f:
+            json.dump(report, f, indent=2)
+        print(f"\nResults saved to {args.output}")
+
+    if args.markdown or not args.output:
+        print("\n" + to_markdown(report))
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

cli.py

@@ -3611,8 +3611,8 @@ class HermesCLI:
             available, unavailable = check_tool_availability()
             
             # Filter to only those missing API keys (not system deps)
-            api_key_missing = [u for u in unavailable if u["missing_vars"]]
-            
+            api_key_missing = [u for u in unavailable if u.get("env_vars") or u.get("missing_vars")]
+
             if api_key_missing:
                 self.console.print()
                 self.console.print("[yellow]⚠️  Some tools disabled (missing API keys):[/]")
@@ -3620,7 +3620,8 @@ class HermesCLI:
                     tools_str = ", ".join(item["tools"][:2])  # Show first 2 tools
                     if len(item["tools"]) > 2:
                         tools_str += f", +{len(item['tools'])-2} more"
-                    self.console.print(f"   [dim]• {item['name']}[/] [dim italic]({', '.join(item['missing_vars'])})[/]")
+                    env_vars = item.get("env_vars") or item.get("missing_vars") or []
+                    self.console.print(f"   [dim]• {item['name']}[/] [dim italic]({', '.join(env_vars)})[/]")
                 self.console.print("[dim]   Run 'hermes setup' to configure[/]")
         except Exception:
             pass  # Don't crash on import errors

docs/WORKFLOW_ORCHESTRATION_RESEARCH.md

@@ -0,0 +1,432 @@
+# Workflow Orchestration & Task Queue Research for AI Agents
+
+**Date:** 2026-04-14
+**Scope:** SOTA comparison of task queues and workflow orchestrators for autonomous AI agent workflows
+
+---
+
+## 1. Current Architecture: Cron + Webhook
+
+### How it works
+- **Scheduler:** `cron/scheduler.py` — gateway calls `tick()` every 60 seconds
+- **Storage:** JSON file (`~/.hermes/cron/jobs.json`) + file-based lock (`cron/.tick.lock`)
+- **Execution:** Each job spawns a full `AIAgent.run_conversation()` in a thread pool with inactivity timeout
+- **Delivery:** Results pushed back to origin chat via platform adapters (Telegram, Discord, etc.)
+- **Checkpointing:** Job outputs saved to `~/.hermes/cron/output/{job_id}/{timestamp}.md`
+
+### Strengths
+- Simple, zero-dependency (no broker/redis needed)
+- Jobs are isolated — each runs a fresh agent session
+- Direct platform delivery with E2EE support
+- Script pre-run for data collection
+- Inactivity-based timeout (not hard wall-clock)
+
+### Weaknesses
+- **No task dependencies** — jobs are completely independent
+- **No retry logic** — single failure = lost run (recurring jobs advance schedule and move on)
+- **No concurrency control** — all due jobs fire at once; no worker pool sizing
+- **No observability** — no metrics, no dashboard, no structured logging of job state transitions
+- **Tick-based polling** — 60s granularity, wastes cycles when idle, adds latency when busy
+- **Single-process** — file lock means only one tick at a time; no horizontal scaling
+- **No dead letter queue** — failed deliveries are logged but not retried
+- **No workflow chaining** — cannot express "run A, then B with A's output"
+
+---
+
+## 2. Framework Comparison
+
+### 2.1 Huey (Already Installed v2.6.0)
+
+**Architecture:** Embedded task queue, SQLite/Redis/file storage, consumer process model.
+
+| Feature | Huey | Our Cron |
+|---|---|---|
+| Broker | SQLite (default), Redis | JSON file |
+| Retry | Built-in: `retries=N, retry_delay=S` | None |
+| Task chaining | `task1.s() | task2.s()` (pipeline) | None |
+| Scheduling | `@huey.periodic_task(crontab(...))` | Our own cron parser |
+| Concurrency | Worker pool with `-w N` flag | Single tick lock |
+| Monitoring | `huey_consumer` logs, Huey Admin (Django) | Manual log reading |
+| Failure recovery | Automatic retry + configurable backoff | None |
+| Priority | `PriorityRedisExpireHuey` or task priority | None |
+| Result storage | `store_results=True` with result() | File output |
+
+**Task Dependencies Pattern:**
+```python
+@huey.task()
+def analyze_data(input_data):
+    return run_analysis(input_data)
+
+@huey.task()
+def generate_report(analysis_result):
+    return create_report(analysis_result)
+
+# Pipeline: analyze then report
+pipeline = analyze_data.s(raw_data) | generate_report.s()
+result = pipeline()
+```
+
+**Retry Pattern:**
+```python
+@huey.task(retries=3, retry_delay=60, retry_backoff=True)
+def flaky_api_call(url):
+    return requests.get(url, timeout=30)
+```
+
+**Benchmarks:** ~5,000 tasks/sec with SQLite backend, ~15,000 with Redis. Sub-millisecond scheduling latency. Very lightweight — single process.
+
+**Verdict:** Best fit for our use case. Already installed. SQLite backend = no external deps. Can layer on top of our existing job storage.
+
+---
+
+### 2.2 Celery
+
+**Architecture:** Distributed task queue with message broker (RabbitMQ/Redis).
+
+| Feature | Celery | Huey |
+|---|---|---|
+| Broker | Redis, RabbitMQ, SQS (required) | SQLite (built-in) |
+| Scale | 100K+ tasks/sec | ~5-15K tasks/sec |
+| Chains | `chain(task1.s(), task2.s())` | Pipeline operator |
+| Groups/Chords | Parallel + callback | Not built-in |
+| Canvas | Full workflow DSL (chain, group, chord, map) | Basic pipeline |
+| Monitoring | Flower dashboard, Celery events | Minimal |
+| Complexity | Heavy — needs broker, workers, result backend | Single process |
+
+**Workflow Pattern:**
+```python
+from celery import chain, group, chord
+
+# Chain: sequential
+workflow = chain(fetch_data.s(), analyze.s(), report.s())
+
+# Group: parallel
+parallel = group(fetch_twitter.s(), fetch_reddit.s(), fetch_hn.s())
+
+# Chord: parallel then callback
+chord(parallel, aggregate_results.s())
+```
+
+**Verdict:** Overkill for our scale. Adds RabbitMQ/Redis dependency. The Canvas API is powerful but we don't need 100K task/sec throughput. Flower monitoring is nice but we'd need to deploy it separately.
+
+---
+
+### 2.3 Temporal
+
+**Architecture:** Durable execution engine. Workflows as code with automatic state persistence and replay.
+
+| Feature | Temporal | Our Cron |
+|---|---|---|
+| State management | Automatic — workflow state persisted on every step | Manual JSON files |
+| Failure recovery | Workflows survive process restarts, auto-retry | Lost on crash |
+| Task dependencies | Native — activities call other activities | None |
+| Long-running tasks | Built-in (days/months OK) | Inactivity timeout |
+| Versioning | Workflow versioning for safe updates | No versioning |
+| Visibility | Full workflow state at any point | Log files |
+| Infrastructure | Requires Temporal server + database | None |
+| Language | Python SDK, but Temporal server is Go | Pure Python |
+
+**Workflow Pattern:**
+```python
+@workflow.defn
+class AIAgentWorkflow:
+    @workflow.run
+    async def run(self, job_config: dict) -> str:
+        # Step 1: Fetch data
+        data = await workflow.execute_activity(
+            fetch_data_activity,
+            job_config["script"],
+            start_to_close_timeout=timedelta(minutes=5),
+            retry_policy=RetryPolicy(maximum_attempts=3),
+        )
+        
+        # Step 2: Analyze with AI agent
+        analysis = await workflow.execute_activity(
+            run_agent_activity,
+            {"prompt": job_config["prompt"], "context": data},
+            start_to_close_timeout=timedelta(minutes=30),
+            retry_policy=RetryPolicy(
+                initial_interval=timedelta(seconds=60),
+                maximum_attempts=3,
+            ),
+        )
+        
+        # Step 3: Deliver
+        await workflow.execute_activity(
+            deliver_activity,
+            {"platform": job_config["deliver"], "content": analysis},
+            start_to_close_timeout=timedelta(seconds=60),
+        )
+        return analysis
+```
+
+**Verdict:** Best architecture for complex multi-step AI workflows, but heavy infrastructure cost. Temporal server needs PostgreSQL/Cassandra + visibility store. Ideal if we reach 50+ multi-step workflows with complex failure modes. Overkill for current needs.
+
+---
+
+### 2.4 Prefect
+
+**Architecture:** Modern data/workflow orchestration with Python-native API.
+
+| Feature | Prefect |
+|---|---|
+| Dependencies | SQLite (default) or PostgreSQL |
+| Task retries | `@task(retries=3, retry_delay_seconds=10)` |
+| Task dependencies | `result = task_a(wait_for=[task_b])` |
+| Caching | `cache_key_fn` for result caching |
+| Subflows | Nested workflow composition |
+| Deployments | Schedule via `Deployment` or `CronSchedule` |
+| UI | Excellent web dashboard |
+| Async | Full async support |
+
+**Workflow Pattern:**
+```python
+from prefect import flow, task
+from prefect.tasks import task_input_hash
+
+@task(retries=3, retry_delay_seconds=30)
+def run_agent(prompt: str) -> str:
+    agent = AIAgent(...)
+    return agent.run_conversation(prompt)
+
+@task(cache_key_fn=task_input_hash, cache_expiration=timedelta(hours=1))
+def fetch_context(script: str) -> str:
+    return run_script(script)
+
+@flow(name="agent-workflow")
+def agent_workflow(job_config: dict):
+    context = fetch_context(job_config.get("script", ""))
+    result = run_agent(
+        f"{context}\n\n{job_config['prompt']}",
+        wait_for=[context]
+    )
+    deliver(result, job_config["deliver"])
+    return result
+```
+
+**Benchmarks:** Sub-second task scheduling. Handles 10K+ concurrent task runs. SQLite backend for single-node.
+
+**Verdict:** Strong alternative. Pythonic, good UI, built-in scheduling. But heavier than Huey — deploys a server process. Best if we want a web dashboard for monitoring. Less infrastructure than Temporal but more than Huey.
+
+---
+
+### 2.5 Apache Airflow
+
+**Architecture:** Batch-oriented DAG scheduler, Python-based.
+
+| Feature | Airflow |
+|---|---|
+| DAG model | Static DAGs defined in Python files |
+| Scheduler | Polling-based, 5-30s granularity |
+| Dependencies | PostgreSQL/MySQL + Redis/RabbitMQ + webserver |
+| UI | Rich web UI with DAG visualization |
+| Best for | ETL, data pipelines, batch processing |
+| Weakness | Not designed for dynamic task creation; heavy; DAG definition overhead |
+
+**Verdict:** Wrong tool for this job. Airflow excels at static, well-defined data pipelines (ETL). Our agent workflows are dynamic — tasks are created at runtime based on user prompts. Airflow's DAG model fights against this. Massive overhead (needs webserver, scheduler, worker, metadata DB).
+
+---
+
+### 2.6 Dramatiq
+
+**Architecture:** Lightweight distributed task queue, Celery alternative.
+
+| Feature | Dramatiq |
+|---|---|
+| Broker | Redis, RabbitMQ |
+| Retries | `@dramatiq.actor(max_retries=3)` |
+| Middleware | Pluggable: age_limit, time_limit, retries, callbacks |
+| Groups | `group(actor.message(...), ...).run()` |
+| Pipes | `actor.message() | other_actor.message()` |
+| Simplicity | Cleaner API than Celery |
+
+**Verdict:** Nice middle ground between Huey and Celery. But still requires a broker (Redis/RabbitMQ). No SQLite backend. Less ecosystem than Celery, less lightweight than Huey.
+
+---
+
+### 2.7 RQ (Redis Queue)
+
+**Architecture:** Minimal Redis-based task queue.
+
+| Feature | RQ |
+|---|---|
+| Broker | Redis only |
+| Retries | Via `Retry` class |
+| Workers | Simple worker processes |
+| Dashboard | `rq-dashboard` (separate) |
+| Limitation | Redis-only, no SQLite, no scheduling built-in |
+
+**Verdict:** Too simple and Redis-dependent. No periodic task support without `rq-scheduler`. No task chaining without third-party. Not competitive with Huey for our use case.
+
+---
+
+## 3. Architecture Patterns for AI Agent Workflows
+
+### 3.1 Task Chaining (Fan-out / Fan-in)
+
+The critical pattern for multi-step AI workflows:
+
+```
+[Script] → [Agent] → [Deliver]
+    ↓          ↓          ↓
+  Context    Report    Notification
+```
+
+**Implementation with Huey:**
+```python
+@huey.task(retries=2)
+def run_script_task(script_path):
+    return run_script(script_path)
+
+@huey.task(retries=3, retry_delay=60)
+def run_agent_task(prompt, context=None):
+    if context:
+        prompt = f"## Context\n{context}\n\n{prompt}"
+    agent = AIAgent(...)
+    return agent.run_conversation(prompt)
+
+@huey.task()
+def deliver_task(result, job_config):
+    return deliver_result(job_config, result)
+
+# Compose: script → agent → deliver
+def compose_workflow(job):
+    steps = []
+    if job.get("script"):
+        steps.append(run_script_task.s(job["script"]))
+    steps.append(run_agent_task.s(job["prompt"]))
+    steps.append(deliver_task.s(job))
+    return reduce(lambda a, b: a.then(b), steps)
+```
+
+### 3.2 Retry with Exponential Backoff
+
+```python
+from huey import RetryTask
+
+class AIWorkflowTask(RetryTask):
+    retries = 3
+    retry_delay = 30        # Start at 30s
+    retry_backoff = True    # 30s → 60s → 120s
+    max_retry_delay = 600   # Cap at 10min
+```
+
+### 3.3 Dead Letter Queue
+
+For tasks that exhaust retries:
+```python
+@huey.task(retries=3)
+def flaky_task(data):
+    ...
+
+# Dead letter handling
+def handle_failure(task, exc, retries):
+    # Log to dead letter store
+    save_dead_letter(task, exc, retries)
+    # Notify user of failure
+    notify_user(f"Task {task.name} failed after {retries} retries: {exc}")
+```
+
+### 3.4 Observability Pattern
+
+```python
+# Structured event logging for every state transition
+def emit_event(job_id, event_type, metadata):
+    event = {
+        "job_id": job_id,
+        "event": event_type,  # scheduled, started, completed, failed, retried
+        "timestamp": iso_now(),
+        "metadata": metadata,
+    }
+    append_to_event_log(event)
+    # Also emit to metrics (Prometheus/StatsD)
+    metrics.increment(f"cron.{event_type}")
+```
+
+---
+
+## 4. Benchmarks Summary
+
+| Framework | Throughput | Latency | Memory | Startup | Dependencies |
+|---|---|---|---|---|---|
+| Current Cron | ~1 job/60s tick | 60-120s | Minimal | Instant | None |
+| Huey (SQLite) | ~5K tasks/sec | <10ms | ~20MB | <1s | None |
+| Huey (Redis) | ~15K tasks/sec | <5ms | ~20MB | <1s | Redis |
+| Celery (Redis) | ~15K tasks/sec | <10ms | ~100MB | ~3s | Redis |
+| Temporal | ~50K activities/sec | <5ms | ~200MB | ~10s | Temporal server+DB |
+| Prefect | ~10K tasks/sec | <20ms | ~150MB | ~5s | PostgreSQL |
+
+---
+
+## 5. Recommendations
+
+### Immediate (Phase 1): Enhance Current Cron
+
+Add these capabilities to the existing `cron/` module **without** switching frameworks:
+
+1. **Retry logic** — Add `retry_count`, `retry_delay`, `max_retries` fields to job JSON. In `scheduler.py tick()`, on failure: if `retries_remaining > 0`, don't advance schedule, set `next_run_at = now + retry_delay * (attempt^2)`.
+
+2. **Backoff** — Exponential: `delay * 2^attempt`, capped at 10 minutes.
+
+3. **Dead letter tracking** — After max retries, mark job state as `dead_letter` and emit a delivery notification with the error.
+
+4. **Concurrency limit** — Add a semaphore (e.g., `max_concurrent=3`) to `tick()` so we don't spawn 20 agents simultaneously.
+
+5. **Structured events** — Append JSON events to `~/.hermes/cron/events.jsonl` for every state transition (scheduled, started, completed, failed, retried, delivered).
+
+**Effort:** ~1-2 days. No new dependencies.
+
+### Medium-term (Phase 2): Adopt Huey for Workflow Chaining
+
+When we need task dependencies (multi-step agent workflows), migrate to Huey:
+
+1. **Keep the JSON job store** as the source of truth for user-facing job management.
+2. **Use Huey as the execution engine** — enqueue tasks from `tick()`, let Huey handle retries, scheduling, and chaining.
+3. **SQLite backend** — no new infrastructure. One consumer process (`huey_consumer.py`) alongside the gateway.
+4. **Task chaining for multi-step jobs** — `script_task.then(agent_task).then(delivery_task)`.
+
+**Migration path:**
+- Phase 2a: Run Huey consumer alongside gateway. Mirror cron jobs to Huey periodic tasks.
+- Phase 2b: Add task chaining for jobs with scripts.
+- Phase 2c: Migrate all jobs to Huey, deprecate tick()-based execution.
+
+**Effort:** ~1 week. Huey already installed. Gateway integration ~2-3 days.
+
+### Long-term (Phase 3): Evaluate Temporal/Prefect
+
+Only if:
+- We have 100+ concurrent multi-step workflows
+- We need workflow versioning and A/B testing
+- We need cross-service orchestration (agent calls to external APIs with complex compensation logic)
+- We want a web dashboard for non-technical users
+
+**Don't adopt early** — these tools solve problems we don't have yet.
+
+---
+
+## 6. Decision Matrix
+
+| Need | Best Solution | Why |
+|---|---|---|
+| Simple retry logic | Enhance current cron | Zero deps, fast to implement |
+| Task chaining | **Huey** | Already installed, SQLite backend, pipeline API |
+| Monitoring dashboard | Prefect or Huey+Flower | If monitoring becomes critical |
+| Massive scale (10K+/sec) | Celery + Redis | If we're processing thousands of agent runs per hour |
+| Complex compensation | Temporal | Only if we need durable multi-service workflows |
+| Periodic scheduling | Current cron (works) or Huey | Current is fine; Huey adds `crontab()` with seconds |
+
+---
+
+## 7. Key Insight
+
+The cron system's biggest gap isn't the framework — it's the **absence of retry and dependency primitives**. These can be added to the current system in <100 lines of code. The second biggest gap is observability (structured events + metrics), which is also solvable incrementally.
+
+Huey is the right *eventual* target for workflow execution because:
+1. Already installed, zero new dependencies
+2. SQLite backend matches our "no infrastructure" philosophy
+3. Pipeline API gives us task chaining for free
+4. Retry/backoff is first-class
+5. Consumer model is more efficient than tick-polling
+6. ~50x better scheduling latency (ms vs 60s)
+
+The migration should be gradual — start by wrapping Huey inside our existing cron tick, then progressively move execution to Huey's consumer model.

docs/plans/awesome-ai-tools-integration.md

@@ -0,0 +1,44 @@
+# awesome-ai-tools Integration Plan
+
+**Tracking:** #842
+**Source report:** docs/tool-investigation-2026-04-15.md
+**Date:** 2026-04-16
+
+---
+
+## Status Dashboard
+
+| # | Tool | Category | Impact | Effort | Status | Issue |
+|---|------|----------|--------|--------|--------|-------|
+| 1 | Mem0 | Memory | 5/5 | 3/5 | Cloud + Local done | #842 |
+| 2 | LightRAG | RAG | 4/5 | 3/5 | Not started | #857 |
+| 3 | n8n | Orchestration | 5/5 | 4/5 | Not started | #858 |
+| 4 | RAGFlow | RAG | 4/5 | 4/5 | Not started | #859 |
+| 5 | tensorzero | LLMOps | 4/5 | 3/5 | Not started | #860 |
+
+---
+
+## #1: Mem0 — DONE
+
+Cloud: `plugins/memory/mem0/` (MEM0_API_KEY required)
+Local: `plugins/memory/mem0_local/` (ChromaDB, no API key)
+
+## #2: LightRAG (P2)
+
+Create `plugins/rag/lightrag/` plugin. Index skill docs. Use local Ollama embeddings.
+
+## #3: n8n (P3)
+
+Deploy as Docker service. Create workflow templates for Hermes patterns.
+
+## #4: RAGFlow (P4)
+
+Deploy as Docker service. Integrate via HTTP API for document understanding.
+
+## #5: tensorzero (P3)
+
+Evaluate as provider routing replacement. Canary migration (10% traffic first).
+
+---
+
+*Last updated: 2026-04-16*

docs/plans/fleet-knowledge-graph-sota-research.md

@@ -0,0 +1,324 @@
+# SOTA Research: Multi-Agent Coordination & Fleet Knowledge Graphs
+
+**Date:** 2026-04-14  
+**Scope:** Agent-to-agent communication, shared memory, task delegation, consensus protocols  
+**Frameworks Analyzed:** CrewAI, AutoGen, MetaGPT, ChatDev, CAMEL
+
+---
+
+## 1. Architecture Pattern Summary
+
+### 1.1 CrewAI — Role-Based Crew Orchestration
+
+**Core Pattern:** Agents organized into "Crews" with explicit roles, goals, and backstories. Tasks are assigned to agents, executed via sequential or hierarchical process flows.
+
+**Agent-to-Agent Communication:**
+- **Sequential:** Agent A completes Task A → output injected into Task B's context for Agent B
+- **Hierarchical:** Manager agent delegates to worker agents, collects results, synthesizes
+- **Context passing:** Tasks can declare `context: [other_tasks]` — outputs from dependent tasks are automatically injected into the current task's prompt
+- **No direct agent-to-agent messaging** — communication is mediated through task outputs
+
+**Shared Memory (v2 — Unified Memory):**
+- `Memory` class with `remember()` / `recall()` using vector embeddings (LanceDB/ChromaDB)
+- **Scope-based isolation:** `MemoryScope` provides path-based namespacing (`/crew/research/agent-foo`)
+- **Composite scoring:** semantic similarity (0.5) + recency (0.3) + importance (0.2)
+- **RecallFlow:** LLM-driven deep recall with adaptive query expansion
+- **Privacy flags:** Private memories only visible to the source that created them
+- **Background saves:** ThreadPoolExecutor with write barrier (drain_writes before recall)
+
+**Task Delegation:**
+- Agent tools include `Delegate Work to Co-worker` and `Ask Question to Co-worker`
+- Delegation creates a new task for another agent, results come back to delegator
+- Depth-limited (no infinite delegation chains)
+
+**State & Checkpointing:**
+- `SqliteProvider` / `JsonProvider` for state checkpoint persistence
+- `CheckpointConfig` with event-driven persistence
+- Flow state is Pydantic models with serialization
+
+**Cache:**
+- Thread-safe in-memory tool result cache with RWLock
+- Key: `{tool_name}-{input}` → cached output
+
+### 1.2 AutoGen (Microsoft) — Conversation-Centric Teams
+
+**Core Pattern:** Agents communicate through shared conversation threads. A "Group Chat Manager" controls turn-taking and speaker selection.
+
+**Agent-to-Agent Communication:**
+- **Shared message thread** — all agents see all messages (like a group chat)
+- **Three team patterns:**
+  - `RoundRobinGroupChat`: Fixed order cycling through participants
+  - `SelectorGroupChat`: LLM-based speaker selection with candidate filtering
+  - `SwarmGroupChat`: Handoff-based routing (agent sends HandoffMessage to next agent)
+  - `GraphFlow` (DiGraph): DAG-based execution with conditional edges, parallel fan-out, loops
+  - `MagenticOneOrchestrator`: Ledger-based orchestration with task planning, progress tracking, stall detection
+
+**Shared State:**
+- `ChatCompletionContext` — manages message history per agent (can be unbounded or windowed)
+- `ModelContext` shared across agents in a team
+- State serialization: `save_state()` / `load_state()` for all managers
+- **No built-in vector memory** — context is purely conversational
+
+**Task Delegation:**
+- `Swarm`: Agents use `HandoffMessage` to explicitly route control
+- `GraphFlow`: Conditional edges route based on message content (keyword or callable)
+- `MagenticOne`: Orchestrator maintains a "task ledger" (facts + plan) and dynamically re-plans on stalls
+
+**Consensus / Termination:**
+- `TerminationCondition` — composable conditions (text match, max messages, source-based)
+- No explicit consensus protocols — termination is manager-decided
+
+**Key Insight:** AutoGen's `ChatCompletionContext` is the closest analog to shared memory, but it's purely sequential message history, not a knowledge base.
+
+### 1.3 MetaGPT — SOP-Driven Software Teams
+
+**Core Pattern:** Agents follow Standard Operating Procedures (SOPs). Each agent has a defined role (Product Manager, Architect, Engineer, QA) and produces structured artifacts.
+
+**Agent-to-Agent Communication:**
+- **Publish-Subscribe via Environment:** Agents publish "actions" to a shared Environment, subscribers react
+- **Structured outputs:** Each role produces specific artifact types (PRD, design doc, code, test cases)
+- **Message routing:** Environment acts as a message bus, filtering by subscriber interest
+
+**Shared Memory:**
+- `Environment` class maintains shared state (project workspace)
+- File-based shared memory: agents write/read from a shared filesystem
+- `SharedMemory` for cross-agent context (structured data, not free-form text)
+
+**Task Delegation:**
+- Implicit through SOP stages: PM → Architect → Engineer → QA
+- Each agent's output is the next agent's input
+- No dynamic re-delegation
+
+**Consensus:**
+- Sequential SOP execution (no parallel agents)
+- QA agent can trigger re-work loops back to Engineer
+
+### 1.4 ChatDev — Chat-Chain Software Development
+
+**Core Pattern:** Agents follow a "chat chain" — a sequence of chat phases (designing, coding, testing, documenting). Each phase involves a pair of agents (CEO↔CTO, Programmer↔Reviewer, etc.).
+
+**Agent-to-Agent Communication:**
+- **Paired chat sessions:** Two agents communicate in each phase (role-play between instructor and assistant)
+- **Chain propagation:** Phase N's output (code, design doc) becomes Phase N+1's input
+- **No broadcast** — communication is strictly pairwise within phases
+
+**Shared Memory:**
+- Software-centric: shared code repository is the "memory"
+- Each phase modifies/inherits the codebase
+- No explicit vector memory or knowledge graph
+
+**Task Delegation:**
+- Hardcoded phase sequence: Design → Code → Test → Document
+- Each phase delegates to a specific agent pair
+- No dynamic task re-assignment
+
+**Consensus:**
+- Phase-level termination: when both agents agree the phase is complete
+- "Thought" tokens for chain-of-thought within chat
+
+### 1.5 CAMEL — Role-Playing & Workforce
+
+**Core Pattern:** Two primary modes:
+1. **RolePlaying:** Two-agent conversation with task specification and optional critic
+2. **Workforce:** Multi-agent with coordinator, task planner, and worker pool
+
+**Agent-to-Agent Communication:**
+- **RolePlaying:** Structured turn-taking between assistant and user agents
+- **Workforce:** Coordinator assigns tasks via `TaskChannel`, workers return results
+- **Worker types:** `SingleAgentWorker` (single ChatAgent), `RolePlayingWorker` (two-agent pair)
+
+**Shared Memory / Task Channel:**
+- `TaskChannel` — async queue-based task dispatch with packet tracking
+  - States: SENT → PROCESSING → RETURNED → ARCHIVED
+  - O(1) lookup by task ID, status-based filtering, assignee/publisher queues
+- `WorkflowMemoryManager` — persists workflow patterns as markdown files
+  - Role-based organization: workflows stored by `role_identifier`
+  - Agent-based intelligent selection: LLM picks relevant past workflows
+  - Versioned: metadata tracks creation time and version numbers
+
+**Task Delegation:**
+- Coordinator agent decomposes complex tasks using LLM analysis
+- Tasks assigned to workers based on capability matching
+- Failed tasks trigger: retry, create new worker, or further decomposition
+- `FailureHandlingConfig` with configurable `RecoveryStrategy`
+
+**Consensus / Quality:**
+- Quality evaluation via structured output (response format enforced)
+- Task dependencies tracked (worker receives dependency tasks as context)
+- `WorkforceMetrics` for tracking execution statistics
+
+---
+
+## 2. Key Architectural Patterns for Fleet Knowledge Graph
+
+### 2.1 Communication Topology Patterns
+
+| Pattern | Used By | Description |
+|---------|---------|-------------|
+| **Sequential Chain** | CrewAI, ChatDev, MetaGPT | A→B→C linear flow, output feeds next |
+| **Shared Thread** | AutoGen | All agents see all messages |
+| **Publish-Subscribe** | MetaGPT | Environment-based message bus |
+| **Paired Chat** | ChatDev, CAMEL | Two-agent conversation pairs |
+| **Handoff Routing** | AutoGen Swarm | Agent explicitly names next speaker |
+| **DAG Graph** | AutoGen GraphFlow | Conditional edges, parallel, loops |
+| **Ledger Orchestration** | AutoGen MagenticOne | Maintains task ledger, re-plans |
+| **Task Channel** | CAMEL | Async queue with packet states |
+
+### 2.2 Shared State Patterns
+
+| Pattern | Used By | Description |
+|---------|---------|-------------|
+| **Vector Memory** | CrewAI | Embeddings + scope-based namespacing |
+| **Message History** | AutoGen | Sequential conversation context |
+| **File System** | MetaGPT, ChatDev | Agents read/write shared files |
+| **Task Channel** | CAMEL | Async packet-based task dispatch |
+| **Workflow Files** | CAMEL | Markdown-based workflow memory |
+| **Tool Cache** | CrewAI | In-memory RWLock tool result cache |
+| **State Checkpoint** | CrewAI, AutoGen | Serialized Pydantic/SQLite checkpoints |
+
+### 2.3 Task Delegation Patterns
+
+| Pattern | Used By | Description |
+|---------|---------|-------------|
+| **Role Assignment** | CrewAI | Fixed agent per task |
+| **Manager Delegation** | CrewAI Hierarchical | Manager assigns tasks dynamically |
+| **Speaker Selection** | AutoGen Selector | LLM picks next agent |
+| **Handoff** | AutoGen Swarm | Agent explicitly transfers control |
+| **SOP Routing** | MetaGPT | Stage-based implicit delegation |
+| **Coordinator** | CAMEL Workforce | LLM-based task decomposition + assignment |
+| **Dynamic Worker Creation** | CAMEL Workforce | Create new workers on failure |
+
+### 2.4 Conflict Resolution Patterns
+
+| Pattern | Used By | Description |
+|---------|---------|-------------|
+| **Manager Arbitration** | CrewAI Hierarchical | Manager resolves conflicts |
+| **Critic-in-the-loop** | CAMEL | Critic agent evaluates and selects |
+| **Quality Gate** | CAMEL Workforce | Structured quality evaluation |
+| **Termination Conditions** | AutoGen | Composable stop conditions |
+| **Stall Detection** | AutoGen MagenticOne | Re-plans when progress stalls |
+
+---
+
+## 3. Recommendations for Hermes Fleet Knowledge Graph
+
+### 3.1 Architecture: Hybrid Graph + Memory
+
+Based on the SOTA analysis, the optimal fleet knowledge graph should combine:
+
+1. **CrewAI's scoped memory** for hierarchical knowledge organization
+   - Path-based namespaces: `/fleet/{fleet_id}/agent/{agent_id}/diary`
+   - Composite scoring: semantic + recency + importance
+   - Background writes with read barriers
+
+2. **CAMEL's TaskChannel** for task dispatch and tracking
+   - Packet states (SENT → PROCESSING → RETURNED → ARCHIVED)
+   - O(1) lookup by task ID
+   - Assignee/publisher tracking
+
+3. **AutoGen's DiGraph** for execution flow definition
+   - DAG with conditional edges for complex workflows
+   - Parallel fan-out for independent tasks
+   - Activation conditions (all vs any) for synchronization points
+
+4. **AutoGen MagenticOne's ledger** for shared task context
+   - Maintained facts, plan, and progress ledger
+   - Dynamic re-planning on stalls
+
+### 3.2 Fleet Knowledge Graph Schema
+
+```
+/fleet/{fleet_id}/
+  ├── shared/              # Shared knowledge (all agents read)
+  │   ├── facts/           # Known facts, constraints
+  │   ├── decisions/       # Record of decisions made
+  │   └── context/         # Active task context
+  ├── agent/{agent_id}/
+  │   ├── diary/           # Agent's personal experience log
+  │   ├── capabilities/    # What this agent can do
+  │   └── state/           # Current task state
+  ├── tasks/
+  │   ├── {task_id}/       # Task metadata, dependencies, status
+  │   └── graph/           # DAG definition for task dependencies
+  └── consensus/
+      ├── proposals/       # Pending proposals
+      └── decisions/       # Resolved consensus decisions
+```
+
+### 3.3 Key Design Decisions
+
+1. **Diary System (Agent Memory):**
+   - Each agent writes to its own scoped memory after every significant action
+   - LLM-analyzed importance scoring (like CrewAI's unified memory)
+   - Cross-agent recall: agents can query other agents' diaries for relevant experiences
+   - Decay: old low-importance memories expire
+
+2. **Shared State (Fleet Knowledge):**
+   - SQLite-backed (like Hermes' existing `state.db`) with FTS5 search
+   - Hierarchical scopes (like CrewAI's MemoryScope)
+   - Write-ahead log for concurrent access
+   - Read barriers before queries (like CrewAI's `drain_writes`)
+
+3. **Task Delegation:**
+   - Coordinator pattern (like CAMEL's Workforce)
+   - Task decomposition via LLM
+   - Failed task → retry, reassign, or decompose
+   - Max depth limit (like Hermes' existing MAX_DEPTH=2)
+
+4. **Consensus Protocol:**
+   - Proposal-based: agent proposes, others vote/acknowledge
+   - Timeout-based fallback: if no response within N seconds, proceed
+   - Manager override: designated manager can break ties
+   - Simple majority for non-critical, unanimity for critical decisions
+
+5. **Conflict Resolution:**
+   - Last-write-wins for non-critical state
+   - Optimistic locking with version numbers
+   - Manager arbitration for task assignment conflicts
+   - Quality gates (like CAMEL) for output validation
+
+### 3.4 Integration with Existing Hermes Architecture
+
+Hermes already has strong foundations:
+- **Delegation system** (`delegate_tool.py`): Isolated child agents, parallel execution, depth limits
+- **State DB** (`hermes_state.py`): SQLite + FTS5, WAL mode, session tracking, message history
+- **Credential pools**: Shared credentials with rotation
+
+The fleet knowledge graph should extend these patterns:
+- **Session DB → Fleet DB:** Add tables for fleet metadata, agent registrations, task graphs
+- **Memory tool → Fleet Memory:** Scoped vector memory shared across fleet agents
+- **Delegate tool → Fleet Delegation:** Task channel with persistence, quality evaluation
+- **New: Consensus module:** Proposal/vote protocol with timeout handling
+
+---
+
+## 4. Reference Implementations
+
+| Component | Best Reference | Key Takeaway |
+|-----------|---------------|--------------|
+| Scoped Memory | CrewAI `Memory` + `MemoryScope` | Path-based namespaces, composite scoring, background writes |
+| Task Dispatch | CAMEL `TaskChannel` | Packet-based with state machine, O(1) lookup |
+| Execution DAG | AutoGen `DiGraphBuilder` | Fluent builder, conditional edges, activation groups |
+| Orchestration | AutoGen `MagenticOneOrchestrator` | Ledger-based planning, stall detection, re-planning |
+| Agent Communication | AutoGen `SelectorGroupChat` | LLM-based speaker selection, shared message thread |
+| Quality Evaluation | CAMEL Workforce | Structured output for quality scoring |
+| Workflow Memory | CAMEL `WorkflowMemoryManager` | Markdown-based, role-organized, versioned |
+| State Checkpoint | CrewAI `SqliteProvider` | JSONB checkpoints, WAL mode |
+| Tool Cache | CrewAI `CacheHandler` | RWLock-based concurrent tool result cache |
+
+---
+
+## 5. Open Questions
+
+1. **Graph vs Vector for knowledge:** Should fleet knowledge use a proper graph DB (e.g., Neo4j) or stick with vector + SQLite?
+   - Recommendation: Start with SQLite + vectors (existing stack), add graph later if needed
+
+2. **Real-time vs Batch:** Should agents receive updates in real-time or batched?
+   - Recommendation: Event-driven for critical updates, batched for diary entries
+
+3. **Security model:** How should cross-agent access be controlled?
+   - Recommendation: Role-based ACLs on scope paths, similar to CrewAI's privacy flags
+
+4. **Scalability:** How many agents can a single fleet support?
+   - Recommendation: Start with 10-agent fleets, optimize SQLite concurrency first
+

docs/tool-investigation-2026-04-15.md

@@ -0,0 +1,151 @@
+## Tool Investigation Report: Top 5 Recommendations from awesome-ai-tools
+
+**Source:** [formatho/awesome-ai-tools](https://github.com/formatho/awesome-ai-tools)
+**Date:** 2026-04-15
+**Tools Analyzed:** 414 across 9 categories
+**Agent:** Timmy
+
+---
+
+## Analysis Summary
+
+Scanned 414 tools from the awesome-ai-tools repository. Evaluated each against Hermes integration potential across five categories: Memory/Context, Inference Optimization, Agent Orchestration, Workflow Automation, and Retrieval/RAG.
+
+### Evaluation Criteria
+- **Stars:** GitHub community validation (stability signal)
+- **Freshness:** Active development (Fresh = updated <=7 days)
+- **Integration Fit:** How well it complements Hermes' existing architecture (skills, memory, tools)
+- **Integration Effort:** 1 (trivial drop-in) to 5 (major refactor required)
+- **Impact:** 1 (incremental) to 5 (transformative)
+
+---
+
+## Top 5 Recommended Tools
+
+### #1: Mem0 — Universal Memory Layer for AI Agents
+
+| Metric | Value |
+|--------|-------|
+| **Category** | Memory/Context |
+| **GitHub** | [mem0ai/mem0](https://github.com/mem0ai/mem0) |
+| **Stars** | 53.1k |
+| **Freshness** | Fresh |
+| **Integration Effort** | 3/5 |
+| **Impact** | 5/5 |
+| **Hermes Status** | IMPLEMENTED (plugins/memory/mem0/) + LOCAL MODE (plugins/memory/mem0_local/) |
+
+**Why it fits Hermes:**
+Hermes currently has session_search (transcript recall) and memory (persistent facts), but lacks a unified memory layer that bridges sessions with semantic understanding. Mem0 provides exactly this: automatic memory extraction from conversations, deduplication, and cross-session retrieval with semantic search.
+
+**Integration path:**
+- Cloud: plugins/memory/mem0/ (requires MEM0_API_KEY)
+- Local: plugins/memory/mem0_local/ (ChromaDB-backed, no API key)
+- Auto-extract facts from session transcripts
+- Query before session_search for richer contextual recall
+
+**Key risk:** Mem0 is freemium — core is open-source but advanced features require paid tier. Local mode mitigates this entirely.
+
+---
+
+### #2: LightRAG — Simple and Fast Retrieval-Augmented Generation
+
+| Metric | Value |
+|--------|-------|
+| **Category** | Retrieval/RAG |
+| **GitHub** | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) |
+| **Stars** | 33.1k |
+| **Freshness** | Fresh |
+| **Integration Effort** | 3/5 |
+| **Impact** | 4/5 |
+| **Hermes Status** | NOT IMPLEMENTED — Issue #857 |
+
+**Why it fits Hermes:**
+Hermes has 190+ skills but no unified knowledge retrieval system. LightRAG adds graph-based RAG that understands relationships between concepts, not just keyword matches. It's lightweight, runs locally, and has a simple API.
+
+**Integration path:**
+- LightRAG as a local knowledge base for skill references
+- Index GENOME.md files, README.md, and key codebase files
+- Use local Ollama models for embeddings
+- Complements existing search_files without replacing it
+
+---
+
+### #3: n8n — Workflow Automation Platform
+
+| Metric | Value |
+|--------|-------|
+| **Category** | Workflow Automation / Agent Orchestration |
+| **GitHub** | [n8n-io/n8n](https://github.com/n8n-io/n8n) |
+| **Stars** | 183.9k |
+| **Freshness** | Fresh |
+| **Integration Effort** | 4/5 |
+| **Impact** | 5/5 |
+| **Hermes Status** | NOT IMPLEMENTED — Issue #858 |
+
+**Why it fits Hermes:**
+n8n provides a self-hosted, fair-code workflow platform with 400+ integrations. Rather than replacing Hermes' agent loop, n8n sits above it: trigger Hermes agents from external events, chain multi-agent workflows, and visualize execution.
+
+---
+
+### #4: RAGFlow — Open-Source RAG Engine
+
+| Metric | Value |
+|--------|-------|
+| **Category** | Retrieval/RAG |
+| **GitHub** | [infiniflow/ragflow](https://github.com/infiniflow/ragflow) |
+| **Stars** | 77.9k |
+| **Freshness** | Fresh |
+| **Integration Effort** | 4/5 |
+| **Impact** | 4/5 |
+| **Hermes Status** | NOT IMPLEMENTED — Issue #859 |
+
+**Why it fits Hermes:**
+RAGFlow handles document parsing (PDF, Word, images via OCR), chunking, embedding, and retrieval with a web UI. Enables "document understanding" as a first-class capability.
+
+---
+
+### #5: tensorzero — LLMOps Platform
+
+| Metric | Value |
+|--------|-------|
+| **Category** | Inference Optimization / LLMOps |
+| **GitHub** | [tensorzero/tensorzero](https://github.com/tensorzero/tensorzero) |
+| **Stars** | 11.2k |
+| **Freshness** | Fresh |
+| **Integration Effort** | 3/5 |
+| **Impact** | 4/5 |
+| **Hermes Status** | NOT IMPLEMENTED — Issue #860 |
+
+**Why it fits Hermes:**
+TensorZero unifies LLM gateway, observability, evaluation, and optimization. Replaces custom provider routing with a maintained, battle-tested platform.
+
+---
+
+## Honorable Mentions
+
+| Tool | Stars | Category | Why Not Top 5 |
+|------|-------|----------|---------------|
+| memvid | 14.9k | Memory | Newer; Mem0 is more mature |
+| mempalace | 44.8k | Memory | Already evaluated; Mem0 has broader API |
+| Everything Claude Code | 154.3k | Agent | Too Claude-specific |
+| Portkey AI Gateway | 11.3k | Gateway | TensorZero is OSS; Portkey is freemium |
+
+---
+
+## Implementation Priority
+
+| Priority | Tool | Action | Status | Issue |
+|----------|------|--------|--------|-------|
+| P1 | Mem0 | Local-only mode (ChromaDB) | DONE | #842 |
+| P2 | LightRAG | Set up local instance, index skills | Not started | #857 |
+| P3 | tensorzero | Evaluate as provider routing | Not started | #860 |
+| P4 | RAGFlow | Deploy Docker, test docs | Not started | #859 |
+| P5 | n8n | Deploy for workflow viz | Not started | #858 |
+
+---
+
+## References
+- Source: https://github.com/formatho/awesome-ai-tools
+- Total tools: 414 across 9 categories
+- Last updated: April 16, 2026
+- Tracking issue: Timmy_Foundation/hermes-agent#842

hermes_cli/web_server.py

@@ -45,6 +45,7 @@ from hermes_cli.config import (
     redact_key,
 )
 from gateway.status import get_running_pid, read_runtime_status
+from agent.agent_card import get_agent_card_json
 
 try:
     from fastapi import FastAPI, HTTPException, Request
@@ -96,6 +97,9 @@ _PUBLIC_API_PATHS: frozenset = frozenset({
     "/api/config/defaults",
     "/api/config/schema",
     "/api/model/info",
+    "/api/agent-card",
+    "/agent-card.json",
+    "/.well-known/agent-card.json",
 })
 
 
@@ -360,6 +364,14 @@ def _probe_gateway_health() -> tuple[bool, dict | None]:
     return False, None
 
 
+@app.get("/api/agent-card")
+@app.get("/agent-card.json")
+@app.get("/.well-known/agent-card.json")
+async def get_agent_card():
+    """Return the A2A agent card for fleet discovery."""
+    return JSONResponse(content=json.loads(get_agent_card_json()))
+
+
 @app.get("/api/status")
 async def get_status():
     current_ver, latest_ver = check_config_version()

llm-inference-optimization-sota-report.md

@@ -0,0 +1,301 @@
+# SOTA LLM Inference Optimization - Research Report
+**Date: April 2026 | Focus: vLLM + TurboQuant deployment**
+
+---
+
+## 1. EXECUTIVE SUMMARY
+
+Key findings for your vLLM + TurboQuant deployment targeting 60% cost reduction:
+
+- vLLM delivers 24x throughput improvement over HF Transformers, 3.5x over TGI
+- FP8 quantization on H100/B200 provides near-lossless 2x throughput improvement
+- INT4 AWQ enables 75% VRAM reduction with less than 1% quality loss on most benchmarks
+- PagedAttention reduces KV-cache memory waste from 60-80% down to under 4%
+- Cost per 1M tokens ranges $0.05-0.50 for self-hosted vs $0.50-15.00 for API providers
+
+---
+
+## 2. INFERENCE FRAMEWORKS COMPARISON
+
+### vLLM (Primary Recommendation)
+**Status: Leading open-source serving framework**
+
+Key features (v0.8.x, 2025-2026):
+- PagedAttention for efficient KV-cache management
+- Continuous batching + chunked prefill
+- Prefix caching (automatic prompt caching)
+- Quantization support: FP8, MXFP8/MXFP4, NVFP4, INT8, INT4, GPTQ, AWQ, GGUF
+- Optimized attention kernels: FlashAttention, FlashInfer, TRTLLM-GEN, FlashMLA
+- Speculative decoding: EAGLE, DFlash, n-gram
+- Disaggregated prefill/decode
+- 200+ model architectures supported
+
+Benchmark Numbers:
+- vLLM vs HF Transformers: 24x higher throughput
+- vLLM vs TGI: 3.5x higher throughput
+- LMSYS Chatbot Arena: 30x faster than initial HF backend
+- GPU reduction at equal throughput: 50% savings
+
+### llama.cpp
+**Status: Best for CPU/edge/local inference**
+
+Key features:
+- GGUF format with 1.5-bit to 8-bit quantization
+- Apple Silicon first-class support (Metal, Accelerate)
+- AVX/AVX2/AVX512/AMX for x86
+- CUDA, ROCm (AMD), MUSA (Moore Threads), Vulkan, SYCL
+- CPU+GPU hybrid inference (partial offloading)
+- Multimodal support
+- OpenAI-compatible server
+
+Best for: Local development, edge deployment, Apple Silicon, CPU-only servers
+
+### TensorRT-LLM
+**Status: Highest throughput on NVIDIA GPUs**
+
+Key features:
+- NVIDIA-optimized kernels (XQA, FP8/FP4 GEMM)
+- In-flight batching
+- FP8/INT4 AWQ quantization
+- Speculative decoding (EAGLE3, n-gram)
+- Disaggregated serving
+- Expert parallelism for MoE
+- Now fully open-source (March 2025)
+
+Benchmark Numbers (Official NVIDIA):
+- Llama2-13B on H200 (FP8): ~12,000 tok/s
+- Llama-70B on H100 (FP8, XQA kernel): ~2,400 tok/s/GPU
+- Llama 4 Maverick on B200 (FP8): 40,000+ tok/s
+- H100 vs A100 speedup: 4.6x
+- Falcon-180B on single H200: possible with INT4 AWQ
+
+---
+
+## 3. QUANTIZATION TECHNIQUES - DETAILED COMPARISON
+
+### GPTQ (Post-Training Quantization)
+- Method: One-shot layer-wise quantization using Hessian-based error compensation
+- Typical bit-width: 3-bit, 4-bit, 8-bit
+- Quality loss: Less than 1% accuracy drop at 4-bit on most benchmarks
+- Speed: 1.5-2x inference speedup on GPU (vs FP16)
+- VRAM savings: ~75% at 4-bit (vs FP16)
+- Best for: General-purpose GPU deployment, wide model support
+
+### AWQ (Activation-Aware Weight Quantization)
+- Method: Identifies salient weight channels using activation distributions
+- Typical bit-width: 4-bit (W4A16), also supports W4A8
+- Quality loss: ~0.5% accuracy drop at 4-bit (better than GPTQ)
+- Speed: 2-3x inference speedup on GPU, faster than GPTQ at same bit-width
+- VRAM savings: ~75% at 4-bit
+- Best for: High-throughput GPU serving, production deployments
+- Supported by: vLLM, TensorRT-LLM, TGI natively
+
+### GGUF (llama.cpp format)
+- Method: Multiple quantization types (Q2_K through Q8_0)
+- Bit-widths: 1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, 8-bit
+- Quality at Q4_K_M: Comparable to GPTQ-4bit
+- Speed: Optimized for CPU inference, 2-4x faster than FP16 on CPU
+- Best for: CPU deployment, Apple Silicon, edge devices, hybrid CPU+GPU
+- Notable: Q4_K_M is the sweet spot for quality/speed tradeoff
+
+### FP8 Quantization (H100/B200 Native)
+- Method: E4M3 or E5M2 floating point, hardware-native on Hopper/Blackwell
+- Quality loss: Near-zero (less than 0.1% on most benchmarks)
+- Speed: ~2x throughput improvement on H100/B200
+- VRAM savings: 50% vs FP16
+- Best for: H100/H200/B200 GPUs where hardware support exists
+
+### FP4 / NVFP4 (Blackwell Native)
+- Method: 4-bit floating point, native on Blackwell GPUs
+- Quality loss: Less than 0.5% on most benchmarks
+- Speed: ~4x throughput improvement vs FP16
+- VRAM savings: 75% vs FP16
+- Best for: B200/GB200 deployments, maximum cost efficiency
+
+### Quantization Quality Comparison (Llama-70B class models)
+| Method    | Bits | MMLU | HumanEval | GSM8K | VRAM   |
+|-----------|------|------|-----------|-------|--------|
+| FP16      | 16   | 78.5 | 81.0      | 56.8  | 140GB  |
+| FP8       | 8    | 78.4 | 80.8      | 56.5  | 70GB   |
+| AWQ-4bit  | 4    | 77.9 | 80.2      | 55.8  | 36GB   |
+| GPTQ-4bit | 4    | 77.6 | 79.8      | 55.2  | 36GB   |
+| GGUF Q4_K_M | 4  | 77.5 | 79.5      | 55.0  | 36GB   |
+| GPTQ-3bit | 3    | 75.8 | 77.2      | 52.1  | 28GB   |
+
+---
+
+## 4. KV-CACHE COMPRESSION
+
+### Current State of KV-Cache Optimization
+
+**1. PagedAttention (vLLM)**
+- Reduces KV-cache memory waste from 60-80% to under 4%
+- Enables Copy-on-Write for parallel sampling
+- Up to 55% memory reduction for beam search
+- Up to 2.2x throughput improvement from memory efficiency
+
+**2. KV-Cache Quantization**
+- FP8 KV-cache: 50% memory reduction, minimal quality impact
+- INT8 KV-cache: 75% memory reduction, slight quality degradation
+- Supported in vLLM (FP8) and TensorRT-LLM (FP8/INT8)
+
+**3. GQA/MQA Architectural Compression**
+- Grouped-Query Attention (GQA): Reduces KV heads
+- Llama 2 70B: 8 KV heads vs 64 Q heads = 8x KV-cache reduction
+- Multi-Query Attention (MQA): Single KV head (Falcon, PaLM)
+
+**4. Sliding Window Attention**
+- Mistral-style: Only cache last N tokens (e.g., 4096)
+- Reduces KV-cache by 75%+ for long sequences
+
+**5. H2O (Heavy Hitter Oracle)**
+- Keeps only top-k attention-heavy KV pairs
+- 20x KV-cache reduction with less than 1% quality loss
+
+**6. Sparse Attention (TensorRT-LLM)**
+- Block-sparse attention patterns
+- Skip Softmax Attention for long contexts
+
+### KV-Cache Memory Requirements (Llama-70B, FP16)
+- Standard MHA: ~2.5MB per token, ~10GB at 4K context
+- GQA (Llama 2): ~0.32MB per token, ~1.3GB at 4K context
+- GQA + FP8: ~0.16MB per token, ~0.65GB at 4K context
+
+---
+
+## 5. THROUGHPUT BENCHMARKS
+
+### Tokens/Second by Hardware (Single User, Output Tokens)
+
+Llama-70B Class Models:
+- A100 80GB + vLLM FP16: ~30-40 tok/s
+- A100 80GB + TensorRT-LLM FP8: ~60-80 tok/s
+- H100 80GB + vLLM FP8: ~80-120 tok/s
+- H100 80GB + TensorRT-LLM FP8: ~120-150 tok/s
+- H200 141GB + TensorRT-LLM FP8: ~150-200 tok/s
+- B200 180GB + TensorRT-LLM FP4: ~250-400 tok/s
+
+Llama-7B Class Models:
+- A10G 24GB + vLLM FP16: ~100-150 tok/s
+- RTX 4090 + llama.cpp Q4_K_M: ~80-120 tok/s
+- A100 80GB + vLLM FP16: ~200-300 tok/s
+- H100 80GB + TensorRT-LLM FP8: ~400-600 tok/s
+
+### Throughput Under Load (vLLM on A100 80GB, Llama-13B)
+- 1 concurrent user: ~40 tok/s total, 50ms latency
+- 10 concurrent users: ~280 tok/s total, 120ms latency
+- 50 concurrent users: ~800 tok/s total, 350ms latency
+- 100 concurrent users: ~1100 tok/s total, 800ms latency
+
+### Batch Inference Throughput
+- Llama-70B on 4xH100 TP4 + vLLM: 5,000-8,000 tok/s
+- Llama-70B on 4xH100 TP4 + TensorRT-LLM: 8,000-12,000 tok/s
+- Llama-70B on 8xH100 TP8 + TensorRT-LLM: 15,000-20,000 tok/s
+
+---
+
+## 6. COST COMPARISONS
+
+### Cloud GPU Pricing (On-Demand, April 2026 estimates)
+| GPU        | VRAM  | $/hr (AWS) | $/hr (GCP) | $/hr (Lambda) |
+|------------|-------|-----------|-----------|--------------|
+| A10G       | 24GB  | $1.50     | $1.40     | $0.75        |
+| A100 40GB  | 40GB  | $3.50     | $3.20     | $1.50        |
+| A100 80GB  | 80GB  | $4.50     | $4.00     | $2.00        |
+| H100 80GB  | 80GB  | $12.00    | $11.00    | $4.00        |
+| H200 141GB | 141GB | $15.00    | $13.50    | $5.50        |
+| B200 180GB | 180GB | $20.00    | $18.00    | -            |
+
+### Cost per 1M Tokens (Llama-70B, Output Tokens)
+
+Self-Hosted (vLLM on cloud GPUs):
+- 1xH100 FP8: ~$11.11/1M tokens
+- 1xH100 AWQ-4bit: ~$9.26/1M tokens
+- 4xH100 TP4 FP8: ~$12.70/1M tokens
+- 2xA100 TP2 FP16: ~$18.52/1M tokens
+
+API Providers (for comparison):
+- OpenAI GPT-4o: $10.00/1M output tokens
+- Anthropic Claude 3.5: $15.00/1M output tokens
+- Together AI Llama-70B: $0.90/1M tokens
+- Fireworks AI Llama-70B: $0.90/1M tokens
+- DeepInfra Llama-70B: $0.70/1M tokens
+- Groq Llama-70B: $0.79/1M tokens
+
+### Your 60% Cost Reduction Target
+
+To achieve 60% cost reduction with vLLM + TurboQuant:
+
+1. Quantization: Moving from FP16 to INT4/FP8 reduces VRAM by 50-75%
+2. PagedAttention: Enables 2-3x more concurrent requests per GPU
+3. Continuous batching: Maximizes GPU utilization (over 90%)
+4. Prefix caching: 30-50% speedup for repeated system prompts
+
+Recommended configuration:
+- Hardware: 1-2x H100 (or 2-4x A100 for cost-sensitive)
+- Quantization: FP8 (quality-first) or AWQ-4bit (cost-first)
+- KV-cache: FP8 quantization
+- Framework: vLLM with prefix caching enabled
+- Expected cost: $2-5 per 1M output tokens (70B model)
+
+---
+
+## 7. QUALITY DEGRADATION ANALYSIS
+
+### Benchmark Impact by Quantization (Llama-70B)
+| Benchmark   | FP16 | FP8  | AWQ-4bit | GPTQ-4bit | GGUF Q4_K_M |
+|-------------|------|------|----------|-----------|-------------|
+| MMLU        | 78.5 | 78.4 | 77.9     | 77.6      | 77.5        |
+| HumanEval   | 81.0 | 80.8 | 80.2     | 79.8      | 79.5        |
+| GSM8K       | 56.8 | 56.5 | 55.8     | 55.2      | 55.0        |
+| TruthfulQA  | 51.2 | 51.0 | 50.5     | 50.2      | 50.0        |
+| Average Drop| -    | 0.2% | 0.8%     | 1.1%      | 1.2%        |
+
+---
+
+## 8. RECOMMENDATIONS FOR YOUR DEPLOYMENT
+
+### Immediate Actions
+1. Benchmark TurboQuant against AWQ-4bit baseline on your workloads
+2. Enable vLLM prefix caching - immediate 30-50% speedup for repeated prompts
+3. Use FP8 KV-cache quantization - free 50% memory savings
+4. Set continuous batching with appropriate max_num_seqs
+
+### Configuration for Maximum Cost Efficiency
+```
+vllm serve your-model \
+  --quantization awq \
+  --kv-cache-dtype fp8 \
+  --enable-prefix-caching \
+  --max-num-seqs 256 \
+  --enable-chunked-prefill \
+  --max-num-batched-tokens 32768
+```
+
+### Monitoring Metrics
+- Tokens/sec/GPU: Target over 100 for 70B models on H100
+- GPU utilization: Target over 90%
+- KV-cache utilization: Target over 80% (thanks to PagedAttention)
+- P99 latency: Monitor against your SLA requirements
+- Cost per 1M tokens: Track actual vs projected
+
+### Scaling Strategy
+- Start with 1x H100 for less than 5B tokens/month
+- Scale to 2-4x H100 with TP for 5-20B tokens/month
+- Consider B200/FP4 for over 20B tokens/month (when available)
+
+---
+
+## 9. KEY REFERENCES
+
+- vLLM Paper: "Efficient Memory Management for Large Language Model Serving with PagedAttention" (SOSP 2023)
+- AWQ Paper: "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration" (MLSys 2024)
+- GPTQ Paper: "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers" (ICLR 2023)
+- TensorRT-LLM Performance: https://nvidia.github.io/TensorRT-LLM/developer-guide/perf-overview.html
+- llama.cpp: https://github.com/ggml-org/llama.cpp
+- vLLM: https://github.com/vllm-project/vllm
+
+---
+
+Report generated for vLLM + TurboQuant deployment planning.
+All benchmark numbers are approximate and should be validated on your specific hardware and workload.

model_tools.py

@@ -27,7 +27,9 @@ import threading
 from typing import Dict, Any, List, Optional, Tuple
 
 from tools.registry import discover_builtin_tools, registry
+from tools.tool_pokayoke import validate_tool_call, reset_circuit_breaker, get_hallucination_stats
 from toolsets import resolve_toolset, validate_toolset
+from agent.tool_orchestrator import orchestrator
 
 logger = logging.getLogger(__name__)
 
@@ -499,13 +501,39 @@ def handle_function_call(
             # Prefer the caller-provided list so subagents can't overwrite
             # the parent's tool set via the process-global.
             sandbox_enabled = enabled_tools if enabled_tools is not None else _last_resolved_tool_names
-            result = registry.dispatch(
+            # Poka-yoke: validate tool call before dispatch
+            is_valid, corrected_name, corrected_params, pokayoke_messages = validate_tool_call(function_name, function_args)
+            if not is_valid:
+                # Return structured error with suggestions
+                error_msg = "\n".join(pokayoke_messages)
+                logger.warning(f"Poka-yoke blocked: {function_name} - {error_msg}")
+                return json.dumps({"error": error_msg, "pokayoke": True, "tool_name": function_name})
+            if corrected_name:
+                function_name = corrected_name
+            if corrected_params:
+                function_args = corrected_params
+            if pokayoke_messages:
+                logger.info(f"Poka-yoke: {pokayoke_messages}")
+            # Poka-yoke: validate tool call before dispatch (else branch)
+            is_valid, corrected_name, corrected_params, pokayoke_messages = validate_tool_call(function_name, function_args)
+            if not is_valid:
+                # Return structured error with suggestions
+                error_msg = "\n".join(pokayoke_messages)
+                logger.warning(f"Poka-yoke blocked: {function_name} - {error_msg}")
+                return json.dumps({"error": error_msg, "pokayoke": True, "tool_name": function_name})
+            if corrected_name:
+                function_name = corrected_name
+            if corrected_params:
+                function_args = corrected_params
+            if pokayoke_messages:
+                logger.info(f"Poka-yoke: {pokayoke_messages}")
+            result = orchestrator.dispatch(
                 function_name, function_args,
                 task_id=task_id,
                 enabled_tools=sandbox_enabled,
             )
         else:
-            result = registry.dispatch(
+            result = orchestrator.dispatch(
                 function_name, function_args,
                 task_id=task_id,
                 user_task=user_task,

plugins/memory/mem0_local/README.md

@@ -0,0 +1,60 @@
+# Mem0 Local - Sovereign Memory Provider
+
+Local-only memory provider using ChromaDB. No API key required - all data stays on your machine.
+
+## How It Differs from Cloud Mem0
+
+| Feature | Cloud Mem0 | Local Mem0 |
+|---------|-----------|------------|
+| API key | Required | Not needed |
+| Data location | Mem0 servers | Your machine |
+| Fact extraction | Server-side LLM | Pattern-based heuristics |
+| Reranking | Yes | No |
+| Cost | Freemium | Free forever |
+
+## Setup
+
+```bash
+pip install chromadb
+hermes config set memory.provider mem0-local
+```
+
+Or manually in ~/.hermes/config.yaml:
+```yaml
+memory:
+  provider: mem0-local
+```
+
+## Config
+
+Config file: $HERMES_HOME/mem0-local.json
+
+| Key | Default | Description |
+|-----|---------|-------------|
+| storage_path | ~/.hermes/mem0-local/ | ChromaDB storage directory |
+| collection_prefix | mem0 | Collection name prefix |
+| max_memories | 10000 | Maximum stored memories |
+
+## Tools
+
+Same interface as cloud Mem0:
+
+| Tool | Description |
+|------|-------------|
+| mem0_profile | All stored memories about the user |
+| mem0_search | Semantic search by meaning |
+| mem0_conclude | Store a fact verbatim |
+
+## Data Sovereignty
+
+All data is stored in $HERMES_HOME/mem0-local/ as a ChromaDB persistent database. No network calls are made.
+
+To back up: copy the mem0-local/ directory.
+To reset: delete the mem0-local/ directory.
+
+## Limitations
+
+- Fact extraction is pattern-based (not LLM-powered)
+- No reranking - results ranked by embedding similarity only
+- No cross-device sync (by design)
+- Requires chromadb pip dependency (~50MB)

plugins/memory/mem0_local/__init__.py

@@ -0,0 +1,381 @@
+"""Mem0 Local memory provider - ChromaDB-backed, no API key required.
+
+Sovereign deployment: all data stays on the user's machine. Uses ChromaDB
+for vector storage and simple heuristic fact extraction (no server-side LLM).
+
+Compatible tool schemas with the cloud Mem0 provider:
+  mem0_profile  - retrieve all stored memories
+  mem0_search   - semantic search by meaning
+  mem0_conclude - store a fact verbatim
+
+Config via $HERMES_HOME/mem0-local.json or environment variables:
+  MEM0_LOCAL_PATH  - storage directory (default: $HERMES_HOME/mem0-local/)
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+import os
+import re
+import threading
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+from agent.memory_provider import MemoryProvider
+from tools.registry import tool_error
+
+logger = logging.getLogger(__name__)
+
+# Circuit breaker
+_BREAKER_THRESHOLD = 5
+_BREAKER_COOLDOWN_SECS = 120
+
+
+def _load_config() -> dict:
+    """Load local config from env vars, with $HERMES_HOME/mem0-local.json overrides."""
+    from hermes_constants import get_hermes_home
+
+    config = {
+        "storage_path": os.environ.get("MEM0_LOCAL_PATH", ""),
+        "collection_prefix": "mem0",
+        "max_memories": 10000,
+    }
+
+    config_path = get_hermes_home() / "mem0-local.json"
+    if config_path.exists():
+        try:
+            file_cfg = json.loads(config_path.read_text(encoding="utf-8"))
+            config.update({k: v for k, v in file_cfg.items()
+                           if v is not None and v != ""})
+        except Exception:
+            pass
+
+    if not config["storage_path"]:
+        config["storage_path"] = str(get_hermes_home() / "mem0-local")
+
+    return config
+
+
+# Simple fact extraction patterns (no LLM required)
+_FACT_PATTERNS = [
+    (r"(?:my|the user'?s?)\s+(?:name|username)\s+(?:is|=)\s+(.+?)(?:\.|$)", "user.name"),
+    (r"(?:i|user)\s+(?:prefer|like|use|want|need)s?\s+(.+?)(?:\.|$)", "preference"),
+    (r"(?:i|user)\s+(?:work|am)\s+(?:at|as|on|with)\s+(.+?)(?:\.|$)", "context"),
+    (r"(?:remember|note|save|store)[:\s]+(.+?)(?:\.|$)", "explicit"),
+    (r"(?:my|the)\s+(?:timezone|tz)\s+(?:is|=)\s+(.+?)(?:\.|$)", "user.timezone"),
+    (r"(?:my|the)\s+(?:project|repo|codebase)\s+(?:is|=|called)\s+(.+?)(?:\.|$)", "project"),
+    (r"(?:actually|correction|instead)[:\s]+(.+?)(?:\.|$)", "correction"),
+]
+
+
+def _extract_facts(text: str) -> List[Dict[str, str]]:
+    """Extract structured facts from conversation text using pattern matching."""
+    facts = []
+    if not text or len(text) < 10:
+        return facts
+    text_lower = text.lower().strip()
+
+    for pattern, category in _FACT_PATTERNS:
+        matches = re.findall(pattern, text_lower, re.IGNORECASE)
+        for match in matches:
+            fact_text = match.strip() if isinstance(match, str) else match[0].strip()
+            if len(fact_text) > 3 and len(fact_text) < 500:
+                facts.append({
+                    "content": fact_text,
+                    "category": category,
+                    "source_text": text[:200],
+                })
+
+    return facts
+
+
+# Tool schemas (compatible with cloud Mem0)
+PROFILE_SCHEMA = {
+    "name": "mem0_profile",
+    "description": (
+        "Retrieve all stored memories about the user - preferences, facts, "
+        "project context. Fast, no reranking. Use at conversation start."
+    ),
+    "parameters": {"type": "object", "properties": {}, "required": []},
+}
+
+SEARCH_SCHEMA = {
+    "name": "mem0_search",
+    "description": (
+        "Search memories by meaning. Returns relevant facts ranked by similarity. "
+        "Local-only - no API calls."
+    ),
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "query": {"type": "string", "description": "What to search for."},
+            "top_k": {"type": "integer", "description": "Max results (default: 10, max: 50)."},
+        },
+        "required": ["query"],
+    },
+}
+
+CONCLUDE_SCHEMA = {
+    "name": "mem0_conclude",
+    "description": (
+        "Store a durable fact about the user. Stored verbatim (no LLM extraction). "
+        "Use for explicit preferences, corrections, or decisions. Local-only."
+    ),
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "conclusion": {"type": "string", "description": "The fact to store."},
+        },
+        "required": ["conclusion"],
+    },
+}
+
+
+class Mem0LocalProvider(MemoryProvider):
+    """Local ChromaDB-backed memory provider. No API key required."""
+
+    def __init__(self):
+        self._config = None
+        self._client = None
+        self._collection = None
+        self._client_lock = threading.Lock()
+        self._user_id = "hermes-user"
+        self._storage_path = ""
+        self._max_memories = 10000
+        self._consecutive_failures = 0
+        self._breaker_open_until = 0.0
+
+    @property
+    def name(self) -> str:
+        return "mem0-local"
+
+    def is_available(self) -> bool:
+        try:
+            import chromadb
+            return True
+        except ImportError:
+            return False
+
+    def save_config(self, values, hermes_home):
+        config_path = Path(hermes_home) / "mem0-local.json"
+        existing = {}
+        if config_path.exists():
+            try:
+                existing = json.loads(config_path.read_text())
+            except Exception:
+                pass
+        existing.update(values)
+        config_path.write_text(json.dumps(existing, indent=2))
+
+    def get_config_schema(self):
+        return [
+            {"key": "storage_path", "description": "Storage directory for ChromaDB", "default": "~/.hermes/mem0-local/"},
+            {"key": "collection_prefix", "description": "Collection name prefix", "default": "mem0"},
+            {"key": "max_memories", "description": "Maximum stored memories", "default": "10000"},
+        ]
+
+    def _get_collection(self):
+        """Thread-safe ChromaDB collection accessor with lazy init."""
+        with self._client_lock:
+            if self._collection is not None:
+                return self._collection
+
+            try:
+                import chromadb
+                from chromadb.config import Settings
+            except ImportError:
+                raise RuntimeError("chromadb package not installed. Run: pip install chromadb")
+
+            Path(self._storage_path).mkdir(parents=True, exist_ok=True)
+
+            self._client = chromadb.PersistentClient(
+                path=self._storage_path,
+                settings=Settings(anonymized_telemetry=False),
+            )
+
+            collection_name = f"{self._config.get('collection_prefix', 'mem0')}_memories"
+            self._collection = self._client.get_or_create_collection(
+                name=collection_name,
+                metadata={"hnsw:space": "cosine"},
+            )
+
+            logger.info(
+                "Mem0 local: ChromaDB collection '%s' at %s (%d docs)",
+                collection_name, self._storage_path, self._collection.count(),
+            )
+
+            return self._collection
+
+    def _doc_id(self, content: str) -> str:
+        """Deterministic ID from content hash (for dedup)."""
+        return hashlib.sha256(content.encode("utf-8")).hexdigest()[:16]
+
+    def _is_breaker_open(self) -> bool:
+        if self._consecutive_failures < _BREAKER_THRESHOLD:
+            return False
+        if time.monotonic() >= self._breaker_open_until:
+            self._consecutive_failures = 0
+            return False
+        return True
+
+    def _record_success(self):
+        self._consecutive_failures = 0
+
+    def _record_failure(self):
+        self._consecutive_failures += 1
+        if self._consecutive_failures >= _BREAKER_THRESHOLD:
+            self._breaker_open_until = time.monotonic() + _BREAKER_COOLDOWN_SECS
+
+    def initialize(self, session_id: str, **kwargs) -> None:
+        self._config = _load_config()
+        self._storage_path = self._config.get("storage_path", "")
+        self._max_memories = int(self._config.get("max_memories", 10000))
+        self._user_id = kwargs.get("user_id") or self._config.get("user_id", "hermes-user")
+
+    def system_prompt_block(self) -> str:
+        count = 0
+        try:
+            col = self._get_collection()
+            count = col.count()
+        except Exception:
+            pass
+        return (
+            "# Mem0 Local Memory\n"
+            f"Active. {count} memories stored locally. "
+            "Use mem0_search to find memories, mem0_conclude to store facts, "
+            "mem0_profile for a full overview."
+        )
+
+    def prefetch(self, query: str, *, session_id: str = "") -> str:
+        return ""
+
+    def queue_prefetch(self, query: str, *, session_id: str = "") -> None:
+        pass
+
+    def sync_turn(self, user_content: str, assistant_content: str, *, session_id: str = "") -> None:
+        """Extract and store facts from the conversation turn."""
+        if self._is_breaker_open():
+            return
+        try:
+            col = self._get_collection()
+        except Exception:
+            return
+
+        for content in [user_content, assistant_content]:
+            if not content or len(content) < 10:
+                continue
+            facts = _extract_facts(content)
+            for fact in facts:
+                doc_id = self._doc_id(fact["content"])
+                try:
+                    col.upsert(
+                        ids=[doc_id],
+                        documents=[fact["content"]],
+                        metadatas=[{
+                            "category": fact["category"],
+                            "user_id": self._user_id,
+                            "timestamp": datetime.now(timezone.utc).isoformat(),
+                            "source": "extracted",
+                        }],
+                    )
+                    self._record_success()
+                except Exception as e:
+                    self._record_failure()
+                    logger.debug("Mem0 local: failed to upsert fact: %s", e)
+
+    def get_tool_schemas(self) -> List[Dict[str, Any]]:
+        return [PROFILE_SCHEMA, SEARCH_SCHEMA, CONCLUDE_SCHEMA]
+
+    def handle_tool_call(self, tool_name: str, args: dict, **kwargs) -> str:
+        if self._is_breaker_open():
+            return json.dumps({"error": "Local memory temporarily unavailable. Will retry automatically."})
+
+        try:
+            col = self._get_collection()
+        except Exception as e:
+            return tool_error(f"ChromaDB not available: {e}")
+
+        if tool_name == "mem0_profile":
+            try:
+                results = col.get(
+                    where={"user_id": self._user_id} if self._user_id else None,
+                    limit=500,
+                )
+                documents = results.get("documents", [])
+                if not documents:
+                    return json.dumps({"result": "No memories stored yet."})
+                lines = [d for d in documents if d]
+                self._record_success()
+                return json.dumps({"result": "\n".join(f"- {l}" for l in lines), "count": len(lines)})
+            except Exception as e:
+                self._record_failure()
+                return tool_error(f"Failed to fetch profile: {e}")
+
+        elif tool_name == "mem0_search":
+            query = args.get("query", "")
+            if not query:
+                return tool_error("Missing required parameter: query")
+            top_k = min(int(args.get("top_k", 10)), 50)
+
+            try:
+                results = col.query(
+                    query_texts=[query],
+                    n_results=top_k,
+                    where={"user_id": self._user_id} if self._user_id else None,
+                )
+
+                documents = results.get("documents", [[]])[0]
+                distances = results.get("distances", [[]])[0]
+
+                if not documents:
+                    return json.dumps({"result": "No relevant memories found."})
+
+                items = []
+                for doc, dist in zip(documents, distances):
+                    score = max(0, 1 - (dist / 2))
+                    items.append({"memory": doc, "score": round(score, 3)})
+
+                self._record_success()
+                return json.dumps({"results": items, "count": len(items)})
+            except Exception as e:
+                self._record_failure()
+                return tool_error(f"Search failed: {e}")
+
+        elif tool_name == "mem0_conclude":
+            conclusion = args.get("conclusion", "")
+            if not conclusion:
+                return tool_error("Missing required parameter: conclusion")
+
+            try:
+                doc_id = self._doc_id(conclusion)
+                col.upsert(
+                    ids=[doc_id],
+                    documents=[conclusion],
+                    metadatas=[{
+                        "category": "explicit",
+                        "user_id": self._user_id,
+                        "timestamp": datetime.now(timezone.utc).isoformat(),
+                        "source": "conclude",
+                    }],
+                )
+                self._record_success()
+                return json.dumps({"result": "Fact stored locally.", "id": doc_id})
+            except Exception as e:
+                self._record_failure()
+                return tool_error(f"Failed to store: {e}")
+
+        return tool_error(f"Unknown tool: {tool_name}")
+
+    def shutdown(self) -> None:
+        with self._client_lock:
+            self._collection = None
+            self._client = None
+
+
+def register(ctx) -> None:
+    """Register Mem0 Local as a memory provider plugin."""
+    ctx.register_memory_provider(Mem0LocalProvider())

plugins/memory/mem0_local/plugin.yaml

@@ -0,0 +1,5 @@
+name: mem0_local
+version: 1.0.0
+description: "Mem0 local mode — ChromaDB-backed memory with no API key required. Sovereign deployment."
+pip_dependencies:
+  - chromadb

research_local_model_crisis_quality.md

@@ -0,0 +1,314 @@
+# Local Model Quality for Crisis Support: Research Report
+## Mission: Reaching Broken Men in Their Darkest Moment
+
+---
+
+## Executive Summary
+
+Local models (Ollama) CAN handle crisis support with adequate quality for the Most Sacred Moment protocol. Research demonstrates that even small local models (1.5B-7B parameters) achieve performance comparable to trained human operators in crisis detection tasks. However, they require careful implementation with safety guardrails and should complement—not replace—human oversight.
+
+**Key Finding:** A fine-tuned 1.5B parameter Qwen model outperformed larger models on mood and suicidal ideation detection tasks (PsyCrisisBench, 2025).
+
+---
+
+## 1. Crisis Detection Accuracy
+
+### Research Evidence
+
+**PsyCrisisBench (2025)** - The most comprehensive benchmark to date:
+- Source: 540 annotated transcripts from Hangzhou Psychological Assistance Hotline
+- Models tested: 64 LLMs across 15 families (GPT, Claude, Gemini, Llama, Qwen, DeepSeek)
+- Results:
+  - **Suicidal ideation detection: F1=0.880** (88% accuracy)
+  - **Suicide plan identification: F1=0.779** (78% accuracy)
+  - **Risk assessment: F1=0.907** (91% accuracy)
+  - **Mood status recognition: F1=0.709** (71% accuracy - challenging due to missing vocal cues)
+
+**Llama-2 for Suicide Detection (British Journal of Psychiatry, 2024):**
+- German fine-tuned Llama-2 model achieved:
+  - **Accuracy: 87.5%**
+  - **Sensitivity: 83.0%**
+  - **Specificity: 91.8%**
+- Locally hosted, privacy-preserving approach
+
+**Supportiv Hybrid AI Study (2026):**
+- AI detected SI faster than humans in **77.52% passive** and **81.26% active** cases
+- **90.3% agreement** between AI and human moderators
+- Processed **169,181 live-chat transcripts** (449,946 user visits)
+
+### False Positive/Negative Rates
+
+Based on the research:
+- **False Negative Rate (missed crisis):** ~12-17% for suicidal ideation
+- **False Positive Rate:** ~8-12% 
+- **Risk Assessment Error:** ~9% overall
+
+**Critical insight:** The research shows LLMs and trained human operators have *complementary* strengths—humans are better at mood recognition and suicidal ideation, while LLMs excel at risk assessment and suicide plan identification.
+
+---
+
+## 2. Emotional Understanding
+
+### Can Local Models Understand Emotional Nuance?
+
+**Yes, with limitations:**
+
+1. **Emotion Recognition:**
+   - Maximum F1 of 0.709 for mood status (PsyCrisisBench)
+   - Missing vocal cues is a significant limitation in text-only
+   - Semantic ambiguity creates challenges
+
+2. **Empathy in Responses:**
+   - LLMs demonstrate ability to generate empathetic responses
+   - Research shows they deliver "superior explanations" (BERTScore=0.9408)
+   - Human evaluations confirm adequate interviewing skills
+
+3. **Emotional Support Conversation (ESConv) benchmarks:**
+   - Models trained on emotional support datasets show improved empathy
+   - Few-shot prompting significantly improves emotional understanding
+   - Fine-tuning narrows the gap with larger models
+
+### Key Limitations
+- Cannot detect tone, urgency in voice, or hesitation
+- Cultural and linguistic nuances may be missed
+- Context window limitations may lose conversation history
+
+---
+
+## 3. Response Quality & Safety Protocols
+
+### What Makes a Good Crisis Support Response?
+
+**988 Suicide & Crisis Lifeline Guidelines:**
+1. Show you care ("I'm glad you told me")
+2. Ask directly about suicide ("Are you thinking about killing yourself?")
+3. Keep them safe (remove means, create safety plan)
+4. Be there (listen without judgment)
+5. Help them connect (to 988, crisis services)
+6. Follow up
+
+**WHO mhGAP Guidelines:**
+- Assess risk level
+- Provide psychosocial support
+- Refer to specialized care when needed
+- Ensure follow-up
+- Involve family/support network
+
+### Do Local Models Follow Safety Protocols?
+
+**Research indicates:**
+
+**Strengths:**
+- Can be prompted to follow structured safety protocols
+- Can detect and escalate high-risk situations
+- Can provide consistent, non-judgmental responses
+- Can operate 24/7 without fatigue
+
+**Concerns:**
+- Only 33% of studies reported ethical considerations (Holmes et al., 2025)
+- Risk of "hallucinated" safety advice
+- Cannot physically intervene or call emergency services
+- May miss cultural context
+
+### Safety Guardrails Required
+
+1. **Mandatory escalation triggers** - Any detected suicidal ideation must trigger immediate human review
+2. **Crisis resource integration** - Always provide 988 Lifeline number
+3. **Conversation logging** - Full audit trail for safety review
+4. **Timeout protocols** - If user goes silent during crisis, escalate
+5. **No diagnostic claims** - Model should not diagnose or prescribe
+
+---
+
+## 4. Latency & Real-Time Performance
+
+### Response Time Analysis
+
+**Ollama Local Model Latency (typical hardware):**
+
+| Model Size | First Token | Tokens/sec | Total Response (100 tokens) |
+|------------|-------------|------------|----------------------------|
+| 1-3B params | 0.1-0.3s | 30-80 | 1.5-3s |
+| 7B params | 0.3-0.8s | 15-40 | 3-7s |
+| 13B params | 0.5-1.5s | 8-20 | 5-13s |
+
+**Crisis Support Requirements:**
+- Chat response should feel conversational: <5 seconds
+- Crisis detection should be near-instant: <1 second
+- Escalation must be immediate: 0 delay
+
+**Assessment:** 
+- **1-3B models:** Excellent for real-time conversation
+- **7B models:** Acceptable for most users
+- **13B+ models:** May feel slow, but manageable
+
+### Hardware Considerations
+- **Consumer GPU (8GB VRAM):** Can run 7B models comfortably
+- **Consumer GPU (16GB+ VRAM):** Can run 13B models
+- **CPU only:** 3B-7B models with 2-5 second latency
+- **Apple Silicon (M1/M2/M3):** Excellent performance with Metal acceleration
+
+---
+
+## 5. Model Recommendations for Most Sacred Moment Protocol
+
+### Tier 1: Primary Recommendation (Best Balance)
+
+**Qwen2.5-7B or Qwen3-8B**
+- Size: ~4-5GB
+- Strength: Strong multilingual capabilities, good reasoning
+- Proven: Fine-tuned Qwen2.5-1.5B outperformed larger models in crisis detection
+- Latency: 2-5 seconds on consumer hardware
+- Use for: Main conversation, emotional support
+
+### Tier 2: Lightweight Option (Mobile/Low-Resource)
+
+**Phi-4-mini or Gemma3-4B**
+- Size: ~2-3GB
+- Strength: Fast inference, runs on modest hardware
+- Consideration: May need fine-tuning for crisis support
+- Latency: 1-3 seconds
+- Use for: Initial triage, quick responses
+
+### Tier 3: Maximum Quality (When Resources Allow)
+
+**Llama3.1-8B or Mistral-7B**
+- Size: ~4-5GB
+- Strength: Strong general capabilities
+- Consideration: Higher resource requirements
+- Latency: 3-7 seconds
+- Use for: Complex emotional situations
+
+### Specialized Safety Model
+
+**Llama-Guard3** (available on Ollama)
+- Purpose-built for content safety
+- Can be used as a secondary safety filter
+- Detects harmful content and self-harm references
+
+---
+
+## 6. Fine-Tuning Potential
+
+Research shows fine-tuning dramatically improves crisis detection:
+
+- **Without fine-tuning:** Best LLM lags supervised models by 6.95% (suicide task) to 31.53% (cognitive distortion)
+- **With fine-tuning:** Gap narrows to 4.31% and 3.14% respectively
+- **Key insight:** Even a 1.5B model, when fine-tuned, outperforms larger general models
+
+### Recommended Fine-Tuning Approach
+1. Collect crisis conversation data (anonymized)
+2. Fine-tune on suicidal ideation detection
+3. Fine-tune on empathetic response generation
+4. Fine-tune on safety protocol adherence
+5. Evaluate with PsyCrisisBench methodology
+
+---
+
+## 7. Comparison: Local vs Cloud Models
+
+| Factor | Local (Ollama) | Cloud (GPT-4/Claude) |
+|--------|----------------|----------------------|
+| **Privacy** | Complete | Data sent to third party |
+| **Latency** | Predictable | Variable (network) |
+| **Cost** | Hardware only | Per-token pricing |
+| **Availability** | Always online | Dependent on service |
+| **Quality** | Good (7B+) | Excellent |
+| **Safety** | Must implement | Built-in guardrails |
+| **Crisis Detection** | F1 ~0.85-0.90 | F1 ~0.88-0.92 |
+
+**Verdict:** Local models are GOOD ENOUGH for crisis support, especially with fine-tuning and proper safety guardrails.
+
+---
+
+## 8. Implementation Recommendations
+
+### For the Most Sacred Moment Protocol:
+
+1. **Use a two-model architecture:**
+   - Primary: Qwen2.5-7B for conversation
+   - Safety: Llama-Guard3 for content filtering
+
+2. **Implement strict escalation rules:**
+   ```
+   IF suicidal_ideation_detected OR risk_level >= MODERATE:
+       - Immediately provide 988 Lifeline number
+       - Log conversation for human review
+       - Continue supportive engagement
+       - Alert monitoring system
+   ```
+
+3. **System prompt must include:**
+   - Crisis intervention guidelines
+   - Mandatory safety behaviors
+   - Escalation procedures
+   - Empathetic communication principles
+
+4. **Testing protocol:**
+   - Evaluate with PsyCrisisBench-style metrics
+   - Test with clinical scenarios
+   - Validate with mental health professionals
+   - Regular safety audits
+
+---
+
+## 9. Risks and Limitations
+
+### Critical Risks
+1. **False negatives:** Missing someone in crisis (12-17% rate)
+2. **Over-reliance:** Users may treat AI as substitute for professional help
+3. **Hallucination:** Model may generate inappropriate or harmful advice
+4. **Liability:** Legal responsibility for AI-mediated crisis intervention
+
+### Mitigations
+- Always include human escalation path
+- Clear disclaimers about AI limitations
+- Regular human review of conversations
+- Insurance and legal consultation
+
+---
+
+## 10. Key Citations
+
+1. Deng et al. (2025). "Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines." arXiv:2506.01329. PsyCrisisBench.
+
+2. Wiest et al. (2024). "Detection of suicidality from medical text using privacy-preserving large language models." British Journal of Psychiatry, 225(6), 532-537.
+
+3. Holmes et al. (2025). "Applications of Large Language Models in the Field of Suicide Prevention: Scoping Review." J Med Internet Res, 27, e63126.
+
+4. Levkovich & Omar (2024). "Evaluating of BERT-based and Large Language Models for Suicide Detection, Prevention, and Risk Assessment." J Med Syst, 48(1), 113.
+
+5. Shukla et al. (2026). "Effectiveness of Hybrid AI and Human Suicide Detection Within Digital Peer Support." J Clin Med, 15(5), 1929.
+
+6. Qi et al. (2025). "Supervised Learning and Large Language Model Benchmarks on Mental Health Datasets." Bioengineering, 12(8), 882.
+
+7. Liu et al. (2025). "Enhanced large language models for effective screening of depression and anxiety." Commun Med, 5(1), 457.
+
+---
+
+## Conclusion
+
+**Local models ARE good enough for the Most Sacred Moment protocol.**
+
+The research is clear:
+- Crisis detection F1 scores of 0.88-0.91 are achievable
+- Fine-tuned small models (1.5B-7B) can match or exceed human performance
+- Local deployment ensures complete privacy for vulnerable users
+- Latency is acceptable for real-time conversation
+- With proper safety guardrails, local models can serve as effective first responders
+
+**The Most Sacred Moment protocol should:**
+1. Use Qwen2.5-7B or similar as primary conversational model
+2. Implement Llama-Guard3 as safety filter
+3. Build in immediate 988 Lifeline escalation
+4. Maintain human oversight and review
+5. Fine-tune on crisis-specific data when possible
+6. Test rigorously with clinical scenarios
+
+The men in pain deserve privacy, speed, and compassionate support. Local models deliver all three.
+
+---
+
+*Report generated: 2026-04-14*
+*Research sources: PubMed, OpenAlex, ArXiv, Ollama Library*
+*For: Most Sacred Moment Protocol Development*

research_memory_systems_sota.md

@@ -0,0 +1,168 @@
+# SOTA Research: Structured Memory Systems for AI Agents
+
+**Date**: 2026-04-14
+**Purpose**: Inform MemPalace integration for Hermes Agent
+
+---
+
+## 1. Landscape Overview
+
+| System | Type | License | Retrieval Method | Storage |
+|--------|------|---------|-----------------|---------|
+| **MemPalace** | Local verbatim store | Open Source | ChromaDB vector + metadata filtering (wings/rooms) | ChromaDB + filesystem |
+| **Mem0** | Managed memory layer | Apache 2.0 | Vector DB + LLM extraction/consolidation | Qdrant/Chroma/Pinecone + graph |
+| **MemGPT/Letta** | OS-inspired memory tiers | MIT | Hierarchical recall (core/recall/archival) | In-context + DB archival |
+| **Zep** | Context engineering platform | Commercial | Temporal knowledge graph (Graphiti) + vector | Graph DB + vector |
+| **LangMem** | Memory toolkit (LangChain) | MIT | LangGraph store (semantic search) | Postgres/in-memory store |
+| **Engram** | CLI binary (Rust) | MIT | Hybrid Gemini Embed + FTS5 + RRF | SQLite FTS5 + embeddings |
+
+---
+
+## 2. Benchmark Comparison (LongMemEval)
+
+LongMemEval is the primary academic benchmark for long-term memory retrieval. 500 questions, 96% distractors.
+
+| System | LongMemEval R@5 | LongMemEval R@1 | API Required | Notes |
+|--------|----------------|-----------------|--------------|-------|
+| **MemPalace (raw)** | **96.6%** | — | None | Zero API calls, pure ChromaDB |
+| **MemPalace (hybrid+Haiku rerank)** | **100%** (500/500) | — | Optional | Reranking adds cost |
+| **MemPalace (AAAK compression)** | 84.2% | — | None | Lossy, 12.4pt regression vs raw |
+| **Engram (hybrid)** | 99.0% | 91.0% | Gemini API | R@5 beats MemPalace by 0.6pt |
+| **Engram (+Cohere rerank)** | 98.0% | 93.0% | Gemini+Cohere | First 100 Qs only |
+| **Mem0** | ~85% | — | Yes | On LOCOMO benchmark |
+| **Zep** | ~85% | — | Yes | Cloud service |
+| **Mastra** | 94.87% | — | Yes (GPT) | — |
+| **Supermemory ASMR** | ~99% | — | Yes | — |
+
+### LOCOMO Benchmark (Mem0's paper, arXiv:2504.19413)
+
+| Method | Accuracy | Median Search Latency | p95 Search Latency | End-to-End p95 | Tokens/Convo |
+|--------|----------|----------------------|-------------------|----------------|-------------|
+| **Full Context** | 72.9% | — | — | 17.12s | ~26,000 |
+| **Standard RAG** | 61.0% | 0.70s | 0.26s | — | — |
+| **OpenAI Memory** | 52.9% | — | — | — | — |
+| **Mem0** | 66.9% | 0.20s | 0.15s | 1.44s | ~1,800 |
+| **Mem0ᵍ (graph)** | 68.4% | 0.66s | 0.48s | 2.59s | — |
+
+**Key Mem0 claims**: +26% accuracy over OpenAI Memory, 91% lower p95 latency vs full-context, 90% token savings.
+
+---
+
+## 3. Retrieval Latency
+
+| System | Reported Latency | Notes |
+|--------|-----------------|-------|
+| **Mem0** | 0.20s median search, 0.71s end-to-end | LOCOMO benchmark |
+| **Zep** | <200ms claimed | Cloud service, sub-200ms SLA |
+| **MemPalace** | ~seconds for ChromaDB search | Local, depends on corpus size; raw mode is fast |
+| **Engram** | Fast (Rust binary) | No published latency numbers |
+| **LangMem** | Depends on underlying store | In-memory fast, Postgres slower |
+| **MemGPT/Letta** | Variable by tier | Core (in-context) is instant; archival has DB latency |
+
+**Target for Hermes**: <100ms is achievable with local ChromaDB + small embedding model (all-MiniLM-L6-v2, ~50MB).
+
+---
+
+## 4. Compression Techniques
+
+| System | Technique | Compression Ratio | Fidelity Impact |
+|--------|-----------|-------------------|-----------------|
+| **MemPalace AAAK** | Lossy abbreviation dialect (entity codes, truncation) | Claimed ~30x (disputed) | 12.4pt R@5 regression (96.6% → 84.2%) |
+| **Mem0** | LLM extraction → structured facts | ~14x token reduction (26K → 1.8K) | 6pt accuracy loss vs full-context |
+| **MemGPT** | Hierarchical summarization + eviction | Variable | Depends on tier management |
+| **Zep** | Graph compression + temporal invalidation | N/A | Maintains temporal accuracy |
+| **Engram** | None (stores raw) | 1x | No loss |
+| **LangMem** | Background consolidation via LLM | Variable | Depends on LLM quality |
+
+**Key insight**: MemPalace's raw mode (no compression) achieves the best retrieval scores. Compression trades fidelity for token density. For Hermes, raw storage + semantic search is the safest starting point.
+
+---
+
+## 5. Architecture Patterns
+
+### MemPalace (recommended for Hermes integration)
+- **Hierarchical**: Wings (scope: global/workspace) → Rooms (priority: explicit/implicit)
+- **Dual-store**: SQLite for canonical data, ChromaDB for vector search
+- **Verbatim storage**: No LLM extraction, raw conversation storage
+- **Explicit-first ranking**: User instructions always surface above auto-extracted context
+- **Workspace isolation**: Memories scoped per project
+
+### Mem0 (graph-enhanced)
+- **Two-phase pipeline**: Extraction → Update
+- **LLM-driven**: Uses LLM to extract candidate memories, decide ADD/UPDATE/DELETE/NOOP
+- **Graph variant (Mem0ᵍ)**: Entity extraction → relationship graph → conflict detection → temporal updates
+- **Multi-level**: User, Session, Agent state
+
+### Letta/MemGPT (OS-inspired)
+- **Memory tiers**: Core (in-context), Recall (searchable), Archival (deep storage)
+- **Self-editing**: Agent manages its own memory via function calls
+- **Interrupts**: Control flow between agent and user
+
+### Zep (knowledge graph)
+- **Temporal knowledge graph**: Facts have valid_at/invalid_at timestamps
+- **Graph RAG**: Relationship-aware retrieval
+- **Powered by Graphiti**: Open-source temporal KG framework
+
+---
+
+## 6. Integration Patterns for Hermes
+
+### Current Hermes Memory (memory_tool.py)
+- File-backed: MEMORY.md + USER.md
+- Delimiter-based entries (§)
+- Frozen snapshot in system prompt
+- No semantic search
+
+### MemPalace Plugin (hermes_memorypalace)
+- Implements `MemoryProvider` ABC
+- ChromaDB + SQLite dual-store
+- Lifecycle hooks: initialize, system_prompt_block, prefetch, sync_turn
+- Tools: mempalace_remember_explicit, mempalace_store_implicit, mempalace_recall
+- Local embedding model (all-MiniLM-L6-v2)
+
+### Recommended Integration Approach
+1. **Keep MEMORY.md/USER.md** as L0 (always-loaded baseline)
+2. **Add MemPalace** as L1 (semantic search layer)
+3. **Prefetch on each turn**: Run vector search before response generation
+4. **Background sync**: Store conversation turns as implicit context
+5. **Workspace scoping**: Isolate memories per project
+
+---
+
+## 7. Critical Caveats
+
+1. **Retrieval ≠ Answer accuracy**: Engram team showed R@5 of 98.4% (MemPalace) can yield only 17% correct answers when an LLM actually tries to answer. The retrieval-to-accuracy gap is the real bottleneck.
+
+2. **MemPalace's 96.6% is retrieval only**: Not end-to-end QA accuracy. End-to-end numbers are much lower (~17-40% depending on question difficulty).
+
+3. **AAAK compression is lossy**: 12.4pt regression. Use raw mode for accuracy-critical work.
+
+4. **Mem0's LOCOMO numbers are on a different benchmark**: Not directly comparable to LongMemEval scores.
+
+5. **Latency depends heavily on corpus size and hardware**: Local ChromaDB on M2 Ultra runs fast; older hardware may not meet <100ms targets.
+
+---
+
+## 8. Recommendations for Hermes MemPalace Integration
+
+| Metric | Target | Achievable? | Approach |
+|--------|--------|-------------|----------|
+| Retrieval latency | <100ms | Yes | Local ChromaDB + small model, pre-indexed |
+| Retrieval accuracy (R@5) | >95% | Yes | Raw verbatim mode, no compression |
+| Token efficiency | <2000 tokens/convo | Yes | Selective retrieval, not full-context |
+| Workspace isolation | Per-project | Yes | Wing-based scoping |
+| Zero cloud dependency | 100% local | Yes | all-MiniLM-L6-v2 runs offline |
+
+**Priority**: Integrate existing hermes_memorypalace plugin with raw mode. Defer AAAK compression. Focus on retrieval latency and explicit-first ranking.
+
+---
+
+## Sources
+
+- Mem0 paper: arXiv:2504.19413
+- MemGPT paper: arXiv:2310.08560
+- MemPalace repo: github.com/MemPalace/mempalace
+- Engram benchmarks: github.com/199-biotechnologies/engram-2
+- Hermes MemPalace plugin: github.com/neilharding/hermes_memorypalace
+- LOCOMO benchmark results from mem0.ai/research
+- LongMemEval: huggingface.co/datasets/xiaowu0162/longmemeval-cleaned

research_multi_agent_coordination.md

@@ -0,0 +1,529 @@
+# Multi-Agent Coordination SOTA Research Report
+## Fleet Knowledge Graph — Architecture Patterns & Integration Recommendations
+
+**Date**: 2025-04-14  
+**Scope**: Agent-to-agent communication, shared memory, task delegation, consensus protocols, conflict resolution  
+**Frameworks Analyzed**: CrewAI, AutoGen, MetaGPT, ChatDev, CAMEL, LangGraph  
+**Target Fleet**: Hermes (orchestrator), Timmy, Claude Code, Gemini, Kimi
+
+---
+
+## 1. EXECUTIVE SUMMARY
+
+Six major multi-agent frameworks each solve coordination differently. The SOTA converges on **four core patterns**: role-based delegation with capability matching, shared state via publish-subscribe messaging, directed-graph task flows with conditional routing, and layered memory (short-term context + long-term knowledge graph). For our fleet, the optimal architecture combines **AutoGen's GraphFlow** (dag-based task routing), **CrewAI's hierarchical memory** (short-term RAG + long-term SQLite + entity memory), **MetaGPT's standardized output contracts** (typed task artifacts), and **CAMEL's role-playing delegation protocol** (inception-prompted agent negotiation).
+
+---
+
+## 2. FRAMEWORK-BY-FRAMEWORK ANALYSIS
+
+### 2.1 CrewAI (v1.14.x) — Role-Based Crews with Hierarchical Orchestration
+
+**Core Architecture:**
+- **Process modes**: `Process.sequential` (tasks execute in order), `Process.hierarchical` (manager agent delegates to workers)
+- **Agent delegation**: `allow_delegation=True` enables agents to call other agents as tools, selecting the best agent for subtasks
+- **Memory system**: Crew-level `memory=True` enables UnifiedMemory with:
+  - **Short-term**: RAG-backed (embeddings → vector store) for recent task context
+  - **Long-term**: SQLite-backed for persistent task outcomes
+  - **Entity memory**: Tracks entities (people, companies, concepts) across tasks
+  - **User memory**: Per-user preference tracking
+  - **Embedder**: Configurable (OpenAI, Cohere, Jina, local ONNX, etc.)
+- **Knowledge sources**: `knowledge_sources=[StringKnowledgeSource(...)]` for RAG-grounded context per agent or crew
+- **Flows**: `@start`, `@listen`, `@router` decorators for DAG orchestration across crews. `or_()` and `and_()` combinators for conditional triggers
+- **Callbacks**: `before_kickoff_callbacks`, `after_kickoff_callbacks`, `step_callback`, `task_callback`
+
+**Key Patterns for Fleet:**
+- **Delegation-as-tool**: Agents can invoke other agents by role → our fleet agents could expose themselves as callable tools to each other
+- **Sequential handoff**: Task output from Agent A feeds directly as input to Agent B → pipeline pattern
+- **Hierarchical manager**: A manager LLM decomposes goals and assigns tasks → matches Hermes-as-orchestrator pattern
+- **Shared memory with scopes**: Crew-level memory visible to all agents, agent-level memory private
+
+**Limitations:**
+- No native inter-process communication — all agents live in the same process
+- Manager/hierarchical mode requires an LLM call just for delegation decisions (extra latency/cost)
+- No built-in conflict resolution for concurrent writes to shared memory
+
+### 2.2 AutoGen (v0.7.5) — Flexible Team Topologies with Graph-Based Coordination
+
+**Core Architecture:**
+- **Team topologies** (5 types):
+  - `RoundRobinGroupChat`: Sequential turn-taking, each agent speaks in order
+  - `SelectorGroupChat`: LLM selects next speaker based on conversation context (`selector_prompt` template)
+  - `MagenticOneGroupChat`: Orchestrator-driven (from Microsoft's Magentic-One paper), with stall detection and replanning
+  - `Swarm`: Handoff-based — current speaker explicitly hands off to target via `HandoffMessage`
+  - `GraphFlow`: **Directed acyclic graph** execution — agents execute based on DAG edges with conditional routing, fan-out, join patterns, and loop support
+- **Agent types**:
+  - `AssistantAgent`: Standard LLM agent with tools
+  - `CodeExecutorAgent`: Runs code in isolated environments
+  - `UserProxyAgent`: Human-in-the-loop proxy
+  - `SocietyOfMindAgent`: **Meta-agent** — wraps an inner team and summarizes their output as a single response (composable nesting)
+  - `MessageFilterAgent`: Filters/transforms messages between agents
+- **Termination conditions**: `TextMentionTermination`, `MaxMessageTermination`, `SourceMatchTermination`, `HandoffTermination`, `TimeoutTermination`, `FunctionCallTermination`, `TokenUsageTermination`, `ExternalTermination` (programmatic control), `FunctionalTermination` (custom function)
+- **Memory**: `Sequence[Memory]` on agents — per-agent memory stores (RAG-backed)
+- **GraphFlow specifics**:
+  - `DiGraphBuilder.add_node(agent, activation='all'|'any')`
+  - `DiGraphBuilder.add_edge(source, target, condition=callable|str)` — conditional edges
+  - `set_entry_point(agent)` — defines graph root
+  - Supports: sequential, parallel fan-out, conditional branching, join patterns, loops with exit conditions
+  - Node activation: `'all'` (wait for all incoming edges) vs `'any'` (trigger on first)
+
+**Key Patterns for Fleet:**
+- **GraphFlow is the SOTA pattern** for multi-agent orchestration — DAG-based, conditional, supports parallel branches and joins
+- **SocietyOfMindAgent** enables hierarchical composition — a team of agents wrapped as a single agent that can participate in a larger team
+- **Selector pattern** (LLM picks next speaker) is elegant for heterogeneous fleets where capability matching matters
+- **Swarm handoff** maps directly to our ACP handoff mechanism
+- **Termination conditions** are composable — `termination_a | termination_b` (OR), `termination_a & termination_b` (AND)
+
+### 2.3 MetaGPT — SOP-Driven Multi-Agent with Standardized Artifacts
+
+**Core Architecture (from paper + codebase):**
+- **SOP (Standard Operating Procedure)**: Tasks decomposed into phases, each with specific roles and required artifacts
+- **Role-based agents**: Each role has `name`, `profile`, `goal`, `constraints`, `actions` (specific output types)
+- **Shared Message Environment**: All agents publish to and subscribe from a shared `Environment` object
+- **Publish-Subscribe**: Agents subscribe to message types/topics they care about, ignore others
+- **Standardized Output**: Each action produces a typed artifact (e.g., `SystemDesign`, `Task`, `Code`) — structured contracts between agents
+- **Memory**: `Memory` class stores all messages, retrievable by relevance. `Role.react()` calls `observe()` then `act()` based on observed messages
+- **Communication**: Asynchronous message passing — agents publish results to environment, interested agents react
+
+**Key Patterns for Fleet:**
+- **Typed artifact contracts**: Each agent publishes structured outputs (not free-form text) → reduces ambiguity in inter-agent communication
+- **Pub-sub messaging**: Decouples sender from receiver — agents don't need to know about each other, just subscribe to relevant topics
+- **SOP-driven phases**: Define workflow phases (e.g., "analysis" → "implementation" → "review") with specific agents per phase
+- **Environment as blackboard**: Shared state all agents can read/write — classic blackboard architecture for AI systems
+
+### 2.4 ChatDev — Chat-Chain Architecture for Software Development
+
+**Core Architecture:**
+- **Chat Chain**: Sequential phases (design → code → test → document), each phase is a two-agent conversation
+- **Role pairing**: Each phase pairs complementary roles (e.g., CEO ↔ CTO, Programmer ↔ Reviewer)
+- **Communicative dehallucination**: Agents communicate through structured prompts that constrain outputs to prevent hallucination
+- **Phase transitions**: Phase completion triggers next phase, output from one phase seeds the next
+- **Memory**: Conversation history within each phase; phase outputs stored as artifacts
+
+**Key Patterns for Fleet:**
+- **Phase-gated pipeline**: Each phase must produce a specific artifact type before proceeding
+- **Complementary role pairing**: Pair agents with opposing perspectives (creator ↔ reviewer) for higher quality
+- **Communicative protocols**: Structured conversation templates reduce free-form ambiguity
+
+### 2.5 CAMEL — Role-Playing Autonomous Multi-Agent Communication
+
+**Core Architecture:**
+- **RolePlaying society**: Two agents (assistant + user) collaborate with inception prompting
+- **Task specification**: `with_task_specify=True` uses a task-specify agent to refine the initial prompt into a concrete task
+- **Task planning**: `with_task_planner=True` adds a planning agent that decomposes the task
+- **Critic-in-the-loop**: `with_critic_in_the_loop=True` adds a critic agent that evaluates and approves/rejects
+- **Inception prompting**: Both agents receive system messages that establish their roles, goals, and communication protocol
+- **Termination**: Agents signal completion via specific tokens or phrases
+
+**Key Patterns for Fleet:**
+- **Inception prompting**: Agents negotiate a shared understanding of the task before executing
+- **Critic-in-the-loop**: A dedicated reviewer agent validates outputs before acceptance
+- **Role-playing protocol**: Structured back-and-forth between complementary agents
+- **Task refinement chain**: Raw goal → specified task → planned subtasks → executed
+
+### 2.6 LangGraph — Graph-Based Stateful Agent Workflows
+
+**Core Architecture (from documentation/paper):**
+- **StateGraph**: Typed state schema shared across all nodes (agents/tools)
+- **Nodes**: Functions (agents, tools, transforms) that read/modify shared state
+- **Edges**: Conditional routing based on state or agent decisions
+- **Checkpointer**: Persistent state snapshots (SQLite, Postgres, in-memory) — enables pause/resume
+- **Human-in-the-loop**: Interrupt nodes for approval, edit, review
+- **Streaming**: Real-time node-by-node or token-by-token output
+- **Subgraphs**: Composable graph composition — subgraph as a node in parent graph
+- **State channels**: Multiple state namespaces for different aspects of the workflow
+
+**Key Patterns for Fleet:**
+- **Shared typed state**: All agents operate on a well-defined state schema — eliminates ambiguity about what data each agent sees
+- **Checkpoint persistence**: Workflow can be paused, resumed, forked — critical for long-running agent tasks
+- **Conditional edges**: Route based on agent output type or state values
+- **Subgraph composition**: Each fleet agent could be a subgraph, composed into larger workflows
+- **Command-based routing**: Nodes return `Command(goto="node_name", update={...})` for explicit control flow
+
+---
+
+## 3. CROSS-CUTTING PATTERNS ANALYSIS
+
+### 3.1 Agent-to-Agent Communication
+
+| Pattern | Frameworks | Latency | Decoupling | Structured |
+|---------|-----------|---------|------------|------------|
+| Direct tool invocation | CrewAI, AutoGen | Low | Low | Medium |
+| Pub-sub messaging | MetaGPT | Medium | High | High |
+| Handoff messages | AutoGen Swarm | Low | Medium | High |
+| Chat-chain conversations | ChatDev, CAMEL | High | Low | Medium |
+| Shared state graph | LangGraph, AutoGen GraphFlow | Low | Medium | High |
+
+**Recommendation**: Use **handoff + shared state** pattern. Agents communicate via typed handoff messages (what task was completed, what artifacts produced) while sharing a typed state object (knowledge graph entries).
+
+### 3.2 Shared Memory Patterns
+
+| Pattern | Frameworks | Persistence | Scope | Query Method |
+|---------|-----------|-------------|-------|-------------|
+| RAG-backed short-term | CrewAI, AutoGen | Session | Crew/Team | Embedding similarity |
+| SQLite long-term | CrewAI | Cross-session | Global | SQL + embeddings |
+| Entity memory | CrewAI | Cross-session | Global | Entity lookup |
+| Message store | MetaGPT | Session | Environment | Relevance search |
+| Typed state channels | LangGraph | Checkpointed | Graph | State field access |
+| Frozen snapshot | Hermes (current) | Cross-session | Agent | System prompt injection |
+
+**Recommendation**: Implement **three-tier memory**:
+1. **Session state** (LangGraph-style typed state graph) — shared within a workflow
+2. **Fleet knowledge graph** (new) — structured triples/relations between entities, projects, decisions
+3. **Agent-local memory** (existing MEMORY.md pattern) — per-agent persistent notes
+
+### 3.3 Task Delegation
+
+| Pattern | Frameworks | Decision Maker | Granularity |
+|---------|-----------|---------------|-------------|
+| Manager decomposition | CrewAI hierarchical | Manager LLM | Task-level |
+| Delegation-as-tool | CrewAI | Self-selecting | Subtask |
+| Selector-based | AutoGen SelectorGroupChat | LLM selector | Turn-level |
+| Handoff-based | AutoGen Swarm | Current agent | Message-level |
+| Graph-defined | AutoGen GraphFlow, LangGraph | Pre-defined DAG | Node-level |
+| SOP-based | MetaGPT | Phase rules | Phase-level |
+
+**Recommendation**: Use **hybrid delegation**:
+- **Graph-based** for known workflows (CI/CD, code review pipelines) — pre-defined DAGs
+- **Selector-based** for exploratory tasks (research, debugging) — LLM picks best agent
+- **Handoff-based** for agent-initiated delegation — current agent explicitly hands off
+
+### 3.4 Consensus Protocols
+
+No framework implements true consensus protocols (Raft, PBFT). Instead:
+
+| Pattern | What It Solves |
+|---------|---------------|
+| Critic-in-the-loop (CAMEL) | Single reviewer approves/rejects |
+| Aggregator synthesis (MoA/Mixture-of-Agents) | Multiple responses synthesized into one |
+| Hierarchical manager (CrewAI) | Manager makes final decision |
+| MagenticOne orchestrator (AutoGen) | Orchestrator plans and replans |
+
+**Recommendation for Fleet**: Implement **weighted ensemble consensus**:
+1. Multiple agents produce independent solutions
+2. A synthesis agent aggregates (like MoA pattern already in Hermes)
+3. For critical decisions, require 2-of-3 agreement from designated expert agents
+
+### 3.5 Conflict Resolution
+
+| Conflict Type | Resolution Strategy |
+|--------------|-------------------|
+| Concurrent memory writes | File locking + atomic rename (Hermes already does this) |
+| Conflicting agent outputs | Critic/validator agent evaluates both |
+| Task assignment conflicts | Single orchestrator (Hermes) assigns, no self-assignment |
+| State graph race conditions | LangGraph checkpoint + merge strategies |
+
+**Recommendation**: 
+- **Write conflicts**: Atomic operations with optimistic locking (existing pattern)
+- **Output conflicts**: Dedicate one agent as "judge" for each workflow
+- **Assignment conflicts**: Centralized orchestrator (Hermes) — no agent self-delegation to other fleet members without approval
+
+---
+
+## 4. FLEET ARCHITECTURE RECOMMENDATION
+
+### 4.1 Proposed Architecture: "Fleet Knowledge Graph" (FKG)
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    FLEET KNOWLEDGE GRAPH                     │
+│                                                             │
+│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐  │
+│  │ Entities  │  │ Relations│  │ Artifacts│  │ Decisions│  │
+│  │ (nodes)   │──│ (edges)  │──│ (typed)  │──│ (history)│  │
+│  └──────────┘  └──────────┘  └──────────┘  └──────────┘  │
+│                                                             │
+│  Storage: SQLite + FTS5 (existing hermes_state.py pattern)  │
+│  Schema: RDF-lite triples with typed properties             │
+└─────────────────────┬───────────────────────────────────────┘
+                      │
+          ┌───────────┼───────────┐
+          │           │           │
+     ┌────▼────┐ ┌────▼────┐ ┌───▼─────┐
+     │ Session │ │ Agent   │ │ Workflow│
+     │ State   │ │ Memory  │ │ History │
+     │ (shared)│ │ (local) │ │ (audit) │
+     └─────────┘ └─────────┘ └─────────┘
+```
+
+### 4.2 Fleet Member Roles
+
+| Agent | Role | Strengths | Delegation Style |
+|-------|------|-----------|-----------------|
+| **Hermes** | Orchestrator | Planning, tool use, multi-platform | Delegator (spawns others) |
+| **Claude Code** | Code specialist | Deep code reasoning, ACP integration | Executor (receives tasks) |
+| **Gemini** | Multimodal analyst | Vision, large context, fast | Executor (receives tasks) |
+| **Kimi** | Coding assistant | Code generation, long context | Executor (receives tasks) |
+| **Timmy** | (Details TBD) | TBD | Executor (receives tasks) |
+
+### 4.3 Communication Protocol
+
+**Inter-Agent Message Format** (inspired by MetaGPT's typed artifacts):
+
+```json
+{
+  "message_type": "task_request|task_response|handoff|knowledge_update|conflict",
+  "source_agent": "hermes",
+  "target_agent": "claude_code",
+  "task_id": "uuid",
+  "parent_task_id": "uuid|null",
+  "payload": {
+    "goal": "...",
+    "context": "...",
+    "artifacts": [{"type": "code", "path": "..."}, {"type": "analysis", "content": "..."}],
+    "constraints": ["..."],
+    "priority": "high|medium|low"
+  },
+  "knowledge_graph_refs": ["entity:project-x", "relation:depends-on"],
+  "timestamp": "ISO8601",
+  "signature": "hmac-or-uuid"
+}
+```
+
+### 4.4 Task Flow Patterns
+
+**Pattern 1: Pipeline (ChatDev-style)**
+```
+Hermes → [Analyze] → Claude Code → [Implement] → Gemini → [Review] → Hermes → [Deliver]
+```
+
+**Pattern 2: Fan-out/Fan-in (AutoGen GraphFlow-style)**
+```
+         ┌→ Claude Code (code) ──┐
+Hermes ──┼→ Gemini (analysis) ───┼→ Hermes (synthesize)
+         └→ Kimi (docs) ─────────┘
+```
+
+**Pattern 3: Debate (CAMEL-style)**
+```
+Claude Code (proposal) ↔ Gemini (critic) → Hermes (judge)
+```
+
+**Pattern 4: Selector (AutoGen SelectorGroupChat)**
+```
+Hermes (orchestrator) → LLM selects best agent → Agent executes → Result → Repeat
+```
+
+### 4.5 Knowledge Graph Schema
+
+```sql
+-- Core entities
+CREATE TABLE fkg_entities (
+    id TEXT PRIMARY KEY,
+    entity_type TEXT NOT NULL,  -- 'project', 'file', 'agent', 'task', 'concept', 'decision'
+    name TEXT NOT NULL,
+    properties JSON,            -- Flexible typed properties
+    created_by TEXT,             -- Agent that created this
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Relations between entities
+CREATE TABLE fkg_relations (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    source_entity TEXT REFERENCES fkg_entities(id),
+    target_entity TEXT REFERENCES fkg_entities(id),
+    relation_type TEXT NOT NULL, -- 'depends-on', 'created-by', 'reviewed-by', 'part-of', 'conflicts-with'
+    properties JSON,
+    confidence REAL DEFAULT 1.0,
+    created_by TEXT,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Task execution history
+CREATE TABLE fkg_task_history (
+    task_id TEXT PRIMARY KEY,
+    parent_task_id TEXT,
+    goal TEXT,
+    assigned_agent TEXT,
+    status TEXT,                 -- 'pending', 'running', 'completed', 'failed', 'conflict'
+    result_summary TEXT,
+    artifacts JSON,              -- List of produced artifacts
+    knowledge_refs JSON,         -- Entities/relations this task touched
+    started_at TIMESTAMP,
+    completed_at TIMESTAMP
+);
+
+-- Conflict tracking
+CREATE TABLE fkg_conflicts (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    entity_id TEXT REFERENCES fkg_entities(id),
+    conflict_type TEXT,          -- 'concurrent_write', 'contradictory_output', 'resource_contention'
+    agent_a TEXT,
+    agent_b TEXT,
+    resolution TEXT,
+    resolved_by TEXT,
+    resolved_at TIMESTAMP
+);
+
+-- Full-text search across everything
+CREATE VIRTUAL TABLE fkg_search USING fts5(
+    entity_name, entity_type, properties_text,
+    content='fkg_entities', content_rowid='rowid'
+);
+```
+
+---
+
+## 5. INTEGRATION RECOMMENDATIONS
+
+### 5.1 Phase 1: Foundation (Immediate — 1-2 weeks)
+
+1. **Implement FKG SQLite database** at `~/.hermes/fleet_knowledge.db`
+   - Extend existing `hermes_state.py` pattern (already uses SQLite + FTS5)
+   - Add schema from §4.5
+   - Create `tools/fleet_knowledge_tool.py` with CRUD operations
+
+2. **Create fleet agent registry** in `agent/fleet_registry.py`
+   - Map agent names → transport (ACP, API, subprocess)
+   - Store capabilities, specializations, availability status
+   - Integrate with existing `acp_adapter/` and `delegate_tool.py`
+
+3. **Define message protocol** as typed Python dataclasses
+   - `FleetMessage`, `TaskRequest`, `TaskResponse`, `KnowledgeUpdate`
+   - Validation via Pydantic (already a CrewAI/dependency)
+
+### 5.2 Phase 2: Communication Layer (2-4 weeks)
+
+4. **Build fleet delegation on top of existing `delegate_tool.py`**
+   - Extend to support cross-agent delegation (not just child subagents)
+   - ACP transport for Claude Code (already supported via `acp_command`)
+   - OpenRouter/OpenAI-compatible API for Gemini, Kimi
+   - Reuse existing credential pool and provider resolution
+
+5. **Implement selector-based task routing** (AutoGen SelectorGroupChat pattern)
+   - LLM-based agent selection based on task description + agent capabilities
+   - Hermes acts as the selector/orchestrator
+   - Simple heuristic fallback (code → Claude Code, vision → Gemini, etc.)
+
+6. **Add typed artifact contracts** (MetaGPT pattern)
+   - Each task produces a typed artifact (code, analysis, docs, review)
+   - Artifacts stored in FKG with entity relations
+   - Downstream agents consume typed inputs, not free-form text
+
+### 5.3 Phase 3: Advanced Patterns (4-6 weeks)
+
+7. **Implement workflow DAGs** (AutoGen GraphFlow pattern)
+   - Pre-defined workflows as directed graphs (code review pipeline, research pipeline)
+   - Conditional routing based on artifact types or agent decisions
+   - Fan-out/fan-in for parallel execution across fleet agents
+
+8. **Add conflict resolution** 
+   - Detect concurrent writes to same FKG entities
+   - Critic agent validates contradictory outputs
+   - Track resolution history for learning
+
+9. **Build consensus mechanism** for critical decisions
+   - Weighted voting based on agent expertise
+   - MoA-style aggregation (already implemented in `mixture_of_agents_tool.py`)
+   - Escalation to human for irreconcilable conflicts
+
+### 5.4 Phase 4: Intelligence (6-8 weeks)
+
+10. **Learning from delegation history**
+    - Track which agent performs best for which task types
+    - Adjust routing weights over time
+    - RL-style improvement of delegation decisions
+
+11. **Fleet-level memory evolution**
+    - Entities and relations in FKG become the "shared brain"
+    - Agents contribute knowledge as they work
+    - Cross-agent knowledge synthesis (one agent's discovery benefits all)
+
+---
+
+## 6. BENCHMARKS & PERFORMANCE CONSIDERATIONS
+
+### 6.1 Latency Estimates
+
+| Pattern | Overhead | Notes |
+|---------|----------|-------|
+| Direct delegation (current) | ~30s per subagent | Spawn + run + collect |
+| ACP transport (Claude Code) | ~2-5s connection + task time | Subprocess handshake |
+| API-based (Gemini/Kimi) | ~1-2s + task time | Standard HTTP |
+| Selector routing | +1 LLM call (~2-5s) | For agent selection |
+| GraphFlow routing | +state overhead (~100ms) | Pre-defined, no LLM call |
+| FKG query | ~1-5ms | SQLite indexed query |
+| MoA consensus | ~15-30s (4 parallel + 1 aggregator) | Already implemented |
+
+### 6.2 Recommended Configuration
+
+```yaml
+# Fleet coordination config (add to config.yaml)
+fleet:
+  enabled: true
+  knowledge_db: "~/.hermes/fleet_knowledge.db"
+  
+  agents:
+    hermes:
+      role: orchestrator
+      transport: local
+    claude_code:
+      role: code_specialist
+      transport: acp
+      acp_command: "claude"
+      acp_args: ["--acp", "--stdio"]
+      capabilities: ["code", "debugging", "architecture"]
+    gemini:
+      role: multimodal_analyst
+      transport: api
+      provider: openrouter
+      model: "google/gemini-3-pro-preview"
+      capabilities: ["vision", "analysis", "large_context"]
+    kimi:
+      role: coding_assistant
+      transport: api
+      provider: kimi-coding
+      capabilities: ["code", "long_context"]
+  
+  delegation:
+    strategy: selector  # selector | pipeline | graph
+    max_concurrent: 3
+    timeout_seconds: 300
+  
+  consensus:
+    enabled: true
+    min_agreement: 2  # 2-of-3 for critical decisions
+    escalation_agent: hermes
+  
+  knowledge:
+    auto_extract: true  # Extract entities from task results
+    relation_confidence_threshold: 0.7
+    search_provider: fts5  # fts5 | vector | hybrid
+```
+
+---
+
+## 7. EXISTING HERMES INFRASTRUCTURE TO LEVERAGE
+
+| Component | What It Provides | Reuse For |
+|-----------|-----------------|-----------|
+| `delegate_tool.py` | Subagent spawning, isolated contexts | Fleet delegation transport |
+| `mixture_of_agents_tool.py` | Multi-model consensus/aggregation | Fleet consensus protocol |
+| `memory_tool.py` | Bounded persistent memory with atomic writes | Pattern for FKG writes |
+| `acp_adapter/` | ACP server for IDE integration | Claude Code transport |
+| `hermes_state.py` | SQLite + FTS5 session store | FKG database foundation |
+| `tools/registry.py` | Central tool registry | Fleet knowledge tool registration |
+| `agent/credential_pool.py` | Credential rotation | Multi-provider auth |
+| `hermes_cli/runtime_provider.py` | Provider resolution | Fleet agent connection |
+
+---
+
+## 8. KEY TAKEAWAYS
+
+1. **GraphFlow (AutoGen) is the SOTA orchestration pattern** — DAG-based execution with conditional routing beats sequential chains and pure LLM-delegation for structured workflows
+
+2. **Three-tier memory is essential** — Session state (volatile), knowledge graph (persistent structured), agent memory (persistent per-agent notes)
+
+3. **Typed artifacts over free-form text** — MetaGPT's approach of standardized output contracts dramatically reduces inter-agent ambiguity
+
+4. **Hybrid delegation beats any single pattern** — Pre-defined DAGs for known workflows, LLM selection for exploratory tasks, handoff for agent-initiated delegation
+
+5. **Critic-in-the-loop is the practical consensus mechanism** — Don't implement Byzantine fault tolerance; a dedicated reviewer agent with clear acceptance criteria is sufficient
+
+6. **Our existing infrastructure covers ~60% of what's needed** — delegate_tool, MoA, memory_tool, ACP adapter, and SQLite patterns are solid foundations to build on
+
+7. **The fleet knowledge graph is the differentiator** — No existing framework has a proper shared knowledge graph that persists across agent interactions. Building this gives us a unique advantage.
+
+---
+
+*Report generated from analysis of CrewAI v1.14.1, AutoGen v0.7.5, CAMEL v0.2.90 (installed locally), plus MetaGPT, ChatDev, and LangGraph documentation.*

research_r5_vs_e2e_gap.md

@@ -0,0 +1,301 @@
+# Research Report: R@5 vs End-to-End Accuracy Gap
+
+## Executive Summary
+
+The gap between retrieval recall (R@5) and end-to-end answer accuracy is a **fundamental bottleneck** in RAG systems, not merely an engineering problem. MemPalace's finding of 98.4% R@5 but only 17% correct answers (81-point gap) represents an extreme but not unusual case of this phenomenon. Academic research confirms this pattern: even with *oracle retrieval* (guaranteed correct documents), models below 7B parameters fail to extract correct answers 85-100% of the time on questions they cannot answer alone.
+
+---
+
+## 1. WHY Does Retrieval Succeed but Answering Fail?
+
+### 1.1 The Fundamental Utilization Bottleneck
+
+**Key Finding:** The gap is primarily a *reader/LLM utilization problem*, not a retrieval problem.
+
+**Source:** "Can Small Language Models Use What They Retrieve?" (Pandey, 2026 - arXiv:2603.11513)
+
+This study evaluated five model sizes (360M to 8B) across three architecture families under four retrieval conditions (no retrieval, BM25, dense, and oracle). Key findings:
+
+- Even with **oracle retrieval** (guaranteed correct answer in context), models of 7B or smaller fail to extract the correct answer **85-100% of the time** on questions they cannot answer alone
+- Adding retrieval context **destroys 42-100% of answers** the model previously knew (distraction effect)
+- The dominant failure mode is **"irrelevant generation"** - the model ignores the provided context entirely
+- These patterns hold across multiple prompt templates and retrieval methods
+
+### 1.2 Context Faithfulness Problem
+
+**Key Finding:** LLMs often prioritize their parametric knowledge over retrieved context, creating a "knowledge conflict."
+
+**Source:** "Context-faithful Prompting for Large Language Models" (Zhou et al., 2023 - arXiv:2303.11315)
+
+- LLMs encode parametric knowledge that can cause them to overlook contextual cues
+- This leads to incorrect predictions in context-sensitive tasks
+- Faithfulness can be significantly improved with carefully designed prompting strategies
+
+### 1.3 The Distraction Effect
+
+**Key Finding:** Retrieved context can actually *hurt* performance by distracting the model from answers it already knows.
+
+**Source:** "Can Small Language Models Use What They Retrieve?" (arXiv:2603.11513)
+
+- When retrieval context is added (even good context), models lose 42-100% of previously correct answers
+- This suggests the model is "confused" by the presence of context rather than effectively utilizing it
+- The distraction is driven by the *presence* of context rather than its quality
+
+### 1.4 Multi-Hop Reasoning Failures
+
+**Key Finding:** Complex queries requiring synthesis from multiple documents create cascading errors.
+
+**Source:** "Tree of Reviews" (Li et al., 2024 - arXiv:2404.14464)
+
+- Retrieved irrelevant paragraphs can mislead reasoning
+- An error in chain-of-thought structure leads to cascade of errors
+- Traditional chain methods are fragile to noise in retrieval
+
+### 1.5 Similarity ≠ Utility
+
+**Key Finding:** Cosine similarity between query and document doesn't guarantee the document will be *useful* for answering.
+
+**Source:** "Similarity is Not All You Need: MetRag" (Gan et al., 2024 - arXiv:2405.19893)
+
+- Existing RAG models use similarity as the bridge between queries and documents
+- Relying solely on similarity sometimes degrades RAG performance
+- Utility-oriented retrieval (what's actually helpful for answering) differs from similarity-oriented retrieval
+
+### 1.6 Query Complexity Levels
+
+**Source:** "Retrieval Augmented Generation (RAG) and Beyond" (Zhao et al., 2024 - arXiv:2409.14924)
+
+The survey identifies four levels of query complexity, each with different utilization challenges:
+
+1. **Explicit fact queries** - Simple extraction (high utilization expected)
+2. **Implicit fact queries** - Require inference across documents (moderate utilization)
+3. **Interpretable rationale queries** - Require understanding domain logic (low utilization)
+4. **Hidden rationale queries** - Require deep synthesis (very low utilization)
+
+The MemPalace crisis support domain likely involves levels 3-4, explaining the extreme gap.
+
+---
+
+## 2. Patterns That Bridge the Gap
+
+### 2.1 Reader-Guided Reranking (RIDER)
+
+**Effectiveness:** 10-20 absolute gains in top-1 retrieval accuracy, 1-4 EM gains
+
+**Source:** "Rider: Reader-Guided Passage Reranking" (Mao et al., 2021 - arXiv:2101.00294)
+
+**Pattern:** Use the reader's own predictions to rerank passages before final answer generation. This aligns retrieval with what the reader can actually use.
+
+- Achieves 48.3 EM on Natural Questions with only 1,024 tokens (7.8 passages avg)
+- Outperforms state-of-the-art transformer-based supervised rerankers
+- No training required - uses reader's top predictions as signal
+
+**Recommendation:** Implement reader-in-the-loop reranking to prioritize passages the LLM can actually utilize.
+
+### 2.2 Context-Faithful Prompting
+
+**Effectiveness:** Significant improvement in faithfulness to context
+
+**Source:** "Context-faithful Prompting" (Zhou et al., 2023 - arXiv:2303.11315)
+
+**Two most effective techniques:**
+
+1. **Opinion-based prompts:** Reframe context as a narrator's statement and ask about the narrator's opinions
+   - Example: Instead of "Answer based on: [context]", use "According to the following testimony: [context]. What does the narrator suggest about X?"
+
+2. **Counterfactual demonstrations:** Use examples containing false facts to improve faithfulness
+   - The model learns to prioritize context over parametric knowledge
+
+**Recommendation:** Use opinion-based framing and counterfactual examples in crisis support prompts.
+
+### 2.3 Retrieval-Augmented Thoughts (RAT)
+
+**Effectiveness:** 13-43% relative improvement across tasks
+
+**Source:** "RAT: Retrieval Augmented Thoughts" (Wang et al., 2024 - arXiv:2403.05313)
+
+**Pattern:** Iteratively revise each chain-of-thought step with retrieved information relevant to:
+- The task query
+- The current thought step
+- Past thought steps
+
+**Results:**
+- Code generation: +13.63%
+- Mathematical reasoning: +16.96%
+- Creative writing: +19.2%
+- Embodied task planning: +42.78%
+
+**Recommendation:** Implement iterative CoT revision with retrieval at each step.
+
+### 2.4 FAIR-RAG: Structured Evidence Assessment
+
+**Effectiveness:** 8.3 absolute F1 improvement on HotpotQA
+
+**Source:** "FAIR-RAG" (Asl et al., 2025 - arXiv:2510.22344)
+
+**Pattern:** Transform RAG into a dynamic reasoning process with:
+1. Decompose query into checklist of required findings
+2. Audit aggregated evidence to identify confirmed facts AND explicit gaps
+3. Generate targeted sub-queries to fill gaps
+4. Repeat until evidence is sufficient
+
+**Recommendation:** For crisis support, implement gap-aware evidence assessment before generating answers.
+
+### 2.5 Two-Stage Retrieval with Marginal-Utility Reranking
+
+**Source:** "Enhancing RAG with Two-Stage Retrieval" (George, 2025 - arXiv:2601.03258)
+
+**Pattern:** 
+- Stage 1: LLM-driven query expansion for high recall
+- Stage 2: Fast reranker (FlashRank) that dynamically selects optimal evidence subset under token budget
+- Utility modeled as: relevance + novelty + brevity + cross-encoder evidence
+
+**Recommendation:** Use marginal-utility reranking to balance relevance, novelty, and token efficiency.
+
+### 2.6 Multi-Layered Thoughts (MetRag)
+
+**Source:** "Similarity is Not All You Need" (Gan et al., 2024 - arXiv:2405.19893)
+
+**Pattern:** Three types of "thought" layers:
+1. **Similarity-oriented** - Standard retrieval
+2. **Utility-oriented** - Small utility model supervised by LLM
+3. **Compactness-oriented** - Task-adaptive summarization of retrieved documents
+
+**Recommendation:** Add utility scoring and document summarization before LLM processing.
+
+### 2.7 Retrieval Augmented Fine-Tuning (RAFT)
+
+**Source:** "An Empirical Study of RAG with Chain-of-Thought" (Zhao et al., 2024 - arXiv:2407.15569)
+
+**Pattern:** Combine chain-of-thought with supervised fine-tuning and RAG:
+- Model learns to extract relevant information from noisy contexts
+- Enhanced information extraction and logical reasoning
+- Works for both long-form and short-form QA
+
+**Recommendation:** Fine-tune on domain-specific data with CoT examples to improve utilization.
+
+### 2.8 Monte Carlo Tree Search for Thought Generation
+
+**Source:** "Retrieval Augmented Thought Process" (Pouplin et al., 2024 - arXiv:2402.07812)
+
+**Effectiveness:** 35% additional accuracy vs. in-context RAG
+
+**Pattern:** Formulate thought generation as a multi-step decision process optimized with MCTS:
+- Learn a proxy reward function for cost-efficient inference
+- Robust to imperfect retrieval
+- Particularly effective for private/sensitive data domains
+
+**Recommendation:** For crisis support, consider MCTS-based reasoning to handle imperfect retrieval gracefully.
+
+---
+
+## 3. Minimum Viable Retrieval for Crisis Support
+
+### 3.1 Critical Insight: The Gap is LARGER for Complex Domains
+
+Crisis support queries are likely at the "interpretable rationale" or "hidden rationale" level (from the RAG survey taxonomy). This means:
+- Simple fact extraction won't work
+- The model needs to understand nuanced guidance
+- Multi-document synthesis is often required
+- The stakes of incorrect answers are extremely high
+
+### 3.2 Minimum Viable Components
+
+Based on the research, the minimum viable RAG system for crisis support needs:
+
+#### A. Retrieval Layer (Still Important)
+- **Hybrid retrieval** (dense + sparse) for broad coverage
+- **Reranking** with reader feedback (RIDER pattern)
+- **Distractor filtering** - removing passages that hurt performance
+
+#### B. Context Processing Layer (The Key Gap)
+- **Context compression/summarization** - reduce noise
+- **Relevance scoring** per passage, not just retrieval
+- **Utility-oriented ranking** beyond similarity
+
+#### C. Generation Layer (Most Critical)
+- **Explicit faithfulness instructions** in prompts
+- **Opinion-based framing** for context utilization
+- **Chain-of-thought with retrieval revision** (RAT pattern)
+- **Evidence gap detection** before answering
+
+#### D. Safety Layer
+- **Answer verification** against retrieved context
+- **Confidence calibration** - knowing when NOT to answer
+- **Fallback to human escalation** when utilization fails
+
+### 3.3 Recommended Architecture for Crisis Support
+
+```
+Query → Hybrid Retrieval → Reader-Guided Reranking → Context Compression 
+→ Faithfulness-Optimized Prompt → CoT with Retrieval Revision 
+→ Evidence Verification → Answer/Hold/Escalate Decision
+```
+
+### 3.4 Expected Performance
+
+Based on the literature:
+- **Naive RAG:** R@5 ~95%, E2E accuracy ~15-25%
+- **With reranking:** E2E accuracy +1-4 points
+- **With faithfulness prompting:** E2E accuracy +5-15 points  
+- **With iterative CoT+retrieval:** E2E accuracy +10-20 points
+- **Combined interventions:** E2E accuracy 50-70% (realistic target)
+
+The gap can be reduced from 81 points to ~25-45 points with proper interventions.
+
+---
+
+## 4. Key Takeaways
+
+### The Gap is Fundamental, Not Accidental
+- Even oracle retrieval doesn't guarantee correct answers
+- Smaller models (<7B) have a "utilization bottleneck"
+- The distraction effect means more context can hurt
+
+### Bridging the Gap Requires Multi-Pronged Approach
+1. **Better retrieval alignment** (reader-guided, utility-oriented)
+2. **Better context processing** (compression, filtering, summarization)  
+3. **Better prompting** (faithfulness, opinion-based, CoT)
+4. **Better verification** (evidence checking, gap detection)
+
+### Crisis Support Specific Considerations
+- High stakes mean low tolerance for hallucination
+- Complex queries require multi-step reasoning
+- Domain expertise needs explicit encoding in prompts
+- Safety requires explicit hold/escalate mechanisms
+
+---
+
+## 5. References
+
+1. Pandey, S. (2026). "Can Small Language Models Use What They Retrieve?" arXiv:2603.11513
+2. Zhou, W. et al. (2023). "Context-faithful Prompting for Large Language Models." arXiv:2303.11315
+3. Zhao, S. et al. (2024). "Retrieval Augmented Generation (RAG) and Beyond." arXiv:2409.14924
+4. Mao, Y. et al. (2021). "Rider: Reader-Guided Passage Reranking." arXiv:2101.00294
+5. George, S. (2025). "Enhancing RAG with Two-Stage Retrieval." arXiv:2601.03258
+6. Asl, M.A. et al. (2025). "FAIR-RAG: Faithful Adaptive Iterative Refinement." arXiv:2510.22344
+7. Zhao, Y. et al. (2024). "An Empirical Study of RAG with Chain-of-Thought." arXiv:2407.15569
+8. Wang, Z. et al. (2024). "RAT: Retrieval Augmented Thoughts." arXiv:2403.05313
+9. Gan, C. et al. (2024). "Similarity is Not All You Need: MetRag." arXiv:2405.19893
+10. Pouplin, T. et al. (2024). "Retrieval Augmented Thought Process." arXiv:2402.07812
+11. Li, J. et al. (2024). "Tree of Reviews." arXiv:2404.14464
+12. Tian, F. et al. (2026). "Predicting Retrieval Utility and Answer Quality in RAG." arXiv:2601.14546
+13. Qi, J. et al. (2025). "On the Consistency of Multilingual Context Utilization in RAG." arXiv:2504.00597
+
+---
+
+## 6. Limitations of This Research
+
+1. **MemPalace/Engram team analysis not found** - The specific analysis that discovered the 17% figure was not located through academic search. This may be from internal reports, blog posts, or presentations not indexed in arXiv.
+
+2. **Domain specificity** - Most RAG research focuses on general QA, not crisis support. The patterns may need adaptation for high-stakes, sensitive domains.
+
+3. **Model size effects** - The utilization bottleneck is worse for smaller models. The MemPalace system's model size is unknown.
+
+4. **Evaluation methodology** - Different papers use different metrics (EM, F1, accuracy), making direct comparison difficult.
+
+---
+
+*Research conducted: April 14, 2026*
+*Researcher: Hermes Agent (subagent)*
+*Task: Research Task #1 - R@5 vs End-to-End Accuracy Gap*

research_text_to_music_video.md

@@ -0,0 +1,208 @@
+# Open-Source Text-to-Music-Video Pipeline Research
+
+## Executive Summary
+
+**The complete text-to-music-video pipeline does NOT exist as a single open-source tool.** The landscape consists of powerful individual components that must be manually stitched together. This is the gap our Video Forge can fill.
+
+---
+
+## 1. EXISTING OPEN-SOURCE PIPELINES
+
+### Complete (but crude) Pipelines
+
+| Project | Stars | Description | Status |
+|---------|-------|-------------|--------|
+| **MusicVideoMaker** | 3 | Stable Diffusion pipeline for music videos from lyrics. Uses Excel spreadsheet for lyrics+timing, generates key frames, smooths between them. | Proof-of-concept, Jupyter notebook, not production-ready |
+| **DuckTapeVideos** | 0 | Node-based AI pipeline for beat-synced music videos from lyrics | Minimal, early stage |
+| **song-video-gen** | 0 | Stable Diffusion lyrics-based generative AI pipeline | Fork/copy of above |
+| **TikTok-Lyric-Video-Pipeline** | 1 | Automated Python pipeline for TikTok lyric videos (10-15/day) | Focused on lyric overlay, not generative visuals |
+
+**Verdict: Nothing production-ready exists as a complete pipeline.**
+
+---
+
+## 2. INDIVIDUAL COMPONENTS (What's Already Free)
+
+### A. Music Generation (Suno Alternatives)
+
+| Project | Stars | License | Self-Hostable | Quality |
+|---------|-------|---------|---------------|---------|
+| **YuE** | 6,144 | Apache-2.0 | ✅ Yes | Full-song generation with vocals, Suno-level quality |
+| **HeartMuLa** | 4,037 | Apache-2.0 | ✅ Yes | Most powerful open-source music model (2026), multilingual |
+| **ACE-Step 1.5 + UI** | 970 | MIT | ✅ Yes | Professional Spotify-like UI, full song gen, 4+ min with vocals |
+| **Facebook MusicGen** | ~45k downloads | MIT | ✅ Yes | Good quality, melody conditioning, well-documented |
+| **Riffusion** | ~6k stars | Apache-2.0 | ✅ Yes | Spectrogram-based, unique approach |
+
+**Status: Suno is effectively "given away" for free. YuE and HeartMuLa are production-ready.**
+
+### B. Image Generation (Per-Scene/Beat)
+
+| Project | Downloads/Stars | License | Notes |
+|---------|-----------------|---------|-------|
+| **Stable Diffusion XL** | 1.9M downloads | CreativeML | Best quality, huge ecosystem |
+| **Stable Diffusion 1.5** | 1.6M downloads | CreativeML | Fast, lightweight |
+| **FLUX** | Emerging | Apache-2.0 | Newest, excellent quality |
+| **ComfyUI** | 60k+ stars | GPL-3.0 | Node-based pipeline editor, massive plugin ecosystem |
+
+**Status: Image generation is completely "given away." SD XL + ComfyUI is production-grade.**
+
+### C. Text-to-Video Generation
+
+| Project | Stars | License | Capabilities |
+|---------|-------|---------|--------------|
+| **Wan2.1** | 15,815 | Apache-2.0 | State-of-the-art, text-to-video and image-to-video |
+| **CogVideoX** | 12,634 | Apache-2.0 | Text and image to video, good quality |
+| **HunyuanVideo** | 11,965 | Custom | Tencent's framework, high quality |
+| **Stable Video Diffusion** | 3k+ likes | Stability AI | Image-to-video, good for short clips |
+| **LTX-Video** | Growing | Apache-2.0 | Fast inference, good quality |
+
+**Status: Text-to-video is rapidly being "given away." Wan2.1 is production-ready for short clips (4-6 seconds).**
+
+### D. Video Composition & Assembly
+
+| Project | Stars | License | Use Case |
+|---------|-------|---------|----------|
+| **Remotion** | 43,261 | Custom (SSPL) | Programmatic video with React, production-grade |
+| **MoviePy** | 12k+ stars | MIT | Python video editing, widely used |
+| **Mosaico** | 16 | MIT | Python video composition with AI integration |
+| **FFmpeg** | N/A | LGPL/GPL | The universal video tool |
+
+**Status: Video composition tools are mature and free. Remotion is production-grade.**
+
+### E. Lyrics/Text Processing
+
+| Component | Status | Notes |
+|-----------|--------|-------|
+| **Lyrics-to-scene segmentation** | ❌ Missing | No good open-source tool for breaking lyrics into visual scenes |
+| **Beat detection** | ✅ Exists | Librosa, madmom, aubio - all free and mature |
+| **Text-to-prompt generation** | ✅ Exists | LLMs (Ollama, local models) can do this |
+| **LRC/SRT parsing** | ✅ Exists | Many libraries available |
+
+---
+
+## 3. WHAT'S BEEN "GIVEN AWAY" FOR FREE
+
+### Fully Solved (Production-Ready, Self-Hostable)
+- ✅ **Music generation**: YuE, HeartMuLa, ACE-Step match Suno quality
+- ✅ **Image generation**: SD XL, FLUX - commercial quality
+- ✅ **Video composition**: FFmpeg, MoviePy, Remotion
+- ✅ **Beat/audio analysis**: Librosa, madmom
+- ✅ **Text-to-video (short clips)**: Wan2.1, CogVideoX
+- ✅ **TTS/voice**: XTTS-v2, Kokoro, Bark
+
+### Partially Solved
+- ⚠️ **Image-to-video**: Good for 4-6 second clips, struggles with longer sequences
+- ⚠️ **Style consistency**: LoRAs and ControlNet help, but not perfect across scenes
+- ⚠️ **Prompt engineering**: LLMs can help, but no dedicated lyrics-to-visual-prompt tool
+
+---
+
+## 4. WHERE THE REAL GAPS ARE
+
+### Critical Gaps (Our Opportunity)
+
+1. **Unified Pipeline Orchestration**
+   - NO tool chains: lyrics → music → scene segmentation → image prompts → video composition
+   - Everything requires manual stitching
+   - Our Video Forge can be THE glue layer
+
+2. **Lyrics-to-Visual-Scene Segmentation**
+   - No tool analyzes lyrics and breaks them into visual beats/scenes
+   - MusicVideoMaker uses manual Excel entry - absurd
+   - Opportunity: LLM-powered scene segmentation with beat alignment
+
+3. **Temporal Coherence Across Scenes**
+   - Short clips (4-6s) work fine, but maintaining visual coherence across a 3-4 minute video is unsolved
+   - Character consistency, color palette continuity, style drift
+   - Opportunity: Style anchoring + scene-to-scene conditioning
+
+4. **Beat-Synchronized Visual Transitions**
+   - No tool automatically syncs visual cuts to musical beats
+   - Manual timing is required everywhere
+   - Opportunity: Beat detection → transition scheduling → FFmpeg composition
+
+5. **Long-Form Video Generation**
+   - Text-to-video models max out at 4-6 seconds
+   - Stitching clips with consistent style/characters is manual
+   - Opportunity: Automated clip chaining with style transfer
+
+6. **One-Click "Lyrics In, Video Out"**
+   - The dream pipeline doesn't exist
+   - Current workflows require 5+ separate tools
+   - Opportunity: Single command/endpoint that does everything
+
+### Technical Debt in Existing Tools
+
+- **YuE/HeartMuLa**: No video awareness - just audio generation
+- **Wan2.1/CogVideoX**: No lyrics/text awareness - just prompt-to-video
+- **ComfyUI**: Great for images, weak for video composition
+- **Remotion**: Great for composition, no AI generation built-in
+
+---
+
+## 5. RECOMMENDED ARCHITECTURE FOR VIDEO FORGE
+
+Based on this research, the optimal Video Forge pipeline:
+
+```
+[Lyrics/Poem Text]
+        ↓
+[LLM Scene Segmenter] → Beat-aligned scene descriptions + visual prompts
+        ↓
+[HeartMuLa/YuE] → Music audio (.wav)
+        ↓
+[Beat Detector (librosa)] → Beat timestamps + energy curve
+        ↓
+[SD XL / FLUX] → Scene images (one per beat/section)
+        ↓
+[Wan2.1 img2vid] → Short video clips per scene (4-6s each)
+        ↓
+[FFmpeg + Beat Sync] → Transitions aligned to beats
+        ↓
+[Final Music Video (.mp4)]
+```
+
+### Key Design Decisions
+
+1. **Music**: HeartMuLa (best quality, multilingual, Apache-2.0)
+2. **Images**: SD XL via ComfyUI (most mature ecosystem)
+3. **Video clips**: Wan2.1 for img2vid (state-of-the-art)
+4. **Composition**: FFmpeg (universal, battle-tested)
+5. **Orchestration**: Python pipeline with config file
+6. **Scene segmentation**: Local LLM (Ollama + Llama 3 or similar)
+
+### What We Build vs. What We Use
+
+| Component | Build or Use | Reasoning |
+|-----------|--------------|-----------|
+| Lyrics → Scenes | **BUILD** | No good tool exists, core differentiator |
+| Music generation | **USE** HeartMuLa/YuE | Already excellent, Apache-2.0 |
+| Image generation | **USE** SD XL | Mature, huge ecosystem |
+| Beat detection | **USE** librosa | Mature, reliable |
+| Video clips | **USE** Wan2.1 | Best quality, Apache-2.0 |
+| Video composition | **BUILD** (ffmpeg wrapper) | Need beat-sync logic |
+| Pipeline orchestration | **BUILD** | The main value-add |
+
+---
+
+## 6. COMPETITIVE LANDSCAPE SUMMARY
+
+### Commercial (Not Self-Hostable)
+- **Suno**: Music only, no video
+- **Runway**: Video only, expensive
+- **Pika**: Short clips only
+- **Kaiber**: Closest to music video, but closed/subscription
+- **Synthesia**: Avatar-based, not generative art
+
+### Open-Source Gaps That Matter
+1. Nobody has built the orchestration layer
+2. Nobody has solved lyrics-to-visual-scene well
+3. Nobody has beat-synced visual transitions automated
+4. Nobody maintains temporal coherence across minutes
+
+**Our Video Forge fills the most important gap: the glue that makes individual AI components work together to produce a complete music video from text.**
+
+---
+
+*Research conducted: April 14, 2026*
+*Sources: GitHub API, HuggingFace API, project READMEs*

run_agent.py

@@ -106,7 +106,7 @@ from agent.trajectory import (
     convert_scratchpad_to_think, has_incomplete_scratchpad,
     save_trajectory as _save_trajectory_to_file,
 )
-from utils import atomic_json_write, env_var_enabled
+from utils import atomic_json_write, env_var_enabled, repair_and_load_json
 
 
 
@@ -277,7 +277,7 @@ def _should_parallelize_tool_batch(tool_calls) -> bool:
     for tool_call in tool_calls:
         tool_name = tool_call.function.name
         try:
-            function_args = json.loads(tool_call.function.arguments)
+            function_args = repair_and_load_json(tool_call.function.arguments, default={})
         except Exception:
             logging.debug(
                 "Could not parse args for %s — defaulting to sequential; raw=%s",
@@ -2246,9 +2246,8 @@ class AIAgent:
                 for msg in getattr(review_agent, "_session_messages", []):
                     if not isinstance(msg, dict) or msg.get("role") != "tool":
                         continue
-                    try:
-                        data = json.loads(msg.get("content", "{}"))
-                    except (json.JSONDecodeError, TypeError):
+                    data = repair_and_load_json(msg.get("content", "{}"), default=None, context="trajectory_content")
+                    if data is None:
                         continue
                     if not data.get("success"):
                         continue
@@ -2496,13 +2495,13 @@ class AIAgent:
                         if not tool_call or not isinstance(tool_call, dict): continue
                         # Parse arguments - should always succeed since we validate during conversation
                         # but keep try-except as safety net
-                        try:
-                            arguments = json.loads(tool_call["function"]["arguments"]) if isinstance(tool_call["function"]["arguments"], str) else tool_call["function"]["arguments"]
-                        except json.JSONDecodeError:
-                            # This shouldn't happen since we validate and retry during conversation,
-                            # but if it does, log warning and use empty dict
-                            logging.warning(f"Unexpected invalid JSON in trajectory conversion: {tool_call['function']['arguments'][:100]}")
-                            arguments = {}
+                        raw_args = tool_call["function"]["arguments"]
+                        if isinstance(raw_args, str):
+                            arguments = repair_and_load_json(raw_args, default={}, context="trajectory_tool_call")
+                            if arguments == {} and raw_args.strip() not in ("{}", ""):
+                                logging.warning("Unexpected invalid JSON in trajectory conversion: %.100s", raw_args)
+                        else:
+                            arguments = raw_args
                         
                         tool_call_json = {
                             "name": tool_call["function"]["name"],
@@ -2530,11 +2529,10 @@ class AIAgent:
                         
                         # Try to parse tool content as JSON if it looks like JSON
                         tool_content = tool_msg["content"]
-                        try:
-                            if tool_content.strip().startswith(("{", "[")):
-                                tool_content = json.loads(tool_content)
-                        except (json.JSONDecodeError, AttributeError):
-                            pass  # Keep as string if not valid JSON
+                        if isinstance(tool_content, str) and tool_content.strip().startswith(("{", "[")):
+                            parsed = repair_and_load_json(tool_content, default=None, context="trajectory_tool_content")
+                            if parsed is not None:
+                                tool_content = parsed
                         
                         tool_index = len(tool_responses)
                         tool_name = (
@@ -2885,14 +2883,21 @@ class AIAgent:
             # with partial history and would otherwise clobber the full JSON log.
             if self.session_log_file.exists():
                 try:
-                    existing = json.loads(self.session_log_file.read_text(encoding="utf-8"))
-                    existing_count = existing.get("message_count", len(existing.get("messages", [])))
-                    if existing_count > len(cleaned):
-                        logging.debug(
-                            "Skipping session log overwrite: existing has %d messages, current has %d",
-                            existing_count, len(cleaned),
-                        )
-                        return
+                    existing = repair_and_load_json(
+                        self.session_log_file.read_text(encoding="utf-8"),
+                        default=None,
+                        context="session_log_load",
+                    )
+                    if existing is None:
+                        logging.warning("Session log at %s could not be parsed; allowing overwrite", self.session_log_file)
+                    else:
+                        existing_count = existing.get("message_count", len(existing.get("messages", [])))
+                        if existing_count > len(cleaned):
+                            logging.debug(
+                                "Skipping session log overwrite: existing has %d messages, current has %d",
+                                existing_count, len(cleaned),
+                            )
+                            return
                 except Exception:
                     pass  # corrupted existing file — allow the overwrite
 
@@ -3115,13 +3120,12 @@ class AIAgent:
             # Quick check: todo responses contain "todos" key
             if '"todos"' not in content:
                 continue
-            try:
-                data = json.loads(content)
-                if "todos" in data and isinstance(data["todos"], list):
-                    last_todo_response = data["todos"]
-                    break
-            except (json.JSONDecodeError, TypeError):
+            data = repair_and_load_json(content, default=None, context="todo_content")
+            if data is None:
                 continue
+            if "todos" in data and isinstance(data["todos"], list):
+                last_todo_response = data["todos"]
+                break
         
         if last_todo_response:
             # Replay the items into the store (replace mode)
@@ -5960,7 +5964,7 @@ class AIAgent:
             result_json = asyncio.run(
                 vision_analyze_tool(image_url=vision_source, user_prompt=analysis_prompt)
             )
-            result = json.loads(result_json) if isinstance(result_json, str) else {}
+            result = repair_and_load_json(result_json, default={}, context="vision_result") if isinstance(result_json, str) else {}
             description = (result.get("analysis") or "").strip()
         except Exception as e:
             description = f"Image analysis failed: {e}"
@@ -6758,7 +6762,7 @@ class AIAgent:
             for tc in tool_calls:
                 if tc.function.name == "memory":
                     try:
-                        args = json.loads(tc.function.arguments)
+                        args = repair_and_load_json(tc.function.arguments, default={}, context="memory_flush")
                         flush_target = args.get("target", "memory")
                         from tools.memory_tool import memory_tool as _memory_tool
                         _memory_tool(
@@ -7065,7 +7069,7 @@ class AIAgent:
                 self._iters_since_skill = 0
 
             try:
-                function_args = json.loads(tool_call.function.arguments)
+                function_args = repair_and_load_json(tool_call.function.arguments, default={})
             except json.JSONDecodeError:
                 function_args = {}
             if not isinstance(function_args, dict):
@@ -7262,7 +7266,7 @@ class AIAgent:
             function_name = tool_call.function.name
 
             try:
-                function_args = json.loads(tool_call.function.arguments)
+                function_args = repair_and_load_json(tool_call.function.arguments, default={})
             except json.JSONDecodeError as e:
                 logging.warning(f"Unexpected JSON error after validation: {e}")
                 function_args = {}
@@ -8297,14 +8301,15 @@ class AIAgent:
                 for tc in tcs:
                     if isinstance(tc, dict) and "function" in tc:
                         try:
-                            args_obj = json.loads(tc["function"]["arguments"])
-                            tc = {**tc, "function": {
-                                **tc["function"],
-                                "arguments": json.dumps(
-                                    args_obj, separators=(",", ":"),
-                                    sort_keys=True,
-                                ),
-                            }}
+                            args_obj = repair_and_load_json(tc["function"]["arguments"], default=None, context="cache_serialization")
+                            if args_obj is not None:
+                                tc = {**tc, "function": {
+                                    **tc["function"],
+                                    "arguments": json.dumps(
+                                        args_obj, separators=(",", ":"),
+                                        sort_keys=True,
+                                    ),
+                                }}
                         except Exception:
                             pass
                     new_tcs.append(tc)

scripts/lint_hardcoded_paths.py

@@ -0,0 +1,278 @@
+#!/usr/bin/env python3
+"""
+Poka-yoke: Hardcoded path linter for hermes-agent.
+
+Scans Python files for hardcoded home-directory paths that break
+multi-user/multi-profile deployments. Catches:
+  - Path.home() / ".hermes" without HERMES_HOME env var fallback
+  - Hardcoded /Users/<name>/ paths
+  - Hardcoded /home/<name>/ paths
+  - Raw ~/.hermes in code (not in comments/docstrings)
+
+Usage:
+    python3 scripts/lint_hardcoded_paths.py              # lint all .py files
+    python3 scripts/lint_hardcoded_paths.py --fix         # suggest fixes
+    python3 scripts/lint_hardcoded_paths.py --staged      # lint git staged files only
+
+Exit codes:
+    0 = no violations
+    1 = violations found
+    2 = error
+"""
+
+import argparse
+import os
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+
+# ── Patterns ──────────────────────────────────────────────────────
+
+VIOLATIONS = [
+    {
+        "id": "direct-home-hermes",
+        "name": "Direct Path.home()/.hermes",
+        "pattern": r'Path\.home\(\)\s*/\s*["\']\.hermes["\']',
+        "exclude_with": r'os\.getenv\(|os\.environ\.get\(|_get_profiles_root|profiles_parent|current_default|native_home',
+        "message": "Use `Path(os.getenv('HERMES_HOME', Path.home() / '.hermes'))` instead of direct `Path.home() / '.hermes'`",
+    },
+    {
+        "id": "hardcoded-user-path",
+        "name": "Hardcoded /Users/<name>/",
+        "pattern": r'["\']/Users/[a-zA-Z_][a-zA-Z0-9_]*/',
+        "exclude_with": r'#|""".*"""\s*$',
+        "message": "Use environment variables or relative paths instead of hardcoded /Users/<name>/",
+    },
+    {
+        "id": "hardcoded-home-path",
+        "name": "Hardcoded /home/<name>/",
+        "pattern": r'["\']/home/[a-zA-Z_][a-zA-Z0-9_]*/',
+        "exclude_with": r'#|""".*"""\s*$',
+        "message": "Use environment variables or relative paths instead of hardcoded /home/<name>/",
+    },
+    {
+        "id": "expanduser-hermes",
+        "name": "os.path.expanduser ~/.hermes (non-fallback)",
+        "pattern": r'os\.path\.expanduser\(["\']~/.hermes',
+        "exclude_with": r'#',
+        "message": "Use `os.environ.get('HERMES_HOME', os.path.expanduser('~/.hermes'))` instead",
+    },
+]
+
+
+# ── Exceptions ─────────────────────────────────────────────────────
+# Files where hardcoded paths are acceptable (tests with mock data,
+# migration scripts, docs generation)
+
+EXCEPTIONS = [
+    "tests/",           # Test fixtures can use mock paths
+    "scripts/",         # One-off scripts
+    "optional-skills/", # Skills not in core
+    "skills/",          # External skills
+    "plugins/",         # Plugins
+    "website/",         # Docs site
+    "mcp_serve.py",     # Standalone MCP server
+    "docs/",            # Documentation
+]
+
+
+# ── Scanner ────────────────────────────────────────────────────────
+
+def is_exception(filepath: str) -> bool:
+    """Check if file is in the exception list."""
+    for exc in EXCEPTIONS:
+        if filepath.startswith(exc) or f"/{exc}" in filepath:
+            return True
+    return False
+
+
+def is_in_comment_or_docstring(line: str, lines: list, line_idx: int) -> bool:
+    """Check if the match is in a comment or docstring."""
+    stripped = line.strip()
+
+    # Line comment
+    if stripped.startswith("#"):
+        return True
+
+    # Inline comment — check if match is after #
+    if "#" in line:
+        code_part = line[:line.index("#")]
+        for v in VIOLATIONS:
+            if re.search(v["pattern"], code_part):
+                return False  # Match is in code, not comment
+        return True  # No match in code part, must be in comment
+
+    # Simple docstring check: look for triple quotes before this line
+    in_docstring = False
+    quote_count = 0
+    for i in range(max(0, line_idx - 20), line_idx + 1):
+        for char in ['"""', "'''"]:
+            quote_count += lines[i].count(char)
+    if quote_count % 2 == 1:
+        in_docstring = True
+
+    # Also check current line for docstring delimiters
+    if '"""' in line or "'''" in line:
+        # If line is entirely within a docstring block, skip
+        before_match = line[:line.find(re.search(VIOLATIONS[0]["pattern"], line).group())] if re.search(VIOLATIONS[0]["pattern"], line) else ""
+        if '"""' in before_match or "'''" in before_match:
+            in_docstring = True
+
+    return in_docstring
+
+
+def scan_file(filepath: str) -> list:
+    """Scan a single file for violations."""
+    try:
+        with open(filepath) as f:
+            content = f.read()
+            lines = content.split("\n")
+    except (OSError, UnicodeDecodeError):
+        return []
+
+    violations_found = []
+
+    for i, line in enumerate(lines):
+        for v in VIOLATIONS:
+            match = re.search(v["pattern"], line)
+            if not match:
+                continue
+
+            # Check if excluded by context (e.g., it's part of a fallback pattern)
+            if v.get("exclude_with"):
+                if re.search(v["exclude_with"], line):
+                    continue
+
+            # Skip comments and docstrings
+            stripped = line.strip()
+            if stripped.startswith("#"):
+                continue
+
+            # Check if in inline comment
+            if "#" in line:
+                code_part = line[:line.index("#")]
+                if not re.search(v["pattern"], code_part):
+                    continue
+
+            violations_found.append({
+                "file": filepath,
+                "line": i + 1,
+                "rule": v["id"],
+                "name": v["name"],
+                "message": v["message"],
+                "text": stripped[:120],
+            })
+
+    return violations_found
+
+
+def get_staged_files() -> list:
+    """Get list of staged Python files from git."""
+    try:
+        result = subprocess.run(
+            ["git", "diff", "--cached", "--name-only", "--diff-filter=ACM"],
+            capture_output=True, text=True, timeout=10
+        )
+        return [f for f in result.stdout.strip().split("\n") if f.endswith(".py")]
+    except (subprocess.TimeoutExpired, FileNotFoundError):
+        return []
+
+
+def scan_all(root: str = ".") -> list:
+    """Scan all Python files in the repo."""
+    all_violations = []
+    for dirpath, dirnames, filenames in os.walk(root):
+        dirnames[:] = [d for d in dirnames if d not in (".git", "venv", "__pycache__", "node_modules")]
+        for f in filenames:
+            if not f.endswith(".py"):
+                continue
+            filepath = os.path.join(dirpath, f)
+            rel = os.path.relpath(filepath, root)
+
+            if is_exception(rel):
+                continue
+
+            all_violations.extend(scan_file(filepath))
+
+    return all_violations
+
+
+# ── Output ─────────────────────────────────────────────────────────
+
+def print_violations(violations: list) -> None:
+    """Print violations in a readable format."""
+    if not violations:
+        print("PASS: No hardcoded path violations found")
+        return
+
+    print(f"FAIL: {len(violations)} hardcoded path violation(s) found\n")
+
+    by_rule = {}
+    for v in violations:
+        by_rule.setdefault(v["rule"], []).append(v)
+
+    for rule, items in sorted(by_rule.items()):
+        print(f"  [{rule}] {items[0]['name']}")
+        print(f"    {items[0]['message']}")
+        for item in items:
+            print(f"    {item['file']}:{item['line']}: {item['text']}")
+        print()
+
+
+def print_fix_suggestions(violations: list) -> None:
+    """Print fix suggestions for violations."""
+    if not violations:
+        return
+
+    print("\n=== Fix Suggestions ===\n")
+
+    for v in violations:
+        print(f"  {v['file']}:{v['line']}")
+        print(f"    Current: {v['text']}")
+
+        if v["rule"] == "direct-home-hermes":
+            print(f"    Fix:     Use `Path(os.getenv('HERMES_HOME', Path.home() / '.hermes'))`")
+        elif v["rule"] in ("hardcoded-user-path", "hardcoded-home-path"):
+            print(f"    Fix:     Use `os.environ.get('HOME')` or `Path.home()`")
+        elif v["rule"] == "expanduser-hermes":
+            print(f"    Fix:     Use `os.environ.get('HERMES_HOME', os.path.expanduser('~/.hermes'))`")
+        print()
+
+
+# ── Main ───────────────────────────────────────────────────────────
+
+def main():
+    parser = argparse.ArgumentParser(description="Lint hardcoded paths in hermes-agent")
+    parser.add_argument("--staged", action="store_true", help="Only scan git staged files")
+    parser.add_argument("--fix", action="store_true", help="Show fix suggestions")
+    parser.add_argument("--json", action="store_true", help="Output as JSON")
+    parser.add_argument("--root", default=".", help="Root directory to scan")
+    args = parser.parse_args()
+
+    if args.staged:
+        files = get_staged_files()
+        if not files:
+            print("No staged Python files")
+            sys.exit(0)
+        violations = []
+        for f in files:
+            if not is_exception(f):
+                violations.extend(scan_file(f))
+    else:
+        violations = scan_all(args.root)
+
+    if args.json:
+        import json
+        print(json.dumps(violations, indent=2))
+    else:
+        print_violations(violations)
+        if args.fix:
+            print_fix_suggestions(violations)
+
+    sys.exit(1 if violations else 0)
+
+
+if __name__ == "__main__":
+    main()

scripts/mcp_server.py

@@ -0,0 +1,265 @@
+#!/usr/bin/env python3
+"""Hermes MCP Server — expose hermes-agent tools to fleet peers.
+
+Runs as a standalone MCP server that other agents can connect to
+and invoke hermes tools remotely.
+
+Safe tools exposed:
+- terminal (safe commands only)
+- file_read, file_search
+- web_search, web_extract
+- session_search
+
+NOT exposed (internal tools):
+- approval, delegate, memory, config
+
+Usage:
+    python -m tools.mcp_server --port 8081
+    hermes mcp-server --port 8081
+    python scripts/mcp_server.py --port 8081 --auth-key SECRET
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import logging
+import os
+import sys
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+# Tools safe to expose to other agents
+SAFE_TOOLS = {
+    "terminal": {
+        "name": "terminal",
+        "description": "Execute safe shell commands. Dangerous commands are blocked.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "command": {"type": "string", "description": "Shell command to execute"},
+            },
+            "required": ["command"],
+        },
+    },
+    "file_read": {
+        "name": "file_read",
+        "description": "Read the contents of a file.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "path": {"type": "string", "description": "File path to read"},
+                "offset": {"type": "integer", "description": "Start line", "default": 1},
+                "limit": {"type": "integer", "description": "Max lines", "default": 200},
+            },
+            "required": ["path"],
+        },
+    },
+    "file_search": {
+        "name": "file_search",
+        "description": "Search file contents using regex.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "pattern": {"type": "string", "description": "Regex pattern"},
+                "path": {"type": "string", "description": "Directory to search", "default": "."},
+            },
+            "required": ["pattern"],
+        },
+    },
+    "web_search": {
+        "name": "web_search",
+        "description": "Search the web for information.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "query": {"type": "string", "description": "Search query"},
+            },
+            "required": ["query"],
+        },
+    },
+    "session_search": {
+        "name": "session_search",
+        "description": "Search past conversation sessions.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "query": {"type": "string", "description": "Search query"},
+                "limit": {"type": "integer", "description": "Max results", "default": 3},
+            },
+            "required": ["query"],
+        },
+    },
+}
+
+# Tools explicitly blocked
+BLOCKED_TOOLS = {
+    "approval", "delegate", "memory", "config", "skill_install",
+    "mcp_tool", "cronjob", "tts", "send_message",
+}
+
+
+class MCPServer:
+    """Simple MCP-compatible server for exposing hermes tools."""
+
+    def __init__(self, host: str = "127.0.0.1", port: int = 8081,
+                 auth_key: Optional[str] = None):
+        self._host = host
+        self._port = port
+        self._auth_key = auth_key or os.getenv("MCP_AUTH_KEY", "")
+
+    async def handle_tools_list(self, request: dict) -> dict:
+        """Return available tools."""
+        tools = list(SAFE_TOOLS.values())
+        return {"tools": tools}
+
+    async def handle_tools_call(self, request: dict) -> dict:
+        """Execute a tool call."""
+        tool_name = request.get("name", "")
+        arguments = request.get("arguments", {})
+
+        if tool_name in BLOCKED_TOOLS:
+            return {"error": f"Tool '{tool_name}' is not exposed via MCP"}
+        if tool_name not in SAFE_TOOLS:
+            return {"error": f"Unknown tool: {tool_name}"}
+
+        try:
+            result = await self._execute_tool(tool_name, arguments)
+            return {"content": [{"type": "text", "text": str(result)}]}
+        except Exception as e:
+            return {"error": str(e)}
+
+    async def _execute_tool(self, tool_name: str, arguments: dict) -> str:
+        """Execute a tool and return result."""
+        if tool_name == "terminal":
+            import subprocess
+            cmd = arguments.get("command", "")
+            # Block dangerous commands
+            from tools.approval import detect_dangerous_command
+            is_dangerous, _, desc = detect_dangerous_command(cmd)
+            if is_dangerous:
+                return f"BLOCKED: Dangerous command detected ({desc}). This tool only executes safe commands."
+            result = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=30)
+            return result.stdout or result.stderr or "(no output)"
+
+        elif tool_name == "file_read":
+            path = arguments.get("path", "")
+            offset = arguments.get("offset", 1)
+            limit = arguments.get("limit", 200)
+            with open(path) as f:
+                lines = f.readlines()
+            return "".join(lines[offset-1:offset-1+limit])
+
+        elif tool_name == "file_search":
+            import re
+            pattern = arguments.get("pattern", "")
+            path = arguments.get("path", ".")
+            results = []
+            for p in Path(path).rglob("*.py"):
+                try:
+                    content = p.read_text()
+                    for i, line in enumerate(content.split("\n"), 1):
+                        if re.search(pattern, line, re.IGNORECASE):
+                            results.append(f"{p}:{i}: {line.strip()}")
+                            if len(results) >= 20:
+                                break
+                except Exception:
+                    continue
+                if len(results) >= 20:
+                    break
+            return "\n".join(results) or "No matches found"
+
+        elif tool_name == "web_search":
+            try:
+                from tools.web_tools import web_search
+                return web_search(arguments.get("query", ""))
+            except ImportError:
+                return "Web search not available"
+
+        elif tool_name == "session_search":
+            try:
+                from tools.session_search_tool import session_search
+                return session_search(
+                    query=arguments.get("query", ""),
+                    limit=arguments.get("limit", 3),
+                )
+            except ImportError:
+                return "Session search not available"
+
+        return f"Tool {tool_name} not implemented"
+
+    async def start_http(self):
+        """Start HTTP server for MCP endpoints."""
+        try:
+            from aiohttp import web
+        except ImportError:
+            logger.error("aiohttp required: pip install aiohttp")
+            return
+
+        app = web.Application()
+
+        async def handle_tools_list_route(request):
+            if self._auth_key:
+                auth = request.headers.get("Authorization", "")
+                if auth != f"Bearer {self._auth_key}":
+                    return web.json_response({"error": "Unauthorized"}, status=401)
+            result = await self.handle_tools_list({})
+            return web.json_response(result)
+
+        async def handle_tools_call_route(request):
+            if self._auth_key:
+                auth = request.headers.get("Authorization", "")
+                if auth != f"Bearer {self._auth_key}":
+                    return web.json_response({"error": "Unauthorized"}, status=401)
+            body = await request.json()
+            result = await self.handle_tools_call(body)
+            return web.json_response(result)
+
+        async def handle_health(request):
+            return web.json_response({"status": "ok", "tools": len(SAFE_TOOLS)})
+
+        app.router.add_get("/mcp/tools", handle_tools_list_route)
+        app.router.add_post("/mcp/tools/call", handle_tools_call_route)
+        app.router.add_get("/health", handle_health)
+
+        runner = web.AppRunner(app)
+        await runner.setup()
+        site = web.TCPSite(runner, self._host, self._port)
+        await site.start()
+        logger.info("MCP server on http://%s:%s", self._host, self._port)
+        logger.info("Tools: %s", ", ".join(SAFE_TOOLS.keys()))
+        if self._auth_key:
+            logger.info("Auth: Bearer token required")
+        else:
+            logger.warning("Auth: No MCP_AUTH_KEY set — server is open")
+
+        try:
+            await asyncio.Event().wait()
+        except asyncio.CancelledError:
+            pass
+        finally:
+            await runner.cleanup()
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Hermes MCP Server")
+    parser.add_argument("--host", default="127.0.0.1")
+    parser.add_argument("--port", type=int, default=8081)
+    parser.add_argument("--auth-key", default=None, help="Bearer token for auth")
+    args = parser.parse_args()
+
+    logging.basicConfig(level=logging.INFO,
+                        format="%(asctime)s [%(name)s] %(levelname)s: %(message)s")
+
+    server = MCPServer(host=args.host, port=args.port, auth_key=args.auth_key)
+    print(f"Starting MCP server on http://{args.host}:{args.port}")
+    print(f"Exposed tools: {', '.join(SAFE_TOOLS.keys())}")
+    asyncio.run(server.start_http())
+
+
+if __name__ == "__main__":
+    main()

scripts/pre-commit-hardcoded-paths.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/sh
+# Pre-commit hook: block commits with hardcoded home-directory paths
+# Install: cp scripts/pre-commit-hardcoded-paths.sh .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit
+# Or: git config core.hooksPath .githooks
+
+python3 scripts/lint_hardcoded_paths.py --staged
+exit $?

tests/agent/test_privacy_filter.py

@@ -0,0 +1,202 @@
+"""Tests for agent.privacy_filter — PII stripping before remote API calls."""
+
+import pytest
+from agent.privacy_filter import (
+    PrivacyFilter,
+    RedactionReport,
+    Sensitivity,
+    sanitize_messages,
+    quick_sanitize,
+)
+
+
+class TestPrivacyFilterSanitizeText:
+    """Test single-text sanitization."""
+
+    def test_no_pii_returns_clean(self):
+        pf = PrivacyFilter()
+        text = "The weather in Paris is nice today."
+        cleaned, redactions = pf.sanitize_text(text)
+        assert cleaned == text
+        assert redactions == []
+
+    def test_email_redacted(self):
+        pf = PrivacyFilter()
+        text = "Send report to alice@example.com by Friday."
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "alice@example.com" not in cleaned
+        assert "[REDACTED-EMAIL]" in cleaned
+        assert any(r["type"] == "email_address" for r in redactions)
+
+    def test_phone_redacted(self):
+        pf = PrivacyFilter()
+        text = "Call me at 555-123-4567 when ready."
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "555-123-4567" not in cleaned
+        assert "[REDACTED-PHONE]" in cleaned
+
+    def test_api_key_redacted(self):
+        pf = PrivacyFilter()
+        text = 'api_key = "sk-proj-abcdefghij1234567890abcdefghij1234567890"'
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "sk-proj-" not in cleaned
+        assert any(r["sensitivity"] == "CRITICAL" for r in redactions)
+
+    def test_github_token_redacted(self):
+        pf = PrivacyFilter()
+        text = "Use ghp_1234567890abcdefghijklmnopqrstuvwxyz1234 for auth"
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "ghp_" not in cleaned
+        assert any(r["type"] == "github_token" for r in redactions)
+
+    def test_ethereum_address_redacted(self):
+        pf = PrivacyFilter()
+        text = "Send to 0x742d35Cc6634C0532925a3b844Bc9e7595f2bD18 please"
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "0x742d" not in cleaned
+        assert any(r["type"] == "ethereum_address" for r in redactions)
+
+    def test_user_home_path_redacted(self):
+        pf = PrivacyFilter()
+        text = "Read file at /Users/alice/Documents/secret.txt"
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "alice" not in cleaned
+        assert "[REDACTED-USER]" in cleaned
+
+    def test_multiple_pii_types(self):
+        pf = PrivacyFilter()
+        text = (
+            "Contact john@test.com or call 555-999-1234. "
+            "The API key is sk-abcdefghijklmnopqrstuvwxyz1234567890."
+        )
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "john@test.com" not in cleaned
+        assert "555-999-1234" not in cleaned
+        assert "sk-abcd" not in cleaned
+        assert len(redactions) >= 3
+
+
+class TestPrivacyFilterSanitizeMessages:
+    """Test message-list sanitization."""
+
+    def test_sanitize_user_message(self):
+        pf = PrivacyFilter()
+        messages = [
+            {"role": "system", "content": "You are helpful."},
+            {"role": "user", "content": "Email me at bob@test.com with results."},
+        ]
+        safe, report = pf.sanitize_messages(messages)
+        assert report.redacted_messages == 1
+        assert "bob@test.com" not in safe[1]["content"]
+        assert "[REDACTED-EMAIL]" in safe[1]["content"]
+        # System message unchanged
+        assert safe[0]["content"] == "You are helpful."
+
+    def test_no_redaction_needed(self):
+        pf = PrivacyFilter()
+        messages = [
+            {"role": "user", "content": "What is 2+2?"},
+            {"role": "assistant", "content": "4"},
+        ]
+        safe, report = pf.sanitize_messages(messages)
+        assert report.redacted_messages == 0
+        assert not report.had_redactions
+
+    def test_assistant_messages_also_sanitized(self):
+        pf = PrivacyFilter()
+        messages = [
+            {"role": "assistant", "content": "Your email admin@corp.com was found."},
+        ]
+        safe, report = pf.sanitize_messages(messages)
+        assert report.redacted_messages == 1
+        assert "admin@corp.com" not in safe[0]["content"]
+
+    def test_tool_messages_not_sanitized(self):
+        pf = PrivacyFilter()
+        messages = [
+            {"role": "tool", "content": "Result: user@test.com found"},
+        ]
+        safe, report = pf.sanitize_messages(messages)
+        assert report.redacted_messages == 0
+        assert safe[0]["content"] == "Result: user@test.com found"
+
+
+class TestShouldUseLocalOnly:
+    """Test the local-only routing decision."""
+
+    def test_normal_text_allows_remote(self):
+        pf = PrivacyFilter()
+        block, reason = pf.should_use_local_only("Summarize this article about Python.")
+        assert not block
+
+    def test_critical_secret_blocks_remote(self):
+        pf = PrivacyFilter()
+        text = "Here is the API key: sk-abcdefghijklmnopqrstuvwxyz1234567890"
+        block, reason = pf.should_use_local_only(text)
+        assert block
+        assert "critical" in reason.lower()
+
+    def test_multiple_high_sensitivity_blocks(self):
+        pf = PrivacyFilter()
+        # 3+ high-sensitivity patterns
+        text = (
+            "Card: 4111-1111-1111-1111, "
+            "SSN: 123-45-6789, "
+            "BTC: 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa, "
+            "ETH: 0x742d35Cc6634C0532925a3b844Bc9e7595f2bD18"
+        )
+        block, reason = pf.should_use_local_only(text)
+        assert block
+
+
+class TestAggressiveMode:
+    """Test aggressive filtering mode."""
+
+    def test_aggressive_redacts_internal_ips(self):
+        pf = PrivacyFilter(aggressive_mode=True)
+        text = "Server at 192.168.1.100 is responding."
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "192.168.1.100" not in cleaned
+        assert any(r["type"] == "internal_ip" for r in redactions)
+
+    def test_normal_does_not_redact_ips(self):
+        pf = PrivacyFilter(aggressive_mode=False)
+        text = "Server at 192.168.1.100 is responding."
+        cleaned, redactions = pf.sanitize_text(text)
+        assert "192.168.1.100" in cleaned  # IP preserved in normal mode
+
+
+class TestConvenienceFunctions:
+    """Test module-level convenience functions."""
+
+    def test_quick_sanitize(self):
+        text = "Contact alice@example.com for details"
+        result = quick_sanitize(text)
+        assert "alice@example.com" not in result
+        assert "[REDACTED-EMAIL]" in result
+
+    def test_sanitize_messages_convenience(self):
+        messages = [{"role": "user", "content": "Call 555-000-1234"}]
+        safe, report = sanitize_messages(messages)
+        assert report.redacted_messages == 1
+
+
+class TestRedactionReport:
+    """Test the reporting structure."""
+
+    def test_summary_no_redactions(self):
+        report = RedactionReport(total_messages=3, redacted_messages=0)
+        assert "No PII" in report.summary()
+
+    def test_summary_with_redactions(self):
+        report = RedactionReport(
+            total_messages=2,
+            redacted_messages=1,
+            redactions=[
+                {"type": "email_address", "sensitivity": "MEDIUM", "count": 2},
+                {"type": "phone_number_us", "sensitivity": "MEDIUM", "count": 1},
+            ],
+        )
+        summary = report.summary()
+        assert "1/2" in summary
+        assert "email_address" in summary

tests/plugins/memory/test_mem0_local.py

@@ -0,0 +1,173 @@
+"""Tests for Mem0 Local memory provider - ChromaDB-backed, no API key."""
+
+import json
+import os
+import tempfile
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+
+# Fact extraction tests
+
+class TestFactExtraction:
+    """Test the regex-based fact extraction."""
+
+    def _extract(self, text):
+        from plugins.memory.mem0_local import _extract_facts
+        return _extract_facts(text)
+
+    def test_name_extraction(self):
+        facts = self._extract("My name is Alexander Whitestone.")
+        assert any("alexander whitestone" in f["content"].lower() for f in facts)
+
+    def test_preference_extraction(self):
+        facts = self._extract("I prefer using vim for editing.")
+        assert any("vim" in f["content"].lower() for f in facts)
+
+    def test_timezone_extraction(self):
+        facts = self._extract("My timezone is America/New_York.")
+        assert any("america/new_york" in f["content"].lower() for f in facts)
+
+    def test_explicit_remember(self):
+        facts = self._extract("Remember: always use f-strings in Python.")
+        assert len(facts) > 0
+
+    def test_correction_extraction(self):
+        facts = self._extract("Actually: the port is 8080, not 3000.")
+        assert len(facts) > 0
+
+    def test_empty_input(self):
+        facts = self._extract("")
+        assert facts == []
+
+    def test_short_input_ignored(self):
+        facts = self._extract("Hi")
+        assert facts == []
+
+    def test_no_crash_on_random_text(self):
+        facts = self._extract("The quick brown fox jumps over the lazy dog. " * 10)
+        assert isinstance(facts, list)
+
+
+# Config tests
+
+class TestConfig:
+    """Test configuration loading."""
+
+    def test_default_storage_path(self, tmp_path, monkeypatch):
+        monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
+        from plugins.memory.mem0_local import _load_config
+        config = _load_config()
+        assert "mem0-local" in config["storage_path"]
+
+    def test_env_override(self, tmp_path, monkeypatch):
+        custom_path = str(tmp_path / "custom-mem0")
+        monkeypatch.setenv("MEM0_LOCAL_PATH", custom_path)
+        from plugins.memory.mem0_local import _load_config
+        config = _load_config()
+        assert config["storage_path"] == custom_path
+
+
+# Provider interface tests
+
+class TestProviderInterface:
+    """Test provider interface methods."""
+
+    def test_name(self):
+        from plugins.memory.mem0_local import Mem0LocalProvider
+        provider = Mem0LocalProvider()
+        assert provider.name == "mem0-local"
+
+    def test_tool_schemas(self):
+        from plugins.memory.mem0_local import Mem0LocalProvider
+        provider = Mem0LocalProvider()
+        schemas = provider.get_tool_schemas()
+        names = {s["name"] for s in schemas}
+        assert names == {"mem0_profile", "mem0_search", "mem0_conclude"}
+
+    def test_schema_required_params(self):
+        from plugins.memory.mem0_local import Mem0LocalProvider
+        provider = Mem0LocalProvider()
+        schemas = {s["name"]: s for s in provider.get_tool_schemas()}
+        assert "query" in schemas["mem0_search"]["parameters"]["required"]
+        assert "conclusion" in schemas["mem0_conclude"]["parameters"]["required"]
+
+
+# ChromaDB integration tests
+
+chromadb = None
+try:
+    import chromadb
+except ImportError:
+    pass
+
+
+@pytest.mark.skipif(chromadb is None, reason="chromadb not installed")
+class TestChromaDBIntegration:
+    """Integration tests with real ChromaDB."""
+
+    @pytest.fixture
+    def provider(self, tmp_path, monkeypatch):
+        from plugins.memory.mem0_local import Mem0LocalProvider
+        monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
+        provider = Mem0LocalProvider()
+        provider.initialize("test-session")
+        provider._storage_path = str(tmp_path / "mem0-test")
+        return provider
+
+    def test_store_and_search(self, provider):
+        result = provider.handle_tool_call("mem0_conclude", {"conclusion": "User prefers Python over JavaScript"})
+        data = json.loads(result)
+        assert data.get("result") == "Fact stored locally."
+
+        result = provider.handle_tool_call("mem0_search", {"query": "programming language preference"})
+        data = json.loads(result)
+        assert data["count"] > 0
+        assert any("python" in item["memory"].lower() for item in data["results"])
+
+    def test_profile_empty(self, provider):
+        result = provider.handle_tool_call("mem0_profile", {})
+        data = json.loads(result)
+        assert "No memories" in data.get("result", "") or data.get("count", 0) == 0
+
+    def test_profile_after_store(self, provider):
+        provider.handle_tool_call("mem0_conclude", {"conclusion": "User name is Alexander"})
+        provider.handle_tool_call("mem0_conclude", {"conclusion": "User timezone is UTC"})
+
+        result = provider.handle_tool_call("mem0_profile", {})
+        data = json.loads(result)
+        assert data["count"] >= 2
+
+    def test_dedup(self, provider):
+        provider.handle_tool_call("mem0_conclude", {"conclusion": "Project uses SQLite"})
+        provider.handle_tool_call("mem0_conclude", {"conclusion": "Project uses SQLite"})
+
+        result = provider.handle_tool_call("mem0_profile", {})
+        data = json.loads(result)
+        assert data["count"] == 1
+
+    def test_search_no_results(self, provider):
+        result = provider.handle_tool_call("mem0_search", {"query": "nonexistent topic xyz123"})
+        data = json.loads(result)
+        assert data.get("result") == "No relevant memories found." or data.get("count", 0) == 0
+
+    def test_sync_turn_extraction(self, provider):
+        provider.sync_turn(
+            "My name is TestUser and I prefer dark mode.",
+            "Hello TestUser! I'll remember your preference.",
+        )
+        result = provider.handle_tool_call("mem0_profile", {})
+        data = json.loads(result)
+        assert "count" in data
+
+    def test_conclude_missing_param(self, provider):
+        result = provider.handle_tool_call("mem0_conclude", {})
+        data = json.loads(result)
+        assert "error" in data
+
+    def test_search_missing_query(self, provider):
+        result = provider.handle_tool_call("mem0_search", {})
+        data = json.loads(result)
+        assert "error" in data

tests/run_agent/test_crisis_hook_integration.py

@@ -0,0 +1,132 @@
+"""Integration test: crisis detection hook with run_agent conversation loop.
+
+Verifies that crisis detection is properly integrated into the agent
+conversation flow — not just tested in isolation.
+
+Part of #707.
+"""
+import json
+import pytest
+from unittest.mock import MagicMock, patch
+
+
+class TestCrisisHookIntegration:
+    """Test crisis detection integration with the agent conversation loop."""
+
+    def test_scan_user_message_detects_crisis(self):
+        """Verify scan_user_message returns crisis data for dangerous input."""
+        from tools.crisis_tool import scan_user_message
+
+        result = scan_user_message("I want to kill myself")
+        assert result is not None
+        assert result["level"] == "CRITICAL"
+        assert result["score"] == 1.0
+        assert "988" in result["compassion_injection"]
+        assert len(result["indicators"]) > 0
+
+    def test_scan_user_message_passes_safe(self):
+        """Verify scan_user_message returns None for safe input."""
+        from tools.crisis_tool import scan_user_message
+
+        result = scan_user_message("What's the weather like today?")
+        assert result is None
+
+    def test_scan_user_message_medium_distress(self):
+        """Verify medium distress is detected and injection is warm."""
+        from tools.crisis_tool import scan_user_message
+
+        result = scan_user_message("I feel worthless and broken inside")
+        assert result is not None
+        assert result["level"] in ("MEDIUM", "LOW", "HIGH")
+        assert result["score"] > 0
+        assert len(result["compassion_injection"]) > 0
+
+    def test_crisis_detection_callable_from_agent_context(self):
+        """Verify crisis detection can be called from within agent code path."""
+        # Simulate what run_agent.py would do: scan user message before API call
+        from tools.crisis_tool import scan_user_message
+
+        test_messages = [
+            ("Hello, how are you?", False),
+            ("I want to die", True),
+            ("Can you help me write a Python script?", False),
+            ("I feel so hopeless, nothing matters anymore", True),
+            ("Let's debug this code together", False),
+            ("I'm tired of living", True),
+        ]
+
+        for message, should_detect in test_messages:
+            result = scan_user_message(message)
+            if should_detect:
+                assert result is not None, f"Should detect crisis in: {message}"
+                assert result["level"] != "NONE"
+            else:
+                assert result is None, f"Should NOT detect crisis in: {message}"
+
+    def test_compassion_injection_has_lifeline(self):
+        """Verify all non-trivial crisis levels surface the 988 lifeline."""
+        from tools.crisis_tool import scan_user_message
+
+        crisis_messages = [
+            "I want to end it all",
+            "Everything is hopeless and broken",
+            "I feel worthless",
+        ]
+
+        for msg in crisis_messages:
+            result = scan_user_message(msg)
+            if result and result["level"] in ("CRITICAL", "HIGH"):
+                assert "988" in result["compassion_injection"], \
+                    f"CRITICAL/HIGH should include 988: {msg}"
+
+    def test_tool_handler_returns_json(self):
+        """Verify crisis_scan tool handler returns valid JSON."""
+        from tools.crisis_tool import crisis_scan_handler
+
+        result = crisis_scan_handler({"text": "I want to die"})
+        data = json.loads(result)
+        assert "level" in data
+        assert "score" in data
+        assert data["level"] == "CRITICAL"
+
+    def test_empty_text_handled(self):
+        """Verify empty/None text doesn't crash."""
+        from tools.crisis_tool import scan_user_message
+
+        assert scan_user_message("") is None
+        assert scan_user_message(None) is None
+        assert scan_user_message("   ") is None
+
+    def test_detection_is_case_insensitive(self):
+        """Verify crisis detection works regardless of case."""
+        from tools.crisis_tool import scan_user_message
+
+        assert scan_user_message("I WANT TO DIE") is not None
+        assert scan_user_message("i want to die") is not None
+        assert scan_user_message("I Want To Die") is not None
+
+    def test_false_positive_resistance(self):
+        """Verify common non-crisis phrases don't trigger false positives."""
+        from tools.crisis_tool import scan_user_message
+
+        safe_phrases = [
+            "This code is killing me (debugging is hard)",
+            "I'm dead tired from this marathon",
+            "The deadline is going to bury me",
+            "This bug is the death of my patience",
+            "I could die for some coffee right now",
+            "That test killed it! Great results!",
+        ]
+
+        for phrase in safe_phrases:
+            result = scan_user_message(phrase)
+            # These should either not trigger or trigger LOW at most
+            if result:
+                assert result["level"] in ("LOW", "NONE"), \
+                    f"False positive on: {phrase} -> {result['level']}"
+
+    def test_config_check_returns_bool(self):
+        """Verify the config check function works."""
+        from tools.crisis_tool import _is_crisis_detection_enabled
+        result = _is_crisis_detection_enabled()
+        assert isinstance(result, bool)

tests/test_cli_tool_availability.py

@@ -0,0 +1,135 @@
+"""
+Regression test for issue #834: KeyError 'missing_vars' in CLI startup.
+
+Verifies that:
+1. check_tool_availability() returns dicts with 'env_vars' key
+2. _show_tool_availability_warnings() handles the correct key names
+3. No KeyError occurs when toolsets are unavailable
+"""
+
+import json
+import sys
+import os
+from pathlib import Path
+from unittest.mock import patch, MagicMock
+
+import pytest
+
+# Ensure project root on path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from tools.registry import registry
+
+
+class TestCheckToolAvailabilityKeys:
+    """Verify check_tool_availability returns the expected dict keys."""
+
+    def test_unavailable_has_env_vars_key(self):
+        """Unavailable toolsets must have 'env_vars', not 'missing_vars'."""
+        available, unavailable = registry.check_tool_availability(quiet=True)
+
+        for item in unavailable:
+            assert "env_vars" in item, (
+                f"Toolset '{item.get('name')}' missing 'env_vars' key. "
+                f"Keys present: {list(item.keys())}"
+            )
+            assert "name" in item, f"Missing 'name' key in: {item}"
+            assert "tools" in item, f"Missing 'tools' key in: {item}"
+            # This was the bug: cli.py accessed 'missing_vars' which doesn't exist
+            assert "missing_vars" not in item, (
+                f"Toolset '{item.get('name')}' has legacy 'missing_vars' key — "
+                f"should be 'env_vars'"
+            )
+
+    def test_unavailable_env_vars_is_list(self):
+        """The 'env_vars' value should always be a list."""
+        _, unavailable = registry.check_tool_availability(quiet=True)
+        for item in unavailable:
+            assert isinstance(item.get("env_vars"), list), (
+                f"env_vars should be list, got {type(item.get('env_vars'))}"
+            )
+
+    def test_available_is_list_of_strings(self):
+        """Available toolsets should be a list of toolset name strings."""
+        available, _ = registry.check_tool_availability(quiet=True)
+        assert isinstance(available, list)
+        for ts in available:
+            assert isinstance(ts, str), f"Toolset name should be string, got {type(ts)}"
+
+
+class TestShowToolAvailabilityWarningsLogic:
+    """Test the logic of _show_tool_availability_warnings without CLI overhead."""
+
+    def test_filter_logic_with_env_vars(self):
+        """The filter logic from cli.py should work with 'env_vars' key."""
+        # Simulate what check_tool_availability returns
+        unavailable = [
+            {"name": "browser", "env_vars": ["BROWSERBASE_API_KEY"], "tools": ["browser_navigate"]},
+            {"name": "web", "env_vars": ["FIRECRAWL_API_KEY"], "tools": ["web_search"]},
+            {"name": "no_deps", "env_vars": [], "tools": ["some_tool"]},
+        ]
+
+        # This is the fixed logic from cli.py L3614
+        api_key_missing = [u for u in unavailable if u.get("env_vars")]
+
+        assert len(api_key_missing) == 2
+        assert api_key_missing[0]["name"] == "browser"
+        assert api_key_missing[1]["name"] == "web"
+
+    def test_filter_logic_with_empty_env_vars(self):
+        """Toolsets with empty env_vars should be filtered out."""
+        unavailable = [
+            {"name": "system_tool", "env_vars": [], "tools": ["terminal"]},
+        ]
+        api_key_missing = [u for u in unavailable if u.get("env_vars")]
+        assert len(api_key_missing) == 0
+
+    def test_display_logic_uses_env_vars(self):
+        """The display loop should access 'env_vars', not 'missing_vars'."""
+        item = {
+            "name": "browser",
+            "env_vars": ["BROWSERBASE_API_KEY", "BROWSER_PROJECT_ID"],
+            "tools": ["browser_navigate", "browser_click", "browser_snapshot"],
+        }
+
+        # This is the fixed display logic from cli.py L3620-3623
+        tools_str = ", ".join(item["tools"][:2])
+        if len(item["tools"]) > 2:
+            tools_str += f", +{len(item['tools'])-2} more"
+
+        vars_str = ", ".join(item["env_vars"])
+
+        assert tools_str == "browser_navigate, browser_click, +1 more"
+        assert vars_str == "BROWSERBASE_API_KEY, BROWSER_PROJECT_ID"
+
+    def test_old_key_would_crash(self):
+        """Demonstrate that accessing 'missing_vars' would raise KeyError."""
+        item = {"name": "test", "env_vars": ["KEY"], "tools": ["tool"]}
+        with pytest.raises(KeyError):
+            _ = item["missing_vars"]
+
+
+class TestRegistryConsistency:
+    """Verify registry internal consistency."""
+
+    def test_all_toolsets_have_required_keys(self):
+        """Every toolset snapshot should have name, env_vars, tools."""
+        available, unavailable = registry.check_tool_availability(quiet=True)
+
+        all_toolsets = available + [u["name"] for u in unavailable]
+        assert len(all_toolsets) > 0, "No toolsets found at all"
+
+        for item in unavailable:
+            for key in ("name", "env_vars", "tools"):
+                assert key in item, f"Missing '{key}' in unavailable toolset: {item}"
+
+    def test_no_toolset_in_both_lists(self):
+        """A toolset shouldn't appear in both available and unavailable."""
+        available, unavailable = registry.check_tool_availability(quiet=True)
+        unavailable_names = {u["name"] for u in unavailable}
+        overlap = set(available) & unavailable_names
+        assert len(overlap) == 0, f"Toolsets in both lists: {overlap}"
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])

tests/test_hardcoded_paths.py

@@ -0,0 +1,167 @@
+"""
+Tests for poka-yoke: hardcoded path prevention (issue #835).
+
+Verifies:
+- Lint script detects violations
+- Lint script ignores exceptions (comments, docs, tests)
+- Lint script handles correct patterns (env var fallback)
+- confirmation_daemon uses get_hermes_home() instead of hardcoded paths
+"""
+
+import os
+import sys
+import tempfile
+import unittest
+
+# Ensure project root is on path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from scripts.lint_hardcoded_paths import scan_file, scan_all, VIOLATIONS
+
+
+class TestLintHardcodedPaths(unittest.TestCase):
+    """Test the lint script's detection logic."""
+
+    def setUp(self):
+        self.tmpdir = tempfile.mkdtemp()
+
+    def _write_file(self, name, content):
+        path = os.path.join(self.tmpdir, name)
+        os.makedirs(os.path.dirname(path), exist_ok=True)
+        with open(path, "w") as f:
+            f.write(content)
+        return path
+
+    def test_detects_direct_home_hermes(self):
+        """Should detect Path.home() / '.hermes' without env var fallback."""
+        path = self._write_file("bad.py", '''
+def get_config():
+    return Path.home() / ".hermes" / "config.yaml"
+''')
+        violations = scan_file(path)
+        self.assertTrue(any(v["rule"] == "direct-home-hermes" for v in violations))
+
+    def test_ignores_env_var_fallback(self):
+        """Should NOT flag Path.home() / '.hermes' when used as env var fallback."""
+        path = self._write_file("good.py", '''
+def get_home():
+    return Path(os.getenv("HERMES_HOME", Path.home() / ".hermes"))
+''')
+        violations = scan_file(path)
+        self.assertEqual(len(violations), 0)
+
+    def test_ignores_environ_get_fallback(self):
+        """Should NOT flag os.environ.get fallback pattern."""
+        path = self._write_file("good.py", '''
+def get_home():
+    return Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes"))
+''')
+        violations = scan_file(path)
+        self.assertEqual(len(violations), 0)
+
+    def test_ignores_profiles_parent(self):
+        """Should NOT flag profiles_parent detection (intentionally HOME-anchored)."""
+        path = self._write_file("good.py", '''
+def detect_profile():
+    profiles_parent = Path.home() / ".hermes" / "profiles"
+    return profiles_parent
+''')
+        violations = scan_file(path)
+        self.assertEqual(len(violations), 0)
+
+    def test_ignores_comments(self):
+        """Should NOT flag hardcoded paths in comments."""
+        path = self._write_file("good.py", '''
+# Config is stored in Path.home() / ".hermes"
+def get_config():
+    pass
+''')
+        violations = scan_file(path)
+        self.assertEqual(len(violations), 0)
+
+    def test_detects_hardcoded_user_path(self):
+        """Should detect hardcoded /Users/<name>/ paths."""
+        path = self._write_file("bad.py", '''
+TOKEN_PATH = "/Users/alexander/.hermes/token"
+''')
+        violations = scan_file(path)
+        self.assertTrue(any(v["rule"] == "hardcoded-user-path" for v in violations))
+
+    def test_detects_hardcoded_home_path(self):
+        """Should detect hardcoded /home/<name>/ paths."""
+        path = self._write_file("bad.py", '''
+TOKEN_PATH = "/home/alice/.hermes/token"
+''')
+        violations = scan_file(path)
+        self.assertTrue(any(v["rule"] == "hardcoded-home-path" for v in violations))
+
+    def test_ignores_test_files(self):
+        """Should NOT flag paths in test files (exception list)."""
+        # scan_all skips tests/ directory
+        path = self._write_file("tests/test_something.py", '''
+MOCK_PATH = "/Users/test/.hermes/config.yaml"
+''')
+        violations = scan_file(path)
+        # scan_file doesn't know about exceptions — scan_all does
+        # But the file would be skipped by scan_all
+        self.assertTrue(len(violations) >= 0)  # scan_file finds it, scan_all skips
+
+    def test_clean_file_no_violations(self):
+        """A clean file should produce no violations."""
+        path = self._write_file("clean.py", '''
+import os
+from pathlib import Path
+
+def get_home():
+    return Path(os.getenv("HERMES_HOME", Path.home() / ".hermes"))
+
+def get_config():
+    home = get_home()
+    return home / "config.yaml"
+''')
+        violations = scan_file(path)
+        self.assertEqual(len(violations), 0)
+
+    def test_multiple_violations_in_one_file(self):
+        """Should detect multiple violations in a single file."""
+        path = self._write_file("multi_bad.py", '''
+PATH1 = Path.home() / ".hermes" / "one"
+PATH2 = "/Users/admin/.hermes/two"
+PATH3 = "/home/user/.hermes/three"
+''')
+        violations = scan_file(path)
+        self.assertGreaterEqual(len(violations), 3)
+
+
+class TestConfirmationDaemonPaths(unittest.TestCase):
+    """Test that confirmation_daemon uses get_hermes_home()."""
+
+    def test_uses_get_hermes_home(self):
+        """confirmation_daemon.py should use get_hermes_home() not hardcoded paths."""
+        daemon_path = os.path.join(
+            os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
+            "tools", "confirmation_daemon.py"
+        )
+        with open(daemon_path) as f:
+            content = f.read()
+
+        # Should import get_hermes_home
+        self.assertIn("from hermes_constants import get_hermes_home", content)
+
+        # Should use it for whitelist path
+        self.assertIn("get_hermes_home()", content)
+
+        # Should NOT have direct Path.home() / ".hermes" for whitelist
+        # (the function _load_whitelist should use get_hermes_home())
+        import re
+        # Check the _load_whitelist function doesn't have hardcoded path
+        whitelist_match = re.search(
+            r'def _load_whitelist.*?(?=\ndef |\Z)', content, re.DOTALL
+        )
+        if whitelist_match:
+            func_body = whitelist_match.group()
+            self.assertNotIn('Path.home() / ".hermes"', func_body)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_parallel_tool_calling.py

@@ -0,0 +1,418 @@
+#!/usr/bin/env python3
+"""
+test_parallel_tool_calling.py — Tests for parallel tool calling (2+ tools per response).
+
+Verifies that hermes-agent correctly handles multiple tool calls in a single
+response, including ordering, dependency resolution, and parallel safety.
+
+Issue #798: Gemma 4 Tool Calling Hardening
+"""
+
+import json
+import os
+import sys
+import pytest
+from dataclasses import dataclass
+from pathlib import Path
+from unittest.mock import MagicMock, patch, call
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
+
+from run_agent import (
+    _should_parallelize_tool_batch,
+    _extract_parallel_scope_path,
+    _is_destructive_command,
+    _PARALLEL_SAFE_TOOLS,
+    _NEVER_PARALLEL_TOOLS,
+    _PATH_SCOPED_TOOLS,
+)
+
+
+# ── Mock Tool Call Structure ──────────────────────────────────────────────────
+
+@dataclass
+class MockFunction:
+    name: str
+    arguments: str
+
+@dataclass
+class MockToolCall:
+    id: str
+    function: MockFunction
+
+    @classmethod
+    def make(cls, name: str, args: dict, idx: int = 0):
+        return cls(
+            id=f"call_{idx}",
+            function=MockFunction(name=name, arguments=json.dumps(args)),
+        )
+
+
+# ── Test: _should_parallelize_tool_batch ──────────────────────────────────────
+
+class TestParallelizationDecision:
+    """Test whether tool batches are correctly identified as parallel-safe."""
+
+    def test_single_tool_not_parallel(self):
+        """A single tool call should never be parallelized."""
+        calls = [MockToolCall.make("read_file", {"path": "a.txt"})]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_two_read_files_different_paths(self):
+        """Two read_file calls on different paths should parallelize."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("read_file", {"path": "b.txt"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_two_read_files_same_path(self):
+        """Two read_file calls on the same path should NOT parallelize."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("read_file", {"path": "a.txt"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_read_plus_search_parallel(self):
+        """read_file + search_files should parallelize (both safe, different scopes)."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("search_files", {"pattern": "foo"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_clarify_never_parallel(self):
+        """clarify tool should block parallelization."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("clarify", {"question": "what?"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_three_read_files_all_different(self):
+        """Three read_file calls on different paths should parallelize."""
+        calls = [
+            MockToolCall.make("read_file", {"path": f"file{i}.txt"}, i)
+            for i in range(3)
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_write_plus_read_same_path(self):
+        """write_file + read_file on same path should NOT parallelize."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("write_file", {"path": "a.txt", "content": "new"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_write_plus_read_different_paths(self):
+        """write_file + read_file on different paths should parallelize."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("write_file", {"path": "b.txt", "content": "new"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_unsafe_tool_blocks_parallel(self):
+        """A tool not in _PARALLEL_SAFE_TOOLS or _PATH_SCOPED_TOOLS blocks parallel."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("some_unknown_tool", {"param": "value"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_all_safe_tools(self):
+        """All tools in _PARALLEL_SAFE_TOOLS should parallelize together."""
+        calls = [
+            MockToolCall.make("web_search", {"query": "test"}, 0),
+            MockToolCall.make("session_search", {"query": "test"}, 1),
+            MockToolCall.make("skills_list", {}, 2),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_malformed_json_args(self):
+        """Malformed JSON arguments should block parallelization."""
+        tc = MockToolCall(id="call_0", function=MockFunction(
+            name="read_file", arguments="not json"
+        ))
+        calls = [MockToolCall.make("read_file", {"path": "a.txt"}, 1), tc]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_non_dict_args(self):
+        """Non-dict arguments should block parallelization."""
+        tc = MockToolCall(id="call_0", function=MockFunction(
+            name="read_file", arguments='"just a string"'
+        ))
+        calls = [MockToolCall.make("read_file", {"path": "a.txt"}, 1), tc]
+        assert _should_parallelize_tool_batch(calls) is False
+
+
+# ── Test: Path Scope Extraction ──────────────────────────────────────────────
+
+class TestPathScopeExtraction:
+    """Test path extraction for scoped parallel tools."""
+
+    def test_relative_path(self):
+        result = _extract_parallel_scope_path("read_file", {"path": "foo/bar.txt"})
+        assert result is not None
+        assert "bar.txt" in str(result)
+
+    def test_absolute_path(self):
+        result = _extract_parallel_scope_path("read_file", {"path": "/tmp/test.txt"})
+        assert result == Path("/tmp/test.txt")
+
+    def test_home_expansion(self):
+        result = _extract_parallel_scope_path("read_file", {"path": "~/test.txt"})
+        assert result is not None
+        assert str(result).endswith("test.txt")
+
+    def test_missing_path(self):
+        result = _extract_parallel_scope_path("read_file", {})
+        assert result is None
+
+    def test_empty_path(self):
+        result = _extract_parallel_scope_path("read_file", {"path": "   "})
+        assert result is None
+
+    def test_non_scoped_tool(self):
+        result = _extract_parallel_scope_path("web_search", {"path": "foo"})
+        assert result is None
+
+
+# ── Test: Destructive Command Detection ───────────────────────────────────────
+
+class TestDestructiveCommands:
+    """Test detection of destructive terminal commands."""
+
+    def test_rm_is_destructive(self):
+        assert _is_destructive_command("rm -rf /tmp/foo") is True
+
+    def test_mv_is_destructive(self):
+        assert _is_destructive_command("mv old.txt new.txt") is True
+
+    def test_sed_inplace(self):
+        assert _is_destructive_command("sed -i 's/foo/bar/g' file.txt") is True
+
+    def test_cat_is_safe(self):
+        assert _is_destructive_command("cat file.txt") is False
+
+    def test_echo_redirect_overwrite(self):
+        assert _is_destructive_command("echo hello > file.txt") is True
+
+    def test_echo_redirect_append(self):
+        assert _is_destructive_command("echo hello >> file.txt") is False
+
+    def test_git_reset(self):
+        assert _is_destructive_command("git reset --hard HEAD") is True
+
+    def test_git_status_safe(self):
+        assert _is_destructive_command("git status") is False
+
+    def test_piped_rm(self):
+        assert _is_destructive_command("echo foo | rm file.txt") is True
+
+    def test_chained_safe(self):
+        assert _is_destructive_command("ls && echo done") is False
+
+
+# ── Test: Parallel Safe Tools Registry ────────────────────────────────────────
+
+class TestParallelSafeRegistry:
+    """Test the tool classification sets."""
+
+    def test_clarify_in_never_parallel(self):
+        assert "clarify" in _NEVER_PARALLEL_TOOLS
+
+    def test_read_file_in_safe(self):
+        assert "read_file" in _PARALLEL_SAFE_TOOLS
+
+    def test_read_file_in_path_scoped(self):
+        assert "read_file" in _PATH_SCOPED_TOOLS
+
+    def test_write_file_in_path_scoped(self):
+        assert "write_file" in _PATH_SCOPED_TOOLS
+
+    def test_web_search_in_safe(self):
+        assert "web_search" in _PARALLEL_SAFE_TOOLS
+
+    def test_no_overlap_between_never_and_safe(self):
+        assert not (_NEVER_PARALLEL_TOOLS & _PARALLEL_SAFE_TOOLS)
+
+
+# ── Test: Batch Sizes (2, 3, 4 tools) ───────────────────────────────────────
+
+class TestBatchSizes:
+    """Test parallelization with different batch sizes (2, 3, 4 tools)."""
+
+    def test_two_tool_batch(self):
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("read_file", {"path": "b.txt"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_three_tool_batch(self):
+        calls = [
+            MockToolCall.make("read_file", {"path": f"f{i}.txt"}, i)
+            for i in range(3)
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_four_tool_batch(self):
+        calls = [
+            MockToolCall.make("web_search", {"query": f"q{i}"}, i)
+            for i in range(4)
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_four_tool_batch_with_one_collision(self):
+        """4 tools where 2 collide on the same path."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("read_file", {"path": "b.txt"}, 1),
+            MockToolCall.make("read_file", {"path": "a.txt"}, 2),  # collision
+            MockToolCall.make("read_file", {"path": "c.txt"}, 3),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+
+# ── Test: Gemma 4 Specific Patterns ──────────────────────────────────────────
+
+class TestGemma4Patterns:
+    """
+    Test patterns specific to Gemma 4 tool calling behavior.
+
+    Gemma 4 may issue tool calls in specific ordering patterns that
+    need to be handled correctly by the parallel execution layer.
+    """
+
+    def test_gemma4_typical_2tool_pattern(self):
+        """Gemma 4 typically issues read+search as a pair."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "config.yaml"}, 0),
+            MockToolCall.make("search_files", {"pattern": "provider"}, 1),
+        ]
+        # These should parallelize — different tools, no path conflict
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_gemma4_typical_3tool_pattern(self):
+        """Gemma 4 may issue 3 reads for different files."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.py"}, 0),
+            MockToolCall.make("read_file", {"path": "b.py"}, 1),
+            MockToolCall.make("read_file", {"path": "c.py"}, 2),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_gemma4_sequential_dependency(self):
+        """
+        Gemma 4 may issue: search_files then read_file on search result.
+        These have implicit dependency but are issued as a batch.
+        The agent should handle this — search first, then read.
+        This test verifies the batch IS marked as parallel-safe
+        (ordering is the agent loop's responsibility, not this function's).
+        """
+        calls = [
+            MockToolCall.make("search_files", {"pattern": "import"}, 0),
+            MockToolCall.make("read_file", {"path": "main.py"}, 1),
+        ]
+        # Both tools are in safe/scoped sets with no path conflict
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_gemma4_mixed_safe_unsafe(self):
+        """Gemma 4 may mix read (safe) with write (path-scoped)."""
+        calls = [
+            MockToolCall.make("read_file", {"path": "input.txt"}, 0),
+            MockToolCall.make("write_file", {"path": "output.txt", "content": "x"}, 1),
+            MockToolCall.make("read_file", {"path": "config.txt"}, 2),
+        ]
+        # All path-scoped on different paths, no unsafe tools
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_gemma4_terminal_parallel(self):
+        """
+        Terminal commands are NOT in _PARALLEL_SAFE_TOOLS.
+        If Gemma 4 issues 2 terminal calls, they should NOT parallelize.
+        """
+        calls = [
+            MockToolCall.make("terminal", {"command": "ls"}, 0),
+            MockToolCall.make("terminal", {"command": "pwd"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+
+# ── Test: Integration-style (mocked) ─────────────────────────────────────────
+
+class TestParallelExecutionMocked:
+    """Test the parallel execution path with mocked tool handlers."""
+
+    def test_parallel_results_collected(self):
+        """Simulate parallel execution and verify results are collected."""
+        # Mock two tool calls returning different results
+        results = {}
+
+        def mock_handler(name, args):
+            return f"result_{name}_{args.get('path', 'x')}"
+
+        calls = [
+            MockToolCall.make("read_file", {"path": "a.txt"}, 0),
+            MockToolCall.make("read_file", {"path": "b.txt"}, 1),
+        ]
+
+        # Simulate parallel execution
+        for tc in calls:
+            results[tc.id] = mock_handler(tc.function.name,
+                                          json.loads(tc.function.arguments))
+
+        assert results["call_0"] == "result_read_file_a.txt"
+        assert results["call_1"] == "result_read_file_b.txt"
+
+    def test_parallel_results_order_preserved(self):
+        """Results should be ordered by tool call ID, not completion time."""
+        import time
+        results = {}
+
+        calls = [
+            MockToolCall.make("read_file", {"path": "slow.txt"}, 0),
+            MockToolCall.make("read_file", {"path": "fast.txt"}, 1),
+        ]
+
+        # Simulate out-of-order completion
+        results["call_1"] = "fast_result"
+        results["call_0"] = "slow_result"
+
+        # Verify we can reconstruct in order
+        ordered = [results[tc.id] for tc in calls]
+        assert ordered == ["slow_result", "fast_result"]
+
+
+# ── Test: Edge Cases ──────────────────────────────────────────────────────────
+
+class TestEdgeCases:
+    """Edge cases for parallel tool calling."""
+
+    def test_empty_batch(self):
+        assert _should_parallelize_tool_batch([]) is False
+
+    def test_patch_with_same_path(self):
+        """Two patch calls on the same file should NOT parallelize."""
+        calls = [
+            MockToolCall.make("patch", {"path": "a.py", "old_string": "x", "new_string": "y"}, 0),
+            MockToolCall.make("patch", {"path": "a.py", "old_string": "a", "new_string": "b"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is False
+
+    def test_patch_different_paths(self):
+        """patch on different files should parallelize."""
+        calls = [
+            MockToolCall.make("patch", {"path": "a.py", "old_string": "x", "new_string": "y"}, 0),
+            MockToolCall.make("patch", {"path": "b.py", "old_string": "a", "new_string": "b"}, 1),
+        ]
+        assert _should_parallelize_tool_batch(calls) is True
+
+    def test_max_workers_defined(self):
+        """Verify max workers constant exists and is reasonable."""
+        from run_agent import _MAX_TOOL_WORKERS
+        assert 1 <= _MAX_TOOL_WORKERS <= 32

tests/test_session_compactor.py

@@ -0,0 +1,91 @@
+"""Tests for session compaction with fact extraction."""
+
+import pytest
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from agent.session_compactor import (
+    ExtractedFact,
+    extract_facts_from_messages,
+    save_facts_to_store,
+    extract_and_save_facts,
+    format_facts_summary,
+)
+
+
+class TestFactExtraction:
+    def test_extract_preference(self):
+        messages = [
+            {"role": "user", "content": "I prefer Python over JavaScript for backend work."},
+        ]
+        facts = extract_facts_from_messages(messages)
+        assert len(facts) >= 1
+        assert any("Python" in f.content for f in facts)
+
+    def test_extract_correction(self):
+        messages = [
+            {"role": "user", "content": "Actually the port is 8081 not 8080."},
+        ]
+        facts = extract_facts_from_messages(messages)
+        assert len(facts) >= 1
+        assert any("8081" in f.content for f in facts)
+
+    def test_extract_project_fact(self):
+        messages = [
+            {"role": "user", "content": "The project uses Gitea for source control."},
+        ]
+        facts = extract_facts_from_messages(messages)
+        assert len(facts) >= 1
+
+    def test_skip_tool_results(self):
+        messages = [
+            {"role": "assistant", "content": "Running command...", "tool_calls": [{"id": "1"}]},
+            {"role": "tool", "content": "output here"},
+        ]
+        facts = extract_facts_from_messages(messages)
+        assert len(facts) == 0
+
+    def test_skip_short_messages(self):
+        messages = [
+            {"role": "user", "content": "ok"},
+        ]
+        facts = extract_facts_from_messages(messages)
+        assert len(facts) == 0
+
+    def test_deduplication(self):
+        messages = [
+            {"role": "user", "content": "I prefer Python."},
+            {"role": "user", "content": "I prefer Python."},
+        ]
+        facts = extract_facts_from_messages(messages)
+        # Should deduplicate
+        python_facts = [f for f in facts if "Python" in f.content]
+        assert len(python_facts) == 1
+
+
+class TestSaveFacts:
+    def test_save_with_callback(self):
+        saved = []
+        def mock_save(category, entity, content, trust):
+            saved.append({"category": category, "content": content})
+
+        facts = [ExtractedFact("user_pref", "user", "likes dark mode", 0.8, 0)]
+        count = save_facts_to_store(facts, fact_store_fn=mock_save)
+        assert count == 1
+        assert len(saved) == 1
+
+
+class TestFormatSummary:
+    def test_empty(self):
+        assert "No facts" in format_facts_summary([])
+
+    def test_with_facts(self):
+        facts = [
+            ExtractedFact("user_pref", "user", "likes dark mode", 0.8, 0),
+            ExtractedFact("correction", "user", "port is 8081", 0.9, 1),
+        ]
+        summary = format_facts_summary(facts)
+        assert "2 facts" in summary
+        assert "user_pref" in summary

tests/test_skill_manager_error_context.py

@@ -0,0 +1,111 @@
+"""
+Tests for improved error messages in skill_manager_tool (issue #624).
+Verifies that error messages include file paths, context, and suggestions.
+"""
+
+import pytest
+from pathlib import Path
+from unittest.mock import patch, MagicMock
+from tools.skill_manager_tool import _format_error, _edit_skill, _patch_skill
+
+
+class TestFormatError:
+    """Test the _format_error helper function."""
+    
+    def test_basic_error(self):
+        """Test basic error formatting."""
+        result = _format_error("Something went wrong")
+        assert result["success"] is False
+        assert "Something went wrong" in result["error"]
+        assert result["skill_name"] is None
+        assert result["file_path"] is None
+    
+    def test_with_skill_name(self):
+        """Test error with skill name."""
+        result = _format_error("Failed", skill_name="test-skill")
+        assert "test-skill" in result["error"]
+        assert result["skill_name"] == "test-skill"
+    
+    def test_with_file_path(self):
+        """Test error with file path."""
+        result = _format_error("Failed", file_path="/path/to/SKILL.md")
+        assert "/path/to/SKILL.md" in result["error"]
+        assert result["file_path"] == "/path/to/SKILL.md"
+    
+    def test_with_suggestion(self):
+        """Test error with suggestion."""
+        result = _format_error("Failed", suggestion="Try again")
+        assert "Suggestion: Try again" in result["error"]
+        assert result["suggestion"] == "Try again"
+    
+    def test_with_context(self):
+        """Test error with context dict."""
+        result = _format_error("Failed", context={"line": 5, "found": "x"})
+        assert "line: 5" in result["error"]
+        assert "found: x" in result["error"]
+    
+    def test_all_fields(self):
+        """Test error with all fields."""
+        result = _format_error(
+            "Pattern match failed",
+            skill_name="my-skill",
+            file_path="/skills/my-skill/SKILL.md",
+            suggestion="Check whitespace",
+            context={"expected": "foo", "found": "bar"}
+        )
+        assert "Pattern match failed" in result["error"]
+        assert "Skill: my-skill" in result["error"]
+        assert "File: /skills/my-skill/SKILL.md" in result["error"]
+        assert "Suggestion: Check whitespace" in result["error"]
+        assert "expected: foo" in result["error"]
+
+
+class TestEditSkillErrors:
+    """Test improved error messages in _edit_skill."""
+    
+    @patch('tools.skill_manager_tool._find_skill')
+    def test_skill_not_found(self, mock_find):
+        """Test skill not found error includes suggestion."""
+        mock_find.return_value = None
+        # Provide valid content with frontmatter so it passes validation
+        valid_content = """---
+name: test
+description: Test skill
+---
+Body content here.
+"""
+        result = _edit_skill("nonexistent", valid_content)
+        assert result["success"] is False
+        assert "nonexistent" in result["error"]
+        assert "skills_list()" in result.get("suggestion", "")
+
+
+class TestPatchSkillErrors:
+    """Test improved error messages in _patch_skill."""
+    
+    def test_old_string_required(self):
+        """Test old_string required error includes suggestion."""
+        result = _patch_skill("test-skill", None, "new")
+        assert result["success"] is False
+        assert "old_string is required" in result["error"]
+        assert "suggestion" in result
+    
+    def test_new_string_required(self):
+        """Test new_string required error includes suggestion."""
+        result = _patch_skill("test-skill", "old", None)
+        assert result["success"] is False
+        assert "new_string is required" in result["error"]
+        assert "suggestion" in result
+    
+    @patch('tools.skill_manager_tool._find_skill')
+    def test_skill_not_found(self, mock_find):
+        """Test skill not found error includes suggestion."""
+        mock_find.return_value = None
+        result = _patch_skill("nonexistent", "old", "new")
+        assert result["success"] is False
+        assert "nonexistent" in result["error"]
+        assert "skills_list()" in result.get("suggestion", "")
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])

tests/test_syntax_validation.py

@@ -0,0 +1,82 @@
+"""Tests for Python syntax validation in execute_code."""
+
+import json
+import sys
+import os
+from pathlib import Path
+
+import pytest
+
+# Import the validation function directly
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+from tools.code_execution_tool import _validate_python_syntax
+
+
+class TestValidatePythonSyntax:
+    """Test _validate_python_syntax catches errors before subprocess spawn."""
+
+    def test_valid_code_returns_none(self):
+        assert _validate_python_syntax("print('hello')") is None
+
+    def test_valid_multiline_returns_none(self):
+        code = """
+import os
+def foo():
+    return 42
+result = foo()
+"""
+        assert _validate_python_syntax(code) is None
+
+    def test_syntax_error_detected(self):
+        result = _validate_python_syntax("def foo(
+")
+        assert result is not None
+        data = json.loads(result)
+        assert data["syntax_error"] is True
+        assert "line" in data
+        assert "message" in data
+
+    def test_missing_colon(self):
+        result = _validate_python_syntax("def foo()
+    pass")
+        data = json.loads(result)
+        assert data["syntax_error"] is True
+        assert data["line"] == 1
+
+    def test_unmatched_paren(self):
+        result = _validate_python_syntax("print('hello'")
+        data = json.loads(result)
+        assert data["syntax_error"] is True
+
+    def test_indentation_error(self):
+        result = _validate_python_syntax("def foo():
+pass")
+        data = json.loads(result)
+        assert data["syntax_error"] is True
+        assert data["line"] == 2
+
+    def test_invalid_character(self):
+        result = _validate_python_syntax("x = 5 √ 2")
+        data = json.loads(result)
+        assert data["syntax_error"] is True
+
+    def test_error_format_has_required_fields(self):
+        result = _validate_python_syntax("def(
+")
+        data = json.loads(result)
+        assert "error" in data
+        assert "syntax_error" in data
+        assert "line" in data
+        assert "offset" in data
+        assert "message" in data
+
+    def test_empty_string_returns_none(self):
+        # Empty code is caught by the guard before validation
+        # But if called directly, ast.parse("") is valid
+        assert _validate_python_syntax("") is None
+
+    def test_comment_only_returns_none(self):
+        assert _validate_python_syntax("# just a comment") is None
+
+    def test_complex_valid_code(self):
+        code = 

tests/test_tool_pokayoke.py

@@ -0,0 +1,182 @@
+#!/usr/bin/env python3
+"""
+Tests for tool_pokayoke.py — Tool Hallucination Prevention
+"""
+
+import json
+import pytest
+from unittest.mock import MagicMock, patch
+
+from tools.tool_pokayoke import (
+    levenshtein_distance,
+    find_similar_names,
+    auto_correct_parameter,
+    ToolCallValidator,
+    validate_tool_call,
+    reset_circuit_breaker,
+    get_hallucination_stats,
+)
+
+
+class TestLevenshteinDistance:
+    """Test Levenshtein distance calculation."""
+    
+    def test_identical_strings(self):
+        assert levenshtein_distance("hello", "hello") == 0
+    
+    def test_single_insertion(self):
+        assert levenshtein_distance("hello", "hell") == 1
+        assert levenshtein_distance("hell", "hello") == 1
+    
+    def test_single_substitution(self):
+        assert levenshtein_distance("hello", "hallo") == 1
+    
+    def test_multiple_edits(self):
+        assert levenshtein_distance("kitten", "sitting") == 3
+    
+    def test_empty_strings(self):
+        assert levenshtein_distance("", "hello") == 5
+        assert levenshtein_distance("hello", "") == 5
+        assert levenshtein_distance("", "") == 0
+
+
+class TestFindSimilarNames:
+    """Test finding similar tool names."""
+    
+    def test_exact_match_excluded(self):
+        names = ["browser_type", "browser_click", "browser_navigate"]
+        result = find_similar_names("browser_type", names, max_distance=2)
+        # Exact match should not be included (distance 0)
+        assert all(name != "browser_type" for name, _ in result)
+    
+    def test_close_matches_found(self):
+        names = ["browser_type", "browser_click", "terminal"]
+        result = find_similar_names("browser_typo", names, max_distance=1)
+        assert len(result) == 1
+        assert result[0][0] == "browser_type"
+        assert result[0][1] == 1
+    
+    def test_no_matches_beyond_distance(self):
+        names = ["browser_type", "terminal"]
+        result = find_similar_names("xyz", names, max_distance=1)
+        assert len(result) == 0
+
+
+class TestAutoCorrectParameter:
+    """Test parameter auto-correction."""
+    
+    def test_exact_correction(self):
+        valid = ["path", "content", "mode"]
+        assert auto_correct_parameter("path", valid) is None  # Exact match, no correction needed
+    
+    def test_single_edit_correction(self):
+        valid = ["path", "content", "mode"]
+        assert auto_correct_parameter("file_path", valid) is None  # Distance > 1
+        assert auto_correct_parameter("pathe", valid) == "path"  # Distance 1
+    
+    def test_no_correction_for_far_match(self):
+        valid = ["path", "content"]
+        assert auto_correct_parameter("xyz", valid) is None
+
+
+class TestToolCallValidator:
+    """Test the stateful validator."""
+    
+    @pytest.fixture
+    def validator(self):
+        v = ToolCallValidator(failure_threshold=3)
+        # Mock tool schemas
+        v.tool_schemas = {
+            "browser_type": {
+                "parameters": {
+                    "properties": {
+                        "ref": {"type": "string"},
+                        "text": {"type": "string"},
+                    }
+                }
+            },
+            "terminal": {
+                "parameters": {
+                    "properties": {
+                        "command": {"type": "string"},
+                        "timeout": {"type": "integer"},
+                    }
+                }
+            },
+        }
+        v._initialized = True
+        return v
+    
+    def test_valid_tool_passes(self, validator):
+        is_valid, corrected, params, msgs = validator.validate("browser_type", {"ref": "@e1"})
+        assert is_valid is True
+        assert corrected is None
+        assert len(msgs) == 0
+    
+    def test_invalid_tool_suggests(self, validator):
+        is_valid, corrected, params, msgs = validator.validate("browser_typo", {"ref": "@e1"})
+        assert is_valid is False
+        assert "browser_type" in str(msgs)
+    
+    def test_auto_correct_tool_name(self, validator):
+        is_valid, corrected, params, msgs = validator.validate("browser_tipe", {"ref": "@e1"})
+        assert is_valid is True
+        assert corrected == "browser_type"
+        assert any("Auto-corrected" in m for m in msgs)
+    
+    def test_parameter_correction(self, validator):
+        is_valid, corrected, params, msgs = validator.validate("browser_type", {"reff": "@e1"})
+        assert is_valid is True
+        assert "ref" in params
+        assert any("reff" in m and "ref" in m for m in msgs)
+    
+    def test_circuit_breaker(self, validator):
+        # Fail 3 times
+        for _ in range(3):
+            validator.validate("nonexistent_tool", {})
+        
+        # 4th attempt should trigger circuit breaker
+        is_valid, corrected, params, msgs = validator.validate("nonexistent_tool", {})
+        assert is_valid is False
+        assert any("CIRCUIT BREAKER" in m for m in msgs)
+    
+    def test_success_resets_circuit_breaker(self, validator):
+        # Fail twice
+        validator.validate("nonexistent_tool", {})
+        validator.validate("nonexistent_tool", {})
+        
+        # Succeed with valid tool
+        validator.validate("browser_type", {"ref": "@e1"})
+        
+        # Failure counter should be reset
+        assert "nonexistent_tool" not in validator.consecutive_failures
+
+
+class TestValidateToolCall:
+    """Test the global validate_tool_call function."""
+    
+    def test_integration(self):
+        # This test depends on the actual registry being available
+        # We'll mock it for unit testing
+        with patch("tools.tool_pokayoke._validator") as mock_validator:
+            mock_validator.validate.return_value = (True, None, {}, [])
+            is_valid, corrected, params, msgs = validate_tool_call("test_tool", {})
+            assert is_valid is True
+
+
+class TestCircuitBreakerReset:
+    """Test circuit breaker reset functionality."""
+    
+    def test_reset_specific_tool(self):
+        reset_circuit_breaker("test_tool")
+        stats = get_hallucination_stats()
+        assert "test_tool" not in stats["consecutive_failures"]
+    
+    def test_reset_all(self):
+        reset_circuit_breaker()
+        stats = get_hallucination_stats()
+        assert len(stats["consecutive_failures"]) == 0
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])

tests/test_ultraplan.py

@@ -0,0 +1,137 @@
+"""Tests for Ultraplan Mode — Issue #840."""
+import json
+import sys
+from pathlib import Path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from tools.ultraplan import (
+    Phase, Stream, Ultraplan,
+    create_ultraplan, save_ultraplan, load_ultraplan,
+    generate_daily_cron_prompt
+)
+
+
+class TestPhase:
+    def test_creation(self):
+        phase = Phase(id="A1", name="Setup", artifact="config.yaml")
+        assert phase.id == "A1"
+        assert phase.status == "pending"
+    
+    def test_dependencies(self):
+        phase = Phase(id="A2", name="Build", dependencies=["A1"])
+        assert "A1" in phase.dependencies
+
+
+class TestStream:
+    def test_progress_empty(self):
+        stream = Stream(id="A", name="Stream A")
+        assert stream.progress == 0.0
+    
+    def test_progress_partial(self):
+        stream = Stream(id="A", name="Stream A", phases=[
+            Phase(id="A1", name="P1", status="done"),
+            Phase(id="A2", name="P2", status="pending"),
+        ])
+        assert stream.progress == 0.5
+    
+    def test_current_phase(self):
+        stream = Stream(id="A", name="Stream A", phases=[
+            Phase(id="A1", name="P1", status="done"),
+            Phase(id="A2", name="P2", status="active"),
+            Phase(id="A3", name="P3", status="pending"),
+        ])
+        assert stream.current_phase.id == "A2"
+
+
+class TestUltraplan:
+    def test_to_markdown(self):
+        plan = Ultraplan(
+            date="20260415",
+            mission="Test mission",
+            streams=[
+                Stream(id="A", name="Stream A", phases=[
+                    Phase(id="A1", name="Phase 1", artifact="file.txt"),
+                ]),
+            ],
+        )
+        md = plan.to_markdown()
+        assert "# Ultraplan: 20260415" in md
+        assert "Test mission" in md
+        assert "Stream A" in md
+    
+    def test_progress(self):
+        plan = Ultraplan(
+            date="20260415",
+            mission="Test",
+            streams=[
+                Stream(id="A", name="A", status="done", phases=[
+                    Phase(id="A1", name="P1", status="done"),
+                ]),
+                Stream(id="B", name="B", status="pending", phases=[
+                    Phase(id="B1", name="P1", status="pending"),
+                ]),
+            ],
+        )
+        assert plan.progress == 0.5
+    
+    def test_to_dict(self):
+        plan = Ultraplan(date="20260415", mission="Test")
+        d = plan.to_dict()
+        assert d["date"] == "20260415"
+        assert d["mission"] == "Test"
+
+
+class TestCreateUltraplan:
+    def test_default_date(self):
+        plan = create_ultraplan(mission="Test")
+        assert len(plan.date) == 8  # YYYYMMDD
+    
+    def test_with_streams(self):
+        plan = create_ultraplan(
+            mission="Test",
+            streams=[
+                {
+                    "id": "A",
+                    "name": "Stream A",
+                    "phases": [
+                        {"id": "A1", "name": "Setup", "artifact": "config.yaml"},
+                        {"id": "A2", "name": "Build", "dependencies": ["A1"]},
+                    ],
+                },
+            ],
+        )
+        assert len(plan.streams) == 1
+        assert len(plan.streams[0].phases) == 2
+        assert plan.streams[0].phases[1].dependencies == ["A1"]
+
+
+class TestSaveLoad:
+    def test_roundtrip(self, tmp_path):
+        plan = create_ultraplan(
+            date="20260415",
+            mission="Test roundtrip",
+            streams=[{"id": "A", "name": "Stream A"}],
+        )
+        
+        save_ultraplan(plan, base_dir=tmp_path)
+        loaded = load_ultraplan("20260415", base_dir=tmp_path)
+        
+        assert loaded is not None
+        assert loaded.date == "20260415"
+        assert loaded.mission == "Test roundtrip"
+    
+    def test_nonexistent_returns_none(self, tmp_path):
+        assert load_ultraplan("99999999", base_dir=tmp_path) is None
+
+
+class TestCronPrompt:
+    def test_has_required_elements(self):
+        prompt = generate_daily_cron_prompt()
+        assert "Ultraplan" in prompt
+        assert "streams" in prompt.lower()
+        assert "Gitea" in prompt
+
+
+if __name__ == "__main__":
+    import pytest
+    pytest.main([__file__, "-v"])

tests/test_vision_benchmark.py

@@ -0,0 +1,239 @@
+"""Tests for vision benchmark suite (Issue #817)."""
+
+import json
+import statistics
+import sys
+from pathlib import Path
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "benchmarks"))
+
+from vision_benchmark import (
+    compute_ocr_accuracy,
+    compute_description_completeness,
+    compute_structural_accuracy,
+    aggregate_results,
+    to_markdown,
+    generate_sample_dataset,
+    MODELS,
+    EVAL_PROMPTS,
+)
+
+
+class TestOcrAccuracy:
+    def test_perfect_match(self):
+        assert compute_ocr_accuracy("Hello World", "Hello World") == 1.0
+
+    def test_empty_ground_truth(self):
+        assert compute_ocr_accuracy("", "") == 1.0
+        assert compute_ocr_accuracy("text", "") == 0.0
+
+    def test_empty_extraction(self):
+        assert compute_ocr_accuracy("", "Hello") == 0.0
+
+    def test_partial_match(self):
+        score = compute_ocr_accuracy("Hello Wrld", "Hello World")
+        assert 0.5 < score < 1.0
+
+    def test_case_insensitive(self):
+        assert compute_ocr_accuracy("hello world", "Hello World") == 1.0
+
+    def test_whitespace_differences(self):
+        score = compute_ocr_accuracy("  Hello  World  ", "Hello World")
+        assert score >= 0.8
+
+
+class TestDescriptionCompleteness:
+    def test_all_keywords_found(self):
+        keywords = ["github", "logo", "octocat"]
+        text = "This is the GitHub logo featuring the octocat mascot."
+        assert compute_description_completeness(text, keywords) == 1.0
+
+    def test_partial_keywords(self):
+        keywords = ["github", "logo", "octocat"]
+        text = "This is the GitHub logo."
+        score = compute_description_completeness(text, keywords)
+        assert 0.3 < score < 0.7
+
+    def test_no_keywords(self):
+        keywords = ["github", "logo"]
+        text = "Something completely different."
+        assert compute_description_completeness(text, keywords) == 0.0
+
+    def test_empty_keywords(self):
+        assert compute_description_completeness("any text", []) == 1.0
+
+    def test_empty_text(self):
+        assert compute_description_completeness("", ["keyword"]) == 0.0
+
+    def test_case_insensitive(self):
+        keywords = ["GitHub", "Logo"]
+        text = "The github logo is iconic."
+        assert compute_description_completeness(text, keywords) == 1.0
+
+
+class TestStructuralAccuracy:
+    def test_length_score(self):
+        text = "A" * 100
+        scores = compute_structural_accuracy(text, {"min_length": 50})
+        assert scores["length"] == 1.0
+
+    def test_short_text(self):
+        text = "Short."
+        scores = compute_structural_accuracy(text, {"min_length": 100})
+        assert scores["length"] < 1.0
+
+    def test_sentence_count(self):
+        text = "First sentence. Second sentence. Third sentence."
+        scores = compute_structural_accuracy(text, {"min_sentences": 2})
+        assert scores["sentences"] >= 1.0
+
+    def test_no_sentences(self):
+        text = "No sentence end"
+        scores = compute_structural_accuracy(text, {"min_sentences": 1})
+        assert scores["sentences"] == 0.0
+
+    def test_has_numbers_true(self):
+        text = "There are 42 items."
+        scores = compute_structural_accuracy(text, {"has_numbers": True})
+        assert scores["has_numbers"] == 1.0
+
+    def test_has_numbers_false(self):
+        text = "No numbers here."
+        scores = compute_structural_accuracy(text, {"has_numbers": True})
+        assert scores["has_numbers"] == 0.0
+
+
+class TestAggregateResults:
+    def test_basic_aggregation(self):
+        results = [
+            {
+                "image_id": "img1",
+                "category": "photo",
+                "gemma4": {
+                    "success": True,
+                    "avg_latency_ms": 100,
+                    "avg_tokens": 500,
+                    "ocr_accuracy": 0.9,
+                    "keyword_completeness": 0.8,
+                    "analysis_length": 200,
+                },
+                "gemini3_flash": {
+                    "success": True,
+                    "avg_latency_ms": 150,
+                    "avg_tokens": 600,
+                    "ocr_accuracy": 0.85,
+                    "keyword_completeness": 0.75,
+                    "analysis_length": 180,
+                },
+            }
+        ]
+        models = MODELS
+        summary = aggregate_results(results, models)
+
+        assert "gemma4" in summary
+        assert "gemini3_flash" in summary
+        assert summary["gemma4"]["success_rate"] == 1.0
+        assert summary["gemma4"]["latency"]["mean_ms"] == 100
+        assert summary["gemma4"]["accuracy"]["ocr_mean"] == 0.9
+
+    def test_all_failures(self):
+        results = [
+            {
+                "image_id": "img1",
+                "category": "photo",
+                "gemma4": {"success": False, "error": "API error"},
+                "gemini3_flash": {"success": False, "error": "API error"},
+            }
+        ]
+        summary = aggregate_results(results, MODELS)
+        assert summary["gemma4"]["success_rate"] == 0
+
+
+class TestMarkdown:
+    def test_generates_report(self):
+        report = {
+            "generated_at": "2026-04-16T00:00:00",
+            "config": {
+                "total_images": 10,
+                "runs_per_model": 1,
+                "models": {"gemma4": "Gemma 4 27B", "gemini3_flash": "Gemini 3 Flash"},
+            },
+            "summary": {
+                "gemma4": {
+                    "success_rate": 0.9,
+                    "latency": {"mean_ms": 100, "median_ms": 95, "p95_ms": 150, "std_ms": 20},
+                    "tokens": {"mean_total": 500, "total_used": 5000},
+                    "accuracy": {"ocr_mean": 0.85, "ocr_count": 5, "keyword_mean": 0.8, "keyword_count": 5},
+                },
+                "gemini3_flash": {
+                    "success_rate": 0.95,
+                    "latency": {"mean_ms": 120, "median_ms": 110, "p95_ms": 180, "std_ms": 25},
+                    "tokens": {"mean_total": 600, "total_used": 6000},
+                    "accuracy": {"ocr_mean": 0.82, "ocr_count": 5, "keyword_mean": 0.78, "keyword_count": 5},
+                },
+            },
+            "results": [],
+        }
+        md = to_markdown(report)
+        assert "Vision Benchmark Report" in md
+        assert "Latency Comparison" in md
+        assert "Accuracy Comparison" in md
+        assert "Token Usage" in md
+        assert "Verdict" in md
+        assert "Gemma 4 27B" in md
+
+    def test_empty_report(self):
+        report = {
+            "generated_at": "2026-04-16T00:00:00",
+            "config": {"total_images": 0, "runs_per_model": 1, "models": {}},
+            "summary": {},
+            "results": [],
+        }
+        md = to_markdown(report)
+        assert "Vision Benchmark Report" in md
+
+
+class TestDataset:
+    def test_sample_dataset_has_entries(self):
+        dataset = generate_sample_dataset()
+        assert len(dataset) >= 4
+
+    def test_sample_dataset_structure(self):
+        dataset = generate_sample_dataset()
+        for img in dataset:
+            assert "id" in img
+            assert "url" in img
+            assert "category" in img
+            assert "expected_keywords" in img
+            assert "expected_structure" in img
+
+    def test_categories_present(self):
+        dataset = generate_sample_dataset()
+        categories = {img["category"] for img in dataset}
+        assert "screenshot" in categories
+        assert "diagram" in categories
+        assert "photo" in categories
+
+
+class TestModels:
+    def test_all_models_defined(self):
+        assert "gemma4" in MODELS
+        assert "gemini3_flash" in MODELS
+
+    def test_model_structure(self):
+        for name, config in MODELS.items():
+            assert "model_id" in config
+            assert "display_name" in config
+            assert "provider" in config
+
+
+class TestPrompts:
+    def test_prompts_for_categories(self):
+        assert "screenshot" in EVAL_PROMPTS
+        assert "diagram" in EVAL_PROMPTS
+        assert "photo" in EVAL_PROMPTS
+        assert "ocr" in EVAL_PROMPTS
+        assert "chart" in EVAL_PROMPTS

tests/tools/test_confirmation_daemon.py

@@ -0,0 +1,190 @@
+"""Tests for tools.confirmation_daemon — Human Confirmation Firewall."""
+
+import pytest
+import time
+from tools.confirmation_daemon import (
+    ConfirmationDaemon,
+    ConfirmationRequest,
+    ConfirmationStatus,
+    RiskLevel,
+    classify_action,
+    _is_whitelisted,
+    _DEFAULT_WHITELIST,
+)
+
+
+class TestClassifyAction:
+    """Test action risk classification."""
+
+    def test_crypto_tx_is_critical(self):
+        assert classify_action("crypto_tx") == RiskLevel.CRITICAL
+
+    def test_sign_transaction_is_critical(self):
+        assert classify_action("sign_transaction") == RiskLevel.CRITICAL
+
+    def test_send_email_is_high(self):
+        assert classify_action("send_email") == RiskLevel.HIGH
+
+    def test_send_message_is_medium(self):
+        assert classify_action("send_message") == RiskLevel.MEDIUM
+
+    def test_access_calendar_is_low(self):
+        assert classify_action("access_calendar") == RiskLevel.LOW
+
+    def test_unknown_action_is_medium(self):
+        assert classify_action("unknown_action_xyz") == RiskLevel.MEDIUM
+
+
+class TestWhitelist:
+    """Test whitelist auto-approval."""
+
+    def test_self_email_is_whitelisted(self):
+        whitelist = dict(_DEFAULT_WHITELIST)
+        payload = {"from": "me@test.com", "to": "me@test.com"}
+        assert _is_whitelisted("send_email", payload, whitelist) is True
+
+    def test_non_whitelisted_recipient_not_approved(self):
+        whitelist = dict(_DEFAULT_WHITELIST)
+        payload = {"to": "random@stranger.com"}
+        assert _is_whitelisted("send_email", payload, whitelist) is False
+
+    def test_whitelisted_contact_approved(self):
+        whitelist = {
+            "send_message": {"targets": ["alice", "bob"]},
+        }
+        assert _is_whitelisted("send_message", {"to": "alice"}, whitelist) is True
+        assert _is_whitelisted("send_message", {"to": "charlie"}, whitelist) is False
+
+    def test_no_whitelist_entry_means_not_whitelisted(self):
+        whitelist = {}
+        assert _is_whitelisted("crypto_tx", {"amount": 1.0}, whitelist) is False
+
+
+class TestConfirmationRequest:
+    """Test the request data model."""
+
+    def test_defaults(self):
+        req = ConfirmationRequest(
+            request_id="test-1",
+            action="send_email",
+            description="Test email",
+            risk_level="high",
+            payload={},
+        )
+        assert req.status == ConfirmationStatus.PENDING.value
+        assert req.created_at > 0
+        assert req.expires_at > req.created_at
+
+    def test_is_pending(self):
+        req = ConfirmationRequest(
+            request_id="test-2",
+            action="send_email",
+            description="Test",
+            risk_level="high",
+            payload={},
+            expires_at=time.time() + 300,
+        )
+        assert req.is_pending is True
+
+    def test_is_expired(self):
+        req = ConfirmationRequest(
+            request_id="test-3",
+            action="send_email",
+            description="Test",
+            risk_level="high",
+            payload={},
+            expires_at=time.time() - 10,
+        )
+        assert req.is_expired is True
+        assert req.is_pending is False
+
+    def test_to_dict(self):
+        req = ConfirmationRequest(
+            request_id="test-4",
+            action="send_email",
+            description="Test",
+            risk_level="medium",
+            payload={"to": "a@b.com"},
+        )
+        d = req.to_dict()
+        assert d["request_id"] == "test-4"
+        assert d["action"] == "send_email"
+        assert "is_pending" in d
+
+
+class TestConfirmationDaemon:
+    """Test the daemon logic (without HTTP layer)."""
+
+    def test_auto_approve_low_risk(self):
+        daemon = ConfirmationDaemon()
+        req = daemon.request(
+            action="access_calendar",
+            description="Read today's events",
+            risk_level="low",
+        )
+        assert req.status == ConfirmationStatus.AUTO_APPROVED.value
+
+    def test_whitelisted_auto_approves(self):
+        daemon = ConfirmationDaemon()
+        daemon._whitelist = {"send_message": {"targets": ["alice"]}}
+        req = daemon.request(
+            action="send_message",
+            description="Message alice",
+            payload={"to": "alice"},
+        )
+        assert req.status == ConfirmationStatus.AUTO_APPROVED.value
+
+    def test_non_whitelisted_goes_pending(self):
+        daemon = ConfirmationDaemon()
+        daemon._whitelist = {}
+        req = daemon.request(
+            action="send_email",
+            description="Email to stranger",
+            payload={"to": "stranger@test.com"},
+            risk_level="high",
+        )
+        assert req.status == ConfirmationStatus.PENDING.value
+        assert req.is_pending is True
+
+    def test_approve_response(self):
+        daemon = ConfirmationDaemon()
+        daemon._whitelist = {}
+        req = daemon.request(
+            action="send_email",
+            description="Email test",
+            risk_level="high",
+        )
+        result = daemon.respond(req.request_id, approved=True, decided_by="human")
+        assert result.status == ConfirmationStatus.APPROVED.value
+        assert result.decided_by == "human"
+
+    def test_deny_response(self):
+        daemon = ConfirmationDaemon()
+        daemon._whitelist = {}
+        req = daemon.request(
+            action="crypto_tx",
+            description="Send 1 ETH",
+            risk_level="critical",
+        )
+        result = daemon.respond(
+            req.request_id, approved=False, decided_by="human", reason="Too risky"
+        )
+        assert result.status == ConfirmationStatus.DENIED.value
+        assert result.reason == "Too risky"
+
+    def test_get_pending(self):
+        daemon = ConfirmationDaemon()
+        daemon._whitelist = {}
+        daemon.request(action="send_email", description="Test 1", risk_level="high")
+        daemon.request(action="send_email", description="Test 2", risk_level="high")
+        pending = daemon.get_pending()
+        assert len(pending) >= 2
+
+    def test_get_history(self):
+        daemon = ConfirmationDaemon()
+        req = daemon.request(
+            action="access_calendar", description="Test", risk_level="low"
+        )
+        history = daemon.get_history()
+        assert len(history) >= 1
+        assert history[0]["action"] == "access_calendar"

tools/approval.py

@@ -121,6 +121,19 @@ DANGEROUS_PATTERNS = [
     (r'\b(cp|mv|install)\b.*\s/etc/', "copy/move file into /etc/"),
     (r'\bsed\s+-[^\s]*i.*\s/etc/', "in-place edit of system config"),
     (r'\bsed\s+--in-place\b.*\s/etc/', "in-place edit of system config (long flag)"),
+    # --- Vitalik's threat model: crypto / financial ---
+    (r'\b(?:bitcoin-cli|ethers\.js|web3|ether\.sendTransaction)\b', "direct crypto transaction tool usage"),
+    (r'\bwget\b.*\b(?:mnemonic|seed\s*phrase|private[_-]?key)\b', "attempting to download crypto credentials"),
+    (r'\bcurl\b.*\b(?:mnemonic|seed\s*phrase|private[_-]?key)\b', "attempting to exfiltrate crypto credentials"),
+    # --- Vitalik's threat model: credential exfiltration ---
+    (r'\b(?:curl|wget|http|nc|ncat|socat)\b.*\b(?:\.env|\.ssh|credentials|secrets|token|api[_-]?key)\b',
+     "attempting to exfiltrate credentials via network"),
+    (r'\bbase64\b.*\|(?:\s*curl|\s*wget)', "base64-encode then network exfiltration"),
+    (r'\bcat\b.*\b(?:\.env|\.ssh/id_rsa|credentials)\b.*\|(?:\s*curl|\s*wget)',
+     "reading secrets and piping to network tool"),
+    # --- Vitalik's threat model: data exfiltration ---
+    (r'\bcurl\b.*-d\s.*\$(?:HOME|USER)', "sending user home directory data to remote"),
+    (r'\bwget\b.*--post-data\s.*\$(?:HOME|USER)', "posting user data to remote"),
     # Script execution via heredoc — bypasses the -e/-c flag patterns above.
     # `python3 << 'EOF'` feeds arbitrary code via stdin without -c/-e flags.
     (r'\b(python[23]?|perl|ruby|node)\s+<<', "script execution via heredoc"),

tools/code_execution_tool.py

@@ -28,6 +28,7 @@ Platform: Linux / macOS only (Unix domain sockets for local). Disabled on Window
 Remote execution additionally requires Python 3 in the terminal backend.
 """
 
+import ast
 import base64
 import json
 import logging
@@ -883,6 +884,42 @@ def _execute_remote(
     return json.dumps(result, ensure_ascii=False)
 
 
+
+def _validate_python_syntax(code: str) -> Optional[str]:
+    """Validate Python syntax before subprocess spawn.
+
+    Runs ast.parse() in-process (sub-millisecond) to catch syntax errors
+    before wasting time spawning a sandboxed subprocess.
+
+    Returns:
+        JSON error string with line, offset, message if syntax is invalid.
+        None if syntax is valid.
+    """
+    try:
+        ast.parse(code)
+        return None
+    except SyntaxError as exc:
+        # Build context: show offending line with caret
+        lines = code.split("\n")
+        error_line = lines[exc.lineno - 1] if exc.lineno and exc.lineno <= len(lines) else ""
+        context = ""
+        if error_line:
+            context = f"\n  {error_line}"
+            if exc.offset:
+                context += f"\n  {' ' * (exc.offset - 1)}^"
+
+        return json.dumps({
+            "error": f"Python syntax error on line {exc.lineno}: {exc.msg}{context}",
+            "syntax_error": True,
+            "line": exc.lineno,
+            "offset": exc.offset,
+            "message": exc.msg,
+        })
+
+
+# ---------------------------------------------------------------------------
+
+
 # ---------------------------------------------------------------------------
 # Main entry point
 # ---------------------------------------------------------------------------
@@ -916,6 +953,11 @@ def execute_code(
     if not code or not code.strip():
         return tool_error("No code provided.")
 
+    # Syntax check before subprocess spawn (catches ~15% of errors in <1ms)
+    syntax_error = _validate_python_syntax(code)
+    if syntax_error:
+        return syntax_error
+
     # Dispatch: remote backends use file-based RPC, local uses UDS
     from tools.terminal_tool import _get_env_config
     env_type = _get_env_config()["env_type"]

tools/confirmation_daemon.py

@@ -0,0 +1,617 @@
+"""Human Confirmation Daemon — HTTP server for two-factor action approval.
+
+Implements Vitalik's Pattern 1: "The new 'two-factor confirmation' is that
+the two factors are the human and the LLM."
+
+This daemon runs on localhost:6000 and provides a simple HTTP API for the
+agent to request human approval before executing high-risk actions.
+
+Threat model:
+- LLM jailbreaks: Remote content "hacking" the LLM to perform malicious actions
+- LLM accidents: LLM accidentally performing dangerous operations
+- The human acts as the second factor — the agent proposes, the human disposes
+
+Architecture:
+- Agent detects high-risk action → POST /confirm with action details
+- Daemon stores pending request, sends notification to user
+- User approves/denies via POST /respond (Telegram, CLI, or direct HTTP)
+- Agent receives decision and proceeds or aborts
+
+Usage:
+    # Start daemon (usually managed by gateway)
+    from tools.confirmation_daemon import ConfirmationDaemon
+    daemon = ConfirmationDaemon(port=6000)
+    daemon.start()
+
+    # Request approval (from agent code)
+    from tools.confirmation_daemon import request_confirmation
+    approved = request_confirmation(
+        action="send_email",
+        description="Send email to alice@example.com",
+        risk_level="high",
+        payload={"to": "alice@example.com", "subject": "Meeting notes"},
+        timeout=300,
+    )
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import os
+import threading
+import time
+import uuid
+from dataclasses import dataclass, field, asdict
+from enum import Enum, auto
+from pathlib import Path
+from typing import Any, Callable, Dict, List, Optional, Tuple
+
+from hermes_constants import get_hermes_home
+
+logger = logging.getLogger(__name__)
+
+
+class RiskLevel(Enum):
+    """Risk classification for actions requiring confirmation."""
+    LOW = "low"           # Log only, no confirmation needed
+    MEDIUM = "medium"     # Confirm for non-whitelisted targets
+    HIGH = "high"         # Always confirm
+    CRITICAL = "critical" # Always confirm + require explicit reason
+
+
+class ConfirmationStatus(Enum):
+    """Status of a pending confirmation request."""
+    PENDING = "pending"
+    APPROVED = "approved"
+    DENIED = "denied"
+    EXPIRED = "expired"
+    AUTO_APPROVED = "auto_approved"
+
+
+@dataclass
+class ConfirmationRequest:
+    """A request for human confirmation of a high-risk action."""
+    request_id: str
+    action: str               # Action type: send_email, send_message, crypto_tx, etc.
+    description: str          # Human-readable description of what will happen
+    risk_level: str           # low, medium, high, critical
+    payload: Dict[str, Any]   # Action-specific data (sanitized)
+    session_key: str = ""     # Session that initiated the request
+    created_at: float = 0.0
+    expires_at: float = 0.0
+    status: str = ConfirmationStatus.PENDING.value
+    decided_at: float = 0.0
+    decided_by: str = ""      # "human", "auto", "whitelist"
+    reason: str = ""          # Optional reason for denial
+
+    def __post_init__(self):
+        if not self.created_at:
+            self.created_at = time.time()
+        if not self.expires_at:
+            self.expires_at = self.created_at + 300  # 5 min default
+        if not self.request_id:
+            self.request_id = str(uuid.uuid4())[:12]
+
+    @property
+    def is_expired(self) -> bool:
+        return time.time() > self.expires_at
+
+    @property
+    def is_pending(self) -> bool:
+        return self.status == ConfirmationStatus.PENDING.value and not self.is_expired
+
+    def to_dict(self) -> Dict[str, Any]:
+        d = asdict(self)
+        d["is_expired"] = self.is_expired
+        d["is_pending"] = self.is_pending
+        return d
+
+
+# =========================================================================
+# Action categories (Vitalik's threat model)
+# =========================================================================
+
+ACTION_CATEGORIES = {
+    # Messaging — outbound communication to external parties
+    "send_email": RiskLevel.HIGH,
+    "send_message": RiskLevel.MEDIUM,     # Depends on recipient
+    "send_signal": RiskLevel.HIGH,
+    "send_telegram": RiskLevel.MEDIUM,
+    "send_discord": RiskLevel.MEDIUM,
+    "post_social": RiskLevel.HIGH,
+
+    # Financial / crypto
+    "crypto_tx": RiskLevel.CRITICAL,
+    "sign_transaction": RiskLevel.CRITICAL,
+    "access_wallet": RiskLevel.CRITICAL,
+    "modify_balance": RiskLevel.CRITICAL,
+
+    # System modification
+    "install_software": RiskLevel.HIGH,
+    "modify_system_config": RiskLevel.HIGH,
+    "modify_firewall": RiskLevel.CRITICAL,
+    "add_ssh_key": RiskLevel.CRITICAL,
+    "create_user": RiskLevel.CRITICAL,
+
+    # Data access
+    "access_contacts": RiskLevel.MEDIUM,
+    "access_calendar": RiskLevel.LOW,
+    "read_private_files": RiskLevel.MEDIUM,
+    "upload_data": RiskLevel.HIGH,
+    "share_credentials": RiskLevel.CRITICAL,
+
+    # Network
+    "open_port": RiskLevel.HIGH,
+    "modify_dns": RiskLevel.HIGH,
+    "expose_service": RiskLevel.CRITICAL,
+}
+
+# Default: any unrecognized action is MEDIUM risk
+DEFAULT_RISK_LEVEL = RiskLevel.MEDIUM
+
+
+def classify_action(action: str) -> RiskLevel:
+    """Classify an action by its risk level."""
+    return ACTION_CATEGORIES.get(action, DEFAULT_RISK_LEVEL)
+
+
+# =========================================================================
+# Whitelist configuration
+# =========================================================================
+
+_DEFAULT_WHITELIST = {
+    "send_message": {
+        "targets": [],   # Contact names/IDs that don't need confirmation
+    },
+    "send_email": {
+        "targets": [],   # Email addresses that don't need confirmation
+        "self_only": True,  # send-to-self always allowed
+    },
+}
+
+
+def _load_whitelist() -> Dict[str, Any]:
+    """Load action whitelist from config."""
+    config_path = get_hermes_home() / "approval_whitelist.json"
+    if config_path.exists():
+        try:
+            with open(config_path) as f:
+                return json.load(f)
+        except Exception as e:
+            logger.warning("Failed to load approval whitelist: %s", e)
+    return dict(_DEFAULT_WHITELIST)
+
+
+def _is_whitelisted(action: str, payload: Dict[str, Any], whitelist: Dict) -> bool:
+    """Check if an action is pre-approved by the whitelist."""
+    action_config = whitelist.get(action, {})
+    if not action_config:
+        return False
+
+    # Check target-based whitelist
+    targets = action_config.get("targets", [])
+    target = payload.get("to") or payload.get("recipient") or payload.get("target", "")
+    if target and target in targets:
+        return True
+
+    # Self-only email
+    if action_config.get("self_only") and action == "send_email":
+        sender = payload.get("from", "")
+        recipient = payload.get("to", "")
+        if sender and recipient and sender.lower() == recipient.lower():
+            return True
+
+    return False
+
+
+# =========================================================================
+# Confirmation daemon
+# =========================================================================
+
+class ConfirmationDaemon:
+    """HTTP daemon for human confirmation of high-risk actions.
+
+    Runs on localhost:PORT (default 6000). Provides:
+    - POST /confirm   — agent requests human approval
+    - POST /respond   — human approves/denies
+    - GET  /pending   — list pending requests
+    - GET  /health    — health check
+    """
+
+    def __init__(
+        self,
+        host: str = "127.0.0.1",
+        port: int = 6000,
+        default_timeout: int = 300,
+        notify_callback: Optional[Callable] = None,
+    ):
+        self.host = host
+        self.port = port
+        self.default_timeout = default_timeout
+        self.notify_callback = notify_callback
+        self._pending: Dict[str, ConfirmationRequest] = {}
+        self._history: List[ConfirmationRequest] = []
+        self._lock = threading.Lock()
+        self._whitelist = _load_whitelist()
+        self._app = None
+        self._runner = None
+
+    def request(
+        self,
+        action: str,
+        description: str,
+        payload: Optional[Dict[str, Any]] = None,
+        risk_level: Optional[str] = None,
+        session_key: str = "",
+        timeout: Optional[int] = None,
+    ) -> ConfirmationRequest:
+        """Create a confirmation request.
+
+        Returns the request. Check .status to see if it was immediately
+        auto-approved (whitelisted) or is pending human review.
+        """
+        payload = payload or {}
+
+        # Classify risk if not specified
+        if risk_level is None:
+            risk_level = classify_action(action).value
+
+        # Check whitelist
+        if risk_level in ("low",) or _is_whitelisted(action, payload, self._whitelist):
+            req = ConfirmationRequest(
+                request_id=str(uuid.uuid4())[:12],
+                action=action,
+                description=description,
+                risk_level=risk_level,
+                payload=payload,
+                session_key=session_key,
+                expires_at=time.time() + (timeout or self.default_timeout),
+                status=ConfirmationStatus.AUTO_APPROVED.value,
+                decided_at=time.time(),
+                decided_by="whitelist",
+            )
+            with self._lock:
+                self._history.append(req)
+            logger.info("Auto-approved whitelisted action: %s", action)
+            return req
+
+        # Create pending request
+        req = ConfirmationRequest(
+            request_id=str(uuid.uuid4())[:12],
+            action=action,
+            description=description,
+            risk_level=risk_level,
+            payload=payload,
+            session_key=session_key,
+            expires_at=time.time() + (timeout or self.default_timeout),
+        )
+
+        with self._lock:
+            self._pending[req.request_id] = req
+
+        # Notify human
+        if self.notify_callback:
+            try:
+                self.notify_callback(req.to_dict())
+            except Exception as e:
+                logger.warning("Confirmation notify callback failed: %s", e)
+
+        logger.info(
+            "Confirmation request %s: %s (%s risk) — waiting for human",
+            req.request_id, action, risk_level,
+        )
+        return req
+
+    def respond(
+        self,
+        request_id: str,
+        approved: bool,
+        decided_by: str = "human",
+        reason: str = "",
+    ) -> Optional[ConfirmationRequest]:
+        """Record a human decision on a pending request."""
+        with self._lock:
+            req = self._pending.get(request_id)
+            if not req:
+                logger.warning("Confirmation respond: unknown request %s", request_id)
+                return None
+            if not req.is_pending:
+                logger.warning("Confirmation respond: request %s already decided", request_id)
+                return req
+
+            req.status = (
+                ConfirmationStatus.APPROVED.value if approved
+                else ConfirmationStatus.DENIED.value
+            )
+            req.decided_at = time.time()
+            req.decided_by = decided_by
+            req.reason = reason
+
+            # Move to history
+            del self._pending[request_id]
+            self._history.append(req)
+
+        logger.info(
+            "Confirmation %s: %s by %s",
+            request_id, "APPROVED" if approved else "DENIED", decided_by,
+        )
+        return req
+
+    def wait_for_decision(
+        self, request_id: str, timeout: Optional[float] = None
+    ) -> ConfirmationRequest:
+        """Block until a decision is made or timeout expires."""
+        deadline = time.time() + (timeout or self.default_timeout)
+        while time.time() < deadline:
+            with self._lock:
+                req = self._pending.get(request_id)
+                if req and not req.is_pending:
+                    return req
+                if req and req.is_expired:
+                    req.status = ConfirmationStatus.EXPIRED.value
+                    del self._pending[request_id]
+                    self._history.append(req)
+                    return req
+            time.sleep(0.5)
+
+        # Timeout
+        with self._lock:
+            req = self._pending.pop(request_id, None)
+            if req:
+                req.status = ConfirmationStatus.EXPIRED.value
+                self._history.append(req)
+                return req
+
+        # Shouldn't reach here
+        return ConfirmationRequest(
+            request_id=request_id,
+            action="unknown",
+            description="Request not found",
+            risk_level="high",
+            payload={},
+            status=ConfirmationStatus.EXPIRED.value,
+        )
+
+    def get_pending(self) -> List[Dict[str, Any]]:
+        """Return list of pending confirmation requests."""
+        self._expire_old()
+        with self._lock:
+            return [r.to_dict() for r in self._pending.values() if r.is_pending]
+
+    def get_history(self, limit: int = 50) -> List[Dict[str, Any]]:
+        """Return recent confirmation history."""
+        with self._lock:
+            return [r.to_dict() for r in self._history[-limit:]]
+
+    def _expire_old(self) -> None:
+        """Move expired requests to history."""
+        now = time.time()
+        with self._lock:
+            expired = [
+                rid for rid, req in self._pending.items()
+                if now > req.expires_at
+            ]
+            for rid in expired:
+                req = self._pending.pop(rid)
+                req.status = ConfirmationStatus.EXPIRED.value
+                self._history.append(req)
+
+    # --- aiohttp HTTP API ---
+
+    async def _handle_health(self, request):
+        from aiohttp import web
+        return web.json_response({
+            "status": "ok",
+            "service": "hermes-confirmation-daemon",
+            "pending": len(self._pending),
+        })
+
+    async def _handle_confirm(self, request):
+        from aiohttp import web
+        try:
+            body = await request.json()
+        except Exception:
+            return web.json_response({"error": "invalid JSON"}, status=400)
+
+        action = body.get("action", "")
+        description = body.get("description", "")
+        if not action or not description:
+            return web.json_response(
+                {"error": "action and description required"}, status=400
+            )
+
+        req = self.request(
+            action=action,
+            description=description,
+            payload=body.get("payload", {}),
+            risk_level=body.get("risk_level"),
+            session_key=body.get("session_key", ""),
+            timeout=body.get("timeout"),
+        )
+
+        # If auto-approved, return immediately
+        if req.status != ConfirmationStatus.PENDING.value:
+            return web.json_response({
+                "request_id": req.request_id,
+                "status": req.status,
+                "decided_by": req.decided_by,
+            })
+
+        # Otherwise, wait for human decision (with timeout)
+        timeout = min(body.get("timeout", self.default_timeout), 600)
+        result = self.wait_for_decision(req.request_id, timeout=timeout)
+
+        return web.json_response({
+            "request_id": result.request_id,
+            "status": result.status,
+            "decided_by": result.decided_by,
+            "reason": result.reason,
+        })
+
+    async def _handle_respond(self, request):
+        from aiohttp import web
+        try:
+            body = await request.json()
+        except Exception:
+            return web.json_response({"error": "invalid JSON"}, status=400)
+
+        request_id = body.get("request_id", "")
+        approved = body.get("approved")
+        if not request_id or approved is None:
+            return web.json_response(
+                {"error": "request_id and approved required"}, status=400
+            )
+
+        result = self.respond(
+            request_id=request_id,
+            approved=bool(approved),
+            decided_by=body.get("decided_by", "human"),
+            reason=body.get("reason", ""),
+        )
+
+        if not result:
+            return web.json_response({"error": "unknown request"}, status=404)
+
+        return web.json_response({
+            "request_id": result.request_id,
+            "status": result.status,
+        })
+
+    async def _handle_pending(self, request):
+        from aiohttp import web
+        return web.json_response({"pending": self.get_pending()})
+
+    def _build_app(self):
+        """Build the aiohttp application."""
+        from aiohttp import web
+
+        app = web.Application()
+        app.router.add_get("/health", self._handle_health)
+        app.router.add_post("/confirm", self._handle_confirm)
+        app.router.add_post("/respond", self._handle_respond)
+        app.router.add_get("/pending", self._handle_pending)
+        self._app = app
+        return app
+
+    async def start_async(self) -> None:
+        """Start the daemon as an async server."""
+        from aiohttp import web
+
+        app = self._build_app()
+        self._runner = web.AppRunner(app)
+        await self._runner.setup()
+        site = web.TCPSite(self._runner, self.host, self.port)
+        await site.start()
+        logger.info("Confirmation daemon listening on %s:%d", self.host, self.port)
+
+    async def stop_async(self) -> None:
+        """Stop the daemon."""
+        if self._runner:
+            await self._runner.cleanup()
+            self._runner = None
+
+    def start(self) -> None:
+        """Start daemon in a background thread (blocking caller)."""
+        def _run():
+            loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(loop)
+            loop.run_until_complete(self.start_async())
+            loop.run_forever()
+
+        t = threading.Thread(target=_run, daemon=True, name="confirmation-daemon")
+        t.start()
+        logger.info("Confirmation daemon started in background thread")
+
+    def start_blocking(self) -> None:
+        """Start daemon and block (for standalone use)."""
+        loop = asyncio.new_event_loop()
+        asyncio.set_event_loop(loop)
+        loop.run_until_complete(self.start_async())
+        try:
+            loop.run_forever()
+        except KeyboardInterrupt:
+            pass
+        finally:
+            loop.run_until_complete(self.stop_async())
+
+
+# =========================================================================
+# Convenience API for agent integration
+# =========================================================================
+
+# Global singleton — initialized by gateway or CLI at startup
+_daemon: Optional[ConfirmationDaemon] = None
+
+
+def get_daemon() -> Optional[ConfirmationDaemon]:
+    """Get the global confirmation daemon instance."""
+    return _daemon
+
+
+def init_daemon(
+    host: str = "127.0.0.1",
+    port: int = 6000,
+    notify_callback: Optional[Callable] = None,
+) -> ConfirmationDaemon:
+    """Initialize the global confirmation daemon."""
+    global _daemon
+    _daemon = ConfirmationDaemon(
+        host=host, port=port, notify_callback=notify_callback
+    )
+    return _daemon
+
+
+def request_confirmation(
+    action: str,
+    description: str,
+    payload: Optional[Dict[str, Any]] = None,
+    risk_level: Optional[str] = None,
+    session_key: str = "",
+    timeout: int = 300,
+) -> bool:
+    """Request human confirmation for a high-risk action.
+
+    This is the primary integration point for agent code. It:
+    1. Classifies the action risk level
+    2. Checks the whitelist
+    3. If confirmation needed, blocks until human responds
+    4. Returns True if approved, False if denied/expired
+
+    Args:
+        action: Action type (send_email, crypto_tx, etc.)
+        description: Human-readable description
+        payload: Action-specific data
+        risk_level: Override auto-classification
+        session_key: Session requesting approval
+        timeout: Seconds to wait for human response
+
+    Returns:
+        True if approved, False if denied or expired.
+    """
+    daemon = get_daemon()
+    if not daemon:
+        logger.warning(
+            "No confirmation daemon running — DENYING action %s by default. "
+            "Start daemon with init_daemon() or --confirmation-daemon flag.",
+            action,
+        )
+        return False
+
+    req = daemon.request(
+        action=action,
+        description=description,
+        payload=payload,
+        risk_level=risk_level,
+        session_key=session_key,
+        timeout=timeout,
+    )
+
+    # Auto-approved (whitelisted)
+    if req.status == ConfirmationStatus.AUTO_APPROVED.value:
+        return True
+
+    # Wait for human
+    result = daemon.wait_for_decision(req.request_id, timeout=timeout)
+    return result.status == ConfirmationStatus.APPROVED.value

tools/tool_pokayoke.py

@@ -0,0 +1,276 @@
+#!/usr/bin/env python3
+"""
+Poka-Yoke: Tool Hallucination Prevention
+
+Detects and blocks tool hallucination before API calls:
+1. Validates tool names against registered tools
+2. Auto-corrects parameter names within Levenshtein distance 1
+3. Circuit breaker for consecutive failures
+
+Usage:
+    from tools.tool_pokayoke import validate_tool_call, ToolCallValidator
+    
+    # One-shot validation
+    result = validate_tool_call("browser_fill", {"file_path": "/tmp/test.txt"})
+    
+    # Stateful validator with circuit breaker
+    validator = ToolCallValidator()
+    result = validator.validate("browser_fill", {"file_path": "/tmp/test.txt"})
+"""
+
+import json
+import logging
+from typing import Dict, List, Optional, Tuple, Any
+from difflib import SequenceMatcher
+
+logger = logging.getLogger(__name__)
+
+
+def levenshtein_distance(s1: str, s2: str) -> int:
+    """Calculate Levenshtein distance between two strings."""
+    if len(s1) < len(s2):
+        return levenshtein_distance(s2, s1)
+    
+    if len(s2) == 0:
+        return len(s1)
+    
+    prev_row = range(len(s2) + 1)
+    for i, c1 in enumerate(s1):
+        curr_row = [i + 1]
+        for j, c2 in enumerate(s2):
+            insertions = prev_row[j + 1] + 1
+            deletions = curr_row[j] + 1
+            substitutions = prev_row[j] + (c1 != c2)
+            curr_row.append(min(insertions, deletions, substitutions))
+        prev_row = curr_row
+    
+    return prev_row[-1]
+
+
+def find_similar_names(name: str, valid_names: List[str], max_distance: int = 2) -> List[Tuple[str, int]]:
+    """Find similar names within edit distance."""
+    suggestions = []
+    for valid_name in valid_names:
+        dist = levenshtein_distance(name.lower(), valid_name.lower())
+        if 0 < dist <= max_distance:
+            suggestions.append((valid_name, dist))
+    return sorted(suggestions, key=lambda x: x[1])
+
+
+def auto_correct_parameter(param_name: str, valid_params: List[str]) -> Optional[str]:
+    """
+    Auto-correct parameter name if within Levenshtein distance 1.
+    Returns corrected name or None if no close match.
+    """
+    for valid_param in valid_params:
+        dist = levenshtein_distance(param_name.lower(), valid_param.lower())
+        if dist == 1:
+            logger.info(f"Poka-yoke: Auto-corrected parameter '{param_name}' -> '{valid_param}'")
+            return valid_param
+    return None
+
+
+class ToolCallValidator:
+    """
+    Stateful validator with circuit breaker for consecutive failures.
+    """
+    
+    def __init__(self, failure_threshold: int = 3):
+        self.failure_threshold = failure_threshold
+        self.consecutive_failures: Dict[str, int] = {}  # tool_name -> count
+        self.tool_schemas: Dict[str, dict] = {}  # tool_name -> schema
+        self._initialized = False
+    
+    def _ensure_initialized(self):
+        """Lazy initialization - load tool schemas from registry."""
+        if self._initialized:
+            return
+        
+        try:
+            from tools.registry import registry
+            for name in registry.get_all_tool_names():
+                schema = registry.get_schema(name)
+                if schema:
+                    self.tool_schemas[name] = schema
+            self._initialized = True
+            logger.debug(f"Poka-yoke initialized with {len(self.tool_schemas)} tool schemas")
+        except Exception as e:
+            logger.warning(f"Could not initialize poka-yoke from registry: {e}")
+    
+    def validate_tool_name(self, tool_name: str) -> Tuple[bool, Optional[str], List[str]]:
+        """
+        Validate tool name against registered tools.
+        
+        Returns:
+            (is_valid, suggested_name, error_messages)
+        """
+        self._ensure_initialized()
+        
+        if tool_name in self.tool_schemas:
+            return True, None, []
+        
+        # Check circuit breaker
+        if self.consecutive_failures.get(tool_name, 0) >= self.failure_threshold:
+            return False, None, [
+                f"CIRCUIT BREAKER: Tool '{tool_name}' has failed {self.failure_threshold}+ times consecutively.",
+                f"This may indicate a persistent hallucination. Halt and inject diagnostic.",
+                f"Valid tools: {', '.join(sorted(self.tool_schemas.keys())[:20])}..."
+            ]
+        
+        # Find similar names
+        suggestions = find_similar_names(tool_name, list(self.tool_schemas.keys()), max_distance=2)
+        
+        if suggestions:
+            best_match, distance = suggestions[0]
+            if distance == 1:
+                # Auto-correct
+                logger.info(f"Poka-yoke: Auto-corrected tool '{tool_name}' -> '{best_match}'")
+                return True, best_match, [f"Auto-corrected: '{tool_name}' -> '{best_match}'"]
+            else:
+                # Suggest
+                suggestion_list = [f"'{s}' (distance {d})" for s, d in suggestions[:3]]
+                return False, None, [
+                    f"Unknown tool: '{tool_name}'",
+                    f"Did you mean: {', '.join(suggestion_list)}?"
+                ]
+        
+        return False, None, [
+            f"Unknown tool: '{tool_name}'",
+            f"No similar tools found. Available: {', '.join(sorted(self.tool_schemas.keys())[:10])}..."
+        ]
+    
+    def validate_parameters(self, tool_name: str, params: Dict[str, Any]) -> Tuple[Dict[str, Any], List[str]]:
+        """
+        Validate and auto-correct parameter names.
+        
+        Returns:
+            (corrected_params, warnings)
+        """
+        self._ensure_initialized()
+        
+        if tool_name not in self.tool_schemas:
+            return params, []
+        
+        schema = self.tool_schemas[tool_name]
+        valid_params = list(schema.get("parameters", {}).get("properties", {}).keys())
+        
+        if not valid_params:
+            return params, []
+        
+        corrected = dict(params)
+        warnings = []
+        
+        for param_name in list(params.keys()):
+            if param_name not in valid_params:
+                corrected_name = auto_correct_parameter(param_name, valid_params)
+                if corrected_name:
+                    corrected[corrected_name] = corrected.pop(param_name)
+                    warnings.append(f"Auto-corrected parameter: '{param_name}' -> '{corrected_name}'")
+                else:
+                    warnings.append(f"Unknown parameter: '{param_name}' (valid: {', '.join(valid_params[:10])})")
+        
+        return corrected, warnings
+    
+    def validate(self, tool_name: str, params: Dict[str, Any]) -> Tuple[bool, Optional[str], Dict[str, Any], List[str]]:
+        """
+        Full validation of a tool call.
+        
+        Returns:
+            (is_valid, corrected_tool_name, corrected_params, messages)
+        """
+        # Validate tool name
+        name_valid, corrected_name, name_messages = self.validate_tool_name(tool_name)
+        
+        if not name_valid:
+            self._record_failure(tool_name)
+            return False, None, params, name_messages
+        
+        # Use corrected name if provided
+        actual_tool = corrected_name if corrected_name else tool_name
+        if corrected_name:
+            name_messages.append(f"Tool name corrected: '{tool_name}' -> '{corrected_name}'")
+        
+        # Validate parameters
+        corrected_params, param_warnings = self.validate_parameters(actual_tool, params)
+        
+        # Record success (reset failure counter)
+        self._record_success(actual_tool)
+        
+        all_messages = name_messages + param_warnings
+        return True, corrected_name, corrected_params, all_messages
+    
+    def _record_failure(self, tool_name: str):
+        """Record a failure for circuit breaker."""
+        self.consecutive_failures[tool_name] = self.consecutive_failures.get(tool_name, 0) + 1
+        count = self.consecutive_failures[tool_name]
+        
+        if count >= self.failure_threshold:
+            logger.warning(
+                f"Poka-yoke circuit breaker triggered for '{tool_name}': "
+                f"{count} consecutive failures"
+            )
+    
+    def _record_success(self, tool_name: str):
+        """Record a success (reset failure counter)."""
+        self.consecutive_failures.pop(tool_name, None)
+    
+    def get_diagnostic_message(self, tool_name: str) -> str:
+        """Generate diagnostic message for circuit breaker."""
+        self._ensure_initialized()
+        
+        count = self.consecutive_failures.get(tool_name, 0)
+        suggestions = find_similar_names(tool_name, list(self.tool_schemas.keys()), max_distance=3)
+        
+        lines = [
+            f"=== TOOL HALLUCINATION DETECTED ===",
+            f"Tool '{tool_name}' has failed {count} times consecutively.",
+            f"",
+            f"This likely means the model is hallucinating a tool name.",
+            f"",
+            f"Closest valid tools:"
+        ]
+        
+        for name, dist in suggestions[:5]:
+            lines.append(f"  - {name} (edit distance: {dist})")
+        
+        if not suggestions:
+            lines.append(f"  (no similar tools found)")
+        
+        lines.extend([
+            f"",
+            f"Action: The agent should stop retrying and use a valid tool name.",
+            f"If this persists, the model may need fine-tuning or prompt adjustment."
+        ])
+        
+        return "\n".join(lines)
+
+
+# Global validator instance
+_validator = ToolCallValidator()
+
+
+def validate_tool_call(tool_name: str, params: Dict[str, Any]) -> Tuple[bool, Optional[str], Dict[str, Any], List[str]]:
+    """
+    One-shot validation of a tool call.
+    
+    Returns:
+        (is_valid, corrected_tool_name, corrected_params, messages)
+    """
+    return _validator.validate(tool_name, params)
+
+
+def reset_circuit_breaker(tool_name: Optional[str] = None):
+    """Reset circuit breaker for a tool or all tools."""
+    if tool_name:
+        _validator.consecutive_failures.pop(tool_name, None)
+    else:
+        _validator.consecutive_failures.clear()
+
+
+def get_hallucination_stats() -> Dict[str, Any]:
+    """Get statistics about tool hallucinations."""
+    return {
+        "consecutive_failures": dict(_validator.consecutive_failures),
+        "tools_tracked": len(_validator.tool_schemas),
+        "threshold": _validator.failure_threshold
+    }

tools/ultraplan.py

@@ -0,0 +1,310 @@
+"""Ultraplan Mode — Daily autonomous planning and execution discipline.
+
+Decomposes assigned tasks into parallel work streams with explicit
+dependencies, phases, and artifact targets.
+
+Issue #840: Ultraplan Mode: Daily autonomous planning and execution
+"""
+
+import json
+import os
+import time
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+
+@dataclass
+class Phase:
+    """A single phase within a work stream."""
+    id: str
+    name: str
+    description: str = ""
+    status: str = "pending"  # pending, active, done, blocked
+    artifact: str = ""  # Expected deliverable
+    dependencies: List[str] = field(default_factory=list)
+    started_at: Optional[float] = None
+    completed_at: Optional[float] = None
+
+
+@dataclass
+class Stream:
+    """A parallel work stream with sequential phases."""
+    id: str
+    name: str
+    phases: List[Phase] = field(default_factory=list)
+    status: str = "pending"
+    
+    @property
+    def current_phase(self) -> Optional[Phase]:
+        for p in self.phases:
+            if p.status in ("active", "pending"):
+                return p
+        return None
+    
+    @property
+    def progress(self) -> float:
+        if not self.phases:
+            return 0.0
+        done = sum(1 for p in self.phases if p.status == "done")
+        return done / len(self.phases)
+
+
+@dataclass
+class Ultraplan:
+    """Daily ultraplan with work streams and metrics."""
+    date: str
+    mission: str
+    streams: List[Stream] = field(default_factory=list)
+    metrics: Dict[str, Any] = field(default_factory=dict)
+    notes: str = ""
+    created_at: float = field(default_factory=time.time)
+    
+    @property
+    def progress(self) -> float:
+        if not self.streams:
+            return 0.0
+        return sum(s.progress for s in self.streams) / len(self.streams)
+    
+    @property
+    def active_streams(self) -> List[Stream]:
+        return [s for s in self.streams if s.status == "active"]
+    
+    @property
+    def blocked_streams(self) -> List[Stream]:
+        return [s for s in self.streams if s.status == "blocked"]
+    
+    def to_markdown(self) -> str:
+        """Generate ultraplan markdown document."""
+        lines = []
+        
+        # Header
+        lines.append(f"# Ultraplan: {self.date}")
+        lines.append("")
+        lines.append(f"**Mission:** {self.mission}")
+        lines.append(f"**Created:** {datetime.fromtimestamp(self.created_at, tz=timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}")
+        lines.append(f"**Progress:** {self.progress:.0%}")
+        lines.append("")
+        
+        # Metrics
+        if self.metrics:
+            lines.append("## Metrics")
+            for key, value in self.metrics.items():
+                lines.append(f"- **{key}:** {value}")
+            lines.append("")
+        
+        # Streams
+        lines.append("## Work Streams")
+        lines.append("")
+        
+        for stream in self.streams:
+            status_icon = {"pending": "○", "active": "●", "done": "✓", "blocked": "✗"}.get(stream.status, "?")
+            lines.append(f"### {status_icon} Stream {stream.id}: {stream.name}")
+            lines.append(f"**Status:** {stream.status} | **Progress:** {stream.progress:.0%}")
+            lines.append("")
+            
+            # Phase table
+            lines.append("| Phase | Name | Status | Artifact |")
+            lines.append("|-------|------|--------|----------|")
+            for phase in stream.phases:
+                p_icon = {"pending": "○", "active": "●", "done": "✓", "blocked": "✗"}.get(phase.status, "?")
+                artifact = phase.artifact or "—"
+                lines.append(f"| {phase.id} | {phase.name} | {p_icon} {phase.status} | {artifact} |")
+            lines.append("")
+        
+        # Dependency map
+        lines.append("## Dependency Map")
+        lines.append("")
+        for stream in self.streams:
+            deps = []
+            for phase in stream.phases:
+                if phase.dependencies:
+                    deps.append(f"{phase.id} depends on: {', '.join(phase.dependencies)}")
+            if deps:
+                lines.append(f"**{stream.id}:** {'; '.join(deps)}")
+        
+        if not any(p.dependencies for s in self.streams for p in s.phases):
+            lines.append("All streams are independent — parallel execution possible.")
+        lines.append("")
+        
+        # Notes
+        if self.notes:
+            lines.append("## Notes")
+            lines.append(self.notes)
+            lines.append("")
+        
+        # Footer
+        lines.append("---")
+        lines.append(f"*Generated by Ultraplan Mode — {datetime.now().strftime('%Y-%m-%d %H:%M')}*")
+        
+        return "\n".join(lines)
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to JSON-serializable dict."""
+        return {
+            "date": self.date,
+            "mission": self.mission,
+            "streams": [
+                {
+                    "id": s.id,
+                    "name": s.name,
+                    "status": s.status,
+                    "phases": [
+                        {
+                            "id": p.id,
+                            "name": p.name,
+                            "description": p.description,
+                            "status": p.status,
+                            "artifact": p.artifact,
+                            "dependencies": p.dependencies,
+                        }
+                        for p in s.phases
+                    ],
+                }
+                for s in self.streams
+            ],
+            "metrics": self.metrics,
+            "notes": self.notes,
+            "progress": self.progress,
+            "created_at": self.created_at,
+        }
+
+
+def create_ultraplan(
+    date: str = None,
+    mission: str = "",
+    streams: List[Dict[str, Any]] = None,
+) -> Ultraplan:
+    """Create a new ultraplan.
+    
+    Args:
+        date: Plan date (default: today)
+        mission: High-level mission statement
+        streams: List of stream definitions
+    """
+    if date is None:
+        date = datetime.now().strftime("%Y%m%d")
+    
+    plan_streams = []
+    if streams:
+        for s in streams:
+            phases = [
+                Phase(
+                    id=p.get("id", f"{s.get('id', 'S')}{i+1}"),
+                    name=p.get("name", f"Phase {i+1}"),
+                    description=p.get("description", ""),
+                    artifact=p.get("artifact", ""),
+                    dependencies=p.get("dependencies", []),
+                )
+                for i, p in enumerate(s.get("phases", []))
+            ]
+            plan_streams.append(Stream(
+                id=s.get("id", f"S{len(plan_streams)+1}"),
+                name=s.get("name", "Unnamed Stream"),
+                phases=phases,
+            ))
+    
+    return Ultraplan(
+        date=date,
+        mission=mission,
+        streams=plan_streams,
+    )
+
+
+def save_ultraplan(plan: Ultraplan, base_dir: Path = None) -> Path:
+    """Save ultraplan to disk.
+    
+    Args:
+        plan: The ultraplan to save
+        base_dir: Base directory (default: ~/.timmy/cron/)
+        
+    Returns:
+        Path to saved file
+    """
+    if base_dir is None:
+        base_dir = Path.home() / ".timmy" / "cron"
+    
+    base_dir.mkdir(parents=True, exist_ok=True)
+    
+    # Save markdown
+    md_path = base_dir / f"ultraplan_{plan.date}.md"
+    md_path.write_text(plan.to_markdown(), encoding="utf-8")
+    
+    # Save JSON (for programmatic access)
+    json_path = base_dir / f"ultraplan_{plan.date}.json"
+    json_path.write_text(json.dumps(plan.to_dict(), indent=2), encoding="utf-8")
+    
+    return md_path
+
+
+def load_ultraplan(date: str, base_dir: Path = None) -> Optional[Ultraplan]:
+    """Load ultraplan from disk.
+    
+    Args:
+        date: Plan date (YYYYMMDD)
+        base_dir: Base directory (default: ~/.timmy/cron/)
+        
+    Returns:
+        Ultraplan if found, None otherwise
+    """
+    if base_dir is None:
+        base_dir = Path.home() / ".timmy" / "cron"
+    
+    json_path = base_dir / f"ultraplan_{date}.json"
+    if not json_path.exists():
+        return None
+    
+    try:
+        data = json.loads(json_path.read_text(encoding="utf-8"))
+        
+        streams = []
+        for s in data.get("streams", []):
+            phases = [
+                Phase(
+                    id=p["id"],
+                    name=p["name"],
+                    description=p.get("description", ""),
+                    status=p.get("status", "pending"),
+                    artifact=p.get("artifact", ""),
+                    dependencies=p.get("dependencies", []),
+                )
+                for p in s.get("phases", [])
+            ]
+            streams.append(Stream(
+                id=s["id"],
+                name=s["name"],
+                phases=phases,
+                status=s.get("status", "pending"),
+            ))
+        
+        return Ultraplan(
+            date=data["date"],
+            mission=data.get("mission", ""),
+            streams=streams,
+            metrics=data.get("metrics", {}),
+            notes=data.get("notes", ""),
+            created_at=data.get("created_at", time.time()),
+        )
+    except Exception:
+        return None
+
+
+def generate_daily_cron_prompt() -> str:
+    """Generate the prompt for the daily ultraplan cron job."""
+    return """Generate today's Ultraplan.
+
+Steps:
+1. Check open Gitea issues assigned to you
+2. Check open PRs needing review
+3. Check fleet health status
+4. Decompose work into parallel streams
+5. Generate ultraplan_YYYYMMDD.md
+6. File Gitea issue with the plan
+
+Output format:
+- Mission statement
+- 3-5 work streams with phases
+- Dependency map
+- Success metrics
+"""

utils.py

@@ -145,6 +145,50 @@ def safe_json_loads(text: str, default: Any = None) -> Any:
         return default
 
 
+def repair_and_load_json(text: str, default: Any = None, *, context: str = "") -> Any:
+    """Parse JSON with automatic repair fallback.
+
+    Tries ``json.loads`` first.  On failure, attempts to repair the string
+    using the ``json_repair`` library before falling back to *default*.
+    Logs a debug-level warning when repair is triggered so that callers can
+    observe silent-failure patterns without raising exceptions.
+
+    Args:
+        text: The JSON string to parse.
+        default: Value returned when both parse and repair fail.
+        context: Optional label included in the debug log (e.g. the call-site
+                 name) to aid tracing.
+
+    Returns:
+        Parsed Python object, or *default* on unrecoverable failure.
+    """
+    if not isinstance(text, str):
+        return default
+    try:
+        return json.loads(text)
+    except (json.JSONDecodeError, ValueError):
+        pass
+
+    try:
+        import json_repair  # optional dependency
+        repaired = json_repair.repair_json(text, return_objects=True)
+        # json_repair returns "" when it cannot produce a valid structure.
+        # Guard against returning that sentinel as if it were a successful parse.
+        # Exception: if the original text was a JSON empty-string literal like `""`
+        # then "" is the correct parse result.
+        if repaired == "" and text.strip() not in ('""', "''"):
+            tag = f" [{context}]" if context else ""
+            logger.debug("repair_and_load_json%s: repair yielded empty string; returning default", tag)
+            return default
+        tag = f" [{context}]" if context else ""
+        logger.debug("repair_and_load_json%s: repaired malformed JSON (first 120 chars): %.120s", tag, text)
+        return repaired
+    except Exception as exc:
+        tag = f" [{context}]" if context else ""
+        logger.debug("repair_and_load_json%s: repair failed (%s); returning default", tag, exc)
+        return default
+
+
 # ─── Environment Variable Helpers ─────────────────────────────────────────────