Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3a7e0e7db4 |
@@ -13,9 +13,11 @@ import time
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
HERMES_HOME = Path.home() / ".hermes"
|
||||
HERMES_HOME = get_hermes_home()
|
||||
CHECKPOINT_DIR = HERMES_HOME / "checkpoints"
|
||||
CHARS_PER_TOKEN = 4
|
||||
|
||||
|
||||
@@ -1,157 +0,0 @@
|
||||
# AI Tools Evaluation Report (#842)
|
||||
|
||||
**Source:** [formatho/awesome-ai-tools](https://github.com/formatho/awesome-ai-tools)
|
||||
**Date:** 2026-04-15
|
||||
**Tools Analyzed:** 414 across 9 categories
|
||||
**Scope:** Hermes-agent integration potential
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Scanned 414 tools from awesome-ai-tools. Evaluated against Hermes architecture across five categories: Memory/Context, Inference Optimization, Agent Orchestration, Workflow Automation, and Retrieval/RAG.
|
||||
|
||||
## Top 5 Recommendations & Implementation Status
|
||||
|
||||
### P1 — Mem0 (Memory/Context) ✅ IMPLEMENTED
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| GitHub | [mem0ai/mem0](https://github.com/mem0ai/mem0) |
|
||||
| Stars | 53.1k ⭐ |
|
||||
| Integration Effort | 3/5 |
|
||||
| Impact | 5/5 |
|
||||
|
||||
**Status:** Both cloud (mem0ai) and local (ChromaDB) variants implemented.
|
||||
|
||||
**Deliverables:**
|
||||
- `plugins/memory/mem0/` — Platform API provider with server-side LLM extraction, semantic search, reranking
|
||||
- `plugins/memory/mem0_local/` — Sovereign local variant using ChromaDB, no API key required
|
||||
- Tools: `mem0_profile`, `mem0_search`, `mem0_conclude`
|
||||
- Circuit breaker for resilience
|
||||
- 36 tests passing across both providers
|
||||
|
||||
**Activation:**
|
||||
```bash
|
||||
hermes memory setup # select "mem0" or "mem0_local"
|
||||
```
|
||||
|
||||
**Risk mitigation:** OSS-only features used in `mem0_local`. Cloud version uses freemium API but has circuit-breaker fallback.
|
||||
|
||||
---
|
||||
|
||||
### P2 — LightRAG (Retrieval/RAG) 🔴 NOT STARTED
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| GitHub | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) |
|
||||
| Stars | 33.1k ⭐ |
|
||||
| Integration Effort | 3/5 |
|
||||
| Impact | 4/5 |
|
||||
|
||||
**Proposed integration:**
|
||||
- Local knowledge base for skill references and codebase understanding
|
||||
- Index GENOME.md, README.md, and key architecture files
|
||||
- Query via tool call when agent needs contextual understanding (not just keyword search)
|
||||
- Complements `search_files` without replacing it
|
||||
|
||||
**Blocker:** Requires OpenAI-compatible embedding endpoint. Can use local Ollama via compatibility layer.
|
||||
|
||||
**Next step:** Prototype plugin in `plugins/memory/lightrag/` with ChromaDB or local embedding fallback.
|
||||
|
||||
---
|
||||
|
||||
### P3 — tensorzero (Inference Optimization / LLMOps) 🔴 NOT STARTED
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| GitHub | [tensorzero/tensorzero](https://github.com/tensorzero/tensorzero) |
|
||||
| Stars | 11.2k ⭐ |
|
||||
| Integration Effort | 3/5 |
|
||||
| Impact | 4/5 |
|
||||
|
||||
**Proposed integration:**
|
||||
- Replace custom provider routing, fallback chains, and token tracking
|
||||
- Intelligent routing across providers with cost/quality optimization
|
||||
- Automatic prompt optimization based on feedback
|
||||
- Evaluation metrics for A/B testing model/provider combinations
|
||||
|
||||
**Blocker:** Rust-based infrastructure. Requires careful migration of existing provider logic. Best done as gradual opt-in, not replacement.
|
||||
|
||||
**Next step:** Evaluate tensorzero gateway as optional `providers.tensorzero` backend.
|
||||
|
||||
---
|
||||
|
||||
### P4 — RAGFlow (Retrieval/RAG) 🔴 NOT STARTED
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| GitHub | [infiniflow/ragflow](https://github.com/infiniflow/ragflow) |
|
||||
| Stars | 77.9k ⭐ |
|
||||
| Integration Effort | 4/5 |
|
||||
| Impact | 4/5 |
|
||||
|
||||
**Proposed integration:**
|
||||
- Deploy as local Docker service for document understanding
|
||||
- Ingest technical docs, research papers, codebases
|
||||
- Query via HTTP API when agents need deep document comprehension
|
||||
|
||||
**Blocker:** Heavy deployment (multi-service Docker). Best suited for always-on infrastructure, not per-session.
|
||||
|
||||
**Next step:** Add RAGFlow API client tool in `tools/ragflow_tool.py` for document querying.
|
||||
|
||||
---
|
||||
|
||||
### P5 — n8n (Workflow Automation) 🔴 NOT STARTED
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| GitHub | [n8n-io/n8n](https://github.com/n8n-io/n8n) |
|
||||
| Stars | 183.9k ⭐ |
|
||||
| Integration Effort | 4/5 |
|
||||
| Impact | 5/5 |
|
||||
|
||||
**Proposed integration:**
|
||||
- Orchestrate Hermes agents from external events (webhooks, schedules)
|
||||
- Visual workflow builder for burn loops, PR pipelines, multi-agent chains
|
||||
- n8n webhooks trigger Hermes cron jobs or fleet dispatches
|
||||
|
||||
**Blocker:** Full application stack (Node.js, PostgreSQL, Redis). Deploy as standalone Docker service.
|
||||
|
||||
**Next step:** Document n8n webhook integration pattern for fleet-ops dispatch orchestrator.
|
||||
|
||||
---
|
||||
|
||||
## Honorable Mentions Already in Stack
|
||||
|
||||
| Tool | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| llama.cpp | ✅ Integrated | Via Ollama local inference |
|
||||
| mempalace | ✅ Integrated | Holographic memory system (44.8k ⭐) |
|
||||
|
||||
---
|
||||
|
||||
## Category Breakdown
|
||||
|
||||
### Memory/Context (9 tools evaluated)
|
||||
- Mem0 → **IMPLEMENTED** (cloud + local)
|
||||
- memvid, mempalace, nocturne_memory, rowboat, byterover-cli, letta-code, hindsight, agentic-context-engine → Evaluated, no action
|
||||
|
||||
### Inference Optimization (5 tools evaluated)
|
||||
- llama.cpp → **Already integrated**
|
||||
- vllm, tensorzero, mistral.rs, pruna → Evaluated, no action
|
||||
|
||||
### Retrieval/RAG (5 tools evaluated)
|
||||
- RAGFlow, LightRAG, PageIndex, WeKnora, RAG-Anything → Evaluated, no action
|
||||
|
||||
### Agent Orchestration (5 tools evaluated)
|
||||
- n8n, Langflow, agent-framework, deepagents, multica → Evaluated, no action
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- Source repository: https://github.com/formatho/awesome-ai-tools
|
||||
- Total tools: 414 across 9 categories
|
||||
- Freshness distribution: 🟢 303 | 🟡 49 | 🟠 22 | 🔴 40
|
||||
- Hermes issue: [#842](https://forge.alexanderwhitestone.com/Timmy_Foundation/hermes-agent/issues/842)
|
||||
@@ -56,7 +56,7 @@ VIOLATIONS = [
|
||||
"id": "expanduser-hermes",
|
||||
"name": "os.path.expanduser ~/.hermes (non-fallback)",
|
||||
"pattern": r'os\.path\.expanduser\(["\']~/.hermes',
|
||||
"exclude_with": r'#',
|
||||
"exclude_with": r'#|HERMES_HOME',
|
||||
"message": "Use `os.environ.get('HERMES_HOME', os.path.expanduser('~/.hermes'))` instead",
|
||||
},
|
||||
]
|
||||
|
||||
@@ -13,9 +13,11 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Tuple
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
HERMES_HOME = Path.home() / ".hermes"
|
||||
HERMES_HOME = get_hermes_home()
|
||||
AUDIT_DIR = HERMES_HOME / "audit"
|
||||
|
||||
# Credential patterns to detect and redact
|
||||
@@ -32,14 +34,14 @@ CREDENTIAL_PATTERNS = [
|
||||
(r"bearer\s+[a-zA-Z0-9._-]{20,}", "[REDACTED: Bearer token]"),
|
||||
|
||||
# Generic tokens/passwords
|
||||
(r"(?:token|TOKEN|Token)[:=]\s*["']?[a-zA-Z0-9._-]{20,}["']?", "[REDACTED: Token]"),
|
||||
(r"(?:password|PASSWORD|Password)[:=]\s*["']?[^\s"']{8,}["']?", "[REDACTED: Password]"),
|
||||
(r"(?:secret|SECRET|Secret)[:=]\s*["']?[a-zA-Z0-9._-]{20,}["']?", "[REDACTED: Secret]"),
|
||||
(r"(?:api_key|API_KEY|apiKey|ApiKey)[:=]\s*["']?[a-zA-Z0-9._-]{20,}["']?", "[REDACTED: API key]"),
|
||||
("(?:token|TOKEN|Token)[:=]\\s*['\"]?[a-zA-Z0-9._-]{20,}['\"]?", "[REDACTED: Token]"),
|
||||
("(?:password|PASSWORD|Password)[:=]\\s*['\"]?[^\\s\"']{8,}['\"]?", "[REDACTED: Password]"),
|
||||
("(?:secret|SECRET|Secret)[:=]\\s*['\"]?[a-zA-Z0-9._-]{20,}['\"]?", "[REDACTED: Secret]"),
|
||||
("(?:api_key|API_KEY|apiKey|ApiKey)[:=]\\s*['\"]?[a-zA-Z0-9._-]{20,}['\"]?", "[REDACTED: API key]"),
|
||||
|
||||
# AWS keys
|
||||
(r"AKIA[0-9A-Z]{16}", "[REDACTED: AWS access key]"),
|
||||
(r"(?:aws_secret_access_key|AWS_SECRET_ACCESS_KEY)[:=]\s*["']?[a-zA-Z0-9/+=]{40}["']?", "[REDACTED: AWS secret]"),
|
||||
("(?:aws_secret_access_key|AWS_SECRET_ACCESS_KEY)[:=]\\s*['\"]?[a-zA-Z0-9/+=]{40}['\"]?", "[REDACTED: AWS secret]"),
|
||||
|
||||
# Private keys
|
||||
(r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----", "[REDACTED: Private key header]"),
|
||||
|
||||
@@ -249,7 +249,8 @@ def detect_crisis(text: str) -> CrisisDetectionResult:
|
||||
# ── Escalation Logging ────────────────────────────────────────────────────
|
||||
|
||||
BRIDGE_URL = os.environ.get("CRISIS_BRIDGE_URL", "")
|
||||
LOG_PATH = os.path.expanduser("~/.hermes/crisis_escalations.jsonl")
|
||||
_HERMES_HOME = os.environ.get("HERMES_HOME")
|
||||
LOG_PATH = os.path.join(_HERMES_HOME or os.path.expanduser("~/.hermes"), "crisis_escalations.jsonl")
|
||||
|
||||
|
||||
def _log_escalation(result: CrisisDetectionResult, text_preview: str = ""):
|
||||
|
||||
@@ -10,10 +10,10 @@ Usage:
|
||||
from tools.hardcoded_path_guard import check_path, validate_tool_args
|
||||
|
||||
# Check a single path
|
||||
err = check_path("/Users/apayne/.hermes/config.yaml")
|
||||
err = check_path("/Users/apayne/.hermes/config.yaml") # noqa: hardcoded-path-ok
|
||||
|
||||
# Validate all path-like args in a tool call
|
||||
clean_args, warnings = validate_tool_args("read_file", {"path": "/home/user/file.txt"})
|
||||
clean_args, warnings = validate_tool_args("read_file", {"path": "/home/user/file.txt"}) # noqa: hardcoded-path-ok
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
@@ -14,9 +14,11 @@ from typing import Dict, List, Optional, Any
|
||||
from dataclasses import dataclass, asdict, field
|
||||
from enum import Enum
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TEMPLATE_DIR = Path.home() / ".hermes" / "session-templates"
|
||||
TEMPLATE_DIR = get_hermes_home() / "session-templates"
|
||||
|
||||
|
||||
class TaskType(Enum):
|
||||
@@ -106,7 +108,7 @@ class Templates:
|
||||
return TaskType.MIXED
|
||||
|
||||
def extract(self, session_id, max_n=10):
|
||||
db = Path.home() / ".hermes" / "state.db"
|
||||
db = get_hermes_home() / "state.db"
|
||||
if not db.exists():
|
||||
return []
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user