Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

[Infra] Hermes Agent local model routing — custom_providers ignored, always routes to Anthropic #950

New Issue
Closed
opened 2026-03-22 17:36:45 +00:00 by Timmy · 1 comment
Timmy commented 2026-03-22 17:36:45 +00:00
Owner
Copy Link

Problem

Hermes agent CLI always routes to Anthropic when an Anthropic API key exists in auth.json, even when:

  • custom_providers is configured in config.yaml with a local Ollama endpoint
  • HERMES_INFERENCE_PROVIDER=custom env var is set
  • OPENAI_BASE_URL=http://localhost:11434/v1 env var is set
  • -m hermes3:latest is passed (a model that only exists locally)

The --provider flag only accepts hardcoded built-in providers, no custom or ollama option.

Impact

Cannot run hermes agent sessions on local models. All development burns Anthropic credits even for tasks a local 8B model could handle. Blocks the two-tier brain architecture where local models handle reflexes and cloud handles strategy.

Benchmark Data (M3 Max, March 22 2026)

Model Latency tok/s Code Quality
hermes3:latest (8B) 4.4s 57 GOOD
llama3.1:latest (8B) 3.6s 54 GOOD
llama3.2:latest (3B) 2.0s 109 GOOD
qwen2.5:14b 6.0s 30 GOOD
qwen3:30b 12.1s 71 EMPTY (thinking mode)

Desired Behavior

  • hermes chat -m hermes3:latest --provider custom routes to configured Ollama endpoint
  • OR: hermes chat -m hermes3:latest --base-url http://localhost:11434/v1 overrides provider
  • OR: when model name matches a custom_providers entry, auto-route there

Workaround Attempted

  • OpenClaw: import errors, depends on CMDOP cloud platform
  • Env vars: ignored when Anthropic auth exists
  • Custom providers config: configured but not used by CLI

Files

  • ~/.hermes/hermes-agent/hermes_cli/runtime_provider.py — resolver logic
  • ~/.hermes/config.yaml — custom_providers section
## Problem Hermes agent CLI always routes to Anthropic when an Anthropic API key exists in auth.json, even when: - `custom_providers` is configured in config.yaml with a local Ollama endpoint - `HERMES_INFERENCE_PROVIDER=custom` env var is set - `OPENAI_BASE_URL=http://localhost:11434/v1` env var is set - `-m hermes3:latest` is passed (a model that only exists locally) The `--provider` flag only accepts hardcoded built-in providers, no `custom` or `ollama` option. ## Impact Cannot run hermes agent sessions on local models. All development burns Anthropic credits even for tasks a local 8B model could handle. Blocks the two-tier brain architecture where local models handle reflexes and cloud handles strategy. ## Benchmark Data (M3 Max, March 22 2026) | Model | Latency | tok/s | Code Quality | |-------|---------|-------|--------------| | hermes3:latest (8B) | 4.4s | 57 | GOOD | | llama3.1:latest (8B) | 3.6s | 54 | GOOD | | llama3.2:latest (3B) | 2.0s | 109 | GOOD | | qwen2.5:14b | 6.0s | 30 | GOOD | | qwen3:30b | 12.1s | 71 | EMPTY (thinking mode) | ## Desired Behavior - `hermes chat -m hermes3:latest --provider custom` routes to configured Ollama endpoint - OR: `hermes chat -m hermes3:latest --base-url http://localhost:11434/v1` overrides provider - OR: when model name matches a custom_providers entry, auto-route there ## Workaround Attempted - OpenClaw: import errors, depends on CMDOP cloud platform - Env vars: ignored when Anthropic auth exists - Custom providers config: configured but not used by CLI ## Files - `~/.hermes/hermes-agent/hermes_cli/runtime_provider.py` — resolver logic - `~/.hermes/config.yaml` — custom_providers section
claude was assigned by Rockachopa 2026-03-22 23:31:28 +00:00
claude added the rejected-direction label 2026-03-23 13:51:25 +00:00
perplexity commented 2026-03-23 13:52:17 +00:00
Collaborator
Copy Link

🧹 Closed — Superseded

The custom_providers routing issue is superseded by the new metabolic cascade router (#966, #1075) which implements proper three-tier local model routing.

Ref: Deep Backlog Triage #1076. Reopen if needed.

🧹 **Closed — Superseded** The custom_providers routing issue is superseded by the new metabolic cascade router (#966, #1075) which implements proper three-tier local model routing. Ref: Deep Backlog Triage #1076. Reopen if needed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label rejected-direction
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees claude
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#950
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?