Timmy_Foundation/timmy-config
16
0
Fork 0
Code Issues 57 Pull Requests 1 Actions 1 Packages Projects Releases Wiki Activity

2.4 - Wolf leaderboard to routing table #213

New Issue
Closed
opened 2026-04-05 20:07:26 +00:00 by Timmy · 1 comment
Timmy commented 2026-04-05 20:07:26 +00:00
Owner
Copy Link

wolf-scores.json generated at ~/.hermes/wolf-scores.json with 3 models:

  • google/gemini-2.5-flash (score=1.0, 9.5s, tier=1)
  • google/gemini-3-flash-preview (score=1.0, 4.69s, tier=1)
  • google/gemini-2.5-flash-lite (score=1.0, 1.18s, tier=1)

Format: {{model: {{score, latency_seconds, tier, success}}}}
Ready for smart_model_routing consumption.

NOTE: Only 3 models evaluated — wolf has only run first-run.json. More wolf evaluations needed for comprehensive routing.

wolf-scores.json generated at ~/.hermes/wolf-scores.json with 3 models: - google/gemini-2.5-flash (score=1.0, 9.5s, tier=1) - google/gemini-3-flash-preview (score=1.0, 4.69s, tier=1) - google/gemini-2.5-flash-lite (score=1.0, 1.18s, tier=1) Format: {{model: {{score, latency_seconds, tier, success}}}} Ready for smart_model_routing consumption. NOTE: Only 3 models evaluated — wolf has only run first-run.json. More wolf evaluations needed for comprehensive routing.
Timmy closed this issue 2026-04-05 23:18:19 +00:00
Timmy commented 2026-04-05 23:18:19 +00:00
Author
Owner
Copy Link

Shipped

~/.hermes/wolf-scores.json created with 3 evaluated models.

All 3 scored 1.0 (perfect) but only 3 models total from first-run.json. Wolf needs to evaluate more models for useful differentiation. The output format is ready for smart_model_routing to consume.

Output format

{
  "google/gemini-2.5-flash": {"score": 1.0, "latency_seconds": 9.5, "tier": 1, "success": true},
  "google/gemini-3-flash-preview": {"score": 1.0, "latency_seconds": 4.69, "tier": 1, "success": true},
  "google/gemini-2.5-flash-lite": {"score": 1.0, "latency_seconds": 1.18, "tier": 1, "success": true}
}

Next: assign more models to wolf for evaluation. Currently only 3 gemini variants — not enough for smart routing across the fleet.

## Shipped `~/.hermes/wolf-scores.json` created with 3 evaluated models. All 3 scored 1.0 (perfect) but only 3 models total from first-run.json. Wolf needs to evaluate more models for useful differentiation. The output format is ready for smart_model_routing to consume. ### Output format ```json { "google/gemini-2.5-flash": {"score": 1.0, "latency_seconds": 9.5, "tier": 1, "success": true}, "google/gemini-3-flash-preview": {"score": 1.0, "latency_seconds": 4.69, "tier": 1, "success": true}, "google/gemini-2.5-flash-lite": {"score": 1.0, "latency_seconds": 1.18, "tier": 1, "success": true} } ``` ### Next: assign more models to wolf for evaluation. Currently only 3 gemini variants — not enough for smart routing across the fleet.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-kimi
Dispatched to Kimi via OpenClaw
 claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 enhancement
epic
kimi-done
Kimi completed this task
 kimi-in-progress
Kimi is actively working on this
 research
velocity-engine
Auto-generated by velocity engine
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
KimiClaw Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok groq hermes kimi manus perplexity
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#213
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?