Timmy_Foundation/hermes-agent
15
0
Fork 0
Code Issues 172 Pull Requests 122 Actions 15 Packages Projects Releases Wiki Activity

Poka-yoke: Detect and block tool hallucination before API calls #294

New Issue
Closed
opened 2026-04-11 20:39:24 +00:00 by Rockachopa · 0 comments
Rockachopa commented 2026-04-11 20:39:24 +00:00
Owner
Copy Link

Poka-Yoke Principle: Detection & Shutdown

Category: Tool Hallucination Prevention
Severity: High (model calls non-existent tools, wasting API budget)

Problem

When a model references a tool that doesn't exist in the current toolset (e.g., calling browser_navigate when the browser toolset is disabled), the agent wastes an API call and produces confusing error messages. Tool schema cross-references make this worse.

Error-Proofing Approach

Poka-yoke says: detect the error state and prevent the failure rather than handling it after.

Acceptance Criteria

  1. Add a pre-flight validation in handle_function_call() that checks if the requested tool exists in the active toolset BEFORE making the API call
  2. When a model calls a non-existent tool, return a structured error including: "Available tools: [list]" so the model can self-correct
  3. Generalize the tool_schema_post_processor pattern: strip cross-tool references from schema descriptions when the referenced tool is not active
  4. Log hallucinated tool calls separately for analysis

References

  • AGENTS.md: "DO NOT hardcode cross-tool references in schema descriptions"
  • model_tools.py: get_tool_definitions() post-processing blocks
  • tools/registry.py: central tool registry
## Poka-Yoke Principle: Detection & Shutdown **Category:** Tool Hallucination Prevention **Severity:** High (model calls non-existent tools, wasting API budget) ### Problem When a model references a tool that doesn't exist in the current toolset (e.g., calling `browser_navigate` when the browser toolset is disabled), the agent wastes an API call and produces confusing error messages. Tool schema cross-references make this worse. ### Error-Proofing Approach Poka-yoke says: **detect the error state and prevent the failure** rather than handling it after. ### Acceptance Criteria 1. [ ] Add a pre-flight validation in `handle_function_call()` that checks if the requested tool exists in the active toolset BEFORE making the API call 2. [ ] When a model calls a non-existent tool, return a structured error including: "Available tools: [list]" so the model can self-correct 3. [ ] Generalize the `tool_schema_post_processor` pattern: strip cross-tool references from schema descriptions when the referenced tool is not active 4. [ ] Log hallucinated tool calls separately for analysis ### References - AGENTS.md: "DO NOT hardcode cross-tool references in schema descriptions" - `model_tools.py`: `get_tool_definitions()` post-processing blocks - `tools/registry.py`: central tool registry
Rockachopa added the poka-yoke label 2026-04-11 20:39:24 +00:00
codex-agent was assigned by Rockachopa 2026-04-11 20:58:16 +00:00
Timmy closed this issue 2026-04-12 22:55:42 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
CI
Continuous integration, runners, workflow issues
 QA
Quality assurance, testing, production audit
 assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-kimi
Task assigned to KimiClaw for processing
 batch-pipeline
batch-pipeline label
 blocked
Blocked by external dependency or merge conflict
 claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 duplicate
Duplicate of another issue
 epic
Epic - large feature with multiple sub-tasks
 epic:memory
Unified Memory Architecture epic
 gaming
Gaming agent capabilities
 infra
Infrastructure, VPS, DNS, deployment
 kimi-done
KimiClaw has completed this task
 kimi-in-progress
KimiClaw is actively working on this
 mcp
MCP (Model Context Protocol) tools & servers
 morrowind
Morrowind Agent gameplay & MCP integration
 needs-review
PR or issue requires reviewer sign-off before merge/close
 p0-critical
p1-important
phase:1-activate
Phase 1: Activate existing systems
 phase:2-operational
Phase 2: Operational memory bridge
 phase:3-episodic
Phase 3: Proactive episodic recall
 phase:4-hygiene
Phase 4: Memory hygiene and quality
 phase:5-integration
Phase 5: Integration and testing
 poka-yoke
research
security
Security hardening, vulnerability fixes
 shared-infra
Shared infrastructure across projects (Playground + LPM)
 stale
No activity, pending triage or closure
 throughput-10x
throughput-10x label
 token-masterplan
token-masterplan label
 tool-evaluation
velocity-engine
Auto-generated by velocity engine
 wont-fix
Closed as intentionally not fixing — explicit descope
No Label poka-yoke
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok hermes kimi manus perplexity sonnet
No Assignees codex-agent
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/hermes-agent#294
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?