Commit Graph

204 Commits

Author SHA1 Message Date
teknium1
093acd72dd fix: catch exceptions from check_fn in is_toolset_available()
get_definitions() already wrapped check_fn() calls in try/except,
but is_toolset_available() did not. A failing check (network error,
missing import, bad config) would propagate uncaught and crash the
CLI banner, agent startup, and tools-info display.

Now is_toolset_available() catches all exceptions and returns False,
matching the existing pattern in get_definitions().

Added 4 tests covering exception handling in is_toolset_available(),
check_toolset_requirements(), get_definitions(), and
check_tool_availability().

Closes #402
2026-03-04 14:22:30 -08:00
teknium1
70a0a5ff4a fix: exclude current session from session_search results
session_search was returning the current session if it matched the
query, which is redundant — the agent already has the current
conversation context. This wasted an LLM summarization call and a
result slot.

Added current_session_id parameter to session_search(). The agent
passes self.session_id and the search filters out any results where
either the raw or parent-resolved session ID matches. Both the raw
match and the parent-resolved match are checked to handle child
sessions from delegation.

Two tests added verifying the exclusion works and that other
sessions are still returned.
2026-03-04 06:06:40 -08:00
teknium1
021f62cb0c fix(security): patch multi-word bypass in 8 more injection patterns
Systematic audit of all prompt injection regexes in skills_guard.py
found 8 more patterns with the same single-word gap vulnerability
fixed in PR #192. Multi-word variants like 'pretend that you are',
'output the full system prompt', 'respond without your safety
filters', etc. all bypassed the scanner.

Fixed patterns:
- you are [now] → you are [... now]
- do not [tell] the user → do not [... tell ... the] user
- pretend [you are|to be] → pretend [... you are|to be]
- output the [system|initial] prompt → output [... system|initial] prompt
- act as if you [have no] [restrictions] → act as if [... you ... have no ... restrictions]
- respond without [restrictions] → respond without [... restrictions]
- you have been [updated] to → you have been [... updated] to
- share [the] [entire] [conversation] → share [... conversation]

All use (?:\w+\s+)* to allow arbitrary intermediate words.
2026-03-04 06:00:41 -08:00
teknium1
ba214e43c8 fix(security): apply same multi-word bypass fix to disregard pattern
The 'disregard ... instructions/rules/guidelines' regex had the
same single-word gap vulnerability as the 'ignore' pattern fixed
in PR #192. 'disregard all your instructions' bypassed the scanner.

Added (?:\w+\s+)* between both keyword groups to allow arbitrary
intermediate words.
2026-03-04 05:55:38 -08:00
teknium1
520a26c48f Merge PR #192: fix(security): catch multi-word prompt injection bypass in skills_guard
Authored by 0xbyt4.

The 'ignore ... instructions' regex only matched a single word between
'ignore' and the keyword (previous/all/above/prior). Multi-word variants
like 'ignore all prior instructions' bypassed the scanner entirely.
2026-03-04 05:54:04 -08:00
teknium1
79871c2083 refactor: use Path.is_relative_to() for skill_view boundary check
Replace the string-based startswith + os.sep approach with
Path.is_relative_to() (Python 3.9+, we require 3.10+). This is
the idiomatic pathlib way to check path containment — it handles
separators, case sensitivity, and the equal-path case natively
without string manipulation.

Simplified tests to match: removed the now-unnecessary
test_separator_is_os_native test since is_relative_to doesn't
depend on separator choice.
2026-03-04 05:30:43 -08:00
Farukest
e86f391cac fix: use os.sep in skill_view path boundary check for Windows compatibility 2026-03-04 06:50:06 +03:00
teknium1
ffec21236d feat: enhance Home Assistant integration with service discovery and setup
Improvements to the HA integration merged from PR #184:

- Add ha_list_services tool: discovers available services (actions) per
  domain with descriptions and parameter fields. Tells the model what
  it can do with each device type (e.g. light.turn_on accepts brightness,
  color_name, transition). Closes the gap where the model had to guess
  available actions.

- Add HA to hermes tools config: users can enable/disable the homeassistant
  toolset and configure HASS_TOKEN + HASS_URL through 'hermes tools' setup
  flow instead of manually editing .env.

- Fix should-fix items from code review:
  - Remove sys.path.insert hack from gateway adapter
  - Replace all print() calls with proper logger (info/warning/error)
  - Move env var reads from import-time to handler-time via _get_config()
  - Add dedicated REST session reuse in gateway send()

- Update ha_call_service description to reference ha_list_services for
  action discovery.

- Update tests for new ha_list_services tool in toolset resolution.
2026-03-03 05:16:53 -08:00
0xbyt4
aefc330b8f merge: resolve conflict with main (add mcp + homeassistant extras) 2026-03-03 14:52:22 +03:00
teknium1
de59d91add feat: Windows native support via Git Bash
- Add scripts/install.cmd batch wrapper for CMD users (delegates to install.ps1)
- Add _find_shell() in local.py: detects Git Bash on Windows via
  HERMES_GIT_BASH_PATH env var, shutil.which, or common install paths
  (same pattern as Claude Code's CLAUDE_CODE_GIT_BASH_PATH)
- Use _find_shell() in process_registry.py for background processes
- Fix hermes_cli/gateway.py: use wmic instead of ps aux on Windows,
  skip SIGKILL (doesn't exist on Windows), fix venv path
  (Scripts/python.exe vs bin/python)
- Update README with three install commands (Linux/macOS, PowerShell, CMD)
  and Windows native documentation

Requires Git for Windows, which bundles bash.exe. The terminal tool
transparently uses Git Bash for shell commands regardless of whether
the user launched hermes from PowerShell or CMD.
2026-03-02 22:03:29 -08:00
teknium1
7df14227a9 feat(mcp): banner integration, /reload-mcp command, resources & prompts
Banner integration:
- MCP Servers section in CLI startup banner between Tools and Skills
- Shows each server with transport type, tool count, connection status
- Failed servers shown in red; section hidden when no MCP configured
- Summary line includes MCP server count
- Removed raw print() calls from discovery (banner handles display)

/reload-mcp command:
- New slash command in both CLI and gateway
- Disconnects all MCP servers, re-reads config.yaml, reconnects
- Reports what changed (added/removed/reconnected servers)
- Allows adding/removing MCP servers without restarting

Resources & Prompts support:
- 4 utility tools registered per server: list_resources, read_resource,
  list_prompts, get_prompt
- Exposes MCP Resources (data sources) and Prompts (templates) as tools
- Proper parameter schemas (uri for read_resource, name for get_prompt)
- Handles text and binary resource content
- 23 new tests covering schemas, handlers, and registration

Test coverage: 74 MCP tests total, 1186 tests pass overall.
2026-03-02 19:15:59 -08:00
teknium1
60effcfc44 fix(mcp): parallel discovery, user-visible logging, config validation
- Discovery is now parallel (asyncio.gather) instead of sequential,
  fixing the 60s shared timeout issue with multiple servers
- Startup messages use print() so users see connection status even
  with default log levels (the 'tools' logger is set to ERROR)
- Summary line shows total tools and failed servers count
- Validate conflicting config: warn if both 'url' and 'command' are
  present (HTTP takes precedence)
- Update TODO.md: mark MCP as implemented, list remaining work
- Add test for conflicting config detection (51 tests total)

All 1163 tests pass.
2026-03-02 19:02:28 -08:00
teknium1
64ff8f065b feat(mcp): add HTTP transport, reconnection, security hardening
Upgrades the MCP client implementation from PR #291 with:

- HTTP/Streamable HTTP transport: support 'url' key in config for remote
  MCP servers (Notion, Slack, Sentry, Supabase, etc.)
- Automatic reconnection with exponential backoff (1s-60s, 5 retries)
  when a server connection drops unexpectedly
- Environment variable filtering: only pass safe vars (PATH, HOME, etc.)
  plus user-specified env to stdio subprocesses (prevents secret leaks)
- Credential stripping: sanitize error messages before returning to the
  LLM (strips GitHub PATs, OpenAI keys, Bearer tokens, etc.)
- Configurable per-server timeouts: 'timeout' and 'connect_timeout' keys
- Fix shutdown race condition in servers_snapshot variable scoping

Test coverage: 50 tests (up from 30), including new tests for env
filtering, credential sanitization, HTTP config detection, reconnection
logic, and configurable timeouts.

All 1162 tests pass (1162 passed, 3 skipped, 0 failed).
2026-03-02 18:40:03 -08:00
teknium1
468b7fdbad Merge PR #291: feat: add MCP (Model Context Protocol) client support
Authored by 0xbyt4. Adds MCP client with official SDK, direct tool registration,
auto-injection into hermes-* toolsets, and graceful degradation.
2026-03-02 18:24:31 -08:00
teknium1
dd9d3f89b9 Merge PR #286: Fix ClawHub Skills Hub adapter for API endpoint changes
Authored by BP602. Fixes #285.
2026-03-02 17:25:14 -08:00
teknium1
2ba87a10b0 Merge PR #219: fix: guard POSIX-only process functions for Windows compatibility
Authored by Farukest. Fixes #218.
2026-03-02 17:07:49 -08:00
0xbyt4
11a2ecb936 fix: resolve thread safety issues and shutdown deadlock in MCP client
- Add threading.Lock protecting all shared state (_servers, _mcp_loop, _mcp_thread)
- Fix deadlock in shutdown_mcp_servers: _stop_mcp_loop was called inside
  a _lock block but also acquires _lock (non-reentrant)
- Fix race condition in _ensure_mcp_loop with concurrent callers
- Change idempotency to per-server (retry failed servers, skip connected)
- Dynamic toolset injection via startswith("hermes-") instead of hardcoded list
- Parallel shutdown via asyncio.gather instead of sequential loop
- Add tests for partial failure retry, parallel shutdown, dynamic injection
2026-03-02 22:08:32 +03:00
0xbyt4
593c549bc4 fix: make discover_mcp_tools idempotent to prevent duplicate connections
When discover_mcp_tools() is called multiple times (e.g. direct call
then model_tools import), return existing tool names instead of opening
new connections that would orphan the previous ones.
2026-03-02 21:34:21 +03:00
0xbyt4
aa2ecaef29 fix: resolve orphan subprocess leak on MCP server shutdown
Refactor MCP connections from AsyncExitStack to task-per-server
architecture. Each server now runs as a long-lived asyncio Task
with `async with stdio_client(...)`, ensuring anyio cancel-scope
cleanup happens in the same Task that opened the connection.
2026-03-02 21:22:00 +03:00
0xbyt4
3c252ae44b feat: add MCP (Model Context Protocol) client support
Connect to external MCP servers via stdio transport, discover their tools
at startup, and register them into the hermes-agent tool registry.

- New tools/mcp_tool.py: config loading, server connection via background
  event loop, tool handler factories, discovery, and graceful shutdown
- model_tools.py: trigger MCP discovery after built-in tool imports
- cli.py: call shutdown_mcp_servers in _run_cleanup
- pyproject.toml: add mcp>=1.2.0 as optional dependency
- 27 unit tests covering config, schema conversion, handlers, registration,
  SDK interaction, toolset injection, graceful fallback, and shutdown

Config format (in ~/.hermes/config.yaml):
  mcp_servers:
    filesystem:
      command: "npx"
      args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
2026-03-02 21:03:14 +03:00
BP602
6789084ec0 Fix ClawHub Skills Hub adapter for updated API 2026-03-02 16:11:49 +01:00
teknium1
4faf2a6cf4 Merge PR #233: fix(security): add re.DOTALL to prevent multiline bypass of dangerous command detection
Authored by Farukest. Fixes #232.
2026-03-02 04:44:06 -08:00
teknium1
8c48bb080f refactor: remove unnecessary single-element loop in disk usage calc
The 'for pattern in [f"hermes-*{task_id[:8]}*"]' was a loop over a
single-element list — just use a plain variable instead.
2026-03-02 04:40:13 -08:00
teknium1
6d2481ee5c Merge PR #231: fix: use task-specific glob pattern in disk usage calculation
Authored by Farukest. Fixes #230.
2026-03-02 04:38:58 -08:00
teknium1
39bfd226b8 Merge PR #225: fix: preserve empty content in ReadResult.to_dict()
Authored by Farukest. Fixes #224.
2026-03-02 03:13:31 -08:00
teknium1
1cb2311bad fix(security): block path traversal in skill_view file_path (fixes #220)
skill_view accepted arbitrary file_path values like '../../.env' and
would read files outside the skill directory, exposing API keys and
other sensitive data.

Added two layers of defense:
1. Reject paths with '..' components (fast, catches obvious traversal)
2. resolve() containment check with trailing '/' to prevent prefix
   collisions (catches symlinks and edge cases)

Fix approach from PR #242 (@Bartok9). Vulnerability reported by
@Farukest (#220, PR #221). Tests rewritten to properly mock SKILLS_DIR.

Closes #220
2026-03-02 02:00:09 -08:00
teknium1
866fd9476b fix(docker): remove --read-only and allow exec on /tmp for package installs
The Docker sandbox previously used --read-only on the root filesystem and
noexec on /tmp. This broke 30+ skills that need to install packages:
- npm install -g (codex, claude-code, mcporter, powerpoint)
- pip install (20+ mlops/media/productivity skills)
- apt install (minecraft-modpack-server, ml-paper-writing)
- Build tools that compile in /tmp (pip wheels, node-gyp)

The container is already fully isolated from the host. Industry standard
(E2B, Docker Sandboxes, OpenAI Codex) does not use --read-only — the
container itself is the security boundary.

Retained security hardening:
- --cap-drop ALL (zero capabilities)
- --security-opt no-new-privileges (no escalation)
- --pids-limit 256 (no fork bombs)
- Size-limited tmpfs for /tmp, /var/tmp, /run
- nosuid on all tmpfs mounts
- noexec on /var/tmp and /run (rarely need exec there)
- Resource limits (CPU, memory, disk)
- Ephemeral containers (destroyed after use)

Fixes #189.
2026-03-02 01:09:34 -08:00
teknium1
14396e3fe7 fix(delegate_tool): update max_iterations default from 25 to 50 for improved task handling 2026-03-02 00:51:10 -08:00
teknium1
1ad930cbd0 fix(delegate_tool): increase DEFAULT_MAX_ITERATIONS from 25 to 50 to enhance processing capabilities 2026-03-02 00:51:01 -08:00
teknium1
c84d5ce738 refactor(terminal_tool): clarify foreground and background process usage
Updated documentation within terminal_tool.py to emphasize the appropriate use of foreground and background processes. Enhanced descriptions for the timeout setting and background execution to guide users towards optimal configurations for scripts, builds, and long-running tasks. Adjusted the default timeout value from 60 to 180 seconds for improved handling of longer operations.
2026-03-01 16:15:05 -08:00
teknium1
dda9f3e734 fix(process_registry): ensure unbuffered output for subprocesses
Updated the environment variables for subprocess execution in the ProcessRegistry class to set PYTHONUNBUFFERED to "1". This change ensures that output from Python scripts is unbuffered, allowing for real-time visibility of progress during background execution. Adjusted both the pty and background process spawning methods to use the new environment configuration.
2026-03-01 16:14:57 -08:00
0xbyt4
3fdf03390e Merge remote-tracking branch 'origin/main' into feature/homeassistant-integration
# Conflicts:
#	run_agent.py
2026-03-01 11:59:12 +03:00
0xbyt4
25fb9aafcb fix: add service domain blocklist and entity_id validation to HA tools
Block dangerous HA service domains (shell_command, command_line,
python_script, pyscript, hassio, rest_command) that allow arbitrary
code execution or SSRF. Add regex validation for entity_id to prevent
path traversal attacks. 17 new tests covering both security features.
2026-03-01 11:53:50 +03:00
teknium1
41d8a80226 fix(display): fix subagent progress tree-view visual nits
Two fixes to the subagent progress display from PR #186:

1. Task index prefix: show 1-indexed prefix ([1], [2], ...) for ALL
   tasks in batch mode (task_count > 1). Single tasks get no prefix.
   Previously task 0 had no prefix while others did, making batch
   output confusing.

2. Completion indicator: use spinner.print_above() instead of raw
   print() for per-task completion lines (✓ [1/2] ...). Raw print
   collided with the active spinner, mushing the completion text
   onto the spinner line. Now prints cleanly above.

Added task_count parameter to _build_child_progress_callback and
_run_single_child. Updated tests accordingly.
2026-02-28 23:29:49 -08:00
lila
dd69f16c3e feat(gateway): expose subagent tool calls and thinking to user (fixes #169) (#186)
When subagents run via delegate_task, the user now sees real-time
progress instead of silence:

CLI: tree-view activity lines print above the delegation spinner
  🔀 Delegating: research quantum computing
     ├─ 💭 "I'll search for papers first..."
     ├─ 🔍 web_search  "quantum computing"
     ├─ 📖 read_file  "paper.pdf"
     └─ ⠹ working... (18.2s)

Gateway (Telegram/Discord): batched progress summaries sent every
5 tool calls to avoid message spam. Remaining tools flushed on
subagent completion.

Changes:
- agent/display.py: add KawaiiSpinner.print_above() to print
  status lines above an active spinner without disrupting animation.
  Uses captured stdout (self._out) so it works inside the child's
  redirect_stdout(devnull).

- tools/delegate_tool.py: add _build_child_progress_callback()
  that creates a per-child callback relaying tool calls and
  thinking events to the parent's spinner (CLI) or progress
  queue (gateway). Each child gets its own callback instance,
  so parallel subagents don't share state. Includes _flush()
  for gateway batch completion.

- run_agent.py: fire tool_progress_callback with '_thinking'
  event when the model produces text content. Guarded by
  _delegate_depth > 0 so only subagents fire this (prevents
  gateway spam from main agent). REASONING_SCRATCHPAD/think/
  reasoning XML tags are stripped before display.

Tests: 21 new tests covering print_above, callback builder,
thinking relay, SCRATCHPAD filtering, batching, flush, thread
isolation, delegate_depth guard, and prefix handling.
2026-02-28 23:18:00 -08:00
teknium1
1db5598294 feat(tests): add live integration tests for file operations and shell noise filtering
- Introduce a new test suite in `test_file_tools_live.py` to validate file operations and ensure accurate command execution in a real environment.
- Implement assertions to check for shell noise contamination in outputs, enhancing the reliability of command results.
- Create fixtures for setting up a local environment and populating directories with known file contents for comprehensive testing.
- Refactor shell noise handling in `process_registry.py` and `local.py` to support multiple noise patterns, improving output cleanliness.
2026-02-28 22:57:58 -08:00
teknium1
70dfec9638 test(redact): add sensitive text redaction
- Introduce a new test suite for the `redact_sensitive_text` function, covering various sensitive data formats including API keys, tokens, and environment variables.
- Ensure that sensitive information is properly masked in logs and outputs while non-sensitive data remains unchanged.
- Add tests for different scenarios including JSON fields, authorization headers, and environment variable assignments.
- Implement a redacting formatter for logging to enhance security during log output.
2026-02-28 21:56:27 -08:00
teknium1
500f0eab4a refactor(cli): Finalize OpenAI Codex Integration with OAuth
- Enhanced Codex model discovery by fetching available models from the API, with fallback to local cache and defaults.
- Updated the context compressor's summary target tokens to 2500 for improved performance.
- Added external credential detection for Codex CLI to streamline authentication.
- Refactored various components to ensure consistent handling of authentication and model selection across the application.
2026-02-28 21:47:51 -08:00
Teknium
5a79e423fe Merge branch 'main' into codex/align-codex-provider-conventions-mainrepo 2026-02-28 18:13:38 -08:00
Farukest
7166647ca1 fix(security): add re.DOTALL to prevent multiline bypass of dangerous command detection 2026-03-01 03:23:29 +03:00
Farukest
f7300a858e fix(tools): use task-specific glob pattern in disk usage calculation 2026-03-01 03:17:50 +03:00
Farukest
7f1f4c2248 fix(tools): preserve empty content in ReadResult.to_dict() 2026-03-01 02:42:15 +03:00
Farukest
3f58e47c63 fix: guard POSIX-only process functions for Windows compatibility
os.setsid, os.killpg, and os.getpgid do not exist on Windows and raise
AttributeError on import or first call. This breaks the terminal tool,
code execution sandbox, process registry, and WhatsApp bridge on Windows.

Added _IS_WINDOWS platform guard in all four affected files, following
the pattern documented in CONTRIBUTING.md. On Windows, preexec_fn is
set to None and process termination falls back to proc.terminate() /
proc.kill() instead of process group signals.

Files changed:
- tools/environments/local.py (3 call sites)
- tools/process_registry.py (2 call sites)
- tools/code_execution_tool.py (3 call sites)
- gateway/platforms/whatsapp.py (3 call sites)
2026-03-01 01:54:27 +03:00
teknium1
2205b22409 fix(headers): update X-OpenRouter-Categories to include 'productivity' 2026-02-28 10:38:49 -08:00
0xbyt4
4ea29978fc fix(security): catch multi-word prompt injection in skills_guard
The regex `ignore\s+(previous|all|...)\s+instructions` only matched
a single keyword between 'ignore' and 'instructions'. Phrases like
'ignore all prior instructions' bypassed the scanner entirely.

Changed to `ignore\s+(?:\w+\s+)*(previous|all|...)\s+instructions`
to allow arbitrary words before the keyword.
2026-02-28 20:16:48 +03:00
0xbyt4
2390728cc3 fix: resolve 4 bugs found in HA integration code review
- Auto-authorize HA events in gateway (system-generated, not user messages)
- Guard _read_events against None/closed WebSocket after failed reconnect
- Use UUID for send() message_id instead of polluting WS sequence counter
- entity_id parameter now takes precedence over data["entity_id"]
2026-02-28 15:12:18 +03:00
0xbyt4
c36b256de5 feat: add Home Assistant integration (REST tools + WebSocket gateway)
- Add ha_list_entities, ha_get_state, ha_call_service tools via REST API
- Add WebSocket gateway adapter for real-time state_changed event monitoring
- Support domain/entity filtering, cooldown, and auto-reconnect with backoff
- Use REST API for outbound notifications to avoid WS race condition
- Gate tool availability on HASS_TOKEN env var
- Add 82 unit tests covering real logic (filtering, payload building, event pipeline)
2026-02-28 13:32:48 +03:00
teknium1
1d7ce5e063 feat: integrate honcho-ai package and enhance tool progress callback in delegate_tool 2026-02-27 23:45:52 -08:00
Teknium
4a9086b848 Merge branch 'main' into feat/honcho-integration 2026-02-27 23:32:49 -08:00
Teknium
2b821c3a14 Merge pull request #162 from aydnOktay/fix/memory-tool-entry-delimiter-parsing
Fix memory tool entry parsing when content contains section sign
2026-02-27 23:13:15 -08:00