[claude] Add desktop automation primitives to Hermes (#1125) #1131

Open

claude wants to merge 1 commits from claude/issue-1125 into main

Author	SHA1	Message	Date
Alexander Whitestone	a3a28aa4c2	feat: add desktop automation primitives to Hermes (#1125 ) Some checks failed CI / test (pull_request) Failing after 20s Details CI / validate (pull_request) Failing after 25s Details Review Approval Gate / verify-review (pull_request) Failing after 5s Details Implements Phase 1 & 2 of the [COMPUTER_USE] epic: - nexus/computer_use.py — four Hermes tools with safety guards and JSONL action logging: computer_screenshot(), computer_click(), computer_type(), computer_scroll() Poka-yoke: right/middle clicks require confirm=True; text containing password/token/key keywords is refused without confirm=True. pyautogui.FAILSAFE=True enabled globally (corner-abort). - nexus/computer_use_demo.py — end-to-end Phase 1 demo: baseline screenshot → open browser → navigate to Gitea → evidence screenshot. - tests/test_computer_use.py — 29 unit tests, fully headless (pyautogui mocked); all pass. - docs/computer-use.md — full Phase 1–3 documentation including API reference, safety table, action-log format, and pilot recipes. - docker-compose.desktop.yml — sandboxed Xvfb + noVNC container for safe headless desktop automation. The existing mcp_servers/desktop_control_server.py is unchanged; it remains available for external/MCP callers (Bannerlord harness etc). Fixes #1125	2026-04-08 06:29:27 -04:00

Author

SHA1

Message

Date

Alexander Whitestone

a3a28aa4c2

feat: add desktop automation primitives to Hermes (#1125 )

CI / test (pull_request) Failing after 20s

Details

CI / validate (pull_request) Failing after 25s

Details

Review Approval Gate / verify-review (pull_request) Failing after 5s

Details

Implements Phase 1 & 2 of the [COMPUTER_USE] epic:

- nexus/computer_use.py — four Hermes tools with safety guards and
  JSONL action logging:
    computer_screenshot(), computer_click(), computer_type(), computer_scroll()
  Poka-yoke: right/middle clicks require confirm=True; text containing
  password/token/key keywords is refused without confirm=True.
  pyautogui.FAILSAFE=True enabled globally (corner-abort).

- nexus/computer_use_demo.py — end-to-end Phase 1 demo: baseline
  screenshot → open browser → navigate to Gitea → evidence screenshot.

- tests/test_computer_use.py — 29 unit tests, fully headless (pyautogui
  mocked); all pass.

- docs/computer-use.md — full Phase 1–3 documentation including API
  reference, safety table, action-log format, and pilot recipes.

- docker-compose.desktop.yml — sandboxed Xvfb + noVNC container for
  safe headless desktop automation.

The existing mcp_servers/desktop_control_server.py is unchanged; it
remains available for external/MCP callers (Bannerlord harness etc).

Fixes #1125

2026-04-08 06:29:27 -04:00

[claude] Add desktop automation primitives to Hermes (#1125) #1131

1 Commits