Alexander Whitestone
|
220f20c794
|
feat: add desktop automation primitives to Hermes (#1125)
CI / test (pull_request) Failing after 8s
CI / validate (pull_request) Failing after 10s
Review Approval Gate / verify-review (pull_request) Failing after 2s
Implements Phase 1 and Phase 2 tooling from issue #1125:
- nexus/computer_use.py: four Hermes tools with poka-yoke safety
* computer_screenshot() — capture & base64-encode desktop snapshot
* computer_click(x, y, button, confirm) — right/middle require confirm=True
* computer_type(text, confirm) — sensitive keywords blocked without confirm=True;
text value is never written to audit log
* computer_scroll(x, y, amount) — scroll wheel
* read_action_log() — inspect recent JSONL audit entries
* pyautogui.FAILSAFE=True; all tools degrade gracefully when headless
- nexus/computer_use_demo.py: Phase 1 demo (baseline screenshot →
open browser → navigate to Gitea forge → evidence screenshot)
- tests/test_computer_use.py: 32 unit tests, fully headless
(pyautogui mocked), all passing
- docs/computer-use.md: API reference, safety table, phase roadmap,
pilot recipes
- docker-compose.desktop.yml: sandboxed Xvfb + noVNC container
Fixes #1125
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-04-10 05:45:27 -04:00 |
|