diff --git a/docs/matrix-setup.md b/docs/matrix-setup.md new file mode 100644 index 000000000..2249baad4 --- /dev/null +++ b/docs/matrix-setup.md @@ -0,0 +1,271 @@ +# Matrix Integration Setup Guide + +Connect Hermes Agent to any Matrix homeserver for sovereign, encrypted messaging. + +## Prerequisites + +- Python 3.10+ +- matrix-nio SDK: `pip install "matrix-nio[e2e]"` +- For E2EE: libolm C library (see below) + +## Option A: matrix.org Public Homeserver (Testing) + +Best for quick evaluation. No server to run. + +### 1. Create a Matrix Account + +Go to https://app.element.io and create an account on matrix.org. +Choose a username like `@hermes-bot:matrix.org`. + +### 2. Get an Access Token + +The recommended auth method. Token avoids storing passwords and survives +password changes. + +```bash +# Using curl (replace user/password): +curl -X POST 'https://matrix-client.matrix.org/_matrix/client/v3/login' \ + -H 'Content-Type: application/json' \ + -d '{ + "type": "m.login.password", + "user": "your-bot-username", + "password": "your-password" + }' +``` + +Look for `access_token` and `device_id` in the response. + +Alternatively, in Element: Settings -> Help & About -> Advanced -> Access Token. + +### 3. Set Environment Variables + +Add to `~/.hermes/.env`: + +```bash +MATRIX_HOMESERVER=https://matrix-client.matrix.org +MATRIX_ACCESS_TOKEN=syt_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +MATRIX_USER_ID=@hermes-bot:matrix.org +MATRIX_DEVICE_ID=HERMES_BOT +``` + +### 4. Install Dependencies + +```bash +pip install "matrix-nio[e2e]" +``` + +### 5. Start Hermes Gateway + +```bash +hermes gateway +``` + +## Option B: Self-Hosted Homeserver (Sovereignty) + +For full control over your data and encryption keys. + +### Popular Homeservers + +- **Synapse** (reference impl): https://github.com/element-hq/synapse +- **Conduit** (lightweight, Rust): https://conduit.rs +- **Dendrite** (Go): https://github.com/matrix-org/dendrite + +### 1. Deploy Your Homeserver + +Follow your chosen server's documentation. Common setup with Docker: + +```bash +# Synapse example: +docker run -d --name synapse \ + -v /opt/synapse/data:/data \ + -e SYNAPSE_SERVER_NAME=your.domain.com \ + -e SYNAPSE_REPORT_STATS=no \ + matrixdotorg/synapse:latest +``` + +### 2. Create Bot Account + +Register on your homeserver: + +```bash +# Synapse: register new user (run inside container) +docker exec -it synapse register_new_matrix_user http://localhost:8008 \ + -c /data/homeserver.yaml -u hermes-bot -p 'secure-password' --admin +``` + +### 3. Configure Hermes + +Set in `~/.hermes/.env`: + +```bash +MATRIX_HOMESERVER=https://matrix.your.domain.com +MATRIX_ACCESS_TOKEN= +MATRIX_USER_ID=@hermes-bot:your.domain.com +MATRIX_DEVICE_ID=HERMES_BOT +``` + +## Environment Variables Reference + +| Variable | Required | Description | +|----------|----------|-------------| +| `MATRIX_HOMESERVER` | Yes | Homeserver URL (e.g. `https://matrix.org`) | +| `MATRIX_ACCESS_TOKEN` | Yes* | Access token (preferred over password) | +| `MATRIX_USER_ID` | With password | Full user ID (`@user:server`) | +| `MATRIX_PASSWORD` | Alt* | Password (alternative to token) | +| `MATRIX_DEVICE_ID` | Recommended | Stable device ID for E2EE persistence | +| `MATRIX_ENCRYPTION` | No | Set `true` to enable E2EE | +| `MATRIX_ALLOWED_USERS` | No | Comma-separated allowed user IDs | +| `MATRIX_HOME_ROOM` | No | Room ID for cron/notifications | +| `MATRIX_REACTIONS` | No | Enable processing reactions (default: true) | +| `MATRIX_REQUIRE_MENTION` | No | Require @mention in rooms (default: true) | +| `MATRIX_FREE_RESPONSE_ROOMS` | No | Room IDs exempt from mention requirement | +| `MATRIX_AUTO_THREAD` | No | Auto-create threads (default: true) | + +\* Either `MATRIX_ACCESS_TOKEN` or `MATRIX_USER_ID` + `MATRIX_PASSWORD` is required. + +## Config YAML Entries + +Add to `~/.hermes/config.yaml` under a `matrix:` key for declarative settings: + +```yaml +matrix: + require_mention: true + free_response_rooms: + - "!roomid1:matrix.org" + - "!roomid2:matrix.org" + auto_thread: true +``` + +These override to env vars only if the env var is not already set. + +## End-to-End Encryption (E2EE) + +E2EE protects messages so only participants can read them. Hermes uses +matrix-nio's Olm/Megolm implementation. + +### 1. Install E2EE Dependencies + +```bash +# macOS +brew install libolm + +# Ubuntu/Debian +sudo apt install libolm-dev + +# Then install matrix-nio with E2EE support: +pip install "matrix-nio[e2e]" +``` + +### 2. Enable Encryption + +Set in `~/.hermes/.env`: + +```bash +MATRIX_ENCRYPTION=true +MATRIX_DEVICE_ID=HERMES_BOT +``` + +### 3. How It Works + +- On first connect, Hermes creates a device and uploads encryption keys. +- Keys are stored in `~/.hermes/platforms/matrix/store/`. +- On shutdown, Megolm session keys are exported to `exported_keys.txt`. +- On next startup, keys are imported so the bot can decrypt old messages. +- The `MATRIX_DEVICE_ID` ensures the bot reuses the same device identity + across restarts. Without it, each restart creates a new "device" in + Matrix and old keys become unusable. + +### 4. Verifying E2EE + +1. Create an encrypted room in Element. +2. Invite your bot user. +3. Send a message — the bot should respond. +4. Check logs: `grep -i "e2ee\|crypto\|encrypt" ~/.hermes/logs/gateway.log` + +## Room Configuration + +### Inviting the Bot + +1. Create a room in Element or any Matrix client. +2. Invite the bot: `/invite @hermes-bot:your.domain.com` +3. The bot auto-accepts invites (controlled by `MATRIX_ALLOWED_USERS`). + +### Home Room + +Set `MATRIX_HOME_ROOM` to a room ID for cron jobs and notifications: + +```bash +MATRIX_HOME_ROOM=!abcde12345:matrix.org +``` + +### Free-Response Rooms + +Rooms where the bot responds to all messages without @mention: + +```bash +MATRIX_FREE_RESPONSE_ROOMS=!room1:matrix.org,!room2:matrix.org +``` + +Or in config.yaml: + +```yaml +matrix: + free_response_rooms: + - "!room1:matrix.org" +``` + +## Troubleshooting + +### "Matrix: need MATRIX_ACCESS_TOKEN or MATRIX_USER_ID + MATRIX_PASSWORD" + +Neither auth method is configured. Set `MATRIX_ACCESS_TOKEN` in `~/.hermes/.env` +or provide `MATRIX_USER_ID` + `MATRIX_PASSWORD`. + +### "Matrix: whoami failed" + +The access token is invalid or expired. Generate a new one via the login API. + +### "Matrix: E2EE dependencies are missing" + +Install libolm and matrix-nio with E2EE support: + +```bash +brew install libolm # macOS +pip install "matrix-nio[e2e]" +``` + +### "Matrix: login failed" + +- Check username and password. +- Ensure the account exists on the target homeserver. +- Some homeservers require admin approval for new registrations. + +### Bot Not Responding in Rooms + +1. Check `MATRIX_REQUIRE_MENTION` — if `true` (default), messages must + @mention the bot. +2. Check `MATRIX_ALLOWED_USERS` — if set, only listed users can interact. +3. Check logs: `tail -f ~/.hermes/logs/gateway.log` + +### E2EE Rooms Show "Unable to Decrypt" + +1. Ensure `MATRIX_DEVICE_ID` is set to a stable value. +2. Check that `~/.hermes/platforms/matrix/store/` has read/write permissions. +3. Verify libolm is installed: `python -c "from nio.crypto import ENCRYPTION_ENABLED; print(ENCRYPTION_ENABLED)"` + +### Slow Message Delivery + +Matrix federation can add latency. For faster responses: +- Use the same homeserver for the bot and users. +- Set `MATRIX_HOME_ROOM` to a local room. +- Check network connectivity between Hermes and the homeserver. + +## Quick Start (Automated) + +Run the interactive setup script: + +```bash +python scripts/setup_matrix.py +``` + +This guides you through homeserver selection, authentication, and verification. diff --git a/scripts/setup_matrix.py b/scripts/setup_matrix.py new file mode 100755 index 000000000..93f042688 --- /dev/null +++ b/scripts/setup_matrix.py @@ -0,0 +1,430 @@ +#!/usr/bin/env python3 +"""Interactive Matrix setup wizard for Hermes Agent. + +Guides you through configuring Matrix integration: + - Homeserver URL + - Token auth or password auth + - Device ID generation + - Config/env file writing + - Optional: test room creation and message send + - E2EE verification + +Usage: + python scripts/setup_matrix.py +""" + +import getpass +import json +import os +import secrets +import sys +import urllib.error +import urllib.request +from pathlib import Path + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def _hermes_home() -> Path: + """Resolve ~/.hermes (or HERMES_HOME override).""" + return Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes")) + + +def _prompt(msg: str, default: str = "") -> str: + """Prompt with optional default. Returns stripped input or default.""" + suffix = f" [{default}]" if default else "" + val = input(f"{msg}{suffix}: ").strip() + return val or default + + +def _prompt_bool(msg: str, default: bool = True) -> bool: + """Yes/no prompt.""" + d = "Y/n" if default else "y/N" + val = input(f"{msg} [{d}]: ").strip().lower() + if not val: + return default + return val in ("y", "yes") + + +def _http_post_json(url: str, data: dict, timeout: int = 15) -> dict: + """POST JSON and return parsed response. Raises on HTTP errors.""" + body = json.dumps(data).encode() + req = urllib.request.Request( + url, + data=body, + headers={"Content-Type": "application/json"}, + method="POST", + ) + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: + return json.loads(resp.read()) + except urllib.error.HTTPError as exc: + detail = exc.read().decode(errors="replace") + raise RuntimeError(f"HTTP {exc.code}: {detail}") from exc + except urllib.error.URLError as exc: + raise RuntimeError(f"Connection error: {exc.reason}") from exc + + +def _http_get_json(url: str, token: str = "", timeout: int = 15) -> dict: + """GET JSON, optionally with Bearer auth.""" + req = urllib.request.Request(url, method="GET") + if token: + req.add_header("Authorization", f"Bearer {token}") + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: + return json.loads(resp.read()) + except urllib.error.HTTPError as exc: + detail = exc.read().decode(errors="replace") + raise RuntimeError(f"HTTP {exc.code}: {detail}") from exc + except urllib.error.URLError as exc: + raise RuntimeError(f"Connection error: {exc.reason}") from exc + + +def _write_env_file(env_path: Path, vars: dict) -> None: + """Write/update ~/.hermes/.env with given variables.""" + existing: dict[str, str] = {} + if env_path.exists(): + for line in env_path.read_text().splitlines(): + line = line.strip() + if line and not line.startswith("#") and "=" in line: + k, v = line.split("=", 1) + existing[k.strip()] = v.strip().strip("'\"") + + existing.update(vars) + + lines = ["# Hermes Agent environment variables"] + for k, v in sorted(existing.items()): + # Quote values with spaces or special chars + if any(c in v for c in " \t#\"'$"): + lines.append(f'{k}="{v}"') + else: + lines.append(f"{k}={v}") + + env_path.parent.mkdir(parents=True, exist_ok=True) + env_path.write_text("\n".join(lines) + "\n") + try: + os.chmod(str(env_path), 0o600) + except (OSError, NotImplementedError): + pass + print(f" -> Wrote {len(vars)} vars to {env_path}") + + +def _write_config_yaml(config_path: Path, matrix_section: dict) -> None: + """Add/update matrix: section in config.yaml (creates file if needed).""" + try: + import yaml + except ImportError: + print(" [!] PyYAML not installed — skipping config.yaml update.") + print(" Add manually under 'matrix:' key.") + return + + config: dict = {} + if config_path.exists(): + try: + config = yaml.safe_load(config_path.read_text()) or {} + except Exception: + config = {} + + config["matrix"] = matrix_section + + config_path.parent.mkdir(parents=True, exist_ok=True) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + try: + os.chmod(str(config_path), 0o600) + except (OSError, NotImplementedError): + pass + print(f" -> Updated matrix section in {config_path}") + + +def _generate_device_id() -> str: + """Generate a stable, human-readable device ID.""" + return f"HERMES_{secrets.token_hex(4).upper()}" + + +# --------------------------------------------------------------------------- +# Login flows +# --------------------------------------------------------------------------- + +def login_with_token(homeserver: str) -> dict: + """Validate an existing access token via whoami.""" + token = getpass.getpass("Access token (hidden): ").strip() + if not token: + print(" [!] Token cannot be empty.") + sys.exit(1) + + whoami_url = f"{homeserver}/_matrix/client/v3/account/whoami" + print(" Validating token...") + resp = _http_get_json(whoami_url, token=token) + + user_id = resp.get("user_id", "") + device_id = resp.get("device_id", "") + print(f" Authenticated as: {user_id}") + if device_id: + print(f" Server device ID: {device_id}") + + return { + "MATRIX_ACCESS_TOKEN": token, + "MATRIX_USER_ID": user_id, + } + + +def login_with_password(homeserver: str) -> dict: + """Login with username + password, get access token.""" + user_id = _prompt("Full user ID (e.g. @bot:matrix.org)") + if not user_id: + print(" [!] User ID cannot be empty.") + sys.exit(1) + + password = getpass.getpass("Password (hidden): ").strip() + if not password: + print(" [!] Password cannot be empty.") + sys.exit(1) + + login_url = f"{homeserver}/_matrix/client/v3/login" + print(" Logging in...") + resp = _http_post_json(login_url, { + "type": "m.login.password", + "identifier": { + "type": "m.id.user", + "user": user_id, + }, + "password": password, + "device_name": "Hermes Agent", + }) + + access_token = resp.get("access_token", "") + device_id = resp.get("device_id", "") + resolved_user = resp.get("user_id", user_id) + + if not access_token: + print(" [!] Login succeeded but no access_token in response.") + sys.exit(1) + + print(f" Authenticated as: {resolved_user}") + if device_id: + print(f" Device ID: {device_id}") + + return { + "MATRIX_ACCESS_TOKEN": access_token, + "MATRIX_USER_ID": resolved_user, + "_server_device_id": device_id, + } + + +# --------------------------------------------------------------------------- +# Test room + message +# --------------------------------------------------------------------------- + +def create_test_room(homeserver: str, token: str) -> str | None: + """Create a private test room and return the room ID.""" + create_url = f"{homeserver}/_matrix/client/v3/createRoom" + try: + resp = _http_post_json(create_url, { + "name": "Hermes Test Room", + "topic": "Auto-created by hermes setup_matrix.py — safe to delete", + "preset": "private_chat", + "visibility": "private", + }, timeout=30) + # Set auth header manually (createRoom needs proper auth) + room_id = resp.get("room_id", "") + if room_id: + print(f" Created test room: {room_id}") + return room_id + except Exception: + pass + + # Fallback: use curl-style with auth + req = urllib.request.Request( + create_url, + data=json.dumps({ + "name": "Hermes Test Room", + "topic": "Auto-created by hermes setup_matrix.py — safe to delete", + "preset": "private_chat", + "visibility": "private", + }).encode(), + headers={ + "Content-Type": "application/json", + "Authorization": f"Bearer {token}", + }, + method="POST", + ) + try: + with urllib.request.urlopen(req, timeout=30) as resp: + data = json.loads(resp.read()) + room_id = data.get("room_id", "") + if room_id: + print(f" Created test room: {room_id}") + return room_id + except Exception as exc: + print(f" [!] Room creation failed: {exc}") + return None + + +def send_test_message(homeserver: str, token: str, room_id: str) -> bool: + """Send a test message to a room. Returns True on success.""" + txn_id = secrets.token_hex(8) + url = ( + f"{homeserver}/_matrix/client/v3/rooms/" + f"{urllib.request.quote(room_id, safe='')}/send/m.room.message/{txn_id}" + ) + req = urllib.request.Request( + url, + data=json.dumps({ + "msgtype": "m.text", + "body": "Hermes Agent setup verified successfully!", + }).encode(), + headers={ + "Content-Type": "application/json", + "Authorization": f"Bearer {token}", + }, + method="PUT", + ) + try: + with urllib.request.urlopen(req, timeout=15) as resp: + data = json.loads(resp.read()) + event_id = data.get("event_id", "") + if event_id: + print(f" Test message sent: {event_id}") + return True + except Exception as exc: + print(f" [!] Test message failed: {exc}") + return False + + +def check_e2ee_support() -> bool: + """Check if E2EE dependencies are available.""" + try: + import nio + from nio.crypto import ENCRYPTION_ENABLED + return bool(ENCRYPTION_ENABLED) + except (ImportError, AttributeError): + return False + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main(): + print("=" * 60) + print(" Hermes Agent — Matrix Setup Wizard") + print("=" * 60) + print() + + # -- Homeserver -- + print("Step 1: Homeserver") + print(" A) matrix.org (public, for testing)") + print(" B) Custom homeserver (self-hosted)") + choice = _prompt("Choose [A/B]", "A").upper() + + if choice == "B": + homeserver = _prompt("Homeserver URL (e.g. https://matrix.example.com)") + if not homeserver: + print(" [!] Homeserver URL is required.") + sys.exit(1) + else: + homeserver = "https://matrix-client.matrix.org" + + homeserver = homeserver.rstrip("/") + print(f" Using: {homeserver}") + print() + + # -- Authentication -- + print("Step 2: Authentication") + print(" A) Access token (recommended)") + print(" B) Username + password") + auth_choice = _prompt("Choose [A/B]", "A").upper() + + if auth_choice == "B": + auth_vars = login_with_password(homeserver) + else: + auth_vars = login_with_token(homeserver) + print() + + # -- Device ID -- + print("Step 3: Device ID (for E2EE persistence)") + server_device = auth_vars.pop("_server_device_id", "") + default_device = server_device or _generate_device_id() + device_id = _prompt("Device ID", default_device) + auth_vars["MATRIX_DEVICE_ID"] = device_id + print() + + # -- E2EE -- + print("Step 4: End-to-End Encryption") + e2ee_available = check_e2ee_support() + if e2ee_available: + enable_e2ee = _prompt_bool("Enable E2EE?", default=False) + if enable_e2ee: + auth_vars["MATRIX_ENCRYPTION"] = "true" + print(" E2EE enabled. Keys will be stored in:") + print(" ~/.hermes/platforms/matrix/store/") + else: + print(" E2EE dependencies not found. Skipping.") + print(" To enable later: pip install 'matrix-nio[e2e]'") + print() + + # -- Optional settings -- + print("Step 5: Optional Settings") + allowed = _prompt("Allowed user IDs (comma-separated, or empty for all)") + if allowed: + auth_vars["MATRIX_ALLOWED_USERS"] = allowed + + home_room = _prompt("Home room ID for notifications (or empty)") + if home_room: + auth_vars["MATRIX_HOME_ROOM"] = home_room + + require_mention = _prompt_bool("Require @mention in rooms?", default=True) + auto_thread = _prompt_bool("Auto-create threads?", default=True) + print() + + # -- Write files -- + print("Step 6: Writing Configuration") + hermes_home = _hermes_home() + + env_path = hermes_home / ".env" + _write_env_file(env_path, auth_vars) + + config_path = hermes_home / "config.yaml" + matrix_cfg = { + "require_mention": require_mention, + "auto_thread": auto_thread, + } + _write_config_yaml(config_path, matrix_cfg) + print() + + # -- Verify connection -- + print("Step 7: Verification") + token = auth_vars.get("MATRIX_ACCESS_TOKEN", "") + + do_test = _prompt_bool("Create test room and send message?", default=True) + if do_test and token: + room_id = create_test_room(homeserver, token) + if room_id: + send_test_message(homeserver, token, room_id) + print() + + # -- Summary -- + print("=" * 60) + print(" Setup Complete!") + print("=" * 60) + print() + print(" Config written to:") + print(f" {env_path}") + print(f" {config_path}") + print() + print(" To start the Matrix gateway:") + print(" hermes gateway --platform matrix") + print() + if not e2ee_available: + print(" To enable E2EE later:") + print(" pip install 'matrix-nio[e2e]'") + print(" Then set MATRIX_ENCRYPTION=true in .env") + print() + print(" Docs: docs/matrix-setup.md") + print() + + +if __name__ == "__main__": + main()