feat: enhance README and improve environment configuration

- Added a new section in the README for Inference Providers, detailing setup instructions for Nous Portal, OpenRouter, and Custom Endpoints, improving user guidance for LLM connections. - Updated messaging platform setup instructions to include Slack and WhatsApp, providing clearer steps for configuration. - Introduced a new environment variable, TERMINAL_SANDBOX_DIR, to allow users to customize the sandbox storage location for Docker and Singularity environments. - Refactored the Docker and Singularity environment classes to utilize the new sandbox directory for persistent workspaces, enhancing organization and usability. - Improved handling of working directories across various environments, ensuring compatibility and clarity in execution paths.
2026-02-23 21:15:35 -08:00
parent 54dd1b3038
commit a183827128
7 changed files with 157 additions and 105 deletions
--- a/README.md
+++ b/README.md
@@ -92,6 +92,20 @@ rm -rf ~/.hermes            # Optional — keep if you plan to reinstall

 ---

+## Inference Providers
+
+You need at least one way to connect to an LLM. Use `hermes model` to switch providers and models interactively, or configure directly:
+
+| Provider | Setup |
+|----------|-------|
+| **Nous Portal** | `hermes login` (OAuth, subscription-based) |
+| **OpenRouter** | `OPENROUTER_API_KEY` in `~/.hermes/.env` |
+| **Custom Endpoint** | `OPENAI_BASE_URL` + `OPENAI_API_KEY` in `~/.hermes/.env` |
+
+**Note:** Even when using Nous Portal or a custom endpoint, some tools (vision, web summarization, MoA) use OpenRouter independently. An `OPENROUTER_API_KEY` enables these tools.
+
+---
+
 ## Configuration

 All your settings are stored in `~/.hermes/` for easy access:
@@ -109,18 +123,6 @@ All your settings are stored in `~/.hermes/` for easy access:
 └── logs/           # Logs
 ```

-### Messaging Platforms (Telegram, Discord, Slack)
-
-If you configured a messaging bot token during setup, **start the gateway** so Hermes can receive and send messages:
-
-```bash
-hermes gateway              # Run in foreground (see output)
-hermes gateway install      # Or install as a background service (Linux)
-hermes gateway start        # Start the background service
-```
-
-The installer will offer to do this automatically if it detects a bot token. See [Messaging Gateway](#messaging-gateway) below for full setup instructions.
-
 ### Managing Configuration

 ```bash
@@ -136,18 +138,6 @@ hermes config set terminal.backend docker
 hermes config set OPENROUTER_API_KEY sk-or-...  # Saves to .env
 ```

-### Inference Providers
-
-You need at least one way to connect to an LLM. Use `hermes model` to switch providers and models interactively, or configure directly:
-
-| Provider | Setup |
-|----------|-------|
-| **Nous Portal** | `hermes login` (OAuth, subscription-based) |
-| **OpenRouter** | `OPENROUTER_API_KEY` in `~/.hermes/.env` |
-| **Custom Endpoint** | `OPENAI_BASE_URL` + `OPENAI_API_KEY` in `~/.hermes/.env` |
-
-**Note:** Even when using Nous Portal or a custom endpoint, some tools (vision, web summarization, MoA) use OpenRouter independently. An `OPENROUTER_API_KEY` enables these tools.
-
 ### Optional API Keys

 | Feature | Provider | Env Variable |
@@ -158,14 +148,12 @@ You need at least one way to connect to an LLM. Use `hermes model` to switch pro
 | Premium TTS voices | [ElevenLabs](https://elevenlabs.io/) | `ELEVENLABS_API_KEY` |
 | OpenAI TTS + voice transcription | [OpenAI](https://platform.openai.com/api-keys) | `VOICE_TOOLS_OPENAI_KEY` |
 | RL Training | [Tinker](https://tinker-console.thinkingmachines.ai/) + [WandB](https://wandb.ai/) | `TINKER_API_KEY`, `WANDB_API_KEY` |
-| Slack integration | [Slack](https://api.slack.com/apps) | `SLACK_BOT_TOKEN`, `SLACK_APP_TOKEN` |
-| Messaging | Telegram, Discord | `TELEGRAM_BOT_TOKEN`, `DISCORD_BOT_TOKEN` |

 ---

 ## Messaging Gateway

-Chat with Hermes from Telegram, Discord, or WhatsApp.
+Chat with Hermes from Telegram, Discord, Slack, or WhatsApp. The gateway is a single background process that connects to all your configured platforms, handles sessions, runs cron jobs, and delivers voice messages.

 ### Starting the Gateway

@@ -177,18 +165,12 @@ hermes gateway stop         # Stop the systemd service
 hermes gateway status       # Check service status
 ```

-### Gateway Commands (inside chat)
-
-| Command | Description |
-|---------|-------------|
-| `/new` or `/reset` | Start fresh conversation |
-| `/status` | Show session info |
-| `/hermes` (Discord) | Slash command — ask, reset, status, stop |
+The installer will offer to set this up automatically if it detects a bot token.

 ### Telegram Setup

 1. **Create a bot:** Message [@BotFather](https://t.me/BotFather) on Telegram, use `/newbot`
-2. **Get your user ID:** Message [@userinfobot](https://t.me/userinfobot) - it replies with your numeric ID
+2. **Get your user ID:** Message [@userinfobot](https://t.me/userinfobot) — it replies with your numeric ID
 3. **Configure:**

 ```bash
@@ -202,8 +184,10 @@ TELEGRAM_ALLOWED_USERS=YOUR_USER_ID    # Comma-separated for multiple users
 ### Discord Setup

 1. **Create a bot:** Go to [Discord Developer Portal](https://discord.com/developers/applications)
-2. **Get your user ID:** Enable Developer Mode in Discord settings, right-click your name → Copy ID
-3. **Configure:**
+2. **Enable intents:** Bot → Privileged Gateway Intents → enable Message Content Intent
+3. **Get your user ID:** Enable Developer Mode in Discord settings, right-click your name → Copy ID
+4. **Invite to your server:** OAuth2 → URL Generator → scopes: `bot`, `applications.commands` → permissions: Send Messages, Read Message History, Attach Files
+5. **Configure:**

 ```bash
 # Add to ~/.hermes/.env:
@@ -227,7 +211,41 @@ SLACK_APP_TOKEN=xapp-...
 SLACK_ALLOWED_USERS=U01234ABCDE    # Comma-separated Slack user IDs
 ```

-5. **Start the gateway:** `hermes gateway`
+### WhatsApp Setup
+
+WhatsApp doesn't have a simple bot API like Telegram or Discord. Hermes supports two approaches:
+
+**Option A — WhatsApp Business API** (requires [Meta Business verification](https://business.facebook.com/)):
+- Production-grade, but requires a verified business account
+- Set `WHATSAPP_ENABLED=true` in `~/.hermes/.env` and configure the Business API credentials
+
+**Option B — whatsapp-web.js bridge** (personal accounts):
+1. Install Node.js if not already present
+2. Set up the bridge:
+
+```bash
+# Add to ~/.hermes/.env:
+WHATSAPP_ENABLED=true
+WHATSAPP_ALLOWED_USERS=YOUR_PHONE_NUMBER    # e.g. 15551234567
+```
+
+3. On first launch, the gateway will display a QR code — scan it with WhatsApp on your phone to link the session
+
+See [docs/messaging.md](docs/messaging.md) for advanced WhatsApp configuration.
+
+### Gateway Commands (inside chat)
+
+| Command | Description |
+|---------|-------------|
+| `/new` or `/reset` | Start fresh conversation |
+| `/model [name]` | Show or change the model |
+| `/personality [name]` | Set a personality |
+| `/retry` | Retry the last message |
+| `/undo` | Remove the last exchange |
+| `/status` | Show session info |
+| `/stop` | Stop the running agent |
+| `/sethome` | Set this chat as the home channel |
+| `/help` | Show available commands |

 ### DM Pairing (Alternative to Allowlists)

@@ -245,7 +263,7 @@ hermes pairing revoke telegram 123456789  # Remove access

 Pairing codes expire after 1 hour, are rate-limited, and use cryptographic randomness.

-### Security (Important!)
+### Security

 **By default, the gateway denies all users who are not in an allowlist or paired via DM.** This is the safe default for a bot with terminal access.

@@ -260,12 +278,17 @@ GATEWAY_ALLOW_ALL_USERS=true

 ### Working Directory

- **CLI (`hermes`)**: Uses current directory where you run the command
- **Messaging**: Uses `MESSAGING_CWD` (default: home directory `~`)
+| Context | Default |
+|---------|---------|
+| **CLI (`hermes`)** | Current directory where you run the command |
+| **Messaging gateway** | Home directory `~` (override with `MESSAGING_CWD`) |
+| **Docker / Singularity / Modal / SSH** | User's home directory (`~`) inside the container or remote machine |

+Override the terminal working directory for any backend:
 ```bash
-# Set custom messaging working directory in ~/.hermes/.env
-MESSAGING_CWD=/home/myuser/projects
+# In ~/.hermes/.env or ~/.hermes/config.yaml:
+MESSAGING_CWD=/home/myuser/projects    # Gateway sessions
+TERMINAL_CWD=/workspace                # All terminal sessions (local or container)
 ```

 ### Tool Progress Notifications
@@ -275,18 +298,9 @@ Get real-time updates as the agent works:
 ```bash
 # Enable in ~/.hermes/.env
 HERMES_TOOL_PROGRESS=true
-HERMES_TOOL_PROGRESS_MODE=new    # or "all" for every tool call
+HERMES_TOOL_PROGRESS_MODE=all    # or "new" for only when tool changes
 ```

-When enabled, you'll see messages like:
-```
-💻 `ls -la`...
-🔍 web_search...
-📄 web_extract...
-```
-
-See [docs/messaging.md](docs/messaging.md) for WhatsApp and advanced setup.
-
 ---

 ## Commands
@@ -474,7 +488,12 @@ terminal:
  container_persistent: true    # Persist filesystem across sessions (default: true)
 ```

-When `container_persistent: true`, the sandbox state (installed packages, files, config) survives across sessions. Docker uses named volumes, Singularity uses persistent overlays, and Modal uses filesystem snapshots.
+When `container_persistent: true`, the sandbox state (installed packages, files, config) survives across sessions. Docker uses bind mounts, Singularity uses persistent overlays, and Modal uses filesystem snapshots. All persistent data is stored under `TERMINAL_SANDBOX_DIR` (default: `~/.hermes/sandboxes/`):
+
+```bash
+# Override where Docker workspaces and Singularity overlays/SIF cache are stored
+TERMINAL_SANDBOX_DIR=/mnt/fast-ssd/hermes-sandboxes
+```

 ### 🧠 Persistent Memory

@@ -1416,13 +1435,14 @@ All variables go in `~/.hermes/.env`. Run `hermes config set VAR value` to set t
 | `TERMINAL_CONTAINER_MEMORY` | Memory in MB for container backends (default: 5120) |
 | `TERMINAL_CONTAINER_DISK` | Disk in MB for container backends (default: 51200) |
 | `TERMINAL_CONTAINER_PERSISTENT` | Persist container filesystem across sessions (default: true) |
+| `TERMINAL_SANDBOX_DIR` | Host directory for Docker workspaces, Singularity overlays/SIF cache (default: `~/.hermes/sandboxes/`) |

 **Agent Behavior:**
 | Variable | Description |
 |----------|-------------|
 | `HERMES_MAX_ITERATIONS` | Max tool-calling iterations per conversation (default: 60) |
 | `HERMES_TOOL_PROGRESS` | Send progress messages when using tools (`true`/`false`) |
-| `HERMES_TOOL_PROGRESS_MODE` | `new` (only when tool changes) or `all` (every call) |
+| `HERMES_TOOL_PROGRESS_MODE` | `all` (every call, default) or `new` (only when tool changes) |

 **Context Compression:**
 | Variable | Description |
--- a/tools/environments/base.py
+++ b/tools/environments/base.py
@@ -1,7 +1,24 @@
 """Base class for all Hermes execution environment backends."""

 from abc import ABC, abstractmethod
+import os
 import subprocess
+from pathlib import Path
+
+
+def get_sandbox_dir() -> Path:
+    """Return the host-side root for all sandbox storage (Docker workspaces,
+    Singularity overlays/SIF cache, etc.).
+
+    Configurable via TERMINAL_SANDBOX_DIR. Defaults to ~/.hermes/sandboxes/.
+    """
+    custom = os.getenv("TERMINAL_SANDBOX_DIR")
+    if custom:
+        p = Path(custom)
+    else:
+        p = Path.home() / ".hermes" / "sandboxes"
+    p.mkdir(parents=True, exist_ok=True)
+    return p


 class BaseEnvironment(ABC):
--- a/tools/environments/docker.py
+++ b/tools/environments/docker.py
@@ -44,7 +44,7 @@ class DockerEnvironment(BaseEnvironment):
    def __init__(
        self,
        image: str,
-        cwd: str = "/",
+        cwd: str = "~",
        timeout: int = 60,
        cpu: float = 0,
        memory: int = 0,
@@ -72,23 +72,26 @@ class DockerEnvironment(BaseEnvironment):
        if not network:
            resource_args.append("--network=none")

-        # Persistent volume for writable workspace that survives container restarts.
-        # Non-persistent mode uses tmpfs (ephemeral, fast, gone on cleanup).
-        self._volume_name: Optional[str] = None
+        # Persistent workspace via bind mounts from a configurable host directory
+        # (TERMINAL_SANDBOX_DIR, default ~/.hermes/sandboxes/). Non-persistent
+        # mode uses tmpfs (ephemeral, fast, gone on cleanup).
+        from tools.environments.base import get_sandbox_dir
+
+        self._workspace_dir: Optional[str] = None
+        self._home_dir: Optional[str] = None
        if self._persistent:
-            self._volume_name = f"hermes-workspace-{task_id}"
-            # Create volume if it doesn't exist
-            subprocess.run(
-                ["docker", "volume", "create", self._volume_name],
-                capture_output=True, timeout=10,
-            )
+            sandbox = get_sandbox_dir() / "docker" / task_id
+            self._workspace_dir = str(sandbox / "workspace")
+            self._home_dir = str(sandbox / "home")
+            os.makedirs(self._workspace_dir, exist_ok=True)
+            os.makedirs(self._home_dir, exist_ok=True)
            writable_args = [
-                "-v", f"{self._volume_name}:{cwd}",
-                "-v", f"{self._volume_name}-home:/root",
+                "-v", f"{self._workspace_dir}:/workspace",
+                "-v", f"{self._home_dir}:/root",
            ]
        else:
            writable_args = [
-                "--tmpfs", f"{cwd}:rw,exec,size=10g",
+                "--tmpfs", "/workspace:rw,exec,size=10g",
                "--tmpfs", "/home:rw,exec,size=1g",
                "--tmpfs", "/root:rw,exec,size=1g",
            ]
@@ -111,6 +114,11 @@ class DockerEnvironment(BaseEnvironment):
        work_dir = cwd or self.cwd
        effective_timeout = timeout or self.timeout

+        # docker exec -w doesn't expand ~, so prepend a cd into the command
+        if work_dir == "~" or work_dir.startswith("~/"):
+            exec_command = f"cd {work_dir} && {exec_command}"
+            work_dir = "/"
+
        assert self._inner.container_id, "Container not started"
        cmd = [self._inner.config.executable, "exec"]
        if stdin_data is not None:
@@ -173,16 +181,11 @@ class DockerEnvironment(BaseEnvironment):
            return {"output": f"Docker execution error: {e}", "returncode": 1}

    def cleanup(self):
-        """Stop and remove the container. Volumes persist if persistent=True."""
+        """Stop and remove the container. Bind-mount dirs persist if persistent=True."""
        self._inner.cleanup()

-        # If NOT persistent, remove the workspace volumes too
-        if not self._persistent and self._volume_name:
-            for vol in [self._volume_name, f"{self._volume_name}-home"]:
-                try:
-                    subprocess.run(
-                        ["docker", "volume", "rm", "-f", vol],
-                        capture_output=True, timeout=10,
-                    )
-                except Exception:
-                    pass
+        if not self._persistent:
+            import shutil
+            for d in (self._workspace_dir, self._home_dir):
+                if d:
+                    shutil.rmtree(d, ignore_errors=True)
--- a/tools/environments/modal.py
+++ b/tools/environments/modal.py
@@ -50,7 +50,7 @@ class ModalEnvironment(BaseEnvironment):
    def __init__(
        self,
        image: str,
-        cwd: str = "/root",
+        cwd: str = "~",
        timeout: int = 60,
        modal_sandbox_kwargs: Optional[Dict[str, Any]] = None,
        persistent_filesystem: bool = True,
--- a/tools/environments/singularity.py
+++ b/tools/environments/singularity.py
@@ -43,13 +43,23 @@ def _save_snapshots(data: Dict[str, str]) -> None:
 # -------------------------------------------------------------------------

 def _get_scratch_dir() -> Path:
-    """Get the best directory for Singularity sandboxes -- prefers /scratch on HPC."""
+    """Get the best directory for Singularity sandboxes.
+
+    Resolution order:
+      1. TERMINAL_SCRATCH_DIR (explicit override)
+      2. TERMINAL_SANDBOX_DIR / singularity (shared sandbox root)
+      3. /scratch (common on HPC clusters)
+      4. ~/.hermes/sandboxes/singularity (fallback)
+    """
    custom_scratch = os.getenv("TERMINAL_SCRATCH_DIR")
    if custom_scratch:
        scratch_path = Path(custom_scratch)
        scratch_path.mkdir(parents=True, exist_ok=True)
        return scratch_path

+    from tools.environments.base import get_sandbox_dir
+    sandbox = get_sandbox_dir() / "singularity"
+
    scratch = Path("/scratch")
    if scratch.exists() and os.access(scratch, os.W_OK):
        user_scratch = scratch / os.getenv("USER", "hermes") / "hermes-agent"
@@ -57,8 +67,8 @@ def _get_scratch_dir() -> Path:
        logger.info("Using /scratch for sandboxes: %s", user_scratch)
        return user_scratch

-    logger.debug("/scratch not available, using /tmp for sandboxes")
-    return Path(tempfile.gettempdir())
+    sandbox.mkdir(parents=True, exist_ok=True)
+    return sandbox


 def _get_apptainer_cache_dir() -> Path:
@@ -149,7 +159,7 @@ class SingularityEnvironment(BaseEnvironment):
    def __init__(
        self,
        image: str,
-        cwd: str = "/root",
+        cwd: str = "~",
        timeout: int = 60,
        cpu: float = 0,
        memory: int = 0,
@@ -217,9 +227,17 @@ class SingularityEnvironment(BaseEnvironment):
            return {"output": "Instance not started", "returncode": -1}

        effective_timeout = timeout or self.timeout
-        cmd = [self.executable, "exec", "--pwd", cwd or self.cwd,
+        work_dir = cwd or self.cwd
+        exec_command = self._prepare_command(command)
+
+        # apptainer exec --pwd doesn't expand ~, so prepend a cd into the command
+        if work_dir == "~" or work_dir.startswith("~/"):
+            exec_command = f"cd {work_dir} && {exec_command}"
+            work_dir = "/tmp"
+
+        cmd = [self.executable, "exec", "--pwd", work_dir,
               f"instance://{self.instance_id}",
-               "bash", "-c", self._prepare_command(command)]
+               "bash", "-c", exec_command]

        try:
            import time as _time
--- a/tools/environments/ssh.py
+++ b/tools/environments/ssh.py
@@ -24,7 +24,7 @@ class SSHEnvironment(BaseEnvironment):
    and a remote kill is attempted over the ControlMaster socket.
    """

-    def __init__(self, host: str, user: str, cwd: str = "/tmp",
+    def __init__(self, host: str, user: str, cwd: str = "~",
                 timeout: int = 60, port: int = 22, key_path: str = ""):
        super().__init__(cwd=cwd, timeout=timeout)
        self.host = host
--- a/tools/terminal_tool.py
+++ b/tools/terminal_tool.py
@@ -405,29 +405,22 @@ def _get_env_config() -> Dict[str, Any]:
    default_image = "nikolaik/python-nodejs:python3.11-nodejs20"
    env_type = os.getenv("TERMINAL_ENV", "local")
    
-    # Default cwd depends on backend:
-    #   - local: host's current working directory
-    #   - ssh: remote user's home (agent code is local, execution is remote)
-    #   - docker: / inside the container
-    #   - singularity/modal: /root (ephemeral cloud/container)
-    if env_type in ("modal", "singularity"):
-        default_cwd = "/root"
-    elif env_type == "docker":
-        default_cwd = "/"
-    elif env_type == "ssh":
-        default_cwd = "~"
-    else:
+    # Default cwd: local uses the host's current directory, everything
+    # else starts in the user's home (~ resolves to whatever account
+    # is running inside the container/remote).
+    if env_type == "local":
        default_cwd = os.getcwd()
+    else:
+        default_cwd = "~"
    
-    # Read TERMINAL_CWD but sanity-check it for non-local backends.
+    # Read TERMINAL_CWD but sanity-check it for container backends.
    # If the CWD looks like a host-local path that can't exist inside a
-    # container/sandbox, fall back to the backend's own default.  This
+    # container/sandbox, fall back to the backend's own default. This
    # catches the case where cli.py (or .env) leaked the host's CWD.
+    # SSH is excluded since /home/ paths are valid on remote machines.
    cwd = os.getenv("TERMINAL_CWD", default_cwd)
-    if env_type in ("modal", "docker", "singularity", "ssh") and cwd:
-        # Paths containing common host-only prefixes are clearly wrong
-        # inside a container.  Also catch Windows-style paths (C:\...).
-        host_prefixes = ("/Users/", "/home/", "C:\\", "C:/")
+    if env_type in ("modal", "docker", "singularity") and cwd:
+        host_prefixes = ("/Users/", "C:\\", "C:/")
        if any(cwd.startswith(p) for p in host_prefixes) and cwd != default_cwd:
            logger.info("Ignoring TERMINAL_CWD=%r for %s backend "
                        "(host path won't exist in sandbox). Using %r instead.",
@@ -1122,6 +1115,7 @@ if __name__ == "__main__":
    print(f"  TERMINAL_SINGULARITY_IMAGE: {os.getenv('TERMINAL_SINGULARITY_IMAGE', f'docker://{default_img}')}")
    print(f"  TERMINAL_MODAL_IMAGE: {os.getenv('TERMINAL_MODAL_IMAGE', default_img)}")
    print(f"  TERMINAL_CWD: {os.getenv('TERMINAL_CWD', os.getcwd())}")
+    print(f"  TERMINAL_SANDBOX_DIR: {os.getenv('TERMINAL_SANDBOX_DIR', '~/.hermes/sandboxes')}")
    print(f"  TERMINAL_TIMEOUT: {os.getenv('TERMINAL_TIMEOUT', '60')}")
    print(f"  TERMINAL_LIFETIME_SECONDS: {os.getenv('TERMINAL_LIFETIME_SECONDS', '300')}")