# Timmy private staging runbook

This runbook is a reviewed host template, not approval to change a live host. Apply it only in a separately approved provisioning change. Staging is Phase 1: Hermes Agent and vision are off.

## Prerequisites

- Linux host with Node.js 22, systemd, and Caddy.
- Existing HTTPS origin and Gitea route inventory.
- A reviewed 40-character commit, source archive, and SHA-256 receipt from the release manifest.
- Root only for one-time account/unit/config installation; promotions use the narrowest available sudo policy.
- A fresh Caddy password hash delivered outside Git. Never put the password or hash in shell history, release notes, screenshots, or this repository.

Confirm that `127.0.0.1:4174` is unused. Do not copy `.env`, credentials, model weights, raw media, or a mutable Git checkout into a release.

## DNS and URL

No DNS change is required for the approved subpage deployment. The private URL is `https://forge.alexanderwhitestone.com/timmy-staging/`. Caddy remains the only public listener; Node binds only to `127.0.0.1:4174`. Verify `/git/` before and after any separately approved Caddy reload.

## Install

These commands are reference commands for an approved maintenance window; they have not been run by this change:

```bash
NODE_SOURCE="$(readlink -f "$(command -v node)")"
"$NODE_SOURCE" --version  # must report v22.x before installation
sudo install -D -o root -g root -m 755 "$NODE_SOURCE" /usr/local/lib/timmy-staging/node
/usr/local/lib/timmy-staging/node --version
sudo useradd --system --home-dir /nonexistent --no-create-home --shell /usr/sbin/nologin timmy-staging
sudo install -d -o root -g root -m 755 /opt/timmy-staging/releases
sudo install -d -o timmy-staging -g timmy-staging -m 700 /var/lib/timmy-staging
sudo install -o root -g root -m 644 deploy/timmy-staging.service /etc/systemd/system/timmy-staging.service
sudo install -o root -g root -m 600 deploy/timmy-staging.env.example /etc/timmy-staging.env
sudo chmod 600 /etc/timmy-staging.env
sudo systemctl daemon-reload
```

Edit only `TIMMY_RELEASE_TAG` and `TIMMY_RELEASE_COMMIT` for the selected artifact. The unit deliberately applies `TIMMY_AGENT_ENABLED=false` and `TIMMY_VISION_ENABLED=false` on the `ExecStart` command after loading the environment file, so values in that file cannot enable either subsystem. Leave the loopback host, port, and base path unchanged. The environment file must stay root-owned mode 600 and absent from release archives. The copied Node 22 executable is root-owned at `/usr/local/lib/timmy-staging/node`, outside every home directory, so `ProtectHome=true` remains enforceable; replace it only through a separately reviewed Node upgrade.

Generate Caddy basic-auth material interactively (for example, `caddy hash-password`) and inject it as `TIMMY_STAGING_PASSWORD_HASH`; do not commit the output. Merge the staging handle before the existing catchall, validate a temporary complete config, then use `caddy reload` rather than restarting unrelated services.

## Promote an immutable release

First inspect without mutation:

```bash
python3 scripts/deploy_staging.py --dry-run promote \
  --tag daily-YYYY-MM-DD.N --archive /secure/inbox/timmy.tar.gz \
  --sha256 64_LOWERCASE_HEX_CHARACTERS --commit 40_LOWERCASE_HEX_CHARACTERS
```

Then run the same command without `--dry-run`. Promotion opens only a non-symlink regular source, copies it while hashing into a mode-400 file in a private temporary directory, and inspects and extracts only that verified copy before cleaning it. It rejects unsafe members and forbidden artifacts, extracts once to `/opt/timmy-staging/releases/<commit>`, and never overwrites that directory. It atomically swaps `current`, restarts only `timmy-staging.service`, checks bounded health, and invokes `npm run test:staging-smoke` as an argv array with `shell=False`. Restart, health, or smoke failure automatically restores and verifies the prior symlink. If there is no prior release, the tool removes `current` and stops the service instead of restarting it against a missing path; the failed release remains unreferenced as inert immutable evidence for operator review.

The operator must update `/etc/timmy-staging.env` release identity to the same reviewed tag and commit before promotion. A narrow wrapper/sudo policy may set the smoke environment without granting arbitrary command execution:

```bash
export TIMMY_STAGING_URL=https://forge.alexanderwhitestone.com/timmy-staging/
export TIMMY_STAGING_USER=staging
# Read TIMMY_STAGING_PASSWORD from an approved secret channel; never paste it here.
sudo -E python3 scripts/deploy_staging.py promote --tag "$TAG" --archive "$ARCHIVE" --sha256 "$SHA256" --commit "$COMMIT"
```

## Smoke test

The promotion runs the synthetic 390×844 Playwright suite. It checks edge auth, build identity, one dominant photo action, manual save, urgent-language interception, Journal visibility, export, no horizontal overflow, browser errors, and suspicious secret-bearing responses. It must never use a real stool photo or medical record.

For an explicit rerun:

```bash
TIMMY_STAGING_URL=https://forge.alexanderwhitestone.com/timmy-staging/ \
TIMMY_EXPECT_RELEASE_TAG="$TAG" TIMMY_EXPECT_RELEASE_COMMIT="$COMMIT" \
TIMMY_STAGING_USER=staging TIMMY_STAGING_PASSWORD="$(secret-reader)" \
npm run test:staging-smoke
```

Also verify `curl --fail http://127.0.0.1:4174/timmy-staging/api/healthz`, authenticated public health, unauthenticated HTTP 401, `/git/`, and that no wildcard/public listener owns port 4174.

## Status and logs

```bash
python3 scripts/deploy_staging.py status
sudo systemctl status timmy-staging.service
sudo journalctl -u timmy-staging.service --since today --no-pager
# Only after an approved environment-file change:
sudo systemctl restart timmy-staging.service
```

Health and logs may show bounded release identity, but must not show environment values, cookies, passwords, tokens, session data, filesystem secrets, or raw user content.

## Rollback

List reviewed immutable commit directories, choose the known-good receipt, dry-run, then roll back:

```bash
python3 scripts/deploy_staging.py --dry-run rollback --commit 40_LOWERCASE_HEX_CHARACTERS
sudo python3 scripts/deploy_staging.py rollback --commit 40_LOWERCASE_HEX_CHARACTERS
```

Rollback atomically repoints `current`, restarts only Timmy, and verifies commit-specific loopback health. If rollback verification fails, the tool restores the release that was current when rollback began. Never repoint the symlink manually while the service is running.

## Backup

Browser journal data is local to each browser and is not server state. Before host maintenance, back up only operator-owned receipts and any explicitly required `/var/lib/timmy-staging` state with root-only permissions. Release archives should be recoverable from their verified release source; do not back up `/etc/timmy-staging.env` into a general artifact store. Test restore procedures without live credentials.

## Template validation

```bash
node --test tests/staging-config.test.js
systemd-analyze verify deploy/timmy-staging.service
HASH="$(caddy hash-password --plaintext validation-only-password)"
TIMMY_STAGING_PASSWORD_HASH="$HASH" caddy validate --config deploy/Caddyfile.staging.example --adapter caddyfile
python3 -m py_compile scripts/deploy_staging.py
```

Use only a disposable validation hash. Validation does not authorize installation or reload.

## Remove staging

In a separately approved maintenance window: disable and stop only `timmy-staging.service`; remove only the reviewed Caddy staging handle and validate/reload Caddy; verify `/git/`; remove the unit and run `systemctl daemon-reload`; archive required receipts; then remove `/opt/timmy-staging` and `/var/lib/timmy-staging`. Delete the dedicated locked user last. Do not delete shared Caddy, Gitea, TLS, or browser data.