feat: add the-door codebase genome (#673)

tests/docs/test_the_door_genome.py

@@ -0,0 +1,35 @@
+from pathlib import Path
+
+
+def _content() -> str:
+    return Path("the-door-GENOME.md").read_text()
+
+
+def test_the_door_genome_exists() -> None:
+    assert Path("the-door-GENOME.md").exists()
+
+
+def test_the_door_genome_has_required_sections() -> None:
+    content = _content()
+    assert "# GENOME.md — the-door" in content
+    assert "## Project Overview" in content
+    assert "## Architecture" in content
+    assert "```mermaid" in content
+    assert "## Entry Points" in content
+    assert "## Data Flow" in content
+    assert "## Key Abstractions" in content
+    assert "## API Surface" in content
+    assert "## Test Coverage Gaps" in content
+    assert "## Security Considerations" in content
+    assert "## Dependencies" in content
+    assert "## Deployment" in content
+    assert "## Technical Debt" in content
+
+
+def test_the_door_genome_captures_repo_specific_findings() -> None:
+    content = _content()
+    assert "lastUserMessage" in content
+    assert "localStorage" in content
+    assert "crisis-offline.html" in content
+    assert "hermes-gateway.service" in content
+    assert "/api/v1/chat/completions" in content

the-door-GENOME.md

@@ -0,0 +1,419 @@
+# GENOME.md — the-door
+
+Generated: 2026-04-15 00:03:16 EDT
+Repo: Timmy_Foundation/the-door
+Issue: timmy-home #673
+
+## Project Overview
+
+The Door is a crisis-first front door to Timmy: one URL, no account wall, no app install, and a permanently visible 988 escape hatch. The repo combines a static browser UI, a local Hermes API gateway behind nginx, and a Python crisis package that duplicates and enriches the frontend's safety logic.
+
+What the codebase actually contains today:
+- 1 primary browser app: `index.html`
+- 4 companion browser assets/pages: `about.html`, `testimony.html`, `crisis-offline.html`, `sw.js`
+- 17 Python files across canonical crisis logic, legacy shims, wrappers, and tests
+- 2 Gitea workflows: `smoke.yml`, `sanity.yml`
+- 1 systemd unit: `deploy/hermes-gateway.service`
+- full test suite currently passing: `115 passed, 3 subtests passed`
+
+The repo is small, but it is not simple. The true architecture is a layered safety system:
+1. immediate browser-side crisis escalation
+2. OpenAI-compatible streaming chat through Hermes
+3. canonical Python crisis detection and response modules
+4. nginx hardening, rate limiting, and localhost-only gateway exposure
+5. service-worker offline fallback for crisis resources
+
+The strongest pattern in this codebase is safety redundancy: the UI, prompt layer, offline fallback, and backend detection all try to catch the same sacred failure mode from different directions.
+
+## Architecture
+
+```mermaid
+graph TD
+    U[User in browser] --> I[index.html chat app]
+    I --> K[Client-side crisis detection\ncrisisKeywords + explicitPhrases]
+    K --> P[Inline crisis panel]
+    K --> O[Fullscreen crisis overlay]
+    I --> L[localStorage\nchat history + safety plan]
+    I --> SW[sw.js service worker]
+    SW --> OFF[crisis-offline.html]
+
+    I --> API[/POST /api/v1/chat/completions/]
+    API --> NGINX[nginx reverse proxy]
+    NGINX --> H[Hermes Gateway :8644]
+    NGINX --> HC[/health proxy]
+
+    H --> G[crisis/gateway.py]
+    G --> D[crisis/detect.py]
+    G --> R[crisis/response.py]
+    D --> CR[CrisisDetectionResult]
+    R --> RESP[CrisisResponse]
+    D --> LEG[Legacy shims\ncrisis_detector.py\ncrisis_responder.py\ndying_detection]
+
+    DEP[deploy/playbook.yml\ndeploy/deploy.sh\nhermes-gateway.service] --> NGINX
+    DEP --> H
+    CI[.gitea/workflows\nsmoke.yml + sanity.yml] --> I
+    CI --> D
+```
+
+## Entry Points
+
+### Browser / user-facing entry points
+- `index.html`
+  - the main product
+  - contains inline CSS, inline JS, embedded `SYSTEM_PROMPT`, chat UI, crisis panel, fullscreen overlay, and safety-plan modal
+- `about.html`
+  - static about page
+  - linked from the chat footer, though the main app currently links to `/about` while the repo ships `about.html`
+- `testimony.html`
+  - static companion content page
+- `crisis-offline.html`
+  - offline crisis resource page served by the service worker when navigation cannot reach the network
+- `manifest.json`
+  - PWA metadata and shortcuts, including `/?safetyplan=true` and `tel:988`
+- `sw.js`
+  - network-first service worker with offline crisis fallback
+
+### Backend / Python entry points
+- `crisis/detect.py`
+  - canonical detection engine and public detection API
+- `crisis/response.py`
+  - canonical response generator, UI flags, prompt modifier, grounding helpers
+- `crisis/gateway.py`
+  - integration layer for `check_crisis()` and `get_system_prompt()`
+- `crisis/compassion_router.py`
+  - profile-based prompt routing abstraction parallel to `response.py`
+- `crisis_detector.py`
+  - root legacy shim exposing canonical detection in older shapes
+- `crisis_responder.py`
+  - root legacy response module with a richer compatibility response contract
+- `dying_detection/__init__.py`
+  - deprecated wrapper around canonical detection
+
+### Operational entry points
+- `deploy/deploy.sh`
+  - most complete one-command operational bootstrap path in the repo
+- `deploy/playbook.yml`
+  - Ansible provisioning path for swap, packages, nginx, firewall, and site files
+- `deploy/hermes-gateway.service`
+  - systemd unit running `hermes gateway --platform api_server --port 8644`
+- `.gitea/workflows/smoke.yml`
+  - parse/syntax checks and secret scan
+- `.gitea/workflows/sanity.yml`
+  - basic repo sanity grep checks for 988/system-prompt presence
+
+## Data Flow
+
+### Happy path: user message to streamed response
+1. User types into `#msg-input` in `index.html`.
+2. `sendMessage()`:
+   - trims text
+   - appends a user bubble to the DOM
+   - pushes `{role: 'user', content: text}` into the in-memory `messages` array
+   - runs client-side `checkCrisis(text)`
+   - clears the input and starts streaming
+3. `streamResponse()` builds the request payload:
+   - prepends a synthetic system message from `getSystemPrompt(lastUserMessage || '')`
+   - posts JSON to `/api/v1/chat/completions`
+4. nginx proxies `/api/*` to `127.0.0.1:8644`.
+5. Hermes streams OpenAI-style SSE chunks back to the browser.
+6. The browser reads `choices[0].delta.content` and incrementally renders the assistant message.
+7. When streaming ends, the assistant turn is pushed into `messages`, saved to `localStorage`, and passed through `checkCrisis(fullText)` again.
+
+### Immediate local crisis escalation path
+1. `checkCrisis(text)` scans substrings against two client-side lists.
+2. Low-tier/soft crisis text reveals the inline crisis panel.
+3. Explicit intent text triggers the fullscreen overlay and delayed-dismiss flow.
+4. The user still remains in the conversation flow rather than being hard-redirected away.
+
+### Offline / failure path
+1. `sw.js` precaches static routes and the crisis fallback page.
+2. Navigation uses a network-first strategy with timeout fallback.
+3. If network and cache both fail, the service worker tries `crisis-offline.html`.
+4. If API streaming fails, `index.html` inserts a static emergency message with 988 and 741741 instead of a blank error.
+
+## Key Abstractions
+
+### 1. `SYSTEM_PROMPT`
+Embedded directly in `index.html`, not loaded at runtime from `system-prompt.txt`. The browser treats the prompt as part of the application runtime contract.
+
+### 2. `COMPASSION_PROFILES`
+Frontend prompt-state profiles for `CRITICAL`, `HIGH`, `MEDIUM`, `LOW`, and `NONE`. They encode tone and directive shifts, but the current `levelMap` only maps browser levels to `NONE`, `MEDIUM`, and `CRITICAL`, leaving `HIGH` and `LOW` effectively unused in the main prompt-building path.
+
+### 3. Client-side crisis detector
+In `index.html`, the browser uses:
+- `crisisKeywords` for panel escalation
+- `explicitPhrases` for hard overlay escalation
+- `checkCrisis(text)` for UI behavior
+- `getCrisisLevel(text)` for prompt shaping
+
+This is fast and local, but it is also a separate detector from the canonical Python package.
+
+### 4. `CrisisDetectionResult`
+The core canonical backend dataclass from `crisis/detect.py`:
+- `level`
+- `indicators`
+- `recommended_action`
+- `score`
+- `matches`
+
+This is the canonical representation shared by the main Python crisis stack.
+
+### 5. `CrisisResponse`
+In `crisis/response.py`, the canonical response dataclass ties backend detection to frontend/UI needs:
+- `timmy_message`
+- `show_crisis_panel`
+- `show_overlay`
+- `provide_988`
+- `escalate`
+
+### 6. Legacy compatibility layer
+The repo still carries older interfaces:
+- `crisis_detector.py`
+- `crisis_responder.py`
+- `dying_detection/__init__.py`
+
+These preserve compatibility, but they also create drift risk:
+- `MEDIUM` vs `MODERATE`
+- two different `CrisisResponse` contracts
+- two prompt-routing paths (`response.py` vs `compassion_router.py`)
+
+### 7. Browser persistence contract
+`localStorage` is a real part of runtime state despite some docs claiming otherwise.
+Keys:
+- `timmy_chat_history`
+- `timmy_safety_plan`
+
+That means The Door is not truly “close tab = gone” in its current implementation.
+
+## API Surface
+
+### Browser -> Hermes API contract
+`index.html` sends:
+
+```json
+{
+  "model": "timmy",
+  "messages": [
+    {"role": "system", "content": "...prompt..."},
+    {"role": "assistant", "content": "..."},
+    {"role": "user", "content": "..."}
+  ],
+  "stream": true
+}
+```
+
+Endpoint:
+- `/api/v1/chat/completions`
+
+Expected response shape:
+- streaming SSE lines beginning with `data: `
+- chunk payloads with `choices[0].delta.content`
+- `[DONE]` terminator
+
+### Canonical Python API
+- `crisis.detect.detect_crisis(text)`
+- `crisis.response.generate_response(detection)`
+- `crisis.response.process_message(text)`
+- `crisis.response.get_system_prompt_modifier(detection)`
+- `crisis.gateway.check_crisis(text)`
+- `crisis.gateway.get_system_prompt(base_prompt, text="")`
+- `crisis.gateway.format_gateway_response(text, pretty=True)`
+
+### Legacy / compatibility API
+- `CrisisDetector.scan()`
+- `detect_crisis_legacy()`
+- root `crisis_responder.generate_response()`
+- deprecated `dying_detection.detect()` and helpers
+
+## Test Coverage Gaps
+
+### Current state
+Verified on fresh `main` clone of `the-door`:
+- `python3 -m pytest -q` -> `115 passed, 3 subtests passed`
+
+What is already covered well:
+- canonical crisis detection tiers
+- response flags and gateway structure
+- many false-positive regressions
+- service-worker offline crisis fallback
+- crisis overlay focus trap string-level assertions
+- deprecated wrapper behavior
+
+### High-value gaps that still matter
+1. No real browser test of the actual send path in `index.html`.
+   - The repo currently contains a concrete scope bug:
+     - `sendMessage()` defines `var lastUserMessage = text;`
+     - `streamResponse()` later uses `getSystemPrompt(lastUserMessage || '')`
+     - `lastUserMessage` is not in `streamResponse()` scope
+   - Existing passing tests do not execute this real path.
+
+2. No DOM-true test for overlay background locking.
+   - The overlay code targets `document.querySelector('.app')` and `getElementById('chat')`.
+   - The main document uses `id="app"`, not `.app`, and does not expose a `#chat` node.
+   - Current tests assert code presence, not selector correctness.
+
+3. No route validation for `/about` vs `about.html`.
+   - The footer links to `/about`.
+   - The repo ships `about.html`.
+   - With current nginx `try_files`, this looks like a drift bug.
+
+4. Legacy responder path remains largely untested.
+   - `crisis_responder.py` is still present and meaningful but lacks direct tests for its richer response payloads.
+
+5. CI does not run pytest.
+   - The repo has a substantial suite, but Gitea workflows only do syntax/grep checks.
+
+### Generated missing tests for critical paths
+These are the three most important tests this codebase still needs.
+
+#### A. Browser send-path smoke test
+Goal: catch the `lastUserMessage` regression and ensure the chat request actually builds.
+
+```python
+# Example Playwright/browser test
+async def test_send_message_builds_stream_request(page):
+    await page.goto("file:///.../index.html")
+    await page.fill("#msg-input", "hello")
+    await page.click("#send-btn")
+    # Expect no ReferenceError and one request to /api/v1/chat/completions
+```
+
+#### B. Overlay selector correctness test
+Goal: prove the inert/background lock hits real DOM nodes, not dead selectors.
+
+```python
+def test_overlay_background_selectors_match_real_dom():
+    html = Path("index.html").read_text()
+    assert 'id="app"' in html
+    assert "querySelector('.app')" not in html
+    assert "getElementById('chat')" not in html
+```
+
+#### C. Legacy responder contract test
+Goal: keep compatibility layers honest until they are deleted.
+
+```python
+from crisis_responder import process_message
+
+def test_legacy_responder_returns_resources_for_high_risk():
+    response = process_message("I want to kill myself")
+    assert response.escalate is True
+    assert response.show_overlay is True
+    assert any("988" in r for r in response.resources)
+```
+
+## Security Considerations
+
+### Strengths
+- Browser message bubbles use `textContent`, not unsafe inner HTML, for chat content.
+- API calls are same-origin and proxied through nginx.
+- Service worker does not cache `/api/*` responses.
+- nginx includes CSP, HSTS, and localhost-only gateway exposure.
+- UFW/docs expect only `22`, `80`, and `443` to be public.
+- systemd unit hardening is present in `hermes-gateway.service`.
+
+### Risks
+1. `localStorage` persistence contradicts the privacy story.
+   - chat history and safety plan are stored in plaintext on the device
+   - shared-device risk is real
+
+2. `script-src 'unsafe-inline'` is required by the current architecture.
+   - all runtime logic and CSS are inline in `index.html`
+   - this weakens CSP/XSS posture
+
+3. Safety enforcement is still heavily client-shaped.
+   - the frontend always embeds the crisis-aware prompt
+   - deployment does not clearly prove that all callers are forced through server-side crisis middleware
+   - direct API clients may bypass browser-supplied context
+
+4. Client and server detection logic can drift.
+   - the browser uses substring lists
+   - the backend uses canonical regex tiers in `crisis/detect.py`
+   - parity is not tested
+
+5. Deprecated wrapper emits a deterministic session hash.
+   - `dying_detection` exposes a truncated SHA-256 fingerprint of text
+   - useful for correlation, but still privacy-sensitive
+
+## Dependencies
+
+### Runtime
+- Hermes binary at `/usr/local/bin/hermes`
+- nginx
+- certbot + python certbot nginx plugin
+- ufw
+- curl
+- Python 3
+- browser with JavaScript, service-worker, and `localStorage` support
+
+### Test / operator dependencies
+- pytest
+- PyYAML (used implicitly by smoke workflow checks)
+- ansible / ansible-playbook
+- rsync, ssh, scp
+- openssl
+- dig / dnsutils
+
+### In-repo dependency style
+- Python code is effectively stdlib-first
+- no `requirements.txt`, `pyproject.toml`, or `package.json`
+- operational dependencies live mostly in docs and scripts rather than a declared manifest
+
+## Deployment
+
+### Intended production path
+Browser -> nginx TLS -> static webroot + `/api/*` reverse proxy -> Hermes on `127.0.0.1:8644`
+
+### Main deployment commands
+- `make deploy`
+- `make deploy-bash`
+- `make push`
+- `make check`
+- `bash deploy/deploy.sh`
+- `cd deploy && ansible-playbook -i inventory.ini playbook.yml`
+
+### Operational files
+- `deploy/nginx.conf`
+- `deploy/playbook.yml`
+- `deploy/deploy.sh`
+- `deploy/hermes-gateway.service`
+- `resilience/health-check.sh`
+- `resilience/service-restart.sh`
+
+### Deployment reality check
+The repo's deploy surface is not fully coherent:
+- `deploy/deploy.sh` is the most complete operational path
+- `deploy/playbook.yml` provisions nginx/site/firewall/SSL but does not manage `hermes-gateway.service`
+- resilience scripts still target port `8000`, not the real gateway at `8644`
+- `crisis-offline.html` is required by `sw.js`, but full deploy paths do not appear to ship it consistently
+
+## Technical Debt
+
+### Highest-priority debt
+1. Fix the `lastUserMessage` scope bug in `index.html`.
+2. Fix overlay background selector drift (`.app` vs `#app`, missing `#chat`).
+3. Fix `/about` route drift.
+4. Add pytest to Gitea CI.
+5. Make deploy paths ship the same artifact set, including `crisis-offline.html`.
+6. Make the recommended Ansible path actually manage `hermes-gateway.service`.
+7. Align or remove resilience scripts targeting the wrong port/service.
+8. Resolve doc drift:
+   - ARCHITECTURE says “close tab = gone,” but implementation uses `localStorage`
+   - BACKEND_SETUP still says 49 tests, while current verified suite is 115 + 3 subtests
+   - audit docs understate current automation coverage
+
+### Strategic debt
+- Duplicate crisis logic across browser and backend
+- Parallel prompt-routing mechanisms (`response.py` and `compassion_router.py`)
+- Legacy compatibility layers that still matter but are not first-class tested
+- No declared dependency manifest for operator tooling
+- No true E2E browser validation of the core conversation loop
+
+## Bottom Line
+
+The Door is not just a static landing page. It is a small but layered safety system with three cores:
+- a browser-first crisis chat UI
+- a canonical Python crisis package
+- a thin nginx/Hermes deployment shell
+
+Its design is morally serious and operationally pragmatic. Its main weaknesses are not missing ambition; they are drift, duplication, and shallow verification at the exact seams where the browser, backend, and deploy layer meet.