Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
58b7feddc8 | ||
|
|
d027a1e791 | ||
|
|
503efaf959 | ||
|
|
29ea045671 |
No files matched your search
@@ -0,0 +1,157 @@
|
||||
# Photo Data-Flow Threat Model — End-to-End Photo Path
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Issue:** [#20 — Write the end-to-end photo data-flow threat model](https://forge.alexanderwhitestone.com/git/stackchain/timmy-talking-turd/issues/20)
|
||||
- **Parent epic:** [#5 — Clinical safety, privacy & security](https://forge.alexanderwhitestone.com/git/stackchain/timmy-talking-turd/issues/5)
|
||||
- **Milestone:** M1 — Sovereign Photo Intelligence
|
||||
- **Date:** 2026-10-10
|
||||
- **Scope:** Every byte of a user's photo from capture to deletion on the `/api/analyze` path, plus the logs, exports, and consent records that touch it.
|
||||
|
||||
## Why this exists
|
||||
|
||||
Photo-first logging asks a user to hand Timmy an image of the most private output their body produces. That image is sensitive medical-adjacent data by default, so the data-flow boundaries must be explicit before any private beta, and every abuse case that is not yet fully mitigated must name an owner and a tracked remediation ticket. This model covers the browser, the API, the model worker, transient buffers, the training opt-in store, logs, exports, and deletion.
|
||||
|
||||
## Product boundary (unchanged by this model)
|
||||
|
||||
Timmy is non-diagnostic. AI may suggest only visible Bristol form, broad color, and image quality; every suggestion stays provisional until the user confirms or corrects it. Timmy must never diagnose disease, identify bleeding conclusively, infer pain, urgency, fever, or vomiting, or clear a food. Symptoms stay user-entered, and journal save is a separate decision from training contribution. Deterministic red-flag escalation runs on user-reported answers and is never model-gated or model-suppressible.
|
||||
|
||||
No real medical images are ever added to this repository, its fixtures, its tests, or its Git history. Every fixture used in the photo path is synthetic.
|
||||
|
||||
## Data-flow diagram
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Browser["Browser — camera or gallery pick, in-memory draft only"]
|
||||
APIIngress["API ingress — server.mjs /api/analyze"]
|
||||
TransientBuffers["Transient buffers — bounded body bytes, decode canvas"]
|
||||
ModelWorker["Model worker — self-hosted vision, private loopback only"]
|
||||
OptInStore["Training opt-in store — consented contributions only"]
|
||||
Logs["Sanitized logs — counts and reasons, never bytes or base64"]
|
||||
Exports["Exports — user-initiated JSON portability"]
|
||||
Deletion["Deletion — withdrawal, purge, and tombstones"]
|
||||
|
||||
Browser -->|"image bytes plus explicit consent"| APIIngress
|
||||
APIIngress -->|"length, dimension, and rate limits first"| TransientBuffers
|
||||
TransientBuffers -->|"re-encoded baseline JPEG, metadata stripped"| ModelWorker
|
||||
ModelWorker -->|"observable fields only — image dropped, never persisted"| APIIngress
|
||||
APIIngress -->|"no image bytes"| Logs
|
||||
Browser --> Exports
|
||||
OptInStore --> Deletion
|
||||
TransientBuffers --> Deletion
|
||||
Logs --> Deletion
|
||||
```
|
||||
|
||||
## Trust boundaries
|
||||
|
||||
| # | Boundary | Trust posture |
|
||||
|---|---|---|
|
||||
| B1 | Browser to API ingress | Hostile network. Treat every upload as hostile and sensitive: sniff content, cap size and dimensions, rate-limit the client, and never trust the filename or declared MIME type. |
|
||||
| B2 | API ingress to model worker | Private loopback only. The worker is never exposed publicly and never receives a user identity. |
|
||||
| B3 | Model worker to provider or hosted profile | Disabled by default. When enabled, the operator carries the decision, the image is the only payload, and no ledger, symptom, or session data travels with it. |
|
||||
| B4 | App process to logs | One-way and lossy: counts, reasons, and durations leave; bytes, base64, filenames, and symptom text do not. |
|
||||
| B5 | App process to consent records | Contribution requires its own opt-in with a versioned receipt; journal save never implies contribution. |
|
||||
| B6 | Host to operator | The operator can reach data the app holds. Least privilege, documented purge, and no image bytes in operator-facing logs. |
|
||||
|
||||
## Components
|
||||
|
||||
### C1 — Browser
|
||||
|
||||
Holds the captured image as an in-memory draft and renders the consent prompt. A photo awaiting consent is deliberately not written to storage, so it cannot survive the tab by accident, and the service worker never caches analysis requests or image bodies. Lifetime: until the tab closes, consent is refused, or the draft is cleared.
|
||||
|
||||
### C2 — API ingress
|
||||
|
||||
`server.mjs` `/api/analyze` is the only door for image bytes. It owns content sniffing, dimension and body caps, re-encode, metadata stripping, rate control, and the sanitized error vocabulary. Lifetimes are per request: nothing about an image is written to disk.
|
||||
|
||||
### C3 — Model worker
|
||||
|
||||
A self-hosted vision process bound to private loopback. It receives the re-encoded image and returns schema-constrained observable fields or an abstention. It returns no free text, and its output is provisional until the user confirms it.
|
||||
|
||||
### C4 — Transient buffers
|
||||
|
||||
Request body buffer, decode canvas, and the re-encode pipe. These are the moments the image exists in readable form; they are bounded, per request, and dropped when the request ends, on success or failure alike.
|
||||
|
||||
### C5 — Training opt-in store
|
||||
|
||||
Only images and labels a user explicitly contributed, each with consent version, receipt, and provenance. This store is not on the logging path: an ordinary journal save never reaches it.
|
||||
|
||||
### C6 — Sanitized logs
|
||||
|
||||
Operational logs for health, latency, abstention, and failure. They carry counts, reason codes, and durations; they never carry image bytes, base64, the original filename, or user symptom text.
|
||||
|
||||
### C7 — Exports
|
||||
|
||||
User-initiated JSON portability of the user's own ledger. Exports are generated on request, are not written server-side, and deliberately exclude photo bytes.
|
||||
|
||||
### C8 — Deletion
|
||||
|
||||
Withdrawal, purge, and tombstones. Deletion is addressable by contribution ID so it can reach derived artifacts, and a tombstone records that deletion happened without keeping what was deleted.
|
||||
|
||||
## Threat register
|
||||
|
||||
Each entry names the abuse case, the mitigation that is already committed or queued, the owning role, the tracked ticket that will prove the mitigation, and the artifact that will count as evidence.
|
||||
|
||||
### T1 — Replay and re-submission
|
||||
|
||||
- **Scenario:** An attacker replays a captured upload to exhaust inference capacity, correlate timing with a user session, or force repeated provider work from one image.
|
||||
- **Impact:** Denial of service for real users, cost amplification on hosted profiles, and weak linkage between a session and its image bytes.
|
||||
- **Mitigation:** Per-client fixed-window rate control on `/api/analyze` that fails with `429` and `Retry-After` before any provider work, opaque client identity held server-side rather than in the image payload, and no retention of over-limit payloads.
|
||||
- **Owner:** Timmy (ingress, area/backend)
|
||||
- **Testable follow-up:** #16
|
||||
- **Evidence:** An ingress test that drives the limiter past its window and asserts a rejected request never reaches the provider, with a separate assertion that no over-limit payload is retained.
|
||||
|
||||
### T2 — Oversized input
|
||||
|
||||
- **Scenario:** A decompression bomb, an enormous dimension count, or a multi-hundred-megabyte body arrives to consume worker memory, block the queue, or stall the request path indefinitely.
|
||||
- **Impact:** Worker exhaustion and queue starvation that degrade logging availability, up to a full outage of photo analysis.
|
||||
- **Mitigation:** Decoded-body, per-dimension, and timeout caps enforced before any provider or worker work; a bounded inference queue with cancellation and an overload fallback that returns the manual path instead of hanging.
|
||||
- **Owner:** Timmy (ingress, area/backend)
|
||||
- **Testable follow-up:** #17
|
||||
- **Evidence:** A queue-full and body-timeout test proving the request is rejected inside its budget and that the manual Bristol path remains one action away, per the acceptance criteria of #17.
|
||||
|
||||
### T3 — Metadata leakage
|
||||
|
||||
- **Scenario:** An uploaded photo carries EXIF, GPS, device identifiers, XMP, or PNG text chunks; or a failure path echoes base64 image bytes into an error message or a log line.
|
||||
- **Impact:** Location and device deanonymization, plus durable copies of the image outside the intended boundary, including in logs.
|
||||
- **Mitigation:** Content sniffing independent of filename and MIME, safe re-encode to baseline JPEG with a fixed argv, metadata dropped by construction, polyglot and malformed inputs rejected, and a fixed sanitized error vocabulary with no bytes, no base64, and no stack traces.
|
||||
- **Owner:** Timmy (ingress, area/backend)
|
||||
- **Testable follow-up:** #16
|
||||
- **Evidence:** A re-encode test asserting EXIF, GPS, XMP, and PNG text chunks are absent from the accepted bytes, paired with a log-scan test asserting no image bytes or base64 appear in request or error logs.
|
||||
|
||||
### T4 — Dataset poisoning
|
||||
|
||||
- **Scenario:** Mislabeled, duplicated, or adversarially crafted contributions enter the training corpus and shift the classifier toward a wrong or unsafe behavior, including teaching it to call a not-stool image a stool form.
|
||||
- **Impact:** Silent accuracy loss that is invisible in aggregate metrics and could undermine abstention exactly where the product must fail closed.
|
||||
- **Mitigation:** Immutable provenance per contribution, contributor-level splits so a single contributor cannot straddle train and evaluation, a reviewer and adjudication tool for form, color, quality, and exclusions, and a published dataset card with the labeling guide and split policy.
|
||||
- **Owner:** Privacy reviewer (human gate), with Timmy as release authority
|
||||
- **Testable follow-up:** #29
|
||||
- **Evidence:** The contributor-held-out evaluation harness and calibration report, with the reviewer and adjudication tool as the labeling control.
|
||||
|
||||
### T5 — Unauthorized training
|
||||
|
||||
- **Scenario:** A journal save is silently treated as consent, an image is trained on without a receipt, or a withdrawal fails to reach a derived artifact so contribution survives deletion.
|
||||
- **Impact:** Use of private data without consent, which breaks the product promise and the beta consent gate, and is irreversible once a model is trained.
|
||||
- **Mitigation:** Contribution is a separate opt-in decision with a versioned consent receipt shown before the data moves; journal save defaults to no contribution; withdrawal is addressable by contribution ID and reaches every derived artifact; journal saving never requires contribution.
|
||||
- **Owner:** Privacy reviewer (human gate)
|
||||
- **Testable follow-up:** #24
|
||||
- **Evidence:** An opt-in and withdrawal test proving journal save contributes nothing by default, that the consent version and receipt are shown before contribution, and that a withdrawal removes every derived artifact by contribution ID, with tombstone handling from #25.
|
||||
|
||||
### T6 — Operator access
|
||||
|
||||
- **Scenario:** The host operator — or anyone who reaches the host with the operator's privilege — reads image bytes, symptom text, or consent records from process memory, on-disk state, backups, or logs.
|
||||
- **Impact:** A single privileged reader can deanonymize a user's most private log, and an undocumented backup can outlive a deletion request.
|
||||
- **Mitigation:** Least-privilege host access, worker bound to private loopback with no public exposure, no image bytes or base64 in operator-facing logs, documented backup and rollback boundaries, and a purge procedure that names every place an image can exist.
|
||||
- **Owner:** Operator (host owner), with Timmy as release authority
|
||||
- **Testable follow-up:** #39
|
||||
- **Evidence:** The production topology, secrets, backup, and rollback runbook, exercised against the deletion path so that purge reaches memory, temporary files, logs, and backups, with observability from #41 confirming no bytes reach the metrics surface.
|
||||
|
||||
## Accepted residual risks
|
||||
|
||||
- **Host compromise.** Data in a running process can be read by a sufficiently privileged attacker. This is accepted for the private beta and bounded by least privilege plus the purge procedure in #39; it is not claimed as eliminated.
|
||||
- **Model accuracy.** The photo prefill remains blocked behind abstention and specialist-model evidence. A wrong or confident suggestion is treated as a user-correction problem, and the user remains the author of every saved entry, per #32 and the calibration work in #4.
|
||||
- **Provider profiles.** A hosted OpenAI-compatible profile moves the image off-host by definition. It stays opt-in, operator-controlled, and off by default, with readiness reporting before it is offered, per #19.
|
||||
|
||||
## Verification
|
||||
|
||||
- This document is enforced by `tests/threat-model.test.js`, which asserts that the diagram covers every byte-bearing component, that each required abuse case is a numbered threat with an owner and a tracked follow-up ticket, and that the model keeps the non-diagnostic boundary and ships no real medical images.
|
||||
- The test runs in `npm test` and therefore inside the CI quality gate and the release build, so the threat model cannot silently drift from the code it describes.
|
||||
- Boundary and boundary-adjacent claims are reviewed by the clinical and privacy gate (#22, #23) before beta; this model does not move that gate, it feeds it.
|
||||
@@ -59,3 +59,4 @@ Routine provisioning, deployment, label sweeps, and release-candidate assembly a
|
||||
- [Product brief and safety boundary](../PRODUCT.md)
|
||||
- [AI execution evidence](../AI-EVIDENCE.md)
|
||||
- [Sovereign vision decision and measured limitations](../research/SELF-HOSTED-STOOL-VISION.md)
|
||||
- [Photo data-flow threat model](./PHOTO-DATAFLOW-THREAT-MODEL.md)
|
||||
+1
-1
@@ -4,7 +4,7 @@
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"test": "node --test tests/domain.test.js tests/analysis.test.js tests/vision-service.test.js tests/vision-config.test.js tests/draft.test.js tests/suggestion.test.js tests/followup.test.js tests/vision-readiness.test.js tests/hermes-agent-service.test.js tests/agent-gateway.acceptance.test.js tests/staging-health.test.js tests/service-worker-runtime.test.js tests/training-ingest.test.js tests/ci-workflow.test.js tests/product-decisions.test.js tests/release-demo.test.js tests/selfhost-bootstrap.test.js tests/staging-config.test.js",
|
||||
"test": "node --test tests/domain.test.js tests/analysis.test.js tests/vision-service.test.js tests/vision-config.test.js tests/draft.test.js tests/suggestion.test.js tests/followup.test.js tests/vision-readiness.test.js tests/hermes-agent-service.test.js tests/agent-gateway.acceptance.test.js tests/staging-health.test.js tests/service-worker-runtime.test.js tests/training-ingest.test.js tests/ci-workflow.test.js tests/product-decisions.test.js tests/release-demo.test.js tests/selfhost-bootstrap.test.js tests/staging-config.test.js tests/threat-model.test.js",
|
||||
"test:ui": "node tests/ui.acceptance.mjs",
|
||||
"test:photo": "node tests/photo-first.acceptance.mjs",
|
||||
"test:mobile-capture": "node tests/mobile-capture.acceptance.mjs",
|
||||
|
||||
@@ -26,6 +26,19 @@ SECRET_PATTERNS = (
|
||||
re.compile(r"(?i)(?:api[_-]?key|access[_-]?token|authorization)\s*[:=]\s*['\"](?!\.\.\.|top-secret|test-|example|placeholder)[A-Za-z0-9_./+:-]{16,}"),
|
||||
)
|
||||
|
||||
# Captions that must be proven on camera (via the demo's own receipt) for the release to pass.
|
||||
DEMO_REQUIRED_CAPTIONS = (
|
||||
"One clear photo action. Manual logging stays one tap away.",
|
||||
"Nothing uploads until explicit consent",
|
||||
"The suggestion stays provisional — nothing is applied or saved until you confirm it",
|
||||
"Type 2 is hard or lumpy, so Timmy asks about soreness, hydration, blood, and black-or-tarry — never guessed from the photo",
|
||||
"Vision worker offline — the AI and consent steps disappear instead of failing after upload",
|
||||
"Every byte-bearing control the model names, in the running app: local ledger, consent-gated upload, portable export, full deletion",
|
||||
"Delete all local data clears the ledger in this browser; there is no server copy of it to keep",
|
||||
"Hermes Agent connected — free-text, contextual, and server-side",
|
||||
"Urgent language is intercepted deterministically before Hermes",
|
||||
)
|
||||
|
||||
|
||||
def run(args: list[str], cwd: Path, *, capture: bool = False, env: dict[str, str] | None = None) -> str:
|
||||
print("+", " ".join(args), flush=True)
|
||||
@@ -74,6 +87,10 @@ def main() -> int:
|
||||
status = run(["git", "status", "--porcelain"], ROOT, capture=True)
|
||||
if status:
|
||||
raise SystemExit("Release requires a clean committed tree")
|
||||
release_delta = os.environ.get("TIMMY_RELEASE_DELTA",
|
||||
"Verified rebuild of the current user-visible feature path — no new feature is claimed.").strip()
|
||||
if not release_delta:
|
||||
raise SystemExit("TIMMY_RELEASE_DELTA must not be empty")
|
||||
commit = run(["git", "rev-parse", "HEAD"], ROOT, capture=True)
|
||||
short = commit[:12]
|
||||
commit_epoch = int(run(["git", "show", "-s", "--format=%ct", commit], ROOT, capture=True))
|
||||
@@ -115,9 +132,21 @@ def main() -> int:
|
||||
run(["npm", "run", "test:provisional"], tree)
|
||||
run(["npm", "run", "test:followup"], tree)
|
||||
demo_raw = release_dir / f"timmy-talking-turd-{version}-demo.raw.webm"
|
||||
caption_receipt = release_dir / f"timmy-talking-turd-{version}-demo-captions.json"
|
||||
demo_env = dict(server_env)
|
||||
demo_env["TIMMY_RELEASE_VERSION"] = version
|
||||
demo_env["TIMMY_RELEASE_DEMO_DELTA"] = release_delta
|
||||
demo_env["TIMMY_DEMO_CAPTIONS_OUT"] = str(caption_receipt)
|
||||
run(["node", "scripts/record_release_demo.mjs", str(demo_raw)], tree, env=demo_env)
|
||||
if not caption_receipt.exists():
|
||||
raise SystemExit("Release demo produced no on-camera caption receipt")
|
||||
recorded = json.loads(caption_receipt.read_text(encoding="utf-8"))
|
||||
onscreen = [str(line) for line in (recorded.get("captions") or [])]
|
||||
if recorded.get("releaseDelta") != release_delta or not any(release_delta in line for line in onscreen):
|
||||
raise SystemExit("Release demo did not state the declared release delta on camera")
|
||||
for required in DEMO_REQUIRED_CAPTIONS:
|
||||
if not any(required in line for line in onscreen):
|
||||
raise SystemExit(f"Release demo is missing a required on-camera caption: {required}")
|
||||
finally:
|
||||
server.terminate()
|
||||
try:
|
||||
@@ -135,7 +164,13 @@ def main() -> int:
|
||||
if len(video_streams) != 1 or video_streams[0].get("codec_name") != "h264" or video_streams[0].get("pix_fmt") != "yuv420p" or video_streams[0].get("width") != 720 or video_streams[0].get("height") != 1280:
|
||||
raise SystemExit("Release demo media probe failed")
|
||||
contact_sheet = release_dir / f"timmy-talking-turd-{version}-demo-contact-sheet.jpg"
|
||||
run(["ffmpeg", "-y", "-v", "error", "-i", str(demo), "-vf", "fps=1/2,scale=180:-1,tile=4x3:padding=4:margin=4", "-frames:v", "1", str(contact_sheet)], tree)
|
||||
# Span the whole recording: a fixed 1/2 fps sheet only covered the first 24s of a longer demo.
|
||||
demo_duration = float(run(["ffprobe", "-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", str(demo)], tree, capture=True))
|
||||
if demo_duration <= 0:
|
||||
raise SystemExit("Release demo has no measurable duration")
|
||||
run(["ffmpeg", "-y", "-v", "error", "-i", str(demo), "-vf", f"fps={12 / demo_duration:.6f},scale=180:-1,tile=4x3:padding=4:margin=4", "-frames:v", "1", str(contact_sheet)], tree)
|
||||
if not contact_sheet.exists() or contact_sheet.stat().st_size == 0:
|
||||
raise SystemExit("Release demo contact sheet was not produced")
|
||||
run(["npm", "audit", "--audit-level=high"], tree)
|
||||
for file in ("app.js", "server.mjs", "service-worker.js", "src/analysis.js", "src/domain.js", "src/draft.js", "src/suggestion.js", "src/followup.js", "src/vision-readiness.js", "src/hermes-agent-service.js", "src/vision-config.js", "src/vision-service.js", "tests/staging.acceptance.mjs"):
|
||||
run(["node", "--check", file], tree)
|
||||
@@ -160,6 +195,7 @@ def main() -> int:
|
||||
f"# Timmy review release {version}\n\n"
|
||||
f"- Source commit: [{commit}]({PUBLIC_REPO}/commit/{commit})\n"
|
||||
f"- Build date (UTC): {release_date}\n"
|
||||
f"- Release delta: {release_delta}\n"
|
||||
"- Sleek three-destination shell with one dominant photo action and quiet manual fallback\n"
|
||||
"- Manual fallback: vision-offline logging stays one action away with an in-session draft and explicit start over\n"
|
||||
"- Smart free-text Timmy chat backed by authenticated server-side Hermes session continuity\n"
|
||||
@@ -197,6 +233,7 @@ def main() -> int:
|
||||
"project": "Timmy the Talking Turd",
|
||||
"version": version,
|
||||
"commit": commit,
|
||||
"release_delta": release_delta,
|
||||
"source_date_utc": source_date.isoformat(),
|
||||
"built_at_utc": dt.datetime.now(dt.timezone.utc).isoformat(),
|
||||
"artifact": archive.name,
|
||||
@@ -227,6 +264,7 @@ def main() -> int:
|
||||
"forbidden_artifacts": "passed",
|
||||
"feature_demo_full_decode": "passed",
|
||||
"feature_demo_media_probe": "passed",
|
||||
"release_delta_visible_on_camera": "passed",
|
||||
},
|
||||
}
|
||||
manifest_path = release_dir / "manifest.json"
|
||||
|
||||
@@ -1,9 +1,15 @@
|
||||
import { chromium } from 'playwright';
|
||||
import { mkdir } from 'node:fs/promises';
|
||||
import { mkdir, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
const output = path.resolve(process.argv[2] || 'release-demo.webm');
|
||||
const version = process.env.TIMMY_RELEASE_VERSION || 'review build';
|
||||
// The release delta is supplied by the builder so the recording states, on camera, what actually
|
||||
// changed in this release instead of implying a feature that did not ship.
|
||||
const releaseDelta = process.env.TIMMY_RELEASE_DEMO_DELTA
|
||||
|| 'Verified rebuild of the current user-visible feature path — no new feature is claimed.';
|
||||
const captionsOut = process.env.TIMMY_DEMO_CAPTIONS_OUT || '';
|
||||
const captionsShown = [];
|
||||
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
|
||||
await mkdir(path.dirname(output), { recursive: true });
|
||||
|
||||
@@ -71,6 +77,7 @@ await page.addStyleTag({ content: `
|
||||
` });
|
||||
|
||||
async function caption(text, duration = 1400) {
|
||||
captionsShown.push(text);
|
||||
await page.evaluate(text => {
|
||||
document.querySelector('#demo-caption')?.remove();
|
||||
const node = document.createElement('div');
|
||||
@@ -80,6 +87,16 @@ async function caption(text, duration = 1400) {
|
||||
node.animate([{ opacity: 0, transform: 'translateY(-8px)' }, { opacity: 1, transform: 'translateY(0)' }], { duration: 260, fill: 'forwards' });
|
||||
}, text);
|
||||
await sleep(duration);
|
||||
// A caption must not outlive its stated duration. It used to stay on screen until the next one
|
||||
// replaced it, so a view change left the previous screen's claim hanging over a different screen —
|
||||
// e.g. a delete-data caption sitting on top of the home view.
|
||||
await page.evaluate(() => {
|
||||
const node = document.querySelector('#demo-caption');
|
||||
if (!node) return;
|
||||
node.animate([{ opacity: 1 }, { opacity: 0 }], { duration: 180, fill: 'forwards' });
|
||||
setTimeout(() => node.remove(), 220);
|
||||
});
|
||||
await sleep(220);
|
||||
}
|
||||
|
||||
async function tap(selector, after = 650) {
|
||||
@@ -120,7 +137,8 @@ async function indicate(selector) {
|
||||
await sleep(650);
|
||||
}
|
||||
|
||||
await caption(`TIMMY ${version} • FEATURE DEMO`, 1200);
|
||||
await caption(`TIMMY ${version} • RELEASE DEMO`, 1200);
|
||||
await caption(`This release: ${releaseDelta}`, 2600);
|
||||
await caption('Automated checks replay this synthetic path before review', 1200);
|
||||
await caption('One clear photo action. Manual logging stays one tap away.', 1500);
|
||||
await tap('[data-scan]', 450);
|
||||
@@ -186,6 +204,17 @@ await caption('One tap still reaches manual Bristol entry, and the draft keeps y
|
||||
await tap('#discard-draft', 500);
|
||||
await caption('Start over clears the kept draft — nothing lingers silently', 1500);
|
||||
await tap('#close-sheet', 450);
|
||||
await caption('The threat model documents this surface, so the recording shows it live rather than describing it', 2200);
|
||||
await tap('[data-view="calendar"]', 550);
|
||||
await page.getByText(/Your private journal/i).waitFor();
|
||||
await tap('[data-view="privacy"]', 550);
|
||||
await page.getByText(/Your poop\. Your phone\./i).waitFor();
|
||||
await caption('Every byte-bearing control the model names, in the running app: local ledger, consent-gated upload, portable export, full deletion', 2800);
|
||||
await indicate('#export');
|
||||
await caption('Export writes a plain JSON file you hold — the model records it as a data-lifetime boundary, not a server sync', 2600);
|
||||
await indicate('#delete-all');
|
||||
await caption('Delete all local data clears the ledger in this browser; there is no server copy of it to keep', 2600);
|
||||
await tap('[data-view="home"]', 500);
|
||||
await tap('[data-view="timmy"]', 550);
|
||||
await page.getByText(/Hermes Agent connected/i).waitFor();
|
||||
await caption('Hermes Agent connected — free-text, contextual, and server-side', 1400);
|
||||
@@ -214,6 +243,11 @@ await page.evaluate(() => {
|
||||
await sleep(2200);
|
||||
|
||||
if (errors.length) throw new Error(`Browser errors: ${errors.join(' | ')}`);
|
||||
// Machine-readable receipt of exactly what was rendered on camera, so the release gate can prove
|
||||
// the recorded delta statement reached the screen instead of trusting the script source.
|
||||
if (captionsOut) {
|
||||
await writeFile(captionsOut, JSON.stringify({ version, releaseDelta, captions: captionsShown }, null, 2) + '\n', 'utf8');
|
||||
}
|
||||
const recording = page.video();
|
||||
await page.close();
|
||||
await recording.saveAs(output);
|
||||
|
||||
@@ -97,3 +97,54 @@ test('release builder gates the hard-stool follow-up the same way CI does', asyn
|
||||
assert.match(builder, /"src\/followup\.js"/);
|
||||
assert.match(builder, /Hard-stool follow-up: a Bristol 1-2 entry asks user-reported questions/);
|
||||
});
|
||||
|
||||
test('release demo states the declared release delta on camera and proves it with a receipt', async () => {
|
||||
const demo = await readFile(demoPath, 'utf8');
|
||||
|
||||
// The delta comes from the builder, so a documentation-only release cannot imply a new feature.
|
||||
assert.match(demo, /TIMMY_RELEASE_DEMO_DELTA/);
|
||||
assert.match(demo, /This release: \$\{releaseDelta\}/);
|
||||
assert.match(demo, /no new feature is claimed/);
|
||||
// A caption alone is not evidence; the demo must emit a machine-readable receipt of what rendered.
|
||||
assert.match(demo, /TIMMY_DEMO_CAPTIONS_OUT/);
|
||||
assert.match(demo, /captionsShown\.push\(text\)/);
|
||||
assert.match(demo, /writeFile\(captionsOut/);
|
||||
});
|
||||
|
||||
test('release demo shows the byte-bearing controls the threat model governs, without inventing UI', async () => {
|
||||
const demo = await readFile(demoPath, 'utf8');
|
||||
|
||||
assert.match(demo, /\[data-view="privacy"\]/);
|
||||
assert.match(demo, /Every byte-bearing control the model names/);
|
||||
assert.match(demo, /indicate\('#export'\)/);
|
||||
assert.match(demo, /indicate\('#delete-all'\)/);
|
||||
assert.match(demo, /Delete all local data clears the ledger in this browser; there is no server copy of it to keep/);
|
||||
// Real controls only: the selectors must exist in the app shell, not be demo-only scaffolding.
|
||||
const app = await readFile(new URL('../app.js', import.meta.url), 'utf8');
|
||||
assert.match(app, /id="export"/);
|
||||
assert.match(app, /id="delete-all"/);
|
||||
});
|
||||
|
||||
test('release builder refuses to publish without the on-camera delta receipt', async () => {
|
||||
const builder = await readFile(builderPath, 'utf8');
|
||||
|
||||
assert.match(builder, /TIMMY_RELEASE_DELTA/);
|
||||
assert.match(builder, /TIMMY_DEMO_CAPTIONS_OUT/);
|
||||
assert.match(builder, /Release demo produced no on-camera caption receipt/);
|
||||
assert.match(builder, /Release demo did not state the declared release delta on camera/);
|
||||
assert.match(builder, /Release demo is missing a required on-camera caption/);
|
||||
assert.match(builder, /"release_delta_visible_on_camera": "passed"/);
|
||||
assert.match(builder, /"release_delta": release_delta/);
|
||||
});
|
||||
|
||||
test('release demo captions expire and the contact sheet spans the whole recording', async () => {
|
||||
const demo = await readFile(demoPath, 'utf8');
|
||||
const builder = await readFile(builderPath, 'utf8');
|
||||
|
||||
// A caption that outlives its stated duration hangs the previous view's claim over the next screen.
|
||||
assert.match(demo, /creates via removal after duration|A caption must not outlive its stated duration/);
|
||||
assert.match(demo, /setTimeout\(\(\) => node\.remove\(\), 220\)/);
|
||||
// The sheet must cover the entire demo, not just its opening.
|
||||
assert.match(builder, /12 \/ demo_duration/);
|
||||
assert.match(builder, /Release demo contact sheet was not produced/);
|
||||
});
|
||||
@@ -0,0 +1,119 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
|
||||
const threatModelPath = new URL('../docs/PHOTO-DATAFLOW-THREAT-MODEL.md', import.meta.url);
|
||||
const productDecisionPath = new URL('../docs/PRODUCT-DECISIONS.md', import.meta.url);
|
||||
const roadmapPath = new URL('../ROADMAP.md', import.meta.url);
|
||||
|
||||
// Every hop that can hold, move, or leak a user's photo bytes has to be named
|
||||
// in the diagram, not only in prose.
|
||||
const REQUIRED_DIAGRAM_NODES = [
|
||||
['browser', /\bbrowser\b/i],
|
||||
['API ingress', /\bapi[- ]ingress\b/i],
|
||||
['model worker', /\bmodel worker\b/i],
|
||||
['transient buffers', /\btransient buffer/i],
|
||||
['training opt-in store', /\btraining opt-in\b/i],
|
||||
['logs', /\blogs?\b/i],
|
||||
['exports', /\bexports?\b/i],
|
||||
['deletion', /\bdeletion\b/i],
|
||||
];
|
||||
|
||||
// The abuse cases the ticket names explicitly, in the order they appear there.
|
||||
const REQUIRED_THREATS = [
|
||||
['replay', /\breplay\b/i],
|
||||
['oversized input', /\boversized input\b/i],
|
||||
['metadata leakage', /\bmetadata leakage\b/i],
|
||||
['poisoning', /\bpoisoning\b/i],
|
||||
['unauthorized training', /\bunauthorized training\b/i],
|
||||
['operator access', /\boperator access\b/i],
|
||||
];
|
||||
|
||||
function diagramBody(doc) {
|
||||
const match = doc.match(/```mermaid\r?\n([\s\S]*?)```/);
|
||||
assert.ok(match, 'the threat model must contain a fenced mermaid data-flow diagram');
|
||||
return match[1];
|
||||
}
|
||||
|
||||
function threatBlocks(doc) {
|
||||
const blocks = new Map();
|
||||
const pattern = /^###\s+(T\d+)\s+—\s+(.+)$/gm;
|
||||
const headings = [...doc.matchAll(pattern)];
|
||||
headings.forEach((heading, index) => {
|
||||
const start = heading.index + heading[0].length;
|
||||
const end = index + 1 < headings.length ? headings[index + 1].index : doc.length;
|
||||
blocks.set(heading[1], { name: heading[2].trim(), body: doc.slice(start, end) });
|
||||
});
|
||||
return blocks;
|
||||
}
|
||||
|
||||
function knownIssueNumbers(roadmap) {
|
||||
return new Set([...roadmap.matchAll(/issues\/(\d+)/g)].map((match) => Number(match[1])));
|
||||
}
|
||||
|
||||
test('photo data-flow threat model diagrams every component that can hold or leak image bytes', async () => {
|
||||
const doc = await readFile(threatModelPath, 'utf8');
|
||||
const diagram = diagramBody(doc);
|
||||
|
||||
for (const [label, pattern] of REQUIRED_DIAGRAM_NODES) {
|
||||
assert.match(diagram, pattern, `the data-flow diagram must show the ${label} component`);
|
||||
}
|
||||
|
||||
// The diagram has to end somewhere: the model worker's bytes must not flow on.
|
||||
assert.match(diagram, /no retention|dropped|discarded|never persist/i,
|
||||
'the diagram must show that worker input is not retained');
|
||||
});
|
||||
|
||||
test('every required abuse case has a mitigation, an owner, and a real follow-up ticket', async () => {
|
||||
const [doc, roadmap] = await Promise.all([
|
||||
readFile(threatModelPath, 'utf8'),
|
||||
readFile(roadmapPath, 'utf8'),
|
||||
]);
|
||||
const known = knownIssueNumbers(roadmap);
|
||||
const blocks = threatBlocks(doc);
|
||||
|
||||
assert.ok(blocks.size > 0, 'threats must be numbered headings so their coverage is checkable');
|
||||
const named = [...blocks.values()].map((block) => block.name).join('\n');
|
||||
|
||||
for (const [label, pattern] of REQUIRED_THREATS) {
|
||||
assert.match(named, pattern, `the ${label} abuse case must be its own numbered threat`);
|
||||
|
||||
const block = [...blocks.values()].find((entry) => pattern.test(entry.name));
|
||||
assert.match(block.body, /^\s*-\s*\*\*Owner:\*\*\s*\S+/m,
|
||||
`${label} needs a named owner`);
|
||||
assert.match(block.body, /^\s*-\s*\*\*Testable follow-up:\*\*\s*#\d+/m,
|
||||
`${label} needs a testable follow-up ticket`);
|
||||
assert.match(block.body, /^\s*-\s*\*\*Mitigation:\*\*\s*\S+/m,
|
||||
`${label} needs a concrete mitigation, not a restatement of the risk`);
|
||||
assert.match(block.body, /^\s*-\s*\*\*Evidence:\*\*\s*\S+/m,
|
||||
`${label} needs the artifact that will prove the mitigation works`);
|
||||
|
||||
const followUp = Number(block.body.match(/^\s*-\s*\*\*Testable follow-up:\*\*\s*#(\d+)/m)[1]);
|
||||
assert.ok(known.has(followUp),
|
||||
`${label} points at #${followUp}, which is not a tracked ticket in ROADMAP.md`);
|
||||
assert.notEqual(followUp, 20, `${label} must delegate to work other than this threat model itself`);
|
||||
}
|
||||
});
|
||||
|
||||
test('threat model keeps the photo path non-diagnostic and free of real medical images', async () => {
|
||||
const doc = await readFile(threatModelPath, 'utf8');
|
||||
|
||||
assert.match(doc, /non-diagnostic/i);
|
||||
assert.match(doc, /never diagnos/i);
|
||||
assert.match(doc, /provisional/i);
|
||||
assert.match(doc, /synthetic/i, 'fixtures must be synthetic, not real medical images');
|
||||
assert.match(doc, /no real (?:medical |stool )?images? (?:are |may )?(?:ever )?(?:added|used|shipped|included)/i,
|
||||
'the model must state plainly that no real medical images enter the repo');
|
||||
assert.match(doc, /deterministic (?:red-flag )?(?:escalation|urgent-symptom)/i,
|
||||
'the model must note that escalation is deterministic and never model-gated');
|
||||
assert.doesNotMatch(doc, /\b(?:bathroom photo|patient photo|clinical image|real stool image)\b/i,
|
||||
'the model must not promise or reference real medical imagery');
|
||||
assert.doesNotMatch(doc, /\b(?:clinically validated|clinically proven|medical device|FDA-cleared|CE-marked)\b/i,
|
||||
'the model must not overstate the clinical status of the vision path');
|
||||
});
|
||||
|
||||
test('the accepted product decision record links the photo data-flow threat model', async () => {
|
||||
const decision = await readFile(productDecisionPath, 'utf8');
|
||||
assert.match(decision, /\.\/PHOTO-DATAFLOW-THREAT-MODEL\.md/,
|
||||
'the governing decision record must point reviewers at the threat model');
|
||||
});
|
||||
Reference in new issue
Block a user