test(release): record the ingress refusal on camera and gate its caption (#16)
Quality gates / quality (pull_request) Successful in 2m57s

The demo now picks a committed text fixture that only claims to be a photo, asserts on camera that
nothing was posted to /api/analyze, and keeps manual entry in the same scene. The builder requires
that caption to have painted and records image_ingress_refusal in the manifest; the threat model
names each implemented control, its module, and the tests that enforce it.
This commit is contained in:
timmy committed 2026-10-20 08:21:24 +00:00
1 parent 9b72872466
commit 2b773ba58c
4 files changed
+62 -1

No files matched your search

+21
View File
@@ -144,6 +144,26 @@ Each entry names the abuse case, the mitigation that is already committed or que
- **Testable follow-up:** #39
- **Evidence:** The production topology, secrets, backup, and rollback runbook, exercised against the deletion path so that purge reaches memory, temporary files, logs, and backups, with observability from #41 confirming no bytes reach the metrics surface.
## Ingress controls implemented for #16
`/api/analyze` is the only route that accepts image bytes, and every limit below is decided inside
`server.mjs` before `analyzePhoto` runs, so a refusal never costs a provider call.
| Control | Where | Behaviour |
|---|---|---|
| Per-caller request budget | `src/rate-limit.js` | Sliding window, 12 requests per 60s by default (`TIMMY_ANALYZE_RATE_LIMIT`, `TIMMY_ANALYZE_RATE_WINDOW_MS`); refusal is `429` with `Retry-After`. The key is a salted SHA-256 of the socket address held in process memory only — no address, no identity, no bytes are stored. |
| Body ceiling and deadline | `src/http-body.js` | 6 MB ceiling and a 15s deadline (`TIMMY_BODY_TIMEOUT_MS`); `413` and `408` respectively, and the half-read socket is dropped after the refusal is written. |
| Container sniffing | `src/image-ingress.js` | The declared data-URL MIME is treated as a claim, never evidence: JPEG/PNG/WebP is decided from magic bytes, and a mismatch is refused. |
| Polyglot / trailing payloads | `src/image-ingress.js` | The container is walked to its own end marker (PNG `IEND`, JPEG `EOI`, RIFF length); anything appended after the image is refused instead of forwarded. |
| Embedded metadata | `src/image-ingress.js` | JPEG APP1/APP13/COM segments and PNG `eXIf`/`tEXt`/`zTXt`/`iTXt`/`tIME` chunks are refused. The app's own upload is a canvas re-encode (JPEG q0.7, longest edge 1200px) and carries none, so the accepted path is already metadata-free. |
| Pixel limits | `src/image-ingress.js` | Dimensions are read from the header (PNG `IHDR`, JPEG SOF, WebP `VP8X`/`VP8`/`VP8L`) and refused above a 4096px edge or 16 megapixels, so a decode bomb never reaches the worker. |
| Log hygiene | `server.mjs` | Refusal copy is fixed prose; no request body, image bytes, hash, or address is written to a log or returned in an error. |
Residual, and not claimed as solved here: the server refuses metadata-bearing or mismatched
containers rather than transcoding them itself, so a client that cannot produce a clean re-encode
must use the app's capture path. PNG chunk CRCs are not validated (structure is), and the request
budget is per process, not per cluster.
## Accepted residual risks
- **Host compromise.** Data in a running process can be read by a sufficiently privileged attacker. This is accepted for the private beta and bounded by least privilege plus the purge procedure in #39; it is not claimed as eliminated.
@@ -154,4 +174,5 @@ Each entry names the abuse case, the mitigation that is already committed or que
- This document is enforced by `tests/threat-model.test.js`, which asserts that the diagram covers every byte-bearing component, that each required abuse case is a numbered threat with an owner and a tracked follow-up ticket, and that the model keeps the non-diagnostic boundary and ships no real medical images.
- The test runs in `npm test` and therefore inside the CI quality gate and the release build, so the threat model cannot silently drift from the code it describes.
- The ingress controls above are enforced by `tests/image-ingress.test.js`, `tests/rate-limit.test.js`, `tests/http-body.test.js`, and `tests/analyze-ingress.acceptance.test.js`, which runs the shipping server against synthetic bytes and a counting provider stub; `tests/ingress-capture.acceptance.mjs` proves the same refusal in the running app. All of them run in `npm test` or the browser gate, so a regression fails the release instead of a promise.
- Boundary and boundary-adjacent claims are reviewed by the clinical and privacy gate (#22, #23) before beta; this model does not move that gate, it feeds it.
+4 -1
View File
@@ -48,6 +48,7 @@ DEMO_REQUIRED_CAPTIONS = (
"Reachable is not loaded: no AI control, no consent, and no upload until the pinned model is resident",
"Every pattern line names the confirmed logs behind it, and following one shows exactly those records",
"Two logs is not a pattern: the summary refuses to describe a trend it cannot support",
"A file that only claims to be a photo is refused on the device — nothing is uploaded",
"The answer states the count and cites the four confirmed logs it counted",
"Following a citation opens exactly the confirmed log behind that answer",
"Ask something the journal cannot answer and Timmy refuses instead of reassuring",
@@ -143,6 +144,7 @@ def main() -> int:
run(["npm", "run", "test:ui"], tree)
run(["npm", "run", "test:photo"], tree)
run(["npm", "run", "test:mobile-capture"], tree)
run(["npm", "run", "test:ingress-capture"], tree)
run(["npm", "run", "test:sleek"], tree)
run(["npm", "run", "test:manual-fallback"], tree)
run(["npm", "run", "test:provisional"], tree)
@@ -199,7 +201,7 @@ def main() -> int:
if not contact_sheet.exists() or contact_sheet.stat().st_size == 0:
raise SystemExit("Release demo contact sheet was not produced")
run(["npm", "audit", "--audit-level=high"], tree)
for file in ("app.js", "server.mjs", "service-worker.js", "src/analysis.js", "src/domain.js", "src/draft.js", "src/suggestion.js", "src/followup.js", "src/vision-readiness.js", "src/hermes-agent-service.js", "src/vision-config.js", "src/vision-service.js", "src/vision-corpus.js", "src/summary.js", "src/photo-lifecycle.js", "scripts/record_worker_benchmark.mjs", "tests/staging.acceptance.mjs", "tests/vision-refusal.acceptance.mjs", "tests/worker-readiness.acceptance.mjs", "tests/pattern-summary.acceptance.mjs", "tests/photo-lifecycle.acceptance.mjs"):
for file in ("app.js", "server.mjs", "service-worker.js", "src/analysis.js", "src/domain.js", "src/draft.js", "src/suggestion.js", "src/followup.js", "src/vision-readiness.js", "src/hermes-agent-service.js", "src/vision-config.js", "src/vision-service.js", "src/vision-corpus.js", "src/summary.js", "src/photo-lifecycle.js", "src/image-ingress.js", "src/rate-limit.js", "src/http-body.js", "scripts/record_worker_benchmark.mjs", "tests/staging.acceptance.mjs", "tests/vision-refusal.acceptance.mjs", "tests/worker-readiness.acceptance.mjs", "tests/pattern-summary.acceptance.mjs", "tests/photo-lifecycle.acceptance.mjs", "tests/ingress-capture.acceptance.mjs"):
run(["node", "--check", file], tree)
run(["bash", "-n", "scripts/bootstrap_selfhost_smolvlm.sh"], tree)
run(["bash", "-n", "scripts/run_selfhost_smolvlm.sh"], tree)
@@ -300,6 +302,7 @@ def main() -> int:
"worker_post_load_readiness": "passed",
"longitudinal_pattern_summary": "passed",
"grounded_chat_citations": "passed",
"image_ingress_refusal": "passed",
"dependency_audit_high": "passed",
"syntax": "passed",
"secret_scan": "passed",
+20
View File
@@ -80,7 +80,9 @@ const suggestionBody = {
warning: 'Visual suggestion only. Confirm it yourself; this is not a diagnosis.',
};
let analyzeOverride = null;
let analyzeCalls = 0;
await page.route('**/api/analyze', route => {
analyzeCalls += 1;
const response = analyzeOverride || { status: 200, body: suggestionBody };
return route.fulfill({ status: response.status, contentType: 'application/json', body: JSON.stringify(response.body) });
});
@@ -226,6 +228,24 @@ await indicate('label[for="camera-photo"]');
await page.locator('#camera-photo').dispatchEvent('cancel');
await page.getByText(/Camera or photo picker closed/i).waitFor();
await caption('Camera closed cleanly — gallery and manual logging are still available', 1600);
// Issue #16, on camera: the file below is committed as text and only claims to be a photo. It must
// be refused on the device, the refusal must say nothing was uploaded, and the app must never post
// it — asserted here, not narrated.
await indicate('label[for="gallery-photo"]');
await page.locator('#gallery-photo').setInputFiles('tests/fixtures/not-a-photo.jpg');
await page.locator('[data-capture-error="unreadable"]').waitFor();
const unreadableCopy = await page.locator('[data-capture-error="unreadable"]').innerText();
if (!/isn’t a photo Timmy can read/.test(unreadableCopy) || !/Nothing was uploaded/.test(unreadableCopy)) {
throw new Error(`An unreadable file must be refused with the upload-free copy, saw: ${unreadableCopy}`);
}
if (await page.getByText(/couldn’t analyze that safely/i).count() !== 0) {
throw new Error('A file that never left the device must not be reported as a failed analysis');
}
if (analyzeCalls !== 0) throw new Error('An unreadable file was posted to /api/analyze');
await caption('A file that only claims to be a photo is refused on the device — nothing is uploaded', 2600);
await indicate('[data-capture-error="unreadable"]');
await caption('Manual logging stays one tap away, and the photo picker is one tap back', 2000);
await tap('#retake-photo', 450);
await indicate('label[for="gallery-photo"]');
await page.locator('#gallery-photo').setInputFiles('tests/fixtures/synthetic-type4.jpg');
await caption('Nothing uploads until explicit consent', 1100);
+17
View File
@@ -113,6 +113,23 @@ test('release demo visibly exercises the red-flag language coverage instead of o
assert.match(builder, /A tarry note already saved in the journal escalates the same way, before any provider call/);
});
test('release demo shows a file that only claims to be a photo being refused before upload', async () => {
const demo = await readFile(demoPath, 'utf8');
const builder = await readFile(builderPath, 'utf8');
// Issue #16: the recording must show the device-side refusal of a non-image, prove nothing was
// posted, and keep manual entry reachable — all asserted in the browser, not narrated.
assert.match(demo, /tests\/fixtures\/not-a-photo\.jpg/);
assert.match(demo, /\[data-capture-error="unreadable"\]/);
assert.match(demo, /Nothing was uploaded/);
assert.match(demo, /An unreadable file was posted to \/api\/analyze/);
assert.match(demo, /A file that only claims to be a photo is refused on the device — nothing is uploaded/);
assert.match(demo, /Manual logging stays one tap away, and the photo picker is one tap back/);
// The caption is a required release gate, so a later release cannot drop the scene silently.
assert.match(builder, /A file that only claims to be a photo is refused on the device — nothing is uploaded/);
assert.match(builder, /"image_ingress_refusal": "passed"/);
});
test('release builder gates the sleek shell, Hermes chat, and bootstrap syntax', async () => {
const builder = await readFile(builderPath, 'utf8');
assert.match(builder, /"test:sleek"/);