test(release): record the ingress refusal on camera and gate its caption (#16)
Quality gates / quality (pull_request) Successful in 2m57s
Quality gates / quality (pull_request) Successful in 2m57s
The demo now picks a committed text fixture that only claims to be a photo, asserts on camera that nothing was posted to /api/analyze, and keeps manual entry in the same scene. The builder requires that caption to have painted and records image_ingress_refusal in the manifest; the threat model names each implemented control, its module, and the tests that enforce it.
This commit is contained in:
1 parent
9b72872466
commit
2b773ba58c
4 files changed
+62
-1
No files matched your search
@@ -144,6 +144,26 @@ Each entry names the abuse case, the mitigation that is already committed or que
|
||||
- **Testable follow-up:** #39
|
||||
- **Evidence:** The production topology, secrets, backup, and rollback runbook, exercised against the deletion path so that purge reaches memory, temporary files, logs, and backups, with observability from #41 confirming no bytes reach the metrics surface.
|
||||
|
||||
## Ingress controls implemented for #16
|
||||
|
||||
`/api/analyze` is the only route that accepts image bytes, and every limit below is decided inside
|
||||
`server.mjs` before `analyzePhoto` runs, so a refusal never costs a provider call.
|
||||
|
||||
| Control | Where | Behaviour |
|
||||
|---|---|---|
|
||||
| Per-caller request budget | `src/rate-limit.js` | Sliding window, 12 requests per 60s by default (`TIMMY_ANALYZE_RATE_LIMIT`, `TIMMY_ANALYZE_RATE_WINDOW_MS`); refusal is `429` with `Retry-After`. The key is a salted SHA-256 of the socket address held in process memory only — no address, no identity, no bytes are stored. |
|
||||
| Body ceiling and deadline | `src/http-body.js` | 6 MB ceiling and a 15s deadline (`TIMMY_BODY_TIMEOUT_MS`); `413` and `408` respectively, and the half-read socket is dropped after the refusal is written. |
|
||||
| Container sniffing | `src/image-ingress.js` | The declared data-URL MIME is treated as a claim, never evidence: JPEG/PNG/WebP is decided from magic bytes, and a mismatch is refused. |
|
||||
| Polyglot / trailing payloads | `src/image-ingress.js` | The container is walked to its own end marker (PNG `IEND`, JPEG `EOI`, RIFF length); anything appended after the image is refused instead of forwarded. |
|
||||
| Embedded metadata | `src/image-ingress.js` | JPEG APP1/APP13/COM segments and PNG `eXIf`/`tEXt`/`zTXt`/`iTXt`/`tIME` chunks are refused. The app's own upload is a canvas re-encode (JPEG q0.7, longest edge 1200px) and carries none, so the accepted path is already metadata-free. |
|
||||
| Pixel limits | `src/image-ingress.js` | Dimensions are read from the header (PNG `IHDR`, JPEG SOF, WebP `VP8X`/`VP8`/`VP8L`) and refused above a 4096px edge or 16 megapixels, so a decode bomb never reaches the worker. |
|
||||
| Log hygiene | `server.mjs` | Refusal copy is fixed prose; no request body, image bytes, hash, or address is written to a log or returned in an error. |
|
||||
|
||||
Residual, and not claimed as solved here: the server refuses metadata-bearing or mismatched
|
||||
containers rather than transcoding them itself, so a client that cannot produce a clean re-encode
|
||||
must use the app's capture path. PNG chunk CRCs are not validated (structure is), and the request
|
||||
budget is per process, not per cluster.
|
||||
|
||||
## Accepted residual risks
|
||||
|
||||
- **Host compromise.** Data in a running process can be read by a sufficiently privileged attacker. This is accepted for the private beta and bounded by least privilege plus the purge procedure in #39; it is not claimed as eliminated.
|
||||
@@ -154,4 +174,5 @@ Each entry names the abuse case, the mitigation that is already committed or que
|
||||
|
||||
- This document is enforced by `tests/threat-model.test.js`, which asserts that the diagram covers every byte-bearing component, that each required abuse case is a numbered threat with an owner and a tracked follow-up ticket, and that the model keeps the non-diagnostic boundary and ships no real medical images.
|
||||
- The test runs in `npm test` and therefore inside the CI quality gate and the release build, so the threat model cannot silently drift from the code it describes.
|
||||
- The ingress controls above are enforced by `tests/image-ingress.test.js`, `tests/rate-limit.test.js`, `tests/http-body.test.js`, and `tests/analyze-ingress.acceptance.test.js`, which runs the shipping server against synthetic bytes and a counting provider stub; `tests/ingress-capture.acceptance.mjs` proves the same refusal in the running app. All of them run in `npm test` or the browser gate, so a regression fails the release instead of a promise.
|
||||
- Boundary and boundary-adjacent claims are reviewed by the clinical and privacy gate (#22, #23) before beta; this model does not move that gate, it feeds it.
|
||||
@@ -48,6 +48,7 @@ DEMO_REQUIRED_CAPTIONS = (
|
||||
"Reachable is not loaded: no AI control, no consent, and no upload until the pinned model is resident",
|
||||
"Every pattern line names the confirmed logs behind it, and following one shows exactly those records",
|
||||
"Two logs is not a pattern: the summary refuses to describe a trend it cannot support",
|
||||
"A file that only claims to be a photo is refused on the device — nothing is uploaded",
|
||||
"The answer states the count and cites the four confirmed logs it counted",
|
||||
"Following a citation opens exactly the confirmed log behind that answer",
|
||||
"Ask something the journal cannot answer and Timmy refuses instead of reassuring",
|
||||
@@ -143,6 +144,7 @@ def main() -> int:
|
||||
run(["npm", "run", "test:ui"], tree)
|
||||
run(["npm", "run", "test:photo"], tree)
|
||||
run(["npm", "run", "test:mobile-capture"], tree)
|
||||
run(["npm", "run", "test:ingress-capture"], tree)
|
||||
run(["npm", "run", "test:sleek"], tree)
|
||||
run(["npm", "run", "test:manual-fallback"], tree)
|
||||
run(["npm", "run", "test:provisional"], tree)
|
||||
@@ -199,7 +201,7 @@ def main() -> int:
|
||||
if not contact_sheet.exists() or contact_sheet.stat().st_size == 0:
|
||||
raise SystemExit("Release demo contact sheet was not produced")
|
||||
run(["npm", "audit", "--audit-level=high"], tree)
|
||||
for file in ("app.js", "server.mjs", "service-worker.js", "src/analysis.js", "src/domain.js", "src/draft.js", "src/suggestion.js", "src/followup.js", "src/vision-readiness.js", "src/hermes-agent-service.js", "src/vision-config.js", "src/vision-service.js", "src/vision-corpus.js", "src/summary.js", "src/photo-lifecycle.js", "scripts/record_worker_benchmark.mjs", "tests/staging.acceptance.mjs", "tests/vision-refusal.acceptance.mjs", "tests/worker-readiness.acceptance.mjs", "tests/pattern-summary.acceptance.mjs", "tests/photo-lifecycle.acceptance.mjs"):
|
||||
for file in ("app.js", "server.mjs", "service-worker.js", "src/analysis.js", "src/domain.js", "src/draft.js", "src/suggestion.js", "src/followup.js", "src/vision-readiness.js", "src/hermes-agent-service.js", "src/vision-config.js", "src/vision-service.js", "src/vision-corpus.js", "src/summary.js", "src/photo-lifecycle.js", "src/image-ingress.js", "src/rate-limit.js", "src/http-body.js", "scripts/record_worker_benchmark.mjs", "tests/staging.acceptance.mjs", "tests/vision-refusal.acceptance.mjs", "tests/worker-readiness.acceptance.mjs", "tests/pattern-summary.acceptance.mjs", "tests/photo-lifecycle.acceptance.mjs", "tests/ingress-capture.acceptance.mjs"):
|
||||
run(["node", "--check", file], tree)
|
||||
run(["bash", "-n", "scripts/bootstrap_selfhost_smolvlm.sh"], tree)
|
||||
run(["bash", "-n", "scripts/run_selfhost_smolvlm.sh"], tree)
|
||||
@@ -300,6 +302,7 @@ def main() -> int:
|
||||
"worker_post_load_readiness": "passed",
|
||||
"longitudinal_pattern_summary": "passed",
|
||||
"grounded_chat_citations": "passed",
|
||||
"image_ingress_refusal": "passed",
|
||||
"dependency_audit_high": "passed",
|
||||
"syntax": "passed",
|
||||
"secret_scan": "passed",
|
||||
|
||||
@@ -80,7 +80,9 @@ const suggestionBody = {
|
||||
warning: 'Visual suggestion only. Confirm it yourself; this is not a diagnosis.',
|
||||
};
|
||||
let analyzeOverride = null;
|
||||
let analyzeCalls = 0;
|
||||
await page.route('**/api/analyze', route => {
|
||||
analyzeCalls += 1;
|
||||
const response = analyzeOverride || { status: 200, body: suggestionBody };
|
||||
return route.fulfill({ status: response.status, contentType: 'application/json', body: JSON.stringify(response.body) });
|
||||
});
|
||||
@@ -226,6 +228,24 @@ await indicate('label[for="camera-photo"]');
|
||||
await page.locator('#camera-photo').dispatchEvent('cancel');
|
||||
await page.getByText(/Camera or photo picker closed/i).waitFor();
|
||||
await caption('Camera closed cleanly — gallery and manual logging are still available', 1600);
|
||||
// Issue #16, on camera: the file below is committed as text and only claims to be a photo. It must
|
||||
// be refused on the device, the refusal must say nothing was uploaded, and the app must never post
|
||||
// it — asserted here, not narrated.
|
||||
await indicate('label[for="gallery-photo"]');
|
||||
await page.locator('#gallery-photo').setInputFiles('tests/fixtures/not-a-photo.jpg');
|
||||
await page.locator('[data-capture-error="unreadable"]').waitFor();
|
||||
const unreadableCopy = await page.locator('[data-capture-error="unreadable"]').innerText();
|
||||
if (!/isn’t a photo Timmy can read/.test(unreadableCopy) || !/Nothing was uploaded/.test(unreadableCopy)) {
|
||||
throw new Error(`An unreadable file must be refused with the upload-free copy, saw: ${unreadableCopy}`);
|
||||
}
|
||||
if (await page.getByText(/couldn’t analyze that safely/i).count() !== 0) {
|
||||
throw new Error('A file that never left the device must not be reported as a failed analysis');
|
||||
}
|
||||
if (analyzeCalls !== 0) throw new Error('An unreadable file was posted to /api/analyze');
|
||||
await caption('A file that only claims to be a photo is refused on the device — nothing is uploaded', 2600);
|
||||
await indicate('[data-capture-error="unreadable"]');
|
||||
await caption('Manual logging stays one tap away, and the photo picker is one tap back', 2000);
|
||||
await tap('#retake-photo', 450);
|
||||
await indicate('label[for="gallery-photo"]');
|
||||
await page.locator('#gallery-photo').setInputFiles('tests/fixtures/synthetic-type4.jpg');
|
||||
await caption('Nothing uploads until explicit consent', 1100);
|
||||
|
||||
@@ -113,6 +113,23 @@ test('release demo visibly exercises the red-flag language coverage instead of o
|
||||
assert.match(builder, /A tarry note already saved in the journal escalates the same way, before any provider call/);
|
||||
});
|
||||
|
||||
test('release demo shows a file that only claims to be a photo being refused before upload', async () => {
|
||||
const demo = await readFile(demoPath, 'utf8');
|
||||
const builder = await readFile(builderPath, 'utf8');
|
||||
|
||||
// Issue #16: the recording must show the device-side refusal of a non-image, prove nothing was
|
||||
// posted, and keep manual entry reachable — all asserted in the browser, not narrated.
|
||||
assert.match(demo, /tests\/fixtures\/not-a-photo\.jpg/);
|
||||
assert.match(demo, /\[data-capture-error="unreadable"\]/);
|
||||
assert.match(demo, /Nothing was uploaded/);
|
||||
assert.match(demo, /An unreadable file was posted to \/api\/analyze/);
|
||||
assert.match(demo, /A file that only claims to be a photo is refused on the device — nothing is uploaded/);
|
||||
assert.match(demo, /Manual logging stays one tap away, and the photo picker is one tap back/);
|
||||
// The caption is a required release gate, so a later release cannot drop the scene silently.
|
||||
assert.match(builder, /A file that only claims to be a photo is refused on the device — nothing is uploaded/);
|
||||
assert.match(builder, /"image_ingress_refusal": "passed"/);
|
||||
});
|
||||
|
||||
test('release builder gates the sleek shell, Hermes chat, and bootstrap syntax', async () => {
|
||||
const builder = await readFile(builderPath, 'utf8');
|
||||
assert.match(builder, /"test:sleek"/);
|
||||
|
||||
Reference in new issue
Block a user