Brings #82 up to today's `main` (1bbcc10) and resolves the two conflicts as ordered
unions: `package.json` `test`/`check:syntax` and the release builder's syntax allow-list
and evidence keys. Both sides only appended, so nothing was dropped.
Verifying the merge against main's RC attack suite (#40) found two real defects, each
fixed test-first:
- An oversized body destroyed the socket mid-upload, so the caller saw `fetch failed`
instead of the 413 it had already earned, and `security_attack_suite.mjs` could not
complete at all. New test in `tests/analyze-ingress.acceptance.test.js` failed with
`fetch failed` before the fix; `src/http-body.js` now keeps draining without buffering
and refuses on `end`, so the ceiling still holds and the refusal is readable.
- #16's per-caller budget turned the suite's own load phase into 8/8 429s, which would
have published refusals as analysis latency. The load phase now runs against a second
permissive target and the suite refuses to publish a load figure if any sample was
refused. `buildReport` also publishes `evidenceVerified`, so the verdict is
re-derivable from the report — asserted by #40's acceptance test, which only trips once
the verdict is GO.
Evidence on the merged tree: `npm test` 277/277, `test:security-attack` 8/8, the RC attack
suite returns VERDICT: GO with both P0 ingress cases passing (main is BLOCKED on exactly
those two), 14 browser acceptance suites green, and the release demo records end to end
(8:13, 97 captions) including the on-camera refusal scene.
The RC security card rendered `http undefined · worker 12` for the flood case,
which records no status, and repeated its bare status for the malformed case,
which records no provider call. Both read as a broken card rather than the
measurement it is. Build each row from the observed fields that exist and leave
the rest out.
Executable abuse suite against the real boundary with a counting stub worker, plus the verdict rule that makes a failing P0 case block RC approval. First run: VERDICT BLOCKED (two P0 ingress holds, one P1 queue hold).
Add an executable abuse suite that drives the real server with synthetic hostile bytes while a
counting stub worker records every call that reaches the model boundary, plus the verdict rule that
turns those observations into a release decision.
- src/release-security-gate.js: nearest-rank latency percentiles, per-case pass/fail derivation that
names the field that broke, and the rule that any failing P0 case blocks RC approval while a
required class with no verified evidence blocks it too. Reports are redacted and fail closed.
- scripts/security_attack_suite.mjs: oversized body, HTML polyglot as JPEG, undecodable bytes,
missing consent, non-conforming worker answer, byte-identical replay, concurrent burst, captured
log scan, and an out-of-surface deletion case counted only against a real test file. Writes the
report, prints the verdict, exits 1 when blocked.
- docs/RC-SECURITY-EVIDENCE.md: procedure, rule, and the current finding stated plainly.
- Demo: a real run of the suite on camera, printing the verdict it derived instead of a flattering
one; CI regenerates the same evidence on every change.
The suite currently returns VERDICT: BLOCKED: bytes are trusted from the data-URL label, so an HTML
polyglot still reaches the worker and only fails afterwards on schema. That is a finding against
#16, not a passing check, and it is why the gate treats a failing case as a block.
The release-delta caption is paragraph length. Rendered at 94% opacity over the running app, the
app's own text bled through it, and the final line sat flush against the bar's bottom edge — the one
caption a reviewer most needs to read was the hardest one to read. The bar is now opaque, long copy
steps down to 10.5/9.5px, and caption() measures the last line box with a Range and fails the run when
the bottom padding is under 3px, so a cramped caption can no longer be recorded as proof.
Release builder runs `git diff --check`; scripts/review_corpus.py ended with an extra newline, so the gate failed on the PR head. Whitespace only; no behaviour change.
Confidence was the only thing standing between a model answer and a
prefilled journal entry, so a confident reading of a blurry photo could
still be offered as fact. Quality is now part of the policy: `poor`
abstains however confident the reply looked, with its own fixed copy.
The confidence floor, accepted quality levels, and refusal copy move into
src/abstention-policy.js so the parser, the corpus, and the browser cannot
drift into three explanations of the same refusal. parseVisionResponse
takes an optional policy, which is how the tests prove the parser carries
no hidden limits of its own.
evaluatePrefillGate turns the #29 calibration report into one release
decision: GO plus a validation-selected threshold scored on the held-out
test split, or prefill is blocked with each reason named. A missing report
is a block, never a default allow. scripts/abstention_policy_receipt.mjs
exposes that as a receipt (exit 0 / 3 / 2).
The committed synthetic predictions currently report BLOCKED and the
receipt says so. No candidate exists until #31, and the builder does not
hard-gate on the receipt yet; the on-camera quality gate is the evidence
this release carries.
The corpus gains a case that replays a 0.94-confidence answer about the
synthetic motion-blur fixture, and the release demo asserts in the browser
that nothing is prefilled before captioning it.
Verified: 226 unit tests, calibration harness (19), corpus manifest (8),
vision-refusal, photo-first, and provisional acceptance, check:syntax,
check:diff, and a full real-browser demo recording whose caption receipt
confirms the new scene painted on screen.
The ROADMAP gates (macro-F1 >= 0.80, precision@coverage >= 0.90, ECE <= 0.05) were prose with no
tool behind them. scripts/calibration_report.py now computes them from prediction records and
refuses input it cannot honestly score:
- a split is re-derived from the pseudonymous subject key with the ingestion rule, so a corpus
cannot re-label its own splits; a contributor spanning two splits is refused before scoring
- the threshold is selected on validation only, and selecting on test or train is refused
- coverage, precision-at-coverage, macro-F1, per-class precision/recall, confusion, and expected
calibration error are computed on accepted suggestions, with classes that abstained away named
in excludedClasses instead of being scored as a silent zero
- exit codes carry the verdict: 0 GO, 3 BLOCKED, 2 refused
The release demo records the real harness against committed synthetic fixtures: a blocked
three-candidate comparison, a fixture engineered to clear every gate, and a leaked corpus refused.
Every number on the card is read back from the report the run wrote, and the captions state that
no model was measured, because a fixture is not evidence about accuracy.
The harness suite is wired into CI, check:syntax, and the release builder's own gates.
CI's diff-hygiene gate rejects a trailing blank line at EOF; the local
check:diff run happened before the first commit existed, so it checked the
previous range and missed it.
Promote the ingest script into an auditable corpus pipeline. Each sample now
records consent (version + timestamp), source kind and raw-input digest, both
hashes, the derivation pipeline, review status, split, and lifecycle state, with
the caller's path never written down.
Withdrawal is a new `--revoke <contribution-id> --reason` operation: it deletes
the derived image and any raw pre-derivation copy, then appends a tombstone that
addresses each removed artifact by contribution id and digest while dropping
labels, subject key, and paths. History is append-only, so the original sample
record survives the withdrawal it documents.
Illegal moves are refused instead of silently applied: revoked -> active,
revoking an unknown or already-revoked contribution, an unknown state, a
duplicate contribution id, and a second label set for the same derived image.
The duplicate guard runs before any byte is written, so a refused ingest cannot
strand an image in the active dataset.
Tests written first and watched fail (tests/corpus-manifest.test.py): provenance,
tombstone + byte removal, active-record filtering, and the transition guards.
The transition helper itself passed on first write and is reported as a
hold-out check rather than a RED cycle. Wired into the quality workflow and
check:syntax; contract documented in docs/CORPUS-MANIFEST.md.
No browser surface changed, so record_release_demo.mjs is untouched and claims
nothing new: the nearest honest visible effect is the withdrawn-receipt
tombstone already recorded for #24.
The opt-in row and its tombstone rendered as one run-on line ('Type 4 ·
brownReceipt contrib-…', 'Withdrawn Oct 21, 12:09 PMraw image, …') because
nothing styled .contribution-row: the label and the receipt/artifact line
shared a line and the Withdraw control butted against the text.
Split label and detail into their own lines, right-align the row action,
mute withdrawn tombstones, and keep long receipt ids wrapping inside the
card. Verified on a 390px viewport: queued, withdrawn, and offer rows are
separately legible with no horizontal overflow.
A journal save now records no training contribution. A contribution is a second
consent: the privacy page states consent version training-consent-v1 and the
receipt (contribution id, included fields, never-included fields) before the
confirm control is enabled, the receipt is queued on this device only and copies
no photo bytes, note text, or symptom answers, and withdrawal keeps a tombstone
naming every derived artifact by contribution id. Delete-all clears the receipts
with the ledger.
Closes#24
The demo now picks a committed text fixture that only claims to be a photo, asserts on camera that
nothing was posted to /api/analyze, and keeps manual entry in the same scene. The builder requires
that caption to have painted and records image_ingress_refusal in the manifest; the threat model
names each implemented control, its module, and the tests that enforce it.
Selecting a file the browser cannot decode used to render "Timmy couldn't analyze that safely",
which described an upload that never happened. It now states that the file isn't a photo Timmy can
read and that nothing was uploaded, with manual entry and the picker both one tap away. The state is
asserted in a new browser suite that also asks the running server to refuse the same bytes, and the
existing mobile capture suite asserts the new copy across both viewports.
The declared data-URL MIME was a claim the boundary believed. Every image decision now comes from
the bytes: the container is sniffed, walked to its own end marker, refused when metadata segments
remain, and refused above a 4096px edge or 16MP. The request path adds a per-caller sliding-window
budget (429 + Retry-After) and a body deadline (408), so a flood, a stalled body, or a decode bomb
fails before a provider call.
Log hygiene is part of the contract: refusal copy is fixed prose, the bucket key is a salted hash
kept in memory only, and the acceptance test asserts no image bytes reach the process log.
A confirmed log and the image attached to it are separate things the
person owns. The journal now reports a saved photo as a label instead of
a thumbnail, reveals the image only on explicit request, removes the
photo without touching any confirmed field, and deletes a log through a
separate two-step decision that takes its photo with it. Delete-all is
regression tested to remove the ledger and every stored photo together.
- src/photo-lifecycle.js: presence decided by stored image bytes;
withoutPhoto/withoutEntry return new lists and never mutate the ledger
- app.js: journal rows expose Open-entry controls; entry sheet states
presence, offers explicit reveal, remove-photo-keeps-log, and a
two-step log delete; privacy copy names photo removal and delete-all
- tests: 6 unit cases + browser acceptance (presence, no default
thumbnail, explicit reveal, remove keeps log, two-step delete,
delete-all leaves no image bytes, Escape dismiss, 44px targets)
- record_release_demo.mjs: on-camera photo lifecycle scenes driven by a
synthetic 1x1 PNG, with delete-all actually performed and asserted;
build_release gates the new captions
- CI/release/syntax gates include the new suite
Closes#36
Chat now has to show its work. Every answer that states a count, a date, or a
visible form cites the confirmed records behind it, and following a citation
opens exactly that log. Questions the journal cannot answer (food safety,
medicine, what a symptom means) get boundary copy with no borrowed evidence,
and a sparse window refuses to describe a trend instead of naming one.
The Hermes path is held to the same rule: the server gives each ledger record a
positional ref, the prompt requires inline [entry:L<n>] citations, and the
browser resolves them against the ledger it sent. A ref that matches no record
is stripped and the answer is labelled unverified rather than rendered as a
source, so an invented citation cannot look like evidence.
The release recording now exercises the grounded local answer, its citation
sheet, the refusal, and a Hermes reply whose refs resolve, each asserted in the
browser with fictional logs.
Deterministic escalation only fired on a narrow set of literal phrases, so real reports
like "there is blood when I wipe" or "I have tarry stools" reached the chat provider
without the fixed medical guidance. Expanding the suite found 18 of 18 checked clinical
phrasings missing.
- domain: cover bleeding reported by wiping/toilet paper/anatomy/mixed-in phrasing,
black-tarry-dark-red stool, reported temperature and high temperature, abdominal-scoped
severe pain in every natural word order, and past-tense/slang inability to pass gas.
Near-miss wording that must stay quiet (paper cut, blood pressure, black beans, dark red
wine, a normal temperature, knee or back pain, passed gas, no gas in the car) is pinned.
- domain: a symptom reported in any truthy shape (1, "yes", "true") now counts as reported.
Reading only the literal boolean `true` silently cleared a symptom, which is the one
direction this boundary may never fail in.
- hermes-agent-service: the gateway ledger sanitizer used the same strict check, so a
browser-supplied ledger entry could have its symptom cleared before the escalation check.
- tests/escalation-regression.test.js: per-flag positive and negative language coverage,
model output can neither set nor clear a symptom/urgency/discomfort/note, and the
escalation copy is byte-identical for every flag and under a missing, throwing, or
malicious provider. Registered in the default unit gate.
- scripts/record_release_demo.mjs: the recording now types the new bleeding and tarry
phrasing, and seeds a fictional ledger whose note says "stool was tarry yesterday", then
asserts in the browser that exactly one new Timmy bubble carries the app's own copy and
that the turn never reaches Hermes. Three new captions are required release gates.
Boundary unchanged: no new inference, no diagnosis, no image claim. Escalation only ever
gets broader on user-reported symptoms.
The home insight card now reports the confirmed-log count and form
groupings for a window, every line names the exact confirmed records it
was built from, and following a citation opens those records. A window
below the minimum states that it is too sparse instead of describing a
trend, and a window with both firm and loose logs says it is mixed rather
than picking a winner. No line explains a cause, rules on food, or offers
a health verdict.
- src/summary.js: window filtering, insufficient-data guard, per-form
lines, mixed-pattern line, boundary copy, record citations
- app.js: explainable home card plus a pattern-source sheet listing the
cited records
- tests/summary.test.js, tests/pattern-summary.acceptance.mjs: unit and
real-browser coverage, gated in CI and the release builder
- scripts/record_release_demo.mjs: fictional multi-day sample logs,
citation follow-through, and the sparse-window refusal on camera
The blank-first ordering let one blank field short-circuit the whole reply, so a
response carrying an impossible value came back as `unusable_visual_fields`:
the user was told the photo was unreadable when the model had in fact returned
something Timmy must never apply.
Blank fields still abstain and still prefill nothing. A filled-in field is a
claim, and a claim outside the contract now fails closed even when the other
field came back blank.
The corpus earns its label too. `out-of-range-fields` drops from confidence 1.4
to 0.9 so the case reaches the field-range check instead of dying at the
confidence gate, and a new `blank-plus-out-of-range` case covers the class the
old ordering let through. The release demo replays that case on camera and the
fail-closed 503 budget moves from one to two, so the scene cannot be dropped
silently.
Closes#73
The worker contract was only proven by one happy-path fixture and a handful of
inline assertions, so nothing continuously checked that a reply Timmy cannot use
is refused instead of guessed at.
Corpus: a CC0 synthetic corpus (stool, non-stool, blur, glare, empty bowl,
malformed response) generated deterministically by scripts/build_vision_corpus.py.
Every fixture records its sha256, provenance, licence, case class, the provider
reply it replays, and the outcome that reply must produce. loadVisionCorpus
refuses a fixture whose bytes no longer match the manifest, so the corpus cannot
be edited into a different test. No real medical imagery, camera, or third-party
content is involved.
Contract fixes the corpus exposed:
- A schema-valid reply with a null form and colour is now an abstention
(abstainedFor: unusable_visual_fields) instead of a hard provider failure. The
declared response schema permits those nulls, so treating the reply as an error
was wrong; it now abstains calmly and still prefill nothing.
- Fail-closed replies carry a machine-readable code (invalid_json, invalid_fields)
instead of prose matching, and visionFailureResponse() maps them to plain,
user-safe copy with no provider text, image bytes, or credentials echoed. A
running server now returns that copy for unparsable and out-of-range replies.
Evidence: unit corpus contract and hash integrity, corpus-wide conformance
through the real analysis path, the same corpus through a running server.mjs with
a stub worker (200 abstain vs 503 fail-closed, no leaked bytes), and a real
browser acceptance run for the refusal UI. The release demo replays three corpus
cases through the real code path so the video shows the status the server would
actually send, with the on-camera caption gated by the release builder.
- docs/PHOTO-DATAFLOW-THREAT-MODEL.md: browser, API ingress, model worker,
transient buffers, training opt-in store, logs, exports, and deletion,
with trust boundaries and a numbered threat register
- every named abuse case (replay, oversized input, metadata leakage,
poisoning, unauthorized training, operator access) carries a mitigation,
an owner, a tracked follow-up ticket, and the artifact that will prove it
- tests/threat-model.test.js enforces diagram coverage, owner/follow-up
presence against ROADMAP ticket numbers, and the non-diagnostic boundary
with synthetic-only fixtures
- link the model from the accepted product decision record
Field finding #67: a hard/lumpy (Bristol 1-2) entry now asks a short follow-up
before saving — difficult to pass, soreness, visible red blood, truly
black/sticky/tarry versus dark brown, and dehydration. Every answer is
user-reported and starts unanswered; nothing is inferred from the photo, and
answering "dark brown" never silently escalates as black stool.
A confirmed red-flag answer writes the same symptom flag manual logging already
uses, so the existing deterministic escalation is the only escalation path.
Mild reports get concise non-diagnostic self-care that states the photo cannot
rule out bleeding, and it is withdrawn whenever a red flag is confirmed.
- src/followup.js: pure follow-up contract (9 unit tests, tests/followup.test.js)
- app.js + styles.css: follow-up block on the safety step for Type 1-2 entries only
- tests/followup.acceptance.mjs: browser acceptance incl. that the AI suggestion
surface still contains no symptom wording
- CI + release builder gates: npm run test:followup, syntax checks, manifest gate
- scripts/record_release_demo.mjs: records the real follow-up, mild guidance, and
the black/tarry alert handoff with synthetic data
Logging must never depend on AI availability, and an unsaved entry must not
disappear when the user navigates away mid-log.
- add src/vision-readiness.js so the photo flow consumes one readiness contract
- when the worker is offline/disabled/unknown, offer no AI action and no
analysis-consent prompt, and keep manual Bristol entry one action away
- add src/draft.js and make the draft own the photo so a chosen photo and
entered context survive a recoverable close or navigation, with a visible
kept-draft summary and an explicit start over
- drafts stay in session memory only: a photo awaiting consent is never
persisted, and reload starts clean
- service worker now refuses to cache image bodies in addition to analysis
requests and no-store responses
- tests: 2 new unit files, 3 service-worker privacy assertions (1 hold-out),
1 browser acceptance file, release-demo and CI/package assertions
- release demo now visibly replays the vision-offline pass and the retained
draft with synthetic data and an in-script assertion
timmy