The first recorded card split 'ANALYZE REQUESTS' and every 'FAILURE' label
across two lines, detaching each label from its count. Long CLI labels are
now folded into values (one wrapped failures line), the key column holds
only short keys, and the card was geometry-probed at the recording's phone
viewport before rebuilding: 706px of content in a 706px panel, no clipped
key.
The release video proved the disclosure sentence but never exercised the
operator surface it describes. This records the real
scripts/model_metrics_report.mjs against the recording's own server, shows
the counters that run produced, and proves a wrong token is refused instead
of answered.
The first run of that CLI printed 'failure ingress_refused0': the longest
failure label exactly filled the 22-character column, so label and count
rendered as one broken token. LABEL_WIDTH is now 30, and a format test
fails if any row loses the gap between its label and its value.
The counters had an endpoint and no way to read them without jq. Adds a pure
formatter plus a CLI that prints the app, queue, and model boundaries.
The formatter is a function of the snapshot and the three frozen vocabularies,
so a caller-supplied failure label cannot reach the terminal: unknown classes
are simply not printed. Refusal paths are explicit — 404 means the route is off
because no token is configured, 403 means the token was rejected — and the token
is never echoed in the report or an error line.
RED: tests/model-metrics-report-format.test.js failed with ERR_MODULE_NOT_FOUND
before src/model-metrics-report.js existed; the refusal-path test failed with a
generic 'answered 404' before the 404 branch was written.
Tests: npm test 298/298, npm run test:metrics-report 8/8, syntax and diff gates
clean, real-Chromium ui acceptance green with the privacy disclosure visible.
The recording allows no browser console errors except the ones a scene
produces on purpose, and it enumerated only the two fail-closed 503s. The new
server-refusal probe adds exactly one 400, so the budget is now: two 503s, one
400, and nothing else. The first recording stopped here instead of shipping a
demo that had never exercised the boundary; the assertion is what caught it.
The #16 demo scene stopped at the client guard, so the boundary the ticket is
actually about — the server deciding from bytes, before any model work — was
never on camera. The recording now lifts its own /api/analyze interceptor for
one probe, posts non-image bytes to the running app, and reads the status and
copy back out of that response. A 200 fails the recording.
Both captions are required release gates, so a later release cannot drop the
scene silently. Rehearsed against the real server: 400 not_an_image, card
measured legible (opaque, in frame, no scroll), interceptor restored before
the next scene.
The release builder runs its browser gates with TIMMY_VISION_ENABLED=false, and
the 503 "AI analysis is disabled" branch ran before the request budget, body
limits, and byte-level ingress guard. Under builder conditions every hostile
upload answered 503, so the refusal boundary was never exercised and the
ingress-capture gate this same PR added failed:
AssertionError: the server must refuse non-image bytes, saw 503
Refusals now run in cost order regardless of model availability: budget, body,
bytes, then the disabled switch. Verified with vision disabled: hostile bytes
-> 400 not_an_image, a valid synthetic photo -> 503 with the existing copy.
npm test 277 pass; ingress-capture and the other browser gates green under the
builder's env.
Brings #82 up to today's `main` (1bbcc10) and resolves the two conflicts as ordered
unions: `package.json` `test`/`check:syntax` and the release builder's syntax allow-list
and evidence keys. Both sides only appended, so nothing was dropped.
Verifying the merge against main's RC attack suite (#40) found two real defects, each
fixed test-first:
- An oversized body destroyed the socket mid-upload, so the caller saw `fetch failed`
instead of the 413 it had already earned, and `security_attack_suite.mjs` could not
complete at all. New test in `tests/analyze-ingress.acceptance.test.js` failed with
`fetch failed` before the fix; `src/http-body.js` now keeps draining without buffering
and refuses on `end`, so the ceiling still holds and the refusal is readable.
- #16's per-caller budget turned the suite's own load phase into 8/8 429s, which would
have published refusals as analysis latency. The load phase now runs against a second
permissive target and the suite refuses to publish a load figure if any sample was
refused. `buildReport` also publishes `evidenceVerified`, so the verdict is
re-derivable from the report — asserted by #40's acceptance test, which only trips once
the verdict is GO.
Evidence on the merged tree: `npm test` 277/277, `test:security-attack` 8/8, the RC attack
suite returns VERDICT: GO with both P0 ingress cases passing (main is BLOCKED on exactly
those two), 14 browser acceptance suites green, and the release demo records end to end
(8:13, 97 captions) including the on-camera refusal scene.
The RC security card rendered `http undefined · worker 12` for the flood case,
which records no status, and repeated its bare status for the malformed case,
which records no provider call. Both read as a broken card rather than the
measurement it is. Build each row from the observed fields that exist and leave
the rest out.
Executable abuse suite against the real boundary with a counting stub worker, plus the verdict rule that makes a failing P0 case block RC approval. First run: VERDICT BLOCKED (two P0 ingress holds, one P1 queue hold).
Add an executable abuse suite that drives the real server with synthetic hostile bytes while a
counting stub worker records every call that reaches the model boundary, plus the verdict rule that
turns those observations into a release decision.
- src/release-security-gate.js: nearest-rank latency percentiles, per-case pass/fail derivation that
names the field that broke, and the rule that any failing P0 case blocks RC approval while a
required class with no verified evidence blocks it too. Reports are redacted and fail closed.
- scripts/security_attack_suite.mjs: oversized body, HTML polyglot as JPEG, undecodable bytes,
missing consent, non-conforming worker answer, byte-identical replay, concurrent burst, captured
log scan, and an out-of-surface deletion case counted only against a real test file. Writes the
report, prints the verdict, exits 1 when blocked.
- docs/RC-SECURITY-EVIDENCE.md: procedure, rule, and the current finding stated plainly.
- Demo: a real run of the suite on camera, printing the verdict it derived instead of a flattering
one; CI regenerates the same evidence on every change.
The suite currently returns VERDICT: BLOCKED: bytes are trusted from the data-URL label, so an HTML
polyglot still reaches the worker and only fails afterwards on schema. That is a finding against
#16, not a passing check, and it is why the gate treats a failing case as a block.
The app keeps the journal in the person's browser, so the server has no diary to lose. What an
operator can lose is the corpus: the append-only manifest.jsonl with its provenance records and its
withdrawal tombstones. scripts/operator_backup.py archives only records and names every path it
refuses (model weights, transient inference images, secrets, logs, symlinks, oversize) with the
reason, writes one SHA-256 per stored member, and produces a reproducible archive.
restore re-verifies every digest, member name, and record count before it writes a byte, and refuses
absolute, traversal, link, unlisted, and non-empty-target cases, so a tampered archive or a lost
tombstone fails loudly. docs/PRODUCTION-RUNBOOK.md covers topology, network boundaries (including
the app's real 0.0.0.0 default and the staging HOST=127.0.0.1 template), the four secrets, retention,
and both rollback paths; its documented commands are parsed against the real CLIs by its test.
The release demo runs the real backup, verify, and restore against a synthetic corpus built by the
real ingest and revoke commands and shows what was stored, what was refused, and what came back.
The release-delta caption is paragraph length. Rendered at 94% opacity over the running app, the
app's own text bled through it, and the final line sat flush against the bar's bottom edge — the one
caption a reviewer most needs to read was the hardest one to read. The bar is now opaque, long copy
steps down to 10.5/9.5px, and caption() measures the last line box with a Range and fails the run when
the bottom padding is under 3px, so a cramped caption can no longer be recorded as proof.
Publish the dataset card and labeling guide from the manifest (#27)
Regression tests: npm test 229 pass; dataset-card 14, corpus-manifest 8, corpus-review 21, calibration-harness 19 OK.
The corpus had an append-only manifest, a reviewer tool, and an evaluation harness, but nothing a
stranger could read: what is in the corpus, what a label means, and how leakage is prevented lived in
code and in reviewers' heads.
scripts/dataset_card.py generates docs/DATASET_CARD.md from manifest.jsonl. Every count is read back
from real records — active samples, contributors, per-split contributor and sample counts, class
balance over active samples only — and every fact the pipeline does not collect (devices, contexts,
licenses) is printed as pending rather than estimated. The generator refuses to render a card for a
corpus where one contributor straddles two splits or where a recorded split is not the one the
subject key derives, and it never publishes a subject key or a withdrawn sample's labels. --check
fails when the committed page is not the generator's own output, so the card cannot drift into a
hand-written document.
docs/LABELING_GUIDE.md is the reviewer's rulebook: visible form, broad visible color, and photo
quality; visible color is not blood; nothing is diagnosed; a model suggestion is provisional input to
a human decision. Exclusions are named, and the split policy is stated where it is enforced.
The demo's new DATASET CARD scene builds a throwaway corpus with the real ingest command from
committed synthetic fixtures, publishes the card, and shows the refusal a planted contributor earns,
then re-checks the committed page. Its captions are release gates in build_release.py, which also runs
tests/dataset-card.test.py before a release exists.
tests/dataset-card.test.py covers the counts, the pending facts, the withdrawal receipt, both
refusals, determinism, the empty-corpus case, the committed-page identity, and both documents'
prohibitions. Every fixture is synthetic.
Closes#27
Release builder runs `git diff --check`; scripts/review_corpus.py ended with an extra newline, so the gate failed on the PR head. Whitespace only; no behaviour change.
The corpus manifest records what was ingested and whether the person still allows it; it does not
record whether the labels are right. This adds the labeling control the data-flow threat model
named: scripts/review_corpus.py, a command-line tool with no app screen.
- The review queue shows the contributor's proposal and carries no model field at all: blindness is
structural, because build_queue() never receives predictions.
- reveal() refuses a contribution with no committed review, so the model answer cannot steer the
human who labels it. Revealing appends nothing.
- A review is one append-only line carrying the proposal it judged; a correction must change it. An
adjudication is a second reviewer's line that names the record it resolves; the review it
overrules survives untouched.
- A withdrawn contribution cannot be queued, reviewed, revealed, adjudicated, or trained on, and the
refusal says it was withdrawn rather than that it is already reviewed.
- trainable_records() returns only active samples with a human decision that was not an exclusion,
so unreviewed proposals are never training data.
Tests are written first against every refusal (tests/corpus-review.test.py, 21 cases); the release
builder runs the suite and gates four on-camera captions, and the demo now runs the real queue ->
review -> reveal -> adjudicate -> withdraw sequence against the committed synthetic sample and draws
the evidence card from that output. docs/REVIEWER-TOOL.md is the contract; no model was measured.
Confidence was the only thing standing between a model answer and a
prefilled journal entry, so a confident reading of a blurry photo could
still be offered as fact. Quality is now part of the policy: `poor`
abstains however confident the reply looked, with its own fixed copy.
The confidence floor, accepted quality levels, and refusal copy move into
src/abstention-policy.js so the parser, the corpus, and the browser cannot
drift into three explanations of the same refusal. parseVisionResponse
takes an optional policy, which is how the tests prove the parser carries
no hidden limits of its own.
evaluatePrefillGate turns the #29 calibration report into one release
decision: GO plus a validation-selected threshold scored on the held-out
test split, or prefill is blocked with each reason named. A missing report
is a block, never a default allow. scripts/abstention_policy_receipt.mjs
exposes that as a receipt (exit 0 / 3 / 2).
The committed synthetic predictions currently report BLOCKED and the
receipt says so. No candidate exists until #31, and the builder does not
hard-gate on the receipt yet; the on-camera quality gate is the evidence
this release carries.
The corpus gains a case that replays a 0.94-confidence answer about the
synthetic motion-blur fixture, and the release demo asserts in the browser
that nothing is prefilled before captioning it.
Verified: 226 unit tests, calibration harness (19), corpus manifest (8),
vision-refusal, photo-first, and provisional acceptance, check:syntax,
check:diff, and a full real-browser demo recording whose caption receipt
confirms the new scene painted on screen.
The ROADMAP gates (macro-F1 >= 0.80, precision@coverage >= 0.90, ECE <= 0.05) were prose with no
tool behind them. scripts/calibration_report.py now computes them from prediction records and
refuses input it cannot honestly score:
- a split is re-derived from the pseudonymous subject key with the ingestion rule, so a corpus
cannot re-label its own splits; a contributor spanning two splits is refused before scoring
- the threshold is selected on validation only, and selecting on test or train is refused
- coverage, precision-at-coverage, macro-F1, per-class precision/recall, confusion, and expected
calibration error are computed on accepted suggestions, with classes that abstained away named
in excludedClasses instead of being scored as a silent zero
- exit codes carry the verdict: 0 GO, 3 BLOCKED, 2 refused
The release demo records the real harness against committed synthetic fixtures: a blocked
three-candidate comparison, a fixture engineered to clear every gate, and a leaked corpus refused.
Every number on the card is read back from the report the run wrote, and the captions state that
no model was measured, because a fixture is not evidence about accuracy.
The harness suite is wired into CI, check:syntax, and the release builder's own gates.
On the CLI ingest path there is no raw pre-derivation copy at all, so the
caption asserted a deletion the recording never showed. The code deletes a raw
copy only when one exists under the dataset's raw/ directory, matched by digest.
The first recording of the card was legible but translucent, so the bottom nav
and the journal's body copy ghosted through its empty lower rows. The card is
now fully opaque and the on-screen proof rejects a background with any alpha, so
a future edit cannot reintroduce the bleed unnoticed.
Issue #25's corpus pipeline is a CLI with no app screen, so the demo runs the
real ingest and revoke commands against the committed synthetic fixture in a
throwaway dataset and draws the on-camera audit card from their actual output:
the provenance record, the deleted derived image, the kept original record, the
tombstone that drops labels and the subject key, zero active samples, and a
second revocation refused without appending anything.
The card is asserted on screen (fixed, topmost, unclipped) and the values on it
are read back from the run, so a build that stops recording provenance or stops
deleting bytes fails instead of recording a claim. build_release.py now runs
tests/corpus-manifest.test.py as a release gate and requires the four new
captions to have painted on camera.
CI's diff-hygiene gate rejects a trailing blank line at EOF; the local
check:diff run happened before the first commit existed, so it checked the
previous range and missed it.
Promote the ingest script into an auditable corpus pipeline. Each sample now
records consent (version + timestamp), source kind and raw-input digest, both
hashes, the derivation pipeline, review status, split, and lifecycle state, with
the caller's path never written down.
Withdrawal is a new `--revoke <contribution-id> --reason` operation: it deletes
the derived image and any raw pre-derivation copy, then appends a tombstone that
addresses each removed artifact by contribution id and digest while dropping
labels, subject key, and paths. History is append-only, so the original sample
record survives the withdrawal it documents.
Illegal moves are refused instead of silently applied: revoked -> active,
revoking an unknown or already-revoked contribution, an unknown state, a
duplicate contribution id, and a second label set for the same derived image.
The duplicate guard runs before any byte is written, so a refused ingest cannot
strand an image in the active dataset.
Tests written first and watched fail (tests/corpus-manifest.test.py): provenance,
tombstone + byte removal, active-record filtering, and the transition guards.
The transition helper itself passed on first write and is reported as a
hold-out check rather than a RED cycle. Wired into the quality workflow and
check:syntax; contract documented in docs/CORPUS-MANIFEST.md.
No browser surface changed, so record_release_demo.mjs is untouched and claims
nothing new: the nearest honest visible effect is the withdrawn-receipt
tombstone already recorded for #24.
A leftover checkout answering on 127.0.0.1:4173 satisfied the readiness probe,
so every acceptance flow and the recorded demo described another tree while the
build reported green. The builder now fails closed if the port already has a
listener, re-checks the spawned server after readiness, and verifies the served
app.js and src/domain.js bytes match the clone under test before the first gate.
Verified against a real stale tree: the occupied-port guard aborts, the
foreign-bytes guard aborts, and the matching tree still passes.
The import screen already states what it restored, and the migration notice
restated the same count, so the on-screen notice read as two stacked sentences
saying one thing. The notice now only describes the upgrade, and a unit test
fails if it starts restating the restored count again.
A backup is the person's own record and has to keep working as the AI fields evolve.
- Export stamps schema version 2 and carries the confirmed logs plus the training
contribution receipts, including withdrawn tombstones; both are written through an
allowlist so unknown keys and credentials never leave the device.
- Import accepts version 1 and 2. A version 1 file is upgraded, the notice names the
version it came from, and entries that predate provenance stay null instead of being
given a source they never recorded.
- A file from a newer Timmy is refused with an update instruction, malformed, oversized
and over-limit files fail closed, and a refused import leaves the ledger unchanged.
- A log saved from a confirmed or corrected AI suggestion records its provenance
(source, suggested visible fields, model, confirmedAt) and the entry sheet states it.
- Service worker shell cache v5 -> v6 so installed clients actually receive the change.
Evidence: tests/portability.test.js (7 unit cases), tests/portability.acceptance.mjs
(real browser: v1 upgrade, future refusal with unchanged ledger, provenance recorded,
export v2, round trip), release demo scenes with on-camera caption proof, and the
release builder gate portable_backup_restore.
Refs #35
The opt-in row and its tombstone rendered as one run-on line ('Type 4 ·
brownReceipt contrib-…', 'Withdrawn Oct 21, 12:09 PMraw image, …') because
nothing styled .contribution-row: the label and the receipt/artifact line
shared a line and the Withdraw control butted against the text.
Split label and detail into their own lines, right-align the row action,
mute withdrawn tombstones, and keep long receipt ids wrapping inside the
card. Verified on a 390px viewport: queued, withdrawn, and offer rows are
separately legible with no horizontal overflow.
A journal save now records no training contribution. A contribution is a second
consent: the privacy page states consent version training-consent-v1 and the
receipt (contribution id, included fields, never-included fields) before the
confirm control is enabled, the receipt is queued on this device only and copies
no photo bytes, note text, or symptom answers, and withdrawal keeps a tombstone
naming every derived artifact by contribution id. Delete-all clears the receipts
with the ledger.
Closes#24
The demo now picks a committed text fixture that only claims to be a photo, asserts on camera that
nothing was posted to /api/analyze, and keeps manual entry in the same scene. The builder requires
that caption to have painted and records image_ingress_refusal in the manifest; the threat model
names each implemented control, its module, and the tests that enforce it.
Selecting a file the browser cannot decode used to render "Timmy couldn't analyze that safely",
which described an upload that never happened. It now states that the file isn't a photo Timmy can
read and that nothing was uploaded, with manual entry and the picker both one tap away. The state is
asserted in a new browser suite that also asks the running server to refuse the same bytes, and the
existing mobile capture suite asserts the new copy across both viewports.
The declared data-URL MIME was a claim the boundary believed. Every image decision now comes from
the bytes: the container is sniffed, walked to its own end marker, refused when metadata segments
remain, and refused above a 4096px edge or 16MP. The request path adds a per-caller sliding-window
budget (429 + Retry-After) and a body deadline (408), so a flood, a stalled body, or a decode bomb
fails before a provider call.
Log hygiene is part of the contract: refusal copy is fixed prose, the bucket key is a salted hash
kept in memory only, and the acceptance test asserts no image bytes reach the process log.
The demo injected its caption CSS with page.addStyleTag(), which writes into the current document
only. The first page.reload() in the recording stripped that stylesheet, so every later caption
rendered as unstyled body text clipped at the bottom of the frame and every touch indicator was
invisible, while the caption receipt still listed each string and the release gate accepted it.
- caption and touch nodes now carry inline styles, which no reload can remove
- caption() proves each caption is a fixed bar at the top of the viewport, on top of what is
beneath it, fully inside the frame, and that the caption band pixels change; the recording fails
rather than shipping a caption nobody can read
- the receipt records per-caption evidence and build_release.py requires every required caption,
including the release delta, to be pixel-verified
- the demo photo is a legible synthetic test card instead of a 1x1 white pixel that rendered on
camera as an empty box
- tests assert the inline styling and the pixel gate
A confirmed log and the image attached to it are separate things the
person owns. The journal now reports a saved photo as a label instead of
a thumbnail, reveals the image only on explicit request, removes the
photo without touching any confirmed field, and deletes a log through a
separate two-step decision that takes its photo with it. Delete-all is
regression tested to remove the ledger and every stored photo together.
- src/photo-lifecycle.js: presence decided by stored image bytes;
withoutPhoto/withoutEntry return new lists and never mutate the ledger
- app.js: journal rows expose Open-entry controls; entry sheet states
presence, offers explicit reveal, remove-photo-keeps-log, and a
two-step log delete; privacy copy names photo removal and delete-all
- tests: 6 unit cases + browser acceptance (presence, no default
thumbnail, explicit reveal, remove keeps log, two-step delete,
delete-all leaves no image bytes, Escape dismiss, 44px targets)
- record_release_demo.mjs: on-camera photo lifecycle scenes driven by a
synthetic 1x1 PNG, with delete-all actually performed and asserted;
build_release gates the new captions
- CI/release/syntax gates include the new suite
Closes#36
Chat now has to show its work. Every answer that states a count, a date, or a
visible form cites the confirmed records behind it, and following a citation
opens exactly that log. Questions the journal cannot answer (food safety,
medicine, what a symptom means) get boundary copy with no borrowed evidence,
and a sparse window refuses to describe a trend instead of naming one.
The Hermes path is held to the same rule: the server gives each ledger record a
positional ref, the prompt requires inline [entry:L<n>] citations, and the
browser resolves them against the ledger it sent. A ref that matches no record
is stripped and the answer is labelled unverified rather than rendered as a
source, so an invented citation cannot look like evidence.
The release recording now exercises the grounded local answer, its citation
sheet, the refusal, and a Hermes reply whose refs resolve, each asserted in the
browser with fictional logs.
Timmy Release
rockachopa
Timmy
timmy
rockachopa
mercury