Gate visual prefill on a versioned abstention policy (#32)
Quality gates / quality (pull_request) Successful in 13m29s

Confidence was the only thing standing between a model answer and a
prefilled journal entry, so a confident reading of a blurry photo could
still be offered as fact. Quality is now part of the policy: `poor`
abstains however confident the reply looked, with its own fixed copy.

The confidence floor, accepted quality levels, and refusal copy move into
src/abstention-policy.js so the parser, the corpus, and the browser cannot
drift into three explanations of the same refusal. parseVisionResponse
takes an optional policy, which is how the tests prove the parser carries
no hidden limits of its own.

evaluatePrefillGate turns the #29 calibration report into one release
decision: GO plus a validation-selected threshold scored on the held-out
test split, or prefill is blocked with each reason named. A missing report
is a block, never a default allow. scripts/abstention_policy_receipt.mjs
exposes that as a receipt (exit 0 / 3 / 2).

The committed synthetic predictions currently report BLOCKED and the
receipt says so. No candidate exists until #31, and the builder does not
hard-gate on the receipt yet; the on-camera quality gate is the evidence
this release carries.

The corpus gains a case that replays a 0.94-confidence answer about the
synthetic motion-blur fixture, and the release demo asserts in the browser
that nothing is prefilled before captioning it.

Verified: 226 unit tests, calibration harness (19), corpus manifest (8),
vision-refusal, photo-first, and provisional acceptance, check:syntax,
check:diff, and a full real-browser demo recording whose caption receipt
confirms the new scene painted on screen.
This commit is contained in:
timmy committed 2026-10-26 10:58:38 +00:00
1 parent 2a10a20518
commit 209e21a951
13 files changed
+428 -14

No files matched your search

+4
View File
@@ -7,6 +7,10 @@ splits that are contributor-held-out and reproducible, comparing candidate model
The implementation is [`scripts/calibration_report.py`](../scripts/calibration_report.py). It reads
predictions something else produced, and it refuses to compute gates it cannot honestly compute.
The GO/BLOCKED verdict this report produces is consumed by the production abstention policy in
[`docs/ABSTENTION-POLICY.md`](ABSTENTION-POLICY.md): a candidate that has not cleared these gates
cannot authorise a visual prefill.
**This harness does not train, does not diagnose, and never reads an image.** It only sees Bristol
labels, predictions, and confidences.