Gate visual prefill on a versioned abstention policy (#32)
Quality gates / quality (pull_request) Successful in 13m29s
Quality gates / quality (pull_request) Successful in 13m29s
Confidence was the only thing standing between a model answer and a prefilled journal entry, so a confident reading of a blurry photo could still be offered as fact. Quality is now part of the policy: `poor` abstains however confident the reply looked, with its own fixed copy. The confidence floor, accepted quality levels, and refusal copy move into src/abstention-policy.js so the parser, the corpus, and the browser cannot drift into three explanations of the same refusal. parseVisionResponse takes an optional policy, which is how the tests prove the parser carries no hidden limits of its own. evaluatePrefillGate turns the #29 calibration report into one release decision: GO plus a validation-selected threshold scored on the held-out test split, or prefill is blocked with each reason named. A missing report is a block, never a default allow. scripts/abstention_policy_receipt.mjs exposes that as a receipt (exit 0 / 3 / 2). The committed synthetic predictions currently report BLOCKED and the receipt says so. No candidate exists until #31, and the builder does not hard-gate on the receipt yet; the on-camera quality gate is the evidence this release carries. The corpus gains a case that replays a 0.94-confidence answer about the synthetic motion-blur fixture, and the release demo asserts in the browser that nothing is prefilled before captioning it. Verified: 226 unit tests, calibration harness (19), corpus manifest (8), vision-refusal, photo-first, and provisional acceptance, check:syntax, check:diff, and a full real-browser demo recording whose caption receipt confirms the new scene painted on screen.
This commit is contained in:
1 parent
2a10a20518
commit
209e21a951
13 files changed
+428
-14
No files matched your search
@@ -7,6 +7,10 @@ splits that are contributor-held-out and reproducible, comparing candidate model
|
||||
The implementation is [`scripts/calibration_report.py`](../scripts/calibration_report.py). It reads
|
||||
predictions something else produced, and it refuses to compute gates it cannot honestly compute.
|
||||
|
||||
The GO/BLOCKED verdict this report produces is consumed by the production abstention policy in
|
||||
[`docs/ABSTENTION-POLICY.md`](ABSTENTION-POLICY.md): a candidate that has not cleared these gates
|
||||
cannot authorise a visual prefill.
|
||||
|
||||
**This harness does not train, does not diagnose, and never reads an image.** It only sees Bristol
|
||||
labels, predictions, and confidences.
|
||||
|
||||
|
||||
Reference in new issue
Block a user