fix(demo): count the deliberate ingress 400 in the browser error budget (#16)
Quality gates / quality (push) Canceled after 0s

The recording allows no browser console errors except the ones a scene
produces on purpose, and it enumerated only the two fail-closed 503s. The new
server-refusal probe adds exactly one 400, so the budget is now: two 503s, one
400, and nothing else. The first recording stopped here instead of shipping a
demo that had never exercised the boundary; the assertion is what caught it.
This commit is contained in:
Timmy Release Engineering committed 2026-10-31 12:21:16 +00:00
1 parent 2019d515c0
commit 9067ce5306
2 files changed
+11 -6

No files matched your search

+7 -5
View File
@@ -1646,12 +1646,14 @@ await page.evaluate(() => {
});
await sleep(2200);
// The refusal pass deliberately produces two fail-closed 503s (an out-of-range reply, and one blank field
// beside an out-of-range one), which the browser logs as failed resources. Requiring exactly those two
// keeps the scene honest: if the boundary stops refusing, this fails instead of quietly recording a happy
// path, and a dropped scene is caught rather than silently shrinking the evidence.
// Two deliberate refusals show up in the browser console as failed resources, and both are counted
// rather than tolerated: the ingress probe above posts non-image bytes to the real server (one 400),
// and the refusal pass produces two fail-closed 503s (an out-of-range reply, and one blank field
// beside an out-of-range one). Enumerating them keeps the scene honest: if the ingress boundary stops
// refusing, or a dropped scene shrinks the evidence, this fails instead of recording a happy path.
const expectedFailClosed = errors.filter(message => /Failed to load resource: the server responded with a status of 503 \(Service Unavailable\)/.test(message));
if (expectedFailClosed.length !== 2 || errors.length !== expectedFailClosed.length) {
const expectedIngressRefusal = errors.filter(message => /Failed to load resource: the server responded with a status of 400 \(Bad Request\)/.test(message));
if (expectedFailClosed.length !== 2 || expectedIngressRefusal.length !== 1 || errors.length !== 3) {
throw new Error(`Browser errors: ${errors.join(' | ') || 'unexpected error count'}`);
}
// Machine-readable receipt of exactly what was rendered on camera, so the release gate can prove
+4 -1
View File
@@ -311,8 +311,11 @@ test('release demo shows a filled-in out-of-range field failing closed even besi
assert.match(demo, /tests\/fixtures\/vision-corpus\/blank-plus-out-of-range\.png/);
assert.match(demo, /A filled-in field that is out of range still fails closed, even when the other field came back blank/);
assert.match(demo, /no longer told the photo was unreadable when the model returned an impossible value/);
// The extra fail-closed scene is a real 503, so the demo's error budget must count it.
// The extra fail-closed scene is a real 503, so the demo's error budget must count it, and the
// deliberate ingress probe is the only 400 the recording is allowed to produce.
assert.match(demo, /expectedFailClosed\.length !== 2/);
assert.match(demo, /expectedIngressRefusal\.length !== 1/);
assert.match(demo, /errors\.length !== 3/);
// The caption is a required gate, so a later release cannot silently drop the scene.
assert.match(builder, /A filled-in field that is out of range still fails closed/);
});