fix(demo): count the deliberate ingress 400 in the browser error budget (#16)
Quality gates / quality (push) Canceled after 0s
Quality gates / quality (push) Canceled after 0s
The recording allows no browser console errors except the ones a scene produces on purpose, and it enumerated only the two fail-closed 503s. The new server-refusal probe adds exactly one 400, so the budget is now: two 503s, one 400, and nothing else. The first recording stopped here instead of shipping a demo that had never exercised the boundary; the assertion is what caught it.
This commit is contained in:
1 parent
2019d515c0
commit
9067ce5306
2 files changed
+11
-6
No files matched your search
@@ -1646,12 +1646,14 @@ await page.evaluate(() => {
|
||||
});
|
||||
await sleep(2200);
|
||||
|
||||
// The refusal pass deliberately produces two fail-closed 503s (an out-of-range reply, and one blank field
|
||||
// beside an out-of-range one), which the browser logs as failed resources. Requiring exactly those two
|
||||
// keeps the scene honest: if the boundary stops refusing, this fails instead of quietly recording a happy
|
||||
// path, and a dropped scene is caught rather than silently shrinking the evidence.
|
||||
// Two deliberate refusals show up in the browser console as failed resources, and both are counted
|
||||
// rather than tolerated: the ingress probe above posts non-image bytes to the real server (one 400),
|
||||
// and the refusal pass produces two fail-closed 503s (an out-of-range reply, and one blank field
|
||||
// beside an out-of-range one). Enumerating them keeps the scene honest: if the ingress boundary stops
|
||||
// refusing, or a dropped scene shrinks the evidence, this fails instead of recording a happy path.
|
||||
const expectedFailClosed = errors.filter(message => /Failed to load resource: the server responded with a status of 503 \(Service Unavailable\)/.test(message));
|
||||
if (expectedFailClosed.length !== 2 || errors.length !== expectedFailClosed.length) {
|
||||
const expectedIngressRefusal = errors.filter(message => /Failed to load resource: the server responded with a status of 400 \(Bad Request\)/.test(message));
|
||||
if (expectedFailClosed.length !== 2 || expectedIngressRefusal.length !== 1 || errors.length !== 3) {
|
||||
throw new Error(`Browser errors: ${errors.join(' | ') || 'unexpected error count'}`);
|
||||
}
|
||||
// Machine-readable receipt of exactly what was rendered on camera, so the release gate can prove
|
||||
|
||||
@@ -311,8 +311,11 @@ test('release demo shows a filled-in out-of-range field failing closed even besi
|
||||
assert.match(demo, /tests\/fixtures\/vision-corpus\/blank-plus-out-of-range\.png/);
|
||||
assert.match(demo, /A filled-in field that is out of range still fails closed, even when the other field came back blank/);
|
||||
assert.match(demo, /no longer told the photo was unreadable when the model returned an impossible value/);
|
||||
// The extra fail-closed scene is a real 503, so the demo's error budget must count it.
|
||||
// The extra fail-closed scene is a real 503, so the demo's error budget must count it, and the
|
||||
// deliberate ingress probe is the only 400 the recording is allowed to produce.
|
||||
assert.match(demo, /expectedFailClosed\.length !== 2/);
|
||||
assert.match(demo, /expectedIngressRefusal\.length !== 1/);
|
||||
assert.match(demo, /errors\.length !== 3/);
|
||||
// The caption is a required gate, so a later release cannot silently drop the scene.
|
||||
assert.match(builder, /A filled-in field that is out of range still fails closed/);
|
||||
});
|
||||
|
||||
Reference in new issue
Block a user