checkin 2026-10-18 (volt): the malformed-input class is measured on every tool -- 15 probes, 6 real defects reproduced at 461eede, 0 at HEAD
This commit is contained in:
1 parent
fdd81f289e
commit
96f96c9e42
1 file changed
+94
@@ -140,3 +140,97 @@ because it teaches readers that stated rules are decoration.
|
||||
The filename half needs no ruling: it is already clean since 2026-10-08 and the
|
||||
25 historical names can stay as the record. I recommend (a) leave the history,
|
||||
gate forward with `--since`.
|
||||
|
||||
---
|
||||
|
||||
# Check-in — Volt, 2026-10-18 (02:3xZ, appended; the block above is 00:16–00:5xZ)
|
||||
|
||||
## Did
|
||||
|
||||
Closed yesterday's filed carry — *"a tool that mis-codes a decodable but
|
||||
malformed input is not measured"* — by measuring it on every tool. Landed
|
||||
`tools/selftest_sweep.py` only; **no tool was edited, because no tool was
|
||||
wrong.**
|
||||
|
||||
**The class.** Input that decodes as UTF-8 and cannot be parsed: a `.json` that
|
||||
is not JSON, a `{}` where an audit file should be, a directory that is not a git
|
||||
repo, a queue with no records. It is the class where a tool is most tempted to
|
||||
answer `0` and exit 0 — exactly the `doc_census_check --tools` bug (`76043b7`).
|
||||
|
||||
**15 probes, each graded against the answer its own tool states**, not against a
|
||||
blanket "must be rc 2" — because that universal is false here: `check_lint`
|
||||
grades a junk log `MISSING` (rc 0 report, rc 1 under `--fail-on-fault`),
|
||||
`checkin_check` grades a heading-less check-in advisory (rc 0, its docstring
|
||||
says "never rc 1"), `queue_claim_check` reports `0 opening records` (rc 0; its
|
||||
`--fail-on-stale` is the gate). Nine tools must refuse at rc 2.
|
||||
|
||||
$ python3 tools/selftest_sweep.py
|
||||
read-refusal: 17 probes, 17 ok, 0 fault
|
||||
malformed-input: 15 probes, 15 ok, 0 fault # rc 0, 12 tools OK
|
||||
|
||||
**15/15 at HEAD is the honest finding, and stated as such: nothing was broken,
|
||||
nothing measured it.** Re-run against the revision's own copies
|
||||
(`git show HEAD:tools/*.py`) so the working tree is not grading itself: 15/15.
|
||||
|
||||
**The gate reproduces the class on real history.** Same axis, tools as they were
|
||||
at `461eede` (before the rc-2 unification `6ee8fce`):
|
||||
|
||||
malformed-input: 15 probes, 8 ok, 7 fault (NO_TOOL, RC_WRONG, TRACEBACK)
|
||||
read-refusal: 17 probes, 4 ok, 13 fault (NO_TOOL, RC_NOT_2, TRACEBACK)
|
||||
|
||||
Six of the seven are defects in tools that existed then — three raw tracebacks
|
||||
(`append_only_audit` not-a-repo; `append_only_owners` malformed-json and
|
||||
wrong-shape) and three rc 1 where the contract says 2 (`census_row_delta` both
|
||||
shapes; `selftest_sweep` not-a-repo). The seventh is `NO_TOOL`
|
||||
(`checkin_check.py` did not exist at that rev) — named, not skipped.
|
||||
|
||||
**It discriminates, which is the only thing that makes a gate worth its lines.**
|
||||
One bug reintroduced into a copy of `append_only_owners.py` (swallow
|
||||
`ValueError`, print `0 owners adjudicated`, `return 0`):
|
||||
|
||||
malformed-input: 15 probes, 14 ok, 1 fault (RC_WRONG)
|
||||
RC_WRONG rc 0, contract says 2: 0 owners adjudicated: Expecting value ...
|
||||
# and that same tool's own --selftest: 15 cells, 15 passed, 0 failed
|
||||
|
||||
The tool's own fixtures never covered the class, so it could regress in silence.
|
||||
That sentence is the whole unit.
|
||||
|
||||
**The axis found its own bug on the first run** — six probes were substituting
|
||||
`{badjson}` as a bare key, leaving a literal path that refused at rc 2 for the
|
||||
wrong reason. The required-substring check caught it; without it those six rows
|
||||
would have printed OK. Fixed and re-run before any of these numbers were written.
|
||||
|
||||
**Selftest 22 → 28 cells**, including c24 (the silent-zero shape: rc 0 where the
|
||||
contract says 2 → `RC_WRONG`), c26 (right code, wrong reason → `UNEXPLAINED`) and
|
||||
c28 (coverage: every tool with a missing-probe also has a malformed probe, so a
|
||||
new tool is visible rather than skipped):
|
||||
|
||||
$ python3 tools/selftest_sweep.py --selftest
|
||||
selftest: 28 cells, 28 passed, 0 failed
|
||||
|
||||
**Queue untouched.** `queue_index.py --check docs/queue-index.md` → rc 0
|
||||
(`49 sections, 45 ids, 4 continuation sections, 9 nested ids`) before this block
|
||||
and after it. No queue item opened or closed; open set unchanged (Q37 `:5278`
|
||||
pipe; career-lane Q53 held for Alexander).
|
||||
|
||||
## Blocked
|
||||
|
||||
Nothing.
|
||||
|
||||
## Next
|
||||
|
||||
- If a tool's malformed contract changes, its probe changes with it: the table is
|
||||
in `tools/selftest_sweep.py:MALFORMED_PROBES`, cited by line in the review, not
|
||||
inferred.
|
||||
- The axis grades the code, a message, and one required word — not whether the
|
||||
message is good. That is the next honest increment if it is ever needed; it is
|
||||
not claimed now.
|
||||
|
||||
## Ask
|
||||
|
||||
The four-heading ruling from the block above still stands (unchanged, still the
|
||||
only thing I need from Alexander). Nothing new.
|
||||
|
||||
Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-18-malformed-axis/`
|
||||
(`decision.md`, `selftest.txt`, `sweep-head.txt`, `head-copies.txt`,
|
||||
`ab-461eede.txt`, `discrimination.txt`).
|
||||
Reference in new issue
Block a user