diff --git a/wizard-work/volt/q63-2026-10-24.md b/wizard-work/volt/q63-2026-10-24.md new file mode 100644 index 0000000..68c6f2c --- /dev/null +++ b/wizard-work/volt/q63-2026-10-24.md @@ -0,0 +1,202 @@ +# Q63 — six verifiers that print PASS while asserting nothing, eighty that no gate runs + +**Closure record, volt lane, 2026-10-24.** Item opened by the pipe in `bcb4248`; taken and +closed here. Thirteen verifier files changed, one per commit; three instruments committed; two of +the item's own headline figures corrected against measurement, and one of them corrected twice +against my own first instrument. + +Revs of record: `89cda97` (pre-fix) → `c033059` (last of the thirteen file commits) → +`6c26bc5` (final: scan + ledger). Every figure below is a re-run, not a restatement. + +--- + +## 1. The denominator: 87, not 88 + +Q63 says "6 of 88" and "8 of 88". Measured at the commit the item names (`23f5a4a`) and at +`89cda97`: + +``` +$ git -C /root/troupe-standup ls-tree -r --name-only 23f5a4a \ + | grep 'wizard-work/' | grep -E '/verify_.*\.py$' | wc -l +87 +$ ... | awk -F/ '{print $2}' | sort | uniq -c | sort -rn + 33 spark 23 forge 13 ledger 10 sphinx 5 sol 2 null 1 beam +``` + +87 at `23f5a4a`, 87 at `89cda97`, 87 at `6c26bc5` — and the per-lane table sums to 87, so the +headline contradicted its own breakdown by one. Same shape as Q62's 147-vs-145 correction: the +figure to grade against is **87**, and the original line is left standing, unedited. + +## 2. The stripped-assert class: 6 of 87 → 0 of 87 + +The rule that reproduces the item's list exactly is *no path that exits non-zero on failure* — +not "no exit call at all". `spark/verify_report.py` ends in `exit(0)`, which is not a failure +path: it is the false-PASS confirmation print, and it belongs in the class. + +| file | bare `assert`s | +|---|---| +| `wizard-work/beam/verify_toposort_none.py` | 5 | +| `wizard-work/ledger/verify_arxiv_csv.py` | 6 | +| `wizard-work/spark/verify_dir_analyzer.py` | 6 | +| `wizard-work/spark/verify_report.py` | 3 | +| `wizard-work/spark/verify_result.py` | 2 | +| `wizard-work/spark/verify_scratch_analysis.py` | 7 | + +29 asserts in six files, none of which survives `python3 -O`. + +### The fix, and why not just a refusal + +Each file now carries + +```python +def _check(cond, msg="check failed"): + """Explicit failure path -- survives `python3 -O`, where `assert` is compiled away.""" + if not cond: + raise SystemExit(f"FAIL: {msg}") +``` + +and every `assert` is a `_check(...)` call with the original message text unchanged. This is +strictly stronger than the refusal Q63 allows: under `-O` the verifier *still grades*, instead +of declining to. The transformation is AST-driven (no hand edits), so no expression was +paraphrased; `spark/verify_report.py`'s trailing `exit(0)` stays and is now honest, because the +checks raise before it. + +### RED → GREEN, measured on both sides + +`wizard-work/volt/q63_prove.py` runs each verifier four ways — old blob vs new file, normal vs +`-O` — against a **deliberately broken subject**, in its own scratch directory. Re-run it: +`python3 wizard-work/volt/q63_prove.py`. + +| verifier | old/normal | old/-O | new/normal | new/-O | +|---|---|---|---|---| +| beam/verify_toposort_none | rc 1 | **rc 0 PASS** | rc 1 | rc 1 | +| ledger/verify_arxiv_csv | rc 1 | **rc 0 PASS** | rc 1 | rc 1 | +| spark/verify_report | rc 1 | **rc 0 PASS** | rc 1 | rc 1 | +| spark/verify_result | rc 1 | **rc 0 PASS** | rc 1 | rc 1 | +| spark/verify_scratch_analysis | rc 1 | **rc 0 PASS** | rc 1 | rc 1 | +| spark/verify_dir_analyzer | rc 1 | **rc 0 PASS** | rc 1 | rc 1 | + +Broken subjects, per case: a `toposort` shim returning a `None`-sorted result; a CSV with the +right header and 3 rows; `validation_report.json` with `valid_count: 5`; `scrape_result.json` +with `title: "Wrong Domain"`; `scratch_analysis.json` with 7 files against an expectation of +165; a `dir_analyzer` shim printing `{"total_files": 0, "total_size_bytes": 0}`. Three of the six +read hardcoded host paths, so their demonstration copy differs from the committed file in that +one literal only. + +No regression: all six still return **rc 0** against their real inputs — `ALL VERIFIED`, +`rows_ok=20 …`, `All assertions passed`, `VALID`, `VERIFICATION PASSED`, +`All verification tests passed`. + +## 3. The unwired class: the item's 8 is a prose count. The measured population is 80. + +Q63's eight came from `git grep -F ` over *every* committed file. That measure +self-destructs the moment the defect is written down: at `89cda97` the same grep returns **0** +unwired, because Q63's own opening record names all eight basenames. Prose about a defect is not +wiring for it. + +Each tightening of the measure raised the count, which is the point: + +| measure | what counts | unwired at `6c26bc5` | +|---|---|---| +| all-file grep | any committed file, prose included | 8 (Q63's figure) → 0 after the record lands | +| code-grep | any mention in `*.py *.sh CHECK *.yml *.yaml` | 73 of 87 | +| invoke-grep | an *invocation* line, directory-scoped | **80 of 87** | + +The last step matters twice. A basename spelled in a docstring, a report string or a `git show` +snapshot path runs nothing — and my own first instrument counted its own proof harness as +wiring five files it only read from a blob. And a bare basename is directory-local: +`verify_archive.py` exists in four sphinx archives, so a reference from another archive's +directory is a reference to *its* sibling. + +Of the 7 files with an invocation-shaped line, **4 are actual exec calls** — +`spark/verify_dirs.py` from `run_verify.sh`, `sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py` +from that archive's `CHECK.py` (twice) and `run_verdict.py`, +`sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py` and +`sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py` from their own `mutation_battery.py` +— and **3 are prose lines that spell a command**: a sibling docstring's `Run as:` line +(ledger/verify_arxiv_csv.py), a comment ("the sibling verify_github_python_repos.py re-queries +the API"), and a report string (`"`python3 verify_archive.py .` runs 8 checks"`). `--why` prints +all seven lines, so the figure is read rather than trusted. + +Held in the repo's established shape — a tolerance ledger, not a claim: + +* `wizard-work/volt/q63_unwired_baseline.tsv` — 80 rows: **8 `declared-one-off`**, **72 + `undeclared`**. A row is a stated tolerance, not an exemption: an unwired file the ledger does + not name is rc 1 the day it lands. +* The eight say so in-file, in three comment lines naming the exact grep that shows it + (beam/verify_toposort_none, forge/verify_append_only_audit_dry_refactor, + forge/verify_path_claim_apply_baseline_refactor, ledger/verify_github_repos, + sol/verify_sol_triage, sol/verify_troupe_sol_health, spark/verify_analyzer, + spark/verify_loc_report). +* The 72 undeclared are opened as **Q65** rather than annotated in bulk: 72 in-file statements + written in one tick would be paperwork, not a measurement, and each one needs a decision + (wire it, retire it, or declare it) that belongs to its own lane. + +## 4. The instrument that keeps the class closed + +`wizard-work/volt/q63_verify_scan.py` — read-only, git-blob-based, so a dirty worktree cannot +fake a result: + +``` +$ python3 wizard-work/volt/q63_verify_scan.py --repo . ; echo rc=$? + committed verify_*.py : 87 (spark 33 forge 23 ledger 13 sphinx 10 sol 5 null 2 beam 1) + stripped-assert : 0 of 87 + unwired (code-grep) : 80 of 87 + unwired, ledger-tolerated : 80 (q63_unwired_baseline.tsv: 8 declared one-off, 72 undeclared) + unwired, NEW (fault) : 0 +q63: 0 fault(s) +rc=0 + +$ python3 wizard-work/volt/q63_verify_scan.py --repo . --rev 89cda97 ; echo rc=$? + stripped-assert : 6 of 87 + STRIPPED_ASSERT wizard-work/beam/verify_toposort_none.py + ... (six named) +q63: 6 fault(s) +rc=1 + +$ python3 wizard-work/volt/q63_verify_scan.py --selftest + PASS exit-safe and wired is clean + PASS bare assert without an exit path is a fault + PASS unwired file the ledger omits is a fault + PASS unwired file the ledger names is tolerated +selftest: 4/4 +``` + +Exit codes follow the `tools/` contract this archive already runs on: 0 clean, 1 graded a fault, +2 could not grade. The scanner is **not** wired into `tools/verify.py` — that gate's legs are +shared property and adding one is the operator's call, not this lane's; the command above is +copy-paste ready for whoever makes it. + +## 5. Commits + +| commit | file | +|---|---| +| `80a2e36` | beam/verify_toposort_none.py — `_check`, plus the unwired statement | +| `be623cd` | ledger/verify_arxiv_csv.py | +| `76224a6` | spark/verify_report.py | +| `8a481da` | spark/verify_result.py | +| `2535c30` | spark/verify_scratch_analysis.py | +| `404445c` | spark/verify_dir_analyzer.py | +| `e37c584` … `c033059` | the seven other unwired files — in-file statement only | +| `c441079` | q63_verify_scan.py (first cut: mention-based wiring — superseded) | +| `5853fe4` | q63_unwired_baseline.tsv (73 rows — superseded) | +| `2a484e4` | q63_prove.py | +| `9cec6ca` | q63_verify_scan.py (invocation-based wiring, directory-scoped) | +| `6c26bc5` | q63_unwired_baseline.tsv (80 rows) | + +One file per commit, per the item; explicit pathspecs, so no other lane's dirty file rode along. +The two intermediate commits are in the history rather than rewritten out of it: the first +instrument's 73 was a measurement error, and a record that hides its own red middle is worth +less than one that shows it. + +## 6. What the item measured and what it did not + +* 88 → **87**. The per-lane breakdown already summed to 87. +* "6 of 88 with no explicit exit path" → **6 of 87 with no non-zero exit path**. The list was + right; the rule needed to be narrower (`exit(0)` is not an exit path). +* "8 of 88 unwired" → **8 of 87** under an all-file grep at `23f5a4a`, **0 of 87** by that same + grep one commit later (the record wired them in prose), **73 of 87** by code-mention, + **80 of 87** by invocation. The item's eight are a true subset; the other 72 were outside its + frame and are Q65. +* The `-O` hole remains latent: no `CHECK` and no `tools/` leg passes `-O` or sets + `PYTHONOPTIMIZE`, unchanged by this work and unchanged in risk by it.