From ac0e211ad19693ab45e8f9224a5fd90473cc8e3b Mon Sep 17 00:00:00 2001 From: Volt Date: Sun, 18 Oct 2026 04:55:56 +0000 Subject: [PATCH] tools (volt): doc_census_check -- the claim separator was a class, not an adjacency The full census at 4ce1aed (the run my 2026-10-18 check-in owed) reported two STALE claims against checkin_check.py:59 -- "read as UTF-8. Never 1, never a traceback." -- where the 8 is the tail of UTF-8 and the 1 is an exit code. Both are false defect claims. Measured over all 12 committed docstrings: 140 claims read, 95 real. The rest were welded hex (83acffe0 -> 83 acffe0, 11), ordinals/sentence ends (1. INTERNAL, 8. Never, 5), compound adjectives (120-char, 5), list markers (0 ran, 4) and trailing commas (1, which -> claim 1 of unit which, 8). Three rules: sep is one space or a closing delimiter + space; num is [0-9]+(?:,[0-9]{3})*, so a trailing comma is punctuation; ordinal_at() refuses a line-leading number followed by 2+ spaces. Six cells: 35 -> 41. Same revision, same docstrings, after: 145 -> 105 verdict rows, STALE 2 -> 0, NOT_PRINTED 87 -> 52, and the 26 MATCHES are the same 26 rows (lost: [], gained: []) -- no adjudicated verification was dropped. selftest 41/41 rc 0; selftest_sweep 12 tools 12 OK 0 fatal; read-refusal 17/17; malformed-input 15/15. Named, not fixed: doc_census_check returns 0 with STALE > 0 and has no --fail, so a gate cannot fail on the class it exists to catch. --- checkins/2026-10-18-volt.md | 106 ++++++++++++++++++++++++++++++++++++ tools/doc_census_check.py | 71 ++++++++++++++++++++---- 2 files changed, 165 insertions(+), 12 deletions(-) diff --git a/checkins/2026-10-18-volt.md b/checkins/2026-10-18-volt.md index 05ca369..e76aa19 100644 --- a/checkins/2026-10-18-volt.md +++ b/checkins/2026-10-18-volt.md @@ -234,3 +234,109 @@ only thing I need from Alexander). Nothing new. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-18-malformed-axis/` (`decision.md`, `selftest.txt`, `sweep-head.txt`, `head-copies.txt`, `ab-461eede.txt`, `discrimination.txt`). + +## Verdict, appended 04:55Z — the carried `doc_census_check` full run, and the two false defects it was reporting + +Reply 20 carried one item on my side: the full census my check-in promised to +append had not landed. It is run, at `4ce1aed`, and it found a defect in the +instrument that was running it. + +**The run.** 12 tracked tools, 145 verdict rows, 4m47s, `rc=0`: + + --- verdict totals (unit: claims) --- + AMBIGUOUS 11 + MATCHES 26 + NOT_PRINTED 87 + REV_UNCHECKED 8 + SELF_NOT_RUN 11 + STALE 2 + +**Both STALE rows are false, and both are in one sentence.** The single-tool +view is the indictment: + + $ python3 tools/doc_census_check.py --repo . --tools tools/checkin_check.py + tools/checkin_check.py: 0 INTERNAL SPLIT, 2 STALE claim(s), 0 leg failure(s), 13 claims checked + STALE tools/checkin_check.py:59 Never claimed 8 at HEAD -- printed 2 [selftest] + STALE tools/checkin_check.py:59 never claimed 1 at HEAD -- printed 2 [selftest] + +Line 59 is `read as UTF-8. Never 1, never a traceback.` — an exit-code sentence. +The `8` is the tail of `UTF-8`, paired across the sentence's period with the +word `Never`; the `1` is the code, paired with `never` across a comma. Neither +is a count of a unit. The oracle then found some two-count elsewhere in the +selftest output and graded both STALE: two defect claims against a docstring +that is correct, which is the failure mode this archive grades other lanes for. + +**The cause is the separator class, measured on the whole corpus, not reasoned +from the two rows.** The same 12 committed docstrings at `HEAD` were read by the +old parser and the new one (`vocab-before.txt`, `vocab-after.txt`, same code, +`git show` corpus): + + before: 140 claims after: 104 claims + sep=' ' 95 sep=' ' 87 + sep='' 11 sep='(unit:value form)' 9 + sep='(unit:value form)' 9 sep='` ' 6 + sep='` ' 6 sep=' ' 1 + sep='-' 5 sep=') ' 1 + sep='. ' 5 + sep=' ' 4 + sep='; ' 2 sep=' " ' 1 sep='+ ' 1 sep=') ' 1 + +Only 95 of 140 were a count and its unit. The rest: a hex run welded to its +letters (`83acffe0` -> 83 `acffe0`, 11), an ordinal or a sentence end (`1. +INTERNAL`, `0. Usage`, `8. Never`, 5), a compound adjective (`120-char`, +`1-based`, 5), an aligned list marker (`0 ran`, 4), punctuation spills (5). + +**Fix — three rules, six cells.** `CLAIM.sep` is now one space, or a closing +delimiter followed by one space. `num` is `[0-9]+(?:,[0-9]{3})*`, so a trailing +comma is punctuation and not a thousands separator — the old class read +`1, which` as claim 1 of unit `which` and `680, refs` as claim 680, which is 8 +of the 40 removed verdict rows on its own. `ordinal_at()` refuses a line-leading number +followed by 2+ spaces, the exit-code block's `0 ran` / `1 ran` / `2 could`. +Six cells pin them (welded, ordinal, list marker, sentence end, compound +adjective, and a positive control that `471 UNRESOLVED` / `1,612 citations` +still read): 35 -> 41. + +**After, at the same revision and over the same docstrings:** + + $ python3 tools/doc_census_check.py --repo . --rev 4ce1aed + --- verdict totals (unit: claims) --- + AMBIGUOUS 11 + MATCHES 26 + NOT_PRINTED 52 + REV_UNCHECKED 8 + SELF_NOT_RUN 8 + + 145 -> 105 rows; STALE 2 -> 0; NOT_PRINTED 87 -> 52; SELF_NOT_RUN 11 -> 8 + +**The invariant that makes this a repair and not a blindfold:** the 26 MATCHES +are the same 26 rows, by tool, unit, value and verdict — + + $ python3 scratch/diff_census.py census-head.json census-head-tightened.json + MATCHES identical: True | lost: [] | gained: [] + +so the tightening dropped no adjudicated verification; it dropped only rows that +were never a count and a unit. `MATCHES` unchanged is the whole evidence claim. + +**Gates, re-run after the change:** `--selftest` 41 cells, 41 passed, 0 failed, +rc 0; `tools/selftest_sweep.py` 12 tools, 12 OK, 0 fatal (`doc_census_check.py +41 cells` claimed and printed), read-refusal 17/17, malformed-input 15/15, rc 0. +The sweep's advertised-count gate caught nothing because the usage line was +updated in the same commit, which is the point of it. + +**What the tightening does not reach, stated rather than implied.** Six claims +behind a backticked token (`` `-07` suffixes ``), one aligned mid-line +(`WORK-QUEUE.md:8003 cites`) and one `120) characters` are still read; none is +in a unit any tool prints, so all are NOT_PRINTED and none can become a STALE. +`NOT_PRINTED` at 52 remains the bulk of the verdict rows, which is this tool's +stated gap (§"WHAT THIS DOES NOT CHECK"), unchanged in kind. + +**A contract gap this run exposed, named not fixed.** `doc_census_check.py` +exits 0 while reporting STALE > 0 (measured above: rc 0 with two STALE rows); +it has no `--fail`, where `checkin_check.py` does. A gate cannot fail on the +class this tool exists to catch without parsing its text. That is the next +increment, not something to fold into this one silently. + +Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-18-doc-census/` +(`census-head.txt`/`.json` pre-fix, `census-head-tightened.txt`/`.json` +post-fix, `diff.txt` row-by-row, `vocab-before.txt`/`vocab-after.txt`, +`sweep-after.txt`). diff --git a/tools/doc_census_check.py b/tools/doc_census_check.py index 72777e1..6ad4f38 100644 --- a/tools/doc_census_check.py +++ b/tools/doc_census_check.py @@ -102,15 +102,16 @@ SCOPE, STATED RATHER THAN IMPLIED WHAT THIS DOES NOT CHECK, and the measured size of the gap. A claim is only adjudicated when the unit it names is a unit the tool PRINTS. Over the three - scanners at d355395 the run read ninety-two claims: twelve were verified - against the run they point at, eight were in units the tool prints with two - values (SECTION_SPLIT_BY_TOOL -- no oracle), and seventy-one were in units the - tool never prints at all. So the oracle's reach is narrow and its precision is - the reason: it reported exactly one bad claim on that corpus, and that one was - the returned defect. A wider rule was tried first and falsified by its own - first run -- num_agreement_check.py flagged a hundred and seventeen - restatements across the archive, too coarse to be read -- so the narrow rule - is the one that ships. Two claim forms are read (` ` and + scanners at d355395 the run read ninety-two claims (with the pre-2026-10-18 + separator rule, which read punctuation and list markers as claims): twelve + were verified against the run they point at, eight were in units the tool + prints with two values (SECTION_SPLIT_BY_TOOL -- no oracle), and seventy-one + were in units the tool never prints at all. So the oracle's reach is narrow + and its precision is the reason: it reported exactly one bad claim on that + corpus, and that one was the returned defect. A wider rule was tried first and + falsified by its own first run -- num_agreement_check.py flagged a hundred and + seventeen restatements across the archive, too coarse to be read -- so the + narrow rule is the one that ships. Two claim forms are read (` ` and `: `); a claim written in neither form is not seen, and that is a stated gap, not a silent one. @@ -123,7 +124,7 @@ SCOPE, STATED RATHER THAN IMPLIED USAGE python3 doc_census_check.py [--repo DIR] [--rev HEAD] [--tools NAME...] - [--json OUT] [--no-run] [--selftest] # selftest: 35 cells, one per claim class + [--json OUT] [--no-run] [--selftest] # selftest: 41 cells, one per claim class """ import argparse import ast @@ -142,11 +143,25 @@ DEFAULT_REPO = "/root/troupe-standup" # ` [: =] ` -- "ACS = 72", "overlap = 20" (markdown bold allowed) # A bare number is not a claim. Deliberately not "any word": `471 cannot be # resolved` is prose, and a checker that flags prose is a checker nobody reads. +# +# THE SEPARATOR IS SPACE, OR A CLOSING DELIMITER AND A SPACE -- tightened +# 2026-10-18 after the census was read off every tool at HEAD (140 claims) and +# the separator vocabulary was printed: `''` (11 claims, a hex run read as +# digits+word: `83acffe0` -> 83 `acffe0`), `'. '` (5, an ordinal or a sentence +# end: `1. INTERNAL`, `615. Cite`), `'-'` (5, a compound adjective: `120-char`, +# `1-based`), `' '` (4, an aligned list marker: `0 ran`), and the punctuation +# spills `'; '` `' "'` `'+ '` `') '`. Only 95 of the 140 were a count and its +# unit. The loose class fired two STALE verdicts in the full run at 4ce1aed +# against `checkin_check.py:59` -- "Never 1, never a traceback", where the digit +# is a code, not a census -- which is a false defect claim of exactly the kind +# this archive grades lanes for. A hex run is welded to its letters and has no +# separator; an ordinal's separator is a period; a compound adjective's is a +# hyphen. None of the three is ` `. CLAIM = re.compile( - r"(?[0-9][0-9,]*)(?P[^A-Za-z0-9\n]{0,2})" + r"(?[0-9]+(?:,[0-9]{3})*)(?P\s{1,2}|[)\]}\"'`*]\s)" r"(?P[A-Za-z_][A-Za-z0-9_]*|[A-Z][A-Z_]{2,})\b") CLAIM_EQ = re.compile( - r"(?P[A-Za-z_][A-Za-z0-9_]*)\s*[:=]\s*\**\s*(?P[0-9][0-9,]*)") + r"(?P[A-Za-z_][A-Za-z0-9_]*)\s*[:=]\s*\**\s*(?P[0-9]+(?:,[0-9]{3})*)") REV_LONG = re.compile(r"\b[0-9a-f]{7,40}\b") # A hex run introduced as a DIGEST is not a revision, and reading it as one costs @@ -321,6 +336,18 @@ def parse_printed(stdout): return {k: dict(v) for k, v in units.items()}, clone +def ordinal_at(raw, pos): + """True when `pos` is the number that OPENS this line and 2+ spaces follow. + + That is a list marker -- the exit-code block's `0 ran; nothing to report`, + `1 ran, and graded ...`, `2 could not grade` -- and the word after it is + prose, not a unit. The same shape mid-line (`WORK-QUEUE.md:8003 cites`) is + a citation's line number, not a count, and is excluded by the same rule. + """ + m = re.match(r"[ \t]*([0-9][0-9,]*)([ \t]{2,})", raw) + return bool(m) and m.start(1) == pos + + def claims_in(blk): """[(line, value, unit, token, offset)] for one block. @@ -336,6 +363,8 @@ def claims_in(blk): u = m.group("unit") if len(u) < 3 or u.lower() in STOPWORDS or re.fullmatch(r"[qQ]\d+", u): continue + if ordinal_at(raw, m.start()): + continue found[m.start()] = (int(m.group("num").replace(",", "")), u, m.group(0)) for m in CLAIM_EQ.finditer(raw): u = m.group("unit") @@ -486,6 +515,24 @@ def _fixture_checks(): [("UNRESOLVED", 471), ("UNRESOLVED", 441)]) check("claims: `471 cannot be resolved` is not a claim", any(u in ("cannot", "resolved") for u, _ in got), False) + # the tightened separator, one cell per form it now refuses, plus the + # positive control -- a rule that refuses everything proves nothing + def claims_of(text): + return [(u, v) for _ln, v, u, _t, _p in claims_in([(1, text)])] + + check("claims: a welded digit-and-letter run is not a claim", + claims_of("sha 83acffe0 / rev 9188dd3 at HEAD"), []) + check("claims: an ordinal is not a claim", + claims_of("1. INTERNAL SPLIT -- no oracle, no run"), []) + check("claims: a list marker is not a claim", + claims_of("0 ran; nothing to report"), []) + check("claims: a sentence end is not a claim", + claims_of("read as UTF-8. Never 1, never a traceback"), []) + check("claims: a compound adjective is not a claim", + claims_of("a 120-char window and a 1-based index"), []) + check("claims: the tightening keeps a count and its unit", + claims_of("471 UNRESOLVED and 1,612 citations"), + [("UNRESOLVED", 471), ("citations", 1612)]) check("claims: the block's rev is read", "babdadc" in revs_seen, True) # the digest rule, one signal per cell, and the negative control between # them -- the text of each is the text of the defect, measured at HEAD