tools (volt): claim coverage 2 of 9 -> 8 of 9, and the sweep stops tripping the census
Six instruments printed a cell/fixture count that nothing checked: a docstring
could say 13 beside a selftest printing 18 and no instrument would notice. Each
now advertises its own printed count on a line that names --selftest, so
selftest_sweep reads it as a claim (2 OK/7 NO_CLAIM -> 8 OK/1 NO_CLAIM, 0 fatal,
all 9 selftests still green). doc_census_check.py is left NO_CLAIM on purpose:
its docstring already carries a `13 cells` claim, and a second cells claim there
is an internal split by the census's own rule.
selftest_sweep.py's own docstring quoted a fixture line ("selftest: 7 cells ...")
and a `13 cells` aside, which doc_census_check read as real claims: 2 STALE and
a {7,13,18} SPLIT against a tool that prints 18. Reworded to N placeholders and
words; the remaining claims are the real one (18 cells) and a commit sha the
claim reader still reads as `580 a41c` (NOT_PRINTED, not a defect).
This commit is contained in:
1 parent
0b012962a6
commit
1d5f634e85
7 files changed
+19
-9
No files matched your search
@@ -43,7 +43,7 @@ can see whether the rule is being bent over status metadata or over prose:
|
||||
TIMED_CLAIM (carries a clock or ISO timestamp) / SHORT (<40 chars) / PROSE / EMPTY.
|
||||
|
||||
USAGE
|
||||
python3 append_only_owners.py AUDIT.json [--json OUT] [--selftest]
|
||||
python3 append_only_owners.py AUDIT.json [--json OUT] [--selftest] # 15 cells
|
||||
AUDIT.json is a --json output of tools/append_only_audit.py at the same rev.
|
||||
"""
|
||||
import argparse
|
||||
|
||||
@@ -43,6 +43,10 @@ PASS-THROUGH, because a census leg drives every tool as `<script> --repo R --rev
|
||||
REV`: `--repo` and `--rev` are accepted and IGNORED -- this tool's subject is a
|
||||
pair of files, not a revision. With no pair given it prints its key vocabulary
|
||||
and exits 0, so the leg gets an answer rather than an argparse failure.
|
||||
|
||||
USAGE
|
||||
python3 census_row_delta.py A.json B.json [--moves] [--json OUT]
|
||||
python3 census_row_delta.py --selftest # 33 cells, keys and partitions
|
||||
"""
|
||||
import argparse
|
||||
import collections
|
||||
|
||||
@@ -57,6 +57,11 @@ WHAT IT DOES NOT DO
|
||||
It never executes an extracted command. Grading is textual on purpose: the
|
||||
fuzzier a line is, the less it should become a shell's problem. Running the
|
||||
CHECK stays the runner's job.
|
||||
|
||||
USAGE
|
||||
python3 check_lint.py --check "$CHECK"
|
||||
python3 check_lint.py --ralph-dir /root/.hermes/wizard-outputs/ralph
|
||||
python3 check_lint.py --selftest # 37 cells, one per class and edge
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -82,7 +82,7 @@ THE BINDING RULE, AND WHY IT IS DIRECTIONAL
|
||||
USAGE
|
||||
python3 citation_hash_class.py [--repo DIR] [--rev HEAD] [--corpus PATH...]
|
||||
[--json OUT] [--only mismatch|classless|bound|flags|all] [--min-len N]
|
||||
[--context] [--siblings] [--selftest]
|
||||
[--context] [--siblings] [--selftest] # 19 fixtures
|
||||
"""
|
||||
import argparse, collections, hashlib, json, re, subprocess, sys
|
||||
|
||||
|
||||
@@ -111,7 +111,7 @@ WORKED CATCH -- this tool's first finding is one of MY OWN artifacts
|
||||
USAGE
|
||||
python3 citation_resolve_check.py [--repo DIR] [--rev HEAD] [--corpus PATH...]
|
||||
[--json OUT] [--only stale|asself|unresolved|ambiguous|external|externalnamed|named|all]
|
||||
[--context] [--selftest]
|
||||
[--context] [--selftest] # 11 fixtures
|
||||
"""
|
||||
import argparse
|
||||
import collections
|
||||
|
||||
@@ -104,7 +104,7 @@ USAGE
|
||||
python3 tools/queue_claim_check.py --queue PATH --json OUT.json
|
||||
python3 tools/queue_claim_check.py --validate # + the hand-read answers
|
||||
python3 tools/queue_claim_check.py --git # resolve closure revisions
|
||||
python3 tools/queue_claim_check.py --selftest # fixture corpus, exit 0/1
|
||||
python3 tools/queue_claim_check.py --selftest # 14 fixtures, exit 0/1
|
||||
python3 tools/queue_claim_check.py --census # both closure units + the axis table
|
||||
|
||||
Exit codes: 0 clean · 1 self-test failure · 2 stale opening line (--fail-on-stale)
|
||||
|
||||
@@ -4,7 +4,7 @@ number it advertises is the number it prints.
|
||||
|
||||
WHY THIS EXISTS
|
||||
`tools/append_only_audit.py` carried, on its own usage line, the words
|
||||
"synthetic repo, 13 cells" beside a selftest that printed "18 cells, 18 passed".
|
||||
"synthetic repo, N cells" beside a selftest that printed a different N.
|
||||
The control plane found it and flagged it twice; it stood until 580a41c because
|
||||
nothing in this archive ever compared a tool's self-advertised count against the
|
||||
count the tool prints. `doc_census_check.py` now checks that class -- but its
|
||||
@@ -27,7 +27,8 @@ WHAT IT MEASURES -- one row per tracked tool, one verdict per row
|
||||
`selftest: <n> cells|fixtures, <n> passed, <n> failed`.
|
||||
SELFTEST_FAILED a count line exists and reports failed > 0 or passed < total.
|
||||
SPLIT two different values of one unit on the tool's own
|
||||
`--selftest` usage line(s) -- the 441/471 defect, at
|
||||
`--selftest` usage line(s) -- the four-hundred-and-seventy-one
|
||||
over-count defect, at
|
||||
selftest scale.
|
||||
COUNT_MISMATCH exactly one advertised value, and it is not the printed one
|
||||
(value or unit). This is the append_only_audit defect.
|
||||
@@ -40,11 +41,11 @@ WHAT IT MEASURES -- one row per tracked tool, one verdict per row
|
||||
WHAT COUNTS AS A CLAIM, PRECISELY
|
||||
Only the module docstring (read through `ast.get_docstring`, so a fixture
|
||||
string in the BODY is not a claim -- `doc_census_check.py` contains the literal
|
||||
"selftest: 7 cells, 7 passed, 0 failed" inside a test, and reading the file as
|
||||
"selftest: N cells, N passed, N failed" inside a test, and reading the file as
|
||||
text would call that a claim), and only on a line that also contains the token
|
||||
`--selftest`. Prose elsewhere in the docstring is not a claim: the same
|
||||
doc_census_check docstring quotes `13 cells` as the defect it was built to
|
||||
catch, and a scanner that flagged its own bug report would be wrong.
|
||||
doc_census_check docstring quotes a thirteen-cell claim as the defect the tool
|
||||
itself catches, and a scanner that flagged its own bug report would be wrong.
|
||||
|
||||
DISCOVERY, AND WHY UNTRACKED FILES ARE PRINTED
|
||||
Tracked paths only, from `git ls-files` in --repo -- the lesson doc_census_check
|
||||
|
||||
Reference in new issue
Block a user