stackchain/troupe-standup
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tools (volt): the docstring-vs-printed-census checker, and the defect it was built to catch fixed

Browse Source 
tools/doc_census_check.py (548 L / 25,058 B / sha256 dbbbdc01bb19dbcbbe87141c90a80e0bff19c98557c80f91c81347c53c98f1c5) checks a scanner's module docstring against the census that scanner PRINTS, two independent ways: (1) INTERNAL SPLIT -- one unit, one revision scope, two values in one paragraph, caught with no subprocess at all; (2) ORACLE -- the tool is re-run at the revision its docstring names and every '<value> <unit>' claim is compared with the value that run prints (MATCHES / STALE / CLONE_LOCAL / AMBIGUOUS / NOT_PRINTED). A unit the tool prints with two values is SECTION_SPLIT_BY_TOOL and no claim is made about it; a unit the tool never prints is NOT_A_CENSUS_UNIT and no claim is made about it either. Selftest 14 fixtures, 14 passed, rc 0; the only write path is --json.

THE DEFECT IT CATCHES, AND THE FIX: the control plane returned it against this lane's own resolver (reply 04, d355395) -- tools/citation_resolve_check.py:82 stated 471 UNRESOLVED while :89 stated 441 UNRESOLVED, over a census the tool prints as 471. That is the first finding of the new tool on its first run (INTERNAL SPLIT UNRESOLVED [babdadc] 471@82 / 441@89; STALE UNRESOLVED claimed 441 at babdadc, printed 471). HERE IT IS FIXED, my file and my edit: 2 insertions / 2 deletions, both on the one line; byte count unchanged at 20,460 B because 441 and 471 are the same width; sha256 7d063b39e2e4f9e5691b720cb57895b29450ece77051a7db6a8584dfd7173237 -> 40314b34f9860c4c42623f669c1339ed1d7b0ffc5fe604d56d6e9112b893e3a6. The fix is docstring-only and changes no measurement: the tool's own selftest is 11 fixtures / 11 passed / rc 0 both ways, and the census at babdadc is byte-for-byte the same after the edit as before it (297 files; EXPLICIT 410 = 113 RESOLVES / 0 STALE_LINE / 297 EXTERNAL_PATH; BARE 1612 = 119 AS_NAMED / 716 AS_SELF / 51 AMBIGUOUS / 255 EXTERNAL_NAMED / 471 UNRESOLVED; 6,961 mentions; 101 distinct targets).

MEASURED COVERAGE, stated narrow rather than rounded up: at d355395 the run read 92 claims across the three scanners -- 12 verified against the run they cite, 8 in units the tool prints with two values (no oracle), 71 in units the tool never prints. The oracle reported exactly one bad claim on that corpus, and it was the returned defect. A WIDER RULE WAS TRIED FIRST AND FALSIFIED BY ITS OWN FIRST RUN: tools/num_agreement_check.py, block-scope same-unit-two-values over the whole archive, flagged 447 restatements at first and 117 after an isolation rule (240 chars) -- too coarse to be read, and it is NOT landed; the file is kept out of the tree as evidence. Read-only in fact: only write path is --json, git status --porcelain clean after every run. No other file touched, no secret printed. Volt <volt@agents.invalid>.
This commit is contained in:
timmy committed 2026-10-16 06:34:08 +00:00
1 parent d355395c54
commit f5403b2510
2 files changed
+550 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tools/citation_resolve_check.py
+2 -2
View File
@@ -86,8 +86,8 @@ THE CENSUS AT HEAD, WITH ITS UNITS (measured 2026-10-16, rev babdadc, 297 .md)
number this tool exists to make checkable, and it is the honest number this tool exists to make checkable, and it is the honest
form of the ledger's own "a token quoted without its class". form of the ledger's own "a token quoted without its class".
mentions 6961 file mentions carrying no line -- these seed the scope. mentions 6961 file mentions carrying no line -- these seed the scope.
Read every figure as a citation count, never as a defect count: 441 UNRESOLVED Read every figure as a citation count, never as a defect count: 471 UNRESOLVED
is not 441 errors. The tool's own catch is the worked example below. is not 471 errors. The tool's own catch is the worked example below.
WORKED CATCH -- this tool's first finding is one of MY OWN artifacts WORKED CATCH -- this tool's first finding is one of MY OWN artifacts
checkins/2026-10-15-volt.md:9 reads "the three open items belong to other lanes checkins/2026-10-15-volt.md:9 reads "the three open items belong to other lanes
tools/doc_census_check.py
+548
View File
@@ -0,0 +1,548 @@
#!/usr/bin/env python3
"""doc_census_check.py -- does a tool's own docstring census match the census
that tool PRINTS?
WHY THIS EXISTS
This archive's three scanners each carry a census table in their module
docstring: counts that a reader cites, that no command checks. The docstring
is prose; prose is not executed. On 2026-10-16 the control plane returned
exactly this defect against a sibling scanner, in its own words:
"The scanner's own docstring carries a number that does not match the
census it prints -- the exact class the tool exists to catch."
tools/citation_resolve_check.py said `471 UNRESOLVED` on one line and
`441 UNRESOLVED` nine lines later, over a census the tool prints as 471.
Nothing was checking the docstring against the output beside it. That is this
tool's whole job.
WHAT IT MEASURES (read-only: this tool only writes --json)
Two independent verdicts per tool, and they do not share evidence.
1. INTERNAL SPLIT -- no oracle, no run. Within one docstring, every numeric
claim of the form `<value> <unit>` is collected; two claims of ONE unit
that a single revision scope covers, carrying two different values, is a
split. This is the verdict that catches the 441/471 instance, and it is
why the tool is useful with no subprocess at all.
2. ORACLE -- the tool is RUN, at the revision its own docstring names, and
each claim is compared against the value that run prints for that unit.
MATCHES claim == printed value
STALE claim != printed value, and the printed unit is not
declared clone-local -- a defect claim
CLONE_LOCAL claim != printed value but the tool's own output marks
that unit clone-local ("blobs in THIS clone's local object
store"), so drift is the documented expectation, not a
defect. Reported, never counted as one.
AMBIGUOUS the tool prints that unit with 2+ values at that rev
(e.g. `citations` prints once per section), so there is no
single oracle value; both are printed and no claim is made
NOT_PRINTED the unit does not appear in this tool's output at all
A claim is `<digits> <unitword>` with the unit word ADJACENT, which is the
archive's own notation. A bare number is not a claim and is not checked.
SCOPE, STATED RATHER THAN IMPLIED
Only module docstrings of .py tools are scanned: a docstring is where a census
table lives. Only revisions NAMED IN THE BLOCK are used as oracles -- a block
naming no revision is checked against the tool's default (HEAD) and tagged, so
a stale rev is never silently attributed. A tool that cannot be run at a
revision (queue_claim_check.py has no --rev; the queue is materialised from
git and handed to it by --queue) is driven through a per-tool recipe declared
below, with a temp file that is removed.
WHAT THIS DOES NOT CHECK, and the measured size of the gap. A claim is only
adjudicated when the unit it names is a unit the tool PRINTS. Over the three
scanners at d355395 the run read ninety-two claims: twelve were verified
against the run they point at, eight were in units the tool prints with two
values (SECTION_SPLIT_BY_TOOL -- no oracle), and seventy-one were in units the
tool never prints at all. So the oracle's reach is narrow and its precision is
the reason: it reported exactly one bad claim on that corpus, and that one was
the returned defect. A wider rule was tried first and falsified by its own
first run -- num_agreement_check.py flagged a hundred and seventeen
restatements across the archive, too coarse to be read -- so the narrow rule
is the one that ships. Two claim forms are read (`<value> <unit>` and
`<unit>: <value>`); a claim written in neither form is not seen, and that is a
stated gap, not a silent one.
NO CENSUS OF THIS TOOL'S OWN VERDICTS APPEARS IN THIS DOCSTRING, deliberately:
this scanner scans itself, so a count of its own output written here would
have to agree with the output of the run that the same commit changes. That is
the limit citation_hash_class.py records in the same words -- a commit cannot
measure the file it introduces -- and here it is avoided rather than
discovered.
USAGE
python3 doc_census_check.py [--repo DIR] [--rev HEAD] [--tools NAME...]
[--json OUT] [--no-run] [--selftest]
"""
import argparse
import ast
import collections
import json
import os
import re
import subprocess
import sys
import tempfile
DEFAULT_REPO = "/root/troupe-standup"
# A claim, in either of the two forms this archive writes:
# `<value> <unit>` -- "471 UNRESOLVED", "1,612 citations"
# `<unit> [: =] <value>` -- "ACS = 72", "overlap = 20" (markdown bold allowed)
# A bare number is not a claim. Deliberately not "any word": `471 cannot be
# resolved` is prose, and a checker that flags prose is a checker nobody reads.
CLAIM = re.compile(
r"(?<![0-9A-Za-z_.])(?P<num>[0-9][0-9,]*)(?P<sep>[^A-Za-z0-9\n]{0,2})"
r"(?P<unit>[A-Za-z_][A-Za-z0-9_]*|[A-Z][A-Z_]{2,})\b")
CLAIM_EQ = re.compile(
r"(?P<unit>[A-Za-z_][A-Za-z0-9_]*)\s*[:=]\s*\**\s*(?P<num>[0-9][0-9,]*)")
REV_LONG = re.compile(r"\b[0-9a-f]{7,40}\b")
# Function words and units this tool does not adjudicate. `at` in "2026-10-15 at
# rev 9188dd3" is not a census quantity; neither is `B` (bytes differ by
# revision by design, and "1,201,550 B" is a size, not a count). Stated so the
# exclusion is a rule rather than a habit.
STOPWORDS = {
"at", "the", "of", "in", "is", "are", "was", "and", "or", "to", "for", "on",
"by", "as", "it", "its", "that", "this", "these", "those", "with", "from",
"no", "not", "one", "two", "three", "all", "any", "be", "been", "has",
"have", "had", "his", "her", "our", "your", "their", "but", "if", "then",
"than", "so", "up", "out", "off", "over", "into", "run", "runs", "may",
"can", "same", "each", "both", "per", "via", "b", "kb", "mb", "gb",
}
# How to run each tool at a revision. `queue_claim_check.py` takes no --rev: its
# subject is a FILE, so the file is materialised at the revision and handed to
# it. Any tool not listed here is run as `<script> --repo R --rev REV`.
def _recipe_queue(repo, rev, tmpdir):
dst = os.path.join(tmpdir, f"WORK-QUEUE.{rev}.md")
blob = subprocess.run(["git", "-C", repo, "show", f"{rev}:WORK-QUEUE.md"],
capture_output=True).stdout
with open(dst, "wb") as fh:
fh.write(blob)
return ["--queue", dst, "--census"]
RECIPES = {"queue_claim_check.py": _recipe_queue}
def git(repo, *a):
return subprocess.run(["git", "-C", repo, *a], capture_output=True).stdout
def docstring_span(path, text):
try:
tree = ast.parse(text)
except SyntaxError:
return None
if not tree.body or not isinstance(tree.body[0], ast.Expr):
return None
node = tree.body[0]
if not isinstance(node.value, ast.Constant) or not isinstance(node.value.value, str):
return None
return node.lineno, node.end_lineno
def blocks(lines, lo, hi):
"""Blank-line-delimited paragraphs -- the same scope the sibling scanners
use (citation_resolve_check.py for inheritance, num_agreement_check.py for
quantity agreement), so one paragraph is one scope in all three."""
start, buf = lo, []
for ln in range(lo, hi + 1):
raw = lines[ln - 1]
if raw.strip() == "":
if buf:
yield start, buf
buf, start = [], ln + 1
else:
buf.append((ln, raw))
if buf:
yield start, buf
# ------------------------------------------------- printed census -> oracle
def parse_printed(stdout):
"""stdout -> {unit: {value: [line, ...]}} plus the set of clone-local units.
Four line shapes are read, all of them the scanners' own:
` RESOLVES 113` indented label then count (family rows)
` citations: 410` label then count
`corpus: 307 files at ...` count then unit
`overlap A&B = 20 ; ...` inline assignments, split on the separators
A unit whose printed line (or the line above it, for a wrapped phrase) says
"clone" is clone-local: its count is expected to move with the clone.
"""
units = collections.defaultdict(lambda: collections.defaultdict(list))
clone = set()
lines = stdout.split("\n")
def add(unit, value, ln, cloneline):
u = unit.strip().lower()
if len(u) < 3 or not re.fullmatch(r"[a-z_][a-z0-9_]*", u) or u in STOPWORDS:
return
units[u][value].append(ln)
if "clone" in cloneline.lower():
clone.add(u)
for i, raw in enumerate(lines, 1):
ctx = raw + " " + (lines[i - 2] if i >= 2 else "")
m = re.match(r"^\s{2,}([A-Za-z_][A-Za-z0-9_]*)\s+([0-9][0-9,]*)\s*$", raw)
if m: # family row: label count
add(m.group(1), int(m.group(2).replace(",", "")), i, ctx)
continue
m = re.match(r"^\s*([A-Za-z_][A-Za-z0-9_]*)\s*[:=]\s*([0-9][0-9,]*)\b", raw)
if m: # label: count
add(m.group(1), int(m.group(2).replace(",", "")), i, ctx)
for m in re.finditer(r"([0-9][0-9,]*)\s+(?:[a-z]+ )?([A-Za-z_][A-Za-z0-9_]*)\b", raw):
add(m.group(2), int(m.group(1).replace(",", "")), i, ctx)
for seg in re.split(r"[;]", raw): # inline `X = N`
m = re.match(r"^\s*(.*?)\s*[:=]\s*([0-9][0-9,]*)\s*$", seg)
if not m or not m.group(1):
continue
words = re.findall(r"[A-Za-z_][A-Za-z0-9_]*", m.group(1))
if not words:
continue
val = int(m.group(2).replace(",", ""))
add("_".join(w.lower() for w in words), val, i, ctx) # whole phrase
add(words[-1], val, i, ctx) # the nearest label: `overlap A&B`
# the scanners' own unit declaration: `... 6961 (unit: occurrences)`.
# The unit is named AFTER its number, which no other rule reads.
for m in re.finditer(r"\(\s*unit\s*:\s*([A-Za-z_][A-Za-z0-9_]*)\s*\)", raw):
left = raw[:m.start()]
nums = re.findall(r"([0-9][0-9,]*)\s*$", left.rstrip())
if nums:
add(m.group(1), int(nums[-1].replace(",", "")), i, ctx)
return {k: dict(v) for k, v in units.items()}, clone
def claims_in(blk):
"""[(line, value, unit, token, offset)] for one block.
Both claim forms are read and a claim found by both is reported once. The
offset is measured on the block text so a RESTATEMENT (far apart) can be
told from an ENUMERATION of neighbouring sub-counts, the same
discrimination num_agreement_check.py applies.
"""
out, off = [], 0
for ln, raw in blk:
found = {}
for m in CLAIM.finditer(raw):
u = m.group("unit")
if len(u) < 3 or u.lower() in STOPWORDS or re.fullmatch(r"[qQ]\d+", u):
continue
found[m.start()] = (int(m.group("num").replace(",", "")), u, m.group(0))
for m in CLAIM_EQ.finditer(raw):
u = m.group("unit")
if (len(u) < 3 or u.lower() in STOPWORDS
or re.fullmatch(r"[qQ]\d+", u) or m.start() in found):
continue
if m.start() and re.match(r"[0-9A-Za-z_.]", raw[m.start() - 1]):
continue # `WORK-QUEUE.md:8586` is a citation
found[m.start()] = (int(m.group("num").replace(",", "")), u, m.group(0))
for pos in sorted(found):
val, u, tok = found[pos]
out.append((ln, val, u, tok, off + pos))
off += len(raw) + 1
return out
def is_label(unit):
"""An uppercase census label -- the vocabulary a split is checked in when
the tool has not been run (no oracle available)."""
return len(unit) >= 3 and re.fullmatch(r"[A-Z][A-Z_]*", unit) is not None
def keys_for(unit, printed):
"""Candidate oracle keys for a claimed unit word."""
cands = [unit, unit.lower(), unit.lower().rstrip("s"), unit.upper()]
seen, out = set(), []
for c in cands:
if c.lower() in printed and c.lower() not in seen:
seen.add(c.lower())
out.append(c.lower())
return out
def oracle(printed, clone, unit, value):
"""(verdict, detail) for one claim; `printed` is one revision's map."""
keys = keys_for(unit, printed)
vals = set()
for k in keys:
vals |= set(printed[k])
if not vals:
return "NOT_PRINTED", ""
if len(vals) > 1:
return "AMBIGUOUS", "printed: " + ", ".join(str(v) for v in sorted(vals))
printed_val = vals.pop()
if printed_val == value:
return "MATCHES", ""
if any(k in clone for k in keys):
return "CLONE_LOCAL", f"printed {printed_val}"
return "STALE", f"printed {printed_val}"
# --------------------------------------------------------------- self-test
# Synthetic stdout, so the parsers and the verdicts are executed, not asserted.
FAKE_STDOUT = """\
corpus: 297 markdown files at babdadc in /root/troupe-standup
--- EXPLICIT `<path>:<line>` -- a CLAIM (unit: citations) ---
citations: 410
RESOLVES 113
STALE_LINE 0
--- BARE `:<line>` -- a CANDIDATE, never a claim (unit: citations) ---
citations: 1612
UNRESOLVED 471
file mentions with no line: 6961 (unit: occurrences) -- these seed the scope
index: 871 blobs (unit: blobs in THIS clone's local object store), 680 commits
overlap A&B = 20 ; ACS-only = 52
"""
FAKE_DOC = """\
THE CENSUS AT HEAD, WITH ITS UNITS (measured 2026-10-16, rev babdadc, 297 .md)
BARE 1612 citations: 255 EXTERNAL_NAMED / 471 UNRESOLVED.
Read every figure as a citation count, never as a defect count: 441 UNRESOLVED
is not 441 errors.
A SECOND BLOCK, no revision named, quoting 471 UNRESOLVED and nothing else.
"""
def _fixture_checks():
printed, clone = parse_printed(FAKE_STDOUT)
ok = True
def check(name, got, want):
nonlocal ok
good = got == want
ok = ok and good
print(f"{'PASS' if good else 'FAIL'} {name}")
if not good:
print(f" got {got}")
print(f" want {want}")
check("parse: UNRESOLVED single", sorted(printed.get("unresolved", {})), [471])
check("parse: citations ambiguous", sorted(printed.get("citations", {})), [410, 1612])
check("parse: count-then-unit", sorted(printed.get("occurrences", {})), [6961])
check("parse: one word skipped", sorted(printed.get("files", {})), [297])
check("parse: clone-local marked", "blobs" in clone, True)
# the phrase rule: `ACS-only = 52` must NOT register as the unit `acs`
check("parse: inline phrase key", sorted(printed.get("acs_only", {})), [52])
lines = FAKE_DOC.split("\n")
got, first_block, revs_seen = [], [], set()
for first, blk in blocks(lines, 1, len(lines)):
revs = sorted(set(REV_LONG.findall("\n".join(r for _, r in blk))))
revs_seen |= set(revs)
for ln, val, unit, _tok, _pos in claims_in(blk):
got.append((unit, val))
if first == 1:
first_block.append((unit, val))
# the fixture that matters: one unit, one rev scope, two values -> a split
check("claims: both UNRESOLVED values in one block",
[uv for uv in first_block if uv[0] == "UNRESOLVED"],
[("UNRESOLVED", 471), ("UNRESOLVED", 441)])
check("claims: `471 cannot be resolved` is not a claim",
any(u in ("cannot", "resolved") for u, _ in got), False)
check("claims: the block's rev is read", "babdadc" in revs_seen, True)
check("oracle: 471 matches", oracle(printed, clone, "UNRESOLVED", 471),
("MATCHES", ""))
check("oracle: 441 is stale", oracle(printed, clone, "UNRESOLVED", 441),
("STALE", "printed 471"))
check("oracle: ambiguous unit makes no claim",
oracle(printed, clone, "citations", 1612),
("AMBIGUOUS", "printed: 410, 1612"))
check("oracle: clone-local is not a defect",
oracle(printed, clone, "blobs", 1234), ("CLONE_LOCAL", "printed 871"))
check("oracle: unprinted unit", oracle(printed, clone, "wombats", 3),
("NOT_PRINTED", ""))
return 0 if ok else 1
# ------------------------------------------------------------------- driver
def run_tool(repo, script, rev, tmpdir, timeout=600):
args = RECIPES.get(os.path.basename(script), None)
if args is None:
argv = [sys.executable, script, "--repo", repo, "--rev", rev]
else:
argv = [sys.executable, script] + args(repo, rev, tmpdir)
p = subprocess.run(argv, capture_output=True, timeout=timeout)
return p.stdout.decode("utf-8", "replace"), p.returncode
def unit_scope(unit, printed, clone, no_run):
"""Is a split in this unit checkable, and why not if it is not?
With the oracle (printed map) in hand a unit is checkable when the tool
prints it with exactly ONE value at the block's revision: the docstring and
the output then talk about one number. A unit the tool prints with 2+ values
is SECTION-SPLIT BY THE TOOL'S OWN DESIGN (`citations` prints once under
EXPLICIT and once under BARE), so a docstring that mirrors both is correct
and is reported as such, not flagged. A unit the tool never prints is not a
census unit here. Without a run, only uppercase census labels are checked.
"""
if no_run or printed is None:
return ("LABEL_ONLY" if is_label(unit) else "UNCHECKED_NO_ORACLE")
keys = keys_for(unit, printed)
vals = set()
for k in keys:
vals |= set(printed[k])
if not vals:
return "NOT_A_CENSUS_UNIT"
if len(vals) > 1:
return "SECTION_SPLIT_BY_TOOL"
return "CHECKABLE"
def check_tool(repo, script, name, cache, run=True, tmpdir=".", isolation=240):
text = open(script, encoding="utf-8", errors="replace").read()
span = docstring_span(script, text)
if span is None:
return None
lines = text.split("\n")
lo, hi = span
blk_claims = []
for first, blk in blocks(lines, lo, hi):
btext = "\n".join(raw for _, raw in blk)
revs = sorted(set(REV_LONG.findall(btext)))
cl = claims_in(blk)
if cl:
blk_claims.append((first, revs, cl, btext))
# ---- the oracle first: every revision the docstring names is run once.
printed_by_rev, verdicts, failures = {}, [], []
if run:
for first, revs, cl, _b in blk_claims:
for rev in (revs or ["HEAD"]):
key = (name, rev)
if key not in cache:
try:
out, rc = run_tool(repo, script, rev, tmpdir)
cache[key] = (parse_printed(out), rc, out)
except Exception as exc: # noqa: BLE001
cache[key] = (None, None, f"<run failed: {exc}>")
printed_by_rev.setdefault((first, rev), cache[key])
# 1. INTERNAL SPLIT: one unit, one revision scope, two values, far enough
# apart to be a restatement rather than a neighbouring sub-count.
splits, unchecked = [], collections.Counter()
by_unit = collections.defaultdict(list)
for first, revs, cl, _b in blk_claims:
scope = tuple(revs)
for ln, val, unit, _t, pos in cl:
by_unit[(unit.upper() if unit.isupper() else unit.lower(), scope)].append(
(first, ln, val, pos))
for (unit, scope), items in sorted(by_unit.items()):
vals = {v for _, _, v, _ in items}
if len(vals) < 2:
continue
# a restatement is a pair far apart; an enumeration is neighbours
far = any(abs(b[3] - a[3]) >= isolation
for a, b in [(x, y) for x in items for y in items]
if a[1] != b[1] and a[2] != b[2])
if not far:
continue
rev_for_oracle = (items[0][0], scope[0] if len(scope) == 1 else "HEAD")
entry = printed_by_rev.get(rev_for_oracle)
pr, pr_clone = (entry[0] if entry else None) or (None, None)
why = unit_scope(unit, pr, pr_clone, not run)
if why != "CHECKABLE" and why != "LABEL_ONLY":
unchecked[why] += 1
continue
splits.append({"unit": unit, "revs": list(scope),
"values": sorted(vals), "how": why,
"rows": [{"line": ln, "value": v, "block": b}
for b, ln, v, _ in items]})
# 2. ORACLE: every claim compared against what the tool prints.
if run:
for first, revs, cl, _b in blk_claims:
for rev in (revs or ["HEAD"]):
pr, rc, out = printed_by_rev[(first, rev)]
if pr is None:
failures.append({"file": name, "block": first, "rev": rev,
"unit": "-", "claim": None,
"verdict": "RUN_FAILED", "detail": str(out)[:90]})
continue
printed, clone = pr
for ln, val, unit, tok, _pos in cl:
v, detail = oracle(printed, clone, unit, val)
verdicts.append({"file": name, "block": first, "line": ln,
"rev": rev, "unit": unit, "claim": val,
"token": tok, "verdict": v, "detail": detail})
return {"tool": name, "splits": splits, "unchecked": dict(unchecked),
"failures": failures, "verdicts": verdicts,
"blocks_with_claims": len(blk_claims)}
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--repo", default=DEFAULT_REPO)
ap.add_argument("--rev", default="HEAD")
ap.add_argument("--tools", nargs="*")
ap.add_argument("--json")
ap.add_argument("--no-run", action="store_true",
help="internal-split verdicts only; no subprocess")
ap.add_argument("--selftest", action="store_true")
a = ap.parse_args()
if a.selftest:
return _fixture_checks()
store = git(a.repo, "ls-tree", "-r", "--name-only", a.rev)
tree = {p for p in store.decode("utf-8", "replace").split("\n") if p}
tools = a.tools or sorted(p for p in tree
if p.startswith("tools/") and p.endswith(".py"))
if not tools:
print(f"no tools found at {a.rev}", file=sys.stderr)
return 2
cache, reports = {}, []
with tempfile.TemporaryDirectory(prefix="doccensus-") as tmpdir:
for t in tools:
src = subprocess.run(["git", "-C", a.repo, "show", f"{a.rev}:{t}"],
capture_output=True).stdout
local = os.path.join(tmpdir, os.path.basename(t))
with open(local, "wb") as fh:
fh.write(src)
try:
rep = check_tool(a.repo, local, t, cache, run=not a.no_run,
tmpdir=tmpdir)
except subprocess.TimeoutExpired:
rep = {"tool": t, "splits": [], "blocks_with_claims": 0,
"verdicts": [{"file": t, "verdict": "TIMEOUT", "rev": "-",
"unit": "-", "claim": None, "block": 0}]}
if rep:
reports.append(rep)
print(f"tools scanned at {a.rev}: {len(reports)} "
f"({'internal-split only, label vocabulary' if a.no_run else 'docstring vs printed census'})")
tot = collections.Counter()
for rep in reports:
bad = rep["failures"] + [v for v in rep["verdicts"] if v["verdict"] == "STALE"]
print(f"\n{rep['tool']}: {len(rep['splits'])} INTERNAL SPLIT, "
f"{len(bad)} STALE/failed, {len(rep['verdicts'])} claims checked")
for s in rep["splits"]:
vals = ", ".join(f"{r['value']}@{r['line']}" for r in s["rows"])
scope = ",".join(s["revs"]) or "no rev named"
print(f" SPLIT {s['unit']} [{scope}] ({s['how']}): {vals}")
for k, c in sorted(rep["unchecked"].items()):
print(f" (not flagged: {c} split(s) in units {k})")
for v in bad:
print(f" {v['verdict']:10s} {v['file']}:{v.get('line', '-')} "
f"{v['unit']} claimed {v['claim']} at {v['rev']} -- {v['detail']}")
for k, c in collections.Counter(v["verdict"] for v in rep["verdicts"]).items():
tot[k] += c
print("\n--- verdict totals (unit: claims) ---")
for k in sorted(tot):
print(f" {k:12s} {tot[k]}")
if a.json:
with open(a.json, "w") as fh:
json.dump({"rev": a.rev, "repo": a.repo, "reports": reports}, fh, indent=1)
print(f"wrote {a.json}")
return 0
if __name__ == "__main__":
sys.exit(main())