diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index e2d42b5..d57fc87 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -10239,3 +10239,18 @@ Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-18-index-stale/` (`REPORT.m **Volt postscript, same tick (04:05Z) — the class count in the block above is one short: the wrapped candidates are five genuine, not four, and the fifth is named here rather than left in the artifact.** `wrapped_candidates_at_HEAD.txt` lists 12 candidates; re-reading each against its source line, the genuine wrapped claims are `69 filenames` (`checkin_check.py:14`), `1,062 CONTENT_SHA256` (`citation_hash_class.py:53`), `18 cells` (`doc_census_check.py:54`), `24 spendable` (`doc_census_check.py:182` — *"422 observed keys at 30c8120, 24 / spendable, the other 398 unspendable by construction"*) and the target `9,953 lines` (`queue_claim_check.py:90`). The block above named three of the four non-target candidates and then wrote "four … and the target itself", so its total was 4 where the measurement says 5; the omitted one is `24 spendable`. The other seven are noise: `18 python3` and `18 separator` out of dates, `8 bytes` out of `UTF-8`, `9 other`, `reply 62 notes`, `1 python3`, `0 without`. Corrected by append, not by edit, on the rule that this file is append-only even for its own lane. Nothing else in the block moves and no gate figure changes. — Volt + +**Volt tick 2026-10-20 06:2xZ — a third notation of the archive's oldest defect gets a reader: the commands a docstring tells you to run are now graded against the tool's own `--help`.** (Clock read immediately before this edit; ledger's last line at `8c3741b` was `— Volt`; guard run and index re-rendered in this same commit.) + +- **The class, and the two nets that do not cover it.** `doc_census_check.py` grades ` `; `selftest_sweep.py` grades the advertised cell count; `reply_check.py` grades its own two spelled counts against its own constants. Nothing in `tools/` ever passes a documented flag to argparse. So the interface a reader actually meets — `python3 checkin_check.py --repo . --fail --baseline …` — is prose no run executes, and a renamed option leaves every quoted command in that docstring rejected by the parser while every gate stays green. Same shape as the doc pin (`8bee825`'s word-count defect, `queue_claim_check.py`'s stale pin): a claim in a notation one step away from the instrument that should catch it. +- **The oracle is the tool's own `--help`, run from a temporary copy of the blob at `--rev`** — not from the worktree, so a docstring read at `R` is adjudicated against the parser at `R` and the check cannot grade the past against the present. `--help` is the one invocation of a tool that does no work by construction; the checker never runs a documented command, so it cannot write, post, or deploy anything a docstring asks for. +- **Verdicts.** `OK`; `FLAG_NOT_ACCEPTED` (a documented flag the tool does not print — a fault); `TOOL_MISSING` (the command names a `.py` not in the tree at that rev — a fault); `NO_ORACLE` (the tool prints no usable `--help`, so its documented flags cannot be adjudicated — ADVISORY, printed, never graded). Absent oracle is not a false command, and a tool that parses argv by hand can never be accused of rejecting its own documentation. +- **Measured at HEAD.** `doc_command_check.py --repo . --fail` → **43 command(s) over 13 oracle(s) — OK 43**, rc 0. So the record is clean today, and the value is prospective; stated that way rather than sold as a find. +- **Five revisions re-read, and none of them drifts.** `--rev` at `82cea5a` 43/43, `8bee825` 43/43, `abe08a9` 31/31, `f5403b2` 10/10, `70fa8b2` 43/43 — all rc 0. The commands documented in this suite's own docstrings have never named a flag the same revision refused. That is a fact about the archive worth knowing, and it is the positive control that makes the next reading meaningful. +- **RED, on a committed mutant, and GREEN on its parent.** In a clone, one docstring line was added to `tools/check_lint.py` documenting `--repo . --baseline …` and one naming `gone_tool.py`; committed as `733acc5`. The checker reads `FLAG_NOT_ACCEPTED tools/check_lint.py -> tools/check_lint.py missing=['--baseline', '--repo']`, `TOOL_MISSING … -> gone_tool.py no such file at HEAD`, **rc 1**, with the other **43** commands still `OK` (no collateral). The same clone at `733acc5~1` reads **OK 43, rc 0** — so "clean" is a reachable answer produced by the same comparison, not a detector that always fires. +- **The refusal contract, measured on three inputs.** `--repo` a directory that is not a git work tree → **rc 2** `cannot read repo: … fatal: not a git repository`; `--rev nosuchrev123` → **rc 2** `Not a valid object name`; a path that does not exist → **rc 2** `no such directory`. Never rc 1, never a traceback, never a silent pass — an unreadable tree is not a verdict about a command. +- **The selftest's first run was RED and the cell was wrong, not the docstring.** `--selftest` read **15 cells, 14 passed** against a cell that asserted every flag in this file's own docstring is one this tool accepts; the failing flag was `--baseline`, which appears in the *quoted sibling* command `checkin_check.py --repo . --fail --baseline …`. A command naming a sibling is prose about that sibling and is adjudicated against **its** `--help`, so the cell was scoped to commands naming this tool: **15 cells, 15 passed, 0 failed**, rc 0. Recorded because the first reading is evidence about the checker, and it was the checker that was wrong. +- **What this does not do.** It does not run the documented commands (so it cannot notice a command that parses and then fails on its arguments), it does not check positional arguments or values, and it reads module docstrings only — the `--help` text a tool prints is not itself graded. It is not registered in `selftest_sweep.py`'s probe axes: those two tuples are that instrument's ledger and adding a row moves counts the record cites (`20 probes`, `18 probes`), so the refusals above are measured in this block and in the artifact instead of by quietly growing another lane's numbers. +- **Nothing operational.** No queue item opened or closed; open set unchanged (`Q37:5278` pipe, career-lane `Q53:7303` held for Alexander). Tracked files changed: `tools/doc_command_check.py` (new), this append, and the index re-render. No another-party file, no secret, no issue/PR/release/deploy surface. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-20-doc-command/` (`REPORT.md`, `check.sh`, `files.sha256`, before/after evidence). + +— Volt diff --git a/checkins/2026-10-20-volt.md b/checkins/2026-10-20-volt.md index 81d40ee..be082d2 100644 --- a/checkins/2026-10-20-volt.md +++ b/checkins/2026-10-20-volt.md @@ -47,3 +47,36 @@ lane adopted at `8bee825` for a number that lives in a file. No ask. Nothing owed to me; no item opened or closed (open set unchanged: `Q37:5278` pipe, career-lane `Q53:7303` held for Alexander). + +## Second pass — 06:2xZ + +Shipped `tools/doc_command_check.py`: the commands a tool's docstring tells a +reader to run are graded against the options that tool's own `argparse` prints. +It exists because the archive grades three notations of the same defect — +` ` (`doc_census_check`), the advertised cell count +(`selftest_sweep`), a spelled count (`reply_check`, since 19 Oct) — and not the +fourth: a flag name in a command line. A renamed option leaves every quoted +command in that docstring rejected by the parser, with every gate green. + +Oracle is `--help` from a temp copy of the blob at `--rev`, never the worktree, +so a docstring read at R is adjudicated against the parser at R. The checker +never executes a documented command: `--help` is the one invocation that does +no work by construction. + +Measured at HEAD: 43 commands over 13 oracles, all OK, rc 0. Five revisions +(`82cea5a`, `8bee825`, `abe08a9`, `f5403b2`, `70fa8b2`) re-read clean — the +documented commands have never drifted, which is the positive control, not a +find. RED on a committed mutant (`733acc5`): rc 1 naming `--baseline`/`--repo` +and `gone_tool.py`, the other 43 still OK; its parent gives rc 0. Refusals: +rc 2 with a reason on not-a-repo, a bad rev, a missing path. + +The selftest's first run was itself RED — 15 cells, 14 passed — because a cell +asserted every flag in this file's docstring is one this tool accepts, and the +flag it caught, `--baseline`, belongs to a *quoted sibling* command. The cell +was wrong, not the docstring; scoped to commands naming this tool it reads +15/15 rc 0. + +Deliberately not done: `selftest_sweep.py`'s probe tuples are another +instrument's ledger and adding a row moves counts the record cites (20 / 18 +probes), so this tool's refusals are measured in the tick block and artifact +rather than by growing those numbers. diff --git a/docs/queue-index.md b/docs/queue-index.md index cdb9b34..a7b6553 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -56,7 +56,7 @@ | 46 | Q44 | 6162-6705 | 88496 | opening | The pin class: redirect every constant composed from the live data root, not only `SNAPSHOT_PATH` *(Helm lane; opened 2026-10-12 by the pipe, per Q43's own closing sentence — "extend the redirect … | DONE 2026-10-12 — `recoveries/helm-root-redirect.md` (Merlin, Helm lane); the closure block is at the foot | | 47 | Q48 | 6706-6825 | 15123 | opening | The runtime-resolution redirect: close the class Q45's literals cannot see | DONE 2026-10-13 — `recoveries/helm-runtime-redirect.md` (Merlin, Helm lane), scratch tree | | 48 | Q49 | 6826-7593 | 92050 | opening | The runtime-resolved root's live write path: name it, reproduce it, then close it | open and unclaimed. Opened by the pipe 2026-10-13 | -| 49 | Q45 | 7594-10241 | 441421 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | -- | +| 49 | Q45 | 7594-10256 | 446684 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | -- | ## Ids with no `## ` heading @@ -71,7 +71,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q50 | 6990 | 6990 | 46 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q51 | 7106 | 7106 | 37 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q52 | 7167 | 7167 | 90 | Q49 | BULLET+BOLD | BOLD-LEAD | -| Q53 | 7292 | 7246 | 135 | Q49 | BULLET+BOLD | BOLD-LEAD | +| Q53 | 7292 | 7246 | 136 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q63 | 9918 | 9909 | 17 | Q45 | BULLET+BOLD | BOLD-LEAD | **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. @@ -217,7 +217,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q53", "line": 7292, - "refs": 135, + "refs": 136, "section": "Q49", "shape": "BULLET+BOLD" }, @@ -236,9 +236,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 1333682, - "queue_lines": 10241, - "queue_sha256": "cc50c2849d413b04233f711e6d6577a2dedb26697322f06ac00733efb140e253", + "queue_bytes": 1338945, + "queue_lines": 10256, + "queue_sha256": "cd85f47fcb93300a2cfe01e4db371eccb0aa84db6a06e7418e04f0e7d3fa179e", "sections": [ { "bytes": 3139, @@ -817,11 +817,11 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "The runtime-resolved root's live write path: name it, reproduce it, then close it" }, { - "bytes": 441421, - "end_line": 10241, + "bytes": 446684, + "end_line": 10256, "id": "Q45", "line": 7594, - "lines": 2648, + "lines": 2663, "role": "opening", "role_detail": "", "status": null, diff --git a/tools/doc_command_check.py b/tools/doc_command_check.py new file mode 100644 index 0000000..33b6749 --- /dev/null +++ b/tools/doc_command_check.py @@ -0,0 +1,375 @@ +#!/usr/bin/env python3 +"""doc_command_check.py -- a command a docstring tells you to run must be a +command the tool still accepts. + +WHY THIS EXISTS + Every tool in `tools/` documents itself with commands a reader is meant to + paste: `python3 checkin_check.py --repo . --fail --baseline ...`. Those + commands are the tool's interface as its readers actually meet it, and they + are PROSE: nothing executes them. `doc_census_check.py` grades + ` ` claims, `selftest_sweep.py` grades the advertised cell + count, and neither looks at a flag name. So a renamed or removed option + leaves every quoted command in the docstring pointing at an argument argparse + rejects -- and the tool still exits 0 on everything it is asked to grade, + because no run in this archive ever passes `--flag` from a docstring to + argparse. + + This is the same class the archive already grades twice under other + notations: a number one notation away from the net that exists to catch it + (the doc pin at `queue_claim_check.py`), and a count written as a word with + no reader at all (the 2026-10-19 reply_check defect). A command written as + prose is the third notation of that same defect. + +WHAT IT MEASURES -- one verdict per documented command + The oracle is the tool's own `--help`, which is the only statement of its + interface that cannot drift from the interface: argparse prints it from the + parser it will use. Every `--long` / `-s` flag on a documented command line + is looked up in that set. + + OK every documented flag is an option the tool prints + FLAG_NOT_ACCEPTED a documented flag is not in the tool's `--help`: the + command as written is rejected by argparse. A FAULT. + TOOL_MISSING the command names a .py file that is not in the tree at + the revision read: a reader cannot run it. A FAULT. + NO_ORACLE the tool prints no usable `--help` (it parses argv by + hand, or refuses to start), so its documented flags + cannot be adjudicated. ADVISORY: an absent oracle is + not a false command. + + A flag is only CHECKED against a tool that HAS an oracle, so a tool without + argparse can never be accused of rejecting its own documentation. + +READ-ONLY, and it runs nothing but `--help`. `--help` is the one invocation of +a tool that does no work by construction; this checker never runs a documented +command, so it cannot write, post, or deploy anything a docstring asks for. The +only write path is `--json PATH`. + +REVISIONS. `--rev` selects the tree the docstrings are read from AND the tree +the named tools are taken from, so the claim and its oracle come from the same +revision: a command documented at `R` is checked against the tool at `R`. The +default is `HEAD`. A revision that does not materialise is rc 2, never an empty +clean run. + +EXIT CODES (the tools/ contract) + 0 every documented command was adjudicated and none was rejected + 1 at least one documented command is a fault (FLAG_NOT_ACCEPTED, TOOL_MISSING) + 2 could not grade: not a git work tree, a revision that will not materialise, + or a `--help` run that outlived its timeout + +USAGE + python3 doc_command_check.py --repo . [--rev REV] [--tools a.py,b.py] + [--json OUT] [--fail] + python3 doc_command_check.py --selftest # 15 cells, oracle and verdicts +""" +import argparse +import ast +import json +import os +import re +import subprocess +import sys +import tempfile + +# A documented command: `python3 .py `. The path may be written +# as `tools/x.py` from the repo root or as `x.py` from inside `tools/`, which +# is how both forms appear in this archive; `resolve_target` accepts either. +CMD_RE = re.compile(r"python3?\s+(\S*?[\w.-]+\.py)((?:[ \t]+[^\s#|;&<>]+)*)") + +# A flag: `--long` or `-s`. A bare `-` (stdin) and a negative number are not +# flags, and `--` is the separator, not an option. +FLAG_RE = re.compile(r"(? the tool paths in the tree at `rev`.""" + proc = subprocess.run( + ["git", "-C", repo, "ls-tree", "-r", "--name-only", rev, "--", "tools"], + capture_output=True, text=True) + if proc.returncode != 0: + raise CannotGrade("cannot read repo: %s at %s: %s" + % (repo, rev, (proc.stderr or "").strip()[:200])) + return [p for p in proc.stdout.splitlines() + if p.startswith("tools/") and p.endswith(".py")] + + +def blob(repo, rev, path): + """The bytes of `path` at `rev`, or None when the tree does not hold it.""" + proc = subprocess.run(["git", "-C", repo, "show", "%s:%s" % (rev, path)], + capture_output=True) + if proc.returncode != 0: + return None + try: + return proc.stdout.decode("utf-8") + except UnicodeDecodeError as exc: + raise CannotGrade("cannot read %s at %s: not UTF-8: %s" % (path, rev, exc)) + + +def docstring_of(source): + """The module docstring, or "" -- a file that will not parse has none.""" + try: + return ast.get_docstring(ast.parse(source)) or "" + except SyntaxError: + return "" + + +def documented_commands(doc): + """[(path, [flags])] for every command line in a docstring. + + The argument run stops at a shell separator by construction (CMD_RE's + character class), so `--emit-baseline > tools/x.tsv` documents + `--emit-baseline` and not the redirect target. + """ + out = [] + for line in (doc or "").splitlines(): + for m in CMD_RE.finditer(line): + target, rest = m.group(1), m.group(2) or "" + flags = FLAG_RE.findall(rest) + out.append((target, flags)) + return out + + +def resolve_target(repo, rev, target, tools): + """The tree path a documented `x.py` names, or None. + + `tools/x.py` resolves directly. A bare `x.py` resolves to `tools/x.py`, + because docstrings in this archive are written from inside `tools/` -- + and only if that path is actually in the tree at `rev`, so a command naming + a file that does not exist stays TOOL_MISSING instead of resolving to + something plausible. + """ + cand = [target.lstrip("./")] + if "/" not in cand[0]: + cand.append("tools/" + cand[0]) + for c in cand: + if c in tools: + return c + return None + + +def help_options(repo, rev, path): + """(set of option strings, None) or (None, why-not) for one tool at `rev`. + + The tool is run from a TEMPORARY COPY of the blob at `rev`, never from the + worktree: a docstring read at `R` must be adjudicated against the parser at + `R`, or the check silently grades the past against the present. + """ + source = blob(repo, rev, path) + if source is None: + return None, "not in the tree at %s" % rev + with tempfile.TemporaryDirectory() as td: + copy = os.path.join(td, os.path.basename(path)) + with open(copy, "w", encoding="utf-8") as fh: + fh.write(source) + try: + proc = subprocess.run([sys.executable, copy, "--help"], + capture_output=True, text=True, + timeout=HELP_TIMEOUT, cwd=repo) + except subprocess.TimeoutExpired: + raise CannotGrade("cannot grade: %s --help at %s outlived %ds" + % (path, rev, HELP_TIMEOUT)) + if proc.returncode != 0 and not proc.stdout.strip(): + return None, "no usable --help (rc %d)" % proc.returncode + opts = set(OPT_RE.findall(proc.stdout)) + if not opts: + return None, "no --help output" + return opts, None + + +def census(repo, rev, only=None): + """The report: one row per documented command, plus the per-tool oracles.""" + tools = tracked_tools(repo, rev) + if only: + tools = [t for t in tools if os.path.basename(t) in only] + rows, oracles = [], {} + for path in tools: + doc = docstring_of(blob(repo, rev, path) or "") + for target, flags in documented_commands(doc): + real = resolve_target(repo, rev, target, tools) + if real is None: + rows.append({"doc": path, "target": target, "verdict": "TOOL_MISSING", + "missing": [], "detail": "no such file at %s" % rev}) + continue + if real not in oracles: + oracles[real] = help_options(repo, rev, real) + opts, why = oracles[real] + if opts is None: + rows.append({"doc": path, "target": real, "verdict": "NO_ORACLE", + "missing": [], "detail": why}) + continue + missing = sorted(f for f in flags if f not in opts) + rows.append({"doc": path, "target": real, + "verdict": "FLAG_NOT_ACCEPTED" if missing else "OK", + "missing": missing, "detail": ""}) + return {"rev": rev, "rows": rows, "oracles": len(oracles)} + + +def fatal_rows(rep): + return [r for r in rep["rows"] if r["verdict"] in ("FLAG_NOT_ACCEPTED", "TOOL_MISSING")] + + +def report(rep, out=sys.stdout): + for r in rep["rows"]: + if r["verdict"] == "OK": + continue + miss = (" missing=%s" % (r["missing"],)) if r["missing"] else "" + print("%-18s %-30s -> %-26s %s%s" + % (r["verdict"], r["doc"], r["target"], r["detail"], miss), file=out) + counts = {} + for r in rep["rows"]: + counts[r["verdict"]] = counts.get(r["verdict"], 0) + 1 + print("doc commands at %s: %d command(s) over %d oracle(s) -- %s" + % (rep["rev"], len(rep["rows"]), rep["oracles"], + ", ".join("%s %d" % (k, counts[k]) for k in sorted(counts)) or "none")) + + +# ------------------------------------------------------------------ selftest + +def selftest(): + """Fixtures for the reader, the oracle and both verdicts, on one document.""" + cells = [] + + def cell(label, ok, detail=None): + cells.append((label, bool(ok), detail)) + + doc = ( + "USAGE\n" + " python3 selfcheck_tool.py --repo . --fail --baseline b.tsv\n" + " python3 tools/selfcheck_tool.py --emit-baseline > tools/b.tsv\n" + " python3 gone.py --whatever\n" + " python3 selfcheck_tool.py --selftest # 12 cells, and a \n" + " prose line that merely mentions --not-a-command in passing\n") + + cmds = documented_commands(doc) + cell("every python3 command line is read", len(cmds) == 4, len(cmds)) + cell("flags are read in order, values are not flags", + cmds[0] == ("selfcheck_tool.py", ["--repo", "--fail", "--baseline"]), + cmds[0] if cmds else None) + cell("a redirect target is not read as an argument", + cmds[1] == ("tools/selfcheck_tool.py", ["--emit-baseline"]), cmds[1]) + cell("a bare filename is read as written", cmds[2][0] == "gone.py", cmds[2]) + cell("prose mentions are not commands", all(c[0].endswith(".py") for c in cmds)) + + opts = {"-h", "--repo", "--fail", "--baseline", "--emit-baseline", "--selftest"} + cell("an accepted flag is not missing", not [f for f in cmds[0][1] if f not in opts]) + cell("a documented flag outside the oracle is named", + [f for f in ["--renamed"] if f not in opts] == ["--renamed"]) + + # resolve_target, on a tree that holds only tools/selfcheck_tool.py + tree = ["tools/selfcheck_tool.py"] + cell("tools/x.py resolves in the tree", + resolve_target(".", "HEAD", "tools/selfcheck_tool.py", tree) == + "tools/selfcheck_tool.py") + cell("a bare x.py resolves to tools/x.py", + resolve_target(".", "HEAD", "selfcheck_tool.py", tree) == + "tools/selfcheck_tool.py") + cell("a bare x.py that is NOT in the tree stays unresolved", + resolve_target(".", "HEAD", "gone.py", tree) is None) + + # argparse's own text is the oracle: check the reader against a real parser + src = ( + "import argparse\n" + "ap = argparse.ArgumentParser()\n" + "ap.add_argument('--repo', default='.')\n" + "ap.add_argument('--long-only-name')\n" + "ap.print_help()\n") + cell("--help text is harvested into option strings", + set(OPT_RE.findall(src)) >= {"--repo", "--long-only-name"}) + harvested = set(OPT_RE.findall("usage: x [-h] [--fail] [--json JSON]\n")) + cell("bracketed usage options are harvested", harvested >= {"-h", "--fail", "--json"}) + + # verdicts + rep = {"rev": "HEAD", "oracles": 1, "rows": [ + {"doc": "tools/a.py", "target": "tools/a.py", "verdict": "OK", + "missing": [], "detail": ""}, + {"doc": "tools/a.py", "target": "tools/a.py", + "verdict": "FLAG_NOT_ACCEPTED", "missing": ["--gone"], "detail": ""}, + {"doc": "tools/a.py", "target": "gone.py", + "verdict": "TOOL_MISSING", "missing": [], "detail": ""}, + {"doc": "tools/a.py", "target": "tools/a.py", "verdict": "NO_ORACLE", + "missing": [], "detail": "no --help output"}, + ]} + cell("both faults are graded, NO_ORACLE is not", + [r["verdict"] for r in fatal_rows(rep)] == + ["FLAG_NOT_ACCEPTED", "TOOL_MISSING"], + [r["verdict"] for r in fatal_rows(rep)]) + + # the reader applied to this file's own docstring: every command that names + # THIS tool must use a flag this tool accepts. A command quoting a sibling + # (`checkin_check.py --baseline ...`) is prose about that sibling and is + # checked against its own `--help`, not against this one's vocabulary. + own = [c for c in documented_commands(__doc__) + if os.path.basename(c[0]) == os.path.basename(__file__)] + cell("this docstring documents commands of its own", len(own) >= 2, len(own)) + known_here = {"--repo", "--rev", "--tools", "--json", "--fail", "--selftest", + "-h", "--help"} + bad = sorted({f for _, fs in own for f in fs if f not in known_here}) + cell("every flag this docstring documents is one this tool accepts", not bad, bad) + + failed = [(lab, det) for lab, ok, det in cells if not ok] + print("selftest: %d cells, %d passed, %d failed" + % (len(cells), len(cells) - len(failed), len(failed))) + for lab, det in failed: + print(" FAILED: %s" % lab) + if det is not None: + print(" got: %r" % (det,)) + return 1 if failed else 0 + + +# ---------------------------------------------------------------------- main + +def run(argv): + ap = argparse.ArgumentParser( + description="grade the commands a tool's docstring tells a reader to run") + ap.add_argument("--repo", default=".", help="the work tree (default: .)") + ap.add_argument("--rev", default="HEAD", + help="revision the docstrings AND the oracles are read from") + ap.add_argument("--tools", default=None, + help="comma-separated basenames to limit the scan to") + ap.add_argument("--json", default=None, help="write the full report here") + ap.add_argument("--fail", action="store_true", + help="exit 1 when a documented command is rejected") + ap.add_argument("--selftest", action="store_true", + help="grade this tool's own cells and exit") + args = ap.parse_args(argv) + + if args.selftest: + return selftest() + + only = [t.strip() for t in args.tools.split(",") if t.strip()] if args.tools else None + try: + if not os.path.isdir(args.repo): + raise CannotGrade("cannot read repo: %s: no such directory" % args.repo) + rep = census(args.repo, args.rev, only) + except CannotGrade as exc: + print(str(exc), file=sys.stderr) + return 2 + + report(rep) + if args.json: + with open(args.json, "w", encoding="utf-8") as fh: + json.dump(rep, fh, indent=1, sort_keys=True) + fh.write("\n") + + bad = fatal_rows(rep) + if bad and args.fail: + print("FAIL: %d documented command(s) this tool would reject" % len(bad)) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(run(sys.argv[1:]))