From e64a40dbeccfca73a934d24a2c6bc4416f93e999 Mon Sep 17 00:00:00 2001 From: Volt Date: Thu, 22 Oct 2026 14:42:02 +0000 Subject: [PATCH] volt: count pin -- every selftest count witnessed by a committed ledger selftest_sweep compared a tool's printed count with a number in the same file, so a commit that deleted fixtures and decremented the docstring agreed with itself: doc_command_check.py cut 15 cells -> 12 read clean (sweep: 21 tools, 21 OK, 0 fatal, rc 0). tools/selftest_baseline.tsv is the second witness, read from disk: PIN_DRIFT/PIN_UNPINNED/PIN_ORPHAN fatal, rc 1; a missing ledger is rc 2, never clean; --no-baseline and --tools-dir say out loud that no pin was graded. +283/-4; selftest 30 -> 39 cells, 39/39. --- WORK-QUEUE.md | 24 +++ docs/queue-index.md | 22 +-- tools/selftest_baseline.tsv | 32 ++++ tools/selftest_sweep.py | 287 +++++++++++++++++++++++++++++++++++- 4 files changed, 350 insertions(+), 15 deletions(-) create mode 100644 tools/selftest_baseline.tsv diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index a181abb..a4e093b 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -10604,3 +10604,27 @@ Verification, read at `6cd2114` (the revision this record was written against; a 7. **What it does not claim.** `DOCREL` is a fallback, so a genuinely broken root-relative citation is still `MISSING` unless a same-named tracked file happens to sit beside the citing document; where that coincidence occurs the resolved path is printed, and the tool does not adjudicate intent. The second base does not reach a queue line that quotes *another artifact's* relative path from the repo root — that stays visible as an unresolved cite inside a `DOCREL` row, and the prose repair would be an append, not an edit to another lane's text. `path_claim_check.py` is not a `verify.py` leg, so no gate changed meaning here and no seventh leg was added. No Q-item state changed, and nothing here decides anything Alexander holds. — Volt + +**Volt postscript, same tick — the artifact's file count moved after point 6 was written, named rather than left to be caught.** Point 6 was measured before the gate transcript existed. The artifact now carries **five** transcripts (`transcript-before`, `-after`, `-mutation`, `-selftest`, `-verify`), so its `files.sha256` hashes **seven** files, not the six the record says. Re-measured at that state: `sh check.sh` → `CHECK: PASS`, rc 0, and `artifact_claim_check.py --repo .` → **7 known, 0 new**, rc 0 — the new artifact introduces no claim fault and its manifest is SELF. No figure in points 1-5 or 7 moves, and no queue item changed state. + +— Volt + +**Count pin 2026-10-22 (volt, Workhorse) — every selftest count is now witnessed outside the tool that prints it: a committed ledger, four graded classes, and the one defect the sweep could not see (a corpus that shrinks while the tool's own numbers agree with each other).** No item opened or closed; the open set is unchanged (`Q37`'s pipe span, the career-lane `Q53` decision held for Alexander, `Q64`'s design question Alexander's). Appended at the foot; `docs/queue-index.md` re-rendered in the same commit. + +1. **The defect, measured on a clone at `e33f4a6` before any edit.** `selftest_sweep.py` graded a tool by comparing the count its `--selftest` prints with a number in the same file — a real check on a stale number and no check at all on a shrinking corpus. On the clone, `tools/doc_command_check.py` lost its three `resolve_target` fixtures and its usage line went `15 cells` → `12 cells`; the tool graded itself `selftest: 12 cells, 12 passed, 0 failed`, rc 0, and **HEAD's unmodified sweep read the tree clean: `sweep: 21 tools, 21 OK, 0 fatal`, rc 0**, its row `doc_command_check.py OK 12 cells 12 cells 12 cells`. A fifth of that tool's fixtures gone, the verdict unmoved. + +2. **The witness is a file, not another number inside the tool.** `tools/selftest_baseline.tsv` (new, **21 data rows**, `toolcountunit`) is written by `--write-baseline` and read from disk on every run; `tools/selftest_sweep.py` grew `load_baseline`/`render_baseline`/`grade_pin`/`report_pin` plus `--baseline`/`--no-baseline`/`--write-baseline` (**+283 / −4** lines). Four graded classes: `PIN_OK`; `PIN_DRIFT` (printed ≠ pinned); `PIN_UNPINNED` (a printed count with no pinned row); `PIN_ORPHAN` (a pinned row whose tool is gone at this rev) — the last three fatal, rc 1. `PIN_NA` (a tool with no `--selftest`) and `PIN_NO_COUNT` (the run already failed above) print and are not counted twice. + +3. **A ledger that cannot be read is rc 2, never green.** Missing or malformed → `pin: cannot grade -- cannot read baseline …`, **rc 2** (`transcript-nopin-file.txt`), because a pin that graded nothing did not grade; `--no-baseline` is the explicit opt-out and the run says so out loud, and `--tools-dir` prints `pin: not graded` because a synthetic directory carries no pinned counts. + +4. **The same clone, the same mutation, after.** `doc_command_check.py PIN_DRIFT pinned 15 cells printed 12 cells`, `pin: 21 pinned, 20 agree, 1 drift (PIN_DRIFT)`, **rc 1** (`transcript-after.txt`) — while the sweep's own line still reads `21 OK, 0 fatal`, so the pin is the only instrument that sees the shrunken corpus. The unmutated tree: `pin: 21 pinned, 21 agree, 0 drift`, rc 0 (`transcript-repo.txt`). + +5. **The gate runs it, from the tree it grades.** `tools/verify.py:349-360` builds the `selftest-sweep` leg as `selftest_sweep.py --repo --quiet` — no `--no-baseline`, no `--tools-dir` — so the ledger committed in the graded tree is the one applied, and `--quiet` drops the per-row table but not the one-line `pin:` summary. + +6. **Cells, and what they prove.** `--selftest` **30 → 39**, `39 cells, 39 passed, 0 failed`, rc 0. `c31`–`c35` grade the four classes and `PIN_NA`; `c36`/`c37` the render/load round-trip and a non-integer count refused by line; `c38` `--write-baseline` end to end through `main()`, not through the pure function; `c39` a drifted pin naming `PIN_DRIFT` and turning the run **rc 1**. + +7. **What it does not claim.** The pin proves a count is the same, never that the fixtures behind it are meaningful. It is visibility, not a lock: one commit can move a tool's count and its ledger row together and pass, and the diff then shows the row moving — which is what a reviewer needs, and is not something the tool can enforce. It pins corpus size, not content, so a fixture replaced rather than deleted still passes. `--tools-dir` mode skips the pin by design. No queue item changed state and nothing here decides anything Alexander holds. + +Artifact `/root/.hermes/wizard-outputs/volt/2026-10-22-count-pin/` — `REPORT.md`, `check.sh`, six transcripts, and a `files.sha256` over its own eight files (`sh check.sh` → `CHECK: PASS`, rc 0; `sh check.sh --full` adds the slow gate leg). + +— Volt diff --git a/docs/queue-index.md b/docs/queue-index.md index e82ea3a..719f797 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -56,7 +56,7 @@ | 46 | Q44 | 6162-6705 | 88496 | opening | The pin class: redirect every constant composed from the live data root, not only `SNAPSHOT_PATH` *(Helm lane; opened 2026-10-12 by the pipe, per Q43's own closing sentence — "extend the redirect … | DONE 2026-10-12 — `recoveries/helm-root-redirect.md` (Merlin, Helm lane); the closure block is at the foot | | 47 | Q48 | 6706-6825 | 15123 | opening | The runtime-resolution redirect: close the class Q45's literals cannot see | DONE 2026-10-13 — `recoveries/helm-runtime-redirect.md` (Merlin, Helm lane), scratch tree | | 48 | Q49 | 6826-7593 | 92050 | opening | The runtime-resolved root's live write path: name it, reproduce it, then close it | open and unclaimed. Opened by the pipe 2026-10-13 | -| 49 | Q45 | 7594-10606 | 547023 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | OPEN — awaiting Alexander's zoom/whole-shop answer; build is the pipe's. | +| 49 | Q45 | 7594-10630 | 551944 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | OPEN — awaiting Alexander's zoom/whole-shop answer; build is the pipe's. | ## Ids with no `## ` heading @@ -71,9 +71,9 @@ Opened as records inside another section. `line` is the record's own opening lin | Q50 | 6990 | 6990 | 46 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q51 | 7106 | 7106 | 37 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q52 | 7167 | 7167 | 90 | Q49 | BULLET+BOLD | BOLD-LEAD | -| Q53 | 7292 | 7246 | 154 | Q49 | BULLET+BOLD | BOLD-LEAD | +| Q53 | 7292 | 7246 | 155 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q63 | 9918 | 9909 | 17 | Q45 | BULLET+BOLD | BOLD-LEAD | -| Q64 | 10308 | 10308 | 17 | Q45 | BULLET+BOLD | BOLD-LEAD | +| Q64 | 10308 | 10308 | 18 | Q45 | BULLET+BOLD | BOLD-LEAD | **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. @@ -218,7 +218,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q53", "line": 7292, - "refs": 154, + "refs": 155, "section": "Q49", "shape": "BULLET+BOLD" }, @@ -236,7 +236,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q64", "line": 10308, - "refs": 17, + "refs": 18, "section": "Q45", "shape": "BULLET+BOLD" } @@ -246,9 +246,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 1439284, - "queue_lines": 10606, - "queue_sha256": "8c12492b6459ff61955aabd5f7c6cb2db623514d01da06161dd157f70a6c8591", + "queue_bytes": 1444205, + "queue_lines": 10630, + "queue_sha256": "89c34d1666c6adc397c6c6d72e89663e94ecb21bd803cd93607063fb12acccc9", "sections": [ { "bytes": 3139, @@ -827,11 +827,11 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "The runtime-resolved root's live write path: name it, reproduce it, then close it" }, { - "bytes": 547023, - "end_line": 10606, + "bytes": 551944, + "end_line": 10630, "id": "Q45", "line": 7594, - "lines": 3013, + "lines": 3037, "role": "opening", "role_detail": "", "status": "OPEN \u2014 awaiting Alexander's zoom/whole-shop answer; build is the pipe's.", diff --git a/tools/selftest_baseline.tsv b/tools/selftest_baseline.tsv new file mode 100644 index 0000000..aad5d30 --- /dev/null +++ b/tools/selftest_baseline.tsv @@ -0,0 +1,32 @@ +# selftest_baseline.tsv -- the cell count each tool's selftest prints, pinned. +# Written by: python3 tools/selftest_sweep.py --repo . --write-baseline tools/selftest_baseline.tsv +# +# WHY THIS FILE EXISTS: the count a tool advertises lives in its own docstring, +# so a commit that deletes fixtures and decrements that number agrees with +# itself and every leg stays green. This ledger is the outside witness. +# `selftest_sweep.py` reads it on every run: a printed count that differs is +# PIN_DRIFT and fatal; a tool with no row is PIN_UNPINNED and fatal; a row +# whose tool is gone is PIN_ORPHAN and fatal. Re-pin on purpose, in the diff. +# +# toolcountunit +append_only_audit.py 18 cells +append_only_owners.py 15 cells +census_row_delta.py 33 cells +check_lint.py 37 cells +checkin_check.py 36 cells +citation_hash_class.py 19 fixtures +citation_resolve_check.py 14 fixtures +doc_census_check.py 104 cells +doc_command_check.py 15 cells +gate_blame.py 21 cells +landing_check.py 26 cells +path_claim_check.py 27 cells +queue_append_audit.py 25 cells +queue_claim_check.py 24 fixtures +queue_index.py 58 cells +reply_check.py 49 cells +selftest_sweep.py 39 cells +served_drift_check.py 25 cells +tool_map.py 20 cells +verify.py 51 cells +write_probe.py 15 cells diff --git a/tools/selftest_sweep.py b/tools/selftest_sweep.py index 8d6737d..0b1c27c 100644 --- a/tools/selftest_sweep.py +++ b/tools/selftest_sweep.py @@ -88,6 +88,26 @@ THE MALFORMED-INPUT CLASS, ADDED 2026-10-18 its --fail-on-stale is the gate). The axis asserts the message too, so a silent 0 cannot pass as a refusal. +THE COUNT PIN, ADDED 2026-10-22 -- the docstring was not a witness + Every verdict above compares a tool's printed count with a number in the SAME + file. That catches a stale number; it does not catch a shrinking corpus, + because a commit that deletes three fixtures and decrements the docstring + agrees with itself, and every leg stays green. Measured on a clone of this + repository: `doc_command_check.py` cut from 15 cells to 12 at both the print + and the usage line reads `OK 12 cells` and `sweep: 21 tools, 21 OK, 0 fatal, + rc 0` -- a third of its fixtures gone, reported clean. + `tools/selftest_baseline.tsv` is the second witness: one row per tool, the + count its selftest prints, committed and read from disk. The sweep compares + every row against it and prints four classes -- + PIN_OK / PIN_DRIFT (printed differs from pinned) / PIN_UNPINNED (a tool with + no pinned row) / PIN_ORPHAN (a pinned row whose tool is gone). DRIFT, + UNPINNED and ORPHAN are FATAL: a corpus may not shrink silently, and a new + tool may not arrive unmeasured. PIN_NA (a tool with no --selftest) and + PIN_NO_COUNT (its run already failed above) are printed and not counted + twice. A missing baseline in repo mode is rc 2 -- could not grade -- never a + clean run; `--no-baseline` is the explicit way to sweep without one, and + `--write-baseline PATH` re-pins after a deliberate change. + WHAT COUNTS AS A CLAIM, PRECISELY Only the module docstring (read through `ast.get_docstring`, so a fixture string in the BODY is not a claim -- `doc_census_check.py` contains the literal @@ -111,14 +131,17 @@ NO_PROBE is not: an ungraded tool cannot be reported clean (see the read axis). USAGE: python3 selftest_sweep.py --repo /root/troupe-standup + python3 selftest_sweep.py --repo /root/troupe-standup --write-baseline tools/selftest_baseline.tsv python3 selftest_sweep.py --tools-dir /tmp/fake --json /tmp/sweep.json - python3 selftest_sweep.py --selftest # 30 cells, parser, sweep and both input axes + python3 selftest_sweep.py --selftest # 39 cells, parser, sweep, pin and both input axes """ from __future__ import annotations import argparse import ast +import contextlib import glob +import io import json import os import re @@ -149,6 +172,10 @@ FATAL = ( "UNREADABLE", ) +# the count pin (2026-10-22): the second witness, kept out of the tool it grades +BASELINE_NAME = "selftest_baseline.tsv" +PIN_FAULT = ("PIN_DRIFT", "PIN_UNPINNED", "PIN_ORPHAN") + # --------------------------------------------------------------- read axis # THE SECOND GATE, ADDED 2026-10-17: the exit code a tool gives an input it # cannot read. The repo-wide contract is three codes and no others -- @@ -490,6 +517,137 @@ def report(results, source, untracked, quiet=False): return 1 if fatal else 0, ok, len(fatal) +# --------------------------------------------------------------- count pin +# WHY THIS IS A FILE. Every verdict in `report()` compares the tool's printed +# count with a number inside the tool. That is a real check on a stale number +# and no check at all on a shrinking corpus: delete three fixtures, decrement +# the docstring, and the file agrees with itself. Measured on a clone of this +# repository at 2026-10-22 -- `doc_command_check.py` cut 15 cells -> 12 on both +# the print and the usage line still reads `OK 12 cells`, `sweep: 21 tools, 21 +# OK, 0 fatal`, rc 0. The ledger is the witness the tool cannot edit in the +# same commit without the change being visible in the diff. + +def default_baseline(repo: str) -> str: + return os.path.join(os.path.abspath(repo), "tools", BASELINE_NAME) + + +def load_baseline(path: str): + """({tool: (count, unit)}, None) or (None, reason). Blank and # lines skipped.""" + rows = {} + try: + with open(path, "r", encoding="utf-8") as fh: + lines = fh.read().splitlines() + except OSError as exc: + return None, f"cannot read baseline {path}: {exc}" + for n, line in enumerate(lines, 1): + if not line.strip() or line.lstrip().startswith("#"): + continue + parts = line.split("\t") + if len(parts) != 3: + return None, (f"{path}:{n}: want 3 tab-separated columns " + f"(tool, count, unit), got {len(parts)}") + name, count_s, unit = (p.strip() for p in parts) + if unit not in UNITS: + return None, f"{path}:{n}: unit {unit!r} is not one of {UNITS}" + try: + count = _int(count_s) + except ValueError: + return None, f"{path}:{n}: count {count_s!r} is not an integer" + if name in rows: + return None, f"{path}:{n}: duplicate row for {name}" + rows[name] = (count, unit) + if not rows: + return None, f"{path}: no pinned rows" + return rows, None + + +def render_baseline(results) -> str: + """The ledger text for these results. Rows the selftest never printed are + skipped here and named by grade_pin() instead -- a pin cannot invent a + number the tool did not print.""" + head = [ + "# selftest_baseline.tsv -- the cell count each tool's selftest prints, pinned.", + "# Written by: python3 tools/selftest_sweep.py --repo . --write-baseline tools/selftest_baseline.tsv", + "#", + "# WHY THIS FILE EXISTS: the count a tool advertises lives in its own docstring,", + "# so a commit that deletes fixtures and decrements that number agrees with", + "# itself and every leg stays green. This ledger is the outside witness.", + "# `selftest_sweep.py` reads it on every run: a printed count that differs is", + "# PIN_DRIFT and fatal; a tool with no row is PIN_UNPINNED and fatal; a row", + "# whose tool is gone is PIN_ORPHAN and fatal. Re-pin on purpose, in the diff.", + "#", + "# toolcountunit", + ] + for r in sorted(results, key=lambda r: r["tool"]): + p = r.get("printed") + if not p: + continue + head.append(f"{r['tool']}\t{p['count']}\t{p['unit']}") + return "\n".join(head) + "\n" + + +def pin_faults(rows): + return [r for r in rows if r["verdict"] in PIN_FAULT] + + +def grade_pin(results, baseline): + """One row per tool plus one per orphaned pin. Pure: no I/O, no clock.""" + rows = [] + for r in sorted(results, key=lambda r: r["tool"]): + name, printed = r["tool"], r.get("printed") + pin = baseline.get(name) + pinned = {"count": pin[0], "unit": pin[1]} if pin else None + seen = ({"count": printed["count"], "unit": printed["unit"]} + if printed else None) + row = {"tool": name, "pinned": pinned, "printed": seen} + if r.get("verdict") == "NO_SELFTEST_FLAG": + row.update(verdict="PIN_NA", + detail="no --selftest to pin; not counted") + elif pin is None: + row.update(verdict="PIN_UNPINNED", + detail=(f"prints {seen['count']} {seen['unit']}; no pinned row" + if seen else "no pinned row, and no count to pin")) + elif seen is None: + row.update(verdict="PIN_NO_COUNT", + detail=(f"pinned {pin[0]} {pin[1]}; the run printed no count " + "(graded above)")) + elif (seen["count"], seen["unit"]) == (pin[0], pin[1]): + row.update(verdict="PIN_OK", detail=f"{pin[0]} {pin[1]}") + else: + row.update(verdict="PIN_DRIFT", + detail=(f"pinned {pin[0]} {pin[1]}; prints " + f"{seen['count']} {seen['unit']}")) + rows.append(row) + seen_tools = {r["tool"] for r in results} + for name in sorted(set(baseline) - seen_tools): + rows.append({"tool": name, "pinned": {"count": baseline[name][0], + "unit": baseline[name][1]}, + "printed": None, "verdict": "PIN_ORPHAN", + "detail": (f"pinned {baseline[name][0]} {baseline[name][1]}; " + "no tool of that name at this rev")}) + return rows + + +def report_pin(rows, path, quiet=False): + bad = pin_faults(rows) + if not quiet: + print() + print(f"count pin: {path} -- the number each selftest prints, pinned " + "outside the tool it grades") + width = max([len(r["tool"]) for r in rows] + [4]) + for r in rows: + pin, seen = r["pinned"], r["printed"] + pin_s = f"{pin['count']} {pin['unit']}" if pin else "-" + seen_s = f"{seen['count']} {seen['unit']}" if seen else "-" + print(f"{r['tool']:<{width}} {r['verdict']:<13} " + f"pinned {pin_s:>12} printed {seen_s:>12} {r['detail']}") + agree = sum(1 for r in rows if r["verdict"] == "PIN_OK") + pinned = sum(1 for r in rows if r["pinned"]) + print(f"pin: {pinned} pinned, {agree} agree, {len(bad)} drift" + + (f" ({', '.join(sorted({r['verdict'] for r in bad}))})" if bad else "")) + return (1 if bad else 0), len(bad) + + # ------------------------------------------------------- read-refusal axis def _first_line(text: str) -> str: @@ -802,7 +960,75 @@ def selftest(): and NO_PROBE in FATAL_READ, str(FATAL_READ)) - print(f"selftest: 30 cells, {30 - len(bad)} passed, {len(bad)} failed") + # c31-c37: the count pin. The defect these grade is the one the classes + # above cannot see: a corpus that shrinks while every number in the tool + # agrees with every other number in the tool. + pin_one = {"fake_good.py": (7, "cells")} + pr = grade_pin([{"tool": "fake_good.py", "verdict": "OK", + "printed": {"count": 7, "unit": "cells"}}], pin_one) + cell("c31 a printed count that matches its pinned row -> PIN_OK", + pr[0]["verdict"] == "PIN_OK" and not pin_faults(pr), pr[0]["verdict"]) + pr = grade_pin([{"tool": "fake_good.py", "verdict": "OK", + "printed": {"count": 4, "unit": "cells"}}], pin_one) + cell("c32 four cells against a pin of seven -> PIN_DRIFT, fatal", + pr[0]["verdict"] == "PIN_DRIFT" and len(pin_faults(pr)) == 1, + f"{pr[0]['verdict']} {pr[0]['detail']}") + pr = grade_pin([{"tool": "new_tool.py", "verdict": "OK", + "printed": {"count": 3, "unit": "cells"}}], pin_one) + mine = [r for r in pr if r["tool"] == "new_tool.py"] + cell("c33 a tool with no pinned row -> PIN_UNPINNED, fatal", + mine and mine[0]["verdict"] == "PIN_UNPINNED" and mine[0] in pin_faults(pr), + mine[0]["verdict"] if mine else "no row") + pr = grade_pin([], pin_one) + cell("c34 a pinned row whose tool is gone -> PIN_ORPHAN, fatal", + pr[0]["verdict"] == "PIN_ORPHAN" and len(pin_faults(pr)) == 1, + pr[0]["verdict"]) + pr = grade_pin([{"tool": "flagless.py", "verdict": "NO_SELFTEST_FLAG", + "printed": None}], pin_one) + mine = [r for r in pr if r["tool"] == "flagless.py"] + cell("c35 a tool with no --selftest is PIN_NA, not a fault", + mine and mine[0]["verdict"] == "PIN_NA" and mine[0] not in pin_faults(pr), + mine[0]["verdict"] if mine else "no row") + + with tempfile.TemporaryDirectory() as td: + bp = os.path.join(td, "pin.tsv") + blob = render_baseline([{"tool": "b.py", "printed": {"count": 9, "unit": "fixtures"}}, + {"tool": "a.py", "printed": {"count": 2, "unit": "cells"}}]) + with open(bp, "w", encoding="utf-8") as fh: + fh.write(blob) + round_trip, why = load_baseline(bp) + cell("c36 render/load round-trip keeps both rows and their units", + round_trip == {"b.py": (9, "fixtures"), "a.py": (2, "cells")}, + f"{round_trip} {why}") + with open(bp, "w", encoding="utf-8") as fh: + fh.write("a.py\tnine\tcells\n") + bad_pin, why = load_baseline(bp) + cell("c37 a non-integer count is refused, naming the line", + bad_pin is None and ":1:" in (why or "") and "not an integer" in (why or ""), + str(why)) + + with tempfile.TemporaryDirectory() as td: + for nm, body in (("fake_good.py", good), ("fake_flagless.py", flagless)): + with open(os.path.join(td, nm), "w") as fh: + fh.write(body) + bp = os.path.join(td, "pin.tsv") + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + main(["--tools-dir", td, "--write-baseline", bp, "--quiet"]) + written, why = load_baseline(bp) + cell("c38 --write-baseline pins exactly what the run printed", + written == {"fake_good.py": (7, "cells")}, f"{written} {why}") + with open(bp, "w", encoding="utf-8") as fh: + fh.write("fake_good.py\t9\tcells\n") + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + rc_pin = main(["--tools-dir", td, "--baseline", bp, "--quiet"]) + out = buf.getvalue() + cell("c39 end to end: a drifted pin names PIN_DRIFT and turns the run red", + "PIN_DRIFT" in out and "1 drift (PIN_DRIFT)" in out and rc_pin == 1, + (out.strip().splitlines() or ["no output"])[-1]) + + print(f"selftest: 39 cells, {39 - len(bad)} passed, {len(bad)} failed") return 1 if bad else 0 @@ -816,6 +1042,13 @@ def main(argv=None): help="sweep this directory instead of the repo's tracked tools/") ap.add_argument("--timeout", type=int, default=120, metavar="S") ap.add_argument("--json", default=None, metavar="PATH") + ap.add_argument("--baseline", default=None, metavar="PATH", + help="the pinned counts to grade against " + f"(default: /tools/{BASELINE_NAME} in repo mode)") + ap.add_argument("--no-baseline", action="store_true", + help="do not grade the count pin (the run says so out loud)") + ap.add_argument("--write-baseline", default=None, metavar="PATH", + help="write the pinned counts from this run to PATH") ap.add_argument("--quiet", action="store_true") ap.add_argument("--verbose", action="store_true") ap.add_argument("--selftest", action="store_true") @@ -840,6 +1073,48 @@ def main(argv=None): axis = read_axis(rows, a.timeout) rrc, read_fatal = report_read(axis, a.quiet) + # ---- the count pin: the witness outside the tool's own docstring + if a.no_baseline: + pin_path, pin_note = None, "pin: not graded (--no-baseline)" + elif a.baseline: + pin_path, pin_note = a.baseline, None + elif a.write_baseline: + pin_path, pin_note = a.write_baseline, None + elif a.tools_dir: + pin_path, pin_note = None, ("pin: not graded (--tools-dir: a synthetic " + "directory carries no pinned counts)") + else: + pin_path, pin_note = default_baseline(a.repo), None + + pin_rc, pin_fatal, pin_rows = 0, 0, None + if pin_path is None: + if not a.quiet and pin_note: + print() + print(pin_note) + else: + if a.write_baseline: + text = render_baseline(results) + try: + with open(a.write_baseline, "w", encoding="utf-8") as fh: + fh.write(text) + except OSError as exc: + print("cannot write --write-baseline " + a.write_baseline + + ": " + str(exc), file=sys.stderr) + return 2 + first = [l for l in text.splitlines() if l and not l.startswith("#")] + print(f"pin: wrote {a.write_baseline} ({len(first)} row(s))") + pinned, why = load_baseline(pin_path) + if pinned is None: + # A missing or malformed ledger is not a clean run: the pin graded + # nothing. rc 2 (could not grade), never rc 1 and never 0. + print("pin: cannot grade -- " + str(why) + + " (run --write-baseline to create one, or --no-baseline to sweep without)", + file=sys.stderr) + pin_rc = 2 + else: + pin_rows = grade_pin(results, pinned) + pin_rc, pin_fatal = report_pin(pin_rows, pin_path, a.quiet) + if a.json: try: with open(a.json, "w") as fh: @@ -851,11 +1126,15 @@ def main(argv=None): "fatal": fatal, "ok": ok, "read_axis": axis, - "read_fatal": read_fatal}, fh, indent=1) + "read_fatal": read_fatal, + "pin": {"path": pin_path, "rows": pin_rows, + "fatal": pin_fatal}}, fh, indent=1) except OSError as exc: print("cannot write --json " + a.json + ": " + str(exc), file=sys.stderr) return 2 - return 1 if (rc or rrc) else 0 + if pin_rc == 2: + return 2 + return 1 if (rc or rrc or pin_rc) else 0 if __name__ == "__main__":