#!/usr/bin/env python3 """verify.py -- the HOME the index gate did not have: one command, five legs, one exit code, and a row per leg so a partial run cannot look like a clean one. WHY THIS EXISTS On 2026-10-18 this lane told Alexander, three ticks running, to "wire `queue_index --check` into the tick's check.sh". There is no `check.sh` in this repo and no `scripts/` directory at all -- `git ls-files` shows `tools/` and nothing else. The advice named a target that does not exist, which is why it was never adopted. The gate did not need a decision about policy; it needed a home. This is that home, and it is a task, not a policy call: nothing here obliges any lane to run it, and nothing is enforced until a lane does. The second reason is narrower and this lane's own: three separate counts of "which commits left the index stale" came out wrong in one week, each from a hand-assembled probe loop. A gate whose legs are remembered is a gate that gets run selectively. Fixed order, one rc -- that is the whole design. The third reason is measured, not argued. On 2026-10-19T09:42Z this lane put a contract-breaking check-in into a clone of HEAD -- `checkins/2026-10-19-volt-02.md`, a second same-day file carrying a suffix -- and ran both instruments on it: `checkin_check.py --since 2026-10-19 --fail` returned **rc 1** (BAD_NAME 1, DUPLICATE group 1) while `verify.py --fast` returned **rc 0** on the same tree, "gate: OK -- 2 of 2 graded leg(s) passed". Three legs that grade the queue, its append discipline and the tools' selftests said nothing about the one artifact every lane writes every tick. A fourth leg closes that hole. The fifth is the same shape one directory over, and it is here for a reason that was measured before it was written. On 2026-10-19 this lane built `reply_check.py` and deliberately did NOT wire it: the archive has one live fault -- `replies/2026-10-15-to-timmy-58.md` is named for Timmy and headed "to Merlin" -- it is Merlin's file and Merlin's to repair, and that tick's reasoning was that a gate wired to a defect its owner has not fixed yet is a gate that gets switched off in a week. That reasoning is right about a gate with no ledger and wrong about this repo, which already solved the same problem for check-ins: `tools/reply_baseline.tsv` states the one tolerance, the fault stays visible in every report as known-faulting, and a second mis-addressed reply -- or a new fault class on the known file -- is rc 1 the day it lands. Waiting for another lane to edit its own file was the only alternative, and it had no date on it. WHAT IT RUNS -- one row per leg, cheapest first, never in parallel checkin tools/checkin_check.py --fail --baseline tools/checkin_baseline.tsv the WHOLE check-in archive against F1-F5 (names, duplicates, unknown handles, dates, emptiness), where the ledger states the faults the archive already carried; only a class it does not name is a fault. `--checkin-day DATE` adds a `--since` floor reply-headings tools/reply_check.py --baseline tools/reply_baseline.tsv every reply's heading against its own filename (addressee and number), on the same ledger rule: `replies/` is the second largest artifact directory and, until 2026-10-19, nothing here read it index-current tools/queue_index.py --check docs/queue-index.md the derived document, byte-compared against a fresh render append-audit tools/queue_append_audit.py --guard --to HEAD the newest queue-touching commit, graded in ITS OWN tree selftest-sweep tools/selftest_sweep.py every tool's --selftest plus its two read-refusal axes (skip with --fast; the row is still printed and counted) The legs are ordered by cost, and a leg is never skipped because an earlier one failed: a stale index and a broken selftest are different defects and the run should name both in one pass. WHY THE CHECK-IN LEG NOW GRADES THE ARCHIVE, NOT ONE DAY Grading the whole archive was, until 2026-10-19, a permanently red gate: at `e98a34b` the unbaselined run exits 1 on every one of the archive's pre-rule rows (25 BAD_NAME, 10 DUPLICATE groups, all dated 2026-09-24 through 2026-10-07) -- that list is now a measured artifact, `tools/checkin_baseline.tsv`, one row per file rather than a number in this docstring. The first version of this leg therefore graded the archive's **newest day** only, on the reasoning that it is the only day on which a new fault can appear. That reasoning is false, and it was measured false rather than argued: a check-in added today under a **backdated** filename -- `checkins/2026-10-02-volt-b.md`, which is what a typo in the date half produces -- is not on the newest day, so the floor excluded it. On a clone of `e98a34b` carrying that one file, `verify.py --fast` printed `checkin_check: 1 files, 0 with a fault` and exited **0**, while the same tree's whole-archive run exited **1** naming the file. The fix keeps both properties -- history is not re-litigated, nothing is invisible. `tools/checkin_baseline.tsv` lists the faults the archive already carried, one `\t` row each, and `--baseline` grades every file against it: a listed class is KNOWN, anything else is a fault. A file that is not in the ledger at all is a fault wherever its date falls, and a file that gains a class it did not have is a fault too -- the ledger states what was tolerated, not that a path is exempt. A missing or malformed ledger is rc 2 (cannot grade), never a silent pass, and stale entries (file gone, class cleared) are reported and never make rc 1. `--checkin-day DATE` survives as an explicit narrowing floor for an operator who wants one. THE EXIT CONTRACT -- the house three codes 0 every graded leg passed 1 at least one leg ran and graded a fault 2 at least one leg could not grade -- a missing tool, a timeout, an unexpected exit code, an unreadable repo -- or no leg was runnable at all rc 2 OUTRANKS rc 1 deliberately. rc 1 means "I ran and I graded a fault"; a run with an ungradable leg has not made that claim about the whole gate, and folding the two (as the 15:16Z correction caught `queue_append_audit` doing with ungradable commits) is how a "3 stale" reading hides a "2 could not be read". The summary always prints both counts, so precedence never erases a fault -- it only refuses to certify the run that found it. A skipped leg is not a pass and not a fault: it is counted as `skipped` and printed as SKIP. `--fast` on a green tree exits 0 with one row SKIP, and the summary says so rather than pretending the sweep ran. TOOLS ARE RESOLVED FROM --repo, NOT FROM THIS FILE'S DIRECTORY `verify.py --repo /path/to/worktree` runs THAT tree's `queue_index.py`, `queue_append_audit.py` and `selftest_sweep.py`. That is the instrument lesson the 15:16Z correction paid for: a gate run in a working tree at HEAD says nothing about a cited revision, so grading a revision means grading its own tools from its own checkout. --queue and --index stay REPO-RELATIVE (defaults WORK-QUEUE.md and docs/queue-index.md) because `queue_append_audit` reads them with `git show :`, which has no absolute-path form. `--selftest` fixture corpus, 42 cells """ import argparse import json import os import re import subprocess import sys import tempfile import time QUEUE_DEFAULT = "WORK-QUEUE.md" INDEX_DEFAULT = "docs/queue-index.md" CHECKINS_DEFAULT = "checkins" PASS, FAULT, UNGRADABLE, SKIP = "PASS", "FAULT", "UNGRADABLE", "SKIP" # Legs, in the fixed order they are printed and run. `tool` is resolved inside # the repo being graded. `slow` marks the leg --fast drops. LEGS = ( {"key": "checkin", "tool": "checkin_check.py", "slow": False, "what": "grade the newest day's check-ins against the naming and " "duplicate rules"}, {"key": "reply-headings", "tool": "reply_check.py", "slow": False, "what": "grade every reply's heading against its own filename"}, {"key": "index-current", "tool": "queue_index.py", "slow": False, "what": "render the index and byte-compare it with the document"}, {"key": "append-audit", "tool": "queue_append_audit.py", "slow": False, "what": "grade the newest queue-touching commit in its own tree"}, {"key": "selftest-sweep", "tool": "selftest_sweep.py", "slow": True, "what": "every tool's --selftest and both read-refusal axes"}, ) # A detail line is a courtesy, not the verdict -- but a verdict with no line # beside it is what the read-refusal axis calls SILENT, so it is graded too. # # STRONG outranks WEAK, and both outrank a per-file SKIP note. That tiering is # not cosmetic: `checkin_check.py` prints " SKIP README.md (the directory's own # README: not a check-in)" as its second line on every real run, "not a " is a # WEAK hint, and before the tiering that note was carried onto a FAULT row -- # the row said FAULT beside a line about the README while the tool's own # "FAIL: 2 file(s) violate the check-in contract" sat further down, unpicked. STRONG_HINTS = ("STALE:", "FAIL:", "FAILED", "fatal", "TIMEOUT", "NO_", "cannot", "error", "Error", "refus") WEAK_HINTS = ("FAIL", "not a ") # A tool's concluding line: `reply_check: 192 file(s), 0 new fault(s) ...`. # Hyphenated labels (`read-refusal:`, `malformed-input:`) are deliberately not # summaries -- they are axis lines inside a tool's report. SUMMARY_RE = re.compile(r"^[a-z][a-z0-9_]*: ") def pick_detail(text, limit=150, summary=False): """First line that names a reason, else the last line that says anything. A summary line that reports ZERO faults is not a reason: `selftest_sweep` prints `sweep: 15 tools, 15 OK, 0 fatal` even on the run where its input axes went red, and that line was being carried onto a FAULT row -- the row said FAULT beside the words "0 fatal". Zero-count summaries are therefore only reachable on the second pass (a clean run still gets its sweep line), and any line that names a real reason outranks them. `summary=True` is the clean-run answer, and it is a different question. On a PASS row the useful detail is the tool's own concluding line, not the first line in its report that happens to contain "refus" -- `reply_check` prints one heading excerpt per advisory row, and one of those headings contains the word "refuses", which was reaching the row as if it were the reason. On a fault row the reason still wins, because that is what a reader needs there. """ lines = [ln.strip() for ln in text.splitlines() if ln.strip()] kept = [ln for ln in lines if not ln.startswith("SKIP ")] or lines if summary: sums = [ln for ln in kept if SUMMARY_RE.match(ln)] if sums: return sums[-1][:limit] hinted = [ln for ln in kept if any(h in ln for h in STRONG_HINTS + WEAK_HINTS)] # A class row that printed zero is not a reason and not a count of anything: # `reply_check` prints its whole class table on every run, so `NO_HEADER 0` # was reaching the detail line of a PASS row ahead of the summary. Same # failure the " 0 fatal" filter catches, one shape further out. class_row = re.compile(r"^[A-Z][A-Z0-9_]+ +(\d+)$") zero_row = re.compile(r"^[A-Z][A-Z0-9_]+ +0$") reasons = [ln for ln in hinted if " 0 fatal" not in ln and " 0 fault" not in ln and not zero_row.match(ln)] # Nor is a nonzero class row a "reason": `NO_RECIPIENT_IN_HEADING 7` is a # count, and `NO_` in its name is not a diagnostic marker. A hint counts # only on a line that is not a class table row. strong = [ln for ln in reasons if any(h in ln for h in STRONG_HINTS) and not class_row.match(ln)] # A tool's own verdict line outranks every other reason in its report: when # `reply_check` names a mis-addressed file it also prints one heading excerpt # per row, and an excerpt containing "refuses" was beating the FAIL line. verdict = [ln for ln in strong if ln.startswith(("FAIL:", "STALE:"))] if verdict: return verdict[0][:limit] if strong: return strong[0][:limit] if reasons: return reasons[0][:limit] if hinted: return hinted[0][:limit] return kept[-1][:limit] if kept else "" def newest_checkin_day(repo): """The newest `` prefix in checkins/ -> (day, why-not). Not the graded window any more -- the leg grades the whole archive against its own ledger -- but still the cheapest way to answer "is there anything here to grade at all", which is what makes a repo with no `checkins/` directory (or no dated file in it) ungradable with a message naming which of the two it is, rather than a spawn and an rc 2 from further down. """ d = os.path.join(repo, CHECKINS_DEFAULT) if not os.path.isdir(d): return None, "no check-in directory: %s" % d days = sorted(m.group(1) for m in (re.match(r"^(\d{4}-\d{2}-\d{2})-", n) for n in os.listdir(d)) if m) if not days: return None, "no dated check-in files in %s" % d return days[-1], None def leg_argv(key, repo, queue, index, checkin_day=None): if key == "checkin": # The ledger is a file in the tree being graded, so a revision is graded # with the tolerance it committed -- not with this host's idea of one. argv = [sys.executable, os.path.join(repo, "tools", "checkin_check.py"), "--repo", repo, "--fail", "--baseline", os.path.join(repo, "tools", "checkin_baseline.tsv")] if checkin_day: argv += ["--since", checkin_day] return argv if key == "reply-headings": # The ledger comes from the tree being graded, for the same reason the # check-in ledger does: a revision is graded with the tolerance it # committed, not with this host's idea of one. return [sys.executable, os.path.join(repo, "tools", "reply_check.py"), "--repo", repo, "--baseline", os.path.join(repo, "tools", "reply_baseline.tsv")] if key == "index-current": return [sys.executable, os.path.join(repo, "tools", "queue_index.py"), "--repo", repo, "--queue", queue, "--check", index] if key == "append-audit": return [sys.executable, os.path.join(repo, "tools", "queue_append_audit.py"), "--repo", repo, "--queue", queue, "--index", index, "--guard", "--to", "HEAD"] if key == "selftest-sweep": return [sys.executable, os.path.join(repo, "tools", "selftest_sweep.py"), "--repo", repo, "--quiet"] raise KeyError(key) def run_leg(leg, repo, queue, index, timeout, checkin_day=None): """One leg -> one row. rc 0/1/2 are graded; anything else is ungradable.""" tool = os.path.join(repo, "tools", leg["tool"]) started = time.time() row = {"key": leg["key"], "tool": leg["tool"], "verdict": None, "rc": None, "detail": "", "what": leg["what"]} if not os.path.exists(tool): row.update(verdict=UNGRADABLE, detail="no such tool: %s" % tool) row["ms"] = 0 return row day = checkin_day if leg["key"] == "checkin": # Presence is still pre-checked here: the tool's own rc-2 message names # the directory, but this row can say so without spawning a process. newest, why = newest_checkin_day(repo) if newest is None: row.update(verdict=UNGRADABLE, detail=why) row["ms"] = 0 return row try: proc = subprocess.run(leg_argv(leg["key"], repo, queue, index, day), cwd=repo, capture_output=True, text=True, timeout=timeout) except subprocess.TimeoutExpired: row.update(verdict=UNGRADABLE, detail="TIMEOUT after %ss" % timeout) row["ms"] = int((time.time() - started) * 1000) return row except OSError as exc: row.update(verdict=UNGRADABLE, detail="could not run: %s" % exc) row["ms"] = int((time.time() - started) * 1000) return row out = (proc.stdout or "") + (proc.stderr or "") row["rc"] = proc.returncode row["detail"] = pick_detail(out, summary=(proc.returncode == 0)) if proc.returncode == 0: row["verdict"] = PASS elif proc.returncode == 1: row["verdict"] = FAULT elif proc.returncode == 2: row["verdict"] = UNGRADABLE else: # rc 3 is retired in this repo (one meaning, two codes); any other code # is a tool that left the contract, and the run cannot grade through it. row["verdict"] = UNGRADABLE row["detail"] = "unexpected rc %d: %s" % (proc.returncode, row["detail"]) row["ms"] = int((time.time() - started) * 1000) return row def counts(rows): c = {PASS: 0, FAULT: 0, UNGRADABLE: 0, SKIP: 0} for r in rows: c[r["verdict"]] += 1 return c def decide(rows): """The precedence, in one place: ungradable outranks fault outranks clean.""" c = counts(rows) if c[UNGRADABLE]: return 2 if c[FAULT]: return 1 return 0 def report(rows, repo, quiet=False): c = counts(rows) if not quiet: print("verify -- repo: %s" % repo) for r in rows: rc = "-" if r["rc"] is None else str(r["rc"]) print(" %-16s %-10s rc=%-2s %-6s %s" % (r["key"], r["verdict"], rc, "%dms" % r["ms"], r["detail"])) print() print("verify: %d legs, %d pass, %d fault, %d ungradable, %d skipped" % (len(rows), c[PASS], c[FAULT], c[UNGRADABLE], c[SKIP])) faulted = [r["key"] for r in rows if r["verdict"] == FAULT] ungraded = [r["key"] for r in rows if r["verdict"] == UNGRADABLE] if faulted: print("fault: %s" % " ".join(faulted)) if ungraded: print("ungradable: %s" % " ".join(ungraded)) rc = decide(rows) if rc == 0: print("gate: OK -- %d of %d graded leg(s) passed%s" % (c[PASS], c[PASS] + c[FAULT], "" if not c[SKIP] else " (%d skipped: not a pass, not a fault)" % c[SKIP])) elif rc == 1: print("gate: FAULT -- %d leg(s) ran and graded a fault: %s" % (c[FAULT], " ".join(faulted))) else: print("gate: NOT CERTIFIED -- %d leg(s) could not grade: %s" % (c[UNGRADABLE], " ".join(ungraded))) return rc def main(argv=None): ap = argparse.ArgumentParser( description="Run the repo's gates as one command: the newest day's " "check-ins, index currency, the append audit, and the " "selftest sweep.") ap.add_argument("--repo", default=".") ap.add_argument("--queue", default=QUEUE_DEFAULT, help="repo-relative queue path (default %s)" % QUEUE_DEFAULT) ap.add_argument("--index", default=INDEX_DEFAULT, help="repo-relative index document (default %s)" % INDEX_DEFAULT) ap.add_argument("--checkin-day", default=None, metavar="DATE", help="add a --since DATE floor to the check-in leg, so only " "check-ins dated DATE or later are graded (default: the " "whole archive, against tools/checkin_baseline.tsv)") ap.add_argument("--fast", action="store_true", help="skip the selftest-sweep leg; the row is printed as SKIP") ap.add_argument("--timeout", type=int, default=300, metavar="S", help="per-leg timeout in seconds (default 300)") ap.add_argument("--json", metavar="PATH", help="write the rows here") ap.add_argument("--quiet", action="store_true") ap.add_argument("--selftest", action="store_true", help="fixture corpus, 42 cells") a = ap.parse_args(argv) if a.selftest: return selftest() repo = os.path.abspath(a.repo) if not os.path.isdir(repo): print("cannot read repo: %s is not a directory" % repo, file=sys.stderr) return 2 if not os.path.isdir(os.path.join(repo, ".git")): print("cannot read repo: %s is not a git repository" % repo, file=sys.stderr) return 2 rows = [] for leg in LEGS: if a.fast and leg["slow"]: rows.append({"key": leg["key"], "tool": leg["tool"], "verdict": SKIP, "rc": None, "detail": "--fast: leg not run", "what": leg["what"], "ms": 0}) continue rows.append(run_leg(leg, repo, a.queue, a.index, a.timeout, a.checkin_day)) rc = report(rows, repo, a.quiet) if a.json: c = counts(rows) with open(a.json, "w") as fh: json.dump({"repo": repo, "queue": a.queue, "index": a.index, "fast": a.fast, "rows": rows, "counts": c, "rc": rc}, fh, indent=1) return rc # ------------------------------------------------------------------- selftest def _stub(path, rc, out="", sleep=0.0, marker=None, argv_marker=None): """A stand-in leg: it prints what the fixture needs and exits with its rc.""" body = ["#!/usr/bin/env python3", "import sys"] if sleep: body.append("import time; time.sleep(%r)" % sleep) if marker: body.append("open(%r, 'a').write('hit\\n')" % marker) if argv_marker: body.append("open(%r, 'w').write(' '.join(sys.argv[1:]))" % argv_marker) if out: body.append("sys.stdout.write(%r)" % out) body.append("sys.exit(%d)" % rc) with open(path, "w") as fh: fh.write("\n".join(body) + "\n") os.chmod(path, 0o755) def _fixture(base, rcs, outs=None, sleep=0.0, markers=None, argv_markers=None, checkins=("2026-01-01-fixture.md",)): """A throwaway repo with stubbed legs. rcs: {tool: rc}. `checkins` seeds `checkins/`: one dated file by default (so the check-in leg has a day to grade), a tuple of names for cells that need a specific set, and None for a repo with no `checkins/` directory at all. """ repo = os.path.join(base, "repo") os.makedirs(os.path.join(repo, "tools")) os.makedirs(os.path.join(repo, ".git")) with open(os.path.join(repo, QUEUE_DEFAULT), "w") as fh: fh.write("## Q1 -- fixture\n") os.makedirs(os.path.join(repo, "docs")) with open(os.path.join(repo, INDEX_DEFAULT), "w") as fh: fh.write("fixture\n") if checkins is not None: os.makedirs(os.path.join(repo, CHECKINS_DEFAULT)) for name in checkins: with open(os.path.join(repo, CHECKINS_DEFAULT, name), "w") as fh: fh.write("Did: fixture\nBlocked: nothing\nNext: nothing\n" "Ask: nothing\n") for tool, rc in rcs.items(): if rc is None: # tool absent on purpose continue _stub(os.path.join(repo, "tools", tool), rc, out=(outs or {}).get(tool, ""), sleep=sleep, marker=(markers or {}).get(tool), argv_marker=(argv_markers or {}).get(tool)) return repo def _verify(*argv): """Run this file as the CLI, so the fixture grades the real entry point.""" proc = subprocess.run([sys.executable, os.path.abspath(__file__)] + list(argv), capture_output=True, text=True) return proc.returncode, (proc.stdout or "") + (proc.stderr or "") GREEN5 = {"checkin_check.py": 0, "reply_check.py": 0, "queue_index.py": 0, "queue_append_audit.py": 0, "selftest_sweep.py": 0} def selftest(): cells = 0 passed = 0 failures = [] def check(label, got, want): nonlocal cells, passed cells += 1 if got == want: passed += 1 else: failures.append("%s\n got: %r\n want: %r" % (label, got, want)) with tempfile.TemporaryDirectory(prefix="verify-selftest-") as td: # 1. five green stubs -> rc 0 repo = _fixture(os.path.join(td, "c1"), GREEN5) rc, out = _verify("--repo", repo) check("1 all five legs green -> rc 0", rc, 0) check("1b summary counts the five legs", "verify: 5 legs, 5 pass, 0 fault, " "0 ungradable, 0 skipped" in out, True) # 2. a stale index is a FAULT, and its own line reaches the row repo = _fixture(os.path.join(td, "c2"), dict(GREEN5, **{"queue_index.py": 1}), outs={"queue_index.py": "STALE: rendered document differs at line 7\n"}) rc, out = _verify("--repo", repo) check("2 stale index -> rc 1", rc, 1) check("2b the STALE line is carried into the row", "index-current" in out and "differs at line 7" in out, True) # 3. a refusal is UNGRADABLE, not a fault repo = _fixture(os.path.join(td, "c3"), dict(GREEN5, **{"queue_index.py": 2}), outs={"queue_index.py": "cannot read queue: nope\n"}) rc, out = _verify("--repo", repo) check("3 refusal -> rc 2", rc, 2) check("3b refusal is named in the summary", "ungradable: index-current" in out, True) # 4. precedence: ungradable outranks fault, and the fault is NOT erased repo = _fixture(os.path.join(td, "c4"), dict(GREEN5, **{"queue_index.py": 1, "selftest_sweep.py": 2}), outs={"queue_index.py": "STALE: line 3\n", "selftest_sweep.py": "cannot read: no tools\n"}) rc, out = _verify("--repo", repo) check("4 ungradable outranks fault -> rc 2", rc, 2) check("4b both classes are still printed in one run", "1 fault, 1 ungradable" in out and "fault: index-current" in out, True) # 5. a missing leg tool is ungradable, and says which path was missing repo = _fixture(os.path.join(td, "c5"), dict(GREEN5, **{"queue_append_audit.py": None})) rc, out = _verify("--repo", repo) check("5 missing tool -> rc 2", rc, 2) check("5b the missing path is named", "no such tool:" in out, True) # 6. --fast skips the sweep and does not run it marker = os.path.join(td, "c6-sweep-ran") repo = _fixture(os.path.join(td, "c6"), GREEN5, markers={"selftest_sweep.py": marker}) rc, out = _verify("--repo", repo, "--fast") check("6 --fast on a green tree -> rc 0", rc, 0) check("6b the skipped leg is printed as SKIP", "selftest-sweep" in out and "SKIP" in out, True) check("6c the skipped stub never ran", os.path.exists(marker), False) # 7. --json is a real machine-readable record repo = _fixture(os.path.join(td, "c7"), GREEN5) jpath = os.path.join(td, "c7.json") rc, out = _verify("--repo", repo, "--json", jpath) obj = json.load(open(jpath)) if os.path.exists(jpath) else {} check("7 --json writes the rows", [r["key"] for r in obj.get("rows", [])], ["checkin", "reply-headings", "index-current", "append-audit", "selftest-sweep"]) check("7b --json carries the same rc the process returned", obj.get("rc"), rc) # 8. a repo that is not there is refused before any leg runs rc, out = _verify("--repo", os.path.join(td, "no-such-repo")) check("8 missing repo -> rc 2", rc, 2) check("8b the refusal names the path", "cannot read repo:" in out, True) # 9. rc 3 is retired: it is ungradable, never a fault and never a pass repo = _fixture(os.path.join(td, "c9"), dict(GREEN5, **{"queue_index.py": 3}), outs={"queue_index.py": "unknown state\n"}) rc, out = _verify("--repo", repo) check("9 rc 3 -> rc 2 (retired code)", rc, 2) check("9b the unexpected code is named", "unexpected rc 3" in out, True) # 10. a hung leg is a timeout, not a pass and not a fault repo = _fixture(os.path.join(td, "c10"), dict(GREEN5, **{"selftest_sweep.py": 0}), sleep=5.0) rc, out = _verify("--repo", repo, "--timeout", "1") check("10 timeout -> rc 2", rc, 2) check("10b the timeout is named", "TIMEOUT after 1s" in out, True) # 11. order is fixed, cheapest first: the table is not re-sorted by outcome repo = _fixture(os.path.join(td, "c11"), dict(GREEN5, **{"queue_index.py": 1}), outs={"queue_index.py": "STALE: line 1\n"}) rc, out = _verify("--repo", repo) order = [out.find(k) for k in ("checkin", "reply-headings", "index-current", "append-audit", "selftest-sweep")] check("11 legs print in fixed order", order == sorted(order) and -1 not in order, True) # 12. a skip is counted as a skip, never folded into pass repo = _fixture(os.path.join(td, "c12"), GREEN5) jpath = os.path.join(td, "c12.json") rc, out = _verify("--repo", repo, "--fast", "--json", jpath) obj = json.load(open(jpath)) if os.path.exists(jpath) else {} check("12 skipped leg is its own count in --json", obj.get("counts", {}).get("SKIP"), 1) check("12b and the skipped leg is not counted as a pass", obj.get("counts", {}).get(PASS), 4) # 13. the detail line must not contradict the verdict: a row graded # FAULT beside "0 fatal" is the SILENT failure this gate exists to catch. check("13 a zero-fault summary is not a reason line", pick_detail("sweep: 15 tools, 15 OK, 0 fatal\n" "read-refusal: 20 probes, 19 ok, 1 fault (NO_PROBE)\n" "malformed-input: 18 probes, 17 ok, 1 fault (NO_PROBE)\n"), "read-refusal: 20 probes, 19 ok, 1 fault (NO_PROBE)") check("13b a clean run still gets its sweep line", pick_detail("sweep: 14 tools, 14 OK, 0 fatal\n" "read-refusal: 19 probes, 19 ok, 0 fault\n"), "sweep: 14 tools, 14 OK, 0 fatal") # 14. THE HOLE THIS LEG CLOSES: a contract-breaking check-in is rc 1, # and the gate is no longer green on a tree that carries one. repo = _fixture(os.path.join(td, "c14"), dict(GREEN5, **{"checkin_check.py": 1}), outs={"checkin_check.py": "FAIL: 2 file(s) violate the check-in contract\n"}) rc, out = _verify("--repo", repo, "--fast") check("14 a broken check-in -> rc 1 (was rc 0 with three legs)", rc, 1) check("14b the fault row is named checkin and carries its reason", "checkin" in out and "violate the check-in contract" in out and "fault: checkin" in out, True) # 15. no checkins/ directory is ungradable, not a pass and not a fault repo = _fixture(os.path.join(td, "c15"), GREEN5, checkins=None) rc, out = _verify("--repo", repo, "--fast") check("15 no checkins dir -> rc 2", rc, 2) check("15b the ungradable row says which directory is missing", "ungradable: checkin" in out and "no check-in directory:" in out, True) # 16. the leg grades the WHOLE archive against its own ledger -- the # fix for a newest-day floor that made a backdated fault invisible. av = os.path.join(td, "c16-argv") repo = _fixture(os.path.join(td, "c16"), GREEN5, argv_markers={"checkin_check.py": av}, checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md")) rc, out = _verify("--repo", repo, "--fast") seen = open(av).read() if os.path.exists(av) else "" check("16 the check-in leg is run with --fail", "--fail" in seen, True) check("16b and with the ledger resolved from --repo, so no day is excluded", "--baseline" in seen and os.path.join(repo, "tools", "checkin_baseline.tsv") in seen and "--since" not in seen, True) # 17. --checkin-day still narrows the window explicitly, in the open av = os.path.join(td, "c17-argv") repo = _fixture(os.path.join(td, "c17"), GREEN5, argv_markers={"checkin_check.py": av}, checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md")) rc, out = _verify("--repo", repo, "--fast", "--checkin-day", "2026-01-01") seen = open(av).read() if os.path.exists(av) else "" check("17 --checkin-day adds a --since floor", "--since 2026-01-01" in seen, True) # 19. a ledger that cannot be read is UNGRADABLE (rc 2), never a pass: # the tool's own reason has to reach the row, because "no ledger" and # "clean archive" print nothing alike and must not grade alike. repo = _fixture(os.path.join(td, "c19"), dict(GREEN5, **{"checkin_check.py": 2}), outs={"checkin_check.py": "cannot grade: baseline %s/tools/checkin_baseline.tsv: " "[Errno 2] No such file or directory\n" % os.path.join(td, "c19", "repo")}) rc, out = _verify("--repo", repo, "--fast") check("19 a missing ledger is rc 2 and names the ledger", rc, 2) check("19b the ungradable row carries the tool's own reason", "ungradable: checkin" in out and "checkin_baseline.tsv" in out, True) # 18. the detail line must survive a real tool's noise: `checkin_check` # prints a README SKIP note carrying "not a " on every run, and a WEAK # hint must not beat the FAIL line that actually names the verdict. noisy = ("== 2 check-in files in /tmp/x/checkins ==\n" "roster: 9 handles; today 2026-10-19\n" "since 2026-10-19: 2 graded, 72 older file(s) not graded\n" " SKIP README.md (the directory's own README: not a check-in)\n" "\n" "2026-10-19-volt-02.md BAD_NAME,DUPLICATE 2 files for volt\n" "checkin_check: 2 files, 2 with a fault (BAD_NAME 1)\n" "FAIL: 2 file(s) violate the check-in contract\n") check("18 a README skip note does not mask the FAIL line", pick_detail(noisy), "FAIL: 2 file(s) violate the check-in contract") # 18b. and on a clean run the same noise must not become the detail: # the row reads PASS beside the summary, not beside a README note. clean = noisy.replace("checkin_check: 2 files, 2 with a fault (BAD_NAME 1)\n" "FAIL: 2 file(s) violate the check-in contract\n", "checkin_check: 2 files, 0 with a fault, 1 advisory\n") check("18b a clean check-in row carries its summary, not the skip note", pick_detail(clean), "checkin_check: 2 files, 0 with a fault, 1 advisory") # 20. THE FIFTH LEG: a reply whose heading names the wrong party is a # FAULT on the gate and the row is named -- the class that was invisible # for as long as `replies/` had no reader. repo = _fixture(os.path.join(td, "c20"), dict(GREEN5, **{"reply_check.py": 1}), outs={"reply_check.py": "reply_check: 192 file(s), 1 new fault(s), " "0 known-faulting in the baseline\n"}) rc, out = _verify("--repo", repo, "--fast") check("20 a new mis-addressed reply -> rc 1", rc, 1) check("20b the fault row is named reply-headings and carries its reason", "fault: reply-headings" in out and "1 new fault(s)" in out, True) # 21. the ledger is read from the tree being graded, not from the host av = os.path.join(td, "c21-argv") repo = _fixture(os.path.join(td, "c21"), GREEN5, argv_markers={"reply_check.py": av}) rc, out = _verify("--repo", repo, "--fast") seen = open(av).read() if os.path.exists(av) else "" check("21 the reply leg is run with the repo's own ledger", "--baseline" in seen and os.path.join(repo, "tools", "reply_baseline.tsv") in seen, True) # 22. on a PASS row the detail is the tool's own concluding summary, not # the first line that happens to contain a hint substring. reply_noise = ("reply_check -- 192 file(s) under /tmp/r (roster: x)\n" " NO_RECIPIENT_IN_HEADING 7\n" " 2026-10-12-to-merlin-19.md | heading: # the precondition refuses\n" "reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline\n") check("22 a clean row's detail is the tool's summary", pick_detail(reply_noise, summary=True), "reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline") check("22b a fault run still prefers the reason", pick_detail("cannot grade: baseline x: No such file\n" "tool: 1 new fault(s)\n"), "cannot grade: baseline x: No such file") print("selftest: %d cells, %d passed, %d failed" % (cells, passed, len(failures))) for f in failures: print(" FAIL %s" % f) return 0 if passed == cells else 1 if __name__ == "__main__": sys.exit(main())