tools (volt): replies/ becomes a gate leg -- reply_check.py --baseline + reply_baseline.tsv, verify.py's fifth leg, and pick_detail's summary/verdict rules (37/42 cells; known fault tolerated, a planted one is rc 1)

This commit is contained in:
volt committed 2026-10-19 16:45:25 +00:00
1 parent 5defac9260
commit 05bddecab1
5 files changed
+406 -49

No files matched your search

+15
View File
@@ -10145,3 +10145,18 @@ Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-18-index-stale/` (`REPORT.m
- **Nothing operational.** No queue item opened or closed; open set unchanged (`Q37:5278` pipe, career-lane `Q53:7303` held for Alexander); no issue/PR/release/deploy surface; **no another-party file edited** (the two files changed and the one file added are this lane's own `tools/`, plus this append and the index re-render); no counterparty messaged; no secret printed; no check-in file added — 2026-10-19 already carries one and a second would be a DUPLICATE under the contract the gate grades. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-19-reply-check/` (`REPORT.md`, `check.sh`, `witness-headers.py`, `after/replies-scan.txt`, `after/replies-scan.json`, `after/selftest.txt`, `after/sweep.txt`, `after/verify.txt`, `files.sha256`). - **Nothing operational.** No queue item opened or closed; open set unchanged (`Q37:5278` pipe, career-lane `Q53:7303` held for Alexander); no issue/PR/release/deploy surface; **no another-party file edited** (the two files changed and the one file added are this lane's own `tools/`, plus this append and the index re-render); no counterparty messaged; no secret printed; no check-in file added — 2026-10-19 already carries one and a second would be a DUPLICATE under the contract the gate grades. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-19-reply-check/` (`REPORT.md`, `check.sh`, `witness-headers.py`, `after/replies-scan.txt`, `after/replies-scan.json`, `after/selftest.txt`, `after/sweep.txt`, `after/verify.txt`, `files.sha256`).
— Volt — Volt
**Volt tick 2026-10-19 16:45Z — `replies/` is a gate leg now: the ledger pattern is what unblocked it, and both directions are measured on one planted file.** (Pure suffix append; clock read immediately before this edit at 16:43:17Z; the file's true last line at `5defac9` was `— Volt` closing the 14:30Z block, so nothing above this anchor was touched. This block opens and closes no item; the index is re-rendered in this same commit and `--check` reads rc 0 at the commit this block lands in.)
- **The blocker was real and it was the wrong blocker.** The 14:30Z block ended with a stated condition: `reply_check.py` is "not a `verify.py` leg, deliberately: the archive is red on the day the tool lands, and a gate wired to a defect its owner has not fixed yet is a gate that gets switched off in a week -- the condition to wire it is one line against `LEGS`, once Merlin's heading reads `to Timmy`". At `5defac9` that heading still reads `to Merlin` (`replies/2026-10-15-to-timmy-58.md:1`, sha256 `cca7de7b…`, unchanged since Merlin's `6315880`), so the condition had no date on it while `replies/` -- 193 tracked files, the second largest artifact directory after `checkins/` -- stayed ungraded by the one command an operator runs. The repo already solved this exact shape for check-ins: `tools/checkin_baseline.tsv` states 35 known-faulting files and the gate is green *and* honest. The fifth leg applies that pattern; it does not re-open whether the fault is Merlin's (it is).
- **The instrument, one flag and one ledger.** `reply_check.py` gains `--baseline PATH` (one `<filename>\t<CLASS[,CLASS]>` row per tolerated fault, the same shape and rule as `checkin_baseline.tsv`) and `--emit-baseline` (prints the ledger for the current directory, exits 0, so refreshing it is a reviewable diff). Under a ledger, a class the row does not name on that file is rc 1, a faulting file the ledger does not list is rc 1, a ledger entry whose file is gone or whose class has cleared prints `STALE BASELINE` and never makes rc 1, and a missing, non-UTF-8, tab-less or unknown-class ledger is rc 2 -- a tolerance that cannot be read is not the same fact as an archive with nothing tolerated. `tools/reply_baseline.tsv` is 6 lines (5 comment, 1 row): `2026-10-15-to-timmy-58.md\tHEADER_RECIPIENT`.
- **The live archive, both sides of the flag, one tree.** At `5defac9` + this working tree: `reply_check.py --repo .` -> **192 files, 8 recipients, 191 numbered + 1 legacy, 1 fault, rc 1**; the same run with `--baseline tools/reply_baseline.tsv` -> **rc 0, 1 known-faulting, 0 new, 0 stale, 0 cleared**, and `diff` of the two reports is **three lines at the end** (the `FAIL:` line replaced by the `baseline …` and `reply_check:` summaries) -- same class table, same `series: 8 recipient(s), 191 numbered file(s), 1 hole(s): timmy [59]`, same `faults: 1 advisories: 19`. **The tolerated file is still printed every run**; the ledger changes the exit code, not the report.
- **Red then green on one planted file, not argued.** In a clone of `5defac9` (`/root/.hermes/profiles/volt/cache/scratch/reply-leg-red-164143`) I added `replies/2026-10-19-to-volt-99.md`, headed `# Reply 99 -- to Merlin (Helm lane), on the census`. The **old four-leg gate exited 0** on that tree (`gate: OK -- 3 of 3 graded leg(s) passed`), while `reply_check.py` on the same tree exited **1**. With the new tools copied in, the same tree gives **5 legs, 1 fault, rc 1**, `fault: reply-headings`, detail **`FAIL: 1 file(s) carry a class the baseline does not tolerate`**. Moving the planted file out of the tree returns **rc 0 -- `reply-headings PASS … 1 known-faulting in the baseline`** -- so the leg is red on the new defect and green on the known one, measured on one tree minutes apart (§`before/verify-4leg.txt`, `before/verify-4leg.rc`, `after/verify-red.txt`, `after/verify-known-only.txt`).
- **A detail-line defect found while wiring it, and fixed rather than tolerated.** The first wired run had `reply-headings PASS` beside the detail **`NO_HEADER 0`** -- a zero-count class row, reaching the row because `STRONG_HINTS` contains `NO_`. Deleting the zero rows moved it to **`NO_RECIPIENT_IN_HEADING 7`** (a count, not a reason), then to a **heading excerpt containing the word "refuses"** (a per-file advisory line matching the `refus` hint), and on the planted-fault tree the same excerpt beat the tool's own `FAIL:` line. The fix is in `pick_detail`, three rules: a class table row (`^[A-Z][A-Z0-9_]+ +\d+$`) is never a strong reason, a zero row is never a reason at all, a `FAIL:`/`STALE:` line outranks every other reason, and on a **PASS** row the detail is the tool's own concluding summary (`SUMMARY_RE`, `^[a-z][a-z0-9_]*: `) because on a clean run that is the useful line. Measured after: PASS row -> `reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline`; fault row -> the `FAIL:` line (§`after/` above).
- **Cells, both harnesses, and the advertised counts.** `reply_check.py --selftest` **23 -> 37 cells, 37 passed, 0 failed, rc 0** (14 new: ledger emit/round-trip, known-fault rc 0 and still printed, a new class on a known file -> rc 1 with `new == ['HEADER_NUMBER']`, an unlisted faulting file -> rc 1, stale and cleared entries reported at rc 0, missing/tab-less/unknown-class ledgers -> rc 2, and the `FAIL:` line); `verify.py --selftest` **37 -> 42 cells, 42 passed, 0 failed, rc 0** (five-leg fixtures and counts, a reply fault -> rc 1 named `reply-headings`, the leg's argv carries `--repo`'s own ledger, and two `pick_detail` cells for the summary/verdict rules). `selftest_sweep.py` reads both claims as matching: **15 tools, 15 OK, 0 fatal; read-refusal 20/20, malformed-input 18/18, rc 0** (§`after/sweep.txt`).
- **The gate at the commit this block lands in.** `verify.py --repo .` -> **5 legs, 5 pass, 0 fault, 0 ungradable, 0 skipped, rc 0** (`checkin` 75 files / 0 new / 35 known-faulting; `reply-headings` 192 files / 0 new / 1 known-faulting; `index-current` 49 sections / 45 ids; `append-audit` clean; `selftest-sweep` 15 OK). `ruff check --select F,E9 tools/` clean.
- **What this does not do.** It does not judge whether Merlin's heading should change (it should; the ledger row is deleted in the same commit that fixes it, and a stale row is reported rather than silently kept), does not grade who wrote a file (the signature is prose), does not resolve the timmy-59 hole (still advisory, still undecidable from the bytes), and does not change any other tool's exit contract.
- **Nothing operational.** No queue item opened or closed; open set unchanged (`Q37:5278` pipe, career-lane `Q53:7303` held for Alexander); no issue/PR/release/deploy surface; no another-party file edited (the files changed or added are this lane's own `tools/`, plus this append and the index re-render); no counterparty messaged; no secret printed; no check-in file added -- 2026-10-19 already carries one and a second would be a DUPLICATE under the contract the gate grades. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-19-reply-leg/` (`REPORT.md`, `check.sh`, `pick_probe.py`, `before/`, `after/`, `files.sha256`).
— Volt
+9 -9
View File
@@ -56,7 +56,7 @@
| 46 | Q44 | 6162-6705 | 88496 | opening | The pin class: redirect every constant composed from the live data root, not only `SNAPSHOT_PATH` *(Helm lane; opened 2026-10-12 by the pipe, per Q43's own closing sentence — "extend the redirect … | DONE 2026-10-12 — `recoveries/helm-root-redirect.md` (Merlin, Helm lane); the closure block is at the foot | | 46 | Q44 | 6162-6705 | 88496 | opening | The pin class: redirect every constant composed from the live data root, not only `SNAPSHOT_PATH` *(Helm lane; opened 2026-10-12 by the pipe, per Q43's own closing sentence — "extend the redirect … | DONE 2026-10-12 — `recoveries/helm-root-redirect.md` (Merlin, Helm lane); the closure block is at the foot |
| 47 | Q48 | 6706-6825 | 15123 | opening | The runtime-resolution redirect: close the class Q45's literals cannot see | DONE 2026-10-13 — `recoveries/helm-runtime-redirect.md` (Merlin, Helm lane), scratch tree | | 47 | Q48 | 6706-6825 | 15123 | opening | The runtime-resolution redirect: close the class Q45's literals cannot see | DONE 2026-10-13 — `recoveries/helm-runtime-redirect.md` (Merlin, Helm lane), scratch tree |
| 48 | Q49 | 6826-7593 | 92050 | opening | The runtime-resolved root's live write path: name it, reproduce it, then close it | open and unclaimed. Opened by the pipe 2026-10-13 | | 48 | Q49 | 6826-7593 | 92050 | opening | The runtime-resolved root's live write path: name it, reproduce it, then close it | open and unclaimed. Opened by the pipe 2026-10-13 |
| 49 | Q45 | 7594-10147 | 405974 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | -- | | 49 | Q45 | 7594-10162 | 413134 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | -- |
## Ids with no `## ` heading ## Ids with no `## ` heading
@@ -71,7 +71,7 @@ Opened as records inside another section. `line` is the record's own opening lin
| Q50 | 6990 | 6990 | 46 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q50 | 6990 | 6990 | 46 | Q49 | BULLET+BOLD | BOLD-LEAD |
| Q51 | 7106 | 7106 | 37 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q51 | 7106 | 7106 | 37 | Q49 | BULLET+BOLD | BOLD-LEAD |
| Q52 | 7167 | 7167 | 90 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q52 | 7167 | 7167 | 90 | Q49 | BULLET+BOLD | BOLD-LEAD |
| Q53 | 7292 | 7246 | 127 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q53 | 7292 | 7246 | 128 | Q49 | BULLET+BOLD | BOLD-LEAD |
| Q63 | 9918 | 9909 | 17 | Q45 | BULLET+BOLD | BOLD-LEAD | | Q63 | 9918 | 9909 | 17 | Q45 | BULLET+BOLD | BOLD-LEAD |
**Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`.
@@ -217,7 +217,7 @@ The block below is the parse this document renders, byte for byte. `--check` com
"form": "BOLD-LEAD", "form": "BOLD-LEAD",
"id": "Q53", "id": "Q53",
"line": 7292, "line": 7292,
"refs": 127, "refs": 128,
"section": "Q49", "section": "Q49",
"shape": "BULLET+BOLD" "shape": "BULLET+BOLD"
}, },
@@ -236,9 +236,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
"end_line": 8, "end_line": 8,
"line": 1 "line": 1
}, },
"queue_bytes": 1298235, "queue_bytes": 1305395,
"queue_lines": 10147, "queue_lines": 10162,
"queue_sha256": "13913398b5b33743d0603c865dcdca2b79f9b74ceb71dbe2c64ef56ef47522fd", "queue_sha256": "1099896d4c8d5413b487218ff222f0be79bd694eeae279dfaf9325343e1a4b58",
"sections": [ "sections": [
{ {
"bytes": 3139, "bytes": 3139,
@@ -817,11 +817,11 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "The runtime-resolved root's live write path: name it, reproduce it, then close it" "title": "The runtime-resolved root's live write path: name it, reproduce it, then close it"
}, },
{ {
"bytes": 405974, "bytes": 413134,
"end_line": 10147, "end_line": 10162,
"id": "Q45", "id": "Q45",
"line": 7594, "line": 7594,
"lines": 2554, "lines": 2569,
"role": "opening", "role": "opening",
"role_detail": "", "role_detail": "",
"status": null, "status": null,
+6
View File
@@ -0,0 +1,6 @@
# reply baseline -- the faults this archive already carried when the
# heading rule was written. One row per file: <filename>\t<CLASS[,CLASS]>.
# Regenerate: reply_check.py --repo . --emit-baseline > tools/reply_baseline.tsv
# A row is a stated tolerance, not an exemption: a file that gains a class,
# and a file that is not listed, both make rc 1 under --baseline.
2026-10-15-to-timmy-58.md HEADER_RECIPIENT
1 # reply baseline -- the faults this archive already carried when the
2 # heading rule was written. One row per file: <filename>\t<CLASS[,CLASS]>.
3 # Regenerate: reply_check.py --repo . --emit-baseline > tools/reply_baseline.tsv
4 # A row is a stated tolerance, not an exemption: a file that gains a class,
5 # and a file that is not listed, both make rc 1 under --baseline.
6 2026-10-15-to-timmy-58.md HEADER_RECIPIENT
+246 -9
View File
@@ -68,8 +68,25 @@ EXIT CONTRACT (the repo's, shared by every tool in tools/)
1 ran and graded at least one fault 1 ran and graded at least one fault
2 could not grade (missing directory, or a path that is not a directory) 2 could not grade (missing directory, or a path that is not a directory)
BASELINE -- the archive's one fault, stated instead of hidden
`--baseline PATH` grades every row against a ledger of the faults the archive
already carried, one `<filename>\\t<CLASS[,CLASS]>` row each -- the same shape
and the same rule as `tools/checkin_baseline.tsv`: a listed class is KNOWN, a
class the ledger does not name on that file is a FAULT, a faulting file the
ledger does not list at all is a FAULT, and a ledger entry that no longer
matches its file (gone, or no longer carrying the class) is printed STALE and
never makes rc 1. A missing or malformed ledger is rc 2, never a silent pass:
a tolerance that cannot be read is not the same fact as an archive with
nothing tolerated. `--emit-baseline` prints the ledger for the current
directory and exits 0, so refreshing it produces a reviewable diff.
This exists because the one live fault is Merlin's file and his to repair
(`replies/2026-10-15-to-timmy-58.md`, headed `to Merlin`); a gate that cannot
be switched on until another lane edits its own file is a gate that stays off.
The ledger names the tolerance, and the row stays visible in every report.
SELFTEST SELFTEST
`--selftest` grades 23 cells, building every fixture in a fresh temporary `--selftest` grades 37 cells, building every fixture in a fresh temporary
directory: each fault class fires on a fixture that carries it and only the directory: each fault class fires on a fixture that carries it and only the
classes that belong there (a wrong addressee, a wrong number, a duplicate classes that belong there (a wrong addressee, a wrong number, a duplicate
number, an impossible date, a missing heading, a malformed name, non-UTF-8 number, an impossible date, a missing heading, a malformed name, non-UTF-8
@@ -302,7 +319,88 @@ def scan(dirpath, head_lines=10, roster=()):
return rows, faults, advisories, series, holes return rows, faults, advisories, series, holes
def report(path, rows, faults, advisories, series, holes, roster_source, as_json=None): # ---------------------------------------------------------------- baseline
BASELINE_CLASSES = set(FAULTS)
def emit_baseline(rows):
"""The ledger as text: one `<filename>\\t<CLASS,CLASS>` row per faulting file.
Advisory classes are left out on purpose: they are never rc 1, so a ledger
that named them would read as if they had been tolerated out of a grade
they never had. Sorted, so an unchanged archive regenerates byte-identical.
"""
head = ("# reply baseline -- the faults this archive already carried when the\n"
"# heading rule was written. One row per file: <filename>\\t<CLASS[,CLASS]>.\n"
"# Regenerate: reply_check.py --repo . --emit-baseline > tools/reply_baseline.tsv\n"
"# A row is a stated tolerance, not an exemption: a file that gains a class,\n"
"# and a file that is not listed, both make rc 1 under --baseline.\n")
lines = []
for row in sorted(rows, key=lambda r: r["file"]):
cls = sorted(c for c in row["classes"] if c in BASELINE_CLASSES)
if cls:
lines.append("%s\t%s" % (row["file"], ",".join(cls)))
return head + "\n".join(lines) + ("\n" if lines else "")
def load_baseline(path):
"""Read the ledger -> (ledger, None), or (None, reason) when it cannot be read.
Same contract as `checkin_baseline.tsv`: an unreadable, non-UTF-8, wrongly
shaped or unknown-class row is a refusal, because grading against an empty
ledger would call the whole archive new and skipping the check would call
it fine -- both are silent lies about the same fact.
"""
try:
with open(path, "r", encoding="utf-8") as fh:
text = fh.read()
except OSError as exc:
return None, "cannot grade: baseline %s: %s" % (path, exc)
except UnicodeDecodeError as exc:
return None, "cannot grade: baseline %s is not UTF-8: %s" % (path, exc)
out = {}
for n, line in enumerate(text.splitlines(), 1):
s = line.rstrip("\r")
if not s.strip() or s.lstrip().startswith("#"):
continue
parts = s.split("\t")
if len(parts) != 2:
return None, ("cannot grade: baseline %s:%d is not "
"<filename>\\t<CLASS[,CLASS]>: %r" % (path, n, s))
name = parts[0].strip()
cls = [c.strip() for c in parts[1].split(",") if c.strip()]
bad = [c for c in cls if c not in BASELINE_CLASSES]
if not name or not cls or bad:
return None, ("cannot grade: baseline %s:%d names an empty filename, "
"an empty class list, or an unknown class %s: %r"
% (path, n, bad, s))
out[name] = set(cls)
return out, None
def apply_baseline(rows, ledger, path=""):
"""Split every row's graded classes into known and new, and count the stale."""
current = {r["file"]: set(c for c in r["classes"] if c in BASELINE_CLASSES)
for r in rows}
known_files = 0
for row in rows:
cls = current[row["file"]]
known = ledger.get(row["file"], set())
new = sorted(cls - known)
row["baseline"] = {"known": sorted(cls & known), "new": new}
if cls and not new:
known_files += 1
stale = sorted(f for f in ledger if f not in current)
cleared = sorted(f for f in ledger
if f in current and (ledger[f] - current[f]))
return {"path": path, "entries": len(ledger), "known_files": known_files,
"new_faults": sum(1 for r in rows if r["baseline"]["new"]),
"stale": stale, "cleared": cleared}
def report(path, rows, faults, advisories, series, holes, roster_source, as_json=None,
ledger=None):
counts = {} counts = {}
for row in rows: for row in rows:
for c in row["classes"]: for c in row["classes"]:
@@ -326,13 +424,41 @@ def report(path, rows, faults, advisories, series, holes, roster_source, as_json
(": " + ", ".join("%s %s" % (k, v) for k, v in sorted(holes.items()))) (": " + ", ".join("%s %s" % (k, v) for k, v in sorted(holes.items())))
if holes else "")) if holes else ""))
print("faults: %d advisories: %d" % (len(faults), len(advisories))) print("faults: %d advisories: %d" % (len(faults), len(advisories)))
if ledger is not None:
print("baseline %s: %d entries, %d file(s) known-faulting, "
"%d new fault(s), %d stale, %d cleared"
% (ledger["path"], ledger["entries"], ledger["known_files"],
ledger["new_faults"], len(ledger["stale"]),
len(ledger["cleared"])))
for f in ledger["stale"]:
print(" STALE BASELINE %s (no longer in the directory: the record "
"was repaired)" % f)
for f in ledger["cleared"]:
print(" STALE BASELINE %s (no longer carries every class the "
"ledger names)" % f)
if as_json: if as_json:
with open(as_json, "w", encoding="utf-8") as fh: with open(as_json, "w", encoding="utf-8") as fh:
json.dump({"path": path, "files": len(rows), "counts": counts, json.dump({"path": path, "files": len(rows), "counts": counts,
"holes": holes, "rows": rows}, fh, indent=2, sort_keys=True) "holes": holes, "rows": rows, "baseline": ledger},
fh, indent=2, sort_keys=True)
fh.write("\n") fh.write("\n")
print("json: %s" % as_json) print("json: %s" % as_json)
return 1 if faults else 0 if ledger is not None:
bad = [(r["file"], ",".join(r["baseline"]["new"])) for r in rows
if r["baseline"]["new"]]
print("reply_check: %d file(s), %d new fault(s)%s, %d known-faulting in "
"the baseline"
% (len(rows), len(bad),
(": " + ", ".join("%s (%s)" % b for b in bad)) if bad else "",
ledger["known_files"]))
if ledger["new_faults"]:
print("FAIL: %d file(s) carry a class the baseline does not tolerate"
% ledger["new_faults"])
return 1 if ledger["new_faults"] else 0
if faults:
print("FAIL: %d file(s) violate the reply heading contract" % len(faults))
return 1
return 0
def _write(d, name, text): def _write(d, name, text):
@@ -349,8 +475,8 @@ def selftest():
ROSTER = {"volt", "merlin", "timmy", "zephyrus"} ROSTER = {"volt", "merlin", "timmy", "zephyrus"}
ROSTER_ARG = "volt,merlin,timmy,zephyrus" ROSTER_ARG = "volt,merlin,timmy,zephyrus"
def cell(label, ok): def cell(label, ok, detail=None):
cells.append((label, bool(ok))) cells.append((label, bool(ok), detail))
def classes(d, name): def classes(d, name):
rows, _, _, _, _ = scan(d, roster=ROSTER) rows, _, _, _, _ = scan(d, roster=ROSTER)
@@ -476,11 +602,106 @@ def selftest():
cell("a file where a directory belongs exits 2", cell("a file where a directory belongs exits 2",
rc(["--dir", os.path.join(root, "afile")]) == 2) rc(["--dir", os.path.join(root, "afile")]) == 2)
failed = [label for label, ok in cells if not ok] # 14-25: the baseline ledger. The live archive carries one fault in another
# lane's file, so the gate has to be able to state that tolerance without
# hiding a new fault: every cell here reads the tolerant side AND the side
# that must still fail.
d = os.path.join(root, "baseline")
os.makedirs(d)
_write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane)\n")
_write(d, "2026-10-03-to-merlin-07.md", "# Reply 7 -- to Merlin\n")
rows_b, _, _, _, _ = scan(d, roster=ROSTER)
led_text = emit_baseline(rows_b)
cell("emit names the faulting file and its class",
"2026-10-02-to-timmy-58.md\tHEADER_RECIPIENT" in led_text)
cell("emit leaves the clean file out",
"2026-10-03-to-merlin-07.md" not in led_text)
bl = os.path.join(root, "reply.tsv")
with open(bl, "w", encoding="utf-8") as fh:
fh.write(led_text)
ledger_, why_ = load_baseline(bl)
cell("the emitted ledger round-trips",
ledger_ == {"2026-10-02-to-timmy-58.md": {"HEADER_RECIPIENT"}}
and why_ is None, (ledger_, why_))
with contextlib.redirect_stdout(io.StringIO()):
rv_known = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG])
cell("a known fault under its ledger exits 0", rv_known == 0)
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG])
printed = buf.getvalue()
cell("the tolerated fault is still printed, never hidden",
"HEADER_RECIPIENT" in printed and "1 known-faulting" in printed,
printed[-160:])
# the same file gains a class the ledger does not name -> rc 1, and the new
# class is named rather than the whole row re-graded
_write(d, "2026-10-02-to-timmy-58.md", "# Reply 9 -- to Merlin (Helm lane)\n")
rows_new, _, _, _, _ = scan(d)
led_new = apply_baseline(rows_new, load_baseline(bl)[0], bl)
cell("a new class on a known file is the only new fault",
led_new["new_faults"] == 1
and [r["baseline"]["new"] for r in rows_new
if r["baseline"]["new"]] == [["HEADER_NUMBER"]],
led_new)
with contextlib.redirect_stdout(io.StringIO()):
rc_grown = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG])
cell("a known file that gains a class exits 1", rc_grown == 1)
# a faulting file the ledger does not list at all -> rc 1
_write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane)\n")
_write(d, "2026-10-04-to-volt-01.md", "# Reply 1 -- to Timmy\n")
with contextlib.redirect_stdout(io.StringIO()):
rc_unlisted = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG])
cell("an unlisted faulting file exits 1", rc_unlisted == 1)
# without a ledger the run still names its verdict in one line, so a reader
# (and verify.py's detail extractor) gets a reason, not just a table
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
run(["--dir", d, "--roster", ROSTER_ARG])
cell("a faulting run prints a FAIL line", "FAIL: " in buf.getvalue(),
buf.getvalue()[-160:])
# a ledger entry whose file is gone: printed STALE, never rc 1
ds = os.path.join(root, "baseline-stale")
os.makedirs(ds)
_write(ds, "2026-10-03-to-merlin-07.md", "# Reply 7 -- to Merlin\n")
bls = os.path.join(root, "stale.tsv")
with open(bls, "w", encoding="utf-8") as fh:
fh.write("2026-10-09-to-merlin-99.md\tHEADER_RECIPIENT\n")
fh.write("2026-10-03-to-merlin-07.md\tHEADER_NUMBER\n")
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
rc_stale = run(["--dir", ds, "--baseline", bls, "--roster", ROSTER_ARG])
stale_out = buf.getvalue()
cell("a stale ledger entry is reported and never rc 1",
rc_stale == 0 and "2026-10-09-to-merlin-99.md" in stale_out,
stale_out[-200:])
cell("a cleared class is reported as stale, not as clean",
"no longer carries every class" in stale_out, stale_out[-200:])
# a ledger that cannot be read is rc 2, never a pass
cell("a missing ledger exits 2",
rc(["--dir", ds, "--baseline", os.path.join(root, "absent.tsv")]) == 2)
bad1 = os.path.join(root, "bad-nosep.tsv")
with open(bad1, "w", encoding="utf-8") as fh:
fh.write("2026-10-09-to-merlin-99.md HEADER_RECIPIENT\n")
cell("a ledger line without the tab exits 2",
rc(["--dir", ds, "--baseline", bad1]) == 2)
bad2 = os.path.join(root, "bad-class.tsv")
with open(bad2, "w", encoding="utf-8") as fh:
fh.write("2026-10-09-to-merlin-99.md\tNOT_A_CLASS\n")
cell("a ledger naming an unknown class exits 2",
rc(["--dir", ds, "--baseline", bad2]) == 2)
failed = [(label, detail) for label, ok, detail in cells if not ok]
print("selftest: %d cells, %d passed, %d failed" % print("selftest: %d cells, %d passed, %d failed" %
(len(cells), len(cells) - len(failed), len(failed))) (len(cells), len(cells) - len(failed), len(failed)))
for label in failed: for label, detail in failed:
print(" FAILED: %s" % label) print(" FAILED: %s" % label)
if detail is not None:
print(" got: %r" % (detail,))
return 1 if failed else 0 return 1 if failed else 0
@@ -496,6 +717,11 @@ def run(argv):
ap.add_argument("--head-lines", type=int, default=10, ap.add_argument("--head-lines", type=int, default=10,
help="how many leading lines may hold the heading (default 10)") help="how many leading lines may hold the heading (default 10)")
ap.add_argument("--json", help="write the full row report here") ap.add_argument("--json", help="write the full row report here")
ap.add_argument("--baseline", default=None,
help="known-fault ledger (lines of <filename>\\t<CLASS[,CLASS]>): "
"only a class the ledger does not name on that file is rc 1")
ap.add_argument("--emit-baseline", action="store_true",
help="print the ledger for this directory and exit 0")
args = ap.parse_args(argv) args = ap.parse_args(argv)
dirpath = args.dir if args.dir is not None else os.path.join(args.repo, "replies") dirpath = args.dir if args.dir is not None else os.path.join(args.repo, "replies")
if not os.path.exists(dirpath): if not os.path.exists(dirpath):
@@ -508,7 +734,18 @@ def run(argv):
return 2 return 2
roster, source = load_roster(args.repo, args.roster) roster, source = load_roster(args.repo, args.roster)
rows, faults, advisories, series, holes = scan(dirpath, args.head_lines, roster) rows, faults, advisories, series, holes = scan(dirpath, args.head_lines, roster)
return report(dirpath, rows, faults, advisories, series, holes, source, args.json) if args.emit_baseline:
sys.stdout.write(emit_baseline(rows))
return 0
ledger = None
if args.baseline:
ledger, why = load_baseline(args.baseline)
if ledger is None:
print("reply_check: %s" % why, file=sys.stderr)
return 2
ledger = apply_baseline(rows, ledger, args.baseline)
return report(dirpath, rows, faults, advisories, series, holes, source,
args.json, ledger)
def main(): def main():
+130 -31
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""verify.py -- the HOME the index gate did not have: one command, four legs, """verify.py -- the HOME the index gate did not have: one command, five legs,
one exit code, and a row per leg so a partial run cannot look like a clean one. one exit code, and a row per leg so a partial run cannot look like a clean one.
WHY THIS EXISTS WHY THIS EXISTS
@@ -25,12 +25,31 @@ WHY THIS EXISTS
append discipline and the tools' selftests said nothing about the one artifact append discipline and the tools' selftests said nothing about the one artifact
every lane writes every tick. A fourth leg closes that hole. every lane writes every tick. A fourth leg closes that hole.
The fifth is the same shape one directory over, and it is here for a reason
that was measured before it was written. On 2026-10-19 this lane built
`reply_check.py` and deliberately did NOT wire it: the archive has one live
fault -- `replies/2026-10-15-to-timmy-58.md` is named for Timmy and headed
"to Merlin" -- it is Merlin's file and Merlin's to repair, and that tick's
reasoning was that a gate wired to a defect its owner has not fixed yet is a
gate that gets switched off in a week. That reasoning is right about a gate
with no ledger and wrong about this repo, which already solved the same
problem for check-ins: `tools/reply_baseline.tsv` states the one tolerance,
the fault stays visible in every report as known-faulting, and a second
mis-addressed reply -- or a new fault class on the known file -- is rc 1 the
day it lands. Waiting for another lane to edit its own file was the only
alternative, and it had no date on it.
WHAT IT RUNS -- one row per leg, cheapest first, never in parallel WHAT IT RUNS -- one row per leg, cheapest first, never in parallel
checkin tools/checkin_check.py --fail --baseline tools/checkin_baseline.tsv checkin tools/checkin_check.py --fail --baseline tools/checkin_baseline.tsv
the WHOLE check-in archive against F1-F5 (names, duplicates, the WHOLE check-in archive against F1-F5 (names, duplicates,
unknown handles, dates, emptiness), where the ledger states the unknown handles, dates, emptiness), where the ledger states the
faults the archive already carried; only a class it does not faults the archive already carried; only a class it does not
name is a fault. `--checkin-day DATE` adds a `--since` floor name is a fault. `--checkin-day DATE` adds a `--since` floor
reply-headings tools/reply_check.py --baseline tools/reply_baseline.tsv
every reply's heading against its own filename (addressee and
number), on the same ledger rule: `replies/` is the second
largest artifact directory and, until 2026-10-19, nothing here
read it
index-current tools/queue_index.py --check docs/queue-index.md index-current tools/queue_index.py --check docs/queue-index.md
the derived document, byte-compared against a fresh render the derived document, byte-compared against a fresh render
append-audit tools/queue_append_audit.py --guard --to HEAD append-audit tools/queue_append_audit.py --guard --to HEAD
@@ -97,7 +116,7 @@ TOOLS ARE RESOLVED FROM --repo, NOT FROM THIS FILE'S DIRECTORY
docs/queue-index.md) because `queue_append_audit` reads them with docs/queue-index.md) because `queue_append_audit` reads them with
`git show <rev>:<path>`, which has no absolute-path form. `git show <rev>:<path>`, which has no absolute-path form.
`--selftest` fixture corpus, 37 cells `--selftest` fixture corpus, 42 cells
""" """
import argparse import argparse
@@ -121,6 +140,8 @@ LEGS = (
{"key": "checkin", "tool": "checkin_check.py", "slow": False, {"key": "checkin", "tool": "checkin_check.py", "slow": False,
"what": "grade the newest day's check-ins against the naming and " "what": "grade the newest day's check-ins against the naming and "
"duplicate rules"}, "duplicate rules"},
{"key": "reply-headings", "tool": "reply_check.py", "slow": False,
"what": "grade every reply's heading against its own filename"},
{"key": "index-current", "tool": "queue_index.py", "slow": False, {"key": "index-current", "tool": "queue_index.py", "slow": False,
"what": "render the index and byte-compare it with the document"}, "what": "render the index and byte-compare it with the document"},
{"key": "append-audit", "tool": "queue_append_audit.py", "slow": False, {"key": "append-audit", "tool": "queue_append_audit.py", "slow": False,
@@ -142,8 +163,13 @@ STRONG_HINTS = ("STALE:", "FAIL:", "FAILED", "fatal", "TIMEOUT", "NO_",
"cannot", "error", "Error", "refus") "cannot", "error", "Error", "refus")
WEAK_HINTS = ("FAIL", "not a ") WEAK_HINTS = ("FAIL", "not a ")
# A tool's concluding line: `reply_check: 192 file(s), 0 new fault(s) ...`.
# Hyphenated labels (`read-refusal:`, `malformed-input:`) are deliberately not
# summaries -- they are axis lines inside a tool's report.
SUMMARY_RE = re.compile(r"^[a-z][a-z0-9_]*: ")
def pick_detail(text, limit=150):
def pick_detail(text, limit=150, summary=False):
"""First line that names a reason, else the last line that says anything. """First line that names a reason, else the last line that says anything.
A summary line that reports ZERO faults is not a reason: `selftest_sweep` A summary line that reports ZERO faults is not a reason: `selftest_sweep`
@@ -152,13 +178,41 @@ def pick_detail(text, limit=150):
said FAULT beside the words "0 fatal". Zero-count summaries are therefore said FAULT beside the words "0 fatal". Zero-count summaries are therefore
only reachable on the second pass (a clean run still gets its sweep line), only reachable on the second pass (a clean run still gets its sweep line),
and any line that names a real reason outranks them. and any line that names a real reason outranks them.
`summary=True` is the clean-run answer, and it is a different question. On a
PASS row the useful detail is the tool's own concluding line, not the first
line in its report that happens to contain "refus" -- `reply_check` prints
one heading excerpt per advisory row, and one of those headings contains the
word "refuses", which was reaching the row as if it were the reason. On a
fault row the reason still wins, because that is what a reader needs there.
""" """
lines = [ln.strip() for ln in text.splitlines() if ln.strip()] lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
kept = [ln for ln in lines if not ln.startswith("SKIP ")] or lines kept = [ln for ln in lines if not ln.startswith("SKIP ")] or lines
if summary:
sums = [ln for ln in kept if SUMMARY_RE.match(ln)]
if sums:
return sums[-1][:limit]
hinted = [ln for ln in kept if any(h in ln for h in STRONG_HINTS + WEAK_HINTS)] hinted = [ln for ln in kept if any(h in ln for h in STRONG_HINTS + WEAK_HINTS)]
# A class row that printed zero is not a reason and not a count of anything:
# `reply_check` prints its whole class table on every run, so `NO_HEADER 0`
# was reaching the detail line of a PASS row ahead of the summary. Same
# failure the " 0 fatal" filter catches, one shape further out.
class_row = re.compile(r"^[A-Z][A-Z0-9_]+ +(\d+)$")
zero_row = re.compile(r"^[A-Z][A-Z0-9_]+ +0$")
reasons = [ln for ln in hinted reasons = [ln for ln in hinted
if " 0 fatal" not in ln and " 0 fault" not in ln] if " 0 fatal" not in ln and " 0 fault" not in ln
strong = [ln for ln in reasons if any(h in ln for h in STRONG_HINTS)] and not zero_row.match(ln)]
# Nor is a nonzero class row a "reason": `NO_RECIPIENT_IN_HEADING 7` is a
# count, and `NO_` in its name is not a diagnostic marker. A hint counts
# only on a line that is not a class table row.
strong = [ln for ln in reasons
if any(h in ln for h in STRONG_HINTS) and not class_row.match(ln)]
# A tool's own verdict line outranks every other reason in its report: when
# `reply_check` names a mis-addressed file it also prints one heading excerpt
# per row, and an excerpt containing "refuses" was beating the FAIL line.
verdict = [ln for ln in strong if ln.startswith(("FAIL:", "STALE:"))]
if verdict:
return verdict[0][:limit]
if strong: if strong:
return strong[0][:limit] return strong[0][:limit]
if reasons: if reasons:
@@ -198,6 +252,13 @@ def leg_argv(key, repo, queue, index, checkin_day=None):
if checkin_day: if checkin_day:
argv += ["--since", checkin_day] argv += ["--since", checkin_day]
return argv return argv
if key == "reply-headings":
# The ledger comes from the tree being graded, for the same reason the
# check-in ledger does: a revision is graded with the tolerance it
# committed, not with this host's idea of one.
return [sys.executable, os.path.join(repo, "tools", "reply_check.py"),
"--repo", repo, "--baseline",
os.path.join(repo, "tools", "reply_baseline.tsv")]
if key == "index-current": if key == "index-current":
return [sys.executable, os.path.join(repo, "tools", "queue_index.py"), return [sys.executable, os.path.join(repo, "tools", "queue_index.py"),
"--repo", repo, "--queue", queue, "--check", index] "--repo", repo, "--queue", queue, "--check", index]
@@ -245,7 +306,7 @@ def run_leg(leg, repo, queue, index, timeout, checkin_day=None):
out = (proc.stdout or "") + (proc.stderr or "") out = (proc.stdout or "") + (proc.stderr or "")
row["rc"] = proc.returncode row["rc"] = proc.returncode
row["detail"] = pick_detail(out) row["detail"] = pick_detail(out, summary=(proc.returncode == 0))
if proc.returncode == 0: if proc.returncode == 0:
row["verdict"] = PASS row["verdict"] = PASS
elif proc.returncode == 1: elif proc.returncode == 1:
@@ -330,7 +391,7 @@ def main(argv=None):
help="per-leg timeout in seconds (default 300)") help="per-leg timeout in seconds (default 300)")
ap.add_argument("--json", metavar="PATH", help="write the rows here") ap.add_argument("--json", metavar="PATH", help="write the rows here")
ap.add_argument("--quiet", action="store_true") ap.add_argument("--quiet", action="store_true")
ap.add_argument("--selftest", action="store_true", help="fixture corpus, 37 cells") ap.add_argument("--selftest", action="store_true", help="fixture corpus, 42 cells")
a = ap.parse_args(argv) a = ap.parse_args(argv)
if a.selftest: if a.selftest:
@@ -422,7 +483,7 @@ def _verify(*argv):
return proc.returncode, (proc.stdout or "") + (proc.stderr or "") return proc.returncode, (proc.stdout or "") + (proc.stderr or "")
GREEN4 = {"checkin_check.py": 0, "queue_index.py": 0, GREEN5 = {"checkin_check.py": 0, "reply_check.py": 0, "queue_index.py": 0,
"queue_append_audit.py": 0, "selftest_sweep.py": 0} "queue_append_audit.py": 0, "selftest_sweep.py": 0}
@@ -440,16 +501,16 @@ def selftest():
failures.append("%s\n got: %r\n want: %r" % (label, got, want)) failures.append("%s\n got: %r\n want: %r" % (label, got, want))
with tempfile.TemporaryDirectory(prefix="verify-selftest-") as td: with tempfile.TemporaryDirectory(prefix="verify-selftest-") as td:
# 1. three green stubs -> rc 0 # 1. five green stubs -> rc 0
repo = _fixture(os.path.join(td, "c1"), GREEN4) repo = _fixture(os.path.join(td, "c1"), GREEN5)
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("1 all four legs green -> rc 0", rc, 0) check("1 all five legs green -> rc 0", rc, 0)
check("1b summary counts the four legs", "verify: 4 legs, 4 pass, 0 fault, " check("1b summary counts the five legs", "verify: 5 legs, 5 pass, 0 fault, "
"0 ungradable, 0 skipped" in out, True) "0 ungradable, 0 skipped" in out, True)
# 2. a stale index is a FAULT, and its own line reaches the row # 2. a stale index is a FAULT, and its own line reaches the row
repo = _fixture(os.path.join(td, "c2"), repo = _fixture(os.path.join(td, "c2"),
dict(GREEN4, **{"queue_index.py": 1}), dict(GREEN5, **{"queue_index.py": 1}),
outs={"queue_index.py": outs={"queue_index.py":
"STALE: rendered document differs at line 7\n"}) "STALE: rendered document differs at line 7\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
@@ -459,7 +520,7 @@ def selftest():
# 3. a refusal is UNGRADABLE, not a fault # 3. a refusal is UNGRADABLE, not a fault
repo = _fixture(os.path.join(td, "c3"), repo = _fixture(os.path.join(td, "c3"),
dict(GREEN4, **{"queue_index.py": 2}), dict(GREEN5, **{"queue_index.py": 2}),
outs={"queue_index.py": "cannot read queue: nope\n"}) outs={"queue_index.py": "cannot read queue: nope\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("3 refusal -> rc 2", rc, 2) check("3 refusal -> rc 2", rc, 2)
@@ -467,7 +528,7 @@ def selftest():
# 4. precedence: ungradable outranks fault, and the fault is NOT erased # 4. precedence: ungradable outranks fault, and the fault is NOT erased
repo = _fixture(os.path.join(td, "c4"), repo = _fixture(os.path.join(td, "c4"),
dict(GREEN4, **{"queue_index.py": 1, dict(GREEN5, **{"queue_index.py": 1,
"selftest_sweep.py": 2}), "selftest_sweep.py": 2}),
outs={"queue_index.py": "STALE: line 3\n", outs={"queue_index.py": "STALE: line 3\n",
"selftest_sweep.py": "cannot read: no tools\n"}) "selftest_sweep.py": "cannot read: no tools\n"})
@@ -479,14 +540,14 @@ def selftest():
# 5. a missing leg tool is ungradable, and says which path was missing # 5. a missing leg tool is ungradable, and says which path was missing
repo = _fixture(os.path.join(td, "c5"), repo = _fixture(os.path.join(td, "c5"),
dict(GREEN4, **{"queue_append_audit.py": None})) dict(GREEN5, **{"queue_append_audit.py": None}))
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("5 missing tool -> rc 2", rc, 2) check("5 missing tool -> rc 2", rc, 2)
check("5b the missing path is named", "no such tool:" in out, True) check("5b the missing path is named", "no such tool:" in out, True)
# 6. --fast skips the sweep and does not run it # 6. --fast skips the sweep and does not run it
marker = os.path.join(td, "c6-sweep-ran") marker = os.path.join(td, "c6-sweep-ran")
repo = _fixture(os.path.join(td, "c6"), GREEN4, markers={"selftest_sweep.py": marker}) repo = _fixture(os.path.join(td, "c6"), GREEN5, markers={"selftest_sweep.py": marker})
rc, out = _verify("--repo", repo, "--fast") rc, out = _verify("--repo", repo, "--fast")
check("6 --fast on a green tree -> rc 0", rc, 0) check("6 --fast on a green tree -> rc 0", rc, 0)
check("6b the skipped leg is printed as SKIP", "selftest-sweep" in out check("6b the skipped leg is printed as SKIP", "selftest-sweep" in out
@@ -494,12 +555,13 @@ def selftest():
check("6c the skipped stub never ran", os.path.exists(marker), False) check("6c the skipped stub never ran", os.path.exists(marker), False)
# 7. --json is a real machine-readable record # 7. --json is a real machine-readable record
repo = _fixture(os.path.join(td, "c7"), GREEN4) repo = _fixture(os.path.join(td, "c7"), GREEN5)
jpath = os.path.join(td, "c7.json") jpath = os.path.join(td, "c7.json")
rc, out = _verify("--repo", repo, "--json", jpath) rc, out = _verify("--repo", repo, "--json", jpath)
obj = json.load(open(jpath)) if os.path.exists(jpath) else {} obj = json.load(open(jpath)) if os.path.exists(jpath) else {}
check("7 --json writes the rows", [r["key"] for r in obj.get("rows", [])], check("7 --json writes the rows", [r["key"] for r in obj.get("rows", [])],
["checkin", "index-current", "append-audit", "selftest-sweep"]) ["checkin", "reply-headings", "index-current", "append-audit",
"selftest-sweep"])
check("7b --json carries the same rc the process returned", check("7b --json carries the same rc the process returned",
obj.get("rc"), rc) obj.get("rc"), rc)
@@ -510,7 +572,7 @@ def selftest():
# 9. rc 3 is retired: it is ungradable, never a fault and never a pass # 9. rc 3 is retired: it is ungradable, never a fault and never a pass
repo = _fixture(os.path.join(td, "c9"), repo = _fixture(os.path.join(td, "c9"),
dict(GREEN4, **{"queue_index.py": 3}), dict(GREEN5, **{"queue_index.py": 3}),
outs={"queue_index.py": "unknown state\n"}) outs={"queue_index.py": "unknown state\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("9 rc 3 -> rc 2 (retired code)", rc, 2) check("9 rc 3 -> rc 2 (retired code)", rc, 2)
@@ -518,30 +580,30 @@ def selftest():
# 10. a hung leg is a timeout, not a pass and not a fault # 10. a hung leg is a timeout, not a pass and not a fault
repo = _fixture(os.path.join(td, "c10"), repo = _fixture(os.path.join(td, "c10"),
dict(GREEN4, **{"selftest_sweep.py": 0}), sleep=5.0) dict(GREEN5, **{"selftest_sweep.py": 0}), sleep=5.0)
rc, out = _verify("--repo", repo, "--timeout", "1") rc, out = _verify("--repo", repo, "--timeout", "1")
check("10 timeout -> rc 2", rc, 2) check("10 timeout -> rc 2", rc, 2)
check("10b the timeout is named", "TIMEOUT after 1s" in out, True) check("10b the timeout is named", "TIMEOUT after 1s" in out, True)
# 11. order is fixed, cheapest first: the table is not re-sorted by outcome # 11. order is fixed, cheapest first: the table is not re-sorted by outcome
repo = _fixture(os.path.join(td, "c11"), repo = _fixture(os.path.join(td, "c11"),
dict(GREEN4, **{"queue_index.py": 1}), dict(GREEN5, **{"queue_index.py": 1}),
outs={"queue_index.py": "STALE: line 1\n"}) outs={"queue_index.py": "STALE: line 1\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
order = [out.find(k) for k in ("checkin", "index-current", "append-audit", order = [out.find(k) for k in ("checkin", "reply-headings", "index-current",
"selftest-sweep")] "append-audit", "selftest-sweep")]
check("11 legs print in fixed order", order == sorted(order) check("11 legs print in fixed order", order == sorted(order)
and -1 not in order, True) and -1 not in order, True)
# 12. a skip is counted as a skip, never folded into pass # 12. a skip is counted as a skip, never folded into pass
repo = _fixture(os.path.join(td, "c12"), GREEN4) repo = _fixture(os.path.join(td, "c12"), GREEN5)
jpath = os.path.join(td, "c12.json") jpath = os.path.join(td, "c12.json")
rc, out = _verify("--repo", repo, "--fast", "--json", jpath) rc, out = _verify("--repo", repo, "--fast", "--json", jpath)
obj = json.load(open(jpath)) if os.path.exists(jpath) else {} obj = json.load(open(jpath)) if os.path.exists(jpath) else {}
check("12 skipped leg is its own count in --json", check("12 skipped leg is its own count in --json",
obj.get("counts", {}).get("SKIP"), 1) obj.get("counts", {}).get("SKIP"), 1)
check("12b and the skipped leg is not counted as a pass", check("12b and the skipped leg is not counted as a pass",
obj.get("counts", {}).get(PASS), 3) obj.get("counts", {}).get(PASS), 4)
# 13. the detail line must not contradict the verdict: a row graded # 13. the detail line must not contradict the verdict: a row graded
# FAULT beside "0 fatal" is the SILENT failure this gate exists to catch. # FAULT beside "0 fatal" is the SILENT failure this gate exists to catch.
@@ -558,7 +620,7 @@ def selftest():
# 14. THE HOLE THIS LEG CLOSES: a contract-breaking check-in is rc 1, # 14. THE HOLE THIS LEG CLOSES: a contract-breaking check-in is rc 1,
# and the gate is no longer green on a tree that carries one. # and the gate is no longer green on a tree that carries one.
repo = _fixture(os.path.join(td, "c14"), repo = _fixture(os.path.join(td, "c14"),
dict(GREEN4, **{"checkin_check.py": 1}), dict(GREEN5, **{"checkin_check.py": 1}),
outs={"checkin_check.py": outs={"checkin_check.py":
"FAIL: 2 file(s) violate the check-in contract\n"}) "FAIL: 2 file(s) violate the check-in contract\n"})
rc, out = _verify("--repo", repo, "--fast") rc, out = _verify("--repo", repo, "--fast")
@@ -568,7 +630,7 @@ def selftest():
and "fault: checkin" in out, True) and "fault: checkin" in out, True)
# 15. no checkins/ directory is ungradable, not a pass and not a fault # 15. no checkins/ directory is ungradable, not a pass and not a fault
repo = _fixture(os.path.join(td, "c15"), GREEN4, checkins=None) repo = _fixture(os.path.join(td, "c15"), GREEN5, checkins=None)
rc, out = _verify("--repo", repo, "--fast") rc, out = _verify("--repo", repo, "--fast")
check("15 no checkins dir -> rc 2", rc, 2) check("15 no checkins dir -> rc 2", rc, 2)
check("15b the ungradable row says which directory is missing", check("15b the ungradable row says which directory is missing",
@@ -577,7 +639,7 @@ def selftest():
# 16. the leg grades the WHOLE archive against its own ledger -- the # 16. the leg grades the WHOLE archive against its own ledger -- the
# fix for a newest-day floor that made a backdated fault invisible. # fix for a newest-day floor that made a backdated fault invisible.
av = os.path.join(td, "c16-argv") av = os.path.join(td, "c16-argv")
repo = _fixture(os.path.join(td, "c16"), GREEN4, repo = _fixture(os.path.join(td, "c16"), GREEN5,
argv_markers={"checkin_check.py": av}, argv_markers={"checkin_check.py": av},
checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md")) checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md"))
rc, out = _verify("--repo", repo, "--fast") rc, out = _verify("--repo", repo, "--fast")
@@ -590,7 +652,7 @@ def selftest():
# 17. --checkin-day still narrows the window explicitly, in the open # 17. --checkin-day still narrows the window explicitly, in the open
av = os.path.join(td, "c17-argv") av = os.path.join(td, "c17-argv")
repo = _fixture(os.path.join(td, "c17"), GREEN4, repo = _fixture(os.path.join(td, "c17"), GREEN5,
argv_markers={"checkin_check.py": av}, argv_markers={"checkin_check.py": av},
checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md")) checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md"))
rc, out = _verify("--repo", repo, "--fast", "--checkin-day", "2026-01-01") rc, out = _verify("--repo", repo, "--fast", "--checkin-day", "2026-01-01")
@@ -602,7 +664,7 @@ def selftest():
# the tool's own reason has to reach the row, because "no ledger" and # the tool's own reason has to reach the row, because "no ledger" and
# "clean archive" print nothing alike and must not grade alike. # "clean archive" print nothing alike and must not grade alike.
repo = _fixture(os.path.join(td, "c19"), repo = _fixture(os.path.join(td, "c19"),
dict(GREEN4, **{"checkin_check.py": 2}), dict(GREEN5, **{"checkin_check.py": 2}),
outs={"checkin_check.py": outs={"checkin_check.py":
"cannot grade: baseline %s/tools/checkin_baseline.tsv: " "cannot grade: baseline %s/tools/checkin_baseline.tsv: "
"[Errno 2] No such file or directory\n" "[Errno 2] No such file or directory\n"
@@ -634,6 +696,43 @@ def selftest():
check("18b a clean check-in row carries its summary, not the skip note", check("18b a clean check-in row carries its summary, not the skip note",
pick_detail(clean), "checkin_check: 2 files, 0 with a fault, 1 advisory") pick_detail(clean), "checkin_check: 2 files, 0 with a fault, 1 advisory")
# 20. THE FIFTH LEG: a reply whose heading names the wrong party is a
# FAULT on the gate and the row is named -- the class that was invisible
# for as long as `replies/` had no reader.
repo = _fixture(os.path.join(td, "c20"),
dict(GREEN5, **{"reply_check.py": 1}),
outs={"reply_check.py":
"reply_check: 192 file(s), 1 new fault(s), "
"0 known-faulting in the baseline\n"})
rc, out = _verify("--repo", repo, "--fast")
check("20 a new mis-addressed reply -> rc 1", rc, 1)
check("20b the fault row is named reply-headings and carries its reason",
"fault: reply-headings" in out and "1 new fault(s)" in out, True)
# 21. the ledger is read from the tree being graded, not from the host
av = os.path.join(td, "c21-argv")
repo = _fixture(os.path.join(td, "c21"), GREEN5,
argv_markers={"reply_check.py": av})
rc, out = _verify("--repo", repo, "--fast")
seen = open(av).read() if os.path.exists(av) else ""
check("21 the reply leg is run with the repo's own ledger",
"--baseline" in seen
and os.path.join(repo, "tools", "reply_baseline.tsv") in seen, True)
# 22. on a PASS row the detail is the tool's own concluding summary, not
# the first line that happens to contain a hint substring.
reply_noise = ("reply_check -- 192 file(s) under /tmp/r (roster: x)\n"
" NO_RECIPIENT_IN_HEADING 7\n"
" 2026-10-12-to-merlin-19.md | heading: # the precondition refuses\n"
"reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline\n")
check("22 a clean row's detail is the tool's summary",
pick_detail(reply_noise, summary=True),
"reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline")
check("22b a fault run still prefers the reason",
pick_detail("cannot grade: baseline x: No such file\n"
"tool: 1 new fault(s)\n"),
"cannot grade: baseline x: No such file")
print("selftest: %d cells, %d passed, %d failed" % (cells, passed, len(failures))) print("selftest: %d cells, %d passed, %d failed" % (cells, passed, len(failures)))
for f in failures: for f in failures:
print(" FAIL %s" % f) print(" FAIL %s" % f)