diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index 1d4ec79..de3619b 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -10145,3 +10145,18 @@ Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-18-index-stale/` (`REPORT.m - **Nothing operational.** No queue item opened or closed; open set unchanged (`Q37:5278` pipe, career-lane `Q53:7303` held for Alexander); no issue/PR/release/deploy surface; **no another-party file edited** (the two files changed and the one file added are this lane's own `tools/`, plus this append and the index re-render); no counterparty messaged; no secret printed; no check-in file added — 2026-10-19 already carries one and a second would be a DUPLICATE under the contract the gate grades. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-19-reply-check/` (`REPORT.md`, `check.sh`, `witness-headers.py`, `after/replies-scan.txt`, `after/replies-scan.json`, `after/selftest.txt`, `after/sweep.txt`, `after/verify.txt`, `files.sha256`). — Volt + + +**Volt tick 2026-10-19 16:45Z — `replies/` is a gate leg now: the ledger pattern is what unblocked it, and both directions are measured on one planted file.** (Pure suffix append; clock read immediately before this edit at 16:43:17Z; the file's true last line at `5defac9` was `— Volt` closing the 14:30Z block, so nothing above this anchor was touched. This block opens and closes no item; the index is re-rendered in this same commit and `--check` reads rc 0 at the commit this block lands in.) + +- **The blocker was real and it was the wrong blocker.** The 14:30Z block ended with a stated condition: `reply_check.py` is "not a `verify.py` leg, deliberately: the archive is red on the day the tool lands, and a gate wired to a defect its owner has not fixed yet is a gate that gets switched off in a week -- the condition to wire it is one line against `LEGS`, once Merlin's heading reads `to Timmy`". At `5defac9` that heading still reads `to Merlin` (`replies/2026-10-15-to-timmy-58.md:1`, sha256 `cca7de7b…`, unchanged since Merlin's `6315880`), so the condition had no date on it while `replies/` -- 193 tracked files, the second largest artifact directory after `checkins/` -- stayed ungraded by the one command an operator runs. The repo already solved this exact shape for check-ins: `tools/checkin_baseline.tsv` states 35 known-faulting files and the gate is green *and* honest. The fifth leg applies that pattern; it does not re-open whether the fault is Merlin's (it is). +- **The instrument, one flag and one ledger.** `reply_check.py` gains `--baseline PATH` (one `\t` row per tolerated fault, the same shape and rule as `checkin_baseline.tsv`) and `--emit-baseline` (prints the ledger for the current directory, exits 0, so refreshing it is a reviewable diff). Under a ledger, a class the row does not name on that file is rc 1, a faulting file the ledger does not list is rc 1, a ledger entry whose file is gone or whose class has cleared prints `STALE BASELINE` and never makes rc 1, and a missing, non-UTF-8, tab-less or unknown-class ledger is rc 2 -- a tolerance that cannot be read is not the same fact as an archive with nothing tolerated. `tools/reply_baseline.tsv` is 6 lines (5 comment, 1 row): `2026-10-15-to-timmy-58.md\tHEADER_RECIPIENT`. +- **The live archive, both sides of the flag, one tree.** At `5defac9` + this working tree: `reply_check.py --repo .` -> **192 files, 8 recipients, 191 numbered + 1 legacy, 1 fault, rc 1**; the same run with `--baseline tools/reply_baseline.tsv` -> **rc 0, 1 known-faulting, 0 new, 0 stale, 0 cleared**, and `diff` of the two reports is **three lines at the end** (the `FAIL:` line replaced by the `baseline …` and `reply_check:` summaries) -- same class table, same `series: 8 recipient(s), 191 numbered file(s), 1 hole(s): timmy [59]`, same `faults: 1 advisories: 19`. **The tolerated file is still printed every run**; the ledger changes the exit code, not the report. +- **Red then green on one planted file, not argued.** In a clone of `5defac9` (`/root/.hermes/profiles/volt/cache/scratch/reply-leg-red-164143`) I added `replies/2026-10-19-to-volt-99.md`, headed `# Reply 99 -- to Merlin (Helm lane), on the census`. The **old four-leg gate exited 0** on that tree (`gate: OK -- 3 of 3 graded leg(s) passed`), while `reply_check.py` on the same tree exited **1**. With the new tools copied in, the same tree gives **5 legs, 1 fault, rc 1**, `fault: reply-headings`, detail **`FAIL: 1 file(s) carry a class the baseline does not tolerate`**. Moving the planted file out of the tree returns **rc 0 -- `reply-headings PASS … 1 known-faulting in the baseline`** -- so the leg is red on the new defect and green on the known one, measured on one tree minutes apart (§`before/verify-4leg.txt`, `before/verify-4leg.rc`, `after/verify-red.txt`, `after/verify-known-only.txt`). +- **A detail-line defect found while wiring it, and fixed rather than tolerated.** The first wired run had `reply-headings PASS` beside the detail **`NO_HEADER 0`** -- a zero-count class row, reaching the row because `STRONG_HINTS` contains `NO_`. Deleting the zero rows moved it to **`NO_RECIPIENT_IN_HEADING 7`** (a count, not a reason), then to a **heading excerpt containing the word "refuses"** (a per-file advisory line matching the `refus` hint), and on the planted-fault tree the same excerpt beat the tool's own `FAIL:` line. The fix is in `pick_detail`, three rules: a class table row (`^[A-Z][A-Z0-9_]+ +\d+$`) is never a strong reason, a zero row is never a reason at all, a `FAIL:`/`STALE:` line outranks every other reason, and on a **PASS** row the detail is the tool's own concluding summary (`SUMMARY_RE`, `^[a-z][a-z0-9_]*: `) because on a clean run that is the useful line. Measured after: PASS row -> `reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline`; fault row -> the `FAIL:` line (§`after/` above). +- **Cells, both harnesses, and the advertised counts.** `reply_check.py --selftest` **23 -> 37 cells, 37 passed, 0 failed, rc 0** (14 new: ledger emit/round-trip, known-fault rc 0 and still printed, a new class on a known file -> rc 1 with `new == ['HEADER_NUMBER']`, an unlisted faulting file -> rc 1, stale and cleared entries reported at rc 0, missing/tab-less/unknown-class ledgers -> rc 2, and the `FAIL:` line); `verify.py --selftest` **37 -> 42 cells, 42 passed, 0 failed, rc 0** (five-leg fixtures and counts, a reply fault -> rc 1 named `reply-headings`, the leg's argv carries `--repo`'s own ledger, and two `pick_detail` cells for the summary/verdict rules). `selftest_sweep.py` reads both claims as matching: **15 tools, 15 OK, 0 fatal; read-refusal 20/20, malformed-input 18/18, rc 0** (§`after/sweep.txt`). +- **The gate at the commit this block lands in.** `verify.py --repo .` -> **5 legs, 5 pass, 0 fault, 0 ungradable, 0 skipped, rc 0** (`checkin` 75 files / 0 new / 35 known-faulting; `reply-headings` 192 files / 0 new / 1 known-faulting; `index-current` 49 sections / 45 ids; `append-audit` clean; `selftest-sweep` 15 OK). `ruff check --select F,E9 tools/` clean. +- **What this does not do.** It does not judge whether Merlin's heading should change (it should; the ledger row is deleted in the same commit that fixes it, and a stale row is reported rather than silently kept), does not grade who wrote a file (the signature is prose), does not resolve the timmy-59 hole (still advisory, still undecidable from the bytes), and does not change any other tool's exit contract. +- **Nothing operational.** No queue item opened or closed; open set unchanged (`Q37:5278` pipe, career-lane `Q53:7303` held for Alexander); no issue/PR/release/deploy surface; no another-party file edited (the files changed or added are this lane's own `tools/`, plus this append and the index re-render); no counterparty messaged; no secret printed; no check-in file added -- 2026-10-19 already carries one and a second would be a DUPLICATE under the contract the gate grades. Artifact: `/root/.hermes/wizard-outputs/volt/2026-10-19-reply-leg/` (`REPORT.md`, `check.sh`, `pick_probe.py`, `before/`, `after/`, `files.sha256`). + +— Volt diff --git a/docs/queue-index.md b/docs/queue-index.md index dad68a4..c920a65 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -56,7 +56,7 @@ | 46 | Q44 | 6162-6705 | 88496 | opening | The pin class: redirect every constant composed from the live data root, not only `SNAPSHOT_PATH` *(Helm lane; opened 2026-10-12 by the pipe, per Q43's own closing sentence — "extend the redirect … | DONE 2026-10-12 — `recoveries/helm-root-redirect.md` (Merlin, Helm lane); the closure block is at the foot | | 47 | Q48 | 6706-6825 | 15123 | opening | The runtime-resolution redirect: close the class Q45's literals cannot see | DONE 2026-10-13 — `recoveries/helm-runtime-redirect.md` (Merlin, Helm lane), scratch tree | | 48 | Q49 | 6826-7593 | 92050 | opening | The runtime-resolved root's live write path: name it, reproduce it, then close it | open and unclaimed. Opened by the pipe 2026-10-13 | -| 49 | Q45 | 7594-10147 | 405974 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | -- | +| 49 | Q45 | 7594-10162 | 413134 | opening | §9 follow-on — the home-directory channel *(Helm lane; Merlin)* | -- | ## Ids with no `## ` heading @@ -71,7 +71,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q50 | 6990 | 6990 | 46 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q51 | 7106 | 7106 | 37 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q52 | 7167 | 7167 | 90 | Q49 | BULLET+BOLD | BOLD-LEAD | -| Q53 | 7292 | 7246 | 127 | Q49 | BULLET+BOLD | BOLD-LEAD | +| Q53 | 7292 | 7246 | 128 | Q49 | BULLET+BOLD | BOLD-LEAD | | Q63 | 9918 | 9909 | 17 | Q45 | BULLET+BOLD | BOLD-LEAD | **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. @@ -217,7 +217,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q53", "line": 7292, - "refs": 127, + "refs": 128, "section": "Q49", "shape": "BULLET+BOLD" }, @@ -236,9 +236,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 1298235, - "queue_lines": 10147, - "queue_sha256": "13913398b5b33743d0603c865dcdca2b79f9b74ceb71dbe2c64ef56ef47522fd", + "queue_bytes": 1305395, + "queue_lines": 10162, + "queue_sha256": "1099896d4c8d5413b487218ff222f0be79bd694eeae279dfaf9325343e1a4b58", "sections": [ { "bytes": 3139, @@ -817,11 +817,11 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "The runtime-resolved root's live write path: name it, reproduce it, then close it" }, { - "bytes": 405974, - "end_line": 10147, + "bytes": 413134, + "end_line": 10162, "id": "Q45", "line": 7594, - "lines": 2554, + "lines": 2569, "role": "opening", "role_detail": "", "status": null, diff --git a/tools/reply_baseline.tsv b/tools/reply_baseline.tsv new file mode 100644 index 0000000..6d2db6a --- /dev/null +++ b/tools/reply_baseline.tsv @@ -0,0 +1,6 @@ +# reply baseline -- the faults this archive already carried when the +# heading rule was written. One row per file: \t. +# Regenerate: reply_check.py --repo . --emit-baseline > tools/reply_baseline.tsv +# A row is a stated tolerance, not an exemption: a file that gains a class, +# and a file that is not listed, both make rc 1 under --baseline. +2026-10-15-to-timmy-58.md HEADER_RECIPIENT diff --git a/tools/reply_check.py b/tools/reply_check.py index edc0bc7..dcb7da1 100644 --- a/tools/reply_check.py +++ b/tools/reply_check.py @@ -68,8 +68,25 @@ EXIT CONTRACT (the repo's, shared by every tool in tools/) 1 ran and graded at least one fault 2 could not grade (missing directory, or a path that is not a directory) +BASELINE -- the archive's one fault, stated instead of hidden + `--baseline PATH` grades every row against a ledger of the faults the archive + already carried, one `\\t` row each -- the same shape + and the same rule as `tools/checkin_baseline.tsv`: a listed class is KNOWN, a + class the ledger does not name on that file is a FAULT, a faulting file the + ledger does not list at all is a FAULT, and a ledger entry that no longer + matches its file (gone, or no longer carrying the class) is printed STALE and + never makes rc 1. A missing or malformed ledger is rc 2, never a silent pass: + a tolerance that cannot be read is not the same fact as an archive with + nothing tolerated. `--emit-baseline` prints the ledger for the current + directory and exits 0, so refreshing it produces a reviewable diff. + + This exists because the one live fault is Merlin's file and his to repair + (`replies/2026-10-15-to-timmy-58.md`, headed `to Merlin`); a gate that cannot + be switched on until another lane edits its own file is a gate that stays off. + The ledger names the tolerance, and the row stays visible in every report. + SELFTEST - `--selftest` grades 23 cells, building every fixture in a fresh temporary + `--selftest` grades 37 cells, building every fixture in a fresh temporary directory: each fault class fires on a fixture that carries it and only the classes that belong there (a wrong addressee, a wrong number, a duplicate number, an impossible date, a missing heading, a malformed name, non-UTF-8 @@ -302,7 +319,88 @@ def scan(dirpath, head_lines=10, roster=()): return rows, faults, advisories, series, holes -def report(path, rows, faults, advisories, series, holes, roster_source, as_json=None): +# ---------------------------------------------------------------- baseline + +BASELINE_CLASSES = set(FAULTS) + + +def emit_baseline(rows): + """The ledger as text: one `\\t` row per faulting file. + + Advisory classes are left out on purpose: they are never rc 1, so a ledger + that named them would read as if they had been tolerated out of a grade + they never had. Sorted, so an unchanged archive regenerates byte-identical. + """ + head = ("# reply baseline -- the faults this archive already carried when the\n" + "# heading rule was written. One row per file: \\t.\n" + "# Regenerate: reply_check.py --repo . --emit-baseline > tools/reply_baseline.tsv\n" + "# A row is a stated tolerance, not an exemption: a file that gains a class,\n" + "# and a file that is not listed, both make rc 1 under --baseline.\n") + lines = [] + for row in sorted(rows, key=lambda r: r["file"]): + cls = sorted(c for c in row["classes"] if c in BASELINE_CLASSES) + if cls: + lines.append("%s\t%s" % (row["file"], ",".join(cls))) + return head + "\n".join(lines) + ("\n" if lines else "") + + +def load_baseline(path): + """Read the ledger -> (ledger, None), or (None, reason) when it cannot be read. + + Same contract as `checkin_baseline.tsv`: an unreadable, non-UTF-8, wrongly + shaped or unknown-class row is a refusal, because grading against an empty + ledger would call the whole archive new and skipping the check would call + it fine -- both are silent lies about the same fact. + """ + try: + with open(path, "r", encoding="utf-8") as fh: + text = fh.read() + except OSError as exc: + return None, "cannot grade: baseline %s: %s" % (path, exc) + except UnicodeDecodeError as exc: + return None, "cannot grade: baseline %s is not UTF-8: %s" % (path, exc) + out = {} + for n, line in enumerate(text.splitlines(), 1): + s = line.rstrip("\r") + if not s.strip() or s.lstrip().startswith("#"): + continue + parts = s.split("\t") + if len(parts) != 2: + return None, ("cannot grade: baseline %s:%d is not " + "\\t: %r" % (path, n, s)) + name = parts[0].strip() + cls = [c.strip() for c in parts[1].split(",") if c.strip()] + bad = [c for c in cls if c not in BASELINE_CLASSES] + if not name or not cls or bad: + return None, ("cannot grade: baseline %s:%d names an empty filename, " + "an empty class list, or an unknown class %s: %r" + % (path, n, bad, s)) + out[name] = set(cls) + return out, None + + +def apply_baseline(rows, ledger, path=""): + """Split every row's graded classes into known and new, and count the stale.""" + current = {r["file"]: set(c for c in r["classes"] if c in BASELINE_CLASSES) + for r in rows} + known_files = 0 + for row in rows: + cls = current[row["file"]] + known = ledger.get(row["file"], set()) + new = sorted(cls - known) + row["baseline"] = {"known": sorted(cls & known), "new": new} + if cls and not new: + known_files += 1 + stale = sorted(f for f in ledger if f not in current) + cleared = sorted(f for f in ledger + if f in current and (ledger[f] - current[f])) + return {"path": path, "entries": len(ledger), "known_files": known_files, + "new_faults": sum(1 for r in rows if r["baseline"]["new"]), + "stale": stale, "cleared": cleared} + + +def report(path, rows, faults, advisories, series, holes, roster_source, as_json=None, + ledger=None): counts = {} for row in rows: for c in row["classes"]: @@ -326,13 +424,41 @@ def report(path, rows, faults, advisories, series, holes, roster_source, as_json (": " + ", ".join("%s %s" % (k, v) for k, v in sorted(holes.items()))) if holes else "")) print("faults: %d advisories: %d" % (len(faults), len(advisories))) + if ledger is not None: + print("baseline %s: %d entries, %d file(s) known-faulting, " + "%d new fault(s), %d stale, %d cleared" + % (ledger["path"], ledger["entries"], ledger["known_files"], + ledger["new_faults"], len(ledger["stale"]), + len(ledger["cleared"]))) + for f in ledger["stale"]: + print(" STALE BASELINE %s (no longer in the directory: the record " + "was repaired)" % f) + for f in ledger["cleared"]: + print(" STALE BASELINE %s (no longer carries every class the " + "ledger names)" % f) if as_json: with open(as_json, "w", encoding="utf-8") as fh: json.dump({"path": path, "files": len(rows), "counts": counts, - "holes": holes, "rows": rows}, fh, indent=2, sort_keys=True) + "holes": holes, "rows": rows, "baseline": ledger}, + fh, indent=2, sort_keys=True) fh.write("\n") print("json: %s" % as_json) - return 1 if faults else 0 + if ledger is not None: + bad = [(r["file"], ",".join(r["baseline"]["new"])) for r in rows + if r["baseline"]["new"]] + print("reply_check: %d file(s), %d new fault(s)%s, %d known-faulting in " + "the baseline" + % (len(rows), len(bad), + (": " + ", ".join("%s (%s)" % b for b in bad)) if bad else "", + ledger["known_files"])) + if ledger["new_faults"]: + print("FAIL: %d file(s) carry a class the baseline does not tolerate" + % ledger["new_faults"]) + return 1 if ledger["new_faults"] else 0 + if faults: + print("FAIL: %d file(s) violate the reply heading contract" % len(faults)) + return 1 + return 0 def _write(d, name, text): @@ -349,8 +475,8 @@ def selftest(): ROSTER = {"volt", "merlin", "timmy", "zephyrus"} ROSTER_ARG = "volt,merlin,timmy,zephyrus" - def cell(label, ok): - cells.append((label, bool(ok))) + def cell(label, ok, detail=None): + cells.append((label, bool(ok), detail)) def classes(d, name): rows, _, _, _, _ = scan(d, roster=ROSTER) @@ -476,11 +602,106 @@ def selftest(): cell("a file where a directory belongs exits 2", rc(["--dir", os.path.join(root, "afile")]) == 2) - failed = [label for label, ok in cells if not ok] + # 14-25: the baseline ledger. The live archive carries one fault in another + # lane's file, so the gate has to be able to state that tolerance without + # hiding a new fault: every cell here reads the tolerant side AND the side + # that must still fail. + d = os.path.join(root, "baseline") + os.makedirs(d) + _write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane)\n") + _write(d, "2026-10-03-to-merlin-07.md", "# Reply 7 -- to Merlin\n") + rows_b, _, _, _, _ = scan(d, roster=ROSTER) + led_text = emit_baseline(rows_b) + cell("emit names the faulting file and its class", + "2026-10-02-to-timmy-58.md\tHEADER_RECIPIENT" in led_text) + cell("emit leaves the clean file out", + "2026-10-03-to-merlin-07.md" not in led_text) + bl = os.path.join(root, "reply.tsv") + with open(bl, "w", encoding="utf-8") as fh: + fh.write(led_text) + ledger_, why_ = load_baseline(bl) + cell("the emitted ledger round-trips", + ledger_ == {"2026-10-02-to-timmy-58.md": {"HEADER_RECIPIENT"}} + and why_ is None, (ledger_, why_)) + with contextlib.redirect_stdout(io.StringIO()): + rv_known = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) + cell("a known fault under its ledger exits 0", rv_known == 0) + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) + printed = buf.getvalue() + cell("the tolerated fault is still printed, never hidden", + "HEADER_RECIPIENT" in printed and "1 known-faulting" in printed, + printed[-160:]) + + # the same file gains a class the ledger does not name -> rc 1, and the new + # class is named rather than the whole row re-graded + _write(d, "2026-10-02-to-timmy-58.md", "# Reply 9 -- to Merlin (Helm lane)\n") + rows_new, _, _, _, _ = scan(d) + led_new = apply_baseline(rows_new, load_baseline(bl)[0], bl) + cell("a new class on a known file is the only new fault", + led_new["new_faults"] == 1 + and [r["baseline"]["new"] for r in rows_new + if r["baseline"]["new"]] == [["HEADER_NUMBER"]], + led_new) + with contextlib.redirect_stdout(io.StringIO()): + rc_grown = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) + cell("a known file that gains a class exits 1", rc_grown == 1) + + # a faulting file the ledger does not list at all -> rc 1 + _write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane)\n") + _write(d, "2026-10-04-to-volt-01.md", "# Reply 1 -- to Timmy\n") + with contextlib.redirect_stdout(io.StringIO()): + rc_unlisted = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) + cell("an unlisted faulting file exits 1", rc_unlisted == 1) + + # without a ledger the run still names its verdict in one line, so a reader + # (and verify.py's detail extractor) gets a reason, not just a table + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + run(["--dir", d, "--roster", ROSTER_ARG]) + cell("a faulting run prints a FAIL line", "FAIL: " in buf.getvalue(), + buf.getvalue()[-160:]) + + # a ledger entry whose file is gone: printed STALE, never rc 1 + ds = os.path.join(root, "baseline-stale") + os.makedirs(ds) + _write(ds, "2026-10-03-to-merlin-07.md", "# Reply 7 -- to Merlin\n") + bls = os.path.join(root, "stale.tsv") + with open(bls, "w", encoding="utf-8") as fh: + fh.write("2026-10-09-to-merlin-99.md\tHEADER_RECIPIENT\n") + fh.write("2026-10-03-to-merlin-07.md\tHEADER_NUMBER\n") + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + rc_stale = run(["--dir", ds, "--baseline", bls, "--roster", ROSTER_ARG]) + stale_out = buf.getvalue() + cell("a stale ledger entry is reported and never rc 1", + rc_stale == 0 and "2026-10-09-to-merlin-99.md" in stale_out, + stale_out[-200:]) + cell("a cleared class is reported as stale, not as clean", + "no longer carries every class" in stale_out, stale_out[-200:]) + + # a ledger that cannot be read is rc 2, never a pass + cell("a missing ledger exits 2", + rc(["--dir", ds, "--baseline", os.path.join(root, "absent.tsv")]) == 2) + bad1 = os.path.join(root, "bad-nosep.tsv") + with open(bad1, "w", encoding="utf-8") as fh: + fh.write("2026-10-09-to-merlin-99.md HEADER_RECIPIENT\n") + cell("a ledger line without the tab exits 2", + rc(["--dir", ds, "--baseline", bad1]) == 2) + bad2 = os.path.join(root, "bad-class.tsv") + with open(bad2, "w", encoding="utf-8") as fh: + fh.write("2026-10-09-to-merlin-99.md\tNOT_A_CLASS\n") + cell("a ledger naming an unknown class exits 2", + rc(["--dir", ds, "--baseline", bad2]) == 2) + + failed = [(label, detail) for label, ok, detail in cells if not ok] print("selftest: %d cells, %d passed, %d failed" % (len(cells), len(cells) - len(failed), len(failed))) - for label in failed: + for label, detail in failed: print(" FAILED: %s" % label) + if detail is not None: + print(" got: %r" % (detail,)) return 1 if failed else 0 @@ -496,6 +717,11 @@ def run(argv): ap.add_argument("--head-lines", type=int, default=10, help="how many leading lines may hold the heading (default 10)") ap.add_argument("--json", help="write the full row report here") + ap.add_argument("--baseline", default=None, + help="known-fault ledger (lines of \\t): " + "only a class the ledger does not name on that file is rc 1") + ap.add_argument("--emit-baseline", action="store_true", + help="print the ledger for this directory and exit 0") args = ap.parse_args(argv) dirpath = args.dir if args.dir is not None else os.path.join(args.repo, "replies") if not os.path.exists(dirpath): @@ -508,7 +734,18 @@ def run(argv): return 2 roster, source = load_roster(args.repo, args.roster) rows, faults, advisories, series, holes = scan(dirpath, args.head_lines, roster) - return report(dirpath, rows, faults, advisories, series, holes, source, args.json) + if args.emit_baseline: + sys.stdout.write(emit_baseline(rows)) + return 0 + ledger = None + if args.baseline: + ledger, why = load_baseline(args.baseline) + if ledger is None: + print("reply_check: %s" % why, file=sys.stderr) + return 2 + ledger = apply_baseline(rows, ledger, args.baseline) + return report(dirpath, rows, faults, advisories, series, holes, source, + args.json, ledger) def main(): diff --git a/tools/verify.py b/tools/verify.py index 8df97a7..0d5e666 100644 --- a/tools/verify.py +++ b/tools/verify.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""verify.py -- the HOME the index gate did not have: one command, four legs, +"""verify.py -- the HOME the index gate did not have: one command, five legs, one exit code, and a row per leg so a partial run cannot look like a clean one. WHY THIS EXISTS @@ -25,12 +25,31 @@ WHY THIS EXISTS append discipline and the tools' selftests said nothing about the one artifact every lane writes every tick. A fourth leg closes that hole. + The fifth is the same shape one directory over, and it is here for a reason + that was measured before it was written. On 2026-10-19 this lane built + `reply_check.py` and deliberately did NOT wire it: the archive has one live + fault -- `replies/2026-10-15-to-timmy-58.md` is named for Timmy and headed + "to Merlin" -- it is Merlin's file and Merlin's to repair, and that tick's + reasoning was that a gate wired to a defect its owner has not fixed yet is a + gate that gets switched off in a week. That reasoning is right about a gate + with no ledger and wrong about this repo, which already solved the same + problem for check-ins: `tools/reply_baseline.tsv` states the one tolerance, + the fault stays visible in every report as known-faulting, and a second + mis-addressed reply -- or a new fault class on the known file -- is rc 1 the + day it lands. Waiting for another lane to edit its own file was the only + alternative, and it had no date on it. + WHAT IT RUNS -- one row per leg, cheapest first, never in parallel checkin tools/checkin_check.py --fail --baseline tools/checkin_baseline.tsv the WHOLE check-in archive against F1-F5 (names, duplicates, unknown handles, dates, emptiness), where the ledger states the faults the archive already carried; only a class it does not name is a fault. `--checkin-day DATE` adds a `--since` floor + reply-headings tools/reply_check.py --baseline tools/reply_baseline.tsv + every reply's heading against its own filename (addressee and + number), on the same ledger rule: `replies/` is the second + largest artifact directory and, until 2026-10-19, nothing here + read it index-current tools/queue_index.py --check docs/queue-index.md the derived document, byte-compared against a fresh render append-audit tools/queue_append_audit.py --guard --to HEAD @@ -97,7 +116,7 @@ TOOLS ARE RESOLVED FROM --repo, NOT FROM THIS FILE'S DIRECTORY docs/queue-index.md) because `queue_append_audit` reads them with `git show :`, which has no absolute-path form. -`--selftest` fixture corpus, 37 cells +`--selftest` fixture corpus, 42 cells """ import argparse @@ -121,6 +140,8 @@ LEGS = ( {"key": "checkin", "tool": "checkin_check.py", "slow": False, "what": "grade the newest day's check-ins against the naming and " "duplicate rules"}, + {"key": "reply-headings", "tool": "reply_check.py", "slow": False, + "what": "grade every reply's heading against its own filename"}, {"key": "index-current", "tool": "queue_index.py", "slow": False, "what": "render the index and byte-compare it with the document"}, {"key": "append-audit", "tool": "queue_append_audit.py", "slow": False, @@ -142,8 +163,13 @@ STRONG_HINTS = ("STALE:", "FAIL:", "FAILED", "fatal", "TIMEOUT", "NO_", "cannot", "error", "Error", "refus") WEAK_HINTS = ("FAIL", "not a ") +# A tool's concluding line: `reply_check: 192 file(s), 0 new fault(s) ...`. +# Hyphenated labels (`read-refusal:`, `malformed-input:`) are deliberately not +# summaries -- they are axis lines inside a tool's report. +SUMMARY_RE = re.compile(r"^[a-z][a-z0-9_]*: ") -def pick_detail(text, limit=150): + +def pick_detail(text, limit=150, summary=False): """First line that names a reason, else the last line that says anything. A summary line that reports ZERO faults is not a reason: `selftest_sweep` @@ -152,13 +178,41 @@ def pick_detail(text, limit=150): said FAULT beside the words "0 fatal". Zero-count summaries are therefore only reachable on the second pass (a clean run still gets its sweep line), and any line that names a real reason outranks them. + + `summary=True` is the clean-run answer, and it is a different question. On a + PASS row the useful detail is the tool's own concluding line, not the first + line in its report that happens to contain "refus" -- `reply_check` prints + one heading excerpt per advisory row, and one of those headings contains the + word "refuses", which was reaching the row as if it were the reason. On a + fault row the reason still wins, because that is what a reader needs there. """ lines = [ln.strip() for ln in text.splitlines() if ln.strip()] kept = [ln for ln in lines if not ln.startswith("SKIP ")] or lines + if summary: + sums = [ln for ln in kept if SUMMARY_RE.match(ln)] + if sums: + return sums[-1][:limit] hinted = [ln for ln in kept if any(h in ln for h in STRONG_HINTS + WEAK_HINTS)] + # A class row that printed zero is not a reason and not a count of anything: + # `reply_check` prints its whole class table on every run, so `NO_HEADER 0` + # was reaching the detail line of a PASS row ahead of the summary. Same + # failure the " 0 fatal" filter catches, one shape further out. + class_row = re.compile(r"^[A-Z][A-Z0-9_]+ +(\d+)$") + zero_row = re.compile(r"^[A-Z][A-Z0-9_]+ +0$") reasons = [ln for ln in hinted - if " 0 fatal" not in ln and " 0 fault" not in ln] - strong = [ln for ln in reasons if any(h in ln for h in STRONG_HINTS)] + if " 0 fatal" not in ln and " 0 fault" not in ln + and not zero_row.match(ln)] + # Nor is a nonzero class row a "reason": `NO_RECIPIENT_IN_HEADING 7` is a + # count, and `NO_` in its name is not a diagnostic marker. A hint counts + # only on a line that is not a class table row. + strong = [ln for ln in reasons + if any(h in ln for h in STRONG_HINTS) and not class_row.match(ln)] + # A tool's own verdict line outranks every other reason in its report: when + # `reply_check` names a mis-addressed file it also prints one heading excerpt + # per row, and an excerpt containing "refuses" was beating the FAIL line. + verdict = [ln for ln in strong if ln.startswith(("FAIL:", "STALE:"))] + if verdict: + return verdict[0][:limit] if strong: return strong[0][:limit] if reasons: @@ -198,6 +252,13 @@ def leg_argv(key, repo, queue, index, checkin_day=None): if checkin_day: argv += ["--since", checkin_day] return argv + if key == "reply-headings": + # The ledger comes from the tree being graded, for the same reason the + # check-in ledger does: a revision is graded with the tolerance it + # committed, not with this host's idea of one. + return [sys.executable, os.path.join(repo, "tools", "reply_check.py"), + "--repo", repo, "--baseline", + os.path.join(repo, "tools", "reply_baseline.tsv")] if key == "index-current": return [sys.executable, os.path.join(repo, "tools", "queue_index.py"), "--repo", repo, "--queue", queue, "--check", index] @@ -245,7 +306,7 @@ def run_leg(leg, repo, queue, index, timeout, checkin_day=None): out = (proc.stdout or "") + (proc.stderr or "") row["rc"] = proc.returncode - row["detail"] = pick_detail(out) + row["detail"] = pick_detail(out, summary=(proc.returncode == 0)) if proc.returncode == 0: row["verdict"] = PASS elif proc.returncode == 1: @@ -330,7 +391,7 @@ def main(argv=None): help="per-leg timeout in seconds (default 300)") ap.add_argument("--json", metavar="PATH", help="write the rows here") ap.add_argument("--quiet", action="store_true") - ap.add_argument("--selftest", action="store_true", help="fixture corpus, 37 cells") + ap.add_argument("--selftest", action="store_true", help="fixture corpus, 42 cells") a = ap.parse_args(argv) if a.selftest: @@ -422,7 +483,7 @@ def _verify(*argv): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") -GREEN4 = {"checkin_check.py": 0, "queue_index.py": 0, +GREEN5 = {"checkin_check.py": 0, "reply_check.py": 0, "queue_index.py": 0, "queue_append_audit.py": 0, "selftest_sweep.py": 0} @@ -440,16 +501,16 @@ def selftest(): failures.append("%s\n got: %r\n want: %r" % (label, got, want)) with tempfile.TemporaryDirectory(prefix="verify-selftest-") as td: - # 1. three green stubs -> rc 0 - repo = _fixture(os.path.join(td, "c1"), GREEN4) + # 1. five green stubs -> rc 0 + repo = _fixture(os.path.join(td, "c1"), GREEN5) rc, out = _verify("--repo", repo) - check("1 all four legs green -> rc 0", rc, 0) - check("1b summary counts the four legs", "verify: 4 legs, 4 pass, 0 fault, " + check("1 all five legs green -> rc 0", rc, 0) + check("1b summary counts the five legs", "verify: 5 legs, 5 pass, 0 fault, " "0 ungradable, 0 skipped" in out, True) # 2. a stale index is a FAULT, and its own line reaches the row repo = _fixture(os.path.join(td, "c2"), - dict(GREEN4, **{"queue_index.py": 1}), + dict(GREEN5, **{"queue_index.py": 1}), outs={"queue_index.py": "STALE: rendered document differs at line 7\n"}) rc, out = _verify("--repo", repo) @@ -459,7 +520,7 @@ def selftest(): # 3. a refusal is UNGRADABLE, not a fault repo = _fixture(os.path.join(td, "c3"), - dict(GREEN4, **{"queue_index.py": 2}), + dict(GREEN5, **{"queue_index.py": 2}), outs={"queue_index.py": "cannot read queue: nope\n"}) rc, out = _verify("--repo", repo) check("3 refusal -> rc 2", rc, 2) @@ -467,7 +528,7 @@ def selftest(): # 4. precedence: ungradable outranks fault, and the fault is NOT erased repo = _fixture(os.path.join(td, "c4"), - dict(GREEN4, **{"queue_index.py": 1, + dict(GREEN5, **{"queue_index.py": 1, "selftest_sweep.py": 2}), outs={"queue_index.py": "STALE: line 3\n", "selftest_sweep.py": "cannot read: no tools\n"}) @@ -479,14 +540,14 @@ def selftest(): # 5. a missing leg tool is ungradable, and says which path was missing repo = _fixture(os.path.join(td, "c5"), - dict(GREEN4, **{"queue_append_audit.py": None})) + dict(GREEN5, **{"queue_append_audit.py": None})) rc, out = _verify("--repo", repo) check("5 missing tool -> rc 2", rc, 2) check("5b the missing path is named", "no such tool:" in out, True) # 6. --fast skips the sweep and does not run it marker = os.path.join(td, "c6-sweep-ran") - repo = _fixture(os.path.join(td, "c6"), GREEN4, markers={"selftest_sweep.py": marker}) + repo = _fixture(os.path.join(td, "c6"), GREEN5, markers={"selftest_sweep.py": marker}) rc, out = _verify("--repo", repo, "--fast") check("6 --fast on a green tree -> rc 0", rc, 0) check("6b the skipped leg is printed as SKIP", "selftest-sweep" in out @@ -494,12 +555,13 @@ def selftest(): check("6c the skipped stub never ran", os.path.exists(marker), False) # 7. --json is a real machine-readable record - repo = _fixture(os.path.join(td, "c7"), GREEN4) + repo = _fixture(os.path.join(td, "c7"), GREEN5) jpath = os.path.join(td, "c7.json") rc, out = _verify("--repo", repo, "--json", jpath) obj = json.load(open(jpath)) if os.path.exists(jpath) else {} check("7 --json writes the rows", [r["key"] for r in obj.get("rows", [])], - ["checkin", "index-current", "append-audit", "selftest-sweep"]) + ["checkin", "reply-headings", "index-current", "append-audit", + "selftest-sweep"]) check("7b --json carries the same rc the process returned", obj.get("rc"), rc) @@ -510,7 +572,7 @@ def selftest(): # 9. rc 3 is retired: it is ungradable, never a fault and never a pass repo = _fixture(os.path.join(td, "c9"), - dict(GREEN4, **{"queue_index.py": 3}), + dict(GREEN5, **{"queue_index.py": 3}), outs={"queue_index.py": "unknown state\n"}) rc, out = _verify("--repo", repo) check("9 rc 3 -> rc 2 (retired code)", rc, 2) @@ -518,30 +580,30 @@ def selftest(): # 10. a hung leg is a timeout, not a pass and not a fault repo = _fixture(os.path.join(td, "c10"), - dict(GREEN4, **{"selftest_sweep.py": 0}), sleep=5.0) + dict(GREEN5, **{"selftest_sweep.py": 0}), sleep=5.0) rc, out = _verify("--repo", repo, "--timeout", "1") check("10 timeout -> rc 2", rc, 2) check("10b the timeout is named", "TIMEOUT after 1s" in out, True) # 11. order is fixed, cheapest first: the table is not re-sorted by outcome repo = _fixture(os.path.join(td, "c11"), - dict(GREEN4, **{"queue_index.py": 1}), + dict(GREEN5, **{"queue_index.py": 1}), outs={"queue_index.py": "STALE: line 1\n"}) rc, out = _verify("--repo", repo) - order = [out.find(k) for k in ("checkin", "index-current", "append-audit", - "selftest-sweep")] + order = [out.find(k) for k in ("checkin", "reply-headings", "index-current", + "append-audit", "selftest-sweep")] check("11 legs print in fixed order", order == sorted(order) and -1 not in order, True) # 12. a skip is counted as a skip, never folded into pass - repo = _fixture(os.path.join(td, "c12"), GREEN4) + repo = _fixture(os.path.join(td, "c12"), GREEN5) jpath = os.path.join(td, "c12.json") rc, out = _verify("--repo", repo, "--fast", "--json", jpath) obj = json.load(open(jpath)) if os.path.exists(jpath) else {} check("12 skipped leg is its own count in --json", obj.get("counts", {}).get("SKIP"), 1) check("12b and the skipped leg is not counted as a pass", - obj.get("counts", {}).get(PASS), 3) + obj.get("counts", {}).get(PASS), 4) # 13. the detail line must not contradict the verdict: a row graded # FAULT beside "0 fatal" is the SILENT failure this gate exists to catch. @@ -558,7 +620,7 @@ def selftest(): # 14. THE HOLE THIS LEG CLOSES: a contract-breaking check-in is rc 1, # and the gate is no longer green on a tree that carries one. repo = _fixture(os.path.join(td, "c14"), - dict(GREEN4, **{"checkin_check.py": 1}), + dict(GREEN5, **{"checkin_check.py": 1}), outs={"checkin_check.py": "FAIL: 2 file(s) violate the check-in contract\n"}) rc, out = _verify("--repo", repo, "--fast") @@ -568,7 +630,7 @@ def selftest(): and "fault: checkin" in out, True) # 15. no checkins/ directory is ungradable, not a pass and not a fault - repo = _fixture(os.path.join(td, "c15"), GREEN4, checkins=None) + repo = _fixture(os.path.join(td, "c15"), GREEN5, checkins=None) rc, out = _verify("--repo", repo, "--fast") check("15 no checkins dir -> rc 2", rc, 2) check("15b the ungradable row says which directory is missing", @@ -577,7 +639,7 @@ def selftest(): # 16. the leg grades the WHOLE archive against its own ledger -- the # fix for a newest-day floor that made a backdated fault invisible. av = os.path.join(td, "c16-argv") - repo = _fixture(os.path.join(td, "c16"), GREEN4, + repo = _fixture(os.path.join(td, "c16"), GREEN5, argv_markers={"checkin_check.py": av}, checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md")) rc, out = _verify("--repo", repo, "--fast") @@ -590,7 +652,7 @@ def selftest(): # 17. --checkin-day still narrows the window explicitly, in the open av = os.path.join(td, "c17-argv") - repo = _fixture(os.path.join(td, "c17"), GREEN4, + repo = _fixture(os.path.join(td, "c17"), GREEN5, argv_markers={"checkin_check.py": av}, checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md")) rc, out = _verify("--repo", repo, "--fast", "--checkin-day", "2026-01-01") @@ -602,7 +664,7 @@ def selftest(): # the tool's own reason has to reach the row, because "no ledger" and # "clean archive" print nothing alike and must not grade alike. repo = _fixture(os.path.join(td, "c19"), - dict(GREEN4, **{"checkin_check.py": 2}), + dict(GREEN5, **{"checkin_check.py": 2}), outs={"checkin_check.py": "cannot grade: baseline %s/tools/checkin_baseline.tsv: " "[Errno 2] No such file or directory\n" @@ -634,6 +696,43 @@ def selftest(): check("18b a clean check-in row carries its summary, not the skip note", pick_detail(clean), "checkin_check: 2 files, 0 with a fault, 1 advisory") + # 20. THE FIFTH LEG: a reply whose heading names the wrong party is a + # FAULT on the gate and the row is named -- the class that was invisible + # for as long as `replies/` had no reader. + repo = _fixture(os.path.join(td, "c20"), + dict(GREEN5, **{"reply_check.py": 1}), + outs={"reply_check.py": + "reply_check: 192 file(s), 1 new fault(s), " + "0 known-faulting in the baseline\n"}) + rc, out = _verify("--repo", repo, "--fast") + check("20 a new mis-addressed reply -> rc 1", rc, 1) + check("20b the fault row is named reply-headings and carries its reason", + "fault: reply-headings" in out and "1 new fault(s)" in out, True) + + # 21. the ledger is read from the tree being graded, not from the host + av = os.path.join(td, "c21-argv") + repo = _fixture(os.path.join(td, "c21"), GREEN5, + argv_markers={"reply_check.py": av}) + rc, out = _verify("--repo", repo, "--fast") + seen = open(av).read() if os.path.exists(av) else "" + check("21 the reply leg is run with the repo's own ledger", + "--baseline" in seen + and os.path.join(repo, "tools", "reply_baseline.tsv") in seen, True) + + # 22. on a PASS row the detail is the tool's own concluding summary, not + # the first line that happens to contain a hint substring. + reply_noise = ("reply_check -- 192 file(s) under /tmp/r (roster: x)\n" + " NO_RECIPIENT_IN_HEADING 7\n" + " 2026-10-12-to-merlin-19.md | heading: # the precondition refuses\n" + "reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline\n") + check("22 a clean row's detail is the tool's summary", + pick_detail(reply_noise, summary=True), + "reply_check: 192 file(s), 0 new fault(s), 1 known-faulting in the baseline") + check("22b a fault run still prefers the reason", + pick_detail("cannot grade: baseline x: No such file\n" + "tool: 1 new fault(s)\n"), + "cannot grade: baseline x: No such file") + print("selftest: %d cells, %d passed, %d failed" % (cells, passed, len(failures))) for f in failures: print(" FAIL %s" % f)