#!/usr/bin/env python3 """reply_check.py -- does every reply file's own header agree with its own name? WHY THIS EXISTS `replies/README.md` states the naming contract in one sentence: a reply is `YYYY-MM-DD-to--.md`, where `` is who the reply ADDRESSES. The header inside the file is the copy a reader actually reads, and nothing in this archive has ever compared the two. The failure this grades is the archive's own most-recurring defect, logged in its ledger as a class rather than an accident: "the identifiers printed for Zephyrus's newest file are the 11th's ... Filename and *Blocked on* text were right, the identifiers wrong" (`citation_resolve_check.py`'s docstring quotes it at length). A reply filed to one party whose header addresses another puts a whole thread under the wrong name, and the file is still readable, still append-only, and still green to every instrument this repo owns -- none of them reads `replies/`. WHAT IT MEASURES (read-only; the only write path is --json) Every `*.md` under --dir except `README.md` is one ROW. The filename gives (date, recipient, number); the first `# ` heading in the file gives what the file says about itself. Seven classes are graded: NAME_MALFORMED the filename is neither the numbered contract form nor either of the two unnumbered forms (the legacy file, the reported interview) -- the row is skipped by the header rules, because there is no name to check it against. BAD_DATE the date half is not a real calendar date. NO_HEADER no `# ` heading in the first --head-lines lines. UNREADABLE the file is not UTF-8 text, so no heading can be read from it at all. FAULT: this archive's files are text. HEADER_RECIPIENT the heading names a recipient (`to `) that is not the filename's recipient. FAULT: the two copies of the addressee disagree, and the reader follows the heading. HEADER_NUMBER the heading says `Reply ` and N is not the filename's N. FAULT: the same file would be cited by two different numbers. DUPLICATE_NUMBER two files in one recipient's series carry the same N. FAULT: "the next free integer in that recipient's own series" cannot produce the same integer twice. Five more are reported and printed, and are ADVISORY by construction -- each is a stated consequence of history that predates the naming rule, not a defect this tool can attribute to a writer. Four of the five reach the `advisories: N` count; HOLE rides the `series:` line instead, because a hole belongs to a recipient's series and not to any one file: LEGACY_UNNUMBERED the one legacy form (no `-`). Its number is implicit in its position, so numbering rules skip it. INTERVIEW_UNNUMBERED the reported-interview form, `-interview` in the number's place. It is not a reply and carries no number by construction, so the numbering rules skip it -- but its date, its heading and its addressee are still graded exactly as a numbered reply's are. NO_RECIPIENT_IN_HEADING a heading that names no addressee at all (the early "`# Reply 14 -- Q31 ...`" house style). Nothing contradicts the filename; the heading is silent. NO_NUMBER_IN_HEADING a heading that names the addressee but not "Reply N" (the "`# 2026-10-07 -- to zephyrus (...)`" style). HOLE a gap in one recipient's number series. Advisory on purpose: a hole is indistinguishable, from the bytes, between a lost reply and a number spent in the other direction of the same correspondence. The archive has a measured case of each shape of confusion (see the report that introduced this tool), so the tool names the hole and does not grade it. Recipient matching is by prefix on a `-` boundary, so a heading that says `to zephyrus-sparkborn` matches the filename's `zephyrus`; a heading that says `to merlin` does not match `to-timmy`. EXIT CONTRACT (the repo's, shared by every tool in tools/) 0 ran clean: no FAULT class fired 1 ran and graded at least one fault 2 could not grade (missing directory, or a path that is not a directory) BASELINE -- the archive's one fault, stated instead of hidden `--baseline PATH` grades every row against a ledger of the faults the archive already carried, one `\\t` row each -- the same shape and the same rule as `tools/checkin_baseline.tsv`: a listed class is KNOWN, a class the ledger does not name on that file is a FAULT, a faulting file the ledger does not list at all is a FAULT, and a ledger entry that no longer matches its file (gone, or no longer carrying the class) is printed STALE and never makes rc 1. A missing or malformed ledger is rc 2, never a silent pass: a tolerance that cannot be read is not the same fact as an archive with nothing tolerated. `--emit-baseline` prints the ledger for the current directory and exits 0, so refreshing it produces a reviewable diff. This exists because the one live fault is Merlin's file and his to repair (`replies/2026-10-15-to-timmy-58.md`, headed `to Merlin`); a gate that cannot be switched on until another lane edits its own file is a gate that stays off. The ledger names the tolerance, and the row stays visible in every report. SELFTEST `--selftest` grades 49 cells, building every fixture in a fresh temporary directory: each fault class fires on a fixture that carries it and only the classes that belong there (a wrong addressee, a wrong number, a duplicate number, an impossible date, a missing heading, a malformed name, non-UTF-8 bytes), the advisory classes stay advisory and exit 0 (legacy form, interview form, hole, heading that names nobody), the prefix match accepts a hyphenated handle, the interview form still fails on a wrong addressee or an impossible date and a misspelled `-interview` is still NAME_MALFORMED, README.md is not a row, and both rc 2 paths (missing directory, file where a directory belongs) answer 2. Four cells grade this docstring's own census -- the two spelled class counts against FAULTS/ADVISORIES, every graded class named here, and the pre-fix words refused on a fixture. A rule that stops firing and a rule that fires everywhere both fail a cell. """ import argparse import contextlib import datetime import io import json import os import re import sys import tempfile NAME_RE = re.compile(r"^(\d{4})-(\d{2})-(\d{2})-to-([a-z][a-z0-9]*)-(\d{2,})\.md$") LEGACY_RE = re.compile(r"^(\d{4})-(\d{2})-(\d{2})-to-([a-z][a-z0-9]*)\.md$") INTERVIEW_RE = re.compile(r"^(\d{4})-(\d{2})-(\d{2})-to-([a-z][a-z0-9]*)-interview\.md$") HEADING_TO = re.compile(r"\bto\s+([A-Za-z][A-Za-z0-9-]*)") HEADING_ARROW = re.compile(r"(?:→|->)\s*([A-Za-z][A-Za-z0-9-]*)") HEADING_NUM = re.compile(r"\breply\s+(\d{1,4})\b", re.IGNORECASE) ROSTER_HANDLE_RE = re.compile(r"^[a-z][a-z0-9]*$") FAULTS = ("NAME_MALFORMED", "BAD_DATE", "NO_HEADER", "UNREADABLE", "HEADER_RECIPIENT", "HEADER_NUMBER", "DUPLICATE_NUMBER") ADVISORIES = ("LEGACY_UNNUMBERED", "INTERVIEW_UNNUMBERED", "NO_RECIPIENT_IN_HEADING", "NO_NUMBER_IN_HEADING", "HOLE") WORD_NUMBERS = {"one": 1, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9, "ten": 10, "eleven": 11, "twelve": 12} # The docstring's two spelled counts, and the sentence each lives in. A spelled # count is invisible to doc_census_check.py, which reads ` ` only, # so these are the claims that drift with no instrument watching them: on # 2026-10-19 the control plane measured `Five` over a seven-row table and # `Three` over four rows, and carried the class to this lane. DOC_GRADED_RE = re.compile(r"\b([A-Z][a-z]+) classes are graded\b") DOC_ADVISORY_RE = re.compile(r"\b([A-Z][a-z]+) more are (?:counted|reported)\b") def doc_counts(doc=None): """The docstring's two spelled counts -> (graded, advisory); None if absent. `doc` is a parameter rather than `__doc__` read inside the function so the selftest can grade a fixture docstring -- including the pre-fix `Five` / `Three` pair -- through the same code path the live docstring takes. """ text = __doc__ if doc is None else doc def word(rx): m = rx.search(text or "") return WORD_NUMBERS.get(m.group(1).lower()) if m else None return word(DOC_GRADED_RE), word(DOC_ADVISORY_RE) def _real_date(y, m, d): try: datetime.date(int(y), int(m), int(d)) return True except ValueError: return False def roster_from_table(text): """Handles from WIZARDS-TABLE.md's `| Wizard | ... |` rows: first token, lowered.""" out, in_table = set(), False for line in text.splitlines(): s = line.strip() if not s.startswith("|"): in_table = False continue cells = [c.strip() for c in s.strip("|").split("|")] if not cells or not cells[0]: continue if cells[0].lower() == "wizard": in_table = True continue if not in_table or set(cells[0]) <= set("-: "): continue first = cells[0].split()[0].lower() if cells[0].split() else "" if ROSTER_HANDLE_RE.match(first): out.add(first) return out def roster_from_mailmap(text): """Handles from the canonical side of `.mailmap`: `Name other `.""" out = set() for line in text.splitlines(): s = line.strip() if not s or s.startswith("#") or "<" not in s: continue first = s.split("<")[0].split() if first and ROSTER_HANDLE_RE.match(first[0].lower()): out.add(first[0].lower()) return out def load_roster(repo, override): """(handles, source). The roster exists only to tell a person's name from an ordinary word after `to` -- `to this host` must not read as an addressee.""" if override: hs = {h.strip().lower() for h in override.split(",") if h.strip()} if hs: return hs, "--roster" out, sources = set(), [] for fname, fn in (("WIZARDS-TABLE.md", roster_from_table), (".mailmap", roster_from_mailmap)): path = os.path.join(repo, fname) if not os.path.exists(path): continue try: with open(path, encoding="utf-8") as fh: got = fn(fh.read()) except (OSError, UnicodeDecodeError): continue if got: out |= got sources.append(fname) return out, "+".join(sources) or "(none)" def _recipient_matches(named, wanted): """Prefix match on a '-' boundary, both directions (a heading may be narrower than the filename or fuller than it).""" named = named.lower() wanted = wanted.lower() return (named == wanted or named.startswith(wanted + "-") or wanted.startswith(named + "-")) def _known(cand, roster): return any(_recipient_matches(cand, handle) for handle in roster) def heading_addressee(heading, roster): """The person a heading says the file is addressed to, or None. Two forms are read: `to `, and the arrow form ` → ` (whose addressee is the arrow's target). A candidate counts only if it is a roster handle -- otherwise `to this host` would read as an addressee named "this", which is how a naive matcher invents a contradiction. """ for m in HEADING_ARROW.finditer(heading): cand = m.group(1).lower() if _known(cand, roster): return cand for m in HEADING_TO.finditer(heading): cand = m.group(1).lower() if _known(cand, roster): return cand return None def read_heading(path, head_lines): """The file's first `# ` heading inside the first `head_lines` lines, or None. Blank lines are skipped; a non-blank line before any heading ends the search (a file that opens with prose has no heading where one belongs). False means the file could not be decoded as UTF-8 at all.""" try: with open(path, encoding="utf-8") as fh: for i, line in enumerate(fh): if i >= head_lines: return None s = line.strip() if not s: continue return s if s.startswith("# ") else None except UnicodeDecodeError: return False except OSError: return None return None def scan(dirpath, head_lines=10, roster=()): """(rows, faults, advisories, series, holes) for one replies directory. rows: one dict per file, in sorted filename order, each with `file`, `date`, `recipient`, `number` (None on both unnumbered forms), `heading`, and `classes` (a list). Nothing is written. """ rows = [] for name in sorted(os.listdir(dirpath)): if not name.endswith(".md") or name == "README.md": continue path = os.path.join(dirpath, name) if not os.path.isfile(path): continue row = {"file": name, "date": None, "recipient": None, "number": None, "heading": None, "classes": []} m = NAME_RE.match(name) legacy = interview = False if m: y, mo, d, row["recipient"], num = m.groups() row["number"] = int(num) else: m = LEGACY_RE.match(name) if m: y, mo, d, row["recipient"] = m.groups() legacy = True else: m = INTERVIEW_RE.match(name) if not m: row["classes"].append("NAME_MALFORMED") rows.append(row) continue y, mo, d, row["recipient"] = m.groups() interview = True row["date"] = "%s-%s-%s" % (y, mo, d) if not _real_date(y, mo, d): row["classes"].append("BAD_DATE") if legacy: row["classes"].append("LEGACY_UNNUMBERED") if interview: row["classes"].append("INTERVIEW_UNNUMBERED") heading = read_heading(path, head_lines) unreadable = heading is False row["heading"] = heading if isinstance(heading, str) else None if unreadable: row["classes"].append("UNREADABLE") elif heading is None: row["classes"].append("NO_HEADER") else: named = heading_addressee(heading, roster) if named is None: row["classes"].append("NO_RECIPIENT_IN_HEADING") elif not _recipient_matches(named, row["recipient"]): row["classes"].append("HEADER_RECIPIENT") if row["number"] is not None: hnum = HEADING_NUM.search(heading) if hnum is not None: if int(hnum.group(1)) != row["number"]: row["classes"].append("HEADER_NUMBER") elif not re.search(r"(? 1}) for n in dup: for row in rows: if row["number"] == n and row["recipient"] == who: row["classes"].append("DUPLICATE_NUMBER") missing = [n for n in range(min(nums), max(nums) + 1) if n not in nums] if missing: holes[who] = missing faults = [r for r in rows if any(c in FAULTS for c in r["classes"])] advisories = [r for r in rows if any(c in ADVISORIES for c in r["classes"])] return rows, faults, advisories, series, holes # ---------------------------------------------------------------- baseline BASELINE_CLASSES = set(FAULTS) def emit_baseline(rows): """The ledger as text: one `\\t` row per faulting file. Advisory classes are left out on purpose: they are never rc 1, so a ledger that named them would read as if they had been tolerated out of a grade they never had. Sorted, so an unchanged archive regenerates byte-identical. """ head = ("# reply baseline -- the faults this archive already carried when the\n" "# heading rule was written. One row per file: \\t.\n" "# Regenerate: reply_check.py --repo . --emit-baseline > tools/reply_baseline.tsv\n" "# A row is a stated tolerance, not an exemption: a file that gains a class,\n" "# and a file that is not listed, both make rc 1 under --baseline.\n") lines = [] for row in sorted(rows, key=lambda r: r["file"]): cls = sorted(c for c in row["classes"] if c in BASELINE_CLASSES) if cls: lines.append("%s\t%s" % (row["file"], ",".join(cls))) return head + "\n".join(lines) + ("\n" if lines else "") def load_baseline(path): """Read the ledger -> (ledger, None), or (None, reason) when it cannot be read. Same contract as `checkin_baseline.tsv`: an unreadable, non-UTF-8, wrongly shaped or unknown-class row is a refusal, because grading against an empty ledger would call the whole archive new and skipping the check would call it fine -- both are silent lies about the same fact. """ try: with open(path, "r", encoding="utf-8") as fh: text = fh.read() except OSError as exc: return None, "cannot grade: baseline %s: %s" % (path, exc) except UnicodeDecodeError as exc: return None, "cannot grade: baseline %s is not UTF-8: %s" % (path, exc) out = {} for n, line in enumerate(text.splitlines(), 1): s = line.rstrip("\r") if not s.strip() or s.lstrip().startswith("#"): continue parts = s.split("\t") if len(parts) != 2: return None, ("cannot grade: baseline %s:%d is not " "\\t: %r" % (path, n, s)) name = parts[0].strip() cls = [c.strip() for c in parts[1].split(",") if c.strip()] bad = [c for c in cls if c not in BASELINE_CLASSES] if not name or not cls or bad: return None, ("cannot grade: baseline %s:%d names an empty filename, " "an empty class list, or an unknown class %s: %r" % (path, n, bad, s)) out[name] = set(cls) return out, None def apply_baseline(rows, ledger, path=""): """Split every row's graded classes into known and new, and count the stale.""" current = {r["file"]: set(c for c in r["classes"] if c in BASELINE_CLASSES) for r in rows} known_files = 0 for row in rows: cls = current[row["file"]] known = ledger.get(row["file"], set()) new = sorted(cls - known) row["baseline"] = {"known": sorted(cls & known), "new": new} if cls and not new: known_files += 1 stale = sorted(f for f in ledger if f not in current) cleared = sorted(f for f in ledger if f in current and (ledger[f] - current[f])) return {"path": path, "entries": len(ledger), "known_files": known_files, "new_faults": sum(1 for r in rows if r["baseline"]["new"]), "stale": stale, "cleared": cleared} def report(path, rows, faults, advisories, series, holes, roster_source, as_json=None, ledger=None): counts = {} for row in rows: for c in row["classes"]: counts[c] = counts.get(c, 0) + 1 print("reply_check -- %d file(s) under %s (roster: %s)" % (len(rows), path, roster_source)) for cls in FAULTS + ADVISORIES: n = counts.get(cls, 0) if n: print(" %-24s %d" % (cls, n)) for row in rows: if cls in row["classes"]: extra = "" if row["heading"]: extra = " | heading: %s" % row["heading"][:96] print(" %s%s" % (row["file"], extra)) else: print(" %-24s 0" % cls) print("series: %d recipient(s), %d numbered file(s), %d hole(s)%s" % (len(series), sum(len(v) for v in series.values()), len(holes), (": " + ", ".join("%s %s" % (k, v) for k, v in sorted(holes.items()))) if holes else "")) print("faults: %d advisories: %d" % (len(faults), len(advisories))) if ledger is not None: print("baseline %s: %d entries, %d file(s) known-faulting, " "%d new fault(s), %d stale, %d cleared" % (ledger["path"], ledger["entries"], ledger["known_files"], ledger["new_faults"], len(ledger["stale"]), len(ledger["cleared"]))) for f in ledger["stale"]: print(" STALE BASELINE %s (no longer in the directory: the record " "was repaired)" % f) for f in ledger["cleared"]: print(" STALE BASELINE %s (no longer carries every class the " "ledger names)" % f) if as_json: with open(as_json, "w", encoding="utf-8") as fh: json.dump({"path": path, "files": len(rows), "counts": counts, "holes": holes, "rows": rows, "baseline": ledger}, fh, indent=2, sort_keys=True) fh.write("\n") print("json: %s" % as_json) if ledger is not None: bad = [(r["file"], ",".join(r["baseline"]["new"])) for r in rows if r["baseline"]["new"]] print("reply_check: %d file(s), %d new fault(s)%s, %d known-faulting in " "the baseline" % (len(rows), len(bad), (": " + ", ".join("%s (%s)" % b for b in bad)) if bad else "", ledger["known_files"])) if ledger["new_faults"]: print("FAIL: %d file(s) carry a class the baseline does not tolerate" % ledger["new_faults"]) return 1 if ledger["new_faults"] else 0 if faults: print("FAIL: %d file(s) violate the reply heading contract" % len(faults)) return 1 return 0 def _write(d, name, text): with open(os.path.join(d, name), "w", encoding="utf-8") as fh: fh.write(text) def selftest(): """Fixture cells. Each cell builds a directory, runs scan()/main(), and asserts the class list AND the exit code -- both directions, so a rule that stops firing and a rule that fires everywhere both fail.""" cells = [] ROSTER = {"volt", "merlin", "timmy", "zephyrus"} ROSTER_ARG = "volt,merlin,timmy,zephyrus" def cell(label, ok, detail=None): cells.append((label, bool(ok), detail)) def classes(d, name): rows, _, _, _, _ = scan(d, roster=ROSTER) for row in rows: if row["file"] == name: return row["classes"] return None def rc(argv): # The fixtures' own reports are noise here; the cell is the verdict. with contextlib.redirect_stdout(io.StringIO()), \ contextlib.redirect_stderr(io.StringIO()): try: return run(argv + ["--roster", ROSTER_ARG]) except SystemExit as e: # pragma: no cover - defensive return e.code root = tempfile.mkdtemp(prefix="reply-selftest-") # 1 clean numbered file, heading agrees in name and number d = os.path.join(root, "clean") os.makedirs(d) _write(d, "2026-10-01-to-zephyrus-02.md", "# Reply 2 -- to Zephyrus\n\nbody\n") _write(d, "README.md", "not a reply\n") cell("clean file has no class", classes(d, "2026-10-01-to-zephyrus-02.md") == []) cell("README.md is not a row", len(scan(d)[0]) == 1) cell("clean run exits 0", rc(["--dir", d]) == 0) # 2 header names the other party -- the money rule d = os.path.join(root, "wrongparty") os.makedirs(d) _write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane), Run 33\n\nbody\n") cell("wrong addressee is HEADER_RECIPIENT", classes(d, "2026-10-02-to-timmy-58.md") == ["HEADER_RECIPIENT"]) cell("wrong addressee exits 1", rc(["--dir", d]) == 1) # 3 heading's number disagrees with the filename's d = os.path.join(root, "wrongnum") os.makedirs(d) _write(d, "2026-10-02-to-merlin-07.md", "# Reply 6 -- to Merlin\n\nbody\n") cell("wrong number is HEADER_NUMBER", classes(d, "2026-10-02-to-merlin-07.md") == ["HEADER_NUMBER"]) cell("wrong number exits 1", rc(["--dir", d]) == 1) # 4 a hyphenated handle in the heading still matches its prefix d = os.path.join(root, "handle") os.makedirs(d) _write(d, "2026-10-03-to-zephyrus-58.md", "# 2026-10-03 -- to zephyrus-sparkborn (reply 58)\n\nbody\n") cell("hyphenated handle matches", classes(d, "2026-10-03-to-zephyrus-58.md") == []) # 5 duplicate number within one recipient's series d = os.path.join(root, "dup") os.makedirs(d) _write(d, "2026-10-04-to-volt-03.md", "# Reply 3 -- to Volt\n") _write(d, "2026-10-05-to-volt-03.md", "# Reply 3 -- to Volt\n") got = classes(d, "2026-10-05-to-volt-03.md") cell("duplicate number is DUPLICATE_NUMBER", got == ["DUPLICATE_NUMBER"]) cell("duplicate exits 1", rc(["--dir", d]) == 1) # 6 a hole is named, never graded d = os.path.join(root, "hole") os.makedirs(d) _write(d, "2026-10-04-to-volt-03.md", "# Reply 3 -- to Volt\n") _write(d, "2026-10-05-to-volt-05.md", "# Reply 5 -- to Volt\n") _, faults, _, _, holes = scan(d) cell("hole is reported with its number", holes == {"volt": [4]}) cell("hole is not a fault", not faults and rc(["--dir", d]) == 0) # 7 the one legacy unnumbered form is advisory, not a fault d = os.path.join(root, "legacy") os.makedirs(d) _write(d, "2026-09-24-to-zephyrus.md", "# 2026-09-24 -- to zephyrus\n\nbody\n") cell("legacy form is advisory", classes(d, "2026-09-24-to-zephyrus.md") == ["LEGACY_UNNUMBERED"]) cell("legacy form exits 0", rc(["--dir", d]) == 0) # 7b the reported-interview form: no number by construction, so the # numbering rules skip it -- but everything a name can still contradict # (its date, its addressee) is graded exactly as on a numbered reply. d = os.path.join(root, "interview") os.makedirs(d) _write(d, "2026-10-19-to-zephyrus-interview.md", "# Interview to zephyrus -- 2026-10-19\n\nbody\n") cell("interview form is advisory, and only that", classes(d, "2026-10-19-to-zephyrus-interview.md") == ["INTERVIEW_UNNUMBERED"]) cell("interview form exits 0", rc(["--dir", d]) == 0) _, _, _, series_i, holes_i = scan(d) cell("interview form contributes no number to any series", series_i == {} and holes_i == {}) d = os.path.join(root, "interview-wrongparty") os.makedirs(d) _write(d, "2026-10-19-to-zephyrus-interview.md", "# Interview to Merlin -- 2026-10-19\n") cell("interview heading still grades its addressee", classes(d, "2026-10-19-to-zephyrus-interview.md") == ["INTERVIEW_UNNUMBERED", "HEADER_RECIPIENT"]) cell("interview with the wrong addressee exits 1", rc(["--dir", d]) == 1) d = os.path.join(root, "interview-baddate") os.makedirs(d) _write(d, "2026-02-30-to-zephyrus-interview.md", "# Interview to zephyrus\n") cell("interview form still grades its date", classes(d, "2026-02-30-to-zephyrus-interview.md") == ["BAD_DATE", "INTERVIEW_UNNUMBERED"]) d = os.path.join(root, "interview-nearmiss") os.makedirs(d) _write(d, "2026-10-19-to-zephyrus-inteview.md", "# Interview to zephyrus\n") cell("a misspelled interview slug is still NAME_MALFORMED", classes(d, "2026-10-19-to-zephyrus-inteview.md") == ["NAME_MALFORMED"]) cell("a misspelled interview slug exits 1", rc(["--dir", d]) == 1) # 8 a real-calendar check, not a shape check d = os.path.join(root, "baddate") os.makedirs(d) _write(d, "2026-02-30-to-volt-04.md", "# Reply 4 -- to Volt\n") cell("impossible date is BAD_DATE", classes(d, "2026-02-30-to-volt-04.md") == ["BAD_DATE"]) # 9 no heading where one belongs d = os.path.join(root, "noheader") os.makedirs(d) _write(d, "2026-10-06-to-volt-06.md", "Reply 6 -- to Volt\n\nbody\n") cell("prose before any heading is NO_HEADER", classes(d, "2026-10-06-to-volt-06.md") == ["NO_HEADER"]) # 10 a name that is neither form d = os.path.join(root, "badname") os.makedirs(d) _write(d, "2026-10-06-to-volt-6.md", "# Reply 6 -- to Volt\n") cell("unpadded number is NAME_MALFORMED", classes(d, "2026-10-06-to-volt-6.md") == ["NAME_MALFORMED"]) cell("NAME_MALFORMED exits 1", rc(["--dir", d]) == 1) # 11 an early heading that names nobody: advisory, never a fault d = os.path.join(root, "norecipient") os.makedirs(d) _write(d, "2026-10-11-to-merlin-14.md", "# Reply 14 -- Q31 (Run 6) picked up from this host\n") cell("heading naming nobody is advisory", classes(d, "2026-10-11-to-merlin-14.md") == ["NO_RECIPIENT_IN_HEADING"]) cell("advisory exits 0", rc(["--dir", d]) == 0) # 12 bytes that are not UTF-8: a fault with the decoding named, never a # silent "no heading" d = os.path.join(root, "binary") os.makedirs(d) with open(os.path.join(d, "2026-10-07-to-volt-07.md"), "wb") as fh: fh.write(b"# Reply 7 \xff\xfe to Volt\n") cell("non-UTF-8 bytes are UNREADABLE", classes(d, "2026-10-07-to-volt-07.md") == ["UNREADABLE"]) # 13 the read contract: a path that cannot be read is rc 2, never 0/1 cell("missing directory exits 2", rc(["--dir", os.path.join(root, "nope")]) == 2) _write(root, "afile", "not a directory\n") cell("a file where a directory belongs exits 2", rc(["--dir", os.path.join(root, "afile")]) == 2) # 14-25: the baseline ledger. The live archive carries one fault in another # lane's file, so the gate has to be able to state that tolerance without # hiding a new fault: every cell here reads the tolerant side AND the side # that must still fail. d = os.path.join(root, "baseline") os.makedirs(d) _write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane)\n") _write(d, "2026-10-03-to-merlin-07.md", "# Reply 7 -- to Merlin\n") rows_b, _, _, _, _ = scan(d, roster=ROSTER) led_text = emit_baseline(rows_b) cell("emit names the faulting file and its class", "2026-10-02-to-timmy-58.md\tHEADER_RECIPIENT" in led_text) cell("emit leaves the clean file out", "2026-10-03-to-merlin-07.md" not in led_text) bl = os.path.join(root, "reply.tsv") with open(bl, "w", encoding="utf-8") as fh: fh.write(led_text) ledger_, why_ = load_baseline(bl) cell("the emitted ledger round-trips", ledger_ == {"2026-10-02-to-timmy-58.md": {"HEADER_RECIPIENT"}} and why_ is None, (ledger_, why_)) with contextlib.redirect_stdout(io.StringIO()): rv_known = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) cell("a known fault under its ledger exits 0", rv_known == 0) buf = io.StringIO() with contextlib.redirect_stdout(buf): run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) printed = buf.getvalue() cell("the tolerated fault is still printed, never hidden", "HEADER_RECIPIENT" in printed and "1 known-faulting" in printed, printed[-160:]) # the same file gains a class the ledger does not name -> rc 1, and the new # class is named rather than the whole row re-graded _write(d, "2026-10-02-to-timmy-58.md", "# Reply 9 -- to Merlin (Helm lane)\n") rows_new, _, _, _, _ = scan(d) led_new = apply_baseline(rows_new, load_baseline(bl)[0], bl) cell("a new class on a known file is the only new fault", led_new["new_faults"] == 1 and [r["baseline"]["new"] for r in rows_new if r["baseline"]["new"]] == [["HEADER_NUMBER"]], led_new) with contextlib.redirect_stdout(io.StringIO()): rc_grown = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) cell("a known file that gains a class exits 1", rc_grown == 1) # a faulting file the ledger does not list at all -> rc 1 _write(d, "2026-10-02-to-timmy-58.md", "# Reply 58 -- to Merlin (Helm lane)\n") _write(d, "2026-10-04-to-volt-01.md", "# Reply 1 -- to Timmy\n") with contextlib.redirect_stdout(io.StringIO()): rc_unlisted = run(["--dir", d, "--baseline", bl, "--roster", ROSTER_ARG]) cell("an unlisted faulting file exits 1", rc_unlisted == 1) # without a ledger the run still names its verdict in one line, so a reader # (and verify.py's detail extractor) gets a reason, not just a table buf = io.StringIO() with contextlib.redirect_stdout(buf): run(["--dir", d, "--roster", ROSTER_ARG]) cell("a faulting run prints a FAIL line", "FAIL: " in buf.getvalue(), buf.getvalue()[-160:]) # a ledger entry whose file is gone: printed STALE, never rc 1 ds = os.path.join(root, "baseline-stale") os.makedirs(ds) _write(ds, "2026-10-03-to-merlin-07.md", "# Reply 7 -- to Merlin\n") bls = os.path.join(root, "stale.tsv") with open(bls, "w", encoding="utf-8") as fh: fh.write("2026-10-09-to-merlin-99.md\tHEADER_RECIPIENT\n") fh.write("2026-10-03-to-merlin-07.md\tHEADER_NUMBER\n") buf = io.StringIO() with contextlib.redirect_stdout(buf): rc_stale = run(["--dir", ds, "--baseline", bls, "--roster", ROSTER_ARG]) stale_out = buf.getvalue() cell("a stale ledger entry is reported and never rc 1", rc_stale == 0 and "2026-10-09-to-merlin-99.md" in stale_out, stale_out[-200:]) cell("a cleared class is reported as stale, not as clean", "no longer carries every class" in stale_out, stale_out[-200:]) # a ledger that cannot be read is rc 2, never a pass cell("a missing ledger exits 2", rc(["--dir", ds, "--baseline", os.path.join(root, "absent.tsv")]) == 2) bad1 = os.path.join(root, "bad-nosep.tsv") with open(bad1, "w", encoding="utf-8") as fh: fh.write("2026-10-09-to-merlin-99.md HEADER_RECIPIENT\n") cell("a ledger line without the tab exits 2", rc(["--dir", ds, "--baseline", bad1]) == 2) bad2 = os.path.join(root, "bad-class.tsv") with open(bad2, "w", encoding="utf-8") as fh: fh.write("2026-10-09-to-merlin-99.md\tNOT_A_CLASS\n") cell("a ledger naming an unknown class exits 2", rc(["--dir", ds, "--baseline", bad2]) == 2) # 38-41: the docstring's own census. The two spelled counts live only in # prose, so nothing graded them until this cell; the enumeration beside each # one is the copy a reader follows, and FAULTS/ADVISORIES are what the code # grades. A class added to either table without its docstring row, or a # count word left behind, fails here rather than in the next reader's quote. graded_w, advisory_w = doc_counts() cell("docstring graded-count word matches FAULTS", graded_w == len(FAULTS), "docstring says %r; FAULTS holds %d" % (graded_w, len(FAULTS))) cell("docstring advisory-count word matches ADVISORIES", advisory_w == len(ADVISORIES), "docstring says %r; ADVISORIES holds %d" % (advisory_w, len(ADVISORIES))) unnamed = [c for c in FAULTS + ADVISORIES if c not in (__doc__ or "")] cell("every class the code grades is named in the docstring", not unnamed, unnamed) # the instrument itself, against the words this file carried before the fix prefix_doc = (" file says about itself. Five classes are graded:\n" " Three more are counted and printed, and are ADVISORY.\n") cell("the pre-fix count words are refused", doc_counts(prefix_doc) != (len(FAULTS), len(ADVISORIES)), doc_counts(prefix_doc)) failed = [(label, detail) for label, ok, detail in cells if not ok] print("selftest: %d cells, %d passed, %d failed" % (len(cells), len(cells) - len(failed), len(failed))) for label, detail in failed: print(" FAILED: %s" % label) if detail is not None: print(" got: %r" % (detail,)) return 1 if failed else 0 def run(argv): ap = argparse.ArgumentParser( description="grade every reply file's heading against its own filename") ap.add_argument("--repo", default=".", help="the repository the roster is read from (default: .)") ap.add_argument("--dir", default=None, help="the replies directory (default: /replies)") ap.add_argument("--roster", default=None, help="comma-separated handles, overriding WIZARDS-TABLE.md and .mailmap") ap.add_argument("--head-lines", type=int, default=10, help="how many leading lines may hold the heading (default 10)") ap.add_argument("--json", help="write the full row report here") ap.add_argument("--baseline", default=None, help="known-fault ledger (lines of \\t): " "only a class the ledger does not name on that file is rc 1") ap.add_argument("--emit-baseline", action="store_true", help="print the ledger for this directory and exit 0") args = ap.parse_args(argv) dirpath = args.dir if args.dir is not None else os.path.join(args.repo, "replies") if not os.path.exists(dirpath): print("reply_check: cannot read replies: %s: no such directory" % dirpath, file=sys.stderr) return 2 if not os.path.isdir(dirpath): print("reply_check: cannot read replies: %s: not a directory" % dirpath, file=sys.stderr) return 2 roster, source = load_roster(args.repo, args.roster) rows, faults, advisories, series, holes = scan(dirpath, args.head_lines, roster) if args.emit_baseline: sys.stdout.write(emit_baseline(rows)) return 0 ledger = None if args.baseline: ledger, why = load_baseline(args.baseline) if ledger is None: print("reply_check: %s" % why, file=sys.stderr) return 2 ledger = apply_baseline(rows, ledger, args.baseline) return report(dirpath, rows, faults, advisories, series, holes, source, args.json, ledger) def main(): argv = sys.argv[1:] if "--selftest" in argv: return selftest() return run(argv) if __name__ == "__main__": sys.exit(main())