tools (volt): the gate gets its fourth leg -- the check-in contract, graded on the newest day

Measured hole: on a clone of 3890c12 carrying checkins/2026-10-19-volt-02.md,
checkin_check.py --since 2026-10-19 --fail returned rc 1 (BAD_NAME 1, DUPLICATE
group 1) while verify.py --fast returned rc 0, 'gate: OK -- 2 of 2 graded leg(s)
passed'. Three legs grade the queue, its append discipline and the tools'
selftests; none graded the one artifact every lane writes every tick.

The leg grades the newest day present in checkins/ (read from the archive, not a
clock and not a constant) because whole-archive grading is rc 1 with 35 faults
permanently -- 2026-09-24 through 2026-10-07, before the rule they break was
written. checkin_check --fail at 3890c12: 73 files, 35 with a fault, rc 1.
--checkin-day DATE sets the --since floor explicitly. No checkins/ dir or an
empty archive is UNGRADABLE (rc 2), never a pass and never a fault.

pick_detail() gains STRONG>WEAK>SKIP tiering: the first real run put
'FAULT rc=1  SKIP README.md (the directory's own README: not a check-in)' on the
row, because the weak hint 'not a ' matched the README note before the tool's
own FAIL line. Silent-class defect, fixed here with cells 18/18b.

Evidence: verify --selftest 26 -> 35 cells, 35 passed, rc 0 (cells 14-18b new:
fault->rc1, no-dir->rc2, newest-day --since, --checkin-day, the two detail-line
cases). ruff -F,E9 tools/ All checks passed. Live gate: 4 legs, 4 pass, rc 0.
Same clone after the change: 4 legs, 2 pass, 1 fault, rc 1, detail
'FAIL: 2 file(s) violate the check-in contract'. doc_census_check --tools
tools/verify.py: 6 claims, 0 STALE, 0 INTERNAL SPLIT, rc 0.
Artifact: /root/.hermes/wizard-outputs/volt/2026-10-19-checkin-leg/
This commit is contained in:
volt committed 2026-10-19 12:14:47 +00:00
1 parent 12041210ff
commit a114eadf3d
1 file changed
+193 -36
+193 -36
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""verify.py -- the HOME the index gate did not have: one command, three legs, """verify.py -- the HOME the index gate did not have: one command, four legs,
one exit code, and a row per leg so a partial run cannot look like a clean one. one exit code, and a row per leg so a partial run cannot look like a clean one.
WHY THIS EXISTS WHY THIS EXISTS
@@ -14,10 +14,22 @@ WHY THIS EXISTS
The second reason is narrower and this lane's own: three separate counts of The second reason is narrower and this lane's own: three separate counts of
"which commits left the index stale" came out wrong in one week, each from a "which commits left the index stale" came out wrong in one week, each from a
hand-assembled probe loop. A gate whose legs are remembered is a gate that hand-assembled probe loop. A gate whose legs are remembered is a gate that
gets run selectively. Three legs, fixed order, one rc -- that is the whole gets run selectively. Fixed order, one rc -- that is the whole design.
design.
The third reason is measured, not argued. On 2026-10-19T09:42Z this lane put a
contract-breaking check-in into a clone of HEAD -- `checkins/2026-10-19-volt-02.md`,
a second same-day file carrying a suffix -- and ran both instruments on it:
`checkin_check.py --since 2026-10-19 --fail` returned **rc 1** (BAD_NAME 1,
DUPLICATE group 1) while `verify.py --fast` returned **rc 0** on the same tree,
"gate: OK -- 2 of 2 graded leg(s) passed". Three legs that grade the queue, its
append discipline and the tools' selftests said nothing about the one artifact
every lane writes every tick. A fourth leg closes that hole.
WHAT IT RUNS -- one row per leg, cheapest first, never in parallel WHAT IT RUNS -- one row per leg, cheapest first, never in parallel
checkin tools/checkin_check.py --since <newest day> --fail
the newest day's check-ins against F1-F5 (names, duplicates,
unknown handles, dates, emptiness); `--checkin-day DATE` sets
the floor explicitly, from DATE forward
index-current tools/queue_index.py --check docs/queue-index.md index-current tools/queue_index.py --check docs/queue-index.md
the derived document, byte-compared against a fresh render the derived document, byte-compared against a fresh render
append-audit tools/queue_append_audit.py --guard --to HEAD append-audit tools/queue_append_audit.py --guard --to HEAD
@@ -30,6 +42,20 @@ WHAT IT RUNS -- one row per leg, cheapest first, never in parallel
one failed: a stale index and a broken selftest are different defects and the one failed: a stale index and a broken selftest are different defects and the
run should name both in one pass. run should name both in one pass.
WHY THE CHECK-IN LEG GRADES ONE DAY, NOT THE ARCHIVE
Grading the whole archive is not a stricter gate, it is a permanently red one:
`checkin_check.py --repo . --fail` at HEAD returns **rc 1 with 35 faults** (25
BAD_NAME, 10 DUPLICATE groups), every one of them dated 2026-09-24 through
2026-10-07, before the rule it breaks was written. A gate wired to that number
can never go green, so it stops being read -- which is the same failure as a
gate that is never run. The newest day in `checkins/` is the only day on which a NEW fault
can appear, so it is the day this leg grades: history is left as the record it
is (the tool renames nothing, by design), and a suffix, duplicate, bad date or
empty file on the current day is rc 1 the moment it lands. `--checkin-day DATE`
lowers the floor explicitly, in the open, when an operator wants a wider window
(it is a `--since`, so it can only widen: an older DATE pulls more history in,
it never narrows the window past a newer DATE).
THE EXIT CONTRACT -- the house three codes THE EXIT CONTRACT -- the house three codes
0 every graded leg passed 0 every graded leg passed
1 at least one leg ran and graded a fault 1 at least one leg ran and graded a fault
@@ -58,12 +84,13 @@ TOOLS ARE RESOLVED FROM --repo, NOT FROM THIS FILE'S DIRECTORY
docs/queue-index.md) because `queue_append_audit` reads them with docs/queue-index.md) because `queue_append_audit` reads them with
`git show <rev>:<path>`, which has no absolute-path form. `git show <rev>:<path>`, which has no absolute-path form.
`--selftest` fixture corpus, 26 cells `--selftest` fixture corpus, 35 cells
""" """
import argparse import argparse
import json import json
import os import os
import re
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
@@ -71,12 +98,16 @@ import time
QUEUE_DEFAULT = "WORK-QUEUE.md" QUEUE_DEFAULT = "WORK-QUEUE.md"
INDEX_DEFAULT = "docs/queue-index.md" INDEX_DEFAULT = "docs/queue-index.md"
CHECKINS_DEFAULT = "checkins"
PASS, FAULT, UNGRADABLE, SKIP = "PASS", "FAULT", "UNGRADABLE", "SKIP" PASS, FAULT, UNGRADABLE, SKIP = "PASS", "FAULT", "UNGRADABLE", "SKIP"
# Legs, in the fixed order they are printed and run. `tool` is resolved inside # Legs, in the fixed order they are printed and run. `tool` is resolved inside
# the repo being graded. `slow` marks the leg --fast drops. # the repo being graded. `slow` marks the leg --fast drops.
LEGS = ( LEGS = (
{"key": "checkin", "tool": "checkin_check.py", "slow": False,
"what": "grade the newest day's check-ins against the naming and "
"duplicate rules"},
{"key": "index-current", "tool": "queue_index.py", "slow": False, {"key": "index-current", "tool": "queue_index.py", "slow": False,
"what": "render the index and byte-compare it with the document"}, "what": "render the index and byte-compare it with the document"},
{"key": "append-audit", "tool": "queue_append_audit.py", "slow": False, {"key": "append-audit", "tool": "queue_append_audit.py", "slow": False,
@@ -87,8 +118,16 @@ LEGS = (
# A detail line is a courtesy, not the verdict -- but a verdict with no line # A detail line is a courtesy, not the verdict -- but a verdict with no line
# beside it is what the read-refusal axis calls SILENT, so it is graded too. # beside it is what the read-refusal axis calls SILENT, so it is graded too.
HINTS = ("STALE:", "FAIL", "FAILED", "fatal", "cannot", "TIMEOUT", "NO_", #
"error", "Error", "refus", "not a ") # STRONG outranks WEAK, and both outrank a per-file SKIP note. That tiering is
# not cosmetic: `checkin_check.py` prints " SKIP README.md (the directory's own
# README: not a check-in)" as its second line on every real run, "not a " is a
# WEAK hint, and before the tiering that note was carried onto a FAULT row --
# the row said FAULT beside a line about the README while the tool's own
# "FAIL: 2 file(s) violate the check-in contract" sat further down, unpicked.
STRONG_HINTS = ("STALE:", "FAIL:", "FAILED", "fatal", "TIMEOUT", "NO_",
"cannot", "error", "Error", "refus")
WEAK_HINTS = ("FAIL", "not a ")
def pick_detail(text, limit=150): def pick_detail(text, limit=150):
@@ -102,17 +141,43 @@ def pick_detail(text, limit=150):
and any line that names a real reason outranks them. and any line that names a real reason outranks them.
""" """
lines = [ln.strip() for ln in text.splitlines() if ln.strip()] lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
hinted = [ln for ln in lines if any(h in ln for h in HINTS)] kept = [ln for ln in lines if not ln.startswith("SKIP ")] or lines
hinted = [ln for ln in kept if any(h in ln for h in STRONG_HINTS + WEAK_HINTS)]
reasons = [ln for ln in hinted reasons = [ln for ln in hinted
if " 0 fatal" not in ln and " 0 fault" not in ln] if " 0 fatal" not in ln and " 0 fault" not in ln]
strong = [ln for ln in reasons if any(h in ln for h in STRONG_HINTS)]
if strong:
return strong[0][:limit]
if reasons: if reasons:
return reasons[0][:limit] return reasons[0][:limit]
if hinted: if hinted:
return hinted[0][:limit] return hinted[0][:limit]
return lines[-1][:limit] if lines else "" return kept[-1][:limit] if kept else ""
def leg_argv(key, repo, queue, index): def newest_checkin_day(repo):
"""The newest `<YYYY-MM-DD>` prefix in checkins/ -> (day, why-not).
The leg grades one day and this is how that day is chosen: the archive's own
latest date, so nothing has to be configured and no clock is consulted. A
repo with no `checkins/` directory or no dated file in it cannot be graded
and says which of the two it is.
"""
d = os.path.join(repo, CHECKINS_DEFAULT)
if not os.path.isdir(d):
return None, "no check-in directory: %s" % d
days = sorted(m.group(1) for m in
(re.match(r"^(\d{4}-\d{2}-\d{2})-", n) for n in os.listdir(d))
if m)
if not days:
return None, "no dated check-in files in %s" % d
return days[-1], None
def leg_argv(key, repo, queue, index, checkin_day=None):
if key == "checkin":
return [sys.executable, os.path.join(repo, "tools", "checkin_check.py"),
"--repo", repo, "--fail", "--since", checkin_day]
if key == "index-current": if key == "index-current":
return [sys.executable, os.path.join(repo, "tools", "queue_index.py"), return [sys.executable, os.path.join(repo, "tools", "queue_index.py"),
"--repo", repo, "--queue", queue, "--check", index] "--repo", repo, "--queue", queue, "--check", index]
@@ -126,7 +191,7 @@ def leg_argv(key, repo, queue, index):
raise KeyError(key) raise KeyError(key)
def run_leg(leg, repo, queue, index, timeout): def run_leg(leg, repo, queue, index, timeout, checkin_day=None):
"""One leg -> one row. rc 0/1/2 are graded; anything else is ungradable.""" """One leg -> one row. rc 0/1/2 are graded; anything else is ungradable."""
tool = os.path.join(repo, "tools", leg["tool"]) tool = os.path.join(repo, "tools", leg["tool"])
started = time.time() started = time.time()
@@ -136,8 +201,15 @@ def run_leg(leg, repo, queue, index, timeout):
row.update(verdict=UNGRADABLE, detail="no such tool: %s" % tool) row.update(verdict=UNGRADABLE, detail="no such tool: %s" % tool)
row["ms"] = 0 row["ms"] = 0
return row return row
day = checkin_day
if leg["key"] == "checkin" and not day:
day, why = newest_checkin_day(repo)
if day is None:
row.update(verdict=UNGRADABLE, detail=why)
row["ms"] = 0
return row
try: try:
proc = subprocess.run(leg_argv(leg["key"], repo, queue, index), proc = subprocess.run(leg_argv(leg["key"], repo, queue, index, day),
cwd=repo, capture_output=True, text=True, cwd=repo, capture_output=True, text=True,
timeout=timeout) timeout=timeout)
except subprocess.TimeoutExpired: except subprocess.TimeoutExpired:
@@ -218,20 +290,25 @@ def report(rows, repo, quiet=False):
def main(argv=None): def main(argv=None):
ap = argparse.ArgumentParser( ap = argparse.ArgumentParser(
description="Run the repo's gates as one command: index currency, the " description="Run the repo's gates as one command: the newest day's "
"append audit, and the selftest sweep.") "check-ins, index currency, the append audit, and the "
"selftest sweep.")
ap.add_argument("--repo", default=".") ap.add_argument("--repo", default=".")
ap.add_argument("--queue", default=QUEUE_DEFAULT, ap.add_argument("--queue", default=QUEUE_DEFAULT,
help="repo-relative queue path (default %s)" % QUEUE_DEFAULT) help="repo-relative queue path (default %s)" % QUEUE_DEFAULT)
ap.add_argument("--index", default=INDEX_DEFAULT, ap.add_argument("--index", default=INDEX_DEFAULT,
help="repo-relative index document (default %s)" % INDEX_DEFAULT) help="repo-relative index document (default %s)" % INDEX_DEFAULT)
ap.add_argument("--checkin-day", default=None, metavar="DATE",
help="grade check-ins from DATE forward instead of from the "
"newest day present in checkins/ (a --since floor: it "
"can widen the window, never narrow it)")
ap.add_argument("--fast", action="store_true", ap.add_argument("--fast", action="store_true",
help="skip the selftest-sweep leg; the row is printed as SKIP") help="skip the selftest-sweep leg; the row is printed as SKIP")
ap.add_argument("--timeout", type=int, default=300, metavar="S", ap.add_argument("--timeout", type=int, default=300, metavar="S",
help="per-leg timeout in seconds (default 300)") help="per-leg timeout in seconds (default 300)")
ap.add_argument("--json", metavar="PATH", help="write the rows here") ap.add_argument("--json", metavar="PATH", help="write the rows here")
ap.add_argument("--quiet", action="store_true") ap.add_argument("--quiet", action="store_true")
ap.add_argument("--selftest", action="store_true", help="fixture corpus, 26 cells") ap.add_argument("--selftest", action="store_true", help="fixture corpus, 35 cells")
a = ap.parse_args(argv) a = ap.parse_args(argv)
if a.selftest: if a.selftest:
@@ -252,7 +329,8 @@ def main(argv=None):
"rc": None, "detail": "--fast: leg not run", "rc": None, "detail": "--fast: leg not run",
"what": leg["what"], "ms": 0}) "what": leg["what"], "ms": 0})
continue continue
rows.append(run_leg(leg, repo, a.queue, a.index, a.timeout)) rows.append(run_leg(leg, repo, a.queue, a.index, a.timeout,
a.checkin_day))
rc = report(rows, repo, a.quiet) rc = report(rows, repo, a.quiet)
if a.json: if a.json:
@@ -266,13 +344,15 @@ def main(argv=None):
# ------------------------------------------------------------------- selftest # ------------------------------------------------------------------- selftest
def _stub(path, rc, out="", sleep=0.0, marker=None): def _stub(path, rc, out="", sleep=0.0, marker=None, argv_marker=None):
"""A stand-in leg: it prints what the fixture needs and exits with its rc.""" """A stand-in leg: it prints what the fixture needs and exits with its rc."""
body = ["#!/usr/bin/env python3", "import sys"] body = ["#!/usr/bin/env python3", "import sys"]
if sleep: if sleep:
body.append("import time; time.sleep(%r)" % sleep) body.append("import time; time.sleep(%r)" % sleep)
if marker: if marker:
body.append("open(%r, 'a').write('hit\\n')" % marker) body.append("open(%r, 'a').write('hit\\n')" % marker)
if argv_marker:
body.append("open(%r, 'w').write(' '.join(sys.argv[1:]))" % argv_marker)
if out: if out:
body.append("sys.stdout.write(%r)" % out) body.append("sys.stdout.write(%r)" % out)
body.append("sys.exit(%d)" % rc) body.append("sys.exit(%d)" % rc)
@@ -281,8 +361,14 @@ def _stub(path, rc, out="", sleep=0.0, marker=None):
os.chmod(path, 0o755) os.chmod(path, 0o755)
def _fixture(base, rcs, outs=None, sleep=0.0, markers=None): def _fixture(base, rcs, outs=None, sleep=0.0, markers=None, argv_markers=None,
"""A throwaway repo with three stub legs. rcs: {tool: rc}.""" checkins=("2026-01-01-fixture.md",)):
"""A throwaway repo with stubbed legs. rcs: {tool: rc}.
`checkins` seeds `checkins/`: one dated file by default (so the check-in leg
has a day to grade), a tuple of names for cells that need a specific set,
and None for a repo with no `checkins/` directory at all.
"""
repo = os.path.join(base, "repo") repo = os.path.join(base, "repo")
os.makedirs(os.path.join(repo, "tools")) os.makedirs(os.path.join(repo, "tools"))
os.makedirs(os.path.join(repo, ".git")) os.makedirs(os.path.join(repo, ".git"))
@@ -291,12 +377,19 @@ def _fixture(base, rcs, outs=None, sleep=0.0, markers=None):
os.makedirs(os.path.join(repo, "docs")) os.makedirs(os.path.join(repo, "docs"))
with open(os.path.join(repo, INDEX_DEFAULT), "w") as fh: with open(os.path.join(repo, INDEX_DEFAULT), "w") as fh:
fh.write("fixture\n") fh.write("fixture\n")
if checkins is not None:
os.makedirs(os.path.join(repo, CHECKINS_DEFAULT))
for name in checkins:
with open(os.path.join(repo, CHECKINS_DEFAULT, name), "w") as fh:
fh.write("Did: fixture\nBlocked: nothing\nNext: nothing\n"
"Ask: nothing\n")
for tool, rc in rcs.items(): for tool, rc in rcs.items():
if rc is None: # tool absent on purpose if rc is None: # tool absent on purpose
continue continue
_stub(os.path.join(repo, "tools", tool), rc, _stub(os.path.join(repo, "tools", tool), rc,
out=(outs or {}).get(tool, ""), sleep=sleep, out=(outs or {}).get(tool, ""), sleep=sleep,
marker=(markers or {}).get(tool)) marker=(markers or {}).get(tool),
argv_marker=(argv_markers or {}).get(tool))
return repo return repo
@@ -307,7 +400,8 @@ def _verify(*argv):
return proc.returncode, (proc.stdout or "") + (proc.stderr or "") return proc.returncode, (proc.stdout or "") + (proc.stderr or "")
THREE = {"queue_index.py": 0, "queue_append_audit.py": 0, "selftest_sweep.py": 0} GREEN4 = {"checkin_check.py": 0, "queue_index.py": 0,
"queue_append_audit.py": 0, "selftest_sweep.py": 0}
def selftest(): def selftest():
@@ -325,15 +419,15 @@ def selftest():
with tempfile.TemporaryDirectory(prefix="verify-selftest-") as td: with tempfile.TemporaryDirectory(prefix="verify-selftest-") as td:
# 1. three green stubs -> rc 0 # 1. three green stubs -> rc 0
repo = _fixture(os.path.join(td, "c1"), THREE) repo = _fixture(os.path.join(td, "c1"), GREEN4)
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("1 all three legs green -> rc 0", rc, 0) check("1 all four legs green -> rc 0", rc, 0)
check("1b summary counts the three legs", "verify: 3 legs, 3 pass, 0 fault, " check("1b summary counts the four legs", "verify: 4 legs, 4 pass, 0 fault, "
"0 ungradable, 0 skipped" in out, True) "0 ungradable, 0 skipped" in out, True)
# 2. a stale index is a FAULT, and its own line reaches the row # 2. a stale index is a FAULT, and its own line reaches the row
repo = _fixture(os.path.join(td, "c2"), repo = _fixture(os.path.join(td, "c2"),
dict(THREE, **{"queue_index.py": 1}), dict(GREEN4, **{"queue_index.py": 1}),
outs={"queue_index.py": outs={"queue_index.py":
"STALE: rendered document differs at line 7\n"}) "STALE: rendered document differs at line 7\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
@@ -343,7 +437,7 @@ def selftest():
# 3. a refusal is UNGRADABLE, not a fault # 3. a refusal is UNGRADABLE, not a fault
repo = _fixture(os.path.join(td, "c3"), repo = _fixture(os.path.join(td, "c3"),
dict(THREE, **{"queue_index.py": 2}), dict(GREEN4, **{"queue_index.py": 2}),
outs={"queue_index.py": "cannot read queue: nope\n"}) outs={"queue_index.py": "cannot read queue: nope\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("3 refusal -> rc 2", rc, 2) check("3 refusal -> rc 2", rc, 2)
@@ -351,7 +445,7 @@ def selftest():
# 4. precedence: ungradable outranks fault, and the fault is NOT erased # 4. precedence: ungradable outranks fault, and the fault is NOT erased
repo = _fixture(os.path.join(td, "c4"), repo = _fixture(os.path.join(td, "c4"),
dict(THREE, **{"queue_index.py": 1, dict(GREEN4, **{"queue_index.py": 1,
"selftest_sweep.py": 2}), "selftest_sweep.py": 2}),
outs={"queue_index.py": "STALE: line 3\n", outs={"queue_index.py": "STALE: line 3\n",
"selftest_sweep.py": "cannot read: no tools\n"}) "selftest_sweep.py": "cannot read: no tools\n"})
@@ -363,14 +457,14 @@ def selftest():
# 5. a missing leg tool is ungradable, and says which path was missing # 5. a missing leg tool is ungradable, and says which path was missing
repo = _fixture(os.path.join(td, "c5"), repo = _fixture(os.path.join(td, "c5"),
dict(THREE, **{"queue_append_audit.py": None})) dict(GREEN4, **{"queue_append_audit.py": None}))
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("5 missing tool -> rc 2", rc, 2) check("5 missing tool -> rc 2", rc, 2)
check("5b the missing path is named", "no such tool:" in out, True) check("5b the missing path is named", "no such tool:" in out, True)
# 6. --fast skips the sweep and does not run it # 6. --fast skips the sweep and does not run it
marker = os.path.join(td, "c6-sweep-ran") marker = os.path.join(td, "c6-sweep-ran")
repo = _fixture(os.path.join(td, "c6"), THREE, markers={"selftest_sweep.py": marker}) repo = _fixture(os.path.join(td, "c6"), GREEN4, markers={"selftest_sweep.py": marker})
rc, out = _verify("--repo", repo, "--fast") rc, out = _verify("--repo", repo, "--fast")
check("6 --fast on a green tree -> rc 0", rc, 0) check("6 --fast on a green tree -> rc 0", rc, 0)
check("6b the skipped leg is printed as SKIP", "selftest-sweep" in out check("6b the skipped leg is printed as SKIP", "selftest-sweep" in out
@@ -378,12 +472,12 @@ def selftest():
check("6c the skipped stub never ran", os.path.exists(marker), False) check("6c the skipped stub never ran", os.path.exists(marker), False)
# 7. --json is a real machine-readable record # 7. --json is a real machine-readable record
repo = _fixture(os.path.join(td, "c7"), THREE) repo = _fixture(os.path.join(td, "c7"), GREEN4)
jpath = os.path.join(td, "c7.json") jpath = os.path.join(td, "c7.json")
rc, out = _verify("--repo", repo, "--json", jpath) rc, out = _verify("--repo", repo, "--json", jpath)
obj = json.load(open(jpath)) if os.path.exists(jpath) else {} obj = json.load(open(jpath)) if os.path.exists(jpath) else {}
check("7 --json writes the rows", [r["key"] for r in obj.get("rows", [])], check("7 --json writes the rows", [r["key"] for r in obj.get("rows", [])],
["index-current", "append-audit", "selftest-sweep"]) ["checkin", "index-current", "append-audit", "selftest-sweep"])
check("7b --json carries the same rc the process returned", check("7b --json carries the same rc the process returned",
obj.get("rc"), rc) obj.get("rc"), rc)
@@ -394,7 +488,7 @@ def selftest():
# 9. rc 3 is retired: it is ungradable, never a fault and never a pass # 9. rc 3 is retired: it is ungradable, never a fault and never a pass
repo = _fixture(os.path.join(td, "c9"), repo = _fixture(os.path.join(td, "c9"),
dict(THREE, **{"queue_index.py": 3}), dict(GREEN4, **{"queue_index.py": 3}),
outs={"queue_index.py": "unknown state\n"}) outs={"queue_index.py": "unknown state\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
check("9 rc 3 -> rc 2 (retired code)", rc, 2) check("9 rc 3 -> rc 2 (retired code)", rc, 2)
@@ -402,29 +496,30 @@ def selftest():
# 10. a hung leg is a timeout, not a pass and not a fault # 10. a hung leg is a timeout, not a pass and not a fault
repo = _fixture(os.path.join(td, "c10"), repo = _fixture(os.path.join(td, "c10"),
dict(THREE, **{"selftest_sweep.py": 0}), sleep=5.0) dict(GREEN4, **{"selftest_sweep.py": 0}), sleep=5.0)
rc, out = _verify("--repo", repo, "--timeout", "1") rc, out = _verify("--repo", repo, "--timeout", "1")
check("10 timeout -> rc 2", rc, 2) check("10 timeout -> rc 2", rc, 2)
check("10b the timeout is named", "TIMEOUT after 1s" in out, True) check("10b the timeout is named", "TIMEOUT after 1s" in out, True)
# 11. order is fixed, cheapest first: the table is not re-sorted by outcome # 11. order is fixed, cheapest first: the table is not re-sorted by outcome
repo = _fixture(os.path.join(td, "c11"), repo = _fixture(os.path.join(td, "c11"),
dict(THREE, **{"queue_index.py": 1}), dict(GREEN4, **{"queue_index.py": 1}),
outs={"queue_index.py": "STALE: line 1\n"}) outs={"queue_index.py": "STALE: line 1\n"})
rc, out = _verify("--repo", repo) rc, out = _verify("--repo", repo)
order = [out.find(k) for k in ("index-current", "append-audit", "selftest-sweep")] order = [out.find(k) for k in ("checkin", "index-current", "append-audit",
"selftest-sweep")]
check("11 legs print in fixed order", order == sorted(order) check("11 legs print in fixed order", order == sorted(order)
and -1 not in order, True) and -1 not in order, True)
# 12. a skip is counted as a skip, never folded into pass # 12. a skip is counted as a skip, never folded into pass
repo = _fixture(os.path.join(td, "c12"), THREE) repo = _fixture(os.path.join(td, "c12"), GREEN4)
jpath = os.path.join(td, "c12.json") jpath = os.path.join(td, "c12.json")
rc, out = _verify("--repo", repo, "--fast", "--json", jpath) rc, out = _verify("--repo", repo, "--fast", "--json", jpath)
obj = json.load(open(jpath)) if os.path.exists(jpath) else {} obj = json.load(open(jpath)) if os.path.exists(jpath) else {}
check("12 skipped leg is its own count in --json", check("12 skipped leg is its own count in --json",
obj.get("counts", {}).get("SKIP"), 1) obj.get("counts", {}).get("SKIP"), 1)
check("12b and the skipped leg is not counted as a pass", check("12b and the skipped leg is not counted as a pass",
obj.get("counts", {}).get(PASS), 2) obj.get("counts", {}).get(PASS), 3)
# 13. the detail line must not contradict the verdict: a row graded # 13. the detail line must not contradict the verdict: a row graded
# FAULT beside "0 fatal" is the SILENT failure this gate exists to catch. # FAULT beside "0 fatal" is the SILENT failure this gate exists to catch.
@@ -438,6 +533,68 @@ def selftest():
"read-refusal: 19 probes, 19 ok, 0 fault\n"), "read-refusal: 19 probes, 19 ok, 0 fault\n"),
"sweep: 14 tools, 14 OK, 0 fatal") "sweep: 14 tools, 14 OK, 0 fatal")
# 14. THE HOLE THIS LEG CLOSES: a contract-breaking check-in is rc 1,
# and the gate is no longer green on a tree that carries one.
repo = _fixture(os.path.join(td, "c14"),
dict(GREEN4, **{"checkin_check.py": 1}),
outs={"checkin_check.py":
"FAIL: 2 file(s) violate the check-in contract\n"})
rc, out = _verify("--repo", repo, "--fast")
check("14 a broken check-in -> rc 1 (was rc 0 with three legs)", rc, 1)
check("14b the fault row is named checkin and carries its reason",
"checkin" in out and "violate the check-in contract" in out
and "fault: checkin" in out, True)
# 15. no checkins/ directory is ungradable, not a pass and not a fault
repo = _fixture(os.path.join(td, "c15"), GREEN4, checkins=None)
rc, out = _verify("--repo", repo, "--fast")
check("15 no checkins dir -> rc 2", rc, 2)
check("15b the ungradable row says which directory is missing",
"ungradable: checkin" in out and "no check-in directory:" in out, True)
# 16. the leg grades the NEWEST day in the archive, not the whole archive
av = os.path.join(td, "c16-argv")
repo = _fixture(os.path.join(td, "c16"), GREEN4,
argv_markers={"checkin_check.py": av},
checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md"))
rc, out = _verify("--repo", repo, "--fast")
seen = open(av).read() if os.path.exists(av) else ""
check("16 the check-in leg is run with --fail", "--fail" in seen, True)
check("16b and --since is the newest day present, not a clock or a constant",
"--since 2026-03-04" in seen, True)
# 17. --checkin-day retargets the window in the open
av = os.path.join(td, "c17-argv")
repo = _fixture(os.path.join(td, "c17"), GREEN4,
argv_markers={"checkin_check.py": av},
checkins=("2026-01-01-fixture.md", "2026-03-04-fixture.md"))
rc, out = _verify("--repo", repo, "--fast", "--checkin-day", "2026-01-01")
seen = open(av).read() if os.path.exists(av) else ""
check("17 --checkin-day overrides the default day",
"--since 2026-01-01" in seen, True)
# 18. the detail line must survive a real tool's noise: `checkin_check`
# prints a README SKIP note carrying "not a " on every run, and a WEAK
# hint must not beat the FAIL line that actually names the verdict.
noisy = ("== 2 check-in files in /tmp/x/checkins ==\n"
"roster: 9 handles; today 2026-10-19\n"
"since 2026-10-19: 2 graded, 72 older file(s) not graded\n"
" SKIP README.md (the directory's own README: not a check-in)\n"
"\n"
"2026-10-19-volt-02.md BAD_NAME,DUPLICATE 2 files for volt\n"
"checkin_check: 2 files, 2 with a fault (BAD_NAME 1)\n"
"FAIL: 2 file(s) violate the check-in contract\n")
check("18 a README skip note does not mask the FAIL line",
pick_detail(noisy), "FAIL: 2 file(s) violate the check-in contract")
# 18b. and on a clean run the same noise must not become the detail:
# the row reads PASS beside the summary, not beside a README note.
clean = noisy.replace("checkin_check: 2 files, 2 with a fault (BAD_NAME 1)\n"
"FAIL: 2 file(s) violate the check-in contract\n",
"checkin_check: 2 files, 0 with a fault, 1 advisory\n")
check("18b a clean check-in row carries its summary, not the skip note",
pick_detail(clean), "checkin_check: 2 files, 0 with a fault, 1 advisory")
print("selftest: %d cells, %d passed, %d failed" % (cells, passed, len(failures))) print("selftest: %d cells, %d passed, %d failed" % (cells, passed, len(failures)))
for f in failures: for f in failures:
print(" FAIL %s" % f) print(" FAIL %s" % f)