gate_blame (volt): a gate red is attributed to the landing that turned it -- 12-commit window walks each rev's own worktree through verify.py's own leg runner; measures the live check-in fault to a013842 (2026-10-19 23:44:47Z, PASS->FAULT), proves 12 further commits unchanged; selftest 20 cells; both sweep probe axes get their row, two readerless prose counts retired
This commit is contained in:
1 parent
dc7fe6f3fc
commit
2c2184cbba
2 files changed
+483
-5
No files matched your search
@@ -0,0 +1,469 @@
|
||||
#!/usr/bin/env python3
|
||||
"""gate_blame.py -- which landing turned a gate leg red?
|
||||
|
||||
WHY THIS EXISTS
|
||||
`verify.py` grades five legs, and it grades ONE tree: the one checked out now.
|
||||
So every question of the form "is this red mine?" costs a manual checkout and a
|
||||
re-run per commit. On 2026-10-20 three replies in a row answered exactly that
|
||||
question by hand, for the same inherited check-in fault: each re-proved the
|
||||
same two filenames against a parent commit, in prose, and each said so in its
|
||||
own words. The gate already holds the leg definitions and the grading logic;
|
||||
what it does not hold is time.
|
||||
|
||||
This tool adds the time axis and nothing else. It walks the last N first-parent
|
||||
commits, materialises each one as its own detached worktree, runs `verify.py`'s
|
||||
OWN leg runner against that tree, and names the oldest commit whose verdict
|
||||
differs from its parent's.
|
||||
|
||||
RULES (stated, not implied)
|
||||
B1 FIDELITY a revision's row is `verify.run_leg(...)` with `--repo` set to that
|
||||
revision's worktree, so it is graded by the revision's own tool
|
||||
file and its own `checkin_baseline.tsv` -- the tolerance it
|
||||
committed, not this host's idea of one. The leg DEFINITION (which
|
||||
argv, which ledger, which queue path) is this checkout's
|
||||
`verify.LEGS`: a leg added after a revision cannot have graded it,
|
||||
and that limit is printed in the header rather than hidden.
|
||||
B2 FLIP a flip is a change of VERDICT (PASS / FAULT / UNGRADABLE) between a
|
||||
commit and its first parent. A detail change at a constant verdict
|
||||
is printed as CHANGED and is never called a flip: two different
|
||||
faults are both FAULT, and naming the newer commit as the landing
|
||||
would be wrong.
|
||||
B3 FIRST the landing reported is the OLDEST commit in the window whose
|
||||
verdict differs from its parent -- the landing that turned it, not
|
||||
the newest commit that still carries it.
|
||||
B4 TRANSPARENT the window is printed even when nothing flips, with every row's
|
||||
verdict and the leg's own detail line. A tool that prints only a
|
||||
landing cannot be checked on the run where there is none.
|
||||
|
||||
EXIT CODES (the repo's contract, `tools/selftest_sweep.py`)
|
||||
0 ran; the leg's verdict is identical at every commit in the window
|
||||
1 ran, and graded at least one verdict change (each landing is named)
|
||||
2 could not grade: not a git work tree, an unknown leg, a range with no
|
||||
commits, git refusing a worktree, or every commit in the window UNGRADABLE.
|
||||
Never 1, never a traceback.
|
||||
|
||||
READ-ONLY on the repository's content. Each revision is read from a detached
|
||||
worktree under a temporary directory, removed with `git worktree remove --force`
|
||||
and swept by `git worktree prune` on the way out; the checked-out tree, the index
|
||||
and the branch are untouched. The only write paths are `--json PATH` and stdout.
|
||||
|
||||
USAGE:
|
||||
python3 gate_blame.py --repo . --leg checkin --range 10
|
||||
python3 gate_blame.py --repo /root/troupe-standup --leg index-current --range 5
|
||||
python3 gate_blame.py --repo . --leg checkin --range 24 --json /tmp/blame.json
|
||||
python3 gate_blame.py --repo . --selftest # fixture corpus, 20 cells
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import verify # noqa: E402 -- the gate's own runner and leg table, reused on purpose
|
||||
|
||||
LEG_KEYS = tuple(leg["key"] for leg in verify.LEGS)
|
||||
PASS, FAULT, UNGRADABLE = verify.PASS, verify.FAULT, verify.UNGRADABLE
|
||||
|
||||
# The selftest size this docstring claims. Graded against the count that ran, the
|
||||
# same way `verify.py` grades its own: a docstring count with no reader is how the
|
||||
# 2026-10-19 check-in defect (docstring 50, help 42, run 50) survived.
|
||||
_DOC_CLAIM = re.search(r"fixture corpus, (\d+) cells", __doc__ or "")
|
||||
DOC_CELLS = int(_DOC_CLAIM.group(1)) if _DOC_CLAIM else 0
|
||||
|
||||
|
||||
class CannotGrade(Exception):
|
||||
"""rc 2: a reason this host cannot produce a timeline."""
|
||||
|
||||
|
||||
def _git(repo, *args, timeout=120):
|
||||
return subprocess.run(["git", "-C", repo] + list(args),
|
||||
capture_output=True, text=True, timeout=timeout)
|
||||
|
||||
|
||||
def _first_line(text):
|
||||
for line in (text or "").splitlines():
|
||||
if line.strip():
|
||||
return line.strip()
|
||||
return ""
|
||||
|
||||
|
||||
def work_tree_root(repo):
|
||||
"""The work-tree root, or CannotGrade. Mirrors verify.py's own resolution."""
|
||||
if not os.path.isdir(repo):
|
||||
raise CannotGrade("no such directory: %s" % repo)
|
||||
proc = _git(repo, "rev-parse", "--show-toplevel")
|
||||
if proc.returncode != 0:
|
||||
raise CannotGrade("not a git work tree: %s (%s)"
|
||||
% (repo, _first_line(proc.stderr) or "rev-parse rc %d" % proc.returncode))
|
||||
return proc.stdout.strip()
|
||||
|
||||
|
||||
def window(root, ref, count):
|
||||
"""The last `count` first-parent commits ending at `ref`, oldest first."""
|
||||
if count < 1:
|
||||
raise CannotGrade("--range must be 1 or more, got %d" % count)
|
||||
proc = _git(root, "rev-list", "--first-parent", "-n", str(count), ref)
|
||||
if proc.returncode != 0:
|
||||
raise CannotGrade("cannot read history at %s: %s"
|
||||
% (ref, _first_line(proc.stderr) or "rc %d" % proc.returncode))
|
||||
revs = [line.strip() for line in proc.stdout.split() if line.strip()]
|
||||
if not revs:
|
||||
raise CannotGrade("no commits reachable from %s" % ref)
|
||||
return list(reversed(revs))
|
||||
|
||||
|
||||
def stamps(root, revs):
|
||||
"""{rev: 'YYYY-MM-DD HH:MM +0000'} in one git call, not one per commit."""
|
||||
proc = _git(root, "log", "--no-walk", "--format=%H%x09%ad", "--date=iso",
|
||||
*revs)
|
||||
out = {}
|
||||
if proc.returncode != 0:
|
||||
return out
|
||||
for line in proc.stdout.splitlines():
|
||||
parts = line.split("\t", 1)
|
||||
if len(parts) == 2:
|
||||
out[parts[0].strip()] = parts[1].strip()
|
||||
return out
|
||||
|
||||
|
||||
def grade_rev(root, rev, leg, queue, index, timeout, scratch):
|
||||
"""One revision -> one row, graded by its own tree via verify.run_leg."""
|
||||
wt = os.path.join(scratch, rev[:12])
|
||||
added = _git(root, "worktree", "add", "--detach", "--quiet", wt, rev)
|
||||
if added.returncode != 0:
|
||||
return {"rev": rev, "verdict": UNGRADABLE, "rc": None,
|
||||
"detail": "worktree add refused: %s"
|
||||
% (_first_line(added.stderr) or "rc %d" % added.returncode)}
|
||||
try:
|
||||
row = verify.run_leg(leg, wt, queue, index, timeout)
|
||||
finally:
|
||||
_git(root, "worktree", "remove", "--force", wt)
|
||||
return {"rev": rev, "verdict": row["verdict"], "rc": row["rc"],
|
||||
"detail": row["detail"]}
|
||||
|
||||
|
||||
def changes(rows):
|
||||
"""Rows that differ from their parent, split into flips (verdict) and CHANGED."""
|
||||
out = []
|
||||
for prev, cur in zip(rows, rows[1:]):
|
||||
if prev["verdict"] != cur["verdict"]:
|
||||
out.append(("FLIP", prev, cur))
|
||||
elif prev["detail"] != cur["detail"]:
|
||||
out.append(("CHANGED", prev, cur))
|
||||
return out
|
||||
|
||||
|
||||
def render(rows, events, leg, ref, queue):
|
||||
print("gate_blame -- repo leg: %s (%s)" % (leg["key"], leg["tool"]))
|
||||
print(" window: %d commit(s) ending at %s, oldest first; leg definition from"
|
||||
" this checkout's verify.LEGS" % (len(rows), ref[:12]))
|
||||
print(" what: %s" % leg["what"])
|
||||
print("")
|
||||
print(" %-12s %-19s %-10s %s" % ("commit", "date", "verdict", "detail"))
|
||||
for row in rows:
|
||||
detail = row["detail"] or ""
|
||||
if len(detail) > 96:
|
||||
detail = detail[:93] + "..."
|
||||
print(" %-12s %-19s %-10s %s"
|
||||
% (row["rev"][:12], (row.get("date") or "-")[:19], row["verdict"], detail))
|
||||
print("")
|
||||
if not events:
|
||||
print(" no change: %s at every commit in the window" % rows[0]["verdict"])
|
||||
for kind, prev, cur in events:
|
||||
print(" %s at %s (%s): %s -> %s"
|
||||
% (kind, cur["rev"][:12], (cur.get("date") or "-")[:19],
|
||||
prev["verdict"], cur["verdict"]))
|
||||
print(" detail: %s" % (cur["detail"] or "")[:200])
|
||||
flips = [e for e in events if e[0] == "FLIP"]
|
||||
verdicts = sorted({r["verdict"] for r in rows})
|
||||
print("")
|
||||
print("gate_blame: %d commit(s), %d flip(s), %d changed-at-same-verdict,"
|
||||
" tip %s, verdicts seen %s"
|
||||
% (len(rows), len(flips), len(events) - len(flips),
|
||||
rows[-1]["verdict"], ",".join(verdicts)))
|
||||
return flips
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--repo", default=".")
|
||||
ap.add_argument("--leg", default="checkin", choices=LEG_KEYS)
|
||||
ap.add_argument("--range", type=int, default=10,
|
||||
help="first-parent commits to walk back from --ref (default 10)")
|
||||
ap.add_argument("--ref", default="HEAD")
|
||||
ap.add_argument("--queue", default=verify.QUEUE_DEFAULT)
|
||||
ap.add_argument("--index", default=verify.INDEX_DEFAULT)
|
||||
ap.add_argument("--timeout", type=int, default=300, help="per-revision leg timeout")
|
||||
ap.add_argument("--json", dest="json_path", default=None)
|
||||
ap.add_argument("--quiet", action="store_true")
|
||||
ap.add_argument("--selftest", action="store_true")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.selftest:
|
||||
return selftest()
|
||||
|
||||
leg = next((l for l in verify.LEGS if l["key"] == args.leg), None)
|
||||
if leg is None: # argparse choices makes this unreachable; kept explicit.
|
||||
print("gate_blame: unknown leg %r (known: %s)" % (args.leg, ", ".join(LEG_KEYS)))
|
||||
return 2
|
||||
try:
|
||||
root = work_tree_root(args.repo)
|
||||
revs = window(root, args.ref, args.range)
|
||||
except CannotGrade as exc:
|
||||
print("gate_blame: cannot grade: %s" % exc)
|
||||
return 2
|
||||
|
||||
scratch = tempfile.mkdtemp(prefix="gate-blame-")
|
||||
rows = []
|
||||
try:
|
||||
dates = stamps(root, revs)
|
||||
for rev in revs:
|
||||
row = grade_rev(root, rev, leg, args.queue, args.index, args.timeout, scratch)
|
||||
row["date"] = dates.get(rev, "")
|
||||
rows.append(row)
|
||||
finally:
|
||||
_git(root, "worktree", "prune")
|
||||
shutil.rmtree(scratch, ignore_errors=True)
|
||||
|
||||
graded = [r for r in rows if r["verdict"] in (PASS, FAULT)]
|
||||
if not graded:
|
||||
if not args.quiet:
|
||||
for r in rows:
|
||||
print(" %-12s %-10s %s" % (r["rev"][:12], r["verdict"], r["detail"]))
|
||||
print("gate_blame: cannot grade: all %d commit(s) in the window are UNGRADABLE"
|
||||
% len(rows))
|
||||
return 2
|
||||
|
||||
events = changes(rows)
|
||||
flips = render(rows, events, leg, revs[-1], args.queue) if not args.quiet else \
|
||||
[e for e in events if e[0] == "FLIP"]
|
||||
|
||||
if args.json_path:
|
||||
payload = {"leg": leg["key"], "tool": leg["tool"], "ref": args.ref,
|
||||
"rows": rows,
|
||||
"flips": [{"at": cur["rev"], "from": prev["verdict"],
|
||||
"to": cur["verdict"], "detail": cur["detail"]}
|
||||
for kind, prev, cur in events if kind == "FLIP"],
|
||||
"changed_same_verdict": [
|
||||
{"at": cur["rev"], "verdict": cur["verdict"],
|
||||
"was": prev["detail"], "now": cur["detail"]}
|
||||
for kind, prev, cur in events if kind == "CHANGED"]}
|
||||
with open(args.json_path, "w", encoding="utf-8") as fh:
|
||||
json.dump(payload, fh, indent=2, sort_keys=True)
|
||||
fh.write("\n")
|
||||
return 1 if flips else 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- selftest ---
|
||||
|
||||
def _sh(root, *args, date=None):
|
||||
env = dict(os.environ)
|
||||
env.update({"GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "f@example.invalid",
|
||||
"GIT_COMMITTER_NAME": "fixture", "GIT_COMMITTER_EMAIL": "f@example.invalid"})
|
||||
if date:
|
||||
env["GIT_AUTHOR_DATE"] = date
|
||||
env["GIT_COMMITTER_DATE"] = date
|
||||
return subprocess.run(["git", "-C", root] + list(args), capture_output=True,
|
||||
text=True, env=env)
|
||||
|
||||
|
||||
TABLE = ("# Wizards\n\n| Wizard | Lane |\n|---|---|\n| volt | workhorse |\n")
|
||||
CLEAN = "# 2026-10-20 -- volt check-in\n\n## Did\n- nothing\n"
|
||||
DUP = "# 2026-10-20 -- volt check-in\n\n## Did\n- second file, same day\n"
|
||||
|
||||
|
||||
def _fixture(base, files, msg, date):
|
||||
"""Write `files` (relpath -> text) into `base`, commit everything, return sha."""
|
||||
for rel, text in files.items():
|
||||
path = os.path.join(base, rel)
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write(text)
|
||||
_sh(base, "add", "-A")
|
||||
proc = _sh(base, "commit", "-q", "-m", msg, date=date)
|
||||
if proc.returncode != 0:
|
||||
raise RuntimeError("fixture commit failed: %s" % (proc.stdout + proc.stderr))
|
||||
return _sh(base, "rev-parse", "HEAD").stdout.strip()
|
||||
|
||||
|
||||
def _repo(base):
|
||||
_sh(base, "init", "-q")
|
||||
_sh(base, "symbolic-ref", "HEAD", "refs/heads/main")
|
||||
|
||||
|
||||
def _grade(base, *args):
|
||||
"""Run main() over a fixture, returning (rc, text)."""
|
||||
buf = io.StringIO()
|
||||
with contextlib.redirect_stdout(buf):
|
||||
rc = main(["--repo", base, *args])
|
||||
return rc, buf.getvalue()
|
||||
|
||||
|
||||
def selftest():
|
||||
here = os.path.dirname(os.path.abspath(__file__))
|
||||
cells, failures = [], []
|
||||
|
||||
def cell(name, ok, detail=""):
|
||||
cells.append(name)
|
||||
if not ok:
|
||||
failures.append("%s%s" % (name, (": " + detail) if detail else ""))
|
||||
|
||||
base = tempfile.mkdtemp(prefix="gate-blame-selftest-")
|
||||
try:
|
||||
# -- fixture A: PASS then FAULT (a suffix file creates a duplicate) -----
|
||||
a = os.path.join(base, "a")
|
||||
os.makedirs(os.path.join(a, "tools"))
|
||||
os.makedirs(os.path.join(a, "checkins"))
|
||||
shutil.copy(os.path.join(here, "checkin_check.py"),
|
||||
os.path.join(a, "tools", "checkin_check.py"))
|
||||
with open(os.path.join(a, "tools", "checkin_baseline.tsv"), "w") as fh:
|
||||
fh.write("# empty ledger: every fault is new\n")
|
||||
with open(os.path.join(a, "WIZARDS-TABLE.md"), "w") as fh:
|
||||
fh.write(TABLE)
|
||||
open(os.path.join(a, "checkins", "2026-10-20-volt.md"), "w").write(CLEAN)
|
||||
_repo(a)
|
||||
c1 = _fixture(a, {}, "c1 clean", "2026-10-20T10:00:00+00:00")
|
||||
c2 = _fixture(a, {"checkins/2026-10-20-volt-02.md": DUP},
|
||||
"c2 duplicate", "2026-10-20T11:00:00+00:00")
|
||||
|
||||
rc, out = _grade(a, "--leg", "checkin", "--range", "2")
|
||||
cell("flip: PASS->FAULT exits 1", rc == 1, "rc=%d" % rc)
|
||||
cell("flip: names the landing commit", c2[:12] in out and "FLIP at %s" % c2[:12] in out,
|
||||
out.strip().splitlines()[-1] if out.strip() else "no output")
|
||||
cell("flip: does not name the parent", "FLIP at %s" % c1[:12] not in out)
|
||||
table = out.split("FLIP at")[0] if "FLIP at" in out else out
|
||||
cell("flip: both rows printed (B4)", c1[:12] in table and c2[:12] in table
|
||||
and "PASS" in table and "FAULT" in table, table)
|
||||
|
||||
# -- fixture B: red at both commits -> no flip, tip FAULT --------------
|
||||
b = os.path.join(base, "b")
|
||||
os.makedirs(os.path.join(b, "tools"))
|
||||
os.makedirs(os.path.join(b, "checkins"))
|
||||
shutil.copy(os.path.join(here, "checkin_check.py"),
|
||||
os.path.join(b, "tools", "checkin_check.py"))
|
||||
with open(os.path.join(b, "tools", "checkin_baseline.tsv"), "w") as fh:
|
||||
fh.write("# empty ledger\n")
|
||||
with open(os.path.join(b, "WIZARDS-TABLE.md"), "w") as fh:
|
||||
fh.write(TABLE)
|
||||
open(os.path.join(b, "checkins", "2026-10-20-volt.md"), "w").write(CLEAN)
|
||||
open(os.path.join(b, "checkins", "2026-10-20-volt-02.md"), "w").write(DUP)
|
||||
_repo(b)
|
||||
_fixture(b, {}, "b1 red", "2026-10-20T10:00:00+00:00")
|
||||
_fixture(b, {"checkins/README.md": "# not a check-in\n", "notes.txt": "x\n"},
|
||||
"b2 still red, other detail", "2026-10-20T11:00:00+00:00")
|
||||
rc, out = _grade(b, "--leg", "checkin", "--range", "2")
|
||||
cell("stable red: rc 0 (no verdict change)", rc == 0, "rc=%d" % rc)
|
||||
cell("stable red: tip reported FAULT", "tip FAULT" in out, out.strip().splitlines()[-1:])
|
||||
cell("stable red: parent not named as a landing", "FLIP at" not in out)
|
||||
|
||||
# -- fixture C: constant verdict, changed detail -> CHANGED not FLIP ---
|
||||
c = os.path.join(base, "c")
|
||||
os.makedirs(os.path.join(c, "tools"))
|
||||
os.makedirs(os.path.join(c, "checkins"))
|
||||
shutil.copy(os.path.join(here, "checkin_check.py"),
|
||||
os.path.join(c, "tools", "checkin_check.py"))
|
||||
with open(os.path.join(c, "tools", "checkin_baseline.tsv"), "w") as fh:
|
||||
fh.write("# empty ledger\n")
|
||||
with open(os.path.join(c, "WIZARDS-TABLE.md"), "w") as fh:
|
||||
fh.write(TABLE)
|
||||
open(os.path.join(c, "checkins", "2026-10-20-volt-02.md"), "w").write(DUP)
|
||||
_repo(c)
|
||||
_fixture(c, {}, "c1 one bad name", "2026-10-20T10:00:00+00:00")
|
||||
_fixture(c, {"checkins/2026-10-20-volt-03.md": DUP},
|
||||
"c2 two bad names", "2026-10-20T11:00:00+00:00")
|
||||
rc, out = _grade(c, "--leg", "checkin", "--range", "2")
|
||||
cell("detail-only change: rc 0", rc == 0, "rc=%d" % rc)
|
||||
cell("detail-only change: CHANGED printed, FLIP not", "CHANGED at" in out
|
||||
and "FLIP at" not in out, out)
|
||||
|
||||
# -- fixture D: no tool at an old rev -> UNGRADABLE, then a real flip ---
|
||||
d = os.path.join(base, "d")
|
||||
os.makedirs(os.path.join(d, "checkins"))
|
||||
open(os.path.join(d, "checkins", "2026-10-20-volt.md"), "w").write(CLEAN)
|
||||
_repo(d)
|
||||
_fixture(d, {}, "d1 no tools dir", "2026-10-20T10:00:00+00:00")
|
||||
os.makedirs(os.path.join(d, "tools"))
|
||||
shutil.copy(os.path.join(here, "checkin_check.py"),
|
||||
os.path.join(d, "tools", "checkin_check.py"))
|
||||
with open(os.path.join(d, "tools", "checkin_baseline.tsv"), "w") as fh:
|
||||
fh.write("# empty ledger\n")
|
||||
_fixture(d, {"WIZARDS-TABLE.md": TABLE}, "d2 adds the tool",
|
||||
"2026-10-20T11:00:00+00:00")
|
||||
rc, out = _grade(d, "--leg", "checkin", "--range", "2")
|
||||
cell("absent tool: graded as UNGRADABLE, not a traceback",
|
||||
"UNGRADABLE" in out and "Traceback" not in out, out)
|
||||
cell("absent tool: UNGRADABLE->PASS is a flip", rc == 1, "rc=%d" % rc)
|
||||
|
||||
# -- fixture E: empty checkins dir -> UNGRADABLE both sides, rc 2 -------
|
||||
e = os.path.join(base, "e")
|
||||
os.makedirs(os.path.join(e, "tools"))
|
||||
os.makedirs(os.path.join(e, "checkins"))
|
||||
shutil.copy(os.path.join(here, "checkin_check.py"),
|
||||
os.path.join(e, "tools", "checkin_check.py"))
|
||||
_repo(e)
|
||||
_fixture(e, {"WIZARDS-TABLE.md": TABLE}, "e1 no check-ins",
|
||||
"2026-10-20T10:00:00+00:00")
|
||||
_fixture(e, {"notes.txt": "y\n"}, "e2 still none", "2026-10-20T11:00:00+00:00")
|
||||
rc, out = _grade(e, "--leg", "checkin", "--range", "2")
|
||||
cell("all-ungradable window: rc 2", rc == 2, "rc=%d" % rc)
|
||||
cell("all-ungradable window: says so", "cannot grade" in out, out)
|
||||
|
||||
# -- refusals ----------------------------------------------------------
|
||||
rc, out = _grade(base, "--leg", "checkin", "--range", "0")
|
||||
cell("refusal: --range 0 is rc 2", rc == 2 and "cannot grade" in out,
|
||||
"rc=%d %s" % (rc, out.strip()))
|
||||
notrepo = os.path.join(base, "notrepo")
|
||||
os.makedirs(notrepo, exist_ok=True)
|
||||
open(os.path.join(notrepo, "x.txt"), "w").write("x\n")
|
||||
rc, out = _grade(notrepo, "--leg", "checkin")
|
||||
cell("refusal: not a git work tree is rc 2",
|
||||
rc == 2 and "not a git work tree" in out, "rc=%d %s" % (rc, out.strip()))
|
||||
proc = subprocess.run([sys.executable, os.path.join(here, "gate_blame.py"),
|
||||
"--repo", a, "--leg", "nope"], capture_output=True, text=True)
|
||||
cell("refusal: unknown leg rejected by argparse (rc 2)",
|
||||
proc.returncode == 2 and "invalid choice" in proc.stderr, proc.stderr[:120])
|
||||
|
||||
# -- JSON shape --------------------------------------------------------
|
||||
jpath = os.path.join(base, "blame.json")
|
||||
rc, out = _grade(a, "--leg", "checkin", "--range", "2", "--json", jpath)
|
||||
try:
|
||||
with open(jpath, encoding="utf-8") as fh:
|
||||
payload = json.load(fh)
|
||||
ok = (payload["leg"] == "checkin" and len(payload["rows"]) == 2
|
||||
and payload["flips"] and payload["flips"][0]["at"] == c2)
|
||||
except Exception as exc: # noqa: BLE001 -- a bad JSON file is the cell
|
||||
ok, payload = False, str(exc)
|
||||
cell("--json: rows, leg and the flip's commit", ok, str(payload)[:160])
|
||||
|
||||
# -- subdirectory --repo resolves to the work-tree root ----------------
|
||||
rc, out = _grade(os.path.join(a, "checkins"), "--leg", "checkin", "--range", "2")
|
||||
cell("subdirectory --repo: resolves to the root", rc == 1
|
||||
and "FLIP at %s" % c2[:12] in out, out.strip().splitlines()[-1:])
|
||||
|
||||
# -- no worktree residue ----------------------------------------------
|
||||
listed = _sh(a, "worktree", "list", "--porcelain").stdout
|
||||
cell("no worktree residue after the run",
|
||||
listed.count("worktree ") == 1, listed)
|
||||
finally:
|
||||
shutil.rmtree(base, ignore_errors=True)
|
||||
|
||||
cell("docstring cell count matches the run", DOC_CELLS == len(cells) + 1,
|
||||
"docstring says %d, run says %d" % (DOC_CELLS, len(cells) + 1))
|
||||
|
||||
print("selftest: %d cells, %d passed, %d failed"
|
||||
% (len(cells), len(cells) - len(failures), len(failures)))
|
||||
for f in failures:
|
||||
print(" FAIL %s" % f)
|
||||
return 0 if not failures else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+14
-5
@@ -58,9 +58,9 @@ THE MALFORMED-INPUT CLASS, ADDED 2026-10-18
|
||||
should be, a directory that is not a git repository, a queue file with no
|
||||
records. Those are the inputs where a tool is most tempted to answer "0" and
|
||||
return 0, which reads exactly like a clean pass (`doc_census_check`'s
|
||||
`--tools` bug, `76043b7`, was this class). The axis now runs 17 such probes
|
||||
and grades each against ITS OWN stated answer, because "must exit 2" is not
|
||||
a universal truth: eleven tools must refuse at rc 2, while `check_lint` grades
|
||||
`--tools` bug, `76043b7`, was this class). The axis runs one such probe per
|
||||
tool and grades each against ITS OWN stated answer, because "must exit 2" is
|
||||
not a universal truth: most tools must refuse at rc 2, while `check_lint` grades
|
||||
a junk log MISSING (rc 0 in report mode, rc 1 under --fail-on-fault),
|
||||
`checkin_check` grades a heading-less check-in as an advisory (rc 0, stated
|
||||
in its docstring) and `queue_claim_check` reports 0 opening records (rc 0,
|
||||
@@ -163,6 +163,7 @@ READ_PROBES = (
|
||||
("citation_resolve_check.py", ["--repo", "{nope}"]),
|
||||
("doc_command_check.py", ["--repo", "{nope}"]),
|
||||
("doc_census_check.py", ["--repo", "{nope}", "--no-run"]),
|
||||
("gate_blame.py", ["--repo", "{nope}"]),
|
||||
("landing_check.py", ["--repo", "{nope}"]),
|
||||
("queue_claim_check.py", ["--queue", "{nope}/q.md"]),
|
||||
("queue_index.py", ["--queue", "{nope}/q.md", "--check", "{nope}/i.md"]),
|
||||
@@ -183,10 +184,14 @@ NOT_UTF8_PROBES = (
|
||||
# THE THIRD CLASS (2026-10-18): input that decodes as UTF-8 but cannot be made
|
||||
# sense of. Each entry carries the answer the TOOL's own contract states --
|
||||
# (tool, label, argv, expected rc, a substring the output must contain) -- so
|
||||
# the axis measures the documented behaviour instead of imposing one. Fourteen
|
||||
# tools must refuse at rc 2; the three that grade a malformed input are listed
|
||||
# the axis measures the documented behaviour instead of imposing one. Most
|
||||
# tools must refuse at rc 2; the ones that grade a malformed input are listed
|
||||
# with the rc and the word their grade prints, which is what makes a silent
|
||||
# rc 0 (the doc_census_check --tools bug class) fail here.
|
||||
# The counts are deliberately NOT written here: a number in a comment has no
|
||||
# reader, which is the defect class this file's own docstring names, and the
|
||||
# two counts that used to sit on these lines were both stale ("eleven",
|
||||
# "Fourteen") before gate_blame.py's row was added on 2026-10-20.
|
||||
# Fixtures: {notrepo} = a directory with a file and no .git; {badjson} = a
|
||||
# UTF-8 file that is not JSON; {emptyjson} = "{}" (valid JSON, wrong shape);
|
||||
# {junklog} = a log with no CHECK: line; {junkqueue} = 3 prose lines, no
|
||||
@@ -217,6 +222,10 @@ MALFORMED_PROBES = (
|
||||
2, "no tools found"),
|
||||
("doc_command_check.py", "not-a-repo", ["--repo", "{notrepo}"],
|
||||
2, "cannot read repo"),
|
||||
# gate_blame.py's input is a git work tree: a directory that is not one is the
|
||||
# malformed shape, and it must name that rather than pass a window of nothing.
|
||||
("gate_blame.py", "not-a-repo", ["--repo", "{notrepo}"],
|
||||
2, "not a git work tree"),
|
||||
("landing_check.py", "not-a-repo", ["--repo", "{notrepo}"],
|
||||
2, "not a git work tree"),
|
||||
("queue_claim_check.py", "junk-queue", ["--queue", "{junkqueue}"],
|
||||
|
||||
Reference in new issue
Block a user