From 17d0180cef542ee23ffa176e19f0f91ae372c63b Mon Sep 17 00:00:00 2001 From: volt Date: Sat, 31 Oct 2026 11:12:45 +0000 Subject: [PATCH] volt: gate the forge lane's 39 verifiers (Q87) wizard-work/forge had 39 committed verify_*.py and no CHECK; tools/verify.py names the lane zero times. verifier_gate.py runs them, keeps one row per file bound to the git blob the rc was measured against, and grades that ledger in under a second. First full run at 7b42050: PASS 21 (18 of them PASS* -- they read an absolute host root), FAIL 18, all named with causes. Five verifiers in this lane clean up with rmtree(os.path.dirname(tmp)), i.e. they delete $TMPDIR itself; each file now runs under its own sandbox so the harness cannot be measured instead of the lane. --- WORK-QUEUE.md | 18 + docs/queue-index.md | 57 +- wizard-work/forge/CHECK | 48 ++ wizard-work/forge/CHECK-q87 | 20 + wizard-work/forge/VERIFIER_LEDGER.tsv | 55 ++ wizard-work/forge/VERIFIER_WINDOWS.tsv | 8 + wizard-work/forge/q87_verify.py | 195 ++++++ wizard-work/forge/verifier_gate.py | 802 +++++++++++++++++++++++++ 8 files changed, 1188 insertions(+), 15 deletions(-) create mode 100755 wizard-work/forge/CHECK create mode 100755 wizard-work/forge/CHECK-q87 create mode 100644 wizard-work/forge/VERIFIER_LEDGER.tsv create mode 100644 wizard-work/forge/VERIFIER_WINDOWS.tsv create mode 100644 wizard-work/forge/q87_verify.py create mode 100644 wizard-work/forge/verifier_gate.py diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index 5d7b99b..2980d0f 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -13809,3 +13809,21 @@ CHECK: bash wizard-work/volt/CHECK-q114 **(5) Not claimed.** That Q112 is closed — its done-condition also asks that the record-form ids be enumerated, and **93** bold-lead `**Q — ` records still have no row and no parser that reads them; this record fixes the artifact's provenance and its staleness, not the census's coverage unit. That Q122 is reopened — its clock leg and its interpreter leg hold and are re-measured here. That the drift advisory should become a fault — it is deliberately advisory and the selftest pins that choice. That a shallow clone grades the same — a clone that does not carry the pinned object fails the pin leg by design, and `--allow-dirty` is the documented way to generate without a pin. That the 97 rows' `status` and `done_implies` columns are now verified — `CHECK` never graded them and still does not; the columns are transcribed, not certified. That any other lane's file is affected — this record touches the census directory only. Files edited for this item: `wizard-work/ledger/workqueue-census-v2/extract_workqueue.py`, `wizard-work/ledger/workqueue-census-v2/CHECK`, `wizard-work/ledger/workqueue-census-v2/README.md`, `wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv`, `wizard-work/ledger/workqueue-census-v2/workqueue-census.md`, `wizard-work/ledger/workqueue-census-v2/extract_manifest.json`, `checkins/2026-10-31-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted. Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q135.md`. CHECK: python3 wizard-work/ledger/workqueue-census-v2/CHECK --selftest + +**Q87 — DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 → 36 → 39 across three ticks without one of them being executed; they are now run by a gate that keeps one row per file bound to the blob the rc was measured against, and the first full run says 21 pass — 18 of those only because they read this host — and 18 fail with their causes named.** + +**(0) Entry state, measured at `7b42050` before any edit.** `git ls-tree -r HEAD wizard-work/forge | grep -c 'verify_.*\.py$'` = **39**; `ls wizard-work/forge/CHECK` → absent; `grep -c 'wizard-work/forge' tools/verify.py` = **0**; the population's last measurement is `q63_unwired_baseline.tsv`'s 36 rows at rev `13c217a`. One correction to the opening record, measured rather than inherited: **3 of the 39 are invoked from a committed line** — the tracked root `CHECK:16` runs `verify_selftest_sweep_q93_refactor.py`, and `_mutant_scan.py:20` / `_mutant_scan_triage_dump_pending.py:20` run two more — so "nothing runs them" was true of 36 and false of 3, which is the distinction the item asked for and a `grep -l` cannot draw. + +**(1) What landed, six files in the lane.** `verifier_gate.py` (802 lines, sha256 `353394765f06…`) — population from `git ls-tree`, one row per verifier (` `), `--check` grades the ledger and executes nothing, `--emit` executes, `--census` classifies invocation, `--strict` promotes every non-PASS row to a fault, `--selftest` 13 cells. `CHECK` (48 lines) — the house-documented wrapper; `--check` by default, so the gate costs under a second. `VERIFIER_LEDGER.tsv` — 39 rows at `7b42050`, emitted `2026-10-31T10:59:01Z`. `VERIFIER_WINDOWS.tsv` — three per-file windows with measured reasons against a 90 s default. `q87_verify.py` + `CHECK-q87` (195 + 20 lines) — the instrument: 7 legs, three of them mutants in a throwaway clone. + +**(2) The measurement.** `--emit` in a clone at `7b42050`; final ledger **39 rows, PASS 21, FAIL 18**, sum of row wall clock 801.0 s. Of the 21 PASS rows, **18 read an absolute host root** (`/root/troupe-standup` 16, `/root/.hermes` 2) and print as `PASS*`, against the 3 host-independent ones: `verify_chunker_refactor.py` 0.06 s, `verify_probe_forge_refactor.py` 0.62 s, `verify_scan_dry_refactor.py` 2.53 s. The 18 reds, by the file's own words: **15** ran and their own checks failed (append_only_audit, check_lint_dry, citation_hash_class_portable_default, citation_resolve_check, doc_census, doc_command_report, gate_blame, mutant_scan, path_claim_apply_baseline, path_claim_dry, queue_append_audit, reconstruct_smell_fix, selftest_sweep_q93, tool_map, write_probe); **1** child timeout (`citation_hash_class_dry`, `subprocess.TimeoutExpired` after 144 s); **1** missing module (`r41_pixel_diff`, numpy); **1** host-pin to a snapshot that is gone (`selftest_sweep_dry`, `/root/.hermes/profiles/forge/cache/scratch/…`). The lane's honest state is therefore **3 of 39 green on a host-independent reading**, which is what the gate prints on every run and what `--strict` turns into rc 1. + +**(3) Why the first two emits disagreed, and what it cost.** Emit 1 read `PASS 21, FAIL 18`; emit 2, `PASS 20, FAIL 19` at the same blob. The flip was not noise. Five files clean up with `shutil.rmtree(os.path.dirname())` — `verify_append_only_audit_dry_refactor.py:304`, `verify_append_only_owners_dry_refactor.py:238`, `verify_gate_blame_dry_refactor.py:178`, `verify_served_drift_summarise_dry.py:192`, `verify_write_probe_dry_refactor.py:133` — which is the parent of the directory they made, i.e. **`$TMPDIR` itself**. Every verifier that ran after the first of them raised `FileNotFoundError` naming a directory it had just created, because `mkdtemp` fails on the missing parent. That is also what destroyed this tick's own first probe at 09:27 (the TSV vanished mid-run), and it is how the pattern was found at all. Negative control, because "the host pruner did it" is the obvious wrong answer: twelve fresh directories under that same TMPDIR survived 60 s untouched (`survive.py`). The runner now gives each file `/tb` as `TMPDIR` and removes `` itself, and selftest cell `c13` plants a verifier that deletes its own parent and requires the caller's directory to survive. Five rows changed their reading under the sandbox, each from a `FileNotFoundError` to the file's real verdict: `citation_resolve_check` (0.23 s → `verify: 6 checks, 2 passed, 4 failed`, 37 s), `path_claim_dry` (35 s), `gate_blame` (`FAIL [5] code-level`), `path_claim_apply_baseline`, `landing_check` (PASS 1.5 s → FAIL). + +**(4) The gate's own defects, each found by reading its output.** (a) The first selftest returned `("cell name", ok)` and the harness unpacked `ok, detail = fn()`, so all 10 cells passed on a truthy string while `c8` printed `False` beside them; the tuples are `(bool, detail)` now, the cells read 9/10 until `c8`'s real bug (no note for a silent `sys.exit(1)`) was fixed, and there are 13 today. (b) The note column took stderr's last line, so `verify_append_only_audit_dry_refactor.py`'s row read a `SyntaxWarning` fragment instead of its `VERDICT: MISMATCH`; the note is now the last verdict-shaped line that is not a passing cell, stderr first when nothing is verdict-shaped. (c) DRIFT can only be read from a commit — a worktree edit is invisible to a blob comparison, so the instrument's drift leg read `clean 0 → edited 0` until it committed its mutant; the gate now prints a named `UNCOMMITTED` note (note, not fault) and `c11` holds it. (d) The census counted `.md` documents as runners — six "invocations" were prose until document suffixes were excluded, leaving the true three. (e) `git()` died with `UnicodeDecodeError` on a binary blob until `errors="replace"`. + +**(5) Evidence at HEAD.** `bash wizard-work/forge/CHECK` → `0 fault(s)`, rc **0** (39 committed, 39 rows, PASS 21 / FAIL 18, PASS* 18, graded-clean 3). `--strict` → rc **1**, 18 faults, every row named. `verifier_gate.py --selftest` → **13 cells, 13 passed**. `bash wizard-work/forge/CHECK-q87` → **7 legs, 7 passed**, rc 0, ~7 s: `selftest`, `check`, `strict` (rc 1, 18), `drift` (a committed one-byte edit → `DRIFT … verify_chunker_refactor.py` → reverted → rc 0), `ungraded` (a new committed verifier → `UNGRADED`), `causeless` (a blanked note → `CAUSELESS`), `portable` (the same verdict read from `cwd=/` against a clone). `--census` → 39 verifiers, **3 `also-run`, 36 `gate`**. `tools/verify.py --repo .` → **10 legs, 10 pass**, `gate: OK`. `tools/check_lint.py --check "bash wizard-work/forge/CHECK-q87"` → `SOUND`. + +**(6) Not claimed.** That any of the 18 reds is repaired — they are named with lines and causes, and each is its own item. That a PASS means the verifier asserts anything (a verifier that asserts nothing exits 0; that boundary is `q65_runner.py`'s and is not moved). That a row is stable — `verify_citation_hash_class_portable_default.py` read 335.90 s, 350.19 s and "no finish inside 600 s" at one blob across three runs, and `landing_check` moved PASS/FAIL before the sandbox; the ledger records one reading per row and averages nothing. That the gate is wired into `tools/verify.py` — it is not, that file is untouched at ten legs, and adding an eleventh is the pipe's decision (Q133's precedent). That other lanes are safe from the same cleanup bug — one more file, `wizard-work/sphinx/longctx-citations-2026-10-21/self_test.py`, carries the identical `rmtree(os.path.dirname(...))` line and was neither run nor edited. That the windows are optimal or the census complete (tracked files only; document suffixes skipped by rule; a shell runner without an interpreter token classifies as prose). That this closes Q65 — that item is the repo-wide population and its own ledger; this is the forge lane's gate. Files edited for this item: `wizard-work/forge/verifier_gate.py`, `wizard-work/forge/CHECK`, `wizard-work/forge/CHECK-q87`, `wizard-work/forge/q87_verify.py`, `wizard-work/forge/VERIFIER_WINDOWS.tsv`, `wizard-work/forge/VERIFIER_LEDGER.tsv` (all new), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted, and no counterparty repository file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q87.md`. + +CHECK: bash wizard-work/forge/CHECK-q87 diff --git a/docs/queue-index.md b/docs/queue-index.md index ef7edea..6e8f0c7 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -4,7 +4,7 @@ **What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it. -**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 45 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 115 record blocks in all, 70 of them a later closure or correction for an id already opened. The same **45 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. +**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 45 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 116 record blocks in all, 71 of them a later closure or correction for an id already opened. The same **45 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. | # | id | kind | lines | bytes | role | title | first `Status:` | |---|---|---|---|---|---|---|---| @@ -219,7 +219,8 @@ | 209 | Q134 | record | 13752-13765 | 6338 | record: BOLD-LEAD | **Q134 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; sol lane by authorship, unclaimed and taken). `tools/inspect_tree.py` was an 8-line one-off … | -- | | 210 | Q118 | record | 13766-13781 | 7225 | cont: DONE | **Q118 — DONE (volt, 2026-10-31, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder read is retired three times before it may be called a failure … | -- | | 211 | Q114 | record | 13782-13796 | 6784 | cont: DONE | **Q114 — DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on … | -- | -| 212 | Q135 | record | 13797-13811 | 7674 | record: BOLD-LEAD | **Q135 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red … | -- | +| 212 | Q135 | record | 13797-13812 | 7675 | record: BOLD-LEAD | **Q135 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red … | -- | +| 213 | Q87 | record | 13813-13829 | 8599 | cont: DONE | **Q87 — DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 → … | -- | ## Ids with no `## ` heading @@ -236,7 +237,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q52 | 7167 | 7167 | 90 | Q51 | BULLET+BOLD | BOLD-LEAD | | Q53 | 7292 | 7246 | 170 | Q52 | BULLET+BOLD | BOLD-LEAD | | Q64 | 10308 | 10308 | 32 | Q63 | BULLET+BOLD | BOLD-LEAD | -| Q65 | 11055 | 11047 | 50 | Q63 | BULLET+BOLD | BOLD-LEAD | +| Q65 | 11055 | 11047 | 51 | Q63 | BULLET+BOLD | BOLD-LEAD | | Q99 | 12749 | 9909 | 34 | Q98 | BULLET+BOLD | BOLD-LEAD | | Q100 | 12882 | 12882 | 29 | Q99 | BULLET+BOLD | BOLD-LEAD | | Q101 | 12909 | 12909 | 18 | Q100 | BULLET+BOLD | BOLD-LEAD | @@ -269,7 +270,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q129 | 13653 | 13653 | 3 | Q128 | BULLET+BOLD | BOLD-LEAD | | Q130 | 13675 | 13675 | 1 | Q129 | BULLET+BOLD | BOLD-LEAD | | Q131 | 13689 | 13685 | 2 | Q115 | BULLET+BOLD | BOLD-LEAD | -| Q133 | 13731 | 13731 | 1 | Q90 | BULLET+BOLD | BOLD-LEAD | +| Q133 | 13731 | 13731 | 2 | Q90 | BULLET+BOLD | BOLD-LEAD | | Q134 | 13752 | 13752 | 1 | Q133 | BULLET+BOLD | BOLD-LEAD | | Q135 | 13797 | 13797 | 1 | Q114 | BULLET+BOLD | BOLD-LEAD | @@ -1766,17 +1767,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q114 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on \u2026" }, { - "bytes": 7674, - "end_line": 13811, + "bytes": 7675, + "end_line": 13812, "id": "Q135", "kind": "record", "line": 13797, - "lines": 15, + "lines": 16, "role": "record", "role_detail": "BOLD-LEAD", "status": null, "status_line": null, "title": "**Q135 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red \u2026" + }, + { + "bytes": 8599, + "end_line": 13829, + "id": "Q87", + "kind": "record", + "line": 13813, + "lines": 17, + "role": "continuation", + "role_detail": "DONE", + "status": null, + "status_line": null, + "title": "**Q87 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 \u2192 \u2026" } ], "ids_multi_section": { @@ -4658,17 +4672,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q114 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on \u2026" }, { - "bytes": 7674, - "end_line": 13811, + "bytes": 7675, + "end_line": 13812, "id": "Q135", "kind": "record", "line": 13797, - "lines": 15, + "lines": 16, "role": "record", "role_detail": "BOLD-LEAD", "status": null, "status_line": null, "title": "**Q135 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red \u2026" + }, + { + "bytes": 8599, + "end_line": 13829, + "id": "Q87", + "kind": "record", + "line": 13813, + "lines": 17, + "role": "continuation", + "role_detail": "DONE", + "status": null, + "status_line": null, + "title": "**Q87 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 \u2192 \u2026" } ], "nested": [ @@ -4758,7 +4785,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q65", "line": 11055, - "refs": 50, + "refs": 51, "section": "Q63", "shape": "BULLET+BOLD" }, @@ -5055,7 +5082,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q133", "line": 13731, - "refs": 1, + "refs": 2, "section": "Q90", "shape": "BULLET+BOLD" }, @@ -5083,9 +5110,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 2259729, - "queue_lines": 13811, - "queue_sha256": "ceeef0103bbfea1472c0d4f9b3d440a5285a4d5bba1415b54d4c872122b77fd0", + "queue_bytes": 2268329, + "queue_lines": 13829, + "queue_sha256": "ecf74b08b6aaa77e155cf0793fafc567ad2af428588ef92363c0adda9dcfb9f7", "sections": [ { "bytes": 3139, diff --git a/wizard-work/forge/CHECK b/wizard-work/forge/CHECK new file mode 100755 index 0000000..e56ce25 --- /dev/null +++ b/wizard-work/forge/CHECK @@ -0,0 +1,48 @@ +#!/bin/bash +# CHECK: wizard-work/forge -- every committed verifier has a ledger row, the row +# is bound to the blob it was measured against, and any red row carries a cause. +# +# Runner note (Q87, 2026-10-31). This lane shipped 39 `verify_*.py` and no gate; +# `tools/verify.py` names `wizard-work/forge` zero times, and the lane would not +# have noticed if all 39 broke at once. This CHECK is `verifier_gate.py --check`: +# it grades a committed ledger (VERIFIER_LEDGER.tsv) against the tree at HEAD and +# runs NOTHING, so it costs well under a second and cannot be skipped for being +# slow. Execution is explicit: +# +# bash CHECK --emit # run every verifier, rewrite the ledger (minutes) +# bash CHECK --strict # also fail on any non-PASS row (the lane's health) +# bash CHECK --census # who else, if anyone, invokes each verifier +# +# The fast path is deliberately a ledger grading and not a run: the run takes +# ~8 minutes here (one recursive sweep and two live-ish harnesses), and a gate +# that takes minutes is a gate that gets skipped -- the condition that produced +# this item. What it can still catch without running anything: a verifier edited +# after its rc was measured (DRIFT, by blob), a verifier added with no row +# (UNGRADED), a row whose file is gone (STALE), a PASS whose rc says otherwise, +# and a red row with no recorded cause (CAUSELESS). `--strict` is the switch +# from "are the books consistent?" to "are the lane's verifiers true?" -- today +# the honest answer to the second is no, and the ledger names which ones. +# +# Every skip is printed rather than implied: the gate's population is the +# committed `verify_*.py` set at HEAD, so an untracked file beside the lane is +# reported as OUTSIDE (a note, not a fault, because it is not a claim about the +# commit) and every non-PASS row is printed by name on every run, strict or not. +# +# Exit codes follow the house contract: 0 clean, 1 graded a fault, 2 could not +# grade (no git, no python3, no lane). +set -o pipefail +cd "$(dirname "$(readlink -f "$0")")" || { echo "CHECK: cannot cd to the lane" >&2; exit 2; } +command -v python3 >/dev/null || { echo "CHECK: python3 not found -- cannot grade" >&2; exit 2; } +command -v git >/dev/null || { echo "CHECK: git not found -- cannot grade" >&2; exit 2; } + +set +e +python3 verifier_gate.py --check "$@" +rc=$? +set -e +case $rc in + 0) ;; + 1) echo "CHECK: the forge verifier ledger graded a fault -- see the named rows above (Q87)." >&2; exit 1 ;; + 2) echo "CHECK: could not grade -- see the gate's message above." >&2; exit 2 ;; + *) echo "CHECK: verifier_gate.py exited $rc -- internal error (a fault)." >&2; exit 1 ;; +esac +exit 0 diff --git a/wizard-work/forge/CHECK-q87 b/wizard-work/forge/CHECK-q87 new file mode 100755 index 0000000..9eeae45 --- /dev/null +++ b/wizard-work/forge/CHECK-q87 @@ -0,0 +1,20 @@ +#!/bin/bash +# CHECK-q87: grade the forge lane gate (Q87, 2026-10-31) +# Seven legs: the gate's own selftest, the shipped CHECK, --strict, and three +# mutants in a throwaway clone (drift, ungraded, causeless) plus a portability +# read from another cwd. See q87_verify.py's header for what each leg asserts. +set -o pipefail +cd "$(dirname "$(readlink -f "$0")")" || { echo "CHECK-q87: cannot cd to the lane" >&2; exit 2; } +command -v python3 >/dev/null || { echo "CHECK-q87: python3 not found -- cannot grade" >&2; exit 2; } +command -v git >/dev/null || { echo "CHECK-q87: git not found -- cannot grade" >&2; exit 2; } +set +e +python3 q87_verify.py --repo "$(dirname "$(readlink -f "$0")")/../.." +rc=$? +set -e +case $rc in + 0) ;; + 1) echo "CHECK-q87: a leg failed -- see the FAIL line above." >&2; exit 1 ;; + 2) echo "CHECK-q87: could not grade -- see the message above." >&2; exit 2 ;; + *) echo "CHECK-q87: q87_verify.py exited $rc -- internal error (a fault)." >&2; exit 1 ;; +esac +exit 0 diff --git a/wizard-work/forge/VERIFIER_LEDGER.tsv b/wizard-work/forge/VERIFIER_LEDGER.tsv new file mode 100644 index 0000000..ac2d773 --- /dev/null +++ b/wizard-work/forge/VERIFIER_LEDGER.tsv @@ -0,0 +1,55 @@ +# Q87 verifier ledger -- one row per committed verify_*.py in wizard-work/forge, +# with the exit code it actually returned at the stated rev. +# row: +# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 / TIMEOUT / ERROR / +# MISSING. A non-PASS row must carry a note: the cause, in the file's words. +# invocation is what runs the file: `gate` when this gate is the only runner, +# `also-run` when another committed line invokes it (the census evidence is +# `--census`'s job, not a column here). +# blob is the git blob id the rc was measured against; --check recomputes it and +# a mismatch is DRIFT, so editing a verifier is a request for --emit. +# host is the absolute host root the blob reads (AST, non-docstring strings), or +# empty. A PASS carrying one prints as PASS* and is counted separately. +# seconds is wall clock under that file's window (VERIFIER_WINDOWS.tsv); a +# TIMEOUT note names the window it got. +# rev: 7b42050 +# emitted_at: 2026-10-31T10:59:01Z +gate wizard-work/forge/verify_append_only_audit_dry_refactor.py FAIL 1 1.32 VERDICT: MISMATCH -- behavior NOT preserved 09d40fbd2ca08689fdae3ac003b27d766b115033 /root/troupe-standup +gate wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.25 VERDICT: ALL GREEN -- per-party DRY collapse behavior preserved 0f650718d98bf28f07be134d49803cc902bf96b7 /root/troupe-standup +gate wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 0.92 VERDICT: ALL GREEN -- census_row_delta exit-contract + testability refactor verified (selftest 37/37, byte-identical on real census pair, DRY preserved) 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 /root/troupe-standup +gate wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.42 VERDICT: ALL GREEN -- report() behavior preserved 62082e082a0472546b415219a9180836ac88908b /root/troupe-standup +gate wizard-work/forge/verify_check_lint_dry_refactor.py FAIL 1 0.86 FAIL [5] code-level: json.dump( count 2 in original, 1 in refactored: orig=1 ref=1 06a5c66c3f319d2fa7b4d9e2f73a2eafa7a99cad /root/troupe-standup +gate wizard-work/forge/verify_checkin_check_dry_refactor.py PASS 0 0.26 VERDICT: ALL GREEN -- checkin_check census 7 scans -> 1 Counter pass, byte-identical report dict d8ba2b221b80365165eb3cace77d1764651d4fba /root/troupe-standup +gate wizard-work/forge/verify_chunker_refactor.py PASS 0 0.06 PASS: fixed=lossless on all cases AND naive=lossy (>=1 case) => fix is necessary and correct. c3c7b5631f4a572c7a630269c0b3ec06e32792fc +gate wizard-work/forge/verify_citation_hash_class_dry_refactor.py FAIL 1 144.38 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/4d67c7b3c02c44279c5af04b35dae120/venv/bin/python3', '/root/.hermes/wi 93abbc78e7f061d3e22483d8db148d434fb9caef /root/troupe-standup +gate wizard-work/forge/verify_citation_hash_class_portable_default.py FAIL 1 350.19 [ok ] selftest: identical stdout (byte-for-byte) 81c481978bb3c8bdd51a0c3fdb038ce385bfc898 /root/troupe-standup +gate wizard-work/forge/verify_citation_resolve_check_dry_refactor.py FAIL 1 21.07 verify: 6 checks, 2 passed, 4 failed cd325ddd210d5fa235024780b1737f16bb15b352 /root/troupe-standup +gate wizard-work/forge/verify_doc_census_dry_refactor.py FAIL 1 25.27 VERDICT: MISMATCH -- [2] json artifact unreadable; [2] json sha mismatch; [2b] stderr drifted; [3b] original does not carry the pre-refactor scans 52a9702aa0e015bfeba92fb7e6efba4caece56d8 /root/troupe-standup +gate wizard-work/forge/verify_doc_command_report_dry.py FAIL 1 4.96 VERDICT: MISMATCH -- behavior NOT preserved 7a2f42a5b95d35d7b16e50261a7dd72d39eccbce /root/troupe-standup +gate wizard-work/forge/verify_dump_seed_refactor.py PASS 0 0.51 === Verification: 16/16 checks passed === 589179dc57966e610deffed23b29c349b247ff78 /root/troupe-standup/inbox/seed-2026-09-26.json +gate wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 6.56 FAIL [5] code-level: refactored has 0 inlined sites; original has 5: ref=0 orig=0 8d01677402c9ac02afb044c0cbdfa5a52a7d279a /root/troupe-standup +gate wizard-work/forge/verify_landing_check_refactor.py PASS 0 1.73 verify: 13 checks, 13 passed, 0 failed 6aa282dade614e3e55874adc3e37cf7c01313f3e /root/troupe-standup +gate wizard-work/forge/verify_mutant_scan_dry_refactor.py FAIL 1 6.35 verify_mutant_scan_dry_refactor: 2 FAILED of 18 checks e9fb4006ba56e49d0ae3f68658db2966aa22cf2d +gate wizard-work/forge/verify_ops_pulse_refactor.py PASS 0 0.54 ALL_IDENTICAL ccbe3f8fa7ccdab659e68d6908b7893903759925 /root/.hermes +gate wizard-work/forge/verify_path_claim_apply_baseline_refactor.py FAIL 1 47.21 verify: 10 checks, 8 passed, 2 failed 999da7274c3cbbd9c84c3e3c106fbd65cf79b4e3 /root/troupe-standup +gate wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 35.15 verify: 11 checks, 9 passed, 2 failed ab35c13f9565e9eebbc80a210001ec37a002a267 /root/troupe-standup +gate wizard-work/forge/verify_pdf_refactor.py PASS 0 3.70 SUMMARY: ALL PASS 44ce76d39b69a7d6798af521f958370bf30ad2b1 /root/.hermes/wizard-outputs/forge/before/pdf_create.py +also-run wizard-work/forge/verify_probe_forge_refactor.py PASS 0 0.62 === Verification: 12/12 checks passed === d3434f0a500a1c1509d80c53164f6a2e352bf4b8 +gate wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 7.20 VERDICT: MISMATCH -- behavior NOT preserved 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 /root/troupe-standup +gate wizard-work/forge/verify_r41_pixel_diff.py FAIL 1 0.16 ModuleNotFoundError: No module named 'numpy' 4c1e5bec352f032e6ecb2be86334e33c2dd5d00e /root/.hermes/wizard-outputs/forge +gate wizard-work/forge/verify_reconstruct_smell_fix.py FAIL 1 0.16 FAIL (c): chunker test suite exited 1 80a9377674062386b037aeac0de6015b5c68799f /root/troupe-standup/wizard-work/forge/chunker.py +gate wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 1.35 verify: 8 checks, 8 passed, 0 failed c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 /root/troupe-standup +gate wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.53 === Verification: 10/10 checks passed === 2394abeec25726726edd91696520d788d72a5d88 /root/troupe-standup +gate wizard-work/forge/verify_roundtrip_refactor.py PASS 0 0.42 === Verification: 6/6 checks passed === 0f1d7ea9d5e542c8cb70c8e3def85ba684641b2c /root/troupe-standup +gate wizard-work/forge/verify_scan_dry_refactor.py PASS 0 2.53 === Verification: 11/11 checks passed === 6fb7aa14b96eea6731c51c25e22572cca009adf4 +gate wizard-work/forge/verify_selftest_sweep_dry_refactor.py FAIL 1 0.10 [setup] BEFORE snapshot missing: /root/.hermes/profiles/forge/cache/scratch/selftest_sweep_before.py 6f5e1621139c9ab98ec5b25ce03bda88a73cf9cf /root/troupe-standup +also-run wizard-work/forge/verify_selftest_sweep_q93_refactor.py FAIL 1 1.48 verify_selftest_sweep_q93_refactor: FAIL (2 failed, 2 checks) 971430df5da46e70371e7f75b8c9d88d095a6ab7 +gate wizard-work/forge/verify_selftest_sweep_refactor.py PASS 0 86.94 verify_selftest_sweep_refactor: 5 checks, 5 passed, 0 failed cc8782b4edfcd08eb5ed69f06308c1185ffb41e8 /root/troupe-standup +gate wizard-work/forge/verify_served_drift_summarise_dry.py PASS 0 4.52 VERDICT: ALL GREEN -- summarise() DRY collapse behavior preserved de5a49160b2c2a8f4759e0607a0f0c7b52b3bec1 /root/troupe-standup +gate wizard-work/forge/verify_smell_check_monotonic_refactor.py PASS 0 0.59 === Verification: 9/9 checks passed === 8b8ccea5be0e938fb4bba67080cc65a97762f3f4 /root/troupe-standup +gate wizard-work/forge/verify_smell_scan_refactor.py PASS 0 0.30 === Verification: 8/8 checks passed === 499c8e000359e8ef3486a89031673059d471178c /root/troupe-standup +gate wizard-work/forge/verify_tool_map_dry_refactor.py FAIL 1 24.84 VERDICT: MISMATCH -- behavior NOT preserved de1d9b7a65cfaaacf029b048228806acbf2f817c /root/troupe-standup +also-run wizard-work/forge/verify_triage_dump_pending_refactor.py PASS 0 1.24 === Verification: 13/13 checks passed === 38894d9812a8dfbda63bfa6d01407dba1b3ec042 /root/troupe-standup/inbox/seed-2026-09-26.json +gate wizard-work/forge/verify_triage_dump_refactor.py PASS 0 1.07 === Verification: 14/14 checks passed === 611e56d4d3561a2e52d810081255f575126472bc /root/troupe-standup/inbox/seed-2026-09-26.json +gate wizard-work/forge/verify_triage_probe_refactor.py PASS 0 1.09 === Verification: 14/14 checks passed === c8c9288477c8170d0b35bd40064ad58e93dcaf6c /root/troupe-standup/inbox/seed-2026-09-26.json +gate wizard-work/forge/verify_write_probe_dry_refactor.py FAIL 1 14.14 VERDICT: MISMATCH -- behavior NOT preserved ef4d8c00e4a84495e1612bec4734a366d4869b67 /root/troupe-standup diff --git a/wizard-work/forge/VERIFIER_WINDOWS.tsv b/wizard-work/forge/VERIFIER_WINDOWS.tsv new file mode 100644 index 0000000..5996990 --- /dev/null +++ b/wizard-work/forge/VERIFIER_WINDOWS.tsv @@ -0,0 +1,8 @@ +# Q87 per-file execution windows -- path seconds why. +# The default window for every file this table does not name is 90 s (--timeout). +# A row here says the file's own work costs more than the default, and a TIMEOUT +# row's note names the window it actually got, so the ledger cannot disagree +# with the run. Every figure below is a measurement at 7b42050, not an estimate. +wizard-work/forge/verify_citation_hash_class_dry_refactor.py 300 measured 133.98 s and 144.38 s at 7b42050 (rc 1 both times); it spawns child sweeps and its traceback is a TimeoutExpired from one of them +wizard-work/forge/verify_citation_hash_class_portable_default.py 900 live HTTP: 335.90 s, then 350.19 s, then no finish inside 600 s -- the same blob at three runs; 900 s is the gate's ceiling for one file +wizard-work/forge/verify_selftest_sweep_refactor.py 300 measured 86.94 s and 95.54 s (rc 0 both); its subject is the repo-wide selftest sweep diff --git a/wizard-work/forge/q87_verify.py b/wizard-work/forge/q87_verify.py new file mode 100644 index 0000000..1bcb285 --- /dev/null +++ b/wizard-work/forge/q87_verify.py @@ -0,0 +1,195 @@ +#!/usr/bin/env python3 +"""q87_verify.py -- the instrument behind CHECK-q87 (2026-10-31, volt lane). + +WHAT IT GRADES + The forge lane gate (`wizard-work/forge/CHECK` -> `verifier_gate.py`) and the + ledger it grades. Seven legs, each of which can fail on its own: + + 1 selftest the gate's own 10 cells pass (a cell that silently passes on a + truthy string is the defect the first draft of the selftest had). + 2 check `CHECK` in this tree exits 0 and prints the population and the + known-red count, so a green cannot hide a red. + 3 strict the same gate with --strict exits 1 and names the non-PASS rows, + which is the switch from "books consistent" to "lane healthy". + 4 drift in a throwaway clone, one byte appended to a PASS verifier makes + --check exit 1 with DRIFT naming that file, and restoring the + bytes makes it exit 0 again. + 5 ungraded a new committed `verify_*.py` with no ledger row makes --check + exit 1 with UNGRADED naming it (a file added tomorrow is seen). + 6 causeless blanking the note of a FAIL row makes --check exit 1 with + CAUSELESS (a red must carry its reason). + 7 portable --check run from a different cwd, against a clone of this tree, + returns the same verdict as in-tree -- so the gate is a function + of the tree, not of this host's path. + + The clone legs never touch the graded tree: every mutation happens in a + TemporaryDirectory copy that is thrown away. + +EXIT CODES + 0 every leg passed / 1 a leg failed / 2 could not grade (no git, no clone). + +Usage: + python3 q87_verify.py [--repo /root/troupe-standup] [--verbose] +""" + +import argparse +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +LANE = "wizard-work/forge" +GATE = "verifier_gate.py" +LEDGER = "VERIFIER_LEDGER.tsv" +WINDOWS = "VERIFIER_WINDOWS.tsv" +CHECK = "CHECK" +FILES = (GATE, LEDGER, WINDOWS, CHECK) + + +def run(argv, cwd=None, timeout=600): + r = subprocess.run(argv, cwd=cwd, capture_output=True, text=True, + errors="replace", timeout=timeout, + env=dict(os.environ, PYTHONDONTWRITEBYTECODE="1")) + return r.returncode, (r.stdout or "") + (r.stderr or "") + + +def gate(repo, *extra, cwd=None): + return run([sys.executable, str(Path(repo) / LANE / GATE), "--repo", str(repo)] + + list(extra), cwd=cwd) + + +def clone_of(repo, dest): + rc, out = run(["git", "clone", "-q", "--no-hardlinks", str(repo), str(dest)]) + if rc != 0: + raise RuntimeError("git clone failed: %s" % out.strip()[:200]) + for name in FILES: + src = Path(repo) / LANE / name + if src.exists(): + shutil.copyfile(src, Path(dest) / LANE / name) + return Path(dest) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--repo", default=str(Path(__file__).resolve().parents[2])) + ap.add_argument("--verbose", action="store_true") + args = ap.parse_args() + repo = Path(args.repo).resolve() + if not (repo / ".git").exists(): + print("q87: %s is not a git worktree -- could not grade" % repo) + return 2 + + legs = [] + + def leg(name, ok, detail): + legs.append((name, bool(ok), detail)) + print(" [%s] %-9s %s" % ("ok " if ok else "FAIL", name, detail), flush=True) + + # 1 -- the gate's own cells. + rc, out = gate(repo, "--selftest") + tail = [l for l in out.splitlines() if "selftest:" in l] + leg("selftest", rc == 0 and bool(tail) and "0 failed" in tail[-1] + and "cells," in tail[-1], + (tail[-1] if tail else "no selftest summary (rc %d)" % rc)) + + # 2 -- the shipped gate is green and says what it is green about. The count + # is read from the ledger, not hardcoded: a ledger that re-emits after a + # verifier changes would otherwise break this leg for the wrong reason. + rc, out = run(["bash", str(repo / LANE / CHECK)]) + pop = [l for l in out.splitlines() if "committed verifier(s)" in l] + red = [l for l in out.splitlines() if l.strip().startswith("known-red")] + led = (repo / LANE / LEDGER).read_text(encoding="utf-8") + n_rows = sum(1 for l in led.splitlines() if l.strip() and not l.startswith("#")) + n_red = sum(1 for l in led.splitlines() + if l.strip() and not l.startswith("#") and l.split("\t")[2] != "PASS") + ok2 = (rc == 0 and bool(pop) and bool(red) + and "%d committed verifier(s), %d ledger row(s)" % (n_rows, n_rows) in pop[0] + and ("%d of %d" % (n_red, n_rows)) in red[0] and n_red >= 1) + leg("check", ok2, "%s; %s" % (pop[0].strip() if pop else "no population line", + red[0].strip() if red else "no known-red line")) + + # 3 -- strict is the lane's health, and today it is not healthy. + rc, out = gate(repo, "--check", "--strict") + strict_n = [l for l in out.splitlines() if l.strip().startswith("verifier_gate:") + and "strict" in l] + n_strict = int(strict_n[-1].split()[1]) if strict_n else 0 + leg("strict", rc == 1 and n_strict >= 18 and "strict: every non-PASS row" in out, + "rc %d, %d fault(s) from the non-PASS rows" % (rc, n_strict)) + + # 4..6 -- mutants in a throwaway clone. + with tempfile.TemporaryDirectory() as tmp: + try: + clone = clone_of(repo, Path(tmp) / "c") + except RuntimeError as exc: + print("q87: %s" % exc) + return 2 + target = clone / LANE / "verify_chunker_refactor.py" + + rc0, _ = gate(clone, "--check") + base_rc, base_sha = run(["git", "-C", str(clone), "rev-parse", "HEAD"]) + original = target.read_text(encoding="utf-8") + target.write_text(original + "# one byte of drift\n", encoding="utf-8") + # the ledger binds to the COMMITTED blob, so the mutant has to be a + # commit -- an edit left in the worktree is the UNCOMMITTED note, not DRIFT + run(["git", "-C", str(clone), "add", str(target.relative_to(clone))]) + run(["git", "-C", str(clone), "-c", "user.email=q87@example.invalid", + "-c", "user.name=q87", "commit", "-qm", "drifts one verifier"]) + new_rc, new_sha = run(["git", "-C", str(clone), "rev-parse", "HEAD"]) + committed = new_rc == 0 and new_sha.strip() != base_sha.strip() + rc1, out1 = gate(clone, "--check") + drift_line = [l for l in out1.splitlines() + if l.strip().startswith("DRIFT ") + and "verify_chunker_refactor.py" in l] + if committed: + run(["git", "-C", str(clone), "reset", "-q", "--hard", "HEAD~1"]) + rc2, _ = gate(clone, "--check") + leg("drift", committed and rc0 == 0 and rc1 == 1 and rc2 == 0 + and bool(drift_line), + "clean %d -> committed edit %d%s -> reverted %d" + % (rc0, rc1, (" (%s)" % drift_line[0].strip()[:52]) if drift_line else "", + rc2)) + + new = clone / LANE / "verify_added_tomorrow.py" + new.write_text("print('nothing')\n", encoding="utf-8") + run(["git", "-C", str(clone), "add", str(new.relative_to(clone))]) + run(["git", "-C", str(clone), "-c", "user.email=q87@example.invalid", + "-c", "user.name=q87", "commit", "-qm", "adds a verifier"]) + rc3, out3 = gate(clone, "--check") + leg("ungraded", rc3 == 1 and "UNGRADED" in out3 and "verify_added_tomorrow" in out3, + "rc %d%s" % (rc3, " naming UNGRADED verify_added_tomorrow.py" + if "UNGRADED" in out3 else " without an UNGRADED fault")) + + # causeless: blank the note of the first FAIL row, in a fresh clone state + led = clone / LANE / LEDGER + lines = led.read_text(encoding="utf-8").splitlines() + hit = None + for i, line in enumerate(lines): + if line.startswith("#") or not line.strip(): + continue + parts = line.split("\t") + if parts[2] == "FAIL": + parts[5] = "" + lines[i] = "\t".join(parts) + hit = parts[1] + break + led.write_text("\n".join(lines) + "\n", encoding="utf-8") + rc4, out4 = gate(clone, "--check") + leg("causeless", rc4 == 1 and hit and ("CAUSELESS" in out4), + "rc %d%s" % (rc4, (" naming CAUSELESS %s" % hit) if "CAUSELESS" in out4 + else " with no CAUSELESS fault")) + + # 7 -- portability: same verdict from another cwd, through the clone. + rc5, out5 = gate(clone, "--check", cwd="/") + leg("portable", rc5 == 1 and "CAUSELESS" in out5, + "from cwd=/ against the clone the same fault is read (rc %d)" % rc5) + + bad = [l for l in legs if not l[1]] + print("q87: %d leg(s), %d passed, %d fault(s)" + % (len(legs), len(legs) - len(bad), len(bad))) + return 0 if not bad else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/wizard-work/forge/verifier_gate.py b/wizard-work/forge/verifier_gate.py new file mode 100644 index 0000000..51b02e2 --- /dev/null +++ b/wizard-work/forge/verifier_gate.py @@ -0,0 +1,802 @@ +#!/usr/bin/env python3 +"""verifier_gate.py -- the runner `wizard-work/forge` never had (Q87, 2026-10-31). + +WHY THIS EXISTS + At `431bca1` the pipe measured 34 `verify_*.py` in this lane and no `CHECK`; + the correction at `d56105a` moved the count to 36 and the runner count stayed + zero. At `7b42050` the lane holds 39 and `tools/verify.py` still names + `wizard-work/forge` 0 times. A verifier nothing runs is an unread claim: its + verdict is whatever its author last saw by hand, and this lane would not + notice if all 39 were broken at once. This gate executes them, keeps one row + per file bound to the blob the rc was measured against, and re-grades that + ledger on every run without re-executing it. + +WHAT IT MEASURES -- one row per committed verifier, one verdict per row + PASS rc 0: the file ran and its checks held. + FAIL rc 1: the file ran and its checks did NOT hold. A finding, not a + bookkeeping error -- the file's claim is false at this tree. + REFUSED rc 2: the file declined to grade. Not a pass. + OTHER_RC rc above 2 that the house contract does not define. + TIMEOUT no finish inside its per-file window (see WINDOWS below). + ERROR the interpreter could not run the file at all (no rc to grade). + MISSING the row's path is not in the tree at --rev. + + A non-PASS row must carry a reason in its note column. A red with no recorded + cause is a FAULT (`CAUSELESS`), because "it is red and nobody wrote down why" + is the state this lane has been in since 2026-10-27. + +THE LEDGER IS BOUND TO CONTENT + Column 7 is the git blob id each rc was measured against. `--check` recomputes + it at its own `--rev`; a mismatch is DRIFT, so editing a verifier is a request + for `--emit` rather than a free pass on an old green. Column 8 is a static + host-pin probe: the committed blob is parsed as an AST and every non-docstring + string constant is tested for an absolute host root (`/root/troupe-standup`, + `/root/.hermes`). A PASS carrying one prints as `PASS*` and is reported + separately, because a verifier that reads this host's copy graded this host + and not the tree that invoked it (Q62/Q106's class). + +WHAT IT DELIBERATELY DOES NOT DO + `--check` (the default, what `CHECK` runs) does NOT execute anything: it + grades the ledger against the tree in well under a second. Execution is + `--emit`, it takes minutes here (measured: 470 s for 39 files at a 75 s + default window, plus three files that need longer), and a gate that takes + minutes is a gate that gets skipped. `--strict` promotes any non-PASS row to + a fatal fault, which is the switch between "are the books consistent?" and + "are the lane's verifiers true?" -- today the honest answer to the second is + no, and the rows say which ones. + + It does not decide whether a verifier is *good*, only whether it exits 0. A + verifier that asserts nothing exits 0. That is the same boundary q65_runner + draws for the repo-wide population and it is not moved here. + +WINDOWS + A single `--timeout` for the whole lane is wrong: three files here need more + than 75 s and one of them is a recursive sweep. `VERIFIER_WINDOWS.tsv` names + one window per path with the reason; `--timeout` stays the fallback. A + TIMEOUT row's note names the window it actually got, so the ledger cannot + quietly disagree with the run. + +EXIT CODES -- the house contract + 0 clean / 1 graded a fault / 2 could not grade (no git, no python3, no lane). + +Usage: + python3 verifier_gate.py --check # grade the ledger (fast, no runs) + python3 verifier_gate.py --check --strict # ...and any non-PASS row is fatal + python3 verifier_gate.py --emit # execute every verifier, rewrite + python3 verifier_gate.py --census # who else runs each file + python3 verifier_gate.py --selftest # the gate's own cells +""" + +import argparse +import ast +import os +import re +import shutil +import subprocess +import sys +import tempfile +import time +from pathlib import Path + +LANE = "wizard-work/forge" +HERE = Path(__file__).resolve().parent +ROOT = HERE.parents[1] +LEDGER = HERE / "VERIFIER_LEDGER.tsv" +WINDOWS = HERE / "VERIFIER_WINDOWS.tsv" + +PASS, FAIL, REFUSED = "PASS", "FAIL", "REFUSED" +OTHER_RC, TIMEOUT, ERROR, MISSING = "OTHER_RC", "TIMEOUT", "ERROR", "MISSING" +VERDICTS = (PASS, FAIL, REFUSED, OTHER_RC, TIMEOUT, ERROR, MISSING) + +HOST_ROOTS = ("/root/troupe-standup", "/root/.hermes") +VERDICT_OF_RC = {0: PASS, 1: FAIL, 2: REFUSED} + + +def git(repo, *args): + """(rc, stdout, stderr) -- never raises, so a missing git is rc 2 upstream.""" + try: + r = subprocess.run(["git", "-C", str(repo)] + list(args), + capture_output=True, text=True, errors="replace", + timeout=120) + return r.returncode, r.stdout, r.stderr + except (OSError, subprocess.SubprocessError) as exc: + return 127, "", str(exc) + + +def population(repo, rev): + """Committed verify_*.py under the lane at `rev` -- from the tree, not a list. + + The population is read from git rather than from a directory glob because + the question is "what did this commit ship", and an untracked scratch file + beside the lane is not a claim about the commit. + """ + rc, out, err = git(repo, "ls-tree", "-r", "--name-only", rev, "--", LANE) + if rc != 0: + raise SystemExit("git ls-tree %s failed: %s" % (rev, (err or "").strip())) + paths = [p.strip() for p in out.splitlines() if p.strip()] + return sorted(p for p in paths + if p.startswith(LANE + "/") + and os.path.basename(p).startswith("verify_") + and p.endswith(".py")) + + +def blob_of(repo, rev, path): + rc, out, _ = git(repo, "rev-parse", "%s:%s" % (rev, path)) + return out.strip() if rc == 0 else "" + + +def host_pin(repo, rev, path): + """The first absolute host root this blob reads, or ''. + + Parsed rather than grepped: a docstring that *names* the host root is a + description, and the Q85 lesson is that a text scan turns description into + finding. Only string constants outside docstrings count. + """ + rc, blob, _ = git(repo, "show", "%s:%s" % (rev, path)) + if rc != 0: + return "" + try: + tree = ast.parse(blob) + except SyntaxError: + return "unparseable" + docs = set() + for node in ast.walk(tree): + if isinstance(node, (ast.Module, ast.FunctionDef, ast.AsyncFunctionDef, + ast.ClassDef)): + body = getattr(node, "body", []) + if body and isinstance(body[0], ast.Expr) \ + and isinstance(body[0].value, ast.Constant) \ + and isinstance(body[0].value.value, str): + docs.add(id(body[0].value)) + for node in ast.walk(tree): + if isinstance(node, ast.Constant) and isinstance(node.value, str): + if id(node) in docs: + continue + for root in HOST_ROOTS: + if root in node.value: + hits = [l for l in node.value.splitlines() if root in l] + return (hits[0] if hits else root).strip()[:120] + return "" + + +def read_windows(path): + """{path: (seconds, why)} -- one window per file, the reason mandatory.""" + win = {} + p = Path(path) + if not p.exists(): + return win + for line in p.read_text(encoding="utf-8").splitlines(): + if not line.strip() or line.lstrip().startswith("#"): + continue + parts = line.split("\t") + if len(parts) < 3: + raise SystemExit("windows row needs pathsecondswhy: %r" % line) + try: + win[parts[0].strip()] = (float(parts[1]), parts[2].strip()) + except ValueError: + raise SystemExit("window seconds is not a number: %r" % line) + return win + + +def run_one(repo, rel, timeout, windows): + """Execute one verifier in its own directory, under its own TMPDIR. + + THE SANDBOX IS NOT A LUXURY. Five files in this lane clean up with + `shutil.rmtree(os.path.dirname())`, which is the PARENT of + the directory they made -- i.e. `$TMPDIR` itself: + verify_append_only_audit_dry_refactor.py:304, verify_append_only_owners_ + dry_refactor.py:238, verify_gate_blame_dry_refactor.py:178, verify_served_ + drift_summarise_dry.py:192, verify_write_probe_dry_refactor.py:133. + Running them with the caller's TMPDIR deleted the caller's scratch tree (this + item's own first measurement, at 09:27, died that way: the probe's TSV was + gone mid-run) and every later file failed with FileNotFoundError naming a + directory it had just created, because `mkdtemp` raises on the missing + PARENT. Each file now gets `/tb` as TMPDIR, so a verifier that + deletes its parent takes out a throwaway directory the gate removes itself. + """ + window, why = windows.get(rel, (timeout, "")) + full = Path(repo) / rel + if not full.exists(): + return {"path": rel, "verdict": MISSING, "rc": None, "seconds": 0.0, + "note": "not in worktree"} + base = tempfile.mkdtemp(prefix="forge_gate_") + tb = os.path.join(base, "tb") + os.makedirs(tb, exist_ok=True) + env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1", PYTHONUNBUFFERED="1", + TMPDIR=tb, TEMP=tb, TMP=tb) + env.pop("PYTHONOPTIMIZE", None) + t0 = time.time() + try: + r = subprocess.run([sys.executable, "-B", full.name], cwd=str(full.parent), + env=env, capture_output=True, timeout=window, + text=True, errors="replace") + rc, so, se = r.returncode, r.stdout, r.stderr + except subprocess.TimeoutExpired: + return {"path": rel, "verdict": TIMEOUT, "rc": None, + "seconds": round(time.time() - t0, 2), + "note": "no finish in %ss%s" + % (window, (" (%s)" % why) if why else "")} + except OSError as exc: + return {"path": rel, "verdict": ERROR, "rc": None, "seconds": 0.0, + "note": str(exc)[:160]} + finally: + shutil.rmtree(base, ignore_errors=True) + secs = round(time.time() - t0, 2) + verdict = VERDICT_OF_RC.get(rc, OTHER_RC if rc > 2 else ERROR) + note = verdict_line(so, se, rc) + return {"path": rel, "verdict": verdict, "rc": rc, "seconds": secs, + "note": note} + + +def verdict_line(so, se, rc): + """The file's own sentence, not whichever stream happened to end last. + + The first draft took stderr's last line, and `verify_append_only_audit_dry_ + refactor.py` wrote its cause to stdout while stderr ended on a SyntaxWarning + fragment -- the row read `(b"dash \\u2014 here\\nplain\\n", ...)` and lost the + `VERDICT: MISMATCH` it had printed. A verdict-looking line is preferred, from + stdout first; the last line of either stream is the fallback. + """ + keys = ("VERDICT", "FAIL", "checks", "PASS", "failed", "Error") + for text in (so, se): + lines = [l.strip() for l in (text or "").splitlines() if l.strip()] + for line in reversed(lines): + if line.startswith("[ok") or line.startswith("ok:"): + continue # a passing cell is not this row's cause + if any(k in line for k in keys): + return line[:160] + # Nothing verdict-shaped: prefer stderr's last line, because that is where a + # traceback puts the reason, and stdout's last line is often a passing cell + # the file printed before the thing that killed it. + for text in (se, so): + lines = [l.strip() for l in (text or "").splitlines() if l.strip()] + if lines: + return lines[-1][:160] + return "" if rc == 0 else "exited rc %d with no stdout/stderr" % rc + + +HEADER = """\ +# Q87 verifier ledger -- one row per committed verify_*.py in wizard-work/forge, +# with the exit code it actually returned at the stated rev. +# row: \t\t\t\t\t\t\t +# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 / TIMEOUT / ERROR / +# MISSING. A non-PASS row must carry a note: the cause, in the file's words. +# invocation is what runs the file: `gate` when this gate is the only runner, +# `also-run` when another committed line invokes it (the census evidence is +# `--census`'s job, not a column here). +# blob is the git blob id the rc was measured against; --check recomputes it and +# a mismatch is DRIFT, so editing a verifier is a request for --emit. +# host is the absolute host root the blob reads (AST, non-docstring strings), or +# empty. A PASS carrying one prints as PASS* and is counted separately. +# seconds is wall clock under that file's window (VERIFIER_WINDOWS.tsv); a +# TIMEOUT note names the window it got. +""" + + +def read_ledger(path): + rows, meta = [], {} + p = Path(path) + if not p.exists(): + return rows, meta, "no ledger at %s (run --emit)" % p + for line in p.read_text(encoding="utf-8").splitlines(): + if not line.strip(): + continue + if line.startswith("#"): + m = re.match(r"#\s*(rev|emitted_at):\s*(\S+)", line) + if m: + meta[m.group(1)] = m.group(2) + continue + parts = line.split("\t") + if len(parts) < 6: + return rows, meta, "ledger row has %d column(s): %r" % (len(parts), line) + while len(parts) < 8: + parts.append("") + rows.append({"invocation": parts[0], "path": parts[1], "verdict": parts[2], + "rc": parts[3], "seconds": parts[4], "note": parts[5], + "blob": parts[6], "host": parts[7]}) + return rows, meta, "" + + +def write_ledger(path, rows, repo, rev): + rc, out, _ = git(repo, "rev-parse", "--short", rev) + sha = out.strip() or rev + body = [HEADER.rstrip("\n"), + "# rev: %s" % sha, + "# emitted_at: %sZ" % time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime())] + for r in sorted(rows, key=lambda x: x["path"]): + body.append("\t".join([r["invocation"], r["path"], r["verdict"], + "" if r["rc"] is None else str(r["rc"]), + "%.2f" % float(r["seconds"]), r["note"], + r["blob"], r["host"]])) + Path(path).write_text("\n".join(body) + "\n", encoding="utf-8") + + +def rc_consistent(row): + """Wording a verdict has to match. Returns a reason string or ''.""" + v, rc = row["verdict"], row["rc"] + if v not in VERDICTS: + return "verdict %r is not one of %s" % (v, "/".join(VERDICTS)) + if v in (TIMEOUT, ERROR, MISSING): + return "" if rc == "" else "%s carries rc %s" % (v, rc) + if rc == "" or not re.fullmatch(r"-?\d+", rc): + return "%s carries rc %r" % (v, rc) + n = int(rc) + if v == PASS and n != 0: + return "PASS with rc %d" % n + if v == FAIL and n != 1: + return "FAIL with rc %d" % n + if v == REFUSED and n != 2: + return "REFUSED with rc %d" % n + if v == OTHER_RC and n <= 2: + return "OTHER_RC with rc %d" % n + return "" + + +# ---------------------------------------------------------------- invocation +DOC_SUFFIXES = (".md", ".txt", ".log", ".tsv", ".json", ".pyc") +RUN_TOKENS = ("python3", "python ", "sys.executable", "subprocess", "os.system", + "pytest", "bash ") + + +def invocation_census(repo, rev, paths, exclude_rel): + """{path: (class, evidence)} -- who runs this file, from committed content. + + A mention is not an invocation. A reference counts as a RUN only when it sits + in an executable file (.py/.sh/CHECK*, not a document) on a line that also + names an interpreter or a spawner; every other reference is prose. That is + the distinction reply 21 asked for, and it is a rule with a test: --selftest + holds one prose mention and one `python3 x.py` line and requires the + classifier to separate them. + """ + rc, out, _ = git(repo, "ls-tree", "-r", "--name-only", rev) + if rc != 0: + raise SystemExit("git ls-tree failed at %s" % rev) + files = [f.strip() for f in out.splitlines() if f.strip()] + texts = {} + for f in files: + base = os.path.basename(f) + if base.endswith(DOC_SUFFIXES): + continue + rc, blob, _ = git(repo, "show", "%s:%s" % (rev, f)) + if rc == 0: + texts[f] = blob + res = {} + for path in paths: + name = os.path.basename(path) + runs = [] + named = 0 + for f, blob in texts.items(): + if f == path or f == exclude_rel: + continue + if name not in blob: + continue + for i, line in enumerate(blob.splitlines(), 1): + if name not in line: + continue + if line.lstrip().startswith(("#", "//", "*", "--")): + named += 1 + continue + if any(t in line for t in RUN_TOKENS): + runs.append("%s:%d" % (f, i)) + else: + named += 1 + if runs: + res[path] = ("also-run", ", ".join(runs[:2])) + elif named: + res[path] = ("gate", "%d prose ref(s), no invocation" % named) + else: + res[path] = ("gate", "named by nothing else at all") + return res + + +def check(repo, rev, ledger_path, strict, quiet=False): + try: + pop = population(repo, rev) + except SystemExit as exc: + print("verifier_gate: %s" % exc) + return 2 + if not pop: + print("verifier_gate: population is empty at %s -- could not grade" % rev) + return 2 + rows, meta, err = read_ledger(ledger_path) + if err: + print("verifier_gate: %s" % err) + return 2 + faults, notes = [], [] + seen = {} + for r in rows: + seen[r["path"]] = seen.get(r["path"], 0) + 1 + why = rc_consistent(r) + if why: + faults.append("INCONSISTENT %s: %s" % (r["path"], why)) + if r["verdict"] != PASS and not r["note"].strip(): + faults.append("CAUSELESS %s: %s row with no reason in the note" + % (r["path"], r["verdict"])) + if not r["blob"]: + faults.append("UNBOUND %s: no blob column -- the row does not say " + "which content its rc describes (re-emit)" % r["path"]) + for p, n in sorted(seen.items()): + if n > 1: + faults.append("DUPLICATE %s: %d rows" % (p, n)) + for r in rows: + if r["path"] not in pop: + continue + now = blob_of(repo, rev, r["path"]) + if r["blob"] and now and now != r["blob"]: + faults.append("DRIFT %s: rc measured against blob %s, tree at %s has %s" + % (r["path"], r["blob"][:10], rev, now[:10])) + # A dirty worktree is not a commit, so it cannot be DRIFT -- but a verifier + # whose working bytes are not the blob its rc describes is exactly the case + # where the ledger's verdict is about content you are not running. Named, + # not fatal: the fix is a commit (then DRIFT/UNGRADED decide) or --emit. + for r in rows: + if r["path"] not in pop or not r["blob"]: + continue + rc_w, out_w, _ = git(repo, "hash-object", "--", r["path"]) + if rc_w == 0 and out_w.strip() and out_w.strip() != r["blob"]: + notes.append("UNCOMMITTED %s: worktree bytes are not the blob its rc " + "was measured against -- commit it, then re-emit or " + "expect DRIFT" % r["path"]) + ungraded = sorted(set(pop) - set(seen)) + for p in ungraded: + faults.append("UNGRADED %s: committed at %s, no ledger row" % (p, rev)) + stale = sorted(p for p in set(seen) - set(pop) + if not (Path(repo) / p).exists()) + for p in stale: + faults.append("STALE %s: ledger row, file not in the tree" % p) + outside = sorted(p for p in set(seen) - set(pop) if (Path(repo) / p).exists()) + for p in outside: + notes.append("OUTSIDE %s: ledger row, not a committed verifier at %s " + "(untracked beside the lane or renamed)" % (p, rev)) + counts = {} + for r in rows: + counts[r["verdict"]] = counts.get(r["verdict"], 0) + 1 + pinned = [r for r in rows if r["verdict"] == PASS and r["host"]] + reds = [r for r in rows if r["verdict"] != PASS] + clean = len(rows) - len(reds) - len(pinned) + print("verifier_gate -- lane %s at %s, %d committed verifier(s), %d ledger row(s)" + % (LANE, rev, len(pop), len(rows))) + if meta.get("rev"): + print(" ledger rev : %s emitted %s" + % (meta["rev"], meta.get("emitted_at", "?"))) + print(" rows : %s" % ", ".join("%s %d" % (v, counts[v]) + for v in VERDICTS if counts.get(v))) + print(" graded-clean : %d of %d (PASS on a host-independent blob)" + % (clean, len(pop))) + if pinned: + print(" host-pinned : %d PASS row(s) read as PASS* -- they read an " + "absolute host root" % len(pinned)) + for r in pinned: + print(" PASS* %s %s" % (r["path"], r["host"])) + if reds: + print(" known-red : %d of %d row(s) -- every one named, every one " + "with its cause" % (len(reds), len(rows))) + for r in sorted(reds, key=lambda x: x["path"]): + print(" %-8s rc %-3s %s %s" % (r["verdict"], r["rc"] or "-", + r["path"], r["note"][:100])) + for n in notes: + print(" note: " + n) + if strict: + for r in reds: + faults.append("%s %s rc %s %s" % (r["verdict"], r["path"], r["rc"] or "-", + r["note"][:80])) + for f in faults: + print(" " + f) + print("verifier_gate: %d fault(s)%s" % (len(faults), + " (strict: every non-PASS row counts)" + if strict else "")) + return 1 if faults else 0 + + +def emit(repo, rev, ledger_path, timeout, windows, only=None, merge=False): + pop = population(repo, rev) + if only: + pop = [p for p in pop if only in p] + if not pop: + print("verifier_gate: nothing to emit") + return 2 + keep = [] + if merge and Path(ledger_path).exists(): + have, _, err = read_ledger(ledger_path) + if err: + print("verifier_gate: %s" % err) + return 2 + keep = [r for r in have if r["path"] not in set(pop)] + rc, before, _ = git(repo, "status", "--porcelain") + t0 = time.time() + rows = list(keep) + for p in pop: + r = run_one(repo, p, timeout, windows) + r["blob"] = blob_of(repo, rev, p) + r["host"] = host_pin(repo, rev, p) + r["invocation"] = "gate" + rows.append(r) + print(" %-8s %6.2fs %s" % (r["verdict"], r["seconds"], p), flush=True) + census = {} + try: + census = invocation_census(repo, rev, [r["path"] for r in rows], + os.path.relpath(ledger_path, repo)) + except SystemExit as exc: + print("verifier_gate: census skipped (%s)" % exc) + for r in rows: + cls, ev = census.get(r["path"], ("gate", "")) + r["invocation"] = "also-run" if cls == "also-run" else "gate" + write_ledger(ledger_path, rows, repo, rev) + rc2, after, _ = git(repo, "status", "--porcelain") + led_rel = os.path.relpath(str(ledger_path), str(repo)) + dirt = sorted(d for d in set(after.splitlines()) - set(before.splitlines()) + if led_rel not in d) + counts = {} + for r in rows: + counts[r["verdict"]] = counts.get(r["verdict"], 0) + 1 + print("verifier_gate: %d row(s) in %.1fs -> %s" + % (len(rows), time.time() - t0, ledger_path)) + print(" " + ", ".join("%s %d" % (v, counts[v]) for v in VERDICTS + if counts.get(v))) + if dirt: + print(" WARNING: the run changed the worktree (these verifiers write " + "into it):") + for d in dirt: + print(" " + d) + return 0 + + +def census_report(repo, rev, ledger_path): + pop = population(repo, rev) + res = invocation_census(repo, rev, pop, os.path.relpath(str(ledger_path), str(repo))) + run = sorted(p for p, (c, _) in res.items() if c == "also-run") + print("verifier_census -- %d committed verifier(s) in %s at %s" + % (len(pop), LANE, rev)) + print(" invoked by another committed line : %d" % len(run)) + for p in run: + print(" %s %s" % (p, res[p][1])) + print(" invoked by this gate only : %d" % (len(pop) - len(run))) + for p in pop: + if p in run: + continue + print(" %s %s" % (p, res[p][1])) + return 0 + + +# ------------------------------------------------------------------- selftest +def selftest(): + """Cells for the rules that can silently pass. Every cell builds a throwaway + git repo, so the gate's logic is graded on a tree it did not ship with.""" + cells = [] + + def cell(name, fn): + try: + ok, detail = fn() + except Exception as exc: # noqa: BLE001 - a cell that raises is a cell that failed + ok, detail = False, "%s: %s" % (type(exc).__name__, exc) + cells.append((name, ok, detail)) + + def mk(tmp, files, commit=True): + repo = Path(tmp) + (repo / LANE).mkdir(parents=True, exist_ok=True) + for rel, body in files.items(): + p = repo / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(body, encoding="utf-8") + git(repo, "init", "-q") + git(repo, "config", "user.email", "t@example.invalid") + git(repo, "config", "user.name", "t") + if commit: + git(repo, "add", "-A") + git(repo, "commit", "-qm", "cell") + return repo + + V = LANE + "/verify_alpha.py" + + def c1(): + bad = {"path": V, "verdict": PASS, "rc": "1", "note": "", "blob": "x", + "host": "", "seconds": "0", "invocation": "gate"} + good = dict(bad, verdict=FAIL) + a, b = rc_consistent(bad), rc_consistent(good) + return (bool(a) and not b, "PASS with rc 1") + + def c2(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + n1 = len(population(repo, "HEAD")) + (repo / LANE / "verify_beta.py").write_text("print(0)\n") + git(repo, "add", "-A") + git(repo, "commit", "-qm", "two") + n2 = len(population(repo, "HEAD")) + return (n1 == 1 and n2 == 2, "population follows the rev") + + def c3(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + led = repo / LANE / "L.tsv" + write_ledger(led, [{"path": V, "verdict": PASS, "rc": 0, "seconds": 0.0, + "note": "", "blob": blob_of(repo, "HEAD", V), + "host": "", "invocation": "gate"}], repo, "HEAD") + r0 = check(repo, "HEAD", led, False) + (repo / V).write_text("print(1)\n") + git(repo, "add", "-A") + git(repo, "commit", "-qm", "edit") + r1 = check(repo, "HEAD", led, False) + return (r0 == 0 and r1 == 1, "a drifted blob is a fault") + + def c4(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + led = repo / LANE / "L.tsv" + write_ledger(led, [], repo, "HEAD") + return (check(repo, "HEAD", led, False) == 1, "a committed verifier with no row faults") + + def c5(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + led = repo / LANE / "L.tsv" + write_ledger(led, [{"path": LANE + "/verify_gone.py", "verdict": PASS, + "rc": 0, "seconds": 0.0, "note": "", "blob": "abc", + "host": "", "invocation": "gate"}], repo, "HEAD") + return (check(repo, "HEAD", led, False) == 1, "a row for a deleted file faults") + + def c6(): + with tempfile.TemporaryDirectory() as tmp: + body = ('"""Reads /root/troupe-standup in a docstring."""\n' + 'P = "/root/troupe-standup/inbox"\n') + repo = mk(tmp, {V: body}) + hit = host_pin(repo, "HEAD", V) + repo2 = mk(tmp + "/b", {V: '"""Reads /root/troupe-standup."""\n'}) + miss = host_pin(repo2, "HEAD", V) + return (hit != "" and miss == "", "host pin is AST, not prose") + + def c7(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n", + "CHECK": "python3 %s\n" % V, + "notes/x.md": "see %s\n" % V}) + res = invocation_census(repo, "HEAD", [V], "") + return (res[V][0] == "also-run", "prose is not an invocation") + + def c8(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "import sys\nsys.exit(1)\n", + LANE + "/verify_ok.py": "print(0)\n"}) + led = repo / LANE / "L.tsv" + rc = emit(repo, "HEAD", led, 30.0, {}, only=None) + rows, _, err = read_ledger(led) + by = {r["path"]: r for r in rows} + ok = (rc == 0 and not err and by[V]["verdict"] == FAIL + and by[V]["note"] != "" and by[LANE + "/verify_ok.py"]["verdict"] == PASS) + return (ok, "emit records rc and a cause") + + def c9(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + led = repo / LANE / "L.tsv" + write_ledger(led, [{"path": V, "verdict": FAIL, "rc": 1, "seconds": 0.0, + "note": "", "blob": blob_of(repo, "HEAD", V), + "host": "", "invocation": "gate"}], repo, "HEAD") + return (check(repo, "HEAD", led, False) == 1, "a red with no cause is a fault") + + def c10(): + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + led = repo / LANE / "L.tsv" + blob = blob_of(repo, "HEAD", V) + write_ledger(led, [{"path": V, "verdict": FAIL, "rc": 1, "seconds": 0.0, + "note": "known red", "blob": blob, "host": "", + "invocation": "gate"}], repo, "HEAD") + loose = check(repo, "HEAD", led, False) + strict = check(repo, "HEAD", led, True) + return (loose == 0 and strict == 1, "strict promotes a red row") + + def c11(): + import contextlib + import io + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: "print(0)\n"}) + led = repo / LANE / "L.tsv" + write_ledger(led, [{"path": V, "verdict": PASS, "rc": 0, "seconds": 0.0, + "note": "", "blob": blob_of(repo, "HEAD", V), + "host": "", "invocation": "gate"}], repo, "HEAD") + (repo / V).write_text("print(0) # uncommitted edit\n") + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + rc = check(repo, "HEAD", led, False) + out = buf.getvalue() + return (rc == 0 and "UNCOMMITTED" in out, + "a dirty worktree is named, not fatal") + + def c12(): + body = ("import sys\n" + "sys.stderr.write('warning: bad escape \\\\u\\n')\n" + "print('VERDICT: MISMATCH -- behavior NOT preserved')\n" + "sys.exit(1)\n") + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: body}) + led = repo / LANE / "L.tsv" + emit(repo, "HEAD", led, 30.0, {}, only=None) + rows, _, err = read_ledger(led) + by = {r["path"]: r for r in rows} + note = by[V]["note"] if V in by else "" + return (not err and "VERDICT: MISMATCH" in note, + "the note is the file's verdict line: %r" % note[:60]) + + def c13(): + body = ("import os, shutil, tempfile\n" + "sandbox = tempfile.mkdtemp(prefix='x_', dir=os.environ['TMPDIR'])\n" + "shutil.rmtree(os.path.dirname(sandbox), ignore_errors=True)\n" + "print('cleaned up my parent')\n") + with tempfile.TemporaryDirectory() as tmp: + repo = mk(tmp, {V: body}) + led = repo / LANE / "L.tsv" + survive = tempfile.mkdtemp(prefix="survive_") + emit(repo, "HEAD", led, 30.0, {}, only=None) + rows, _, err = read_ledger(led) + by = {r["path"]: r for r in rows} + alive = os.path.isdir(survive) + shutil.rmtree(survive, ignore_errors=True) + return (not err and by[V]["verdict"] == PASS and alive, + "a verifier that deletes its TMPDIR leaves the caller's alone") + + # the cell fixture needs a caller-level temp dir to watch + for name, fn in (("c1 verdict/rc pairing", c1), + ("c2 population from the rev", c2), + ("c3 blob drift", c3), + ("c4 ungraded file", c4), + ("c5 stale row", c5), + ("c6 host-pin AST", c6), + ("c7 named-vs-run", c7), + ("c8 emit records", c8), + ("c9 causeless red", c9), + ("c10 strict", c10), + ("c11 uncommitted worktree", c11), + ("c12 note is the verdict line", c12), + ("c13 sandboxed TMPDIR", c13)): + cell(name, fn) + + bad = [c for c in cells if not c[1]] + for name, ok, detail in cells: + print(" [%s] %s -- %s" % ("ok " if ok else "FAIL", name, detail)) + print("verifier_gate selftest: %d cells, %d passed, %d failed" + % (len(cells), len(cells) - len(bad), len(bad))) + return 0 if not bad else 1 + + +def main(): + ap = argparse.ArgumentParser(add_help=True) + ap.add_argument("--repo", default=str(ROOT)) + ap.add_argument("--rev", default="HEAD") + ap.add_argument("--ledger", default=str(LEDGER)) + ap.add_argument("--windows", default=str(WINDOWS)) + ap.add_argument("--timeout", type=float, default=90.0) + ap.add_argument("--check", action="store_true") + ap.add_argument("--emit", action="store_true") + ap.add_argument("--census", action="store_true") + ap.add_argument("--strict", action="store_true") + ap.add_argument("--only") + ap.add_argument("--merge", action="store_true", + help="with --emit: keep the rows this run did not re-measure") + ap.add_argument("--selftest", action="store_true") + args = ap.parse_args() + + if args.selftest: + return selftest() + repo = Path(args.repo).resolve() + if not (repo / ".git").exists(): + print("verifier_gate: %s is not a git worktree -- could not grade" % repo) + return 2 + ledger = Path(args.ledger) + if not ledger.is_absolute(): + ledger = repo / ledger + if args.emit: + try: + win = read_windows(args.windows) + except SystemExit as exc: + print("verifier_gate: %s" % exc) + return 2 + return emit(repo, args.rev, ledger, args.timeout, win, only=args.only, + merge=args.merge) + if args.census: + return census_report(repo, args.rev, ledger) + return check(repo, args.rev, ledger, args.strict) + + +if __name__ == "__main__": + sys.exit(main())