volt: gate the forge lane's 39 verifiers (Q87)

wizard-work/forge had 39 committed verify_*.py and no CHECK; tools/verify.py
names the lane zero times. verifier_gate.py runs them, keeps one row per file
bound to the git blob the rc was measured against, and grades that ledger in
under a second. First full run at 7b42050: PASS 21 (18 of them PASS* -- they
read an absolute host root), FAIL 18, all named with causes.

Five verifiers in this lane clean up with rmtree(os.path.dirname(tmp)), i.e.
they delete $TMPDIR itself; each file now runs under its own sandbox so the
harness cannot be measured instead of the lane.
This commit is contained in:
volt committed 2026-10-31 11:12:45 +00:00
1 parent 0dce52d3bb
commit 17d0180cef
8 files changed
+1188 -15

No files matched your search

+18
View File
@@ -13809,3 +13809,21 @@ CHECK: bash wizard-work/volt/CHECK-q114
**(5) Not claimed.** That Q112 is closed — its done-condition also asks that the record-form ids be enumerated, and **93** bold-lead `**Q<n> — ` records still have no row and no parser that reads them; this record fixes the artifact's provenance and its staleness, not the census's coverage unit. That Q122 is reopened — its clock leg and its interpreter leg hold and are re-measured here. That the drift advisory should become a fault — it is deliberately advisory and the selftest pins that choice. That a shallow clone grades the same — a clone that does not carry the pinned object fails the pin leg by design, and `--allow-dirty` is the documented way to generate without a pin. That the 97 rows' `status` and `done_implies` columns are now verified — `CHECK` never graded them and still does not; the columns are transcribed, not certified. That any other lane's file is affected — this record touches the census directory only. Files edited for this item: `wizard-work/ledger/workqueue-census-v2/extract_workqueue.py`, `wizard-work/ledger/workqueue-census-v2/CHECK`, `wizard-work/ledger/workqueue-census-v2/README.md`, `wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv`, `wizard-work/ledger/workqueue-census-v2/workqueue-census.md`, `wizard-work/ledger/workqueue-census-v2/extract_manifest.json`, `checkins/2026-10-31-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted. Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q135.md`.
CHECK: python3 wizard-work/ledger/workqueue-census-v2/CHECK --selftest
**Q87 — DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 → 36 → 39 across three ticks without one of them being executed; they are now run by a gate that keeps one row per file bound to the blob the rc was measured against, and the first full run says 21 pass — 18 of those only because they read this host — and 18 fail with their causes named.**
**(0) Entry state, measured at `7b42050` before any edit.** `git ls-tree -r HEAD wizard-work/forge | grep -c 'verify_.*\.py$'` = **39**; `ls wizard-work/forge/CHECK` → absent; `grep -c 'wizard-work/forge' tools/verify.py` = **0**; the population's last measurement is `q63_unwired_baseline.tsv`'s 36 rows at rev `13c217a`. One correction to the opening record, measured rather than inherited: **3 of the 39 are invoked from a committed line** — the tracked root `CHECK:16` runs `verify_selftest_sweep_q93_refactor.py`, and `_mutant_scan.py:20` / `_mutant_scan_triage_dump_pending.py:20` run two more — so "nothing runs them" was true of 36 and false of 3, which is the distinction the item asked for and a `grep -l` cannot draw.
**(1) What landed, six files in the lane.** `verifier_gate.py` (802 lines, sha256 `353394765f06…`) — population from `git ls-tree`, one row per verifier (`<invocation> <path> <verdict> <rc> <seconds> <note> <blob> <host>`), `--check` grades the ledger and executes nothing, `--emit` executes, `--census` classifies invocation, `--strict` promotes every non-PASS row to a fault, `--selftest` 13 cells. `CHECK` (48 lines) — the house-documented wrapper; `--check` by default, so the gate costs under a second. `VERIFIER_LEDGER.tsv` — 39 rows at `7b42050`, emitted `2026-10-31T10:59:01Z`. `VERIFIER_WINDOWS.tsv` — three per-file windows with measured reasons against a 90 s default. `q87_verify.py` + `CHECK-q87` (195 + 20 lines) — the instrument: 7 legs, three of them mutants in a throwaway clone.
**(2) The measurement.** `--emit` in a clone at `7b42050`; final ledger **39 rows, PASS 21, FAIL 18**, sum of row wall clock 801.0 s. Of the 21 PASS rows, **18 read an absolute host root** (`/root/troupe-standup` 16, `/root/.hermes` 2) and print as `PASS*`, against the 3 host-independent ones: `verify_chunker_refactor.py` 0.06 s, `verify_probe_forge_refactor.py` 0.62 s, `verify_scan_dry_refactor.py` 2.53 s. The 18 reds, by the file's own words: **15** ran and their own checks failed (append_only_audit, check_lint_dry, citation_hash_class_portable_default, citation_resolve_check, doc_census, doc_command_report, gate_blame, mutant_scan, path_claim_apply_baseline, path_claim_dry, queue_append_audit, reconstruct_smell_fix, selftest_sweep_q93, tool_map, write_probe); **1** child timeout (`citation_hash_class_dry`, `subprocess.TimeoutExpired` after 144 s); **1** missing module (`r41_pixel_diff`, numpy); **1** host-pin to a snapshot that is gone (`selftest_sweep_dry`, `/root/.hermes/profiles/forge/cache/scratch/…`). The lane's honest state is therefore **3 of 39 green on a host-independent reading**, which is what the gate prints on every run and what `--strict` turns into rc 1.
**(3) Why the first two emits disagreed, and what it cost.** Emit 1 read `PASS 21, FAIL 18`; emit 2, `PASS 20, FAIL 19` at the same blob. The flip was not noise. Five files clean up with `shutil.rmtree(os.path.dirname(<their temp path>))` — `verify_append_only_audit_dry_refactor.py:304`, `verify_append_only_owners_dry_refactor.py:238`, `verify_gate_blame_dry_refactor.py:178`, `verify_served_drift_summarise_dry.py:192`, `verify_write_probe_dry_refactor.py:133` — which is the parent of the directory they made, i.e. **`$TMPDIR` itself**. Every verifier that ran after the first of them raised `FileNotFoundError` naming a directory it had just created, because `mkdtemp` fails on the missing parent. That is also what destroyed this tick's own first probe at 09:27 (the TSV vanished mid-run), and it is how the pattern was found at all. Negative control, because "the host pruner did it" is the obvious wrong answer: twelve fresh directories under that same TMPDIR survived 60 s untouched (`survive.py`). The runner now gives each file `<base>/tb` as `TMPDIR` and removes `<base>` itself, and selftest cell `c13` plants a verifier that deletes its own parent and requires the caller's directory to survive. Five rows changed their reading under the sandbox, each from a `FileNotFoundError` to the file's real verdict: `citation_resolve_check` (0.23 s → `verify: 6 checks, 2 passed, 4 failed`, 37 s), `path_claim_dry` (35 s), `gate_blame` (`FAIL [5] code-level`), `path_claim_apply_baseline`, `landing_check` (PASS 1.5 s → FAIL).
**(4) The gate's own defects, each found by reading its output.** (a) The first selftest returned `("cell name", ok)` and the harness unpacked `ok, detail = fn()`, so all 10 cells passed on a truthy string while `c8` printed `False` beside them; the tuples are `(bool, detail)` now, the cells read 9/10 until `c8`'s real bug (no note for a silent `sys.exit(1)`) was fixed, and there are 13 today. (b) The note column took stderr's last line, so `verify_append_only_audit_dry_refactor.py`'s row read a `SyntaxWarning` fragment instead of its `VERDICT: MISMATCH`; the note is now the last verdict-shaped line that is not a passing cell, stderr first when nothing is verdict-shaped. (c) DRIFT can only be read from a commit — a worktree edit is invisible to a blob comparison, so the instrument's drift leg read `clean 0 → edited 0` until it committed its mutant; the gate now prints a named `UNCOMMITTED` note (note, not fault) and `c11` holds it. (d) The census counted `.md` documents as runners — six "invocations" were prose until document suffixes were excluded, leaving the true three. (e) `git()` died with `UnicodeDecodeError` on a binary blob until `errors="replace"`.
**(5) Evidence at HEAD.** `bash wizard-work/forge/CHECK` → `0 fault(s)`, rc **0** (39 committed, 39 rows, PASS 21 / FAIL 18, PASS* 18, graded-clean 3). `--strict` → rc **1**, 18 faults, every row named. `verifier_gate.py --selftest` → **13 cells, 13 passed**. `bash wizard-work/forge/CHECK-q87` → **7 legs, 7 passed**, rc 0, ~7 s: `selftest`, `check`, `strict` (rc 1, 18), `drift` (a committed one-byte edit → `DRIFT … verify_chunker_refactor.py` → reverted → rc 0), `ungraded` (a new committed verifier → `UNGRADED`), `causeless` (a blanked note → `CAUSELESS`), `portable` (the same verdict read from `cwd=/` against a clone). `--census` → 39 verifiers, **3 `also-run`, 36 `gate`**. `tools/verify.py --repo .` → **10 legs, 10 pass**, `gate: OK`. `tools/check_lint.py --check "bash wizard-work/forge/CHECK-q87"` → `SOUND`.
**(6) Not claimed.** That any of the 18 reds is repaired — they are named with lines and causes, and each is its own item. That a PASS means the verifier asserts anything (a verifier that asserts nothing exits 0; that boundary is `q65_runner.py`'s and is not moved). That a row is stable — `verify_citation_hash_class_portable_default.py` read 335.90 s, 350.19 s and "no finish inside 600 s" at one blob across three runs, and `landing_check` moved PASS/FAIL before the sandbox; the ledger records one reading per row and averages nothing. That the gate is wired into `tools/verify.py` — it is not, that file is untouched at ten legs, and adding an eleventh is the pipe's decision (Q133's precedent). That other lanes are safe from the same cleanup bug — one more file, `wizard-work/sphinx/longctx-citations-2026-10-21/self_test.py`, carries the identical `rmtree(os.path.dirname(...))` line and was neither run nor edited. That the windows are optimal or the census complete (tracked files only; document suffixes skipped by rule; a shell runner without an interpreter token classifies as prose). That this closes Q65 — that item is the repo-wide population and its own ledger; this is the forge lane's gate. Files edited for this item: `wizard-work/forge/verifier_gate.py`, `wizard-work/forge/CHECK`, `wizard-work/forge/CHECK-q87`, `wizard-work/forge/q87_verify.py`, `wizard-work/forge/VERIFIER_WINDOWS.tsv`, `wizard-work/forge/VERIFIER_LEDGER.tsv` (all new), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted, and no counterparty repository file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q87.md`.
CHECK: bash wizard-work/forge/CHECK-q87
+42 -15
View File
@@ -4,7 +4,7 @@
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 45 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 115 record blocks in all, 70 of them a later closure or correction for an id already opened. The same **45 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 45 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 116 record blocks in all, 71 of them a later closure or correction for an id already opened. The same **45 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
| # | id | kind | lines | bytes | role | title | first `Status:` |
|---|---|---|---|---|---|---|---|
@@ -219,7 +219,8 @@
| 209 | Q134 | record | 13752-13765 | 6338 | record: BOLD-LEAD | **Q134 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; sol lane by authorship, unclaimed and taken). `tools/inspect_tree.py` was an 8-line one-off … | -- |
| 210 | Q118 | record | 13766-13781 | 7225 | cont: DONE | **Q118 — DONE (volt, 2026-10-31, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder read is retired three times before it may be called a failure … | -- |
| 211 | Q114 | record | 13782-13796 | 6784 | cont: DONE | **Q114 — DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on … | -- |
| 212 | Q135 | record | 13797-13811 | 7674 | record: BOLD-LEAD | **Q135 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red … | -- |
| 212 | Q135 | record | 13797-13812 | 7675 | record: BOLD-LEAD | **Q135 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red … | -- |
| 213 | Q87 | record | 13813-13829 | 8599 | cont: DONE | **Q87 — DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 → … | -- |
## Ids with no `## ` heading
@@ -236,7 +237,7 @@ Opened as records inside another section. `line` is the record's own opening lin
| Q52 | 7167 | 7167 | 90 | Q51 | BULLET+BOLD | BOLD-LEAD |
| Q53 | 7292 | 7246 | 170 | Q52 | BULLET+BOLD | BOLD-LEAD |
| Q64 | 10308 | 10308 | 32 | Q63 | BULLET+BOLD | BOLD-LEAD |
| Q65 | 11055 | 11047 | 50 | Q63 | BULLET+BOLD | BOLD-LEAD |
| Q65 | 11055 | 11047 | 51 | Q63 | BULLET+BOLD | BOLD-LEAD |
| Q99 | 12749 | 9909 | 34 | Q98 | BULLET+BOLD | BOLD-LEAD |
| Q100 | 12882 | 12882 | 29 | Q99 | BULLET+BOLD | BOLD-LEAD |
| Q101 | 12909 | 12909 | 18 | Q100 | BULLET+BOLD | BOLD-LEAD |
@@ -269,7 +270,7 @@ Opened as records inside another section. `line` is the record's own opening lin
| Q129 | 13653 | 13653 | 3 | Q128 | BULLET+BOLD | BOLD-LEAD |
| Q130 | 13675 | 13675 | 1 | Q129 | BULLET+BOLD | BOLD-LEAD |
| Q131 | 13689 | 13685 | 2 | Q115 | BULLET+BOLD | BOLD-LEAD |
| Q133 | 13731 | 13731 | 1 | Q90 | BULLET+BOLD | BOLD-LEAD |
| Q133 | 13731 | 13731 | 2 | Q90 | BULLET+BOLD | BOLD-LEAD |
| Q134 | 13752 | 13752 | 1 | Q133 | BULLET+BOLD | BOLD-LEAD |
| Q135 | 13797 | 13797 | 1 | Q114 | BULLET+BOLD | BOLD-LEAD |
@@ -1766,17 +1767,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q114 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on \u2026"
},
{
"bytes": 7674,
"end_line": 13811,
"bytes": 7675,
"end_line": 13812,
"id": "Q135",
"kind": "record",
"line": 13797,
"lines": 15,
"lines": 16,
"role": "record",
"role_detail": "BOLD-LEAD",
"status": null,
"status_line": null,
"title": "**Q135 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red \u2026"
},
{
"bytes": 8599,
"end_line": 13829,
"id": "Q87",
"kind": "record",
"line": 13813,
"lines": 17,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q87 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 \u2192 \u2026"
}
],
"ids_multi_section": {
@@ -4658,17 +4672,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q114 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on \u2026"
},
{
"bytes": 7674,
"end_line": 13811,
"bytes": 7675,
"end_line": 13812,
"id": "Q135",
"kind": "record",
"line": 13797,
"lines": 15,
"lines": 16,
"role": "record",
"role_detail": "BOLD-LEAD",
"status": null,
"status_line": null,
"title": "**Q135 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red \u2026"
},
{
"bytes": 8599,
"end_line": 13829,
"id": "Q87",
"kind": "record",
"line": 13813,
"lines": 17,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q87 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The lane's 39 committed verifiers were run by nothing, and the count had moved 34 \u2192 \u2026"
}
],
"nested": [
@@ -4758,7 +4785,7 @@ The block below is the parse this document renders, byte for byte. `--check` com
"form": "BOLD-LEAD",
"id": "Q65",
"line": 11055,
"refs": 50,
"refs": 51,
"section": "Q63",
"shape": "BULLET+BOLD"
},
@@ -5055,7 +5082,7 @@ The block below is the parse this document renders, byte for byte. `--check` com
"form": "BOLD-LEAD",
"id": "Q133",
"line": 13731,
"refs": 1,
"refs": 2,
"section": "Q90",
"shape": "BULLET+BOLD"
},
@@ -5083,9 +5110,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
"end_line": 8,
"line": 1
},
"queue_bytes": 2259729,
"queue_lines": 13811,
"queue_sha256": "ceeef0103bbfea1472c0d4f9b3d440a5285a4d5bba1415b54d4c872122b77fd0",
"queue_bytes": 2268329,
"queue_lines": 13829,
"queue_sha256": "ecf74b08b6aaa77e155cf0793fafc567ad2af428588ef92363c0adda9dcfb9f7",
"sections": [
{
"bytes": 3139,
+48
View File
@@ -0,0 +1,48 @@
#!/bin/bash
# CHECK: wizard-work/forge -- every committed verifier has a ledger row, the row
# is bound to the blob it was measured against, and any red row carries a cause.
#
# Runner note (Q87, 2026-10-31). This lane shipped 39 `verify_*.py` and no gate;
# `tools/verify.py` names `wizard-work/forge` zero times, and the lane would not
# have noticed if all 39 broke at once. This CHECK is `verifier_gate.py --check`:
# it grades a committed ledger (VERIFIER_LEDGER.tsv) against the tree at HEAD and
# runs NOTHING, so it costs well under a second and cannot be skipped for being
# slow. Execution is explicit:
#
# bash CHECK --emit # run every verifier, rewrite the ledger (minutes)
# bash CHECK --strict # also fail on any non-PASS row (the lane's health)
# bash CHECK --census # who else, if anyone, invokes each verifier
#
# The fast path is deliberately a ledger grading and not a run: the run takes
# ~8 minutes here (one recursive sweep and two live-ish harnesses), and a gate
# that takes minutes is a gate that gets skipped -- the condition that produced
# this item. What it can still catch without running anything: a verifier edited
# after its rc was measured (DRIFT, by blob), a verifier added with no row
# (UNGRADED), a row whose file is gone (STALE), a PASS whose rc says otherwise,
# and a red row with no recorded cause (CAUSELESS). `--strict` is the switch
# from "are the books consistent?" to "are the lane's verifiers true?" -- today
# the honest answer to the second is no, and the ledger names which ones.
#
# Every skip is printed rather than implied: the gate's population is the
# committed `verify_*.py` set at HEAD, so an untracked file beside the lane is
# reported as OUTSIDE (a note, not a fault, because it is not a claim about the
# commit) and every non-PASS row is printed by name on every run, strict or not.
#
# Exit codes follow the house contract: 0 clean, 1 graded a fault, 2 could not
# grade (no git, no python3, no lane).
set -o pipefail
cd "$(dirname "$(readlink -f "$0")")" || { echo "CHECK: cannot cd to the lane" >&2; exit 2; }
command -v python3 >/dev/null || { echo "CHECK: python3 not found -- cannot grade" >&2; exit 2; }
command -v git >/dev/null || { echo "CHECK: git not found -- cannot grade" >&2; exit 2; }
set +e
python3 verifier_gate.py --check "$@"
rc=$?
set -e
case $rc in
0) ;;
1) echo "CHECK: the forge verifier ledger graded a fault -- see the named rows above (Q87)." >&2; exit 1 ;;
2) echo "CHECK: could not grade -- see the gate's message above." >&2; exit 2 ;;
*) echo "CHECK: verifier_gate.py exited $rc -- internal error (a fault)." >&2; exit 1 ;;
esac
exit 0
+20
View File
@@ -0,0 +1,20 @@
#!/bin/bash
# CHECK-q87: grade the forge lane gate (Q87, 2026-10-31)
# Seven legs: the gate's own selftest, the shipped CHECK, --strict, and three
# mutants in a throwaway clone (drift, ungraded, causeless) plus a portability
# read from another cwd. See q87_verify.py's header for what each leg asserts.
set -o pipefail
cd "$(dirname "$(readlink -f "$0")")" || { echo "CHECK-q87: cannot cd to the lane" >&2; exit 2; }
command -v python3 >/dev/null || { echo "CHECK-q87: python3 not found -- cannot grade" >&2; exit 2; }
command -v git >/dev/null || { echo "CHECK-q87: git not found -- cannot grade" >&2; exit 2; }
set +e
python3 q87_verify.py --repo "$(dirname "$(readlink -f "$0")")/../.."
rc=$?
set -e
case $rc in
0) ;;
1) echo "CHECK-q87: a leg failed -- see the FAIL line above." >&2; exit 1 ;;
2) echo "CHECK-q87: could not grade -- see the message above." >&2; exit 2 ;;
*) echo "CHECK-q87: q87_verify.py exited $rc -- internal error (a fault)." >&2; exit 1 ;;
esac
exit 0
+55
View File
@@ -0,0 +1,55 @@
# Q87 verifier ledger -- one row per committed verify_*.py in wizard-work/forge,
# with the exit code it actually returned at the stated rev.
# row: <invocation> <path> <verdict> <rc> <seconds> <note> <blob> <host>
# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 / TIMEOUT / ERROR /
# MISSING. A non-PASS row must carry a note: the cause, in the file's words.
# invocation is what runs the file: `gate` when this gate is the only runner,
# `also-run` when another committed line invokes it (the census evidence is
# `--census`'s job, not a column here).
# blob is the git blob id the rc was measured against; --check recomputes it and
# a mismatch is DRIFT, so editing a verifier is a request for --emit.
# host is the absolute host root the blob reads (AST, non-docstring strings), or
# empty. A PASS carrying one prints as PASS* and is counted separately.
# seconds is wall clock under that file's window (VERIFIER_WINDOWS.tsv); a
# TIMEOUT note names the window it got.
# rev: 7b42050
# emitted_at: 2026-10-31T10:59:01Z
gate wizard-work/forge/verify_append_only_audit_dry_refactor.py FAIL 1 1.32 VERDICT: MISMATCH -- behavior NOT preserved 09d40fbd2ca08689fdae3ac003b27d766b115033 /root/troupe-standup
gate wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.25 VERDICT: ALL GREEN -- per-party DRY collapse behavior preserved 0f650718d98bf28f07be134d49803cc902bf96b7 /root/troupe-standup
gate wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 0.92 VERDICT: ALL GREEN -- census_row_delta exit-contract + testability refactor verified (selftest 37/37, byte-identical on real census pair, DRY preserved) 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 /root/troupe-standup
gate wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.42 VERDICT: ALL GREEN -- report() behavior preserved 62082e082a0472546b415219a9180836ac88908b /root/troupe-standup
gate wizard-work/forge/verify_check_lint_dry_refactor.py FAIL 1 0.86 FAIL [5] code-level: json.dump( count 2 in original, 1 in refactored: orig=1 ref=1 06a5c66c3f319d2fa7b4d9e2f73a2eafa7a99cad /root/troupe-standup
gate wizard-work/forge/verify_checkin_check_dry_refactor.py PASS 0 0.26 VERDICT: ALL GREEN -- checkin_check census 7 scans -> 1 Counter pass, byte-identical report dict d8ba2b221b80365165eb3cace77d1764651d4fba /root/troupe-standup
gate wizard-work/forge/verify_chunker_refactor.py PASS 0 0.06 PASS: fixed=lossless on all cases AND naive=lossy (>=1 case) => fix is necessary and correct. c3c7b5631f4a572c7a630269c0b3ec06e32792fc
gate wizard-work/forge/verify_citation_hash_class_dry_refactor.py FAIL 1 144.38 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/4d67c7b3c02c44279c5af04b35dae120/venv/bin/python3', '/root/.hermes/wi 93abbc78e7f061d3e22483d8db148d434fb9caef /root/troupe-standup
gate wizard-work/forge/verify_citation_hash_class_portable_default.py FAIL 1 350.19 [ok ] selftest: identical stdout (byte-for-byte) 81c481978bb3c8bdd51a0c3fdb038ce385bfc898 /root/troupe-standup
gate wizard-work/forge/verify_citation_resolve_check_dry_refactor.py FAIL 1 21.07 verify: 6 checks, 2 passed, 4 failed cd325ddd210d5fa235024780b1737f16bb15b352 /root/troupe-standup
gate wizard-work/forge/verify_doc_census_dry_refactor.py FAIL 1 25.27 VERDICT: MISMATCH -- [2] json artifact unreadable; [2] json sha mismatch; [2b] stderr drifted; [3b] original does not carry the pre-refactor scans 52a9702aa0e015bfeba92fb7e6efba4caece56d8 /root/troupe-standup
gate wizard-work/forge/verify_doc_command_report_dry.py FAIL 1 4.96 VERDICT: MISMATCH -- behavior NOT preserved 7a2f42a5b95d35d7b16e50261a7dd72d39eccbce /root/troupe-standup
gate wizard-work/forge/verify_dump_seed_refactor.py PASS 0 0.51 === Verification: 16/16 checks passed === 589179dc57966e610deffed23b29c349b247ff78 /root/troupe-standup/inbox/seed-2026-09-26.json
gate wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 6.56 FAIL [5] code-level: refactored has 0 inlined sites; original has 5: ref=0 orig=0 8d01677402c9ac02afb044c0cbdfa5a52a7d279a /root/troupe-standup
gate wizard-work/forge/verify_landing_check_refactor.py PASS 0 1.73 verify: 13 checks, 13 passed, 0 failed 6aa282dade614e3e55874adc3e37cf7c01313f3e /root/troupe-standup
gate wizard-work/forge/verify_mutant_scan_dry_refactor.py FAIL 1 6.35 verify_mutant_scan_dry_refactor: 2 FAILED of 18 checks e9fb4006ba56e49d0ae3f68658db2966aa22cf2d
gate wizard-work/forge/verify_ops_pulse_refactor.py PASS 0 0.54 ALL_IDENTICAL ccbe3f8fa7ccdab659e68d6908b7893903759925 /root/.hermes
gate wizard-work/forge/verify_path_claim_apply_baseline_refactor.py FAIL 1 47.21 verify: 10 checks, 8 passed, 2 failed 999da7274c3cbbd9c84c3e3c106fbd65cf79b4e3 /root/troupe-standup
gate wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 35.15 verify: 11 checks, 9 passed, 2 failed ab35c13f9565e9eebbc80a210001ec37a002a267 /root/troupe-standup
gate wizard-work/forge/verify_pdf_refactor.py PASS 0 3.70 SUMMARY: ALL PASS 44ce76d39b69a7d6798af521f958370bf30ad2b1 /root/.hermes/wizard-outputs/forge/before/pdf_create.py
also-run wizard-work/forge/verify_probe_forge_refactor.py PASS 0 0.62 === Verification: 12/12 checks passed === d3434f0a500a1c1509d80c53164f6a2e352bf4b8
gate wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 7.20 VERDICT: MISMATCH -- behavior NOT preserved 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 /root/troupe-standup
gate wizard-work/forge/verify_r41_pixel_diff.py FAIL 1 0.16 ModuleNotFoundError: No module named 'numpy' 4c1e5bec352f032e6ecb2be86334e33c2dd5d00e /root/.hermes/wizard-outputs/forge
gate wizard-work/forge/verify_reconstruct_smell_fix.py FAIL 1 0.16 FAIL (c): chunker test suite exited 1 80a9377674062386b037aeac0de6015b5c68799f /root/troupe-standup/wizard-work/forge/chunker.py
gate wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 1.35 verify: 8 checks, 8 passed, 0 failed c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 /root/troupe-standup
gate wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.53 === Verification: 10/10 checks passed === 2394abeec25726726edd91696520d788d72a5d88 /root/troupe-standup
gate wizard-work/forge/verify_roundtrip_refactor.py PASS 0 0.42 === Verification: 6/6 checks passed === 0f1d7ea9d5e542c8cb70c8e3def85ba684641b2c /root/troupe-standup
gate wizard-work/forge/verify_scan_dry_refactor.py PASS 0 2.53 === Verification: 11/11 checks passed === 6fb7aa14b96eea6731c51c25e22572cca009adf4
gate wizard-work/forge/verify_selftest_sweep_dry_refactor.py FAIL 1 0.10 [setup] BEFORE snapshot missing: /root/.hermes/profiles/forge/cache/scratch/selftest_sweep_before.py 6f5e1621139c9ab98ec5b25ce03bda88a73cf9cf /root/troupe-standup
also-run wizard-work/forge/verify_selftest_sweep_q93_refactor.py FAIL 1 1.48 verify_selftest_sweep_q93_refactor: FAIL (2 failed, 2 checks) 971430df5da46e70371e7f75b8c9d88d095a6ab7
gate wizard-work/forge/verify_selftest_sweep_refactor.py PASS 0 86.94 verify_selftest_sweep_refactor: 5 checks, 5 passed, 0 failed cc8782b4edfcd08eb5ed69f06308c1185ffb41e8 /root/troupe-standup
gate wizard-work/forge/verify_served_drift_summarise_dry.py PASS 0 4.52 VERDICT: ALL GREEN -- summarise() DRY collapse behavior preserved de5a49160b2c2a8f4759e0607a0f0c7b52b3bec1 /root/troupe-standup
gate wizard-work/forge/verify_smell_check_monotonic_refactor.py PASS 0 0.59 === Verification: 9/9 checks passed === 8b8ccea5be0e938fb4bba67080cc65a97762f3f4 /root/troupe-standup
gate wizard-work/forge/verify_smell_scan_refactor.py PASS 0 0.30 === Verification: 8/8 checks passed === 499c8e000359e8ef3486a89031673059d471178c /root/troupe-standup
gate wizard-work/forge/verify_tool_map_dry_refactor.py FAIL 1 24.84 VERDICT: MISMATCH -- behavior NOT preserved de1d9b7a65cfaaacf029b048228806acbf2f817c /root/troupe-standup
also-run wizard-work/forge/verify_triage_dump_pending_refactor.py PASS 0 1.24 === Verification: 13/13 checks passed === 38894d9812a8dfbda63bfa6d01407dba1b3ec042 /root/troupe-standup/inbox/seed-2026-09-26.json
gate wizard-work/forge/verify_triage_dump_refactor.py PASS 0 1.07 === Verification: 14/14 checks passed === 611e56d4d3561a2e52d810081255f575126472bc /root/troupe-standup/inbox/seed-2026-09-26.json
gate wizard-work/forge/verify_triage_probe_refactor.py PASS 0 1.09 === Verification: 14/14 checks passed === c8c9288477c8170d0b35bd40064ad58e93dcaf6c /root/troupe-standup/inbox/seed-2026-09-26.json
gate wizard-work/forge/verify_write_probe_dry_refactor.py FAIL 1 14.14 VERDICT: MISMATCH -- behavior NOT preserved ef4d8c00e4a84495e1612bec4734a366d4869b67 /root/troupe-standup
1 # Q87 verifier ledger -- one row per committed verify_*.py in wizard-work/forge,
2 # with the exit code it actually returned at the stated rev.
3 # row: <invocation> <path> <verdict> <rc> <seconds> <note> <blob> <host>
4 # verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 / TIMEOUT / ERROR /
5 # MISSING. A non-PASS row must carry a note: the cause, in the file's words.
6 # invocation is what runs the file: `gate` when this gate is the only runner,
7 # `also-run` when another committed line invokes it (the census evidence is
8 # `--census`'s job, not a column here).
9 # blob is the git blob id the rc was measured against; --check recomputes it and
10 # a mismatch is DRIFT, so editing a verifier is a request for --emit.
11 # host is the absolute host root the blob reads (AST, non-docstring strings), or
12 # empty. A PASS carrying one prints as PASS* and is counted separately.
13 # seconds is wall clock under that file's window (VERIFIER_WINDOWS.tsv); a
14 # TIMEOUT note names the window it got.
15 # rev: 7b42050
16 # emitted_at: 2026-10-31T10:59:01Z
17 gate wizard-work/forge/verify_append_only_audit_dry_refactor.py FAIL 1 1.32 VERDICT: MISMATCH -- behavior NOT preserved 09d40fbd2ca08689fdae3ac003b27d766b115033 /root/troupe-standup
18 gate wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.25 VERDICT: ALL GREEN -- per-party DRY collapse behavior preserved 0f650718d98bf28f07be134d49803cc902bf96b7 /root/troupe-standup
19 gate wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 0.92 VERDICT: ALL GREEN -- census_row_delta exit-contract + testability refactor verified (selftest 37/37, byte-identical on real census pair, DRY preserved) 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 /root/troupe-standup
20 gate wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.42 VERDICT: ALL GREEN -- report() behavior preserved 62082e082a0472546b415219a9180836ac88908b /root/troupe-standup
21 gate wizard-work/forge/verify_check_lint_dry_refactor.py FAIL 1 0.86 FAIL [5] code-level: json.dump( count 2 in original, 1 in refactored: orig=1 ref=1 06a5c66c3f319d2fa7b4d9e2f73a2eafa7a99cad /root/troupe-standup
22 gate wizard-work/forge/verify_checkin_check_dry_refactor.py PASS 0 0.26 VERDICT: ALL GREEN -- checkin_check census 7 scans -> 1 Counter pass, byte-identical report dict d8ba2b221b80365165eb3cace77d1764651d4fba /root/troupe-standup
23 gate wizard-work/forge/verify_chunker_refactor.py PASS 0 0.06 PASS: fixed=lossless on all cases AND naive=lossy (>=1 case) => fix is necessary and correct. c3c7b5631f4a572c7a630269c0b3ec06e32792fc
24 gate wizard-work/forge/verify_citation_hash_class_dry_refactor.py FAIL 1 144.38 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/4d67c7b3c02c44279c5af04b35dae120/venv/bin/python3', '/root/.hermes/wi 93abbc78e7f061d3e22483d8db148d434fb9caef /root/troupe-standup
25 gate wizard-work/forge/verify_citation_hash_class_portable_default.py FAIL 1 350.19 [ok ] selftest: identical stdout (byte-for-byte) 81c481978bb3c8bdd51a0c3fdb038ce385bfc898 /root/troupe-standup
26 gate wizard-work/forge/verify_citation_resolve_check_dry_refactor.py FAIL 1 21.07 verify: 6 checks, 2 passed, 4 failed cd325ddd210d5fa235024780b1737f16bb15b352 /root/troupe-standup
27 gate wizard-work/forge/verify_doc_census_dry_refactor.py FAIL 1 25.27 VERDICT: MISMATCH -- [2] json artifact unreadable; [2] json sha mismatch; [2b] stderr drifted; [3b] original does not carry the pre-refactor scans 52a9702aa0e015bfeba92fb7e6efba4caece56d8 /root/troupe-standup
28 gate wizard-work/forge/verify_doc_command_report_dry.py FAIL 1 4.96 VERDICT: MISMATCH -- behavior NOT preserved 7a2f42a5b95d35d7b16e50261a7dd72d39eccbce /root/troupe-standup
29 gate wizard-work/forge/verify_dump_seed_refactor.py PASS 0 0.51 === Verification: 16/16 checks passed === 589179dc57966e610deffed23b29c349b247ff78 /root/troupe-standup/inbox/seed-2026-09-26.json
30 gate wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 6.56 FAIL [5] code-level: refactored has 0 inlined sites; original has 5: ref=0 orig=0 8d01677402c9ac02afb044c0cbdfa5a52a7d279a /root/troupe-standup
31 gate wizard-work/forge/verify_landing_check_refactor.py PASS 0 1.73 verify: 13 checks, 13 passed, 0 failed 6aa282dade614e3e55874adc3e37cf7c01313f3e /root/troupe-standup
32 gate wizard-work/forge/verify_mutant_scan_dry_refactor.py FAIL 1 6.35 verify_mutant_scan_dry_refactor: 2 FAILED of 18 checks e9fb4006ba56e49d0ae3f68658db2966aa22cf2d
33 gate wizard-work/forge/verify_ops_pulse_refactor.py PASS 0 0.54 ALL_IDENTICAL ccbe3f8fa7ccdab659e68d6908b7893903759925 /root/.hermes
34 gate wizard-work/forge/verify_path_claim_apply_baseline_refactor.py FAIL 1 47.21 verify: 10 checks, 8 passed, 2 failed 999da7274c3cbbd9c84c3e3c106fbd65cf79b4e3 /root/troupe-standup
35 gate wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 35.15 verify: 11 checks, 9 passed, 2 failed ab35c13f9565e9eebbc80a210001ec37a002a267 /root/troupe-standup
36 gate wizard-work/forge/verify_pdf_refactor.py PASS 0 3.70 SUMMARY: ALL PASS 44ce76d39b69a7d6798af521f958370bf30ad2b1 /root/.hermes/wizard-outputs/forge/before/pdf_create.py
37 also-run wizard-work/forge/verify_probe_forge_refactor.py PASS 0 0.62 === Verification: 12/12 checks passed === d3434f0a500a1c1509d80c53164f6a2e352bf4b8
38 gate wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 7.20 VERDICT: MISMATCH -- behavior NOT preserved 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 /root/troupe-standup
39 gate wizard-work/forge/verify_r41_pixel_diff.py FAIL 1 0.16 ModuleNotFoundError: No module named 'numpy' 4c1e5bec352f032e6ecb2be86334e33c2dd5d00e /root/.hermes/wizard-outputs/forge
40 gate wizard-work/forge/verify_reconstruct_smell_fix.py FAIL 1 0.16 FAIL (c): chunker test suite exited 1 80a9377674062386b037aeac0de6015b5c68799f /root/troupe-standup/wizard-work/forge/chunker.py
41 gate wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 1.35 verify: 8 checks, 8 passed, 0 failed c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 /root/troupe-standup
42 gate wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.53 === Verification: 10/10 checks passed === 2394abeec25726726edd91696520d788d72a5d88 /root/troupe-standup
43 gate wizard-work/forge/verify_roundtrip_refactor.py PASS 0 0.42 === Verification: 6/6 checks passed === 0f1d7ea9d5e542c8cb70c8e3def85ba684641b2c /root/troupe-standup
44 gate wizard-work/forge/verify_scan_dry_refactor.py PASS 0 2.53 === Verification: 11/11 checks passed === 6fb7aa14b96eea6731c51c25e22572cca009adf4
45 gate wizard-work/forge/verify_selftest_sweep_dry_refactor.py FAIL 1 0.10 [setup] BEFORE snapshot missing: /root/.hermes/profiles/forge/cache/scratch/selftest_sweep_before.py 6f5e1621139c9ab98ec5b25ce03bda88a73cf9cf /root/troupe-standup
46 also-run wizard-work/forge/verify_selftest_sweep_q93_refactor.py FAIL 1 1.48 verify_selftest_sweep_q93_refactor: FAIL (2 failed, 2 checks) 971430df5da46e70371e7f75b8c9d88d095a6ab7
47 gate wizard-work/forge/verify_selftest_sweep_refactor.py PASS 0 86.94 verify_selftest_sweep_refactor: 5 checks, 5 passed, 0 failed cc8782b4edfcd08eb5ed69f06308c1185ffb41e8 /root/troupe-standup
48 gate wizard-work/forge/verify_served_drift_summarise_dry.py PASS 0 4.52 VERDICT: ALL GREEN -- summarise() DRY collapse behavior preserved de5a49160b2c2a8f4759e0607a0f0c7b52b3bec1 /root/troupe-standup
49 gate wizard-work/forge/verify_smell_check_monotonic_refactor.py PASS 0 0.59 === Verification: 9/9 checks passed === 8b8ccea5be0e938fb4bba67080cc65a97762f3f4 /root/troupe-standup
50 gate wizard-work/forge/verify_smell_scan_refactor.py PASS 0 0.30 === Verification: 8/8 checks passed === 499c8e000359e8ef3486a89031673059d471178c /root/troupe-standup
51 gate wizard-work/forge/verify_tool_map_dry_refactor.py FAIL 1 24.84 VERDICT: MISMATCH -- behavior NOT preserved de1d9b7a65cfaaacf029b048228806acbf2f817c /root/troupe-standup
52 also-run wizard-work/forge/verify_triage_dump_pending_refactor.py PASS 0 1.24 === Verification: 13/13 checks passed === 38894d9812a8dfbda63bfa6d01407dba1b3ec042 /root/troupe-standup/inbox/seed-2026-09-26.json
53 gate wizard-work/forge/verify_triage_dump_refactor.py PASS 0 1.07 === Verification: 14/14 checks passed === 611e56d4d3561a2e52d810081255f575126472bc /root/troupe-standup/inbox/seed-2026-09-26.json
54 gate wizard-work/forge/verify_triage_probe_refactor.py PASS 0 1.09 === Verification: 14/14 checks passed === c8c9288477c8170d0b35bd40064ad58e93dcaf6c /root/troupe-standup/inbox/seed-2026-09-26.json
55 gate wizard-work/forge/verify_write_probe_dry_refactor.py FAIL 1 14.14 VERDICT: MISMATCH -- behavior NOT preserved ef4d8c00e4a84495e1612bec4734a366d4869b67 /root/troupe-standup
+8
View File
@@ -0,0 +1,8 @@
# Q87 per-file execution windows -- path <TAB> seconds <TAB> why.
# The default window for every file this table does not name is 90 s (--timeout).
# A row here says the file's own work costs more than the default, and a TIMEOUT
# row's note names the window it actually got, so the ledger cannot disagree
# with the run. Every figure below is a measurement at 7b42050, not an estimate.
wizard-work/forge/verify_citation_hash_class_dry_refactor.py 300 measured 133.98 s and 144.38 s at 7b42050 (rc 1 both times); it spawns child sweeps and its traceback is a TimeoutExpired from one of them
wizard-work/forge/verify_citation_hash_class_portable_default.py 900 live HTTP: 335.90 s, then 350.19 s, then no finish inside 600 s -- the same blob at three runs; 900 s is the gate's ceiling for one file
wizard-work/forge/verify_selftest_sweep_refactor.py 300 measured 86.94 s and 95.54 s (rc 0 both); its subject is the repo-wide selftest sweep
1 # Q87 per-file execution windows -- path <TAB> seconds <TAB> why.
2 # The default window for every file this table does not name is 90 s (--timeout).
3 # A row here says the file's own work costs more than the default, and a TIMEOUT
4 # row's note names the window it actually got, so the ledger cannot disagree
5 # with the run. Every figure below is a measurement at 7b42050, not an estimate.
6 wizard-work/forge/verify_citation_hash_class_dry_refactor.py 300 measured 133.98 s and 144.38 s at 7b42050 (rc 1 both times); it spawns child sweeps and its traceback is a TimeoutExpired from one of them
7 wizard-work/forge/verify_citation_hash_class_portable_default.py 900 live HTTP: 335.90 s, then 350.19 s, then no finish inside 600 s -- the same blob at three runs; 900 s is the gate's ceiling for one file
8 wizard-work/forge/verify_selftest_sweep_refactor.py 300 measured 86.94 s and 95.54 s (rc 0 both); its subject is the repo-wide selftest sweep
+195
View File
@@ -0,0 +1,195 @@
#!/usr/bin/env python3
"""q87_verify.py -- the instrument behind CHECK-q87 (2026-10-31, volt lane).
WHAT IT GRADES
The forge lane gate (`wizard-work/forge/CHECK` -> `verifier_gate.py`) and the
ledger it grades. Seven legs, each of which can fail on its own:
1 selftest the gate's own 10 cells pass (a cell that silently passes on a
truthy string is the defect the first draft of the selftest had).
2 check `CHECK` in this tree exits 0 and prints the population and the
known-red count, so a green cannot hide a red.
3 strict the same gate with --strict exits 1 and names the non-PASS rows,
which is the switch from "books consistent" to "lane healthy".
4 drift in a throwaway clone, one byte appended to a PASS verifier makes
--check exit 1 with DRIFT naming that file, and restoring the
bytes makes it exit 0 again.
5 ungraded a new committed `verify_*.py` with no ledger row makes --check
exit 1 with UNGRADED naming it (a file added tomorrow is seen).
6 causeless blanking the note of a FAIL row makes --check exit 1 with
CAUSELESS (a red must carry its reason).
7 portable --check run from a different cwd, against a clone of this tree,
returns the same verdict as in-tree -- so the gate is a function
of the tree, not of this host's path.
The clone legs never touch the graded tree: every mutation happens in a
TemporaryDirectory copy that is thrown away.
EXIT CODES
0 every leg passed / 1 a leg failed / 2 could not grade (no git, no clone).
Usage:
python3 q87_verify.py [--repo /root/troupe-standup] [--verbose]
"""
import argparse
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
LANE = "wizard-work/forge"
GATE = "verifier_gate.py"
LEDGER = "VERIFIER_LEDGER.tsv"
WINDOWS = "VERIFIER_WINDOWS.tsv"
CHECK = "CHECK"
FILES = (GATE, LEDGER, WINDOWS, CHECK)
def run(argv, cwd=None, timeout=600):
r = subprocess.run(argv, cwd=cwd, capture_output=True, text=True,
errors="replace", timeout=timeout,
env=dict(os.environ, PYTHONDONTWRITEBYTECODE="1"))
return r.returncode, (r.stdout or "") + (r.stderr or "")
def gate(repo, *extra, cwd=None):
return run([sys.executable, str(Path(repo) / LANE / GATE), "--repo", str(repo)]
+ list(extra), cwd=cwd)
def clone_of(repo, dest):
rc, out = run(["git", "clone", "-q", "--no-hardlinks", str(repo), str(dest)])
if rc != 0:
raise RuntimeError("git clone failed: %s" % out.strip()[:200])
for name in FILES:
src = Path(repo) / LANE / name
if src.exists():
shutil.copyfile(src, Path(dest) / LANE / name)
return Path(dest)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--repo", default=str(Path(__file__).resolve().parents[2]))
ap.add_argument("--verbose", action="store_true")
args = ap.parse_args()
repo = Path(args.repo).resolve()
if not (repo / ".git").exists():
print("q87: %s is not a git worktree -- could not grade" % repo)
return 2
legs = []
def leg(name, ok, detail):
legs.append((name, bool(ok), detail))
print(" [%s] %-9s %s" % ("ok " if ok else "FAIL", name, detail), flush=True)
# 1 -- the gate's own cells.
rc, out = gate(repo, "--selftest")
tail = [l for l in out.splitlines() if "selftest:" in l]
leg("selftest", rc == 0 and bool(tail) and "0 failed" in tail[-1]
and "cells," in tail[-1],
(tail[-1] if tail else "no selftest summary (rc %d)" % rc))
# 2 -- the shipped gate is green and says what it is green about. The count
# is read from the ledger, not hardcoded: a ledger that re-emits after a
# verifier changes would otherwise break this leg for the wrong reason.
rc, out = run(["bash", str(repo / LANE / CHECK)])
pop = [l for l in out.splitlines() if "committed verifier(s)" in l]
red = [l for l in out.splitlines() if l.strip().startswith("known-red")]
led = (repo / LANE / LEDGER).read_text(encoding="utf-8")
n_rows = sum(1 for l in led.splitlines() if l.strip() and not l.startswith("#"))
n_red = sum(1 for l in led.splitlines()
if l.strip() and not l.startswith("#") and l.split("\t")[2] != "PASS")
ok2 = (rc == 0 and bool(pop) and bool(red)
and "%d committed verifier(s), %d ledger row(s)" % (n_rows, n_rows) in pop[0]
and ("%d of %d" % (n_red, n_rows)) in red[0] and n_red >= 1)
leg("check", ok2, "%s; %s" % (pop[0].strip() if pop else "no population line",
red[0].strip() if red else "no known-red line"))
# 3 -- strict is the lane's health, and today it is not healthy.
rc, out = gate(repo, "--check", "--strict")
strict_n = [l for l in out.splitlines() if l.strip().startswith("verifier_gate:")
and "strict" in l]
n_strict = int(strict_n[-1].split()[1]) if strict_n else 0
leg("strict", rc == 1 and n_strict >= 18 and "strict: every non-PASS row" in out,
"rc %d, %d fault(s) from the non-PASS rows" % (rc, n_strict))
# 4..6 -- mutants in a throwaway clone.
with tempfile.TemporaryDirectory() as tmp:
try:
clone = clone_of(repo, Path(tmp) / "c")
except RuntimeError as exc:
print("q87: %s" % exc)
return 2
target = clone / LANE / "verify_chunker_refactor.py"
rc0, _ = gate(clone, "--check")
base_rc, base_sha = run(["git", "-C", str(clone), "rev-parse", "HEAD"])
original = target.read_text(encoding="utf-8")
target.write_text(original + "# one byte of drift\n", encoding="utf-8")
# the ledger binds to the COMMITTED blob, so the mutant has to be a
# commit -- an edit left in the worktree is the UNCOMMITTED note, not DRIFT
run(["git", "-C", str(clone), "add", str(target.relative_to(clone))])
run(["git", "-C", str(clone), "-c", "user.email=q87@example.invalid",
"-c", "user.name=q87", "commit", "-qm", "drifts one verifier"])
new_rc, new_sha = run(["git", "-C", str(clone), "rev-parse", "HEAD"])
committed = new_rc == 0 and new_sha.strip() != base_sha.strip()
rc1, out1 = gate(clone, "--check")
drift_line = [l for l in out1.splitlines()
if l.strip().startswith("DRIFT ")
and "verify_chunker_refactor.py" in l]
if committed:
run(["git", "-C", str(clone), "reset", "-q", "--hard", "HEAD~1"])
rc2, _ = gate(clone, "--check")
leg("drift", committed and rc0 == 0 and rc1 == 1 and rc2 == 0
and bool(drift_line),
"clean %d -> committed edit %d%s -> reverted %d"
% (rc0, rc1, (" (%s)" % drift_line[0].strip()[:52]) if drift_line else "",
rc2))
new = clone / LANE / "verify_added_tomorrow.py"
new.write_text("print('nothing')\n", encoding="utf-8")
run(["git", "-C", str(clone), "add", str(new.relative_to(clone))])
run(["git", "-C", str(clone), "-c", "user.email=q87@example.invalid",
"-c", "user.name=q87", "commit", "-qm", "adds a verifier"])
rc3, out3 = gate(clone, "--check")
leg("ungraded", rc3 == 1 and "UNGRADED" in out3 and "verify_added_tomorrow" in out3,
"rc %d%s" % (rc3, " naming UNGRADED verify_added_tomorrow.py"
if "UNGRADED" in out3 else " without an UNGRADED fault"))
# causeless: blank the note of the first FAIL row, in a fresh clone state
led = clone / LANE / LEDGER
lines = led.read_text(encoding="utf-8").splitlines()
hit = None
for i, line in enumerate(lines):
if line.startswith("#") or not line.strip():
continue
parts = line.split("\t")
if parts[2] == "FAIL":
parts[5] = ""
lines[i] = "\t".join(parts)
hit = parts[1]
break
led.write_text("\n".join(lines) + "\n", encoding="utf-8")
rc4, out4 = gate(clone, "--check")
leg("causeless", rc4 == 1 and hit and ("CAUSELESS" in out4),
"rc %d%s" % (rc4, (" naming CAUSELESS %s" % hit) if "CAUSELESS" in out4
else " with no CAUSELESS fault"))
# 7 -- portability: same verdict from another cwd, through the clone.
rc5, out5 = gate(clone, "--check", cwd="/")
leg("portable", rc5 == 1 and "CAUSELESS" in out5,
"from cwd=/ against the clone the same fault is read (rc %d)" % rc5)
bad = [l for l in legs if not l[1]]
print("q87: %d leg(s), %d passed, %d fault(s)"
% (len(legs), len(legs) - len(bad), len(bad)))
return 0 if not bad else 1
if __name__ == "__main__":
sys.exit(main())
+802
View File
@@ -0,0 +1,802 @@
#!/usr/bin/env python3
"""verifier_gate.py -- the runner `wizard-work/forge` never had (Q87, 2026-10-31).
WHY THIS EXISTS
At `431bca1` the pipe measured 34 `verify_*.py` in this lane and no `CHECK`;
the correction at `d56105a` moved the count to 36 and the runner count stayed
zero. At `7b42050` the lane holds 39 and `tools/verify.py` still names
`wizard-work/forge` 0 times. A verifier nothing runs is an unread claim: its
verdict is whatever its author last saw by hand, and this lane would not
notice if all 39 were broken at once. This gate executes them, keeps one row
per file bound to the blob the rc was measured against, and re-grades that
ledger on every run without re-executing it.
WHAT IT MEASURES -- one row per committed verifier, one verdict per row
PASS rc 0: the file ran and its checks held.
FAIL rc 1: the file ran and its checks did NOT hold. A finding, not a
bookkeeping error -- the file's claim is false at this tree.
REFUSED rc 2: the file declined to grade. Not a pass.
OTHER_RC rc above 2 that the house contract does not define.
TIMEOUT no finish inside its per-file window (see WINDOWS below).
ERROR the interpreter could not run the file at all (no rc to grade).
MISSING the row's path is not in the tree at --rev.
A non-PASS row must carry a reason in its note column. A red with no recorded
cause is a FAULT (`CAUSELESS`), because "it is red and nobody wrote down why"
is the state this lane has been in since 2026-10-27.
THE LEDGER IS BOUND TO CONTENT
Column 7 is the git blob id each rc was measured against. `--check` recomputes
it at its own `--rev`; a mismatch is DRIFT, so editing a verifier is a request
for `--emit` rather than a free pass on an old green. Column 8 is a static
host-pin probe: the committed blob is parsed as an AST and every non-docstring
string constant is tested for an absolute host root (`/root/troupe-standup`,
`/root/.hermes`). A PASS carrying one prints as `PASS*` and is reported
separately, because a verifier that reads this host's copy graded this host
and not the tree that invoked it (Q62/Q106's class).
WHAT IT DELIBERATELY DOES NOT DO
`--check` (the default, what `CHECK` runs) does NOT execute anything: it
grades the ledger against the tree in well under a second. Execution is
`--emit`, it takes minutes here (measured: 470 s for 39 files at a 75 s
default window, plus three files that need longer), and a gate that takes
minutes is a gate that gets skipped. `--strict` promotes any non-PASS row to
a fatal fault, which is the switch between "are the books consistent?" and
"are the lane's verifiers true?" -- today the honest answer to the second is
no, and the rows say which ones.
It does not decide whether a verifier is *good*, only whether it exits 0. A
verifier that asserts nothing exits 0. That is the same boundary q65_runner
draws for the repo-wide population and it is not moved here.
WINDOWS
A single `--timeout` for the whole lane is wrong: three files here need more
than 75 s and one of them is a recursive sweep. `VERIFIER_WINDOWS.tsv` names
one window per path with the reason; `--timeout` stays the fallback. A
TIMEOUT row's note names the window it actually got, so the ledger cannot
quietly disagree with the run.
EXIT CODES -- the house contract
0 clean / 1 graded a fault / 2 could not grade (no git, no python3, no lane).
Usage:
python3 verifier_gate.py --check # grade the ledger (fast, no runs)
python3 verifier_gate.py --check --strict # ...and any non-PASS row is fatal
python3 verifier_gate.py --emit # execute every verifier, rewrite
python3 verifier_gate.py --census # who else runs each file
python3 verifier_gate.py --selftest # the gate's own cells
"""
import argparse
import ast
import os
import re
import shutil
import subprocess
import sys
import tempfile
import time
from pathlib import Path
LANE = "wizard-work/forge"
HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[1]
LEDGER = HERE / "VERIFIER_LEDGER.tsv"
WINDOWS = HERE / "VERIFIER_WINDOWS.tsv"
PASS, FAIL, REFUSED = "PASS", "FAIL", "REFUSED"
OTHER_RC, TIMEOUT, ERROR, MISSING = "OTHER_RC", "TIMEOUT", "ERROR", "MISSING"
VERDICTS = (PASS, FAIL, REFUSED, OTHER_RC, TIMEOUT, ERROR, MISSING)
HOST_ROOTS = ("/root/troupe-standup", "/root/.hermes")
VERDICT_OF_RC = {0: PASS, 1: FAIL, 2: REFUSED}
def git(repo, *args):
"""(rc, stdout, stderr) -- never raises, so a missing git is rc 2 upstream."""
try:
r = subprocess.run(["git", "-C", str(repo)] + list(args),
capture_output=True, text=True, errors="replace",
timeout=120)
return r.returncode, r.stdout, r.stderr
except (OSError, subprocess.SubprocessError) as exc:
return 127, "", str(exc)
def population(repo, rev):
"""Committed verify_*.py under the lane at `rev` -- from the tree, not a list.
The population is read from git rather than from a directory glob because
the question is "what did this commit ship", and an untracked scratch file
beside the lane is not a claim about the commit.
"""
rc, out, err = git(repo, "ls-tree", "-r", "--name-only", rev, "--", LANE)
if rc != 0:
raise SystemExit("git ls-tree %s failed: %s" % (rev, (err or "").strip()))
paths = [p.strip() for p in out.splitlines() if p.strip()]
return sorted(p for p in paths
if p.startswith(LANE + "/")
and os.path.basename(p).startswith("verify_")
and p.endswith(".py"))
def blob_of(repo, rev, path):
rc, out, _ = git(repo, "rev-parse", "%s:%s" % (rev, path))
return out.strip() if rc == 0 else ""
def host_pin(repo, rev, path):
"""The first absolute host root this blob reads, or ''.
Parsed rather than grepped: a docstring that *names* the host root is a
description, and the Q85 lesson is that a text scan turns description into
finding. Only string constants outside docstrings count.
"""
rc, blob, _ = git(repo, "show", "%s:%s" % (rev, path))
if rc != 0:
return ""
try:
tree = ast.parse(blob)
except SyntaxError:
return "unparseable"
docs = set()
for node in ast.walk(tree):
if isinstance(node, (ast.Module, ast.FunctionDef, ast.AsyncFunctionDef,
ast.ClassDef)):
body = getattr(node, "body", [])
if body and isinstance(body[0], ast.Expr) \
and isinstance(body[0].value, ast.Constant) \
and isinstance(body[0].value.value, str):
docs.add(id(body[0].value))
for node in ast.walk(tree):
if isinstance(node, ast.Constant) and isinstance(node.value, str):
if id(node) in docs:
continue
for root in HOST_ROOTS:
if root in node.value:
hits = [l for l in node.value.splitlines() if root in l]
return (hits[0] if hits else root).strip()[:120]
return ""
def read_windows(path):
"""{path: (seconds, why)} -- one window per file, the reason mandatory."""
win = {}
p = Path(path)
if not p.exists():
return win
for line in p.read_text(encoding="utf-8").splitlines():
if not line.strip() or line.lstrip().startswith("#"):
continue
parts = line.split("\t")
if len(parts) < 3:
raise SystemExit("windows row needs path<TAB>seconds<TAB>why: %r" % line)
try:
win[parts[0].strip()] = (float(parts[1]), parts[2].strip())
except ValueError:
raise SystemExit("window seconds is not a number: %r" % line)
return win
def run_one(repo, rel, timeout, windows):
"""Execute one verifier in its own directory, under its own TMPDIR.
THE SANDBOX IS NOT A LUXURY. Five files in this lane clean up with
`shutil.rmtree(os.path.dirname(<their temp path>))`, which is the PARENT of
the directory they made -- i.e. `$TMPDIR` itself:
verify_append_only_audit_dry_refactor.py:304, verify_append_only_owners_
dry_refactor.py:238, verify_gate_blame_dry_refactor.py:178, verify_served_
drift_summarise_dry.py:192, verify_write_probe_dry_refactor.py:133.
Running them with the caller's TMPDIR deleted the caller's scratch tree (this
item's own first measurement, at 09:27, died that way: the probe's TSV was
gone mid-run) and every later file failed with FileNotFoundError naming a
directory it had just created, because `mkdtemp` raises on the missing
PARENT. Each file now gets `<own base>/tb` as TMPDIR, so a verifier that
deletes its parent takes out a throwaway directory the gate removes itself.
"""
window, why = windows.get(rel, (timeout, ""))
full = Path(repo) / rel
if not full.exists():
return {"path": rel, "verdict": MISSING, "rc": None, "seconds": 0.0,
"note": "not in worktree"}
base = tempfile.mkdtemp(prefix="forge_gate_")
tb = os.path.join(base, "tb")
os.makedirs(tb, exist_ok=True)
env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1", PYTHONUNBUFFERED="1",
TMPDIR=tb, TEMP=tb, TMP=tb)
env.pop("PYTHONOPTIMIZE", None)
t0 = time.time()
try:
r = subprocess.run([sys.executable, "-B", full.name], cwd=str(full.parent),
env=env, capture_output=True, timeout=window,
text=True, errors="replace")
rc, so, se = r.returncode, r.stdout, r.stderr
except subprocess.TimeoutExpired:
return {"path": rel, "verdict": TIMEOUT, "rc": None,
"seconds": round(time.time() - t0, 2),
"note": "no finish in %ss%s"
% (window, (" (%s)" % why) if why else "")}
except OSError as exc:
return {"path": rel, "verdict": ERROR, "rc": None, "seconds": 0.0,
"note": str(exc)[:160]}
finally:
shutil.rmtree(base, ignore_errors=True)
secs = round(time.time() - t0, 2)
verdict = VERDICT_OF_RC.get(rc, OTHER_RC if rc > 2 else ERROR)
note = verdict_line(so, se, rc)
return {"path": rel, "verdict": verdict, "rc": rc, "seconds": secs,
"note": note}
def verdict_line(so, se, rc):
"""The file's own sentence, not whichever stream happened to end last.
The first draft took stderr's last line, and `verify_append_only_audit_dry_
refactor.py` wrote its cause to stdout while stderr ended on a SyntaxWarning
fragment -- the row read `(b"dash \\u2014 here\\nplain\\n", ...)` and lost the
`VERDICT: MISMATCH` it had printed. A verdict-looking line is preferred, from
stdout first; the last line of either stream is the fallback.
"""
keys = ("VERDICT", "FAIL", "checks", "PASS", "failed", "Error")
for text in (so, se):
lines = [l.strip() for l in (text or "").splitlines() if l.strip()]
for line in reversed(lines):
if line.startswith("[ok") or line.startswith("ok:"):
continue # a passing cell is not this row's cause
if any(k in line for k in keys):
return line[:160]
# Nothing verdict-shaped: prefer stderr's last line, because that is where a
# traceback puts the reason, and stdout's last line is often a passing cell
# the file printed before the thing that killed it.
for text in (se, so):
lines = [l.strip() for l in (text or "").splitlines() if l.strip()]
if lines:
return lines[-1][:160]
return "" if rc == 0 else "exited rc %d with no stdout/stderr" % rc
HEADER = """\
# Q87 verifier ledger -- one row per committed verify_*.py in wizard-work/forge,
# with the exit code it actually returned at the stated rev.
# row: <invocation>\t<path>\t<verdict>\t<rc>\t<seconds>\t<note>\t<blob>\t<host>
# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 / TIMEOUT / ERROR /
# MISSING. A non-PASS row must carry a note: the cause, in the file's words.
# invocation is what runs the file: `gate` when this gate is the only runner,
# `also-run` when another committed line invokes it (the census evidence is
# `--census`'s job, not a column here).
# blob is the git blob id the rc was measured against; --check recomputes it and
# a mismatch is DRIFT, so editing a verifier is a request for --emit.
# host is the absolute host root the blob reads (AST, non-docstring strings), or
# empty. A PASS carrying one prints as PASS* and is counted separately.
# seconds is wall clock under that file's window (VERIFIER_WINDOWS.tsv); a
# TIMEOUT note names the window it got.
"""
def read_ledger(path):
rows, meta = [], {}
p = Path(path)
if not p.exists():
return rows, meta, "no ledger at %s (run --emit)" % p
for line in p.read_text(encoding="utf-8").splitlines():
if not line.strip():
continue
if line.startswith("#"):
m = re.match(r"#\s*(rev|emitted_at):\s*(\S+)", line)
if m:
meta[m.group(1)] = m.group(2)
continue
parts = line.split("\t")
if len(parts) < 6:
return rows, meta, "ledger row has %d column(s): %r" % (len(parts), line)
while len(parts) < 8:
parts.append("")
rows.append({"invocation": parts[0], "path": parts[1], "verdict": parts[2],
"rc": parts[3], "seconds": parts[4], "note": parts[5],
"blob": parts[6], "host": parts[7]})
return rows, meta, ""
def write_ledger(path, rows, repo, rev):
rc, out, _ = git(repo, "rev-parse", "--short", rev)
sha = out.strip() or rev
body = [HEADER.rstrip("\n"),
"# rev: %s" % sha,
"# emitted_at: %sZ" % time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime())]
for r in sorted(rows, key=lambda x: x["path"]):
body.append("\t".join([r["invocation"], r["path"], r["verdict"],
"" if r["rc"] is None else str(r["rc"]),
"%.2f" % float(r["seconds"]), r["note"],
r["blob"], r["host"]]))
Path(path).write_text("\n".join(body) + "\n", encoding="utf-8")
def rc_consistent(row):
"""Wording a verdict has to match. Returns a reason string or ''."""
v, rc = row["verdict"], row["rc"]
if v not in VERDICTS:
return "verdict %r is not one of %s" % (v, "/".join(VERDICTS))
if v in (TIMEOUT, ERROR, MISSING):
return "" if rc == "" else "%s carries rc %s" % (v, rc)
if rc == "" or not re.fullmatch(r"-?\d+", rc):
return "%s carries rc %r" % (v, rc)
n = int(rc)
if v == PASS and n != 0:
return "PASS with rc %d" % n
if v == FAIL and n != 1:
return "FAIL with rc %d" % n
if v == REFUSED and n != 2:
return "REFUSED with rc %d" % n
if v == OTHER_RC and n <= 2:
return "OTHER_RC with rc %d" % n
return ""
# ---------------------------------------------------------------- invocation
DOC_SUFFIXES = (".md", ".txt", ".log", ".tsv", ".json", ".pyc")
RUN_TOKENS = ("python3", "python ", "sys.executable", "subprocess", "os.system",
"pytest", "bash ")
def invocation_census(repo, rev, paths, exclude_rel):
"""{path: (class, evidence)} -- who runs this file, from committed content.
A mention is not an invocation. A reference counts as a RUN only when it sits
in an executable file (.py/.sh/CHECK*, not a document) on a line that also
names an interpreter or a spawner; every other reference is prose. That is
the distinction reply 21 asked for, and it is a rule with a test: --selftest
holds one prose mention and one `python3 x.py` line and requires the
classifier to separate them.
"""
rc, out, _ = git(repo, "ls-tree", "-r", "--name-only", rev)
if rc != 0:
raise SystemExit("git ls-tree failed at %s" % rev)
files = [f.strip() for f in out.splitlines() if f.strip()]
texts = {}
for f in files:
base = os.path.basename(f)
if base.endswith(DOC_SUFFIXES):
continue
rc, blob, _ = git(repo, "show", "%s:%s" % (rev, f))
if rc == 0:
texts[f] = blob
res = {}
for path in paths:
name = os.path.basename(path)
runs = []
named = 0
for f, blob in texts.items():
if f == path or f == exclude_rel:
continue
if name not in blob:
continue
for i, line in enumerate(blob.splitlines(), 1):
if name not in line:
continue
if line.lstrip().startswith(("#", "//", "*", "--")):
named += 1
continue
if any(t in line for t in RUN_TOKENS):
runs.append("%s:%d" % (f, i))
else:
named += 1
if runs:
res[path] = ("also-run", ", ".join(runs[:2]))
elif named:
res[path] = ("gate", "%d prose ref(s), no invocation" % named)
else:
res[path] = ("gate", "named by nothing else at all")
return res
def check(repo, rev, ledger_path, strict, quiet=False):
try:
pop = population(repo, rev)
except SystemExit as exc:
print("verifier_gate: %s" % exc)
return 2
if not pop:
print("verifier_gate: population is empty at %s -- could not grade" % rev)
return 2
rows, meta, err = read_ledger(ledger_path)
if err:
print("verifier_gate: %s" % err)
return 2
faults, notes = [], []
seen = {}
for r in rows:
seen[r["path"]] = seen.get(r["path"], 0) + 1
why = rc_consistent(r)
if why:
faults.append("INCONSISTENT %s: %s" % (r["path"], why))
if r["verdict"] != PASS and not r["note"].strip():
faults.append("CAUSELESS %s: %s row with no reason in the note"
% (r["path"], r["verdict"]))
if not r["blob"]:
faults.append("UNBOUND %s: no blob column -- the row does not say "
"which content its rc describes (re-emit)" % r["path"])
for p, n in sorted(seen.items()):
if n > 1:
faults.append("DUPLICATE %s: %d rows" % (p, n))
for r in rows:
if r["path"] not in pop:
continue
now = blob_of(repo, rev, r["path"])
if r["blob"] and now and now != r["blob"]:
faults.append("DRIFT %s: rc measured against blob %s, tree at %s has %s"
% (r["path"], r["blob"][:10], rev, now[:10]))
# A dirty worktree is not a commit, so it cannot be DRIFT -- but a verifier
# whose working bytes are not the blob its rc describes is exactly the case
# where the ledger's verdict is about content you are not running. Named,
# not fatal: the fix is a commit (then DRIFT/UNGRADED decide) or --emit.
for r in rows:
if r["path"] not in pop or not r["blob"]:
continue
rc_w, out_w, _ = git(repo, "hash-object", "--", r["path"])
if rc_w == 0 and out_w.strip() and out_w.strip() != r["blob"]:
notes.append("UNCOMMITTED %s: worktree bytes are not the blob its rc "
"was measured against -- commit it, then re-emit or "
"expect DRIFT" % r["path"])
ungraded = sorted(set(pop) - set(seen))
for p in ungraded:
faults.append("UNGRADED %s: committed at %s, no ledger row" % (p, rev))
stale = sorted(p for p in set(seen) - set(pop)
if not (Path(repo) / p).exists())
for p in stale:
faults.append("STALE %s: ledger row, file not in the tree" % p)
outside = sorted(p for p in set(seen) - set(pop) if (Path(repo) / p).exists())
for p in outside:
notes.append("OUTSIDE %s: ledger row, not a committed verifier at %s "
"(untracked beside the lane or renamed)" % (p, rev))
counts = {}
for r in rows:
counts[r["verdict"]] = counts.get(r["verdict"], 0) + 1
pinned = [r for r in rows if r["verdict"] == PASS and r["host"]]
reds = [r for r in rows if r["verdict"] != PASS]
clean = len(rows) - len(reds) - len(pinned)
print("verifier_gate -- lane %s at %s, %d committed verifier(s), %d ledger row(s)"
% (LANE, rev, len(pop), len(rows)))
if meta.get("rev"):
print(" ledger rev : %s emitted %s"
% (meta["rev"], meta.get("emitted_at", "?")))
print(" rows : %s" % ", ".join("%s %d" % (v, counts[v])
for v in VERDICTS if counts.get(v)))
print(" graded-clean : %d of %d (PASS on a host-independent blob)"
% (clean, len(pop)))
if pinned:
print(" host-pinned : %d PASS row(s) read as PASS* -- they read an "
"absolute host root" % len(pinned))
for r in pinned:
print(" PASS* %s %s" % (r["path"], r["host"]))
if reds:
print(" known-red : %d of %d row(s) -- every one named, every one "
"with its cause" % (len(reds), len(rows)))
for r in sorted(reds, key=lambda x: x["path"]):
print(" %-8s rc %-3s %s %s" % (r["verdict"], r["rc"] or "-",
r["path"], r["note"][:100]))
for n in notes:
print(" note: " + n)
if strict:
for r in reds:
faults.append("%s %s rc %s %s" % (r["verdict"], r["path"], r["rc"] or "-",
r["note"][:80]))
for f in faults:
print(" " + f)
print("verifier_gate: %d fault(s)%s" % (len(faults),
" (strict: every non-PASS row counts)"
if strict else ""))
return 1 if faults else 0
def emit(repo, rev, ledger_path, timeout, windows, only=None, merge=False):
pop = population(repo, rev)
if only:
pop = [p for p in pop if only in p]
if not pop:
print("verifier_gate: nothing to emit")
return 2
keep = []
if merge and Path(ledger_path).exists():
have, _, err = read_ledger(ledger_path)
if err:
print("verifier_gate: %s" % err)
return 2
keep = [r for r in have if r["path"] not in set(pop)]
rc, before, _ = git(repo, "status", "--porcelain")
t0 = time.time()
rows = list(keep)
for p in pop:
r = run_one(repo, p, timeout, windows)
r["blob"] = blob_of(repo, rev, p)
r["host"] = host_pin(repo, rev, p)
r["invocation"] = "gate"
rows.append(r)
print(" %-8s %6.2fs %s" % (r["verdict"], r["seconds"], p), flush=True)
census = {}
try:
census = invocation_census(repo, rev, [r["path"] for r in rows],
os.path.relpath(ledger_path, repo))
except SystemExit as exc:
print("verifier_gate: census skipped (%s)" % exc)
for r in rows:
cls, ev = census.get(r["path"], ("gate", ""))
r["invocation"] = "also-run" if cls == "also-run" else "gate"
write_ledger(ledger_path, rows, repo, rev)
rc2, after, _ = git(repo, "status", "--porcelain")
led_rel = os.path.relpath(str(ledger_path), str(repo))
dirt = sorted(d for d in set(after.splitlines()) - set(before.splitlines())
if led_rel not in d)
counts = {}
for r in rows:
counts[r["verdict"]] = counts.get(r["verdict"], 0) + 1
print("verifier_gate: %d row(s) in %.1fs -> %s"
% (len(rows), time.time() - t0, ledger_path))
print(" " + ", ".join("%s %d" % (v, counts[v]) for v in VERDICTS
if counts.get(v)))
if dirt:
print(" WARNING: the run changed the worktree (these verifiers write "
"into it):")
for d in dirt:
print(" " + d)
return 0
def census_report(repo, rev, ledger_path):
pop = population(repo, rev)
res = invocation_census(repo, rev, pop, os.path.relpath(str(ledger_path), str(repo)))
run = sorted(p for p, (c, _) in res.items() if c == "also-run")
print("verifier_census -- %d committed verifier(s) in %s at %s"
% (len(pop), LANE, rev))
print(" invoked by another committed line : %d" % len(run))
for p in run:
print(" %s %s" % (p, res[p][1]))
print(" invoked by this gate only : %d" % (len(pop) - len(run)))
for p in pop:
if p in run:
continue
print(" %s %s" % (p, res[p][1]))
return 0
# ------------------------------------------------------------------- selftest
def selftest():
"""Cells for the rules that can silently pass. Every cell builds a throwaway
git repo, so the gate's logic is graded on a tree it did not ship with."""
cells = []
def cell(name, fn):
try:
ok, detail = fn()
except Exception as exc: # noqa: BLE001 - a cell that raises is a cell that failed
ok, detail = False, "%s: %s" % (type(exc).__name__, exc)
cells.append((name, ok, detail))
def mk(tmp, files, commit=True):
repo = Path(tmp)
(repo / LANE).mkdir(parents=True, exist_ok=True)
for rel, body in files.items():
p = repo / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(body, encoding="utf-8")
git(repo, "init", "-q")
git(repo, "config", "user.email", "t@example.invalid")
git(repo, "config", "user.name", "t")
if commit:
git(repo, "add", "-A")
git(repo, "commit", "-qm", "cell")
return repo
V = LANE + "/verify_alpha.py"
def c1():
bad = {"path": V, "verdict": PASS, "rc": "1", "note": "", "blob": "x",
"host": "", "seconds": "0", "invocation": "gate"}
good = dict(bad, verdict=FAIL)
a, b = rc_consistent(bad), rc_consistent(good)
return (bool(a) and not b, "PASS with rc 1")
def c2():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
n1 = len(population(repo, "HEAD"))
(repo / LANE / "verify_beta.py").write_text("print(0)\n")
git(repo, "add", "-A")
git(repo, "commit", "-qm", "two")
n2 = len(population(repo, "HEAD"))
return (n1 == 1 and n2 == 2, "population follows the rev")
def c3():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
led = repo / LANE / "L.tsv"
write_ledger(led, [{"path": V, "verdict": PASS, "rc": 0, "seconds": 0.0,
"note": "", "blob": blob_of(repo, "HEAD", V),
"host": "", "invocation": "gate"}], repo, "HEAD")
r0 = check(repo, "HEAD", led, False)
(repo / V).write_text("print(1)\n")
git(repo, "add", "-A")
git(repo, "commit", "-qm", "edit")
r1 = check(repo, "HEAD", led, False)
return (r0 == 0 and r1 == 1, "a drifted blob is a fault")
def c4():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
led = repo / LANE / "L.tsv"
write_ledger(led, [], repo, "HEAD")
return (check(repo, "HEAD", led, False) == 1, "a committed verifier with no row faults")
def c5():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
led = repo / LANE / "L.tsv"
write_ledger(led, [{"path": LANE + "/verify_gone.py", "verdict": PASS,
"rc": 0, "seconds": 0.0, "note": "", "blob": "abc",
"host": "", "invocation": "gate"}], repo, "HEAD")
return (check(repo, "HEAD", led, False) == 1, "a row for a deleted file faults")
def c6():
with tempfile.TemporaryDirectory() as tmp:
body = ('"""Reads /root/troupe-standup in a docstring."""\n'
'P = "/root/troupe-standup/inbox"\n')
repo = mk(tmp, {V: body})
hit = host_pin(repo, "HEAD", V)
repo2 = mk(tmp + "/b", {V: '"""Reads /root/troupe-standup."""\n'})
miss = host_pin(repo2, "HEAD", V)
return (hit != "" and miss == "", "host pin is AST, not prose")
def c7():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n",
"CHECK": "python3 %s\n" % V,
"notes/x.md": "see %s\n" % V})
res = invocation_census(repo, "HEAD", [V], "")
return (res[V][0] == "also-run", "prose is not an invocation")
def c8():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "import sys\nsys.exit(1)\n",
LANE + "/verify_ok.py": "print(0)\n"})
led = repo / LANE / "L.tsv"
rc = emit(repo, "HEAD", led, 30.0, {}, only=None)
rows, _, err = read_ledger(led)
by = {r["path"]: r for r in rows}
ok = (rc == 0 and not err and by[V]["verdict"] == FAIL
and by[V]["note"] != "" and by[LANE + "/verify_ok.py"]["verdict"] == PASS)
return (ok, "emit records rc and a cause")
def c9():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
led = repo / LANE / "L.tsv"
write_ledger(led, [{"path": V, "verdict": FAIL, "rc": 1, "seconds": 0.0,
"note": "", "blob": blob_of(repo, "HEAD", V),
"host": "", "invocation": "gate"}], repo, "HEAD")
return (check(repo, "HEAD", led, False) == 1, "a red with no cause is a fault")
def c10():
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
led = repo / LANE / "L.tsv"
blob = blob_of(repo, "HEAD", V)
write_ledger(led, [{"path": V, "verdict": FAIL, "rc": 1, "seconds": 0.0,
"note": "known red", "blob": blob, "host": "",
"invocation": "gate"}], repo, "HEAD")
loose = check(repo, "HEAD", led, False)
strict = check(repo, "HEAD", led, True)
return (loose == 0 and strict == 1, "strict promotes a red row")
def c11():
import contextlib
import io
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: "print(0)\n"})
led = repo / LANE / "L.tsv"
write_ledger(led, [{"path": V, "verdict": PASS, "rc": 0, "seconds": 0.0,
"note": "", "blob": blob_of(repo, "HEAD", V),
"host": "", "invocation": "gate"}], repo, "HEAD")
(repo / V).write_text("print(0) # uncommitted edit\n")
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
rc = check(repo, "HEAD", led, False)
out = buf.getvalue()
return (rc == 0 and "UNCOMMITTED" in out,
"a dirty worktree is named, not fatal")
def c12():
body = ("import sys\n"
"sys.stderr.write('warning: bad escape \\\\u\\n')\n"
"print('VERDICT: MISMATCH -- behavior NOT preserved')\n"
"sys.exit(1)\n")
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: body})
led = repo / LANE / "L.tsv"
emit(repo, "HEAD", led, 30.0, {}, only=None)
rows, _, err = read_ledger(led)
by = {r["path"]: r for r in rows}
note = by[V]["note"] if V in by else ""
return (not err and "VERDICT: MISMATCH" in note,
"the note is the file's verdict line: %r" % note[:60])
def c13():
body = ("import os, shutil, tempfile\n"
"sandbox = tempfile.mkdtemp(prefix='x_', dir=os.environ['TMPDIR'])\n"
"shutil.rmtree(os.path.dirname(sandbox), ignore_errors=True)\n"
"print('cleaned up my parent')\n")
with tempfile.TemporaryDirectory() as tmp:
repo = mk(tmp, {V: body})
led = repo / LANE / "L.tsv"
survive = tempfile.mkdtemp(prefix="survive_")
emit(repo, "HEAD", led, 30.0, {}, only=None)
rows, _, err = read_ledger(led)
by = {r["path"]: r for r in rows}
alive = os.path.isdir(survive)
shutil.rmtree(survive, ignore_errors=True)
return (not err and by[V]["verdict"] == PASS and alive,
"a verifier that deletes its TMPDIR leaves the caller's alone")
# the cell fixture needs a caller-level temp dir to watch
for name, fn in (("c1 verdict/rc pairing", c1),
("c2 population from the rev", c2),
("c3 blob drift", c3),
("c4 ungraded file", c4),
("c5 stale row", c5),
("c6 host-pin AST", c6),
("c7 named-vs-run", c7),
("c8 emit records", c8),
("c9 causeless red", c9),
("c10 strict", c10),
("c11 uncommitted worktree", c11),
("c12 note is the verdict line", c12),
("c13 sandboxed TMPDIR", c13)):
cell(name, fn)
bad = [c for c in cells if not c[1]]
for name, ok, detail in cells:
print(" [%s] %s -- %s" % ("ok " if ok else "FAIL", name, detail))
print("verifier_gate selftest: %d cells, %d passed, %d failed"
% (len(cells), len(cells) - len(bad), len(bad)))
return 0 if not bad else 1
def main():
ap = argparse.ArgumentParser(add_help=True)
ap.add_argument("--repo", default=str(ROOT))
ap.add_argument("--rev", default="HEAD")
ap.add_argument("--ledger", default=str(LEDGER))
ap.add_argument("--windows", default=str(WINDOWS))
ap.add_argument("--timeout", type=float, default=90.0)
ap.add_argument("--check", action="store_true")
ap.add_argument("--emit", action="store_true")
ap.add_argument("--census", action="store_true")
ap.add_argument("--strict", action="store_true")
ap.add_argument("--only")
ap.add_argument("--merge", action="store_true",
help="with --emit: keep the rows this run did not re-measure")
ap.add_argument("--selftest", action="store_true")
args = ap.parse_args()
if args.selftest:
return selftest()
repo = Path(args.repo).resolve()
if not (repo / ".git").exists():
print("verifier_gate: %s is not a git worktree -- could not grade" % repo)
return 2
ledger = Path(args.ledger)
if not ledger.is_absolute():
ledger = repo / ledger
if args.emit:
try:
win = read_windows(args.windows)
except SystemExit as exc:
print("verifier_gate: %s" % exc)
return 2
return emit(repo, args.rev, ledger, args.timeout, win, only=args.only,
merge=args.merge)
if args.census:
return census_report(repo, args.rev, ledger)
return check(repo, args.rev, ledger, args.strict)
if __name__ == "__main__":
sys.exit(main())