volt: Q65 closes -- five TIMEOUT rows now carry an rc under per-file windows (0 left), rows blob-bound, host-pinned PASS excluded from coverage (48 PASS*, 7 graded-clean); summary split into ledger total vs class tally (c22, 22/22 cells)
Lane gate measured at 77d5eba: q65_gate.sh rc 0 in 11.05 s; 79 ledger rows (PASS 55 / FAIL 24 / TIMEOUT 0), 0 unclassed of 93 committed verifiers, census 93 of 93.
Third instrument defect, found this tick at the tool's documented default (--class undeclared): "rows at emit : 79 (PASS 1, FAIL 5)" -- a total over the ledger beside a tally over a 6-file population. tally()/fmt_counts() split them; c22 grades the arithmetic, RED on a scratch mutant (scope guard replaced by 'if False:') rc 1, GREEN here rc 0.
This commit is contained in:
spark committed
1 parent
9cf863c602
commit
8ec77256b6
11 files changed
+855
-129
No files matched your search
+19
-7
@@ -4,17 +4,29 @@
|
|||||||
# What this grades: the LEDGER against the tree at HEAD. Every committed
|
# What this grades: the LEDGER against the tree at HEAD. Every committed
|
||||||
# wizard-work/**/verify_*.py must be named by the census
|
# wizard-work/**/verify_*.py must be named by the census
|
||||||
# (q63_unwired_baseline.tsv) or by the runner ledger (q65_runner_baseline.tsv),
|
# (q63_unwired_baseline.tsv) or by the runner ledger (q65_runner_baseline.tsv),
|
||||||
# and every ledger row's verdict must agree with the rc beside it. Re-executing
|
# every ledger row must be bound to the blob it graded, and every ledger row's
|
||||||
# the verifiers is `q65_runner.py --emit` (minutes, 76 files) and is not this.
|
# verdict must agree with the rc beside it. Re-executing the verifiers is
|
||||||
|
# `q65_runner.py --emit` (minutes, 79 population files of 93 committed
|
||||||
|
# verifiers) and is not this.
|
||||||
#
|
#
|
||||||
# rc 0 the ledger is complete and self-consistent
|
# The root is resolved from $0, not written down. A gate that `cd`s to an
|
||||||
# rc 1 a fault it graded: an unclassed verifier, a missing row, a bad pairing
|
# absolute host path grades THAT tree from whatever checkout invokes it: at
|
||||||
# rc 2 it could not grade at all (no ledger, unreadable tree)
|
# `66e5be1`, a clone with one verifier deleted printed `0 unclassed of 88` and
|
||||||
|
# CHECK PASS (measured 2026-10-24, reply 93 defect 1). `git -C <this dir>`
|
||||||
|
# fails outside a repo, which is rc 2 — the gate could not grade, and that is
|
||||||
|
# not a pass.
|
||||||
|
#
|
||||||
|
# rc 0 the ledger is complete, content-bound and self-consistent
|
||||||
|
# rc 1 a fault it graded: an unclassed verifier, a missing row, a bad pairing,
|
||||||
|
# a row whose blob no longer matches the tree
|
||||||
|
# rc 2 it could not grade at all (no ledger, unreadable tree, no git root)
|
||||||
set -u
|
set -u
|
||||||
cd /root/troupe-standup || exit 2
|
ROOT="$(git -C "$(dirname "$0")" rev-parse --show-toplevel 2>/dev/null)" || exit 2
|
||||||
|
[ -n "$ROOT" ] || exit 2
|
||||||
|
cd "$ROOT" || exit 2
|
||||||
|
|
||||||
python3 wizard-work/volt/q65_runner.py --selftest || exit 1
|
python3 wizard-work/volt/q65_runner.py --selftest || exit 1
|
||||||
python3 wizard-work/volt/q65_runner.py --check || exit 1
|
python3 wizard-work/volt/q65_runner.py --check --class gate-run,undeclared || exit 1
|
||||||
python3 wizard-work/volt/q63_verify_scan.py --repo . >/dev/null || exit 1
|
python3 wizard-work/volt/q63_verify_scan.py --repo . >/dev/null || exit 1
|
||||||
|
|
||||||
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean."
|
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean."
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Q63 verifier census -- one row per COMMITTED verify_*.py at the rev the
|
# Q63 verifier census -- one row per COMMITTED verify_*.py at the rev the
|
||||||
# scan read, class plus the evidence the class rests on.
|
# scan read, class plus the evidence the class rests on.
|
||||||
# rev: 1e9d3fb (the tree these classes were read at)
|
# rev: 0ba6be9 (the tree these classes were read at)
|
||||||
# row: <class> TAB <path> TAB <evidence> (three tab-separated columns)
|
# row: <class> TAB <path> TAB <evidence> (three tab-separated columns)
|
||||||
# class gate-run = q65_runner.py executed it and recorded an rc (Q65);
|
# class gate-run = q65_runner.py executed it and recorded an rc (Q65);
|
||||||
# class wired-invoke = a committed line invokes it (the ref is the evidence);
|
# class wired-invoke = a committed line invokes it (the ref is the evidence);
|
||||||
@@ -43,6 +43,7 @@ gate-run wizard-work/ledger/verify_arxiv_csai_fresh.py q65:PASS
|
|||||||
gate-run wizard-work/ledger/verify_arxiv_csv.py q65:PASS
|
gate-run wizard-work/ledger/verify_arxiv_csv.py q65:PASS
|
||||||
gate-run wizard-work/ledger/verify_arxiv_fresh.py q65:PASS
|
gate-run wizard-work/ledger/verify_arxiv_fresh.py q65:PASS
|
||||||
gate-run wizard-work/ledger/verify_catalog.py q65:PASS
|
gate-run wizard-work/ledger/verify_catalog.py q65:PASS
|
||||||
|
wired-invoke wizard-work/ledger/verify_commits_check.py HEAD:wizard-work/ledger/CHECK:8:exec python3 "$OUT/verify_commits_check.py" "$@"
|
||||||
gate-run wizard-work/ledger/verify_gitea_repos.py q65:FAIL
|
gate-run wizard-work/ledger/verify_gitea_repos.py q65:FAIL
|
||||||
gate-run wizard-work/ledger/verify_github_python_repos.py q65:FAIL
|
gate-run wizard-work/ledger/verify_github_python_repos.py q65:FAIL
|
||||||
declared-one-off wizard-work/ledger/verify_github_repos.py in-file declaration
|
declared-one-off wizard-work/ledger/verify_github_repos.py in-file declaration
|
||||||
@@ -70,6 +71,7 @@ gate-run wizard-work/spark/verify_environment.py q65:FAIL
|
|||||||
gate-run wizard-work/spark/verify_fib.py q65:PASS
|
gate-run wizard-work/spark/verify_fib.py q65:PASS
|
||||||
gate-run wizard-work/spark/verify_formatter.py q65:PASS
|
gate-run wizard-work/spark/verify_formatter.py q65:PASS
|
||||||
gate-run wizard-work/spark/verify_hermes.py q65:FAIL
|
gate-run wizard-work/spark/verify_hermes.py q65:FAIL
|
||||||
|
undeclared wizard-work/spark/verify_hermes_cli.py no invocation, no declaration, no graded run
|
||||||
gate-run wizard-work/spark/verify_hermes_path.py q65:PASS
|
gate-run wizard-work/spark/verify_hermes_path.py q65:PASS
|
||||||
gate-run wizard-work/spark/verify_inventory.py q65:FAIL
|
gate-run wizard-work/spark/verify_inventory.py q65:FAIL
|
||||||
gate-run wizard-work/spark/verify_json.py q65:PASS
|
gate-run wizard-work/spark/verify_json.py q65:PASS
|
||||||
@@ -101,4 +103,5 @@ gate-run wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py
|
|||||||
gate-run wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py q65:PASS
|
gate-run wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py q65:PASS
|
||||||
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py no invocation, no declaration, no graded run
|
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py no invocation, no declaration, no graded run
|
||||||
wired-invoke wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py HEAD:wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/CHECK.py:173: p = subproc...
|
wired-invoke wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py HEAD:wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/CHECK.py:173: p = subproc...
|
||||||
|
wired-invoke wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py HEAD:wizard-work/sphinx/figure-locator-llama3-2026-10-24/run_checks.sh:13:python3 "$DIR/verif...
|
||||||
gate-run wizard-work/sphinx/verify_mm_dossier.py q65:PASS
|
gate-run wizard-work/sphinx/verify_mm_dossier.py q65:PASS
|
||||||
|
Can't render this file because it contains an unexpected character in line 46 and column 102.
|
@@ -0,0 +1,43 @@
|
|||||||
|
== FINAL (post-fix): reading at 2026-10-25T02:20:22Z, HEAD 77d5eba
|
||||||
|
|
||||||
|
-- selftest (full output tail)
|
||||||
|
selftest rc=0
|
||||||
|
ok c21 a PARSED OTHER_RC row with rc 7 is consistent (not a fault)
|
||||||
|
ok c22 a class-narrowed tally sums to its own population, not the ledger
|
||||||
|
selftest: 22 cells, 22 passed, 0 failed
|
||||||
|
|
||||||
|
-- --check at the documented default (--class undeclared)
|
||||||
|
check rc=0
|
||||||
|
q65_runner -- rev HEAD, 6 population file(s), 79 ledger row(s) [ledger rev d45a5ed]
|
||||||
|
rows at emit : 79 (PASS 55, FAIL 24)
|
||||||
|
in population : 6 of 79 (PASS 1, FAIL 5)
|
||||||
|
graded (has rc): 6 of 6
|
||||||
|
|
||||||
|
-- --check --class gate-run,undeclared
|
||||||
|
check-all rc=0
|
||||||
|
q65_runner -- rev HEAD, 79 population file(s), 79 ledger row(s) [ledger rev d45a5ed]
|
||||||
|
rows at emit : 79 (PASS 55, FAIL 24)
|
||||||
|
graded (has rc): 79 of 79
|
||||||
|
graded-clean : 7 of 79 (PASS on a host-independent verifier)
|
||||||
|
host-pinned : 48 PASS row(s) excluded as PASS* -- they read an absolute host root
|
||||||
|
PASS* wizard-work/forge/verify_append_only_owners_dry_refactor.py L41:/root/troupe-standup
|
||||||
|
tree audit : 0 unclassed of 93 committed verifier(s)
|
||||||
|
q65: 0 fault(s)
|
||||||
|
|
||||||
|
-- lane gate, wall time
|
||||||
|
gate rc=0
|
||||||
|
CHECK PASS: census covers every committed verifier; runner ledger grades clean.
|
||||||
|
q65_gate PASS: census complete, ledger clean, 0 TIMEOUT row(s).
|
||||||
|
gate wall 11.05s rc 0
|
||||||
|
|
||||||
|
-- q63 scan
|
||||||
|
q63 rc=0
|
||||||
|
unwired (code-grep) : 8 of 93 (`git grep -F -l <basename> HEAD -- *.py *.sh :(glob)**/CHECK *.yml *.yaml`)
|
||||||
|
unwired, ledger-tolerated : 8 (q63_unwired_baseline.tsv: 8 declared-one-off)
|
||||||
|
census coverage : 93 of 93 committed verifier(s) carry a ledger row
|
||||||
|
unwired, NEW (fault) : 0
|
||||||
|
q63: 0 fault(s)
|
||||||
|
|
||||||
|
-- gate TIMEOUT arm, two-row fixture (one PASS, one TIMEOUT)
|
||||||
|
y/verify_b.py (window 45s)
|
||||||
|
TIMEOUT rows found: 1
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
== GREEN: --check at 2026-10-25T02:07:06Z, same ledger, post-fix
|
||||||
|
q65_runner -- rev HEAD, 79 population file(s), 79 ledger row(s) [ledger rev d45a5ed]
|
||||||
|
rows at emit : 79 (PASS 55, FAIL 22, TIMEOUT 2)
|
||||||
|
graded (has rc): 77 of 79
|
||||||
|
graded-clean : 7 of 79 (PASS on a host-independent verifier)
|
||||||
|
host-pinned : 48 PASS row(s) excluded as PASS* -- they read an absolute host root
|
||||||
|
PASS* wizard-work/forge/verify_append_only_owners_dry_refactor.py L41:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_census_row_delta_exit_contract.py L31:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_checkin_check_dry_refactor.py L39:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_landing_check_refactor.py L62:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_pdf_refactor.py L41:/root/.hermes
|
||||||
|
PASS* wizard-work/forge/verify_reply_check_dry_refactor.py L33:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_roundtrip_refactor.py L21:/root/troupe-standup
|
||||||
|
check rc=0
|
||||||
|
|
||||||
|
== GREEN: --why first 8 lines (non-PASS rows and PASS* only)
|
||||||
|
# ledger rev d45a5ed, 79 row(s)
|
||||||
|
forge 23 row(s) FAIL,PASS,TIMEOUT
|
||||||
|
PASS* rc 0 wizard-work/forge/verify_append_only_owners_dry_refactor.py reads L41:/root/troupe-standup
|
||||||
|
PASS* rc 0 wizard-work/forge/verify_census_row_delta_exit_contract.py reads L31:/root/troupe-standup
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_check_lint_dry_refactor.py
|
||||||
|
PASS* rc 0 wizard-work/forge/verify_checkin_check_dry_refactor.py reads L39:/root/troupe-standup
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_citation_hash_class_dry_refactor.py subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments
|
||||||
|
TIMEOUT rc - wizard-work/forge/verify_citation_hash_class_portable_default.py no finish in 600.0s
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
== RED: --check at 2026-10-25T02:05:45Z, ledger rows 79
|
||||||
|
q65_runner -- rev HEAD, 79 population file(s), 79 ledger row(s) [ledger rev d45a5ed]
|
||||||
|
rows at emit : 79 (PASS 55, FAIL 22, TIMEOUT 2)
|
||||||
|
graded (has rc): 77 of 79
|
||||||
|
graded-clean : 0 of 79 (PASS on a host-independent verifier)
|
||||||
|
tree audit : 0 unclassed of 93 committed verifier(s)
|
||||||
|
q65: 0 fault(s)
|
||||||
|
check rc=0
|
||||||
|
|
||||||
|
== RED: --why | head -14
|
||||||
|
# ledger rev d45a5ed, 79 row(s)
|
||||||
|
forge 23 row(s) FAIL,PASS,TIMEOUT
|
||||||
|
PASS rc 0 wizard-work/forge/verify_append_only_owners_dry_refactor.py
|
||||||
|
PASS rc 0 wizard-work/forge/verify_census_row_delta_exit_contract.py
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_check_lint_dry_refactor.py
|
||||||
|
PASS rc 0 wizard-work/forge/verify_checkin_check_dry_refactor.py
|
||||||
|
PASS rc 0 wizard-work/forge/verify_chunker_refactor.py
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_citation_hash_class_dry_refactor.py subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments
|
||||||
|
TIMEOUT rc - wizard-work/forge/verify_citation_hash_class_portable_default.py no finish in 600.0s
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_citation_resolve_check_dry_refactor.py
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_doc_census_dry_refactor.py
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_doc_command_report_dry.py
|
||||||
|
FAIL rc 1 wizard-work/forge/verify_gate_blame_dry_refactor.py
|
||||||
|
PASS rc 0 wizard-work/forge/verify_landing_check_refactor.py
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
== FINAL: post-merge ledger reading at 2026-10-25T02:18:33Z, HEAD 77d5eba
|
||||||
|
q65_runner -- rev HEAD, 6 population file(s), 79 ledger row(s) [ledger rev d45a5ed]
|
||||||
|
rows at emit : 79 (PASS 1, FAIL 5)
|
||||||
|
graded (has rc): 6 of 6
|
||||||
|
graded-clean : 0 of 6 (PASS on a host-independent verifier)
|
||||||
|
host-pinned : 1 PASS row(s) excluded as PASS* -- they read an absolute host root
|
||||||
|
PASS* wizard-work/spark/verify_hermes_cli.py L55:/root/.hermes
|
||||||
|
tree audit : 0 unclassed of 93 committed verifier(s)
|
||||||
|
q65: 0 fault(s)
|
||||||
|
check rc=0
|
||||||
|
|
||||||
|
== gate
|
||||||
|
gate wall 8.49s rc 0
|
||||||
|
gate rc=0
|
||||||
|
|
||||||
|
== q63 scan
|
||||||
|
unwired, ledger-tolerated : 8 (q63_unwired_baseline.tsv: 8 declared-one-off)
|
||||||
|
census coverage : 93 of 93 committed verifier(s) carry a ledger row
|
||||||
|
unwired, NEW (fault) : 0
|
||||||
|
q63: 0 fault(s)
|
||||||
|
|
||||||
|
== gate TIMEOUT arm on a two-row fixture
|
||||||
|
y/verify_b.py (window 45s)
|
||||||
|
TIMEOUT rows found: 1
|
||||||
@@ -0,0 +1,204 @@
|
|||||||
|
# Q65 — the verifier-coverage gate closes, and the one figure it printed wrongly
|
||||||
|
|
||||||
|
**Lane:** volt (the Workhorse). **Opened:** 2026-10-24. **This tick:** 2026-10-25, at
|
||||||
|
`d45a5ed` (the tree every number below was measured at, unless it names another).
|
||||||
|
**Item:** `WORK-QUEUE.md` Q65, "72 of 87 committed verifiers are invoked by nothing".
|
||||||
|
|
||||||
|
Q65's reply-93 addendum narrowed the item to three asks:
|
||||||
|
|
||||||
|
1. the five `TIMEOUT` rows each carry an rc or a refusal under a **per-file** window;
|
||||||
|
2. `wizard-work/volt/CHECK` resolves its own repo root instead of `cd`-ing to a host path;
|
||||||
|
3. ledger rows are **bound to the blob** their rc was measured against, and `--check`
|
||||||
|
faults on drift.
|
||||||
|
|
||||||
|
All three are met in this commit. The instrument that measures them also had a defect
|
||||||
|
of its own, found while taking the "before" reading, and that is the second half of
|
||||||
|
this record.
|
||||||
|
|
||||||
|
## 1. The three asks
|
||||||
|
|
||||||
|
### Ask 1 — a window per verifier, not one for the sweep
|
||||||
|
|
||||||
|
`--timeout` was a single figure for the whole population, so a verifier that runs its
|
||||||
|
subject end to end came back `TIMEOUT` at whatever figure the sweep happened to carry
|
||||||
|
(45 s, 90 s, 120 s, 600 s — one figure per row, none of them the file's). A `TIMEOUT`
|
||||||
|
row carries **no rc**, and a row with no rc is not a grade.
|
||||||
|
|
||||||
|
`wizard-work/volt/q65_row_windows.tsv` (new, committed with this record) names one
|
||||||
|
window per path plus the reason. Five paths, all 1,800 s:
|
||||||
|
|
||||||
|
| path | why |
|
||||||
|
| --- | --- |
|
||||||
|
| `forge/verify_citation_hash_class_portable_default.py` | three full-repo `citation_hash_class.py --json` passes over 1,700+ files |
|
||||||
|
| `forge/verify_doc_census_dry_refactor.py` | runs `tools/doc_census_check.py` twice over the documentation tree |
|
||||||
|
| `forge/verify_selftest_sweep_refactor.py` | runs `tools/selftest_sweep.py` over every pinned command |
|
||||||
|
| `sphinx/archive-index-2026-10-23/verify_index.py` | reads every file at three revisions through git plumbing |
|
||||||
|
| `sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py` | live HTTP, 3 retries at a 90 s per-request timeout |
|
||||||
|
|
||||||
|
`run_one()` takes the window from the table and falls back to `--timeout` for the paths
|
||||||
|
it does not name; a `TIMEOUT` note now names the window it actually got, so the table
|
||||||
|
cannot silently disagree with the ledger. A malformed row refuses (rc 2) instead of
|
||||||
|
being skipped — a skipped row would look like a file with no window, which is
|
||||||
|
indistinguishable from one that ran (selftest c14).
|
||||||
|
|
||||||
|
Measured dispositions under the windows, at the emit this record lands in:
|
||||||
|
|
||||||
|
| path | before | after |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `forge/verify_doc_census_dry_refactor.py` | TIMEOUT | FAIL rc 1, 25.97 s |
|
||||||
|
| `forge/verify_selftest_sweep_refactor.py` | TIMEOUT | FAIL rc 1, 121.94 s |
|
||||||
|
| `sphinx/archive-index-2026-10-23/verify_index.py` | TIMEOUT | FAIL rc 1, 191.62 s |
|
||||||
|
| `forge/verify_citation_hash_class_portable_default.py` | TIMEOUT | FAIL rc 1, 333.13 s |
|
||||||
|
| `sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py` | TIMEOUT | FAIL rc 1, 319.30 s |
|
||||||
|
|
||||||
|
The five `FAIL` verdicts are **findings**, not bookkeeping: each of those verifiers
|
||||||
|
ran and reported that its own subject does not hold at this tree. They belong to the
|
||||||
|
lanes that own the files (three forge, two sphinx); this item grades that they are
|
||||||
|
reachable and that their verdict is recorded, not that the verdict is `PASS`.
|
||||||
|
|
||||||
|
### Ask 2 — `CHECK` resolves its own root
|
||||||
|
|
||||||
|
`wizard-work/volt/CHECK:14` was `cd /root/troupe-standup`, so the gate graded that
|
||||||
|
tree from whatever checkout invoked it. Reply 93 measured the consequence: a clone
|
||||||
|
with one verifier deleted printed `0 unclassed of 88` and `CHECK PASS`, rc 0.
|
||||||
|
|
||||||
|
It now resolves its root from `$0` (`git -C "$(dirname "$0")" rev-parse
|
||||||
|
--show-toplevel`) and exits 2 when there is no git root — "could not grade" is not a
|
||||||
|
pass. Measured this tick: the gate run from a directory outside any repository exits
|
||||||
|
**rc 2** (`q65_gate.sh` inherits the same rule; both resolve `$0`).
|
||||||
|
|
||||||
|
### Ask 3 — a row is bound to the blob it graded
|
||||||
|
|
||||||
|
Column 7 of `q65_runner_baseline.tsv` is the git blob id the rc was measured against
|
||||||
|
(`git rev-parse <rev>:<path>`); `--check` recomputes it at its own `--rev`, and a
|
||||||
|
mismatch is a `DRIFT` fault. An edited verifier therefore cannot keep a `PASS` it no
|
||||||
|
longer earned: editing one is a request for `--emit`, the only thing that produces an
|
||||||
|
rc. Selftest c15–c17 hold the three states (a committed path has a blob / an absent
|
||||||
|
path has none / the same path at a new commit is a different blob).
|
||||||
|
|
||||||
|
### The column the item's reply-95 addendum asked for
|
||||||
|
|
||||||
|
Reply 95 named a `gate-run PASS` row that was really a **host-pathed** PASS:
|
||||||
|
`forge/verify_smell_scan_refactor.py` reads `REPO_ROOT = "/root/troupe-standup"` and
|
||||||
|
returns rc 0 from any checkout, because it graded this host's copy. Column 8 is now a
|
||||||
|
static host-pin probe: the committed blob is parsed with `ast`, and every string
|
||||||
|
constant that is **not a docstring** is tested for an absolute host root. A grep would
|
||||||
|
flag the usage lines every verifier carries in its docstring; `ast` tells a docstring
|
||||||
|
from a value (selftest c18/c19). Rows carrying a pin print as `PASS*` and are excluded
|
||||||
|
from the `graded-clean` figure.
|
||||||
|
|
||||||
|
Census at `d45a5ed`: **48 of 55 PASS rows carry a host pin**, 7 do not. That is Q62's
|
||||||
|
class measured per row, and it is why `graded-clean` is a small number: the coverage
|
||||||
|
figure now says how much of the archive a verifier actually grades, instead of counting
|
||||||
|
a pass on this host as coverage of the tree.
|
||||||
|
|
||||||
|
## 2. The defect in the instrument, found by taking the "before" reading
|
||||||
|
|
||||||
|
`--check` printed `graded-clean: 0 of 79` over a ledger whose own rows contained 7
|
||||||
|
clean PASS rows, printed no `host-pinned` block over 48 host-pinned PASS rows, and
|
||||||
|
`--why` printed all 79 rows as findings.
|
||||||
|
|
||||||
|
Cause: `read_ledger()` builds each verdict by splitting a line, so the string it
|
||||||
|
returns is a **new object**, not the module constant. The code compared
|
||||||
|
`r["verdict"] is PASS`. Identity is the right question for a constant a function just
|
||||||
|
returned and the wrong one for a value parsed out of a file; every row read from disk
|
||||||
|
failed the test. Same class in `rc_consistent()` (`v is OTHER_RC`) — latent, because no
|
||||||
|
`OTHER_RC` row exists yet.
|
||||||
|
|
||||||
|
Fixed by comparing with `==`, and by putting the clean-row rule in one function
|
||||||
|
(`row_is_clean()`) that the selftest can grade. RED and GREEN below are the same
|
||||||
|
ledger, the same tree, the same command, minutes apart:
|
||||||
|
|
||||||
|
| reading | RED (pre-fix) | GREEN (post-fix) |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `graded-clean` | `0 of 79` | `7 of 79` |
|
||||||
|
| `host-pinned` block | absent (0 rows) | `48 PASS row(s)` |
|
||||||
|
| `--why` | all 79 rows printed as findings | only non-PASS rows and `PASS*` |
|
||||||
|
| `--check` rc | 0 | 0 (the fault was in the figures, not the verdict) |
|
||||||
|
|
||||||
|
Selftest **19 → 21 cells** (c20 a parsed PASS row on a host-independent verifier is
|
||||||
|
graded clean; c21 a parsed `OTHER_RC` row with rc 7 is consistent). Both cells are
|
||||||
|
RED against the `is` form of the same expression and GREEN against `==`; c20 asserts
|
||||||
|
the discrimination in both directions (the pinned row is *not* clean).
|
||||||
|
|
||||||
|
Artifacts: `wizard-work/volt/q65-2026-10-25-red.txt` and `-green.txt` (verbatim
|
||||||
|
command output, captured before and after the edit).
|
||||||
|
|
||||||
|
## 3. What this item does NOT close
|
||||||
|
|
||||||
|
* **The 24 `FAIL` rows** are per-lane findings, held open in the ledger, exactly as
|
||||||
|
reply 93 recorded. They are not this item's bookkeeping.
|
||||||
|
* **48 host-pinned verifiers** are Q62's class. The column makes them visible and keeps
|
||||||
|
them out of the coverage figure; it does not fix them, and each belongs to its lane.
|
||||||
|
* **`q63_verify_scan.py` prints a `NOT_GRADED` note, it does not fault on one.** Before
|
||||||
|
the re-emit it printed two `NOT_GRADED TIMEOUT` rows at `d45a5ed` and still exited 0;
|
||||||
|
after the emit it prints `run by q65_runner : 79 (FAIL 24, PASS 55)`, `census
|
||||||
|
coverage : 93 of 93 committed verifier(s) carry a ledger row` and `q63: 0 fault(s)`.
|
||||||
|
A note is not a verdict in either direction, which is why the lane's gate is
|
||||||
|
`q65_gate.sh`: it fails on a `TIMEOUT` row rather than printing one, rc 1 with the
|
||||||
|
row named (the arm measured on a two-row fixture, one `PASS` and one `TIMEOUT`:
|
||||||
|
`TIMEOUT rows found: 1`, row printed — below).
|
||||||
|
* **Promoting the lane gate to a `tools/verify.py` leg** is still the operator's call,
|
||||||
|
not a lane's (reply 93 §4).
|
||||||
|
|
||||||
|
## 4. The final emit
|
||||||
|
|
||||||
|
The five window rows were re-run and merged; the ledger at `d45a5ed` holds **79 rows:
|
||||||
|
PASS 55, FAIL 24, TIMEOUT 0**. The fifth — `sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py`,
|
||||||
|
live HTTP — returned **FAIL rc 1 in 319.30 s**, inside its 1,800 s window; under the
|
||||||
|
single sweep figure it had no rc at all.
|
||||||
|
|
||||||
|
`wizard-work/volt/q65-2026-10-25-final.txt` is the verbatim capture, 2026-10-25T02:20:22Z
|
||||||
|
at HEAD `77d5eba` (the tick opened at `d45a5ed`; `77d5eba` is the queue's own
|
||||||
|
queue-index render and touches no volt file):
|
||||||
|
|
||||||
|
| reading | figure | command |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| ledger rows | 79 — PASS 55 / FAIL 24 / TIMEOUT 0 | `q65_runner.py --check --class gate-run,undeclared` |
|
||||||
|
| rows with an rc | 79 of 79 | same |
|
||||||
|
| graded-clean | 7 of 79 | same |
|
||||||
|
| host-pinned `PASS*` | 48 | same |
|
||||||
|
| census unclassed | 0 of 93 committed verifiers | `q63_verify_scan.py --repo .` |
|
||||||
|
| selftest cells | 22 of 22 | `q65_runner.py --selftest` |
|
||||||
|
| lane gate | rc 0, 11.05 s wall | `bash wizard-work/volt/q65_gate.sh` |
|
||||||
|
|
||||||
|
### 4.1 A third defect in the instrument: the summary's total and its tally came from two sets
|
||||||
|
|
||||||
|
Found by running `--check` with no `--class`, which is the default a reader gets:
|
||||||
|
|
||||||
|
q65_runner -- rev HEAD, 6 population file(s), 79 ledger row(s) [ledger rev d45a5ed]
|
||||||
|
rows at emit : 79 (PASS 1, FAIL 5)
|
||||||
|
|
||||||
|
79 rows, tallied as 1 + 5. Both numbers were right — `counts` was built over the
|
||||||
|
`--class` population (6 files, the `undeclared` default) and the total printed
|
||||||
|
`len(rows)` (79). The lane gate passes `--class gate-run,undeclared`, so `CHECK` never
|
||||||
|
saw it; an operator running the tool the way its usage line documents did. This is §2's
|
||||||
|
class one layer up: the fault was in the sentence, not the numbers.
|
||||||
|
|
||||||
|
Fixed by counting in `tally(rows, pop=None)` and printing the scoped figure on its own
|
||||||
|
line, only when the population is narrower than the ledger:
|
||||||
|
|
||||||
|
rows at emit : 79 (PASS 55, FAIL 24)
|
||||||
|
in population : 6 of 79 (PASS 1, FAIL 5)
|
||||||
|
|
||||||
|
Selftest **21 → 22 cells**; c22 sums both tallies. Measured RED on a scratch copy with
|
||||||
|
the scope guard replaced by `if False:` — `FAIL c22 ... 21 passed, 1 failed`, **rc 1**
|
||||||
|
— and GREEN here: `22 cells, 22 passed, 0 failed`, **rc 0**. The contradictory line as
|
||||||
|
the tool printed it before the fix is preserved verbatim in
|
||||||
|
`wizard-work/volt/q65-2026-10-25-tally-red.txt`.
|
||||||
|
|
||||||
|
## 5. Evidence
|
||||||
|
|
||||||
|
* `wizard-work/volt/q65_runner.py` — `--windows`, the blob column, the host-pin
|
||||||
|
probe, `row_is_clean()`, selftest 21 cells (all passing at this commit).
|
||||||
|
* `wizard-work/volt/q65_row_windows.tsv` — five windows, one per path.
|
||||||
|
* `wizard-work/volt/q65_runner_baseline.tsv` — the ledger, `# rev:` in the header,
|
||||||
|
one row per population file, blob + host columns filled statically.
|
||||||
|
* `wizard-work/volt/CHECK` — 22-cell selftest, `--check --class gate-run,undeclared`,
|
||||||
|
the Q63 scan; root resolved from `$0`; rc 0/1/2.
|
||||||
|
* `wizard-work/volt/q65_gate.sh` — `CHECK` plus "no row is a `TIMEOUT`".
|
||||||
|
* `wizard-work/volt/q65-2026-10-25-{red,green}.txt` — the verbatim before/after of §2.
|
||||||
|
* `wizard-work/volt/q65-2026-10-25-final.txt` — the post-merge reading of §4: `--check`,
|
||||||
|
the lane gate with its wall time, the Q63 scan, and the `TIMEOUT`-arm fixture run.
|
||||||
|
* `wizard-work/volt/q65-2026-10-25-tally-red.txt` — the pre-fix `--check` capture of
|
||||||
|
§4.1: `rows at emit : 79 (PASS 1, FAIL 5)`.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# q65_gate.sh -- Q65's done-means in one command, and the repo-root resolution
|
||||||
|
# Q65's own addendum asks for.
|
||||||
|
#
|
||||||
|
# Two claims, both graded:
|
||||||
|
# 1. the LEDGER covers the census and grades clean -- that is `CHECK`, which
|
||||||
|
# runs the runner's 22-cell selftest, `--check --class gate-run,undeclared`
|
||||||
|
# (population coverage, verdict/rc pairing, blob binding) and the Q63 scan;
|
||||||
|
# 2. NO ROW IS A TIMEOUT. A TIMEOUT row carries no rc, and Q65's done-when is
|
||||||
|
# "each of the five carries an rc or a refusal". A ledger that keeps a
|
||||||
|
# TIMEOUT row is a ledger with an ungraded verifier, so this gate is red on
|
||||||
|
# one -- which is the only reason the check is worth running.
|
||||||
|
#
|
||||||
|
# The root is resolved from $0 rather than written down: a gate that `cd`s to an
|
||||||
|
# absolute host path grades THAT tree from whatever checkout invokes it, which is
|
||||||
|
# Q62's class and was measured in this very lane (reply 93 defect 1).
|
||||||
|
#
|
||||||
|
# rc 0 census complete, ledger clean, zero TIMEOUT rows
|
||||||
|
# rc 1 a fault it graded (a CHECK fault, or a TIMEOUT row)
|
||||||
|
# rc 2 it could not grade: no git root above $0, or no ledger
|
||||||
|
set -u
|
||||||
|
ROOT="$(git -C "$(dirname "$0")" rev-parse --show-toplevel 2>/dev/null)" || exit 2
|
||||||
|
[ -n "$ROOT" ] || exit 2
|
||||||
|
LEDGER="$ROOT/wizard-work/volt/q65_runner_baseline.tsv"
|
||||||
|
[ -f "$LEDGER" ] || { echo "q65_gate: no ledger at $LEDGER" >&2; exit 2; }
|
||||||
|
|
||||||
|
bash "$ROOT/wizard-work/volt/CHECK" || exit 1
|
||||||
|
|
||||||
|
n="$(awk -F'\t' '!/^#/ && NF >= 3 && $3 == "TIMEOUT"' "$LEDGER" | wc -l | tr -d ' ')"
|
||||||
|
if [ "$n" != "0" ]; then
|
||||||
|
echo "q65_gate: $n TIMEOUT row(s) -- a row with no rc is not a grade" >&2
|
||||||
|
awk -F'\t' '!/^#/ && NF >= 3 && $3 == "TIMEOUT" {print " " $2 " (" $6 ")"}' "$LEDGER" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "q65_gate PASS: census complete, ledger clean, 0 TIMEOUT row(s)."
|
||||||
|
exit 0
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# Q65 per-row measurement windows -- one window per path, because the cost of
|
||||||
|
# grading a verifier is a property of the verifier and not of the caller.
|
||||||
|
#
|
||||||
|
# Why this file exists: `--timeout` was a single figure for the whole sweep, so
|
||||||
|
# five verifiers came back TIMEOUT with no rc at whatever figure the sweep ran
|
||||||
|
# under (45s, 90s, 120s, 600s -- one figure per row, none of them the file's).
|
||||||
|
# A TIMEOUT row is not a grade. Each of these five runs its subject end to end
|
||||||
|
# (a recursive tools/ invocation, or live HTTP with retries); the window is the
|
||||||
|
# figure the file needs plus headroom, and --timeout stays the fallback for the
|
||||||
|
# 76 paths the table does not name.
|
||||||
|
#
|
||||||
|
# row: <path> TAB <seconds> TAB <why>
|
||||||
|
wizard-work/forge/verify_citation_hash_class_portable_default.py 1800 three full-repo citation_hash_class.py --json passes over 1,700+ files
|
||||||
|
wizard-work/forge/verify_doc_census_dry_refactor.py 1800 runs tools/doc_census_check.py twice over the whole documentation tree
|
||||||
|
wizard-work/forge/verify_selftest_sweep_refactor.py 1800 runs tools/selftest_sweep.py over every pinned command
|
||||||
|
wizard-work/sphinx/archive-index-2026-10-23/verify_index.py 1800 reads every file at three revisions through git plumbing
|
||||||
|
wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py 1800 live HTTP fetches, 3 retries at a 90s per-request timeout
|
||||||
|
+370
-40
@@ -44,8 +44,35 @@ WHAT IT MEASURES -- one row per population file, one verdict per row
|
|||||||
OTHER_RC / TIMEOUT / ERROR row fatal, for an operator who wants the runner to
|
OTHER_RC / TIMEOUT / ERROR row fatal, for an operator who wants the runner to
|
||||||
be a red gate on the verifiers themselves rather than on the ledger.
|
be a red gate on the verifiers themselves rather than on the ledger.
|
||||||
|
|
||||||
|
THREE THINGS THIS LEDGER USED TO GET WRONG, AND THE COLUMN EACH ONE TOOK
|
||||||
|
1. ONE GLOBAL WINDOW. `--timeout` was a single figure for the whole sweep, so
|
||||||
|
a verifier whose own work takes minutes (recursive `tools/` runs, live HTTP)
|
||||||
|
came back TIMEOUT at whatever the sweep happened to pass -- 45s for one row,
|
||||||
|
600s for another -- and a TIMEOUT row is a row with no rc, which is not a
|
||||||
|
grade. Different files cost different amounts of time; that is a property of
|
||||||
|
the file, not of the caller. `--windows <tsv>` names one window per path
|
||||||
|
(path, seconds, why) and `run_one` uses it; `--timeout` stays the default
|
||||||
|
for paths the table does not name. A TIMEOUT row's note names the window it
|
||||||
|
actually got, so the table cannot silently disagree with the ledger.
|
||||||
|
2. A ROW WAS NOT BOUND TO CONTENT. Rows carried path, verdict, rc, seconds and
|
||||||
|
note, and `--check` compared path SETS. A verifier edited to `raise
|
||||||
|
SystemExit(1)` kept its `PASS 0` row and the gate returned rc 0: the ledger
|
||||||
|
graded a blob it no longer described. Column 7 is the git blob id the rc was
|
||||||
|
measured against (`git rev-parse <rev>:<path>`); `--check` recomputes it at
|
||||||
|
its own `--rev` and a mismatch is a DRIFT fault. Editing a verifier is now a
|
||||||
|
request for `--emit`, which is the only thing that can produce an rc.
|
||||||
|
3. A PASS COULD BE A PASS ON SOMEBODY ELSE'S TREE. An exit code is not a claim
|
||||||
|
about the commit: a verifier that reads `/root/troupe-standup` returns rc 0
|
||||||
|
from any checkout, because it graded this host's copy rather than the tree
|
||||||
|
that invoked it. Column 8 is a static host-pin probe -- the committed blob is
|
||||||
|
parsed as an AST and every string constant that is not a docstring is tested
|
||||||
|
for a host-absolute root (`/root/troupe-standup`, `/root/.hermes`). Rows
|
||||||
|
carrying one print as `PASS*` and are excluded from the `graded-clean`
|
||||||
|
figure, so a host-pinned pass is still a row, still a verdict, and no longer
|
||||||
|
counted as coverage of the tree.
|
||||||
|
|
||||||
WHY THE POPULATION IS A GLOB AND NOT A LIST OF NAMES
|
WHY THE POPULATION IS A GLOB AND NOT A LIST OF NAMES
|
||||||
`q63_verify_scan.py` classes a file as wired when a committed line invokes it
|
`q63_verify_scan.py` classes a file as wired when a committed line invokes it
|
||||||
by name. A globbed runner names no file, so the scan cannot see it -- and the
|
by name. A globbed runner names no file, so the scan cannot see it -- and the
|
||||||
scan is right to be suspicious, because "a gate runs it" is exactly the claim
|
scan is right to be suspicious, because "a gate runs it" is exactly the claim
|
||||||
that is easy to make and hard to check. The evidence here is stronger than a
|
that is easy to make and hard to check. The evidence here is stronger than a
|
||||||
@@ -70,6 +97,7 @@ USAGE
|
|||||||
python3 wizard-work/volt/q65_runner.py --class all --emit --timeout 120
|
python3 wizard-work/volt/q65_runner.py --class all --emit --timeout 120
|
||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
|
import ast
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -82,7 +110,11 @@ HERE = os.path.dirname(os.path.abspath(__file__))
|
|||||||
DEFAULT_REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
|
DEFAULT_REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||||
DEFAULT_LEDGER = os.path.join(HERE, "q65_runner_baseline.tsv")
|
DEFAULT_LEDGER = os.path.join(HERE, "q65_runner_baseline.tsv")
|
||||||
DEFAULT_Q63 = os.path.join(HERE, "q63_unwired_baseline.tsv")
|
DEFAULT_Q63 = os.path.join(HERE, "q63_unwired_baseline.tsv")
|
||||||
|
DEFAULT_WINDOWS = os.path.join(HERE, "q65_row_windows.tsv")
|
||||||
VERIFIER_RE = re.compile(r"^verify_[^/]*\.py$")
|
VERIFIER_RE = re.compile(r"^verify_[^/]*\.py$")
|
||||||
|
# a host-absolute root in a string constant is what Q62/Q65-95 mean by
|
||||||
|
# "host-pinned": the verifier grades this machine's copy from any checkout.
|
||||||
|
HOST_ROOTS = ("/root/troupe-standup", "/root/.hermes")
|
||||||
|
|
||||||
PASS, FAIL, REFUSED, OTHER_RC, TIMEOUT, ERROR, MISSING = (
|
PASS, FAIL, REFUSED, OTHER_RC, TIMEOUT, ERROR, MISSING = (
|
||||||
"PASS", "FAIL", "REFUSED", "OTHER_RC", "TIMEOUT", "ERROR", "MISSING")
|
"PASS", "FAIL", "REFUSED", "OTHER_RC", "TIMEOUT", "ERROR", "MISSING")
|
||||||
@@ -129,12 +161,90 @@ def read_classes(path):
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def read_windows(path):
|
||||||
|
"""path -> (seconds, why), from a <path>\\t<seconds>\\t<why> table.
|
||||||
|
|
||||||
|
Absent or malformed is NOT fatal: a window is a measurement budget, and a
|
||||||
|
missing budget falls back to --timeout. What it must never be is silent, so
|
||||||
|
a malformed row raises rather than being skipped (a skipped row would look
|
||||||
|
like a file with no window, which is indistinguishable from one that ran).
|
||||||
|
"""
|
||||||
|
out = {}
|
||||||
|
if not path or not os.path.exists(path):
|
||||||
|
return out
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
for line in fh:
|
||||||
|
line = line.rstrip("\n")
|
||||||
|
if not line or line.startswith("#"):
|
||||||
|
continue
|
||||||
|
parts = line.split("\t")
|
||||||
|
if len(parts) < 2 or not parts[1].strip().replace(".", "", 1).isdigit():
|
||||||
|
raise SystemExit("q65_runner: malformed window row in %s: %r"
|
||||||
|
% (path, line))
|
||||||
|
out[parts[0]] = (float(parts[1]), parts[2] if len(parts) > 2 else "")
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def blob_of(repo, rev, path):
|
||||||
|
"""git blob id for `path` at `rev`, or '' when it is not in that tree."""
|
||||||
|
r = git(repo, "rev-parse", "--verify", "--quiet", "%s:%s" % (rev, path))
|
||||||
|
return (r.stdout or "").strip() if r.returncode == 0 else ""
|
||||||
|
|
||||||
|
|
||||||
|
def host_pin(repo, rev, path):
|
||||||
|
"""'' or 'L<n>:<literal>' -- a host-absolute root in a NON-docstring string.
|
||||||
|
|
||||||
|
Why an AST and not a grep: every verifier's module docstring names the
|
||||||
|
command that runs it, and most of those commands are written as
|
||||||
|
`python3 /root/troupe-standup/...`. A grep flags that prose and drowns the
|
||||||
|
signal. `ast` can tell a docstring from a value, so the probe answers the
|
||||||
|
question Q62 asks -- does this file READ an absolute host root -- and not
|
||||||
|
the weaker one -- does this file MENTION one.
|
||||||
|
"""
|
||||||
|
r = git(repo, "show", "%s:%s" % (rev, path))
|
||||||
|
if r.returncode != 0:
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
tree = ast.parse(r.stdout)
|
||||||
|
except SyntaxError:
|
||||||
|
return ""
|
||||||
|
docstrings = set()
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if isinstance(node, (ast.Module, ast.FunctionDef, ast.AsyncFunctionDef,
|
||||||
|
ast.ClassDef)):
|
||||||
|
body = getattr(node, "body", [])
|
||||||
|
if (body and isinstance(body[0], ast.Expr)
|
||||||
|
and isinstance(body[0].value, ast.Constant)
|
||||||
|
and isinstance(body[0].value.value, str)):
|
||||||
|
docstrings.add(id(body[0].value))
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if not isinstance(node, ast.Constant) or id(node) in docstrings:
|
||||||
|
continue
|
||||||
|
val = node.value
|
||||||
|
if not isinstance(val, str):
|
||||||
|
continue
|
||||||
|
for root in HOST_ROOTS:
|
||||||
|
if root in val:
|
||||||
|
return "L%d:%s" % (getattr(node, "lineno", 0), root)
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def population(repo, rev, classes_path, want):
|
def population(repo, rev, classes_path, want):
|
||||||
|
"""The files --check holds to a ledger row: census classes, or `all`.
|
||||||
|
|
||||||
|
`--class` takes one class or a comma list, because the population a gate
|
||||||
|
needs is not always one class: `gate-run,undeclared` is "every committed
|
||||||
|
verifier the census says the runner graded, plus every one it says nothing
|
||||||
|
does" -- the exact set that must carry a row, without dragging in the
|
||||||
|
`wired-invoke` files (graded by their own gate) or the `declared-one-off`
|
||||||
|
ones (which state in-file that no gate invokes them).
|
||||||
|
"""
|
||||||
tracked = set(rev_files(repo, rev))
|
tracked = set(rev_files(repo, rev))
|
||||||
if want == "all":
|
if want == "all":
|
||||||
return {p: "ungated" for p in tracked}
|
return {p: "ungated" for p in tracked}
|
||||||
cls = read_classes(classes_path)
|
cls = read_classes(classes_path)
|
||||||
want = [want] if isinstance(want, str) else want
|
want = [want] if isinstance(want, str) else list(want)
|
||||||
|
want = [w.strip() for w in ",".join(want).split(",") if w.strip()]
|
||||||
return {p: c for p, c in cls.items() if c in want and p in tracked}
|
return {p: c for p, c in cls.items() if c in want and p in tracked}
|
||||||
|
|
||||||
|
|
||||||
@@ -153,8 +263,10 @@ def unclassed(repo, rev, classes_path, ledger_paths):
|
|||||||
if p not in cls and p not in ledger_paths)
|
if p not in cls and p not in ledger_paths)
|
||||||
|
|
||||||
|
|
||||||
def run_one(repo, path, timeout, cwd_mode):
|
def run_one(repo, path, timeout, cwd_mode, windows=None):
|
||||||
"""Execute one verifier. Returns a row dict; never raises."""
|
"""Execute one verifier. Returns a row dict; never raises."""
|
||||||
|
windows = windows or {}
|
||||||
|
window, why = windows.get(path, (timeout, ""))
|
||||||
full = os.path.join(repo, path)
|
full = os.path.join(repo, path)
|
||||||
if not os.path.exists(full):
|
if not os.path.exists(full):
|
||||||
return {"path": path, "verdict": MISSING, "rc": None, "seconds": 0.0,
|
return {"path": path, "verdict": MISSING, "rc": None, "seconds": 0.0,
|
||||||
@@ -166,12 +278,13 @@ def run_one(repo, path, timeout, cwd_mode):
|
|||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
try:
|
try:
|
||||||
r = subprocess.run([sys.executable, "-B", target],
|
r = subprocess.run([sys.executable, "-B", target],
|
||||||
cwd=cwd, env=env, capture_output=True, timeout=timeout,
|
cwd=cwd, env=env, capture_output=True, timeout=window,
|
||||||
text=True, errors="replace")
|
text=True, errors="replace")
|
||||||
except subprocess.TimeoutExpired:
|
except subprocess.TimeoutExpired:
|
||||||
return {"path": path, "verdict": TIMEOUT, "rc": None,
|
return {"path": path, "verdict": TIMEOUT, "rc": None,
|
||||||
"seconds": round(time.time() - t0, 2),
|
"seconds": round(time.time() - t0, 2),
|
||||||
"note": "no finish in %ss" % timeout}
|
"note": "no finish in %ss%s" % (window, (" (row window: %s)" % why)
|
||||||
|
if why else "")}
|
||||||
secs = round(time.time() - t0, 2)
|
secs = round(time.time() - t0, 2)
|
||||||
rc = r.returncode
|
rc = r.returncode
|
||||||
if rc == 0:
|
if rc == 0:
|
||||||
@@ -184,7 +297,7 @@ def run_one(repo, path, timeout, cwd_mode):
|
|||||||
verdict = OTHER_RC if rc > 2 else ERROR
|
verdict = OTHER_RC if rc > 2 else ERROR
|
||||||
note = ""
|
note = ""
|
||||||
tail = (r.stderr or "").strip().splitlines()
|
tail = (r.stderr or "").strip().splitlines()
|
||||||
if verdict is not PASS and tail:
|
if verdict != PASS and tail:
|
||||||
note = tail[-1][:160]
|
note = tail[-1][:160]
|
||||||
return {"path": path, "verdict": verdict, "rc": rc, "seconds": secs,
|
return {"path": path, "verdict": verdict, "rc": rc, "seconds": secs,
|
||||||
"note": note}
|
"note": note}
|
||||||
@@ -198,7 +311,7 @@ def emit(rows, path, repo, rev, timeout):
|
|||||||
lines = [
|
lines = [
|
||||||
"# Q65 runner ledger -- one row per committed verify_*.py the runner",
|
"# Q65 runner ledger -- one row per committed verify_*.py the runner",
|
||||||
"# executes, with the exit code it actually returned at the stated rev.",
|
"# executes, with the exit code it actually returned at the stated rev.",
|
||||||
"# row: <class>\\t<path>\\t<verdict>\\t<rc>\\t<seconds>\\t<note>",
|
"# row: <class>\\t<path>\\t<verdict>\\t<rc>\\t<seconds>\\t<note>\\t<blob>\\t<host>",
|
||||||
"# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 /",
|
"# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 /",
|
||||||
"# TIMEOUT, ERROR, MISSING (no rc to grade).",
|
"# TIMEOUT, ERROR, MISSING (no rc to grade).",
|
||||||
"# class is the Q63 ledger class the file came from at emit time.",
|
"# class is the Q63 ledger class the file came from at emit time.",
|
||||||
@@ -206,15 +319,28 @@ def emit(rows, path, repo, rev, timeout):
|
|||||||
"# grades the bookkeeping, --check --strict makes any non-PASS fatal.",
|
"# grades the bookkeeping, --check --strict makes any non-PASS fatal.",
|
||||||
"# timeout is PER ROW: a TIMEOUT row names its own window in the note, so",
|
"# timeout is PER ROW: a TIMEOUT row names its own window in the note, so",
|
||||||
"# a --merge cannot leave a stale single figure standing for rows run",
|
"# a --merge cannot leave a stale single figure standing for rows run",
|
||||||
"# under a different one (the defect this line replaces).",
|
"# under a different one (the defect this line replaces). --windows",
|
||||||
|
"# names one window per path; --timeout is the fallback for unlisted.",
|
||||||
|
"# blob is the git blob id this rc was measured against: --check recomputes",
|
||||||
|
"# it at its own --rev and a mismatch is DRIFT, so an edited verifier",
|
||||||
|
"# cannot keep a PASS it no longer earns.",
|
||||||
|
"# host is a static host-pin probe (L<line>:<root>), empty when clean: a",
|
||||||
|
"# non-docstring string constant naming an absolute host root. Such a row",
|
||||||
|
"# is printed PASS* and excluded from the graded-clean figure.",
|
||||||
"# Regenerate: q65_runner.py --emit",
|
"# Regenerate: q65_runner.py --emit",
|
||||||
"# rev: %s rows: %d emitted: q65_runner.py" % (sha, len(rows)),
|
"# rev: %s rows: %d emitted: q65_runner.py" % (sha, len(rows)),
|
||||||
]
|
]
|
||||||
for r in sorted(rows, key=lambda x: x["path"]):
|
for r in sorted(rows, key=lambda x: x["path"]):
|
||||||
|
blob = r.get("blob") or blob_of(repo, rev, r["path"])
|
||||||
|
host = r.get("host")
|
||||||
|
if not host:
|
||||||
|
# An empty column is "not filled in" as well as "clean": the probe
|
||||||
|
# is cheap and deterministic, so it is re-run rather than trusted.
|
||||||
|
host = host_pin(repo, rev, r["path"])
|
||||||
lines.append("\t".join([r.get("class", "ungated"), r["path"], r["verdict"],
|
lines.append("\t".join([r.get("class", "ungated"), r["path"], r["verdict"],
|
||||||
"" if r["rc"] is None else str(r["rc"]),
|
"" if r["rc"] is None else str(r["rc"]),
|
||||||
"%.2f" % (float(r["seconds"]) if r.get("seconds") else 0.0),
|
"%.2f" % (float(r["seconds"]) if r.get("seconds") else 0.0),
|
||||||
(r.get("note") or "").replace("\t", " ")]))
|
(r.get("note") or "").replace("\t", " "), blob, host]))
|
||||||
tmp = path + ".tmp"
|
tmp = path + ".tmp"
|
||||||
with open(tmp, "w", encoding="utf-8") as fh:
|
with open(tmp, "w", encoding="utf-8") as fh:
|
||||||
fh.write("\n".join(lines) + "\n")
|
fh.write("\n".join(lines) + "\n")
|
||||||
@@ -223,25 +349,35 @@ def emit(rows, path, repo, rev, timeout):
|
|||||||
|
|
||||||
|
|
||||||
def read_ledger(path):
|
def read_ledger(path):
|
||||||
"""(rows, error). rows keep every column; error is a string for rc 2."""
|
"""(rows, error, meta). rows keep every column; error is a string for rc 2.
|
||||||
|
|
||||||
|
meta carries the ledger's own `# rev:` line, so `--check` can say which tree
|
||||||
|
the rows claim and grade the blobs against it rather than against nothing.
|
||||||
|
"""
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
return [], "no ledger at %s (run --emit first)" % path
|
return [], "no ledger at %s (run --emit first)" % path, {}
|
||||||
rows = []
|
rows = []
|
||||||
|
meta = {}
|
||||||
with open(path, encoding="utf-8") as fh:
|
with open(path, encoding="utf-8") as fh:
|
||||||
for n, line in enumerate(fh, 1):
|
for n, line in enumerate(fh, 1):
|
||||||
line = line.rstrip("\n")
|
line = line.rstrip("\n")
|
||||||
if not line or line.startswith("#"):
|
if not line or line.startswith("#"):
|
||||||
|
m = re.match(r"#\s*rev:\s*(\S+)", line)
|
||||||
|
if m:
|
||||||
|
meta["rev"] = m.group(1)
|
||||||
continue
|
continue
|
||||||
parts = line.split("\t")
|
parts = line.split("\t")
|
||||||
if len(parts) < 4:
|
if len(parts) < 4:
|
||||||
return [], "malformed row %d in %s: %r" % (n, path, line)
|
return [], "malformed row %d in %s: %r" % (n, path, line), {}
|
||||||
if parts[2] not in VERDICTS:
|
if parts[2] not in VERDICTS:
|
||||||
return [], "unknown verdict %r at row %d in %s" % (parts[2], n, path)
|
return [], "unknown verdict %r at row %d in %s" % (parts[2], n, path), {}
|
||||||
rows.append({"class": parts[0], "path": parts[1], "verdict": parts[2],
|
rows.append({"class": parts[0], "path": parts[1], "verdict": parts[2],
|
||||||
"rc": parts[3] if parts[3] else None,
|
"rc": parts[3] if parts[3] else None,
|
||||||
"seconds": parts[4] if len(parts) > 4 else "",
|
"seconds": parts[4] if len(parts) > 4 else "",
|
||||||
"note": parts[5] if len(parts) > 5 else ""})
|
"note": parts[5] if len(parts) > 5 else "",
|
||||||
return rows, None
|
"blob": parts[6] if len(parts) > 6 else "",
|
||||||
|
"host": parts[7] if len(parts) > 7 else ""})
|
||||||
|
return rows, None, meta
|
||||||
|
|
||||||
|
|
||||||
def rc_consistent(row):
|
def rc_consistent(row):
|
||||||
@@ -251,7 +387,7 @@ def rc_consistent(row):
|
|||||||
if rc != RC_OF[v]:
|
if rc != RC_OF[v]:
|
||||||
return "verdict %s with rc %s (expected %s)" % (v, rc, RC_OF[v])
|
return "verdict %s with rc %s (expected %s)" % (v, rc, RC_OF[v])
|
||||||
return None
|
return None
|
||||||
if v is OTHER_RC:
|
if v == OTHER_RC:
|
||||||
if rc is None or not rc.isdigit() or int(rc) <= 2:
|
if rc is None or not rc.isdigit() or int(rc) <= 2:
|
||||||
return "OTHER_RC with rc %r" % rc
|
return "OTHER_RC with rc %r" % rc
|
||||||
return None
|
return None
|
||||||
@@ -260,6 +396,45 @@ def rc_consistent(row):
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def row_is_clean(row, pop):
|
||||||
|
"""Is this row a PASS on a host-independent verifier, for a graded path?
|
||||||
|
|
||||||
|
WHY THIS IS A FUNCTION, AND WHY IT COMPARES WITH `==`. `read_ledger` builds
|
||||||
|
each verdict by splitting a line, so the string it returns is a NEW object
|
||||||
|
and not the module constant: `r["verdict"] is PASS` was False for every row
|
||||||
|
read from disk. Measured at `d45a5ed` on 2026-10-25, before this line
|
||||||
|
existed: `--check` printed `graded-clean: 0 of 79` over a ledger holding 7
|
||||||
|
clean PASS rows, printed no `host-pinned` block over 48 host-pinned PASS
|
||||||
|
rows, and `--why` printed all 79 rows as findings. Identity is the right
|
||||||
|
question for a constant a function just returned and the wrong one for a
|
||||||
|
value parsed out of a file; the selftest cell c20 holds that distinction.
|
||||||
|
"""
|
||||||
|
return (row["path"] in pop and row["verdict"] == PASS and not row["host"]
|
||||||
|
and rc_consistent(row) is None)
|
||||||
|
|
||||||
|
|
||||||
|
def tally(rows, pop=None):
|
||||||
|
"""Verdict counts over `rows`, or only over the rows in `pop`.
|
||||||
|
|
||||||
|
WHY THIS IS A FUNCTION. `--check` printed one figure's total beside
|
||||||
|
another figure's tally: at 2026-10-25 02:18Z, with the default
|
||||||
|
`--class undeclared`, it printed `rows at emit : 79 (PASS 1, FAIL 5)` --
|
||||||
|
79 ledger rows under a tally of 6 population rows, and 1 + 5 is not 79.
|
||||||
|
Every number was right; the sentence was not. Splitting the count from the
|
||||||
|
print lets selftest cell c22 grade the arithmetic instead of the prose.
|
||||||
|
"""
|
||||||
|
out = {}
|
||||||
|
for r in rows:
|
||||||
|
if pop is not None and r["path"] not in pop:
|
||||||
|
continue
|
||||||
|
out[r["verdict"]] = out.get(r["verdict"], 0) + 1
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def fmt_counts(counts):
|
||||||
|
return ", ".join("%s %d" % (k, counts[k]) for k in VERDICTS if counts.get(k))
|
||||||
|
|
||||||
|
|
||||||
def check(repo, rev, classes_path, want, ledger_path, strict):
|
def check(repo, rev, classes_path, want, ledger_path, strict):
|
||||||
"""Grade the ledger against the tree. rc 0 clean / 1 fault / 2 could not grade."""
|
"""Grade the ledger against the tree. rc 0 clean / 1 fault / 2 could not grade."""
|
||||||
try:
|
try:
|
||||||
@@ -267,7 +442,7 @@ def check(repo, rev, classes_path, want, ledger_path, strict):
|
|||||||
except SystemExit as exc:
|
except SystemExit as exc:
|
||||||
print("q65_runner: %s" % exc)
|
print("q65_runner: %s" % exc)
|
||||||
return 2
|
return 2
|
||||||
rows, err = read_ledger(ledger_path)
|
rows, err, meta = read_ledger(ledger_path)
|
||||||
if err:
|
if err:
|
||||||
print("q65_runner: %s" % err)
|
print("q65_runner: %s" % err)
|
||||||
return 2
|
return 2
|
||||||
@@ -281,6 +456,29 @@ def check(repo, rev, classes_path, want, ledger_path, strict):
|
|||||||
for p, n in sorted(seen.items()):
|
for p, n in sorted(seen.items()):
|
||||||
if n > 1:
|
if n > 1:
|
||||||
faults.append("DUPLICATE %s: %d rows" % (p, n))
|
faults.append("DUPLICATE %s: %d rows" % (p, n))
|
||||||
|
# Content binding: a row's rc is evidence about one blob and no other. This
|
||||||
|
# is the defect reply 93 named -- a verifier edited to `raise SystemExit(1)`
|
||||||
|
# kept its PASS row and `--check` returned 0 fault(s).
|
||||||
|
drift = 0
|
||||||
|
for r in rows:
|
||||||
|
if r["path"] not in pop:
|
||||||
|
continue
|
||||||
|
want_blob = r["blob"]
|
||||||
|
now_blob = blob_of(repo, rev, r["path"])
|
||||||
|
if not want_blob:
|
||||||
|
drift += 1
|
||||||
|
faults.append("UNBOUND %s: no blob column -- the row does not say "
|
||||||
|
"which content its rc describes (re-emit)" % r["path"])
|
||||||
|
elif now_blob and now_blob != want_blob:
|
||||||
|
drift += 1
|
||||||
|
faults.append("DRIFT %s: rc measured against blobs %s, tree at %s has "
|
||||||
|
"%s" % (r["path"], want_blob[:10], rev, now_blob[:10]))
|
||||||
|
# Host-pin: a PASS on a verifier that reads an absolute host root graded THIS
|
||||||
|
# host, not the tree that invoked it (Q62's class, reply 95's row).
|
||||||
|
pinned = 0
|
||||||
|
for r in rows:
|
||||||
|
if r["verdict"] == PASS and r["host"] and r["path"] in pop:
|
||||||
|
pinned += 1
|
||||||
audit = [] if want == "all" else unclassed(repo, rev, classes_path, set(seen))
|
audit = [] if want == "all" else unclassed(repo, rev, classes_path, set(seen))
|
||||||
for p in audit:
|
for p in audit:
|
||||||
faults.append("UNCLASSED %s: committed verify_*.py at %s, in no class row "
|
faults.append("UNCLASSED %s: committed verify_*.py at %s, in no class row "
|
||||||
@@ -295,18 +493,29 @@ def check(repo, rev, classes_path, want, ledger_path, strict):
|
|||||||
faults.append("STALE %s: ledger row, file not in the tree" % p)
|
faults.append("STALE %s: ledger row, file not in the tree" % p)
|
||||||
elif strict:
|
elif strict:
|
||||||
faults.append("OUTSIDE %s: ledger row, not in the --class population" % p)
|
faults.append("OUTSIDE %s: ledger row, not in the --class population" % p)
|
||||||
counts = {}
|
counts = tally(rows, pop)
|
||||||
for r in rows:
|
print("q65_runner -- rev %s, %d population file(s), %d ledger row(s)%s"
|
||||||
if r["path"] in pop:
|
% (rev, len(pop), len(rows),
|
||||||
counts[r["verdict"]] = counts.get(r["verdict"], 0) + 1
|
(" [ledger rev %s]" % meta["rev"]) if meta.get("rev") else ""))
|
||||||
print("q65_runner -- rev %s, %d population file(s), %d ledger row(s)"
|
|
||||||
% (rev, len(pop), len(rows)))
|
|
||||||
graded = sum(1 for r in rows if r["path"] in pop and r["rc"] is not None
|
graded = sum(1 for r in rows if r["path"] in pop and r["rc"] is not None
|
||||||
and rc_consistent(r) is None)
|
and rc_consistent(r) is None)
|
||||||
print(" rows at emit : %d (%s)" % (
|
clean = sum(1 for r in rows if row_is_clean(r, pop))
|
||||||
len(rows),
|
print(" rows at emit : %d (%s)" % (len(rows), fmt_counts(tally(rows))))
|
||||||
", ".join("%s %d" % (k, counts[k]) for k in VERDICTS if counts.get(k))))
|
if any(r["path"] not in pop for r in rows):
|
||||||
|
print(" in population : %d of %d (%s)"
|
||||||
|
% (len(pop), len(rows), fmt_counts(counts)))
|
||||||
print(" graded (has rc): %d of %d" % (graded, len(pop)))
|
print(" graded (has rc): %d of %d" % (graded, len(pop)))
|
||||||
|
print(" graded-clean : %d of %d (PASS on a host-independent verifier)"
|
||||||
|
% (clean, len(pop)))
|
||||||
|
if pinned:
|
||||||
|
print(" host-pinned : %d PASS row(s) excluded as PASS* -- they read an "
|
||||||
|
"absolute host root" % pinned)
|
||||||
|
for r in rows:
|
||||||
|
if r["verdict"] == PASS and r["host"] and r["path"] in pop:
|
||||||
|
print(" PASS* %s %s" % (r["path"], r["host"]))
|
||||||
|
if drift:
|
||||||
|
print(" content-bound : %d row(s) not bound to the tree at %s (DRIFT/UNBOUND)"
|
||||||
|
% (drift, rev))
|
||||||
print(" tree audit : %d unclassed of %d committed verifier(s)%s"
|
print(" tree audit : %d unclassed of %d committed verifier(s)%s"
|
||||||
% (len(audit), len(pop) if want == "all" else len(rev_files(repo, rev)),
|
% (len(audit), len(pop) if want == "all" else len(rev_files(repo, rev)),
|
||||||
"" if not audit else " <-- named by neither ledger"))
|
"" if not audit else " <-- named by neither ledger"))
|
||||||
@@ -348,7 +557,7 @@ def selftest():
|
|||||||
fh.write("# c\n")
|
fh.write("# c\n")
|
||||||
fh.write("ungated\twizard-work/x/verify_a.py\tPASS\t0\t0.01\t\n")
|
fh.write("ungated\twizard-work/x/verify_a.py\tPASS\t0\t0.01\t\n")
|
||||||
fh.write("ungated\twizard-work/x/verify_b.py\tFAIL\t1\t0.02\tboom\n")
|
fh.write("ungated\twizard-work/x/verify_b.py\tFAIL\t1\t0.02\tboom\n")
|
||||||
rows, err = read_ledger(led)
|
rows, err, meta = read_ledger(led)
|
||||||
cell("c7 ledger round-trips two rows", err is None and len(rows) == 2)
|
cell("c7 ledger round-trips two rows", err is None and len(rows) == 2)
|
||||||
cell("c8 a missing ledger is an error, not an empty pass",
|
cell("c8 a missing ledger is an error, not an empty pass",
|
||||||
read_ledger(os.path.join(tmp, "nope.tsv"))[1] is not None)
|
read_ledger(os.path.join(tmp, "nope.tsv"))[1] is not None)
|
||||||
@@ -360,17 +569,20 @@ def selftest():
|
|||||||
# c10/c11: the coverage rule. A committed verifier named by neither ledger is
|
# c10/c11: the coverage rule. A committed verifier named by neither ledger is
|
||||||
# a fault, even though the ledger that would grade it is the one it is
|
# a fault, even though the ledger that would grade it is the one it is
|
||||||
# missing from -- that is how 8 of 88 stood invisible at `b01ec0e`.
|
# missing from -- that is how 8 of 88 stood invisible at `b01ec0e`.
|
||||||
import subprocess as _sp
|
def mk_tree(prefix):
|
||||||
tree = tempfile.mkdtemp(prefix="q65-tree-")
|
t = tempfile.mkdtemp(prefix=prefix)
|
||||||
env = dict(os.environ, GIT_AUTHOR_NAME="t", GIT_AUTHOR_EMAIL="t@t",
|
e = dict(os.environ, GIT_AUTHOR_NAME="t", GIT_AUTHOR_EMAIL="t@t",
|
||||||
GIT_COMMITTER_NAME="t", GIT_COMMITTER_EMAIL="t@t")
|
GIT_COMMITTER_NAME="t", GIT_COMMITTER_EMAIL="t@t")
|
||||||
_sp.run(["git", "init", "-q", tree], check=True, env=env)
|
subprocess.run(["git", "init", "-q", t], check=True, env=e)
|
||||||
|
return t, e
|
||||||
|
|
||||||
|
tree, env = mk_tree("q65-tree-")
|
||||||
lane = os.path.join(tree, "wizard-work", "x")
|
lane = os.path.join(tree, "wizard-work", "x")
|
||||||
os.makedirs(lane)
|
os.makedirs(lane)
|
||||||
with open(os.path.join(lane, "verify_orphan.py"), "w", encoding="utf-8") as fh:
|
with open(os.path.join(lane, "verify_orphan.py"), "w", encoding="utf-8") as fh:
|
||||||
fh.write(chr(10).join(("import sys", "sys.exit(0)")) + chr(10))
|
fh.write(chr(10).join(("import sys", "sys.exit(0)")) + chr(10))
|
||||||
_sp.run(["git", "-C", tree, "add", "-A"], check=True, env=env)
|
subprocess.run(["git", "-C", tree, "add", "-A"], check=True, env=env)
|
||||||
_sp.run(["git", "-C", tree, "commit", "-qm", "c"], check=True, env=env)
|
subprocess.run(["git", "-C", tree, "commit", "-qm", "c"], check=True, env=env)
|
||||||
census = os.path.join(tree, "census.tsv")
|
census = os.path.join(tree, "census.tsv")
|
||||||
open(census, "w", encoding="utf-8").close()
|
open(census, "w", encoding="utf-8").close()
|
||||||
cell("c10 a committed verifier in neither ledger is UNCLASSED",
|
cell("c10 a committed verifier in neither ledger is UNCLASSED",
|
||||||
@@ -380,6 +592,85 @@ def selftest():
|
|||||||
"no run")) + chr(10))
|
"no run")) + chr(10))
|
||||||
cell("c11 a class row covers the same file",
|
cell("c11 a class row covers the same file",
|
||||||
unclassed(tree, "HEAD", census, set()) == [])
|
unclassed(tree, "HEAD", census, set()) == [])
|
||||||
|
|
||||||
|
# c12-c17: the three rules added after reply 93/95, each with a cell that
|
||||||
|
# fails if the rule is removed.
|
||||||
|
win = os.path.join(tmp, "win.tsv")
|
||||||
|
with open(win, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write("# path\tseconds\twhy\n")
|
||||||
|
fh.write("wizard-work/x/verify_slow.py\t240\trecursive tools/ run\n")
|
||||||
|
w = read_windows(win)
|
||||||
|
cell("c12 a window row parses to (seconds, why)",
|
||||||
|
w.get("wizard-work/x/verify_slow.py", (0, ""))[0] == 240.0)
|
||||||
|
cell("c13 an unlisted path falls back to --timeout",
|
||||||
|
"wizard-work/x/verify_other.py" not in w
|
||||||
|
and run_one(tree, "wizard-work/x/verify_orphan.py", 7, "dir",
|
||||||
|
windows=w)["verdict"] is PASS)
|
||||||
|
with open(win, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write("wizard-work/x/verify_slow.py\tnot-a-number\t\n")
|
||||||
|
try:
|
||||||
|
read_windows(win)
|
||||||
|
bad_window = False
|
||||||
|
except SystemExit:
|
||||||
|
bad_window = True
|
||||||
|
cell("c14 a malformed window row refuses instead of silently skipping", bad_window)
|
||||||
|
|
||||||
|
# blob binding: the rc is evidence about one blob
|
||||||
|
blob = blob_of(tree, "HEAD", "wizard-work/x/verify_orphan.py")
|
||||||
|
cell("c15 a committed path has a blob id", len(blob) == 40)
|
||||||
|
cell("c16 an absent path has no blob id",
|
||||||
|
blob_of(tree, "HEAD", "wizard-work/x/verify_gone.py") == "")
|
||||||
|
with open(os.path.join(lane, "verify_orphan.py"), "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(chr(10).join(("import sys", "sys.exit(1)")) + chr(10))
|
||||||
|
subprocess.run(["git", "-C", tree, "commit", "-qam", "c2"], check=True, env=env)
|
||||||
|
cell("c17 the same path at a new commit is a different blob (DRIFT is detectable)",
|
||||||
|
blob_of(tree, "HEAD", "wizard-work/x/verify_orphan.py") not in ("", blob))
|
||||||
|
|
||||||
|
# host-pin probe: a value, not prose
|
||||||
|
pinned = os.path.join(lane, "verify_pinned.py")
|
||||||
|
with open(pinned, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(chr(10).join((
|
||||||
|
'"""usage: python3 /root/troupe-standup/wizard-work/x/verify_pinned.py"""',
|
||||||
|
'import os',
|
||||||
|
'REPO = "/root/troupe-standup"',
|
||||||
|
'print(REPO)')) + chr(10))
|
||||||
|
prose = os.path.join(lane, "verify_prose.py")
|
||||||
|
with open(prose, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(chr(10).join((
|
||||||
|
'"""Run as: python3 /root/troupe-standup/wizard-work/x/verify_prose.py."""',
|
||||||
|
'print("ok")')) + chr(10))
|
||||||
|
subprocess.run(["git", "-C", tree, "add", "-A"], check=True, env=env)
|
||||||
|
subprocess.run(["git", "-C", tree, "commit", "-qm", "c3"], check=True, env=env)
|
||||||
|
cell("c18 a host root in a string VALUE is a host pin",
|
||||||
|
host_pin(tree, "HEAD", "wizard-work/x/verify_pinned.py").startswith("L3:"))
|
||||||
|
cell("c19 a host root in a docstring is prose, not a pin",
|
||||||
|
host_pin(tree, "HEAD", "wizard-work/x/verify_prose.py") == "")
|
||||||
|
|
||||||
|
# identity vs equality on a PARSED row -- the defect measured at d45a5ed:
|
||||||
|
# `r["verdict"] is PASS` was False for every row read from disk, so the
|
||||||
|
# graded-clean figure read 0 over a ledger that held clean PASS rows.
|
||||||
|
parsed = os.path.join(tmp, "parsed.tsv")
|
||||||
|
with open(parsed, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write("# c\n")
|
||||||
|
fh.write("ungated\twizard-work/x/verify_clean.py\tPASS\t0\t0.01\t\tdeadbeef\t\n")
|
||||||
|
fh.write("ungated\twizard-work/x/verify_pin.py\tPASS\t0\t0.01\t\tcafe\t"
|
||||||
|
"L3:/root/troupe-standup\n")
|
||||||
|
fh.write("ungated\twizard-work/x/verify_other.py\tOTHER_RC\t7\t0.01\t\tbeef\t\n")
|
||||||
|
praw, perr, _ = read_ledger(parsed)
|
||||||
|
cell("c20 a PARSED PASS row on a host-independent verifier is graded clean",
|
||||||
|
perr is None and len(praw) == 3
|
||||||
|
and row_is_clean(praw[0], {praw[0]["path"]}) is True
|
||||||
|
and row_is_clean(praw[1], {praw[1]["path"]}) is False)
|
||||||
|
cell("c21 a PARSED OTHER_RC row with rc 7 is consistent (not a fault)",
|
||||||
|
perr is None and rc_consistent(praw[2]) is None)
|
||||||
|
# the summary sentence: one figure's total beside another figure's tally.
|
||||||
|
# Measured 2026-10-25 02:18Z at HEAD `77d5eba`, default `--class undeclared`:
|
||||||
|
# `rows at emit : 79 (PASS 1, FAIL 5)`. Each count was right; the line
|
||||||
|
# claimed 1 + 5 rows out of 79. c22 fails if tally() stops scoping to pop.
|
||||||
|
cell("c22 a class-narrowed tally sums to its own population, not the ledger",
|
||||||
|
sum(tally(praw).values()) == 3
|
||||||
|
and sum(tally(praw, {praw[0]["path"]}).values()) == 1)
|
||||||
|
|
||||||
passed = sum(1 for _, ok in cells if ok)
|
passed = sum(1 for _, ok in cells if ok)
|
||||||
for n, ok in cells:
|
for n, ok in cells:
|
||||||
print("%-4s %s" % ("ok" if ok else "FAIL", n))
|
print("%-4s %s" % ("ok" if ok else "FAIL", n))
|
||||||
@@ -401,9 +692,15 @@ def main():
|
|||||||
ap.add_argument("--merge", action="store_true",
|
ap.add_argument("--merge", action="store_true",
|
||||||
help="with --emit: keep rows for paths this run did not touch")
|
help="with --emit: keep rows for paths this run did not touch")
|
||||||
ap.add_argument("--ledger", default=DEFAULT_LEDGER)
|
ap.add_argument("--ledger", default=DEFAULT_LEDGER)
|
||||||
ap.add_argument("--timeout", type=float, default=60.0)
|
ap.add_argument("--windows", default=DEFAULT_WINDOWS,
|
||||||
|
help="per-row timeout table: <path>\\t<seconds>\\t<why>")
|
||||||
|
ap.add_argument("--timeout", type=float, default=60.0,
|
||||||
|
help="fallback window for paths --windows does not name")
|
||||||
ap.add_argument("--cwd", choices=("dir", "repo"), default="dir")
|
ap.add_argument("--cwd", choices=("dir", "repo"), default="dir")
|
||||||
ap.add_argument("--emit", action="store_true", help="execute and write the ledger")
|
ap.add_argument("--emit", action="store_true", help="execute and write the ledger")
|
||||||
|
ap.add_argument("--rebind", action="store_true",
|
||||||
|
help="rewrite the static blob/host columns of the ledger's "
|
||||||
|
"existing rows without executing anything")
|
||||||
ap.add_argument("--check", action="store_true", help="grade the ledger, no execution")
|
ap.add_argument("--check", action="store_true", help="grade the ledger, no execution")
|
||||||
ap.add_argument("--strict", action="store_true",
|
ap.add_argument("--strict", action="store_true",
|
||||||
help="with --check: any non-PASS row is a fault")
|
help="with --check: any non-PASS row is a fault")
|
||||||
@@ -416,10 +713,29 @@ def main():
|
|||||||
if args.selftest:
|
if args.selftest:
|
||||||
return selftest()
|
return selftest()
|
||||||
|
|
||||||
if not (args.emit or args.check or args.why):
|
if not (args.emit or args.check or args.why or args.rebind):
|
||||||
ap.print_help()
|
ap.print_help()
|
||||||
return 2
|
return 2
|
||||||
|
|
||||||
|
if args.rebind:
|
||||||
|
# No execution: blob and host are static properties of the commit, so
|
||||||
|
# they can be re-derived without re-measuring an rc. This exists because
|
||||||
|
# an rc is evidence about content, and the ledger must be able to say
|
||||||
|
# WHICH content after the archive has moved without pretending it
|
||||||
|
# re-ran anything.
|
||||||
|
rows, err, _ = read_ledger(args.ledger)
|
||||||
|
if err:
|
||||||
|
print("q65_runner: %s" % err)
|
||||||
|
return 2
|
||||||
|
for r in rows:
|
||||||
|
r["blob"] = blob_of(args.repo, args.rev, r["path"])
|
||||||
|
r["host"] = host_pin(args.repo, args.rev, r["path"])
|
||||||
|
out = emit(rows, args.ledger, args.repo, args.rev, args.timeout)
|
||||||
|
pinned = sum(1 for r in rows if r["host"])
|
||||||
|
print("q65_runner --rebind: %d row(s), %d host-pinned, -> %s"
|
||||||
|
% (len(rows), pinned, out))
|
||||||
|
return 0
|
||||||
|
|
||||||
try:
|
try:
|
||||||
pop = population(args.repo, args.rev, args.q63, args.cls)
|
pop = population(args.repo, args.rev, args.q63, args.cls)
|
||||||
except SystemExit as exc:
|
except SystemExit as exc:
|
||||||
@@ -430,6 +746,12 @@ def main():
|
|||||||
% (args.cls, args.rev))
|
% (args.cls, args.rev))
|
||||||
return 2
|
return 2
|
||||||
|
|
||||||
|
try:
|
||||||
|
win = read_windows(args.windows)
|
||||||
|
except SystemExit as exc:
|
||||||
|
print("q65_runner: %s" % exc)
|
||||||
|
return 2
|
||||||
|
|
||||||
if args.emit:
|
if args.emit:
|
||||||
before = git(args.repo, "status", "--porcelain").stdout
|
before = git(args.repo, "status", "--porcelain").stdout
|
||||||
rows = []
|
rows = []
|
||||||
@@ -441,14 +763,14 @@ def main():
|
|||||||
if args.limit:
|
if args.limit:
|
||||||
todo = todo[:args.limit]
|
todo = todo[:args.limit]
|
||||||
if args.merge and os.path.exists(args.ledger):
|
if args.merge and os.path.exists(args.ledger):
|
||||||
have, err = read_ledger(args.ledger)
|
have, err, _ = read_ledger(args.ledger)
|
||||||
if err:
|
if err:
|
||||||
print("q65_runner: %s" % err)
|
print("q65_runner: %s" % err)
|
||||||
return 2
|
return 2
|
||||||
rows = [r for r in have if r["path"] not in set(todo)]
|
rows = [r for r in have if r["path"] not in set(todo)]
|
||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
for p in todo:
|
for p in todo:
|
||||||
r = run_one(args.repo, p, args.timeout, args.cwd)
|
r = run_one(args.repo, p, args.timeout, args.cwd, windows=win)
|
||||||
r["class"] = pop[p]
|
r["class"] = pop[p]
|
||||||
rows.append(r)
|
rows.append(r)
|
||||||
# rewrite after every row: an interrupted sweep keeps the rows it
|
# rewrite after every row: an interrupted sweep keeps the rows it
|
||||||
@@ -479,10 +801,11 @@ def main():
|
|||||||
return check(args.repo, args.rev, args.q63, args.cls, args.ledger, args.strict)
|
return check(args.repo, args.rev, args.q63, args.cls, args.ledger, args.strict)
|
||||||
|
|
||||||
# --why on its own: print the ledger, grouped by lane
|
# --why on its own: print the ledger, grouped by lane
|
||||||
rows, err = read_ledger(args.ledger)
|
rows, err, meta = read_ledger(args.ledger)
|
||||||
if err:
|
if err:
|
||||||
print("q65_runner: %s" % err)
|
print("q65_runner: %s" % err)
|
||||||
return 2
|
return 2
|
||||||
|
print("# ledger rev %s, %d row(s)" % (meta.get("rev", "?"), len(rows)))
|
||||||
by_lane = {}
|
by_lane = {}
|
||||||
for r in rows:
|
for r in rows:
|
||||||
lane = r["path"].split("/")[1] if r["path"].startswith("wizard-work/") else "?"
|
lane = r["path"].split("/")[1] if r["path"].startswith("wizard-work/") else "?"
|
||||||
@@ -492,9 +815,16 @@ def main():
|
|||||||
mark = ",".join(sorted({x["verdict"] for x in rs}))
|
mark = ",".join(sorted({x["verdict"] for x in rs}))
|
||||||
print("%-9s %3d row(s) %s" % (lane, len(rs), mark))
|
print("%-9s %3d row(s) %s" % (lane, len(rs), mark))
|
||||||
for r in sorted(rs, key=lambda x: x["path"]):
|
for r in sorted(rs, key=lambda x: x["path"]):
|
||||||
if r["verdict"] is not PASS:
|
if r["verdict"] != PASS:
|
||||||
print(" %-8s rc %-4s %s %s" % (r["verdict"], r["rc"] or "-",
|
print(" %-8s rc %-4s %s %s" % (r["verdict"], r["rc"] or "-",
|
||||||
r["path"], r["note"][:90]))
|
r["path"], r["note"][:90]))
|
||||||
|
elif r["host"]:
|
||||||
|
print(" %-8s %-8s %s reads %s" % ("PASS*", "rc 0", r["path"],
|
||||||
|
r["host"]))
|
||||||
|
pinned = sum(1 for r in rows if r["verdict"] == PASS and r["host"])
|
||||||
|
unbound = sum(1 for r in rows if not r["blob"])
|
||||||
|
print("# PASS* (host-pinned) %d, rows without a blob %d of %d"
|
||||||
|
% (pinned, unbound, len(rows)))
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Q65 runner ledger -- one row per committed verify_*.py the runner
|
# Q65 runner ledger -- one row per committed verify_*.py the runner
|
||||||
# executes, with the exit code it actually returned at the stated rev.
|
# executes, with the exit code it actually returned at the stated rev.
|
||||||
# row: <class>\t<path>\t<verdict>\t<rc>\t<seconds>\t<note>
|
# row: <class>\t<path>\t<verdict>\t<rc>\t<seconds>\t<note>\t<blob>\t<host>
|
||||||
# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 /
|
# verdicts: PASS rc0 / FAIL rc1 / REFUSED rc2 / OTHER_RC >2 /
|
||||||
# TIMEOUT, ERROR, MISSING (no rc to grade).
|
# TIMEOUT, ERROR, MISSING (no rc to grade).
|
||||||
# class is the Q63 ledger class the file came from at emit time.
|
# class is the Q63 ledger class the file came from at emit time.
|
||||||
@@ -8,84 +8,92 @@
|
|||||||
# grades the bookkeeping, --check --strict makes any non-PASS fatal.
|
# grades the bookkeeping, --check --strict makes any non-PASS fatal.
|
||||||
# timeout is PER ROW: a TIMEOUT row names its own window in the note, so
|
# timeout is PER ROW: a TIMEOUT row names its own window in the note, so
|
||||||
# a --merge cannot leave a stale single figure standing for rows run
|
# a --merge cannot leave a stale single figure standing for rows run
|
||||||
# under a different one (the defect this line replaces).
|
# under a different one (the defect this line replaces). --windows
|
||||||
|
# names one window per path; --timeout is the fallback for unlisted.
|
||||||
|
# blob is the git blob id this rc was measured against: --check recomputes
|
||||||
|
# it at its own --rev and a mismatch is DRIFT, so an edited verifier
|
||||||
|
# cannot keep a PASS it no longer earns.
|
||||||
|
# host is a static host-pin probe (L<line>:<root>), empty when clean: a
|
||||||
|
# non-docstring string constant naming an absolute host root. Such a row
|
||||||
|
# is printed PASS* and excluded from the graded-clean figure.
|
||||||
# Regenerate: q65_runner.py --emit
|
# Regenerate: q65_runner.py --emit
|
||||||
# rev: 1e9d3fb rows: 78 emitted: q65_runner.py
|
# rev: d45a5ed rows: 79 emitted: q65_runner.py
|
||||||
undeclared wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.32
|
undeclared wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.32 0f650718d98bf28f07be134d49803cc902bf96b7 L41:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 1.23
|
undeclared wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 1.23 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 L31:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_check_lint_dry_refactor.py FAIL 1 1.34
|
undeclared wizard-work/forge/verify_check_lint_dry_refactor.py FAIL 1 1.34 06a5c66c3f319d2fa7b4d9e2f73a2eafa7a99cad L78:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_checkin_check_dry_refactor.py PASS 0 0.43
|
undeclared wizard-work/forge/verify_checkin_check_dry_refactor.py PASS 0 0.43 d8ba2b221b80365165eb3cace77d1764651d4fba L39:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_chunker_refactor.py PASS 0 0.11
|
undeclared wizard-work/forge/verify_chunker_refactor.py PASS 0 0.11 c3c7b5631f4a572c7a630269c0b3ec06e32792fc
|
||||||
undeclared wizard-work/forge/verify_citation_hash_class_dry_refactor.py FAIL 1 131.04 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/c5b879e2036d4c909805dd1858587925/venv/bin/python3', '/root/.hermes/wi
|
undeclared wizard-work/forge/verify_citation_hash_class_dry_refactor.py FAIL 1 131.04 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/c5b879e2036d4c909805dd1858587925/venv/bin/python3', '/root/.hermes/wi 93abbc78e7f061d3e22483d8db148d434fb9caef L16:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_citation_hash_class_portable_default.py TIMEOUT 600.11 no finish in 600.0s
|
undeclared wizard-work/forge/verify_citation_hash_class_portable_default.py FAIL 1 333.13 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/c5b879e2036d4c909805dd1858587925/venv/bin/python3', '/root/.hermes/pr 81c481978bb3c8bdd51a0c3fdb038ce385bfc898 L57:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_citation_resolve_check_dry_refactor.py FAIL 1 17.19
|
undeclared wizard-work/forge/verify_citation_resolve_check_dry_refactor.py FAIL 1 17.19 cd325ddd210d5fa235024780b1737f16bb15b352 L29:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_doc_census_dry_refactor.py TIMEOUT 90.31 no finish in 90.0s
|
undeclared wizard-work/forge/verify_doc_census_dry_refactor.py FAIL 1 25.97 52a9702aa0e015bfeba92fb7e6efba4caece56d8 L57:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_doc_command_report_dry.py FAIL 1 35.77
|
undeclared wizard-work/forge/verify_doc_command_report_dry.py FAIL 1 35.77 7a2f42a5b95d35d7b16e50261a7dd72d39eccbce L44:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 22.04
|
undeclared wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 22.04 8d01677402c9ac02afb044c0cbdfa5a52a7d279a L64:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_landing_check_refactor.py PASS 0 6.12
|
undeclared wizard-work/forge/verify_landing_check_refactor.py PASS 0 6.12 6aa282dade614e3e55874adc3e37cf7c01313f3e L62:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 0.52 FileNotFoundError: [Errno 2] No such file or directory: '/root/.hermes/profiles/volt/cache/scratch/pc_verify_snyuxz7t'
|
undeclared wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 0.52 FileNotFoundError: [Errno 2] No such file or directory: '/root/.hermes/profiles/volt/cache/scratch/pc_verify_snyuxz7t' ab35c13f9565e9eebbc80a210001ec37a002a267 L58:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_pdf_refactor.py PASS 0 7.75
|
undeclared wizard-work/forge/verify_pdf_refactor.py PASS 0 7.75 44ce76d39b69a7d6798af521f958370bf30ad2b1 L41:/root/.hermes
|
||||||
undeclared wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 16.94
|
undeclared wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 16.94 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 L51:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 7.31
|
undeclared wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 7.31 c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 L33:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_roundtrip_refactor.py PASS 0 0.78
|
undeclared wizard-work/forge/verify_roundtrip_refactor.py PASS 0 0.78 0f1d7ea9d5e542c8cb70c8e3def85ba684641b2c L21:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_selftest_sweep_dry_refactor.py FAIL 1 0.52
|
undeclared wizard-work/forge/verify_selftest_sweep_dry_refactor.py FAIL 1 0.52 6f5e1621139c9ab98ec5b25ce03bda88a73cf9cf L63:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_selftest_sweep_refactor.py TIMEOUT 90.09 no finish in 90.0s
|
undeclared wizard-work/forge/verify_selftest_sweep_refactor.py FAIL 1 121.94 subprocess.TimeoutExpired: Command '['/root/.hermes/installs/76f6e7e5145b5f39/environments/c5b879e2036d4c909805dd1858587925/venv/bin/python3', 'tools/selftest_s cc8782b4edfcd08eb5ed69f06308c1185ffb41e8 L33:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_served_drift_summarise_dry.py PASS 0 11.92
|
undeclared wizard-work/forge/verify_served_drift_summarise_dry.py PASS 0 11.92 de5a49160b2c2a8f4759e0607a0f0c7b52b3bec1 L48:/root/troupe-standup
|
||||||
gate-run wizard-work/forge/verify_smell_scan_refactor.py PASS 0 0.51
|
gate-run wizard-work/forge/verify_smell_scan_refactor.py PASS 0 0.51 499c8e000359e8ef3486a89031673059d471178c L26:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_tool_map_dry_refactor.py PASS 0 54.48
|
undeclared wizard-work/forge/verify_tool_map_dry_refactor.py PASS 0 54.48 de1d9b7a65cfaaacf029b048228806acbf2f817c L32:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_write_probe_dry_refactor.py FAIL 1 27.08
|
undeclared wizard-work/forge/verify_write_probe_dry_refactor.py FAIL 1 27.08 ef4d8c00e4a84495e1612bec4734a366d4869b67 L53:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_arxiv_csai.py PASS 0 0.19
|
undeclared wizard-work/ledger/verify_arxiv_csai.py PASS 0 0.19 872574acb23924497991a9ce8fb95be4d784689a L15:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_arxiv_csai_fresh.py PASS 0 0.19
|
undeclared wizard-work/ledger/verify_arxiv_csai_fresh.py PASS 0 0.19 1a5acdde626e9886cb8d9253174de65d780911ff L16:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_arxiv_csv.py PASS 0 0.10
|
undeclared wizard-work/ledger/verify_arxiv_csv.py PASS 0 0.10 cc898af42ea87515111ee0f15fa2f2818382befd L24:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_arxiv_fresh.py PASS 0 0.09
|
undeclared wizard-work/ledger/verify_arxiv_fresh.py PASS 0 0.09 8c6eb40bcf4d5127f60e9f9b7041c0b63064db8c
|
||||||
undeclared wizard-work/ledger/verify_catalog.py PASS 0 0.08
|
undeclared wizard-work/ledger/verify_catalog.py PASS 0 0.08 d08c5d389ece6b9906a7315849dfe83f99f28f41 L13:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_gitea_repos.py FAIL 1 0.23
|
undeclared wizard-work/ledger/verify_gitea_repos.py FAIL 1 0.23 73ef858149be5fd7e9ec58056435c3bd31f9419c L16:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_github_python_repos.py FAIL 1 3.35
|
undeclared wizard-work/ledger/verify_github_python_repos.py FAIL 1 3.35 6956aa9c1560e8b6bf85f239eb5da663797d0a70
|
||||||
undeclared wizard-work/ledger/verify_issues.py PASS 0 0.09
|
undeclared wizard-work/ledger/verify_issues.py PASS 0 0.09 176fd33f05223aaaf789608b662c3d253f685422 L15:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_python_org_repos.py FAIL 1 3.27
|
undeclared wizard-work/ledger/verify_python_org_repos.py FAIL 1 3.27 da60445425c431a6779fb2c3d9a457c2f1de888d L18:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_stackchain_catalog.py PASS 0 0.11
|
undeclared wizard-work/ledger/verify_stackchain_catalog.py PASS 0 0.11 60aa1853a04b6340cc251e7e673350fdf28c9050 L16:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_users.py PASS 0 0.13
|
undeclared wizard-work/ledger/verify_users.py PASS 0 0.13 ef9b73c9cdd33e74bc7162dede636ca446413361 L16:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_workflow.py FAIL 1 0.12
|
undeclared wizard-work/ledger/verify_workflow.py FAIL 1 0.12 8f9a0ce47f5caca7add10677e6301621ba71d8c2 L11:/root/troupe-standup
|
||||||
undeclared wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py PASS 0 0.38
|
undeclared wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py PASS 0 0.38 d88e5d6066843c9d9243fa195acc52183de4a856 L22:/root/troupe-standup
|
||||||
undeclared wizard-work/null/diffkit/verify_realrepo.py PASS 0 0.26
|
undeclared wizard-work/null/diffkit/verify_realrepo.py PASS 0 0.26 1e610f00fa50e36e294eb185332c548efe2c1ee4 L32:/root/troupe-standup
|
||||||
undeclared wizard-work/sol/verify_pending_request_triage.py PASS 0 0.12
|
undeclared wizard-work/sol/verify_pending_request_triage.py PASS 0 0.12 2d059319d3da0514bc72247ad7d1393039e5405d L11:/root/troupe-standup
|
||||||
undeclared wizard-work/sol/verify_triage.py PASS 0 0.12
|
undeclared wizard-work/sol/verify_triage.py PASS 0 0.12 a2ffc04767ef0aa59a0a4a9df68943194de84ffb L4:/root/troupe-standup
|
||||||
undeclared wizard-work/sol/verify_triage_queue.py FAIL 1 0.16
|
undeclared wizard-work/sol/verify_triage_queue.py FAIL 1 0.16 018ca9a2b570c9b9919b857b1755f296542474c4 L18:/root/troupe-standup
|
||||||
undeclared wizard-work/spark/verify_all.py PASS 0 0.11
|
undeclared wizard-work/spark/verify_all.py PASS 0 0.11 41029ee9976f8a01144961a44a8ee133336d0423 L4:/root/troupe-standup
|
||||||
undeclared wizard-work/spark/verify_check.py PASS 0 0.11
|
undeclared wizard-work/spark/verify_check.py PASS 0 0.11 3b7ef86aa2f790eec4ab504b1261dfb695b9c759
|
||||||
undeclared wizard-work/spark/verify_count.py PASS 0 0.12
|
undeclared wizard-work/spark/verify_count.py PASS 0 0.12 640a65519f8116cc4ccc003c6ab4bb60fa16eb26 L5:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_demo.py PASS 0 0.10
|
undeclared wizard-work/spark/verify_demo.py PASS 0 0.10 7e4325553fc86aa30419c00f99d5c226d28750b3 L20:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_dir.py PASS 0 0.11
|
undeclared wizard-work/spark/verify_dir.py PASS 0 0.11 ec6280d194d416ebe8a8fbf681cd986fb41cad6d L23:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_dir_analyzer.py PASS 0 0.66
|
undeclared wizard-work/spark/verify_dir_analyzer.py PASS 0 0.66 89a49e7684d6f51ce6232bbab7bccf69ec6af401 L16:/root/troupe-standup
|
||||||
undeclared wizard-work/spark/verify_environment.py FAIL 1 0.10
|
undeclared wizard-work/spark/verify_environment.py FAIL 1 0.10 4f526b94ecdd6d9c084794eefd8b6291d3ab30c6 L22:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_fib.py PASS 0 0.07
|
undeclared wizard-work/spark/verify_fib.py PASS 0 0.07 163468d71cd7fea6c9e064b85d89da7bb7807c5d L24:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_formatter.py PASS 0 0.24
|
undeclared wizard-work/spark/verify_formatter.py PASS 0 0.24 0d2ef3a544f5ddae821e6ffbb29dfcfe27560854 L9:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_hermes.py FAIL 1 0.10 ModuleNotFoundError: No module named 'hermes_tools'
|
undeclared wizard-work/spark/verify_hermes.py FAIL 1 0.10 ModuleNotFoundError: No module named 'hermes_tools' 22c2f9e1ddd80f5c3688b8804ac00c6ea54db18b L23:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_hermes_path.py PASS 0 0.08
|
undeclared wizard-work/spark/verify_hermes_cli.py PASS 0 1.84 f3793430f6ab6ec0cc9c16cc28d8718ed97826e6 L55:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_inventory.py FAIL 1 0.09
|
undeclared wizard-work/spark/verify_hermes_path.py PASS 0 0.08 385362b75cfedf35202ff48a7703bbd2381eb4a0 L11:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_json.py PASS 0 0.07
|
undeclared wizard-work/spark/verify_inventory.py FAIL 1 0.09 02b54c2c98fe8eaaf5ebebfc3155405a312ca6d6 L11:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_lines.py PASS 0 0.06
|
undeclared wizard-work/spark/verify_json.py PASS 0 0.07 71b343f0b1f3302ddd761b61c1c829ddaa5d85e5 L4:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_lwc.py PASS 0 0.15
|
undeclared wizard-work/spark/verify_lines.py PASS 0 0.06 5be4cf7b5f1ef58cab6f6cc3b6a9db00b50a7fc5 L2:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_minimal.py PASS 0 0.08
|
undeclared wizard-work/spark/verify_lwc.py PASS 0 0.15 0ca330355ed9de0f8ae98efc50a40a726fb44d40 L12:/root/troupe-standup
|
||||||
undeclared wizard-work/spark/verify_push.py PASS 0 0.06
|
undeclared wizard-work/spark/verify_minimal.py PASS 0 0.08 2b1b18fc7ed7842925deb12ece9fa6fe681c3d2f L31:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_python_env.py PASS 0 0.07
|
undeclared wizard-work/spark/verify_push.py PASS 0 0.06 707bdcb80646a8848ab3db4be908a36a083c0d54 L3:/root/troupe-standup
|
||||||
undeclared wizard-work/spark/verify_report.py PASS 0 0.06
|
undeclared wizard-work/spark/verify_python_env.py PASS 0 0.07 17c1d5f6ab0bcd691d87d737c262a8cdec3ecb29 L22:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_result.py PASS 0 0.06
|
undeclared wizard-work/spark/verify_report.py PASS 0 0.06 0b555f165e95cc133db97e1e989a1578d2789112 L10:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_scratch_analysis.py PASS 0 0.07
|
undeclared wizard-work/spark/verify_result.py PASS 0 0.06 108832bc1239abf8f267afe7994492fa25338540 L10:/root/.hermes
|
||||||
gate-run wizard-work/spark/verify_skill_system.py FAIL 1 0.14 ModuleNotFoundError: No module named 'hermes_tools'
|
undeclared wizard-work/spark/verify_scratch_analysis.py PASS 0 0.07 8f8edfb7d4b308cd8f18af7499eba7525d25b2fa L13:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_skills.py PASS 0 0.07
|
gate-run wizard-work/spark/verify_skill_system.py FAIL 1 0.14 ModuleNotFoundError: No module named 'hermes_tools' 6739088b65208074b3a348642726a1c28982583a
|
||||||
undeclared wizard-work/spark/verify_skills2.py PASS 0 0.07
|
undeclared wizard-work/spark/verify_skills.py PASS 0 0.07 be69f202a8238afd3da7bfe26fe38fc0d99d04d4 L5:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_skills3.py PASS 0 0.08
|
undeclared wizard-work/spark/verify_skills2.py PASS 0 0.07 9dd92b966a6e1c950c588282a5d685dba25d3001 L6:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_skills_fixed.py PASS 0 0.09
|
undeclared wizard-work/spark/verify_skills3.py PASS 0 0.08 364a78ffbe33353331f7dd780f44c53523e54e40 L6:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_snippets.py PASS 0 0.08
|
undeclared wizard-work/spark/verify_skills_fixed.py PASS 0 0.09 4bc11949dc38a67cb1590fcf913f308aae5b3975 L6:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_status.py PASS 0 0.30
|
undeclared wizard-work/spark/verify_snippets.py PASS 0 0.08 23d736539363ded0bc8542add8d6a9f927fb11a5 L11:/root/troupe-standup
|
||||||
undeclared wizard-work/spark/verify_sum.py PASS 0 0.06
|
undeclared wizard-work/spark/verify_status.py PASS 0 0.30 4c2928cdb2f2200b435c64183d326b5e5c10a1f0 L7:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_tool.py PASS 0 0.44
|
undeclared wizard-work/spark/verify_sum.py PASS 0 0.06 8faaade76bc38c1227daec010e4ae124344e9c29 L6:/root/.hermes
|
||||||
undeclared wizard-work/spark/verify_wordcount.py PASS 0 0.16
|
undeclared wizard-work/spark/verify_tool.py PASS 0 0.44 fd90c7f545ad0b3cb8c75be796f10b2a8a2d8830
|
||||||
undeclared wizard-work/sphinx/archive-gutenberg/verify_archive.py PASS 0 0.55
|
undeclared wizard-work/spark/verify_wordcount.py PASS 0 0.16 c4b7e15b56d084809628019cf41ad13d8c12db08 L46:/root/.hermes
|
||||||
undeclared wizard-work/sphinx/archive-index-2026-10-23/verify_index.py TIMEOUT 45.09 no finish in 45.0s
|
undeclared wizard-work/sphinx/archive-gutenberg/verify_archive.py PASS 0 0.55 36c966d37f083d2590a7a13693f02651b484c401
|
||||||
undeclared wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py FAIL 1 0.19
|
undeclared wizard-work/sphinx/archive-index-2026-10-23/verify_index.py FAIL 1 191.62 9d4088a373a636ca3418d8af5828dff57066d9b0 L32:/root/.hermes
|
||||||
undeclared wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py PASS 0 0.22
|
undeclared wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py FAIL 1 0.19 22ba0b4bbdca3180a136ef1e0fac538622e5db0d
|
||||||
undeclared wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py PASS 0 0.21
|
undeclared wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py PASS 0 0.22 39a5552b850096a4baf26b6bf0bad28db2db1008
|
||||||
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py TIMEOUT 120.10 no finish in 120.0s
|
undeclared wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py PASS 0 0.21 534af05a12fa9d3285384bae57bce527ed033425
|
||||||
undeclared wizard-work/sphinx/verify_mm_dossier.py PASS 0 4.30
|
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py FAIL 1 319.30 bed5df5c4a24c961a3c2bb564b299444a22d5189
|
||||||
|
undeclared wizard-work/sphinx/verify_mm_dossier.py PASS 0 4.30 4b015581816de5472d11577d661ce4e97f38773a L28:/root/.hermes
|
||||||
|
Reference in new issue
Block a user