volt: Q65 cycle 6's rc-probe leg lands (it was on disk, uncommitted, since 7cd4dd5) + forge's new verifier graded -- census 110 of 110, runner 91 rows, CHECK rc 0

Two facts in one instrument commit, stated rather than implied:

1. cycle 6 was never committed. `q65_rc_probe.py` and its 109-row TSV were
   untracked in the worktree, the CHECK's rc-probe legs existed only on disk,
   and the two ledgers carried 3+2 uncommitted rows. Measured before landing,
   not trusted: `q65_rc_probe.py --selftest` 25/25 and `--check` 0 faults at
   the worktree state; the state is now the one the gate reads.

2. cycle 7 is the file that reopened the census. forge's 6e8f577 adds
   `wizard-work/forge/verify_probe_forge_refactor.py` (9,890 B, blob
   0b9040ad68), invoked by nothing, rowed by
   nothing: `bash wizard-work/volt/CHECK` rc 1, `UNCLASSED ... 1 unclassed of
   110`, and `q63_verify_scan.py` `NEW_UNWIRED 1` -- both legs, two classes,
   one file. Graded by execution through the runner (PASS rc 0, 0.42s, host
   column empty so the row is `graded-clean`, not `PASS*`) and rc-probed
   (sensitive, moved at L67 after 21 sites were enumerated).

Re-emitted at 6e8f577: census 110 rows (gate-run 91, wired-invoke 11,
declared-one-off 8), runner ledger 91 rows (PASS 67, FAIL 24), rc-probe
110 rows (sensitive 59). The re-emit is not a pure append and the record
names the non-append hunk: `ledger/verify_gdp_per_capita.py` moved
`undeclared` -> `gate-run q65:PASS`, derived from a graded row that was in
the uncommitted ledger and is reproduced by the tree.
This commit is contained in:
volt committed 2026-10-25 17:01:51 +00:00
1 parent 6e8f577c8c
commit eb345839ac
12 files changed
+1464 -3

No files matched your search

+18 -1
View File
@@ -16,6 +16,21 @@
# have none -- all spark, recorded as findings, not repaired here; 14 of the
# runner ledger's 64 PASS rows are rows on such a file and cannot fail.
#
# Since cycle 6 it grades that same claim dynamically, and about a process that
# actually ran. `q65_rc_probe.py --emit` inverts each verifier's own comparisons
# one at a time in a detached worktree and asks whether the exit code moves;
# `--check` is the fast half run here: every row in q65_rc_probe.tsv must still
# match the blob at HEAD (STALE) and must not contradict its own class. A file
# cannot be given a row it did not earn, and a file that changed after its row
# was written cannot keep it.
#
# `inert` is the one class whose negative is unbounded by construction, so the
# sweep tries EVERY comparison site before writing it; a file stopped by
# `--exhaustive-cap` is `inert-partial`, and --check faults on either word used
# outside its coverage. This is cycle 6's own correction: a 3-of-11 sample wrote
# `inert` on forge/verify_chunker_refactor.py, whose rc moves at 3 of its 11
# sites (L123, L139, L142) that the sample did not pick.
#
# The root is resolved from $0, not written down. A gate that `cd`s to an
# absolute host path grades THAT tree from whatever checkout invokes it: at
# `66e5be1`, a clone with one verifier deleted printed `0 unclassed of 88` and
@@ -37,6 +52,8 @@ python3 wizard-work/volt/q65_runner.py --check --class gate-run,undeclared || ex
python3 wizard-work/volt/q63_verify_scan.py --repo . >/dev/null || exit 1
python3 wizard-work/volt/q65_no_verdict.py --selftest >/dev/null || exit 1
python3 wizard-work/volt/q65_no_verdict.py --repo . --check >/dev/null || exit 1
python3 wizard-work/volt/q65_rc_probe.py --selftest >/dev/null || exit 1
python3 wizard-work/volt/q65_rc_probe.py --repo . --check >/dev/null || exit 1
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean; no unrowed printer."
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean; no unrowed printer; rc-sensitivity ledger content-bound."
exit 0
+5 -1
View File
@@ -1,6 +1,6 @@
# Q63 verifier census -- one row per COMMITTED verify_*.py at the rev the
# scan read, class plus the evidence the class rests on.
# rev: 25b04d0 (the tree these classes were read at)
# rev: 6e8f577 (the tree these classes were read at)
# row: <class> TAB <path> TAB <evidence> (three tab-separated columns)
# class gate-run = q65_runner.py executed it and recorded an rc (Q65);
# class wired-invoke = a committed line invokes it (the ref is the evidence);
@@ -30,6 +30,7 @@ gate-run wizard-work/forge/verify_landing_check_refactor.py q65:PASS
declared-one-off wizard-work/forge/verify_path_claim_apply_baseline_refactor.py in-file declaration
gate-run wizard-work/forge/verify_path_claim_dry_refactor.py q65:FAIL
gate-run wizard-work/forge/verify_pdf_refactor.py q65:PASS
gate-run wizard-work/forge/verify_probe_forge_refactor.py q65:PASS
gate-run wizard-work/forge/verify_queue_append_audit_dry_refactor.py q65:FAIL
gate-run wizard-work/forge/verify_reply_check_dry_refactor.py q65:PASS
gate-run wizard-work/forge/verify_repro_bounds_refactor.py q65:PASS
@@ -42,6 +43,7 @@ gate-run wizard-work/forge/verify_smell_check_monotonic_refactor.py q65:PASS
gate-run wizard-work/forge/verify_smell_scan_refactor.py q65:PASS
gate-run wizard-work/forge/verify_tool_map_dry_refactor.py q65:PASS
gate-run wizard-work/forge/verify_write_probe_dry_refactor.py q65:FAIL
wired-invoke wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py HEAD:wizard-work/ledger/2026-10-25-git-history-live-b1edd09/CHECK:14:exec python3 "$DIR/verif...
wired-invoke wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py HEAD:wizard-work/ledger/2026-10-25-head-20-commits/CHECK:7:exec python3 verify_head_commits.p...
gate-run wizard-work/ledger/verify_arxiv_csai.py q65:PASS
gate-run wizard-work/ledger/verify_arxiv_csai_fresh.py q65:PASS
@@ -51,6 +53,7 @@ gate-run wizard-work/ledger/verify_catalog.py q65:PASS
wired-invoke wizard-work/ledger/verify_commits_check.py HEAD:wizard-work/ledger/CHECK:8:exec python3 "$OUT/verify_commits_check.py" "$@"
gate-run wizard-work/ledger/verify_dates.py q65:PASS
gate-run wizard-work/ledger/verify_gdp.py q65:PASS
gate-run wizard-work/ledger/verify_gdp_per_capita.py q65:PASS
gate-run wizard-work/ledger/verify_gitea_repos.py q65:FAIL
gate-run wizard-work/ledger/verify_github_python_repos.py q65:FAIL
declared-one-off wizard-work/ledger/verify_github_repos.py in-file declaration
@@ -116,5 +119,6 @@ gate-run wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py q
gate-run wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py q65:PASS
gate-run wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py q65:FAIL
wired-invoke wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py HEAD:wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/CHECK.py:173: p = subproc...
gate-run wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py q65:PASS
wired-invoke wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py HEAD:wizard-work/sphinx/figure-locator-llama3-2026-10-24/run_checks.sh:13:python3 "$DIR/verif...
gate-run wizard-work/sphinx/verify_mm_dossier.py q65:PASS
Can't render this file because it contains an unexpected character in line 46 and column 173.
@@ -0,0 +1,7 @@
rev1 da538ca5c0a4 blob b752e5467fb8
rev2 64733c9e977e blob 365068a8f0f5
ok A stale blob rc=1 expect=1 FAULT STALE wizard-work/x/verify_ctl.py: row graded blob b752e5467fb8, tree holds 365068a8f0f5
ok B rebound blob rc=0 expect=0 (no fault line)
ok C inert 1 of 3 sites rc=1 expect=1 FAULT CONTRADICTION wizard-work/x/verify_ctl.py: inert with valid=1 tried=1 sites=3 moved=None
ok D inert-partial 1/3 rc=0 expect=0 (no fault line)
4 of 4 controls behaved
@@ -0,0 +1,49 @@
dynamic dead-rc files (inert or no-comparison): 19 of 109 rows
class static runner sites/tried path
inert weak-subprocess PASS 2/2 wizard-work/ledger/verify_dates.py
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_all.py
inert no-verdict PASS 3/3 wizard-work/spark/verify_check.py
inert no-verdict PASS 1/1 wizard-work/spark/verify_count.py
nocomp no-verdict - 0/0 wizard-work/spark/verify_dirs.py
nocomp - PASS 0/0 wizard-work/spark/verify_env.py
inert no-verdict PASS 4/4 wizard-work/spark/verify_fib.py
inert - PASS 9/9 wizard-work/spark/verify_hermes_cli.py
inert - PASS 1/1 wizard-work/spark/verify_hermes_path.py
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_json.py
inert no-verdict PASS 1/1 wizard-work/spark/verify_lines.py
inert no-verdict PASS 3/3 wizard-work/spark/verify_minimal.py
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_push.py
inert no-verdict PASS 1/1 wizard-work/spark/verify_python_env.py
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_skills.py
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_skills2.py
inert no-verdict PASS 1/1 wizard-work/spark/verify_skills3.py
inert no-verdict PASS 1/1 wizard-work/spark/verify_skills_fixed.py
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_sum.py
static no-verdict (cycle 5): 15
dynamic dead rc (cycle 6) : 19
in both : 15
static only (rc moves) : 0 []
dynamic only (has a path) : 4 ['wizard-work/ledger/verify_dates.py', 'wizard-work/spark/verify_env.py', 'wizard-work/spark/verify_hermes_cli.py', 'wizard-work/spark/verify_hermes_path.py']
dead-rc files carrying a runner PASS row: 18
PASS wizard-work/ledger/verify_dates.py
PASS wizard-work/spark/verify_all.py
PASS wizard-work/spark/verify_check.py
PASS wizard-work/spark/verify_count.py
PASS wizard-work/spark/verify_env.py
PASS wizard-work/spark/verify_fib.py
PASS wizard-work/spark/verify_hermes_cli.py
PASS wizard-work/spark/verify_hermes_path.py
PASS wizard-work/spark/verify_json.py
PASS wizard-work/spark/verify_lines.py
PASS wizard-work/spark/verify_minimal.py
PASS wizard-work/spark/verify_push.py
PASS wizard-work/spark/verify_python_env.py
PASS wizard-work/spark/verify_skills.py
PASS wizard-work/spark/verify_skills2.py
PASS wizard-work/spark/verify_skills3.py
PASS wizard-work/spark/verify_skills_fixed.py
PASS wizard-work/spark/verify_sum.py
of those, PASS* (host-pinned): 17
+325
View File
@@ -0,0 +1,325 @@
q65_rc_probe -- rev 7cd4dd5d9226, 109 population file(s), work copy /root/.hermes/wizard-outputs/volt/rcprobe-work-c6
[1/109] wizard-work/beam/verify_toposort_none.py
L21 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=21 0.22s
[2/109] wizard-work/forge/verify_append_only_audit_dry_refactor.py
<unknown>:142: SyntaxWarning: "\u" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\u"? A raw string is also an option.
<unknown>:157: SyntaxWarning: "\u" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\u"? A raw string is also an option.
-> unprobed-baseline rc=1 sites=17 tried=0 valid=0 moved_at=- 2.16s
[3/109] wizard-work/forge/verify_append_only_owners_dry_refactor.py
L56 valid mutant rc=1
-> sensitive rc=0 sites=13 tried=1 valid=1 moved_at=56 0.65s
[4/109] wizard-work/forge/verify_census_row_delta_exit_contract.py
L46 valid mutant rc=1
-> sensitive rc=0 sites=23 tried=1 valid=1 moved_at=46 1.2000000000000002s
[5/109] wizard-work/forge/verify_check_lint_classes_dry.py
L80 valid mutant rc=1
-> sensitive rc=0 sites=15 tried=1 valid=1 moved_at=80 0.68s
[6/109] wizard-work/forge/verify_check_lint_dry_refactor.py
-> unprobed-baseline rc=1 sites=14 tried=0 valid=0 moved_at=- 1.22s
[7/109] wizard-work/forge/verify_checkin_check_dry_refactor.py
L103 valid mutant rc=1
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=103 0.9299999999999999s
[8/109] wizard-work/forge/verify_chunker_refactor.py
L58 invalid mutant (1)
L105 valid mutant rc=0
L150 valid mutant rc=0
L61 valid mutant rc=0
L64 valid mutant rc=0
L68 valid mutant rc=0
L101 valid mutant rc=0
L123 valid mutant rc=1
-> sensitive rc=0 sites=11 tried=8 valid=7 moved_at=123 0.8499999999999999s
[9/109] wizard-work/forge/verify_citation_hash_class_dry_refactor.py
-> unprobed-timeout rc=TIMEOUT sites=12 tried=0 valid=0 moved_at=- 90.08s
[10/109] wizard-work/forge/verify_citation_hash_class_portable_default.py
-> unprobed-window window 1800s exceeds --skip-over 300s
[11/109] wizard-work/forge/verify_citation_resolve_check_dry_refactor.py
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 15.52s
[12/109] wizard-work/forge/verify_doc_census_dry_refactor.py
-> unprobed-window window 1800s exceeds --skip-over 300s
[13/109] wizard-work/forge/verify_doc_command_report_dry.py
-> unprobed-baseline rc=1 sites=8 tried=0 valid=0 moved_at=- 7.96s
[14/109] wizard-work/forge/verify_gate_blame_dry_refactor.py
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 10.81s
[15/109] wizard-work/forge/verify_landing_check_refactor.py
L103 valid mutant rc=1
-> sensitive rc=0 sites=25 tried=1 valid=1 moved_at=103 3.92s
[16/109] wizard-work/forge/verify_path_claim_apply_baseline_refactor.py
-> unprobed-baseline rc=1 sites=15 tried=0 valid=0 moved_at=- 0.31s
[17/109] wizard-work/forge/verify_path_claim_dry_refactor.py
-> unprobed-baseline rc=1 sites=17 tried=0 valid=0 moved_at=- 0.55s
[18/109] wizard-work/forge/verify_pdf_refactor.py
L79 valid mutant rc=1
-> sensitive rc=0 sites=14 tried=1 valid=1 moved_at=79 7.27s
[19/109] wizard-work/forge/verify_queue_append_audit_dry_refactor.py
-> unprobed-baseline rc=1 sites=13 tried=0 valid=0 moved_at=- 7.81s
[20/109] wizard-work/forge/verify_reply_check_dry_refactor.py
L72 valid mutant rc=1
-> sensitive rc=0 sites=11 tried=1 valid=1 moved_at=72 3.0s
[21/109] wizard-work/forge/verify_repro_bounds_refactor.py
L69 valid mutant rc=1
-> sensitive rc=0 sites=17 tried=1 valid=1 moved_at=69 0.94s
[22/109] wizard-work/forge/verify_roundtrip_refactor.py
L39 valid mutant rc=1
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=39 1.26s
[23/109] wizard-work/forge/verify_scan_dry_refactor.py
L78 valid mutant rc=1
-> sensitive rc=0 sites=21 tried=1 valid=1 moved_at=78 6.3s
[24/109] wizard-work/forge/verify_selftest_sweep_dry_refactor.py
-> unprobed-baseline rc=1 sites=15 tried=0 valid=0 moved_at=- 0.25s
[25/109] wizard-work/forge/verify_selftest_sweep_refactor.py
-> unprobed-window window 1800s exceeds --skip-over 300s
[26/109] wizard-work/forge/verify_served_drift_summarise_dry.py
L100 valid mutant rc=1
-> sensitive rc=0 sites=16 tried=1 valid=1 moved_at=100 12.190000000000001s
[27/109] wizard-work/forge/verify_smell_check_monotonic_refactor.py
L64 valid mutant rc=1
-> sensitive rc=0 sites=16 tried=1 valid=1 moved_at=64 1.6800000000000002s
[28/109] wizard-work/forge/verify_smell_scan_refactor.py
L55 valid mutant rc=1
-> sensitive rc=0 sites=11 tried=1 valid=1 moved_at=55 1.0s
[29/109] wizard-work/forge/verify_tool_map_dry_refactor.py
L40 valid mutant rc=1
-> sensitive rc=0 sites=9 tried=1 valid=1 moved_at=40 49.33s
[30/109] wizard-work/forge/verify_write_probe_dry_refactor.py
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 23.35s
[31/109] wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py
L49 valid mutant rc=0
L90 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=2 valid=2 moved_at=90 3.45s
[32/109] wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py
L122 valid mutant rc=1
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=122 0.69s
[33/109] wizard-work/ledger/verify_arxiv_csai.py
L37 valid mutant rc=1
-> sensitive rc=0 sites=14 tried=1 valid=1 moved_at=37 0.49s
[34/109] wizard-work/ledger/verify_arxiv_csai_fresh.py
L42 valid mutant rc=1
-> sensitive rc=0 sites=18 tried=1 valid=1 moved_at=42 0.47s
[35/109] wizard-work/ledger/verify_arxiv_csv.py
L24 valid mutant rc=0
L29 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=2 valid=2 moved_at=29 0.33999999999999997s
[36/109] wizard-work/ledger/verify_arxiv_fresh.py
L48 valid mutant rc=1
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=48 0.19s
[37/109] wizard-work/ledger/verify_catalog.py
L30 invalid mutant (1)
L77 valid mutant rc=1
-> sensitive rc=0 sites=14 tried=2 valid=1 moved_at=77 0.39s
[38/109] wizard-work/ledger/verify_commits_check.py
L122 valid mutant rc=1
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=122 0.58s
[39/109] wizard-work/ledger/verify_dates.py
L26 valid mutant rc=0
L36 valid mutant rc=0
-> inert rc=0 sites=2 tried=2 valid=2 moved_at=- 0.48s
[40/109] wizard-work/ledger/verify_gdp.py
L3 valid mutant rc=0
L27 valid mutant rc=1
-> sensitive rc=0 sites=12 tried=2 valid=2 moved_at=27 0.29000000000000004s
[41/109] wizard-work/ledger/verify_gdp_per_capita.py
L42 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=42 1.52s
[42/109] wizard-work/ledger/verify_gitea_repos.py
-> unprobed-baseline rc=1 sites=8 tried=0 valid=0 moved_at=- 0.24s
[43/109] wizard-work/ledger/verify_github_python_repos.py
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 2.86s
[44/109] wizard-work/ledger/verify_github_repos.py
L24 valid mutant rc=0
L107 valid mutant rc=1
-> sensitive rc=0 sites=14 tried=2 valid=2 moved_at=107 5.529999999999999s
[45/109] wizard-work/ledger/verify_issues.py
L48 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=48 0.22s
[46/109] wizard-work/ledger/verify_python_org_repos.py
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 2.02s
[47/109] wizard-work/ledger/verify_stackchain_catalog.py
L105 valid mutant rc=1
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=105 0.16999999999999998s
[48/109] wizard-work/ledger/verify_users.py
L32 valid mutant rc=0
L40 valid mutant rc=1
-> sensitive rc=0 sites=6 tried=2 valid=2 moved_at=40 0.32s
[49/109] wizard-work/ledger/verify_wizards.py
-> unprobed-baseline rc=1 sites=19 tried=0 valid=0 moved_at=- 15.69s
[50/109] wizard-work/ledger/verify_workflow.py
-> unprobed-baseline rc=1 sites=4 tried=0 valid=0 moved_at=- 0.22s
[51/109] wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py
L57 valid mutant rc=0
L85 valid mutant rc=1
-> sensitive rc=0 sites=9 tried=2 valid=2 moved_at=85 1.47s
[52/109] wizard-work/null/diffkit/verify_realrepo.py
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 0.38s
[53/109] wizard-work/sol/verify_pending_request_triage.py
L27 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=27 0.19s
[54/109] wizard-work/sol/verify_sol_profile_triage.py
L37 valid mutant rc=1
-> sensitive rc=0 sites=5 tried=1 valid=1 moved_at=37 1.65s
[55/109] wizard-work/sol/verify_sol_triage.py
-> unprobed-baseline rc=1 sites=11 tried=0 valid=0 moved_at=- 0.09s
[56/109] wizard-work/sol/verify_triage.py
L9 valid mutant rc=1
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=9 0.18s
[57/109] wizard-work/sol/verify_triage_queue.py
-> unprobed-baseline rc=1 sites=11 tried=0 valid=0 moved_at=- 0.13s
[58/109] wizard-work/sol/verify_troupe_sol_health.py
L43 valid mutant rc=1
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=43 0.18s
[59/109] wizard-work/spark/verify_all.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.09s
[60/109] wizard-work/spark/verify_analyzer.py
L19 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=19 0.44s
[61/109] wizard-work/spark/verify_check.py
L6 valid mutant rc=0
L10 valid mutant rc=0
L11 valid mutant rc=0
-> inert rc=0 sites=3 tried=3 valid=3 moved_at=- 0.31s
[62/109] wizard-work/spark/verify_count.py
L11 valid mutant rc=0
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.19s
[63/109] wizard-work/spark/verify_demo.py
L32 valid mutant rc=1
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=32 0.2s
[64/109] wizard-work/spark/verify_dir.py
L15 valid mutant rc=1
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=15 0.15000000000000002s
[65/109] wizard-work/spark/verify_dir_analyzer.py
L19 valid mutant rc=1
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=19 2.7800000000000002s
[66/109] wizard-work/spark/verify_dirs.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.1s
[67/109] wizard-work/spark/verify_env.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.08s
[68/109] wizard-work/spark/verify_environment.py
-> unprobed-baseline rc=1 sites=3 tried=0 valid=0 moved_at=- 0.07s
[69/109] wizard-work/spark/verify_fib.py
L28 valid mutant rc=0
L31 valid mutant rc=0
L35 valid mutant rc=0
L30 valid mutant rc=0
-> inert rc=0 sites=4 tried=4 valid=4 moved_at=- 0.4100000000000001s
[70/109] wizard-work/spark/verify_formatter.py
L14 valid mutant rc=1
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=14 0.52s
[71/109] wizard-work/spark/verify_hermes.py
-> unprobed-baseline rc=1 sites=5 tried=0 valid=0 moved_at=- 0.13s
[72/109] wizard-work/spark/verify_hermes_cli.py
L31 valid mutant rc=0
L35 valid mutant rc=0
L64 valid mutant rc=0
L32 valid mutant rc=0
L33 valid mutant rc=0
L34 valid mutant rc=0
L36 valid mutant rc=0
L45 valid mutant rc=0
L50 valid mutant rc=0
-> inert rc=0 sites=9 tried=9 valid=9 moved_at=- 10.519999999999998s
[73/109] wizard-work/spark/verify_hermes_path.py
L24 valid mutant rc=0
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.16s
[74/109] wizard-work/spark/verify_inventory.py
-> unprobed-baseline rc=1 sites=2 tried=0 valid=0 moved_at=- 0.1s
[75/109] wizard-work/spark/verify_json.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.13s
[76/109] wizard-work/spark/verify_lines.py
L6 valid mutant rc=0
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.25s
[77/109] wizard-work/spark/verify_loc_report.py
L14 valid mutant rc=1
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=14 0.25s
[78/109] wizard-work/spark/verify_lwc.py
L18 valid mutant rc=1
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=18 0.56s
[79/109] wizard-work/spark/verify_minimal.py
L24 valid mutant rc=0
L41 valid mutant rc=0
L71 valid mutant rc=0
-> inert rc=0 sites=3 tried=3 valid=3 moved_at=- 0.68s
[80/109] wizard-work/spark/verify_push.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.14s
[81/109] wizard-work/spark/verify_python_env.py
L31 valid mutant rc=0
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.19s
[82/109] wizard-work/spark/verify_report.py
L11 valid mutant rc=1
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=11 0.2s
[83/109] wizard-work/spark/verify_result.py
L12 valid mutant rc=1
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=12 0.25s
[84/109] wizard-work/spark/verify_scratch_analysis.py
L17 valid mutant rc=1
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=17 0.19s
[85/109] wizard-work/spark/verify_skill_system.py
-> unprobed-baseline rc=1 sites=6 tried=0 valid=0 moved_at=- 0.16s
[86/109] wizard-work/spark/verify_skills.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.11s
[87/109] wizard-work/spark/verify_skills2.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.09s
[88/109] wizard-work/spark/verify_skills3.py
L41 valid mutant rc=0
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.24000000000000002s
[89/109] wizard-work/spark/verify_skills_fixed.py
L31 valid mutant rc=0
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.22s
[90/109] wizard-work/spark/verify_snippets.py
L25 valid mutant rc=1
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=25 0.19s
[91/109] wizard-work/spark/verify_status.py
L10 valid mutant rc=1
-> sensitive rc=0 sites=5 tried=1 valid=1 moved_at=10 0.63s
[92/109] wizard-work/spark/verify_sum.py
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.1s
[93/109] wizard-work/spark/verify_tool.py
L12 valid mutant rc=1
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=12 1.08s
[94/109] wizard-work/spark/verify_wordcount.py
L15 valid mutant rc=1
-> sensitive rc=0 sites=7 tried=1 valid=1 moved_at=15 0.46s
[95/109] wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py
L49 invalid mutant (1)
L88 valid mutant rc=1
-> sensitive rc=0 sites=17 tried=2 valid=1 moved_at=88 8.68s
[96/109] wizard-work/sphinx/archive-gutenberg/verify_archive.py
L25 valid mutant rc=0
L63 valid mutant rc=1
-> sensitive rc=0 sites=12 tried=2 valid=2 moved_at=63 1.2s
[97/109] wizard-work/sphinx/archive-index-2026-10-23/verify_index.py
-> unprobed-window window 1800s exceeds --skip-over 300s
[98/109] wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py
L57 valid mutant rc=1
-> sensitive rc=0 sites=44 tried=1 valid=1 moved_at=57 0.37s
[99/109] wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py
-> unprobed-baseline rc=1 sites=29 tried=0 valid=0 moved_at=- 4.96s
[100/109] wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py
L106 valid mutant rc=1
-> sensitive rc=0 sites=35 tried=1 valid=1 moved_at=106 13.06s
[101/109] wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py
-> unprobed-baseline rc=1 sites=2 tried=0 valid=0 moved_at=- 0.14s
[102/109] wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py
L61 valid mutant rc=1
-> sensitive rc=0 sites=10 tried=1 valid=1 moved_at=61 0.48s
[103/109] wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py
L44 valid mutant rc=1
-> sensitive rc=0 sites=19 tried=1 valid=1 moved_at=44 0.38s
[104/109] wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py
L93 valid mutant rc=1
-> sensitive rc=0 sites=22 tried=1 valid=1 moved_at=93 7.75s
[105/109] wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py
-> unprobed-window window 1800s exceeds --skip-over 300s
[106/109] wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py
L40 invalid mutant (1)
L250 valid mutant rc=1
-> sensitive rc=0 sites=69 tried=2 valid=1 moved_at=250 9.260000000000002s
[107/109] wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py
-> unprobed-baseline rc=1 sites=10 tried=0 valid=0 moved_at=- 40.7s
[108/109] wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py
L101 valid mutant rc=1
-> sensitive rc=0 sites=21 tried=1 valid=1 moved_at=101 0.77s
[109/109] wizard-work/sphinx/verify_mm_dossier.py
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 3.49s
wrote wizard-work/volt/q65_rc_probe.tsv (109 rows)
work copy left at /root/.hermes/wizard-outputs/volt/rcprobe-work-c6
@@ -0,0 +1,20 @@
# The boundary of 'inert': comparison-inert is not 'cannot fail'.
# Substrate: spark/verify_hermes_path.py copied unchanged to a scratch dir, then
# ONE required path renamed to a directory that does not exist.
== control: unmodified copy at HEAD (blob 385362b75cfe)
OK: /root/.hermes/wizard-outputs/spark
OK: /root/troupe-standup/wizard-work/spark
OK: /root/.hermes/profiles/spark
rc=0
== falsified subject: /root/.hermes/profiles/spark -> /root/.hermes/profiles/spark-NO-SUCH-DIR-c6
OK: /root/.hermes/wizard-outputs/spark
OK: /root/troupe-standup/wizard-work/spark
MISSING: /root/.hermes/profiles/spark-NO-SUCH-DIR-c6
rc=1
== the other 16 comparison-inert rows cannot fail at all:
spark/verify_check.py (13 loc) computes factorial(5) and prints it -- no exit path, no assertion;
it carries a runner PASS row with an empty host column, i.e. a graded-clean PASS on a file that cannot fail.
15 static no-verdict printers (q65_no_verdict.py --check) + ledger/verify_dates.py (cycle 4: prints impossible dates, rc 0).
@@ -0,0 +1,24 @@
== wizard-work/forge/verify_chunker_refactor.py (6328 bytes, sha256 fc82c86f32ab)
sites=11 sample(spread,3)=[58, 105, 150] baseline rc=0
L58 rc=1 invalid <- in sample
L61 rc=0 -
L64 rc=0 -
L68 rc=0 -
L101 rc=0 -
L105 rc=0 - <- in sample
L123 rc=1 MOVED
L124 rc=0 -
L139 rc=1 MOVED
L142 rc=2 MOVED
L150 rc=0 - <- in sample
moved at 3 of 11 sites: [123, 139, 142]
sampled sites that moved: []
== wizard-work/ledger/verify_dates.py (1172 bytes, sha256 717e81b33a0f)
sites=2 sample(spread,3)=[26, 36] baseline rc=0
L26 rc=0 - <- in sample
L36 rc=0 - <- in sample
moved at 0 of 2 sites: []
sampled sites that moved: []
done
@@ -0,0 +1,89 @@
# provenance: bash wizard-work/volt/CHECK at 6e8f577 (2026-10-25T16:50:03+00:00), 2026-10-25T16:57:38Z
ok c1 PASS rc 0 is consistent
ok c2 PASS rc 1 is a fault
ok c3 TIMEOUT carrying an rc is a fault
ok c4 OTHER_RC rc 3 is consistent
ok c5 OTHER_RC rc 7 is consistent
ok c6 OTHER_RC rc 2 is a fault (that is REFUSED)
ok c7 ledger round-trips two rows
ok c8 a missing ledger is an error, not an empty pass
ok c9 an unknown verdict refuses the ledger
ok c10 a committed verifier in neither ledger is UNCLASSED
ok c11 a class row covers the same file
ok c12 a window row parses to (seconds, why)
ok c13 an unlisted path falls back to --timeout
ok c14 a malformed window row refuses instead of silently skipping
ok c15 a committed path has a blob id
ok c16 an absent path has no blob id
ok c17 the same path at a new commit is a different blob (DRIFT is detectable)
ok c18 a host root in a string VALUE is a host pin
ok c19 a host root in a docstring is prose, not a pin
ok c20 a PARSED PASS row on a host-independent verifier is graded clean
ok c21 a PARSED OTHER_RC row with rc 7 is consistent (not a fault)
ok c22 a class-narrowed tally sums to its own population, not the ledger
selftest: 22 cells, 22 passed, 0 failed
q65_runner -- rev HEAD, 90 population file(s), 90 ledger row(s) [ledger rev 7cd4dd5]
rows at emit : 90 (PASS 66, FAIL 24)
graded (has rc): 90 of 90
graded-clean : 10 of 90 (PASS on a host-independent verifier)
host-pinned : 56 PASS row(s) excluded as PASS* -- they read an absolute host root
PASS* wizard-work/forge/verify_append_only_owners_dry_refactor.py L41:/root/troupe-standup
PASS* wizard-work/forge/verify_census_row_delta_exit_contract.py L31:/root/troupe-standup
PASS* wizard-work/forge/verify_check_lint_classes_dry.py L44:/root/troupe-standup
PASS* wizard-work/forge/verify_checkin_check_dry_refactor.py L39:/root/troupe-standup
PASS* wizard-work/forge/verify_landing_check_refactor.py L62:/root/troupe-standup
PASS* wizard-work/forge/verify_pdf_refactor.py L41:/root/.hermes
PASS* wizard-work/forge/verify_reply_check_dry_refactor.py L33:/root/troupe-standup
PASS* wizard-work/forge/verify_repro_bounds_refactor.py L33:/root/troupe-standup
PASS* wizard-work/forge/verify_roundtrip_refactor.py L21:/root/troupe-standup
PASS* wizard-work/forge/verify_served_drift_summarise_dry.py L48:/root/troupe-standup
PASS* wizard-work/forge/verify_smell_check_monotonic_refactor.py L32:/root/troupe-standup
PASS* wizard-work/forge/verify_smell_scan_refactor.py L26:/root/troupe-standup
PASS* wizard-work/forge/verify_tool_map_dry_refactor.py L32:/root/troupe-standup
PASS* wizard-work/ledger/verify_arxiv_csai.py L15:/root/.hermes
PASS* wizard-work/ledger/verify_arxiv_csai_fresh.py L16:/root/troupe-standup
PASS* wizard-work/ledger/verify_arxiv_csv.py L24:/root/troupe-standup
PASS* wizard-work/ledger/verify_catalog.py L13:/root/.hermes
PASS* wizard-work/ledger/verify_dates.py L6:/root/troupe-standup
PASS* wizard-work/ledger/verify_gdp.py L3:/root/troupe-standup
PASS* wizard-work/ledger/verify_gdp_per_capita.py L19:/root/troupe-standup
PASS* wizard-work/ledger/verify_issues.py L15:/root/.hermes
PASS* wizard-work/ledger/verify_stackchain_catalog.py L16:/root/troupe-standup
PASS* wizard-work/ledger/verify_users.py L16:/root/troupe-standup
PASS* wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py L22:/root/troupe-standup
PASS* wizard-work/null/diffkit/verify_realrepo.py L32:/root/troupe-standup
PASS* wizard-work/sol/verify_pending_request_triage.py L11:/root/troupe-standup
PASS* wizard-work/sol/verify_triage.py L4:/root/troupe-standup
PASS* wizard-work/spark/verify_all.py L4:/root/troupe-standup
PASS* wizard-work/spark/verify_count.py L5:/root/.hermes
PASS* wizard-work/spark/verify_demo.py L20:/root/.hermes
PASS* wizard-work/spark/verify_dir.py L23:/root/.hermes
PASS* wizard-work/spark/verify_dir_analyzer.py L16:/root/troupe-standup
PASS* wizard-work/spark/verify_env.py L9:/root/.hermes
PASS* wizard-work/spark/verify_fib.py L24:/root/.hermes
PASS* wizard-work/spark/verify_formatter.py L9:/root/.hermes
PASS* wizard-work/spark/verify_hermes_cli.py L55:/root/.hermes
PASS* wizard-work/spark/verify_hermes_path.py L11:/root/.hermes
PASS* wizard-work/spark/verify_json.py L4:/root/.hermes
PASS* wizard-work/spark/verify_lines.py L2:/root/.hermes
PASS* wizard-work/spark/verify_lwc.py L12:/root/troupe-standup
PASS* wizard-work/spark/verify_minimal.py L31:/root/.hermes
PASS* wizard-work/spark/verify_push.py L3:/root/troupe-standup
PASS* wizard-work/spark/verify_python_env.py L22:/root/.hermes
PASS* wizard-work/spark/verify_report.py L10:/root/.hermes
PASS* wizard-work/spark/verify_result.py L10:/root/.hermes
PASS* wizard-work/spark/verify_scratch_analysis.py L13:/root/.hermes
PASS* wizard-work/spark/verify_skills.py L5:/root/.hermes
PASS* wizard-work/spark/verify_skills2.py L6:/root/.hermes
PASS* wizard-work/spark/verify_skills3.py L6:/root/.hermes
PASS* wizard-work/spark/verify_skills_fixed.py L6:/root/.hermes
PASS* wizard-work/spark/verify_snippets.py L11:/root/troupe-standup
PASS* wizard-work/spark/verify_status.py L7:/root/.hermes
PASS* wizard-work/spark/verify_sum.py L6:/root/.hermes
PASS* wizard-work/spark/verify_wordcount.py L46:/root/.hermes
PASS* wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py L59:/root/.hermes
PASS* wizard-work/sphinx/verify_mm_dossier.py L28:/root/.hermes
tree audit : 1 unclassed of 110 committed verifier(s) <-- named by neither ledger
UNCLASSED wizard-work/forge/verify_probe_forge_refactor.py: committed verify_*.py at HEAD, in no class row in q63_unwired_baseline.tsv and no ledger row
q65: 1 fault(s)
rc=1
+804
View File
@@ -0,0 +1,804 @@
#!/usr/bin/env python3
"""q65_rc_probe.py -- ask a committed verifier, by execution, whether its exit
code can move at all (Q65, volt lane, 2026-10-25, cycle 6).
WHY THIS EXISTS
Cycle 5 answered "does a verifier have a failure path" statically: an AST scan
for `raise` / `assert` / non-zero `exit` / `unittest.main` found 15 of 106
committed verifiers with none, 14 of them carrying a PASS row in the runner
ledger. Static presence is a proxy, and it has two blind spots the tool said
so about itself: an exit path inside a `try/except` that swallows it still
passes the scan, and a present exit path can be unreachable from any verdict.
This is the dynamic question instead, and it does not read the source for
intent. It perturbs the verifier's OWN decisions and watches the process:
flip one comparison operator at a time, re-run the file, and see whether the
exit code changes. If a verifier's rc is unchanged after its comparisons have
been inverted, its rc is not connected to what it measured -- no input could
have made that file say no. That is a stronger statement than "no raise in the
AST", because it is made about a process that actually ran.
The mutation is of the verifier, not of the subject. Conventionally, mutation
testing mutates the SUBJECT and asks whether the test still passes (a killed
mutant means the test discriminates). Here the subject stays exactly as it is
and the ASSERTION is mutated, which inverts the question into the one Q65
needs: does an inverted assertion reach the exit code? A verifier whose rc is
stable under its own inverted assertions has no verdict channel, whatever its
source looks like.
WHAT IT MEASURES -- one row per probed file, one class per row
sensitive baseline rc 0, and >=1 VALID mutant returned a different rc.
The rc is wired to a comparison; the file can fail.
inert baseline rc 0, >=1 valid mutant, every valid mutant returned
the baseline rc, AND every mutable comparison site in the file
was tried. The rc does not respond to any of the file's own
comparisons. This is the finding.
It is a statement about COMPARISONS, not about the world: a
verifier whose rc is `all(exists and is_dir ...)` over
`os.path.isdir` is comparison-inert and can still fail on a
falsified subject (measured: spark/verify_hermes_path.py with
one required path renamed -> rc 1). 3 of the 19 comparison-inert
rows at 7cd4dd5 are of that kind; the other 16 cannot fail at
all, 15 of them static no-verdict printers.
inert-partial the same, but the file has more sites than `--exhaustive-cap`
allowed, so the negative is bounded: nothing moved in the
first N sites, and sites N+1.. were never tried. Explicitly
not the same claim as `inert` -- see WHY THE CLASSES ARE SPLIT.
no-comparison baseline rc 0 and no single-operator comparison to mutate.
Nothing in the file is a decision the rc could carry.
unprobed-baseline the unmutated file already returned a non-zero rc: there is
no pass state to defend, so the question does not apply.
unprobed-timeout the unmutated file did not finish inside its window.
unprobed-mutant every mutant tried was invalid (see below), so nothing was
measured. Not a finding either way.
unprobed-window the verifier's own measurement window (`q65_row_windows.tsv`,
shared with the runner) exceeds `--skip-over`, so it was not
mutated. A 30-minute live-HTTP verifier is not inverted three
times inside a sweep; its rc is the runner's row.
unprobed-error the sweep could not touch the file (the work copy lost it
mid-run). Recorded as a row with its message, so a pruned
scratch directory leaves a readable ledger instead of a
traceback halfway through the population.
A MUTANT IS VALID when its run (a) compiled and (b) finished inside the window
and (c) did NOT print a traceback. Flipping `==` to `!=` can turn a type check
into a TypeError; an rc that moves because the mutant crashed says nothing
about whether a verdict reaches the exit code. Invalid mutants are counted and
discarded, and a file left with none is `unprobed-mutant`, not `inert`.
HOW MUCH IS TRIED -- deterministic spread, so the figure is reproducible
`--max-mutants` (default 3) sites are taken from the sorted list of mutable
comparison line numbers by even spacing, so a 76-site file is probed near its
beginning, middle and end rather than at its first three lines. No randomness:
the same rev and the same `--max-mutants` select the same sites.
If those first sites all return the baseline rc, the sample is NOT the
verdict: every remaining site is tried too, up to `--exhaustive-cap`
(default 40) per file; 0 means no cap. Only a file whose whole site list was
tried can be called `inert`; a file stopped by the cap is `inert-partial`.
WHY THE CLASSES ARE SPLIT -- a measured false positive, not a hypothetical
Cycle 6's first sweep wrote `inert` for `forge/verify_chunker_refactor.py`
from a 3-of-11 sample: sites 58, 105, 150 (one of which crashed). Swept
exhaustively in a scratch copy, that file's rc moves at 3 of its 11 sites
(L123 -> 1, L139 -> 1, L142 -> 2) and the sample happened to hit none of
them. The file can fail; the sample said it could not. `inert` is therefore
a statement about every comparison in the file or it is not made.
WHERE IT WRITES
`--emit` mutates a COPY, never this worktree. The copy is a detached `git
worktree` at `--rev` -- a real checkout, not an export, because a verifier
that reads git plumbing returns a non-zero rc in a bare export and that
non-zero would be the copy's defect graded as the verifier's baseline. The
tool refuses (rc 2) a `--work` equal to the repository root -- a gate that
edits the tree it grades is the defect reply 93 §1 already measured once.
Every mutant is written, run, and restored inside a `try/finally`; the file
hash is re-checked after each file and a mismatch is rc 1, not a silent pass.
`--emit` writes the TSV (`--tsv`) and appends progress as it goes, so a sweep
that is interrupted leaves a readable partial ledger rather than nothing.
`--check` is the fast half and what the lane's CHECK runs: it re-reads the TSV,
binds every row to the blob at `--rev` (a row graded a file that has since
changed is STALE), re-derives the class counts from the rows, and faults on a
row that contradicts its own class (a `sensitive` row with no moving line, an
`inert` row with no valid mutant, a `no-comparison` row that lists sites).
It does NOT require population coverage: that is the census's job
(`q63_verify_scan.py`) and the runner's, and duplicating it here would make
two gates red for one added file. It prints the coverage gap instead.
rc 0 the ledger is internally consistent and content-bound
rc 1 a fault it graded: STALE, MISSING, an unknown class, or a row that
contradicts its class
rc 2 it could not grade at all (no TSV, not a git repo, --work is the tree)
"""
import argparse
import ast
import hashlib
import json
import os
import subprocess
import sys
import tempfile
import time
HERE = os.path.dirname(os.path.abspath(__file__))
DEFAULT_TSV = os.path.join(HERE, "q65_rc_probe.tsv")
DEFAULT_REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
SCRATCH_ROOT = "/root/.hermes/profiles/volt/cache/scratch"
CLASSES = ("sensitive", "inert", "inert-partial", "no-comparison",
"unprobed-baseline", "unprobed-timeout", "unprobed-mutant",
"unprobed-window", "unprobed-error")
FLIP = {
ast.Eq: ast.NotEq, ast.NotEq: ast.Eq,
ast.Lt: ast.Gt, ast.Gt: ast.Lt,
ast.LtE: ast.GtE, ast.GtE: ast.LtE,
ast.In: ast.NotIn, ast.NotIn: ast.In,
ast.Is: ast.IsNot, ast.IsNot: ast.Is,
}
TRACEBACK = "Traceback (most recent call last)"
# ---------------------------------------------------------------- source side
def mutable_lines(src):
"""Line numbers of single-operator comparisons that FLIP can invert."""
try:
tree = ast.parse(src)
except SyntaxError:
return []
lines = set()
for node in ast.walk(tree):
if (isinstance(node, ast.Compare) and len(node.ops) == 1
and type(node.ops[0]) in FLIP):
lines.add(node.lineno)
return sorted(lines)
def spread(picks, n):
"""Even-spaced deterministic subset of a sorted list, n or fewer."""
if n <= 0 or not picks:
return []
if len(picks) <= n:
return list(picks)
if n == 1:
return [picks[0]]
step = (len(picks) - 1) / float(n - 1)
out = []
for i in range(n):
idx = int(round(i * step))
if picks[idx] not in out:
out.append(picks[idx])
return out
def mutate(src, line):
"""Return source with the comparison at `line` inverted, or None."""
tree = ast.parse(src)
hits = [n for n in ast.walk(tree)
if isinstance(n, ast.Compare) and n.lineno == line
and len(n.ops) == 1 and type(n.ops[0]) in FLIP]
if not hits:
return None
hits[0].ops[0] = FLIP[type(hits[0].ops[0])]()
return ast.unparse(ast.fix_missing_locations(tree)) + "\n"
# ---------------------------------------------------------------- process side
def run_file(path, timeout):
"""Run one verifier the way the ledger does: cwd is the file's directory."""
t0 = time.time()
try:
p = subprocess.run([sys.executable, os.path.basename(path)],
cwd=os.path.dirname(path) or ".",
capture_output=True, text=True, timeout=timeout)
return {"rc": p.returncode, "traceback": TRACEBACK in p.stderr,
"seconds": round(time.time() - t0, 2)}
except subprocess.TimeoutExpired:
return {"rc": "TIMEOUT", "traceback": False,
"seconds": round(time.time() - t0, 2)}
def read_windows(path):
"""path -> seconds, from the runner's per-row window table (may be absent)."""
table = {}
if not path or not os.path.isfile(path):
return table
with open(path, encoding="utf-8") as fh:
for line in fh:
line = line.rstrip("\n")
if not line or line.startswith("#"):
continue
cells = line.split("\t")
if len(cells) >= 2:
try:
table[cells[0]] = float(cells[1])
except ValueError:
continue
return table
def probe(path, max_mutants, timeout, log=None, exhaustive_cap=40):
"""Baseline, up to max_mutants sample sites, then every remaining site.
The sample is the cheap question; the exhaustive pass is what licenses the
word `inert`. A file stopped by `exhaustive_cap` is inert-partial.
"""
with open(path, encoding="utf-8") as fh:
src = fh.read()
before = hashlib.sha256(src.encode()).hexdigest()
sites = mutable_lines(src)
base = run_file(path, timeout)
row = {"path": path, "baseline_rc": base["rc"], "sites": len(sites),
"tried": 0, "valid": 0, "moved_at": "-", "seconds": base["seconds"],
"cls": None, "note": ""}
if base["rc"] == "TIMEOUT":
row["cls"] = "unprobed-timeout"
return row
if base["rc"] != 0:
row["cls"] = "unprobed-baseline"
row["note"] = "baseline rc %s" % base["rc"]
return row
if not sites:
row["cls"] = "no-comparison"
return row
tries = spread(sites, max_mutants)
if len(tries) < len(sites):
# The negative half of the class needs every site, not the sample.
rest = [s for s in sites if s not in tries]
cap = len(rest) if exhaustive_cap == 0 else max(0, exhaustive_cap - len(tries))
tries += spread(rest, cap)
for line in tries:
mutant = mutate(src, line)
if mutant is None:
continue
row["tried"] += 1
with open(path, "w", encoding="utf-8") as fh:
fh.write(mutant)
try:
got = run_file(path, timeout)
finally:
with open(path, "w", encoding="utf-8") as fh:
fh.write(src)
row["seconds"] += got["seconds"]
valid = got["rc"] != "TIMEOUT" and not got["traceback"]
if not valid:
if log:
log(" L%d invalid mutant (%s)" % (line, got["rc"]))
continue
row["valid"] += 1
if log:
log(" L%d valid mutant rc=%s" % (line, got["rc"]))
if got["rc"] != base["rc"]:
row["cls"] = "sensitive"
row["moved_at"] = line
break
if row["cls"] != "sensitive":
if not row["valid"]:
row["cls"] = "unprobed-mutant"
elif row["tried"] >= len(sites):
row["cls"] = "inert"
else:
row["cls"] = "inert-partial"
row["note"] = "%d of %d sites tried (cap %d)" % (
row["tried"], len(sites), exhaustive_cap)
after = hashlib.sha256(open(path, encoding="utf-8").read().encode()).hexdigest()
if after != before:
raise RuntimeError("restore failed for %s" % path)
return row
# ---------------------------------------------------------------- git plumbing
def tree_blobs(repo, rev):
out = subprocess.run(["git", "-C", repo, "ls-tree", "-r", rev],
capture_output=True, text=True)
if out.returncode != 0:
raise SystemExit("q65_rc_probe: cannot read tree at %s: %s"
% (rev, out.stderr.strip()))
blobs = {}
for line in out.stdout.splitlines():
meta, path = line.split("\t", 1)
parts = meta.split()
if len(parts) >= 3 and parts[1] == "blob":
blobs[path] = parts[2]
return blobs
def population(blobs):
return sorted(p for p in blobs
if os.path.basename(p).startswith("verify_")
and p.endswith(".py")
and p.startswith("wizard-work/"))
def strip_basename(path):
return path
# ---------------------------------------------------------------------- modes
def read_tsv(path):
rows, head = [], {}
with open(path, encoding="utf-8") as fh:
for line in fh:
line = line.rstrip("\n")
if not line.strip():
continue
if line.startswith("#"):
for key in ("rev", "rows", "committed"):
if line.startswith("# %s " % key):
head[key] = line.split(None, 2)[2].split()[0]
continue
cells = line.split("\t")
if len(cells) != 9:
raise ValueError("row has %d cells, want 9: %r" % (len(cells), line))
rows.append(dict(zip(
("cls", "path", "baseline_rc", "sites", "tried", "valid",
"moved_at", "blob", "seconds"), cells)))
return head, rows
def write_header(fh, rev, rows, committed):
fh.write("# Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.\n")
fh.write("# Written by q65_rc_probe.py --emit. A row states a measurement, not an\n")
fh.write("# exemption: --check faults on a row that contradicts its class or whose\n")
fh.write("# file has changed since (STALE).\n")
fh.write("# columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds\n")
fh.write("# rev %s\n" % rev)
fh.write("# rows %d\n" % rows)
fh.write("# committed %d\n" % committed)
def do_emit(args):
repo = os.path.abspath(args.repo)
rev = subprocess.run(["git", "-C", repo, "rev-parse", args.rev],
capture_output=True, text=True)
if rev.returncode != 0:
print("q65_rc_probe: not a git tree at %s" % repo, file=sys.stderr)
return 2
rev = rev.stdout.strip()
blobs = tree_blobs(repo, rev)
pop = population(blobs)
windows = read_windows(args.windows)
prev_rows = []
if args.merge and os.path.isfile(args.tsv):
try:
_, prev_rows = read_tsv(args.tsv)
except ValueError:
prev_rows = []
print("q65_rc_probe: merging into %d existing row(s) from %s"
% (len(prev_rows), args.tsv))
if args.only:
pop = [p for p in pop if args.only in p]
if args.limit:
pop = pop[:args.limit]
if args.work:
work = os.path.abspath(args.work)
if work == os.path.realpath(repo) or work == repo:
print("q65_rc_probe: --work is the repository itself; refusing to "
"mutate the tree this gate grades", file=sys.stderr)
return 2
else:
work = os.path.join(SCRATCH_ROOT, "rcprobe-%d" % int(time.time()))
if not os.path.isdir(work):
os.makedirs(work)
if not os.path.isdir(os.path.join(work, ".git")) and not os.path.exists(
os.path.join(work, ".git")):
# A real git worktree, not an export: a verifier that reads git plumbing
# (rev-parse, ls-tree, log) returns a non-zero rc in a bare export, and a
# non-zero BASELINE is a row with no pass state to defend -- the sweep
# would grade its own copy's defect instead of the verifier.
add = subprocess.run(["git", "-C", repo, "worktree", "add", "--detach",
work, rev], capture_output=True, text=True)
if add.returncode != 0:
print("q65_rc_probe: cannot add a worktree at %s: %s"
% (rev[:12], add.stderr.strip()), file=sys.stderr)
return 2
print("q65_rc_probe -- rev %s, %d population file(s), work copy %s"
% (rev[:12], len(pop), work))
rows = []
for i, rel in enumerate(pop, 1):
path = os.path.join(work, rel)
if not os.path.isfile(path):
rows.append({"cls": "unprobed-mutant", "path": rel, "baseline_rc": "-",
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
"blob": blobs.get(rel, "-")[:12], "seconds": 0.0,
"note": "absent from the work copy"})
continue
print("[%d/%d] %s" % (i, len(pop), rel))
window = windows.get(rel, args.timeout)
if window > args.skip_over:
row = {"cls": "unprobed-window", "path": rel, "baseline_rc": "-",
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
"seconds": 0.0, "note": "%ss window" % int(window)}
row["blob"] = blobs.get(rel, "-")[:12]
rows.append(row)
print(" -> %-17s window %ss exceeds --skip-over %ss"
% (row["cls"], int(window), int(args.skip_over)))
continue
try:
row = probe(path, args.max_mutants, window, log=lambda m: print(m),
exhaustive_cap=getattr(args, "exhaustive_cap", 40))
except (OSError, RuntimeError) as exc:
# A work copy that lost a file mid-sweep (a scratch pruner, a full
# disk) must be a row, not a traceback: the sweep's ledger stays
# readable and the row says what happened.
row = {"cls": "unprobed-error", "path": rel, "baseline_rc": "-",
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
"seconds": 0.0, "note": str(exc)[:120]}
row["blob"] = blobs.get(rel, "-")[:12]
rows.append(row)
print(" -> %-17s %s" % (row["cls"], str(exc)[:90]))
continue
row["blob"] = blobs.get(rel, "-")[:12]
row["path"] = rel
rows.append(row)
print(" -> %-17s rc=%s sites=%s tried=%s valid=%s moved_at=%s %ss"
% (row["cls"], row["baseline_rc"], row["sites"], row["tried"],
row["valid"], row["moved_at"], row["seconds"]))
with open(args.tsv, "w", encoding="utf-8") as fh:
write_header(fh, rev, len(rows), len(pop))
for r in rows:
fh.write("\t".join(str(r.get(k, "-")) for k in
("cls", "path", "baseline_rc", "sites", "tried",
"valid", "moved_at", "blob", "seconds")) + "\n")
with open(args.tsv, "w", encoding="utf-8") as fh:
merged = rows
if args.merge and prev_rows:
by_path = {r["path"]: r for r in prev_rows}
for r in rows:
by_path[r["path"]] = r
merged = [by_path[k] for k in sorted(by_path)]
write_header(fh, rev, len(merged), len(population(tree_blobs(repo, rev))))
for r in merged:
fh.write("\t".join(str(r.get(k, "-")) for k in
("cls", "path", "baseline_rc", "sites", "tried",
"valid", "moved_at", "blob", "seconds")) + "\n")
print("wrote %s (%d rows)" % (args.tsv, len(merged)))
print("work copy left at %s" % work)
return 0
def do_check(args):
if not os.path.isfile(args.tsv):
print("q65_rc_probe: no ledger at %s -- cannot grade" % args.tsv,
file=sys.stderr)
return 2
try:
head, rows = read_tsv(args.tsv)
except ValueError as exc:
print("q65_rc_probe: %s" % exc, file=sys.stderr)
return 1
repo = os.path.abspath(args.repo)
try:
blobs = tree_blobs(repo, args.rev)
except SystemExit as exc:
print(str(exc), file=sys.stderr)
return 2
faults = []
counts = {c: 0 for c in CLASSES}
for r in rows:
if r["cls"] not in CLASSES:
faults.append("UNKNOWN_CLASS %s: %r" % (r["path"], r["cls"]))
continue
counts[r["cls"]] += 1
if r["path"] not in blobs:
faults.append("MISSING %s: row in the ledger, absent at %s" % (r["path"], args.rev[:12]))
continue
if not blobs[r["path"]].startswith(r["blob"]):
faults.append("STALE %s: row graded blob %s, tree holds %s"
% (r["path"], r["blob"], blobs[r["path"]][:12]))
try:
moved = int(r["moved_at"]) if r["moved_at"] not in ("-", "") else None
tried, valid = int(r["tried"]), int(r["valid"])
sites = int(r["sites"]) if r["sites"] not in ("-", "") else None
except ValueError:
faults.append("BAD_CELL %s: non-numeric tried/valid/moved_at" % r["path"])
continue
if r["cls"] == "sensitive" and not moved:
faults.append("CONTRADICTION %s: sensitive with no moving line" % r["path"])
if r["cls"] == "inert" and not (valid >= 1 and sites is not None
and tried >= sites and moved is None):
faults.append("CONTRADICTION %s: inert with valid=%d tried=%s sites=%s moved=%s"
% (r["path"], valid, tried, sites, moved))
if r["cls"] == "inert-partial" and not (valid >= 1 and sites is not None
and tried < sites and moved is None):
faults.append("CONTRADICTION %s: inert-partial with valid=%d tried=%s "
"sites=%s moved=%s"
% (r["path"], valid, tried, sites, moved))
if r["cls"] == "no-comparison" and sites:
faults.append("CONTRADICTION %s: no-comparison but lists %d site(s)"
% (r["path"], sites))
if r["cls"] == "unprobed-mutant" and valid:
faults.append("CONTRADICTION %s: unprobed-mutant with %d valid mutant(s)"
% (r["path"], valid))
committed = population(blobs)
probed = {r["path"] for r in rows}
print("q65_rc_probe -- rev %s, %d row(s), %d committed verifier(s)"
% (args.rev[:12], len(rows), len(committed)))
for c in CLASSES:
print(" %-18s %d" % (c, counts[c]))
print(" rc-sensitive %d of %d probed of %d committed"
% (counts["sensitive"], counts["sensitive"] + counts["inert"]
+ counts["inert-partial"] + counts["no-comparison"], len(committed)))
gap = [p for p in committed if p not in probed]
if gap:
print(" not probed %d (population coverage is the census's leg, "
"not this one)" % len(gap))
if faults:
for f in faults:
print(" FAULT %s" % f)
print("q65_rc_probe: %d fault(s)" % len(faults))
return 1
print("q65_rc_probe: 0 fault(s) -- every row content-bound and consistent "
"with its class")
return 0
# -------------------------------------------------------------------- selftest
def selftest():
tmp = tempfile.mkdtemp(prefix="rcprobe-selftest-", dir=SCRATCH_ROOT)
cells = []
def cell(name, ok, detail=""):
cells.append((name, bool(ok), detail))
def mk(name, body):
d = os.path.join(tmp, name)
os.makedirs(d, exist_ok=True)
p = os.path.join(d, "verify_x.py")
with open(p, "w", encoding="utf-8") as fh:
fh.write(body)
return p
# c1 sensitive: the inverted assertion reaches sys.exit
p = mk("sensitive", "import sys\nx = 2\nif x != 2:\n sys.exit(0)\n"
"if x == 2:\n print('ok')\nelse:\n sys.exit(1)\n")
r = probe(p, 3, 20)
cell("c1 an inverted assertion that reaches sys.exit is sensitive",
r["cls"] == "sensitive" and r["moved_at"], str(r))
# c2 inert: comparisons whose result is only printed
p = mk("inert", "x = 2\ny = 3\nprint('x==2', x == 2)\nprint('y<x', y < x)\n")
r = probe(p, 3, 20)
cell("c2 comparisons that only print leave the rc unmoved (inert)",
r["cls"] == "inert" and r["valid"] >= 1, str(r))
# c3 no comparison at all
p = mk("nocomp", "print('all checks passed')\n")
r = probe(p, 3, 20)
cell("c3 a file with no comparison is no-comparison",
r["cls"] == "no-comparison", str(r))
# c4 baseline non-zero
p = mk("baseline", "import sys\nsys.exit(2)\n")
r = probe(p, 3, 20)
cell("c4 a non-zero baseline is unprobed-baseline, not a pass",
r["cls"] == "unprobed-baseline", str(r))
# c5 every mutant invalid (the flip reaches an unreachable branch and crashes)
p = mk("invalid", "x = None\nif x == None:\n print('none')\nelse:\n"
" print(x.attr)\n")
r = probe(p, 3, 20)
cell("c5 a mutant that crashes is invalid, and no valid mutant is not inert",
r["cls"] == "unprobed-mutant" and r["tried"] == 1 and r["valid"] == 0, str(r))
cell("c5b the crashed-mutant file was restored byte for byte",
open(p).read() == "x = None\nif x == None:\n print('none')\nelse:\n"
" print(x.attr)\n")
# c5c one invalid mutant beside a valid one: counted, discarded, still graded
p = mk("mixed", "x = None\ny = 1\nif x == None:\n print('none')\nelse:\n"
" print(x.attr)\nif y == 1:\n print('y')\n")
r = probe(p, 3, 20)
cell("c5c an invalid mutant is counted and discarded, not counted as a test",
r["cls"] == "inert" and r["tried"] == 2 and r["valid"] == 1, str(r))
# c5d the sample is not the verdict: a moving site outside the 3-site
# even-spaced sample is found by the exhaustive pass
body5d = ("import sys\n"
"x1 = 1\n"
"x2 = 2\n"
"x3 = 3\n"
"x4 = 4\n"
"x5 = 5\n"
"print(x1 == 1)\n"
"if x2 == 2:\n"
" sys.exit(0)\n"
"print(x3 == 3)\n"
"print(x4 == 4)\n"
"print(x5 == 5)\n"
"sys.exit(1)\n")
p = mk("outside_sample", body5d)
r = probe(p, 3, 20)
cell("c5d a moving site the 3-site sample misses is still found",
r["cls"] == "sensitive" and r["moved_at"] == 8 and r["sites"] == 5,
str(r))
cell("c5d2 the sample really did miss it (site 8 is not in spread(sites,3))",
8 not in spread(mutable_lines(body5d), 3),
str(spread(mutable_lines(body5d), 3)))
# c5e the same file under a cap that stops before the moving site is
# inert-partial, not inert and not sensitive
p = mk("capped", body5d)
r = probe(p, 3, 20, exhaustive_cap=3)
cell("c5e a cap that stops before the moving site is inert-partial",
r["cls"] == "inert-partial" and r["tried"] == 3 and r["sites"] == 5,
str(r))
# c6 restore on a sensitive file
p = mk("restore", "import sys\nx = 1\nif x == 1:\n sys.exit(0)\n")
before = open(p, "rb").read()
probe(p, 3, 20)
cell("c6 a probed file is restored byte for byte", open(p, "rb").read() == before)
# c7 spread is deterministic and even
picks = list(range(0, 100))
cell("c7 spread picks evenly and deterministically",
spread(picks, 3) == spread(picks, 3) and len(spread(picks, 3)) == 3
and spread(picks, 3)[0] == 0 and spread(picks, 3)[-1] == 99,
str(spread(picks, 3)))
cell("c7b spread never invents a site", all(s in picks for s in spread(picks, 7)))
# c8 tsv round-trip
tsv = os.path.join(tmp, "led.tsv")
rows = [
{"cls": "sensitive", "path": "a/verify_a.py", "baseline_rc": 0, "sites": 4,
"tried": 2, "valid": 2, "moved_at": 12, "blob": "abc123", "seconds": 0.1},
{"cls": "inert", "path": "a/verify_b.py", "baseline_rc": 0, "sites": 1,
"tried": 1, "valid": 1, "moved_at": "-", "blob": "def456", "seconds": 0.2},
]
with open(tsv, "w") as fh:
write_header(fh, "f" * 40, len(rows), 2)
for r in rows:
fh.write("\t".join(str(r[k]) for k in ("cls", "path", "baseline_rc",
"sites", "tried", "valid",
"moved_at", "blob", "seconds")) + "\n")
head, back = read_tsv(tsv)
cell("c8 the ledger round-trips its rows and its header",
len(back) == 2 and head.get("rows") == "2" and back[0]["moved_at"] == "12",
str(head))
# c9 a short row is refused, not silently skipped
bad = os.path.join(tmp, "short.tsv")
open(bad, "w").write("# rev " + "f" * 40 + "\n# rows 1\n# committed 1\n"
"inert\ta/verify_a.py\t0\t1\n")
try:
read_tsv(bad)
cell("c9 a short row is refused", False)
except ValueError:
cell("c9 a short row is refused", True)
# c10 --check refuses a --work that is the tree
ns = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=DEFAULT_TSV,
only=None, limit=0, work=os.path.realpath(DEFAULT_REPO),
max_mutants=1, timeout=1,
windows=os.path.join(HERE, "q65_row_windows.tsv"),
skip_over=300.0, merge=False)
cell("c10 --emit refuses --work equal to the repo root", do_emit(ns) == 2)
# c12 the window table parses, and a malformed row is skipped not fatal
wt = os.path.join(tmp, "win.tsv")
open(wt, "w").write("# path\ts\twhy\na/verify_a.py\t1800\tlive HTTP\n"
"b/verify_b.py\tnotanumber\tbroken\n")
w = read_windows(wt)
cell("c12 the window table parses a path and skips a malformed row",
w.get("a/verify_a.py") == 1800.0 and "b/verify_b.py" not in w, str(w))
# c13 a verifier over the skip-over is unprobed-window, and the emit still succeeds
one = os.path.join(tmp, "one.tsv")
ns3 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=one,
only="wizard-work/sphinx/archive-index-2026-10-23/verify_index.py",
limit=0, work=os.path.join(tmp, "work"),
max_mutants=1, timeout=1,
windows=os.path.join(HERE, "q65_row_windows.tsv"),
skip_over=1.0, merge=False)
rc = do_emit(ns3)
head2, rows2 = read_tsv(one)
cell("c13 a file whose window exceeds --skip-over is unprobed-window, rc 0",
rc == 0 and len(rows2) == 1 and rows2[0]["cls"] == "unprobed-window",
str(rows2))
# c15 --merge keeps a row this run did not probe
ns4 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=one,
only="wizard-work/spark/verify_sum.py",
limit=0, work=os.path.join(tmp, "work"),
max_mutants=1, timeout=30,
windows=os.path.join(HERE, "q65_row_windows.tsv"),
skip_over=300.0, merge=True)
rc = do_emit(ns4)
head3, rows3 = read_tsv(one)
kept = [r for r in rows3 if r["path"].endswith("archive-index-2026-10-23/verify_index.py")]
fresh = [r for r in rows3 if r["path"].endswith("spark/verify_sum.py")]
cell("c15 --merge keeps the unprobed row beside the new one",
rc == 0 and len(rows3) == 2 and kept and fresh, str(rows3))
# c14 a file the sweep cannot read raises (do_emit turns it into a row)
try:
probe(os.path.join(tmp, "absent", "verify_x.py"), 1, 5)
cell("c14 an unreadable file raises OSError, caught into a row", False)
except OSError:
cell("c14 an unreadable file raises OSError, caught into a row", True)
# c16 --check enforces the coverage half of the inert claim
blobs_now = tree_blobs(DEFAULT_REPO, "HEAD")
real = sorted(p for p in blobs_now
if p.startswith("wizard-work/")
and os.path.basename(p).startswith("verify_"))[0]
b12 = blobs_now[real][:12]
def _led(cls, tried, sites):
f = os.path.join(tmp, "chk-%s-%d-%d.tsv" % (cls, tried, sites))
with open(f, "w", encoding="utf-8") as fh:
write_header(fh, "HEAD", 1, 1)
fh.write("\t".join([cls, real, "0", str(sites), str(tried),
"1", "-", b12, "0.1"]) + "\n")
return do_check(argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=f))
cell("c16 inert with sites untried is a contradiction (rc 1)",
_led("inert", 2, 5) == 1)
cell("c16b inert with every site tried is consistent (rc 0)",
_led("inert", 5, 5) == 0)
cell("c16c inert-partial with sites untried is consistent (rc 0)",
_led("inert-partial", 2, 5) == 0)
cell("c16d inert-partial with every site tried is a contradiction (rc 1)",
_led("inert-partial", 5, 5) == 1)
# c11 --check on a missing ledger is rc 2 (cannot grade), not rc 0
ns2 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD",
tsv=os.path.join(tmp, "nope.tsv"))
cell("c11 a missing ledger is rc 2, not a pass", do_check(ns2) == 2)
ok = sum(1 for _, o, _ in cells if o)
for name, o, detail in cells:
print("%s %s%s" % ("ok " if o else "FAIL", name,
"" if o else " [%s]" % detail))
print("selftest: %d cells, %d passed, %d failed" % (len(cells), ok, len(cells) - ok))
return 0 if ok == len(cells) else 1
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--repo", default=DEFAULT_REPO)
ap.add_argument("--rev", default="HEAD")
ap.add_argument("--tsv", default=DEFAULT_TSV)
ap.add_argument("--work", help="copy dir to mutate (default: fresh scratch dir)")
ap.add_argument("--max-mutants", type=int, default=3)
ap.add_argument("--exhaustive-cap", type=int, default=40,
help="sites per file tried before an inert verdict is "
"downgraded to inert-partial; 0 = no cap")
ap.add_argument("--timeout", type=float, default=90.0,
help="window for a file the window table does not name")
ap.add_argument("--windows", default=os.path.join(HERE, "q65_row_windows.tsv"),
help="per-path measurement windows, shared with the runner")
ap.add_argument("--skip-over", type=float, default=300.0,
help="a verifier whose window exceeds this is not mutated; "
"grading it is the runner's row, not this sweep's")
ap.add_argument("--only", help="with --emit: only paths containing this substring")
ap.add_argument("--limit", type=int, default=0)
ap.add_argument("--emit", action="store_true")
ap.add_argument("--merge", action="store_true",
help="with --emit: keep rows for paths this run did not probe")
ap.add_argument("--check", action="store_true")
ap.add_argument("--selftest", action="store_true")
args = ap.parse_args()
if args.selftest:
return selftest()
if args.check:
return do_check(args)
if args.emit:
return do_emit(args)
ap.print_help()
return 2
if __name__ == "__main__":
sys.exit(main())
+118
View File
@@ -0,0 +1,118 @@
# Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.
# Written by q65_rc_probe.py --emit. A row states a measurement, not an
# exemption: --check faults on a row that contradicts its class or whose
# file has changed since (STALE).
# columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds
# rev 6e8f577c8cdd17ae7e6d5d2d20513a183ec19d94
# rows 110
# committed 110
sensitive wizard-work/beam/verify_toposort_none.py 0 4 1 1 21 a0df9ceb99a1 0.22
unprobed-baseline wizard-work/forge/verify_append_only_audit_dry_refactor.py 1 17 0 0 - 09d40fbd2ca0 2.16
sensitive wizard-work/forge/verify_append_only_owners_dry_refactor.py 0 13 1 1 56 0f650718d98b 0.65
sensitive wizard-work/forge/verify_census_row_delta_exit_contract.py 0 23 1 1 46 6afe76a665fa 1.2000000000000002
sensitive wizard-work/forge/verify_check_lint_classes_dry.py 0 15 1 1 80 62082e082a04 0.68
unprobed-baseline wizard-work/forge/verify_check_lint_dry_refactor.py 1 14 0 0 - 06a5c66c3f31 1.22
sensitive wizard-work/forge/verify_checkin_check_dry_refactor.py 0 8 1 1 103 d8ba2b221b80 0.9299999999999999
sensitive wizard-work/forge/verify_chunker_refactor.py 0 11 8 7 123 c3c7b5631f4a 0.8499999999999999
unprobed-timeout wizard-work/forge/verify_citation_hash_class_dry_refactor.py TIMEOUT 12 0 0 - 93abbc78e7f0 90.08
unprobed-window wizard-work/forge/verify_citation_hash_class_portable_default.py - - 0 0 - 81c481978bb3 0.0
unprobed-baseline wizard-work/forge/verify_citation_resolve_check_dry_refactor.py 1 9 0 0 - cd325ddd210d 15.52
unprobed-window wizard-work/forge/verify_doc_census_dry_refactor.py - - 0 0 - 52a9702aa0e0 0.0
unprobed-baseline wizard-work/forge/verify_doc_command_report_dry.py 1 8 0 0 - 7a2f42a5b95d 7.96
unprobed-baseline wizard-work/forge/verify_gate_blame_dry_refactor.py 1 9 0 0 - 8d01677402c9 10.81
sensitive wizard-work/forge/verify_landing_check_refactor.py 0 25 1 1 103 6aa282dade61 3.92
unprobed-baseline wizard-work/forge/verify_path_claim_apply_baseline_refactor.py 1 15 0 0 - 999da7274c3c 0.31
unprobed-baseline wizard-work/forge/verify_path_claim_dry_refactor.py 1 17 0 0 - ab35c13f9565 0.55
sensitive wizard-work/forge/verify_pdf_refactor.py 0 14 1 1 79 44ce76d39b69 7.27
sensitive wizard-work/forge/verify_probe_forge_refactor.py 0 21 1 1 67 0b9040ad6834 0.83
unprobed-baseline wizard-work/forge/verify_queue_append_audit_dry_refactor.py 1 13 0 0 - 0d5e5ef90a0d 7.81
sensitive wizard-work/forge/verify_reply_check_dry_refactor.py 0 11 1 1 72 c94ebf0e965e 3.0
sensitive wizard-work/forge/verify_repro_bounds_refactor.py 0 17 1 1 69 2394abeec257 0.94
sensitive wizard-work/forge/verify_roundtrip_refactor.py 0 8 1 1 39 0f1d7ea9d5e5 1.26
sensitive wizard-work/forge/verify_scan_dry_refactor.py 0 21 1 1 78 6fb7aa14b96e 6.3
unprobed-baseline wizard-work/forge/verify_selftest_sweep_dry_refactor.py 1 15 0 0 - 6f5e1621139c 0.25
unprobed-window wizard-work/forge/verify_selftest_sweep_refactor.py - - 0 0 - cc8782b4edfc 0.0
sensitive wizard-work/forge/verify_served_drift_summarise_dry.py 0 16 1 1 100 de5a49160b2c 12.190000000000001
sensitive wizard-work/forge/verify_smell_check_monotonic_refactor.py 0 16 1 1 64 8b8ccea5be0e 1.6800000000000002
sensitive wizard-work/forge/verify_smell_scan_refactor.py 0 11 1 1 55 499c8e000359 1.0
sensitive wizard-work/forge/verify_tool_map_dry_refactor.py 0 9 1 1 40 de1d9b7a65cf 49.33
unprobed-baseline wizard-work/forge/verify_write_probe_dry_refactor.py 1 12 0 0 - ef4d8c00e4a8 23.35
sensitive wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py 0 4 2 2 90 77048ae1e3f1 3.45
sensitive wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py 0 2 1 1 122 e12a07ca75fc 0.69
sensitive wizard-work/ledger/verify_arxiv_csai.py 0 14 1 1 37 872574acb239 0.49
sensitive wizard-work/ledger/verify_arxiv_csai_fresh.py 0 18 1 1 42 1a5acdde626e 0.47
sensitive wizard-work/ledger/verify_arxiv_csv.py 0 4 2 2 29 cc898af42ea8 0.33999999999999997
sensitive wizard-work/ledger/verify_arxiv_fresh.py 0 8 1 1 48 8c6eb40bcf4d 0.19
sensitive wizard-work/ledger/verify_catalog.py 0 14 2 1 77 d08c5d389ece 0.39
sensitive wizard-work/ledger/verify_commits_check.py 0 2 1 1 122 e84e0c7dd3ed 0.58
inert wizard-work/ledger/verify_dates.py 0 2 2 2 - dd3f643441c9 0.48
sensitive wizard-work/ledger/verify_gdp.py 0 12 2 2 27 ba577cd8ae05 0.29000000000000004
sensitive wizard-work/ledger/verify_gdp_per_capita.py 0 4 1 1 42 7f74b350c60e 1.52
unprobed-baseline wizard-work/ledger/verify_gitea_repos.py 1 8 0 0 - 73ef858149be 0.24
unprobed-baseline wizard-work/ledger/verify_github_python_repos.py 1 9 0 0 - 6956aa9c1560 2.86
sensitive wizard-work/ledger/verify_github_repos.py 0 14 2 2 107 269870d5e95a 5.529999999999999
sensitive wizard-work/ledger/verify_issues.py 0 4 1 1 48 176fd33f0522 0.22
unprobed-baseline wizard-work/ledger/verify_python_org_repos.py 1 9 0 0 - da60445425c4 2.02
sensitive wizard-work/ledger/verify_stackchain_catalog.py 0 8 1 1 105 60aa1853a04b 0.16999999999999998
sensitive wizard-work/ledger/verify_users.py 0 6 2 2 40 ef9b73c9cdd3 0.32
unprobed-baseline wizard-work/ledger/verify_wizards.py 1 19 0 0 - e08c8675a4e4 15.69
unprobed-baseline wizard-work/ledger/verify_workflow.py 1 4 0 0 - 8f9a0ce47f5c 0.22
sensitive wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py 0 9 2 2 85 d88e5d606684 1.47
unprobed-baseline wizard-work/null/diffkit/verify_realrepo.py 1 12 0 0 - 1e610f00fa50 0.38
sensitive wizard-work/sol/verify_pending_request_triage.py 0 4 1 1 27 2d059319d3da 0.19
sensitive wizard-work/sol/verify_sol_profile_triage.py 0 5 1 1 37 84386c791a61 1.65
unprobed-baseline wizard-work/sol/verify_sol_triage.py 1 11 0 0 - 63792506caf3 0.09
sensitive wizard-work/sol/verify_triage.py 0 6 1 1 9 a2ffc04767ef 0.18
unprobed-baseline wizard-work/sol/verify_triage_queue.py 1 11 0 0 - 018ca9a2b570 0.13
sensitive wizard-work/sol/verify_troupe_sol_health.py 0 8 1 1 43 4b1a409c9b28 0.18
no-comparison wizard-work/spark/verify_all.py 0 0 0 0 - 41029ee9976f 0.09
sensitive wizard-work/spark/verify_analyzer.py 0 4 1 1 19 af7609cfadc1 0.44
inert wizard-work/spark/verify_check.py 0 3 3 3 - 3b7ef86aa2f7 0.31
inert wizard-work/spark/verify_count.py 0 1 1 1 - 640a65519f81 0.19
sensitive wizard-work/spark/verify_demo.py 0 3 1 1 32 7e4325553fc8 0.2
sensitive wizard-work/spark/verify_dir.py 0 3 1 1 15 ec6280d194d4 0.15000000000000002
sensitive wizard-work/spark/verify_dir_analyzer.py 0 6 1 1 19 89a49e7684d6 2.7800000000000002
no-comparison wizard-work/spark/verify_dirs.py 0 0 0 0 - d2846e746204 0.1
no-comparison wizard-work/spark/verify_env.py 0 0 0 0 - d28af1c48c1c 0.08
unprobed-baseline wizard-work/spark/verify_environment.py 1 3 0 0 - 4f526b94ecdd 0.07
inert wizard-work/spark/verify_fib.py 0 4 4 4 - 163468d71cd7 0.4100000000000001
sensitive wizard-work/spark/verify_formatter.py 0 4 1 1 14 0d2ef3a544f5 0.52
unprobed-baseline wizard-work/spark/verify_hermes.py 1 5 0 0 - 22c2f9e1ddd8 0.13
inert wizard-work/spark/verify_hermes_cli.py 0 9 9 9 - f3793430f6ab 10.519999999999998
inert wizard-work/spark/verify_hermes_path.py 0 1 1 1 - 385362b75cfe 0.16
unprobed-baseline wizard-work/spark/verify_inventory.py 1 2 0 0 - 02b54c2c98fe 0.1
no-comparison wizard-work/spark/verify_json.py 0 0 0 0 - 71b343f0b1f3 0.13
inert wizard-work/spark/verify_lines.py 0 1 1 1 - 5be4cf7b5f1e 0.25
sensitive wizard-work/spark/verify_loc_report.py 0 8 1 1 14 8e8aa89896b3 0.25
sensitive wizard-work/spark/verify_lwc.py 0 3 1 1 18 0ca330355ed9 0.56
inert wizard-work/spark/verify_minimal.py 0 3 3 3 - 2b1b18fc7ed7 0.68
no-comparison wizard-work/spark/verify_push.py 0 0 0 0 - 707bdcb80646 0.14
inert wizard-work/spark/verify_python_env.py 0 1 1 1 - 17c1d5f6ab0b 0.19
sensitive wizard-work/spark/verify_report.py 0 3 1 1 11 0b555f165e95 0.2
sensitive wizard-work/spark/verify_result.py 0 2 1 1 12 108832bc1239 0.25
sensitive wizard-work/spark/verify_scratch_analysis.py 0 6 1 1 17 8f8edfb7d4b3 0.19
unprobed-baseline wizard-work/spark/verify_skill_system.py 1 6 0 0 - 6739088b6520 0.16
no-comparison wizard-work/spark/verify_skills.py 0 0 0 0 - be69f202a823 0.11
no-comparison wizard-work/spark/verify_skills2.py 0 0 0 0 - 9dd92b966a6e 0.09
inert wizard-work/spark/verify_skills3.py 0 1 1 1 - 364a78ffbe33 0.24000000000000002
inert wizard-work/spark/verify_skills_fixed.py 0 1 1 1 - 4bc11949dc38 0.22
sensitive wizard-work/spark/verify_snippets.py 0 3 1 1 25 23d736539363 0.19
sensitive wizard-work/spark/verify_status.py 0 5 1 1 10 4c2928cdb2f2 0.63
no-comparison wizard-work/spark/verify_sum.py 0 0 0 0 - 8faaade76bc3 0.1
sensitive wizard-work/spark/verify_tool.py 0 3 1 1 12 fd90c7f545ad 1.08
sensitive wizard-work/spark/verify_wordcount.py 0 7 1 1 15 c4b7e15b56d0 0.46
sensitive wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py 0 17 2 1 88 50644ea156a1 8.68
sensitive wizard-work/sphinx/archive-gutenberg/verify_archive.py 0 12 2 2 63 36c966d37f08 1.2
unprobed-window wizard-work/sphinx/archive-index-2026-10-23/verify_index.py - - 0 0 - 9d4088a373a6 0.0
sensitive wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py 0 44 1 1 57 b252edd8547d 0.37
unprobed-baseline wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py 1 29 0 0 - a15fab347cf6 4.96
sensitive wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py 0 35 1 1 106 3c4b48a678a8 13.06
unprobed-baseline wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py 1 2 0 0 - 22ba0b4bbdca 0.14
sensitive wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py 0 10 1 1 61 39a5552b8500 0.48
sensitive wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py 0 19 1 1 44 534af05a12fa 0.38
sensitive wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py 0 22 1 1 93 c1f03701bf9a 7.75
unprobed-window wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py - - 0 0 - bed5df5c4a24 0.0
sensitive wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py 0 69 2 1 250 95997f60d905 9.260000000000002
unprobed-baseline wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 1 10 0 0 - c106248aea18 40.7
sensitive wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py 0 21 1 1 101 4b656a6d1d45 0.77
unprobed-baseline wizard-work/sphinx/verify_mm_dossier.py 1 12 0 0 - 4b015581816d 3.49
1 # Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.
2 # Written by q65_rc_probe.py --emit. A row states a measurement, not an
3 # exemption: --check faults on a row that contradicts its class or whose
4 # file has changed since (STALE).
5 # columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds
6 # rev 6e8f577c8cdd17ae7e6d5d2d20513a183ec19d94
7 # rows 110
8 # committed 110
9 sensitive wizard-work/beam/verify_toposort_none.py 0 4 1 1 21 a0df9ceb99a1 0.22
10 unprobed-baseline wizard-work/forge/verify_append_only_audit_dry_refactor.py 1 17 0 0 - 09d40fbd2ca0 2.16
11 sensitive wizard-work/forge/verify_append_only_owners_dry_refactor.py 0 13 1 1 56 0f650718d98b 0.65
12 sensitive wizard-work/forge/verify_census_row_delta_exit_contract.py 0 23 1 1 46 6afe76a665fa 1.2000000000000002
13 sensitive wizard-work/forge/verify_check_lint_classes_dry.py 0 15 1 1 80 62082e082a04 0.68
14 unprobed-baseline wizard-work/forge/verify_check_lint_dry_refactor.py 1 14 0 0 - 06a5c66c3f31 1.22
15 sensitive wizard-work/forge/verify_checkin_check_dry_refactor.py 0 8 1 1 103 d8ba2b221b80 0.9299999999999999
16 sensitive wizard-work/forge/verify_chunker_refactor.py 0 11 8 7 123 c3c7b5631f4a 0.8499999999999999
17 unprobed-timeout wizard-work/forge/verify_citation_hash_class_dry_refactor.py TIMEOUT 12 0 0 - 93abbc78e7f0 90.08
18 unprobed-window wizard-work/forge/verify_citation_hash_class_portable_default.py - - 0 0 - 81c481978bb3 0.0
19 unprobed-baseline wizard-work/forge/verify_citation_resolve_check_dry_refactor.py 1 9 0 0 - cd325ddd210d 15.52
20 unprobed-window wizard-work/forge/verify_doc_census_dry_refactor.py - - 0 0 - 52a9702aa0e0 0.0
21 unprobed-baseline wizard-work/forge/verify_doc_command_report_dry.py 1 8 0 0 - 7a2f42a5b95d 7.96
22 unprobed-baseline wizard-work/forge/verify_gate_blame_dry_refactor.py 1 9 0 0 - 8d01677402c9 10.81
23 sensitive wizard-work/forge/verify_landing_check_refactor.py 0 25 1 1 103 6aa282dade61 3.92
24 unprobed-baseline wizard-work/forge/verify_path_claim_apply_baseline_refactor.py 1 15 0 0 - 999da7274c3c 0.31
25 unprobed-baseline wizard-work/forge/verify_path_claim_dry_refactor.py 1 17 0 0 - ab35c13f9565 0.55
26 sensitive wizard-work/forge/verify_pdf_refactor.py 0 14 1 1 79 44ce76d39b69 7.27
27 sensitive wizard-work/forge/verify_probe_forge_refactor.py 0 21 1 1 67 0b9040ad6834 0.83
28 unprobed-baseline wizard-work/forge/verify_queue_append_audit_dry_refactor.py 1 13 0 0 - 0d5e5ef90a0d 7.81
29 sensitive wizard-work/forge/verify_reply_check_dry_refactor.py 0 11 1 1 72 c94ebf0e965e 3.0
30 sensitive wizard-work/forge/verify_repro_bounds_refactor.py 0 17 1 1 69 2394abeec257 0.94
31 sensitive wizard-work/forge/verify_roundtrip_refactor.py 0 8 1 1 39 0f1d7ea9d5e5 1.26
32 sensitive wizard-work/forge/verify_scan_dry_refactor.py 0 21 1 1 78 6fb7aa14b96e 6.3
33 unprobed-baseline wizard-work/forge/verify_selftest_sweep_dry_refactor.py 1 15 0 0 - 6f5e1621139c 0.25
34 unprobed-window wizard-work/forge/verify_selftest_sweep_refactor.py - - 0 0 - cc8782b4edfc 0.0
35 sensitive wizard-work/forge/verify_served_drift_summarise_dry.py 0 16 1 1 100 de5a49160b2c 12.190000000000001
36 sensitive wizard-work/forge/verify_smell_check_monotonic_refactor.py 0 16 1 1 64 8b8ccea5be0e 1.6800000000000002
37 sensitive wizard-work/forge/verify_smell_scan_refactor.py 0 11 1 1 55 499c8e000359 1.0
38 sensitive wizard-work/forge/verify_tool_map_dry_refactor.py 0 9 1 1 40 de1d9b7a65cf 49.33
39 unprobed-baseline wizard-work/forge/verify_write_probe_dry_refactor.py 1 12 0 0 - ef4d8c00e4a8 23.35
40 sensitive wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py 0 4 2 2 90 77048ae1e3f1 3.45
41 sensitive wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py 0 2 1 1 122 e12a07ca75fc 0.69
42 sensitive wizard-work/ledger/verify_arxiv_csai.py 0 14 1 1 37 872574acb239 0.49
43 sensitive wizard-work/ledger/verify_arxiv_csai_fresh.py 0 18 1 1 42 1a5acdde626e 0.47
44 sensitive wizard-work/ledger/verify_arxiv_csv.py 0 4 2 2 29 cc898af42ea8 0.33999999999999997
45 sensitive wizard-work/ledger/verify_arxiv_fresh.py 0 8 1 1 48 8c6eb40bcf4d 0.19
46 sensitive wizard-work/ledger/verify_catalog.py 0 14 2 1 77 d08c5d389ece 0.39
47 sensitive wizard-work/ledger/verify_commits_check.py 0 2 1 1 122 e84e0c7dd3ed 0.58
48 inert wizard-work/ledger/verify_dates.py 0 2 2 2 - dd3f643441c9 0.48
49 sensitive wizard-work/ledger/verify_gdp.py 0 12 2 2 27 ba577cd8ae05 0.29000000000000004
50 sensitive wizard-work/ledger/verify_gdp_per_capita.py 0 4 1 1 42 7f74b350c60e 1.52
51 unprobed-baseline wizard-work/ledger/verify_gitea_repos.py 1 8 0 0 - 73ef858149be 0.24
52 unprobed-baseline wizard-work/ledger/verify_github_python_repos.py 1 9 0 0 - 6956aa9c1560 2.86
53 sensitive wizard-work/ledger/verify_github_repos.py 0 14 2 2 107 269870d5e95a 5.529999999999999
54 sensitive wizard-work/ledger/verify_issues.py 0 4 1 1 48 176fd33f0522 0.22
55 unprobed-baseline wizard-work/ledger/verify_python_org_repos.py 1 9 0 0 - da60445425c4 2.02
56 sensitive wizard-work/ledger/verify_stackchain_catalog.py 0 8 1 1 105 60aa1853a04b 0.16999999999999998
57 sensitive wizard-work/ledger/verify_users.py 0 6 2 2 40 ef9b73c9cdd3 0.32
58 unprobed-baseline wizard-work/ledger/verify_wizards.py 1 19 0 0 - e08c8675a4e4 15.69
59 unprobed-baseline wizard-work/ledger/verify_workflow.py 1 4 0 0 - 8f9a0ce47f5c 0.22
60 sensitive wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py 0 9 2 2 85 d88e5d606684 1.47
61 unprobed-baseline wizard-work/null/diffkit/verify_realrepo.py 1 12 0 0 - 1e610f00fa50 0.38
62 sensitive wizard-work/sol/verify_pending_request_triage.py 0 4 1 1 27 2d059319d3da 0.19
63 sensitive wizard-work/sol/verify_sol_profile_triage.py 0 5 1 1 37 84386c791a61 1.65
64 unprobed-baseline wizard-work/sol/verify_sol_triage.py 1 11 0 0 - 63792506caf3 0.09
65 sensitive wizard-work/sol/verify_triage.py 0 6 1 1 9 a2ffc04767ef 0.18
66 unprobed-baseline wizard-work/sol/verify_triage_queue.py 1 11 0 0 - 018ca9a2b570 0.13
67 sensitive wizard-work/sol/verify_troupe_sol_health.py 0 8 1 1 43 4b1a409c9b28 0.18
68 no-comparison wizard-work/spark/verify_all.py 0 0 0 0 - 41029ee9976f 0.09
69 sensitive wizard-work/spark/verify_analyzer.py 0 4 1 1 19 af7609cfadc1 0.44
70 inert wizard-work/spark/verify_check.py 0 3 3 3 - 3b7ef86aa2f7 0.31
71 inert wizard-work/spark/verify_count.py 0 1 1 1 - 640a65519f81 0.19
72 sensitive wizard-work/spark/verify_demo.py 0 3 1 1 32 7e4325553fc8 0.2
73 sensitive wizard-work/spark/verify_dir.py 0 3 1 1 15 ec6280d194d4 0.15000000000000002
74 sensitive wizard-work/spark/verify_dir_analyzer.py 0 6 1 1 19 89a49e7684d6 2.7800000000000002
75 no-comparison wizard-work/spark/verify_dirs.py 0 0 0 0 - d2846e746204 0.1
76 no-comparison wizard-work/spark/verify_env.py 0 0 0 0 - d28af1c48c1c 0.08
77 unprobed-baseline wizard-work/spark/verify_environment.py 1 3 0 0 - 4f526b94ecdd 0.07
78 inert wizard-work/spark/verify_fib.py 0 4 4 4 - 163468d71cd7 0.4100000000000001
79 sensitive wizard-work/spark/verify_formatter.py 0 4 1 1 14 0d2ef3a544f5 0.52
80 unprobed-baseline wizard-work/spark/verify_hermes.py 1 5 0 0 - 22c2f9e1ddd8 0.13
81 inert wizard-work/spark/verify_hermes_cli.py 0 9 9 9 - f3793430f6ab 10.519999999999998
82 inert wizard-work/spark/verify_hermes_path.py 0 1 1 1 - 385362b75cfe 0.16
83 unprobed-baseline wizard-work/spark/verify_inventory.py 1 2 0 0 - 02b54c2c98fe 0.1
84 no-comparison wizard-work/spark/verify_json.py 0 0 0 0 - 71b343f0b1f3 0.13
85 inert wizard-work/spark/verify_lines.py 0 1 1 1 - 5be4cf7b5f1e 0.25
86 sensitive wizard-work/spark/verify_loc_report.py 0 8 1 1 14 8e8aa89896b3 0.25
87 sensitive wizard-work/spark/verify_lwc.py 0 3 1 1 18 0ca330355ed9 0.56
88 inert wizard-work/spark/verify_minimal.py 0 3 3 3 - 2b1b18fc7ed7 0.68
89 no-comparison wizard-work/spark/verify_push.py 0 0 0 0 - 707bdcb80646 0.14
90 inert wizard-work/spark/verify_python_env.py 0 1 1 1 - 17c1d5f6ab0b 0.19
91 sensitive wizard-work/spark/verify_report.py 0 3 1 1 11 0b555f165e95 0.2
92 sensitive wizard-work/spark/verify_result.py 0 2 1 1 12 108832bc1239 0.25
93 sensitive wizard-work/spark/verify_scratch_analysis.py 0 6 1 1 17 8f8edfb7d4b3 0.19
94 unprobed-baseline wizard-work/spark/verify_skill_system.py 1 6 0 0 - 6739088b6520 0.16
95 no-comparison wizard-work/spark/verify_skills.py 0 0 0 0 - be69f202a823 0.11
96 no-comparison wizard-work/spark/verify_skills2.py 0 0 0 0 - 9dd92b966a6e 0.09
97 inert wizard-work/spark/verify_skills3.py 0 1 1 1 - 364a78ffbe33 0.24000000000000002
98 inert wizard-work/spark/verify_skills_fixed.py 0 1 1 1 - 4bc11949dc38 0.22
99 sensitive wizard-work/spark/verify_snippets.py 0 3 1 1 25 23d736539363 0.19
100 sensitive wizard-work/spark/verify_status.py 0 5 1 1 10 4c2928cdb2f2 0.63
101 no-comparison wizard-work/spark/verify_sum.py 0 0 0 0 - 8faaade76bc3 0.1
102 sensitive wizard-work/spark/verify_tool.py 0 3 1 1 12 fd90c7f545ad 1.08
103 sensitive wizard-work/spark/verify_wordcount.py 0 7 1 1 15 c4b7e15b56d0 0.46
104 sensitive wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py 0 17 2 1 88 50644ea156a1 8.68
105 sensitive wizard-work/sphinx/archive-gutenberg/verify_archive.py 0 12 2 2 63 36c966d37f08 1.2
106 unprobed-window wizard-work/sphinx/archive-index-2026-10-23/verify_index.py - - 0 0 - 9d4088a373a6 0.0
107 sensitive wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py 0 44 1 1 57 b252edd8547d 0.37
108 unprobed-baseline wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py 1 29 0 0 - a15fab347cf6 4.96
109 sensitive wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py 0 35 1 1 106 3c4b48a678a8 13.06
110 unprobed-baseline wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py 1 2 0 0 - 22ba0b4bbdca 0.14
111 sensitive wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py 0 10 1 1 61 39a5552b8500 0.48
112 sensitive wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py 0 19 1 1 44 534af05a12fa 0.38
113 sensitive wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py 0 22 1 1 93 c1f03701bf9a 7.75
114 unprobed-window wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py - - 0 0 - bed5df5c4a24 0.0
115 sensitive wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py 0 69 2 1 250 95997f60d905 9.260000000000002
116 unprobed-baseline wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 1 10 0 0 - c106248aea18 40.7
117 sensitive wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py 0 21 1 1 101 4b656a6d1d45 0.77
118 unprobed-baseline wizard-work/sphinx/verify_mm_dossier.py 1 12 0 0 - 4b015581816d 3.49
+1
View File
@@ -15,3 +15,4 @@ wizard-work/forge/verify_doc_census_dry_refactor.py 1800 runs tools/doc_census_c
wizard-work/forge/verify_selftest_sweep_refactor.py 1800 runs tools/selftest_sweep.py over every pinned command
wizard-work/sphinx/archive-index-2026-10-23/verify_index.py 1800 reads every file at three revisions through git plumbing
wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py 1800 live HTTP fetches, 3 retries at a 90s per-request timeout
wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 120 live HTTP: 1210 exact id_list lookups plus 3 listing pages; measured 19.87s at 5095e89, 6x headroom
1 # Q65 per-row measurement windows -- one window per path, because the cost of
15 wizard-work/forge/verify_selftest_sweep_refactor.py
16 wizard-work/sphinx/archive-index-2026-10-23/verify_index.py
17 wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py
18 wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py
+4 -1
View File
@@ -17,7 +17,7 @@
# non-docstring string constant naming an absolute host root. Such a row
# is printed PASS* and excluded from the graded-clean figure.
# Regenerate: q65_runner.py --emit
# rev: 25b04d0 rows: 88 emitted: q65_runner.py
# rev: 6e8f577 rows: 91 emitted: q65_runner.py
undeclared wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.32 0f650718d98bf28f07be134d49803cc902bf96b7 L41:/root/troupe-standup
undeclared wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 1.23 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 L31:/root/troupe-standup
gate-run wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.53 62082e082a0472546b415219a9180836ac88908b L44:/root/troupe-standup
@@ -33,6 +33,7 @@ undeclared wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 22.04 8d0
undeclared wizard-work/forge/verify_landing_check_refactor.py PASS 0 6.12 6aa282dade614e3e55874adc3e37cf7c01313f3e L62:/root/troupe-standup
undeclared wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 0.52 FileNotFoundError: [Errno 2] No such file or directory: '/root/.hermes/profiles/volt/cache/scratch/pc_verify_snyuxz7t' ab35c13f9565e9eebbc80a210001ec37a002a267 L58:/root/troupe-standup
undeclared wizard-work/forge/verify_pdf_refactor.py PASS 0 7.75 44ce76d39b69a7d6798af521f958370bf30ad2b1 L41:/root/.hermes
undeclared wizard-work/forge/verify_probe_forge_refactor.py PASS 0 0.42 0b9040ad683464475d77b58c744f16dfbdb8438e
undeclared wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 16.94 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 L51:/root/troupe-standup
undeclared wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 7.31 c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 L33:/root/troupe-standup
undeclared wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.35 2394abeec25726726edd91696520d788d72a5d88 L33:/root/troupe-standup
@@ -52,6 +53,7 @@ undeclared wizard-work/ledger/verify_arxiv_fresh.py PASS 0 0.09 8c6eb40bcf4d512
undeclared wizard-work/ledger/verify_catalog.py PASS 0 0.08 d08c5d389ece6b9906a7315849dfe83f99f28f41 L13:/root/.hermes
undeclared wizard-work/ledger/verify_dates.py PASS 0 0.13 dd3f643441c9fd985639c0ac7f250e05a6184aeb L6:/root/troupe-standup
undeclared wizard-work/ledger/verify_gdp.py PASS 0 0.10 ba577cd8ae0502f57bfd7b8a7dadd8ef54455dbd L3:/root/troupe-standup
undeclared wizard-work/ledger/verify_gdp_per_capita.py PASS 0 0.84 7f74b350c60ee7fd0e361f8d60a2e669ae703a97 L19:/root/troupe-standup
undeclared wizard-work/ledger/verify_gitea_repos.py FAIL 1 0.23 73ef858149be5fd7e9ec58056435c3bd31f9419c L16:/root/.hermes
undeclared wizard-work/ledger/verify_github_python_repos.py FAIL 1 3.35 6956aa9c1560e8b6bf85f239eb5da663797d0a70
undeclared wizard-work/ledger/verify_issues.py PASS 0 0.09 176fd33f05223aaaf789608b662c3d253f685422 L15:/root/.hermes
@@ -105,4 +107,5 @@ undeclared wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.
undeclared wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py PASS 0 0.21 534af05a12fa9d3285384bae57bce527ed033425
wired-invoke wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py PASS 0 2.28 c1f03701bf9af0018e7aac9a567255dd8218488d
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py FAIL 1 837.56 bed5df5c4a24c961a3c2bb564b299444a22d5189
undeclared wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py PASS 0 19.87 c106248aea1856d29d987497af4fe95dfe6d9768 L59:/root/.hermes
undeclared wizard-work/sphinx/verify_mm_dossier.py PASS 0 4.30 4b015581816de5472d11577d661ce4e97f38773a L28:/root/.hermes
1 # Q65 runner ledger -- one row per committed verify_*.py the runner
17 # non-docstring string constant naming an absolute host root. Such a row
18 # is printed PASS* and excluded from the graded-clean figure.
19 # Regenerate: q65_runner.py --emit
20 # rev: 25b04d0 rows: 88 emitted: q65_runner.py # rev: 6e8f577 rows: 91 emitted: q65_runner.py
21 undeclared
22 undeclared
23 gate-run
33 undeclared
34 undeclared
35 undeclared
36 undeclared
37 undeclared
38 undeclared
39 undeclared
53 undeclared
54 undeclared
55 undeclared
56 undeclared
57 undeclared
58 undeclared
59 undeclared
107 undeclared
108 wired-invoke
109 undeclared
110 undeclared
111 undeclared