volt: Q65 cycle 6's rc-probe leg lands (it was on disk, uncommitted, since 7cd4dd5) + forge's new verifier graded -- census 110 of 110, runner 91 rows, CHECK rc 0
Two facts in one instrument commit, stated rather than implied: 1. cycle 6 was never committed. `q65_rc_probe.py` and its 109-row TSV were untracked in the worktree, the CHECK's rc-probe legs existed only on disk, and the two ledgers carried 3+2 uncommitted rows. Measured before landing, not trusted: `q65_rc_probe.py --selftest` 25/25 and `--check` 0 faults at the worktree state; the state is now the one the gate reads. 2. cycle 7 is the file that reopened the census. forge's6e8f577adds `wizard-work/forge/verify_probe_forge_refactor.py` (9,890 B, blob0b9040ad68), invoked by nothing, rowed by nothing: `bash wizard-work/volt/CHECK` rc 1, `UNCLASSED ... 1 unclassed of 110`, and `q63_verify_scan.py` `NEW_UNWIRED 1` -- both legs, two classes, one file. Graded by execution through the runner (PASS rc 0, 0.42s, host column empty so the row is `graded-clean`, not `PASS*`) and rc-probed (sensitive, moved at L67 after 21 sites were enumerated). Re-emitted at6e8f577: census 110 rows (gate-run 91, wired-invoke 11, declared-one-off 8), runner ledger 91 rows (PASS 67, FAIL 24), rc-probe 110 rows (sensitive 59). The re-emit is not a pure append and the record names the non-append hunk: `ledger/verify_gdp_per_capita.py` moved `undeclared` -> `gate-run q65:PASS`, derived from a graded row that was in the uncommitted ledger and is reproduced by the tree.
This commit is contained in:
1 parent
6e8f577c8c
commit
eb345839ac
12 files changed
+1464
-3
No files matched your search
+18
-1
@@ -16,6 +16,21 @@
|
||||
# have none -- all spark, recorded as findings, not repaired here; 14 of the
|
||||
# runner ledger's 64 PASS rows are rows on such a file and cannot fail.
|
||||
#
|
||||
# Since cycle 6 it grades that same claim dynamically, and about a process that
|
||||
# actually ran. `q65_rc_probe.py --emit` inverts each verifier's own comparisons
|
||||
# one at a time in a detached worktree and asks whether the exit code moves;
|
||||
# `--check` is the fast half run here: every row in q65_rc_probe.tsv must still
|
||||
# match the blob at HEAD (STALE) and must not contradict its own class. A file
|
||||
# cannot be given a row it did not earn, and a file that changed after its row
|
||||
# was written cannot keep it.
|
||||
#
|
||||
# `inert` is the one class whose negative is unbounded by construction, so the
|
||||
# sweep tries EVERY comparison site before writing it; a file stopped by
|
||||
# `--exhaustive-cap` is `inert-partial`, and --check faults on either word used
|
||||
# outside its coverage. This is cycle 6's own correction: a 3-of-11 sample wrote
|
||||
# `inert` on forge/verify_chunker_refactor.py, whose rc moves at 3 of its 11
|
||||
# sites (L123, L139, L142) that the sample did not pick.
|
||||
#
|
||||
# The root is resolved from $0, not written down. A gate that `cd`s to an
|
||||
# absolute host path grades THAT tree from whatever checkout invokes it: at
|
||||
# `66e5be1`, a clone with one verifier deleted printed `0 unclassed of 88` and
|
||||
@@ -37,6 +52,8 @@ python3 wizard-work/volt/q65_runner.py --check --class gate-run,undeclared || ex
|
||||
python3 wizard-work/volt/q63_verify_scan.py --repo . >/dev/null || exit 1
|
||||
python3 wizard-work/volt/q65_no_verdict.py --selftest >/dev/null || exit 1
|
||||
python3 wizard-work/volt/q65_no_verdict.py --repo . --check >/dev/null || exit 1
|
||||
python3 wizard-work/volt/q65_rc_probe.py --selftest >/dev/null || exit 1
|
||||
python3 wizard-work/volt/q65_rc_probe.py --repo . --check >/dev/null || exit 1
|
||||
|
||||
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean; no unrowed printer."
|
||||
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean; no unrowed printer; rc-sensitivity ledger content-bound."
|
||||
exit 0
|
||||
@@ -1,6 +1,6 @@
|
||||
# Q63 verifier census -- one row per COMMITTED verify_*.py at the rev the
|
||||
# scan read, class plus the evidence the class rests on.
|
||||
# rev: 25b04d0 (the tree these classes were read at)
|
||||
# rev: 6e8f577 (the tree these classes were read at)
|
||||
# row: <class> TAB <path> TAB <evidence> (three tab-separated columns)
|
||||
# class gate-run = q65_runner.py executed it and recorded an rc (Q65);
|
||||
# class wired-invoke = a committed line invokes it (the ref is the evidence);
|
||||
@@ -30,6 +30,7 @@ gate-run wizard-work/forge/verify_landing_check_refactor.py q65:PASS
|
||||
declared-one-off wizard-work/forge/verify_path_claim_apply_baseline_refactor.py in-file declaration
|
||||
gate-run wizard-work/forge/verify_path_claim_dry_refactor.py q65:FAIL
|
||||
gate-run wizard-work/forge/verify_pdf_refactor.py q65:PASS
|
||||
gate-run wizard-work/forge/verify_probe_forge_refactor.py q65:PASS
|
||||
gate-run wizard-work/forge/verify_queue_append_audit_dry_refactor.py q65:FAIL
|
||||
gate-run wizard-work/forge/verify_reply_check_dry_refactor.py q65:PASS
|
||||
gate-run wizard-work/forge/verify_repro_bounds_refactor.py q65:PASS
|
||||
@@ -42,6 +43,7 @@ gate-run wizard-work/forge/verify_smell_check_monotonic_refactor.py q65:PASS
|
||||
gate-run wizard-work/forge/verify_smell_scan_refactor.py q65:PASS
|
||||
gate-run wizard-work/forge/verify_tool_map_dry_refactor.py q65:PASS
|
||||
gate-run wizard-work/forge/verify_write_probe_dry_refactor.py q65:FAIL
|
||||
wired-invoke wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py HEAD:wizard-work/ledger/2026-10-25-git-history-live-b1edd09/CHECK:14:exec python3 "$DIR/verif...
|
||||
wired-invoke wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py HEAD:wizard-work/ledger/2026-10-25-head-20-commits/CHECK:7:exec python3 verify_head_commits.p...
|
||||
gate-run wizard-work/ledger/verify_arxiv_csai.py q65:PASS
|
||||
gate-run wizard-work/ledger/verify_arxiv_csai_fresh.py q65:PASS
|
||||
@@ -51,6 +53,7 @@ gate-run wizard-work/ledger/verify_catalog.py q65:PASS
|
||||
wired-invoke wizard-work/ledger/verify_commits_check.py HEAD:wizard-work/ledger/CHECK:8:exec python3 "$OUT/verify_commits_check.py" "$@"
|
||||
gate-run wizard-work/ledger/verify_dates.py q65:PASS
|
||||
gate-run wizard-work/ledger/verify_gdp.py q65:PASS
|
||||
gate-run wizard-work/ledger/verify_gdp_per_capita.py q65:PASS
|
||||
gate-run wizard-work/ledger/verify_gitea_repos.py q65:FAIL
|
||||
gate-run wizard-work/ledger/verify_github_python_repos.py q65:FAIL
|
||||
declared-one-off wizard-work/ledger/verify_github_repos.py in-file declaration
|
||||
@@ -116,5 +119,6 @@ gate-run wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py q
|
||||
gate-run wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py q65:PASS
|
||||
gate-run wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py q65:FAIL
|
||||
wired-invoke wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py HEAD:wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/CHECK.py:173: p = subproc...
|
||||
gate-run wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py q65:PASS
|
||||
wired-invoke wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py HEAD:wizard-work/sphinx/figure-locator-llama3-2026-10-24/run_checks.sh:13:python3 "$DIR/verif...
|
||||
gate-run wizard-work/sphinx/verify_mm_dossier.py q65:PASS
|
||||
|
Can't render this file because it contains an unexpected character in line 46 and column 173.
|
@@ -0,0 +1,7 @@
|
||||
rev1 da538ca5c0a4 blob b752e5467fb8
|
||||
rev2 64733c9e977e blob 365068a8f0f5
|
||||
ok A stale blob rc=1 expect=1 FAULT STALE wizard-work/x/verify_ctl.py: row graded blob b752e5467fb8, tree holds 365068a8f0f5
|
||||
ok B rebound blob rc=0 expect=0 (no fault line)
|
||||
ok C inert 1 of 3 sites rc=1 expect=1 FAULT CONTRADICTION wizard-work/x/verify_ctl.py: inert with valid=1 tried=1 sites=3 moved=None
|
||||
ok D inert-partial 1/3 rc=0 expect=0 (no fault line)
|
||||
4 of 4 controls behaved
|
||||
@@ -0,0 +1,49 @@
|
||||
dynamic dead-rc files (inert or no-comparison): 19 of 109 rows
|
||||
|
||||
class static runner sites/tried path
|
||||
inert weak-subprocess PASS 2/2 wizard-work/ledger/verify_dates.py
|
||||
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_all.py
|
||||
inert no-verdict PASS 3/3 wizard-work/spark/verify_check.py
|
||||
inert no-verdict PASS 1/1 wizard-work/spark/verify_count.py
|
||||
nocomp no-verdict - 0/0 wizard-work/spark/verify_dirs.py
|
||||
nocomp - PASS 0/0 wizard-work/spark/verify_env.py
|
||||
inert no-verdict PASS 4/4 wizard-work/spark/verify_fib.py
|
||||
inert - PASS 9/9 wizard-work/spark/verify_hermes_cli.py
|
||||
inert - PASS 1/1 wizard-work/spark/verify_hermes_path.py
|
||||
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_json.py
|
||||
inert no-verdict PASS 1/1 wizard-work/spark/verify_lines.py
|
||||
inert no-verdict PASS 3/3 wizard-work/spark/verify_minimal.py
|
||||
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_push.py
|
||||
inert no-verdict PASS 1/1 wizard-work/spark/verify_python_env.py
|
||||
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_skills.py
|
||||
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_skills2.py
|
||||
inert no-verdict PASS 1/1 wizard-work/spark/verify_skills3.py
|
||||
inert no-verdict PASS 1/1 wizard-work/spark/verify_skills_fixed.py
|
||||
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_sum.py
|
||||
|
||||
static no-verdict (cycle 5): 15
|
||||
dynamic dead rc (cycle 6) : 19
|
||||
in both : 15
|
||||
static only (rc moves) : 0 []
|
||||
dynamic only (has a path) : 4 ['wizard-work/ledger/verify_dates.py', 'wizard-work/spark/verify_env.py', 'wizard-work/spark/verify_hermes_cli.py', 'wizard-work/spark/verify_hermes_path.py']
|
||||
|
||||
dead-rc files carrying a runner PASS row: 18
|
||||
PASS wizard-work/ledger/verify_dates.py
|
||||
PASS wizard-work/spark/verify_all.py
|
||||
PASS wizard-work/spark/verify_check.py
|
||||
PASS wizard-work/spark/verify_count.py
|
||||
PASS wizard-work/spark/verify_env.py
|
||||
PASS wizard-work/spark/verify_fib.py
|
||||
PASS wizard-work/spark/verify_hermes_cli.py
|
||||
PASS wizard-work/spark/verify_hermes_path.py
|
||||
PASS wizard-work/spark/verify_json.py
|
||||
PASS wizard-work/spark/verify_lines.py
|
||||
PASS wizard-work/spark/verify_minimal.py
|
||||
PASS wizard-work/spark/verify_push.py
|
||||
PASS wizard-work/spark/verify_python_env.py
|
||||
PASS wizard-work/spark/verify_skills.py
|
||||
PASS wizard-work/spark/verify_skills2.py
|
||||
PASS wizard-work/spark/verify_skills3.py
|
||||
PASS wizard-work/spark/verify_skills_fixed.py
|
||||
PASS wizard-work/spark/verify_sum.py
|
||||
of those, PASS* (host-pinned): 17
|
||||
@@ -0,0 +1,325 @@
|
||||
q65_rc_probe -- rev 7cd4dd5d9226, 109 population file(s), work copy /root/.hermes/wizard-outputs/volt/rcprobe-work-c6
|
||||
[1/109] wizard-work/beam/verify_toposort_none.py
|
||||
L21 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=21 0.22s
|
||||
[2/109] wizard-work/forge/verify_append_only_audit_dry_refactor.py
|
||||
<unknown>:142: SyntaxWarning: "\u" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\u"? A raw string is also an option.
|
||||
<unknown>:157: SyntaxWarning: "\u" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\u"? A raw string is also an option.
|
||||
-> unprobed-baseline rc=1 sites=17 tried=0 valid=0 moved_at=- 2.16s
|
||||
[3/109] wizard-work/forge/verify_append_only_owners_dry_refactor.py
|
||||
L56 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=13 tried=1 valid=1 moved_at=56 0.65s
|
||||
[4/109] wizard-work/forge/verify_census_row_delta_exit_contract.py
|
||||
L46 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=23 tried=1 valid=1 moved_at=46 1.2000000000000002s
|
||||
[5/109] wizard-work/forge/verify_check_lint_classes_dry.py
|
||||
L80 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=15 tried=1 valid=1 moved_at=80 0.68s
|
||||
[6/109] wizard-work/forge/verify_check_lint_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=14 tried=0 valid=0 moved_at=- 1.22s
|
||||
[7/109] wizard-work/forge/verify_checkin_check_dry_refactor.py
|
||||
L103 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=103 0.9299999999999999s
|
||||
[8/109] wizard-work/forge/verify_chunker_refactor.py
|
||||
L58 invalid mutant (1)
|
||||
L105 valid mutant rc=0
|
||||
L150 valid mutant rc=0
|
||||
L61 valid mutant rc=0
|
||||
L64 valid mutant rc=0
|
||||
L68 valid mutant rc=0
|
||||
L101 valid mutant rc=0
|
||||
L123 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=11 tried=8 valid=7 moved_at=123 0.8499999999999999s
|
||||
[9/109] wizard-work/forge/verify_citation_hash_class_dry_refactor.py
|
||||
-> unprobed-timeout rc=TIMEOUT sites=12 tried=0 valid=0 moved_at=- 90.08s
|
||||
[10/109] wizard-work/forge/verify_citation_hash_class_portable_default.py
|
||||
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||
[11/109] wizard-work/forge/verify_citation_resolve_check_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 15.52s
|
||||
[12/109] wizard-work/forge/verify_doc_census_dry_refactor.py
|
||||
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||
[13/109] wizard-work/forge/verify_doc_command_report_dry.py
|
||||
-> unprobed-baseline rc=1 sites=8 tried=0 valid=0 moved_at=- 7.96s
|
||||
[14/109] wizard-work/forge/verify_gate_blame_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 10.81s
|
||||
[15/109] wizard-work/forge/verify_landing_check_refactor.py
|
||||
L103 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=25 tried=1 valid=1 moved_at=103 3.92s
|
||||
[16/109] wizard-work/forge/verify_path_claim_apply_baseline_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=15 tried=0 valid=0 moved_at=- 0.31s
|
||||
[17/109] wizard-work/forge/verify_path_claim_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=17 tried=0 valid=0 moved_at=- 0.55s
|
||||
[18/109] wizard-work/forge/verify_pdf_refactor.py
|
||||
L79 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=14 tried=1 valid=1 moved_at=79 7.27s
|
||||
[19/109] wizard-work/forge/verify_queue_append_audit_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=13 tried=0 valid=0 moved_at=- 7.81s
|
||||
[20/109] wizard-work/forge/verify_reply_check_dry_refactor.py
|
||||
L72 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=11 tried=1 valid=1 moved_at=72 3.0s
|
||||
[21/109] wizard-work/forge/verify_repro_bounds_refactor.py
|
||||
L69 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=17 tried=1 valid=1 moved_at=69 0.94s
|
||||
[22/109] wizard-work/forge/verify_roundtrip_refactor.py
|
||||
L39 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=39 1.26s
|
||||
[23/109] wizard-work/forge/verify_scan_dry_refactor.py
|
||||
L78 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=21 tried=1 valid=1 moved_at=78 6.3s
|
||||
[24/109] wizard-work/forge/verify_selftest_sweep_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=15 tried=0 valid=0 moved_at=- 0.25s
|
||||
[25/109] wizard-work/forge/verify_selftest_sweep_refactor.py
|
||||
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||
[26/109] wizard-work/forge/verify_served_drift_summarise_dry.py
|
||||
L100 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=16 tried=1 valid=1 moved_at=100 12.190000000000001s
|
||||
[27/109] wizard-work/forge/verify_smell_check_monotonic_refactor.py
|
||||
L64 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=16 tried=1 valid=1 moved_at=64 1.6800000000000002s
|
||||
[28/109] wizard-work/forge/verify_smell_scan_refactor.py
|
||||
L55 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=11 tried=1 valid=1 moved_at=55 1.0s
|
||||
[29/109] wizard-work/forge/verify_tool_map_dry_refactor.py
|
||||
L40 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=9 tried=1 valid=1 moved_at=40 49.33s
|
||||
[30/109] wizard-work/forge/verify_write_probe_dry_refactor.py
|
||||
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 23.35s
|
||||
[31/109] wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py
|
||||
L49 valid mutant rc=0
|
||||
L90 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=2 valid=2 moved_at=90 3.45s
|
||||
[32/109] wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py
|
||||
L122 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=122 0.69s
|
||||
[33/109] wizard-work/ledger/verify_arxiv_csai.py
|
||||
L37 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=14 tried=1 valid=1 moved_at=37 0.49s
|
||||
[34/109] wizard-work/ledger/verify_arxiv_csai_fresh.py
|
||||
L42 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=18 tried=1 valid=1 moved_at=42 0.47s
|
||||
[35/109] wizard-work/ledger/verify_arxiv_csv.py
|
||||
L24 valid mutant rc=0
|
||||
L29 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=2 valid=2 moved_at=29 0.33999999999999997s
|
||||
[36/109] wizard-work/ledger/verify_arxiv_fresh.py
|
||||
L48 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=48 0.19s
|
||||
[37/109] wizard-work/ledger/verify_catalog.py
|
||||
L30 invalid mutant (1)
|
||||
L77 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=14 tried=2 valid=1 moved_at=77 0.39s
|
||||
[38/109] wizard-work/ledger/verify_commits_check.py
|
||||
L122 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=122 0.58s
|
||||
[39/109] wizard-work/ledger/verify_dates.py
|
||||
L26 valid mutant rc=0
|
||||
L36 valid mutant rc=0
|
||||
-> inert rc=0 sites=2 tried=2 valid=2 moved_at=- 0.48s
|
||||
[40/109] wizard-work/ledger/verify_gdp.py
|
||||
L3 valid mutant rc=0
|
||||
L27 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=12 tried=2 valid=2 moved_at=27 0.29000000000000004s
|
||||
[41/109] wizard-work/ledger/verify_gdp_per_capita.py
|
||||
L42 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=42 1.52s
|
||||
[42/109] wizard-work/ledger/verify_gitea_repos.py
|
||||
-> unprobed-baseline rc=1 sites=8 tried=0 valid=0 moved_at=- 0.24s
|
||||
[43/109] wizard-work/ledger/verify_github_python_repos.py
|
||||
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 2.86s
|
||||
[44/109] wizard-work/ledger/verify_github_repos.py
|
||||
L24 valid mutant rc=0
|
||||
L107 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=14 tried=2 valid=2 moved_at=107 5.529999999999999s
|
||||
[45/109] wizard-work/ledger/verify_issues.py
|
||||
L48 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=48 0.22s
|
||||
[46/109] wizard-work/ledger/verify_python_org_repos.py
|
||||
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 2.02s
|
||||
[47/109] wizard-work/ledger/verify_stackchain_catalog.py
|
||||
L105 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=105 0.16999999999999998s
|
||||
[48/109] wizard-work/ledger/verify_users.py
|
||||
L32 valid mutant rc=0
|
||||
L40 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=6 tried=2 valid=2 moved_at=40 0.32s
|
||||
[49/109] wizard-work/ledger/verify_wizards.py
|
||||
-> unprobed-baseline rc=1 sites=19 tried=0 valid=0 moved_at=- 15.69s
|
||||
[50/109] wizard-work/ledger/verify_workflow.py
|
||||
-> unprobed-baseline rc=1 sites=4 tried=0 valid=0 moved_at=- 0.22s
|
||||
[51/109] wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py
|
||||
L57 valid mutant rc=0
|
||||
L85 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=9 tried=2 valid=2 moved_at=85 1.47s
|
||||
[52/109] wizard-work/null/diffkit/verify_realrepo.py
|
||||
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 0.38s
|
||||
[53/109] wizard-work/sol/verify_pending_request_triage.py
|
||||
L27 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=27 0.19s
|
||||
[54/109] wizard-work/sol/verify_sol_profile_triage.py
|
||||
L37 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=5 tried=1 valid=1 moved_at=37 1.65s
|
||||
[55/109] wizard-work/sol/verify_sol_triage.py
|
||||
-> unprobed-baseline rc=1 sites=11 tried=0 valid=0 moved_at=- 0.09s
|
||||
[56/109] wizard-work/sol/verify_triage.py
|
||||
L9 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=9 0.18s
|
||||
[57/109] wizard-work/sol/verify_triage_queue.py
|
||||
-> unprobed-baseline rc=1 sites=11 tried=0 valid=0 moved_at=- 0.13s
|
||||
[58/109] wizard-work/sol/verify_troupe_sol_health.py
|
||||
L43 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=43 0.18s
|
||||
[59/109] wizard-work/spark/verify_all.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.09s
|
||||
[60/109] wizard-work/spark/verify_analyzer.py
|
||||
L19 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=19 0.44s
|
||||
[61/109] wizard-work/spark/verify_check.py
|
||||
L6 valid mutant rc=0
|
||||
L10 valid mutant rc=0
|
||||
L11 valid mutant rc=0
|
||||
-> inert rc=0 sites=3 tried=3 valid=3 moved_at=- 0.31s
|
||||
[62/109] wizard-work/spark/verify_count.py
|
||||
L11 valid mutant rc=0
|
||||
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.19s
|
||||
[63/109] wizard-work/spark/verify_demo.py
|
||||
L32 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=32 0.2s
|
||||
[64/109] wizard-work/spark/verify_dir.py
|
||||
L15 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=15 0.15000000000000002s
|
||||
[65/109] wizard-work/spark/verify_dir_analyzer.py
|
||||
L19 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=19 2.7800000000000002s
|
||||
[66/109] wizard-work/spark/verify_dirs.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.1s
|
||||
[67/109] wizard-work/spark/verify_env.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.08s
|
||||
[68/109] wizard-work/spark/verify_environment.py
|
||||
-> unprobed-baseline rc=1 sites=3 tried=0 valid=0 moved_at=- 0.07s
|
||||
[69/109] wizard-work/spark/verify_fib.py
|
||||
L28 valid mutant rc=0
|
||||
L31 valid mutant rc=0
|
||||
L35 valid mutant rc=0
|
||||
L30 valid mutant rc=0
|
||||
-> inert rc=0 sites=4 tried=4 valid=4 moved_at=- 0.4100000000000001s
|
||||
[70/109] wizard-work/spark/verify_formatter.py
|
||||
L14 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=14 0.52s
|
||||
[71/109] wizard-work/spark/verify_hermes.py
|
||||
-> unprobed-baseline rc=1 sites=5 tried=0 valid=0 moved_at=- 0.13s
|
||||
[72/109] wizard-work/spark/verify_hermes_cli.py
|
||||
L31 valid mutant rc=0
|
||||
L35 valid mutant rc=0
|
||||
L64 valid mutant rc=0
|
||||
L32 valid mutant rc=0
|
||||
L33 valid mutant rc=0
|
||||
L34 valid mutant rc=0
|
||||
L36 valid mutant rc=0
|
||||
L45 valid mutant rc=0
|
||||
L50 valid mutant rc=0
|
||||
-> inert rc=0 sites=9 tried=9 valid=9 moved_at=- 10.519999999999998s
|
||||
[73/109] wizard-work/spark/verify_hermes_path.py
|
||||
L24 valid mutant rc=0
|
||||
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.16s
|
||||
[74/109] wizard-work/spark/verify_inventory.py
|
||||
-> unprobed-baseline rc=1 sites=2 tried=0 valid=0 moved_at=- 0.1s
|
||||
[75/109] wizard-work/spark/verify_json.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.13s
|
||||
[76/109] wizard-work/spark/verify_lines.py
|
||||
L6 valid mutant rc=0
|
||||
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.25s
|
||||
[77/109] wizard-work/spark/verify_loc_report.py
|
||||
L14 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=14 0.25s
|
||||
[78/109] wizard-work/spark/verify_lwc.py
|
||||
L18 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=18 0.56s
|
||||
[79/109] wizard-work/spark/verify_minimal.py
|
||||
L24 valid mutant rc=0
|
||||
L41 valid mutant rc=0
|
||||
L71 valid mutant rc=0
|
||||
-> inert rc=0 sites=3 tried=3 valid=3 moved_at=- 0.68s
|
||||
[80/109] wizard-work/spark/verify_push.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.14s
|
||||
[81/109] wizard-work/spark/verify_python_env.py
|
||||
L31 valid mutant rc=0
|
||||
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.19s
|
||||
[82/109] wizard-work/spark/verify_report.py
|
||||
L11 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=11 0.2s
|
||||
[83/109] wizard-work/spark/verify_result.py
|
||||
L12 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=12 0.25s
|
||||
[84/109] wizard-work/spark/verify_scratch_analysis.py
|
||||
L17 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=17 0.19s
|
||||
[85/109] wizard-work/spark/verify_skill_system.py
|
||||
-> unprobed-baseline rc=1 sites=6 tried=0 valid=0 moved_at=- 0.16s
|
||||
[86/109] wizard-work/spark/verify_skills.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.11s
|
||||
[87/109] wizard-work/spark/verify_skills2.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.09s
|
||||
[88/109] wizard-work/spark/verify_skills3.py
|
||||
L41 valid mutant rc=0
|
||||
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.24000000000000002s
|
||||
[89/109] wizard-work/spark/verify_skills_fixed.py
|
||||
L31 valid mutant rc=0
|
||||
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.22s
|
||||
[90/109] wizard-work/spark/verify_snippets.py
|
||||
L25 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=25 0.19s
|
||||
[91/109] wizard-work/spark/verify_status.py
|
||||
L10 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=5 tried=1 valid=1 moved_at=10 0.63s
|
||||
[92/109] wizard-work/spark/verify_sum.py
|
||||
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.1s
|
||||
[93/109] wizard-work/spark/verify_tool.py
|
||||
L12 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=12 1.08s
|
||||
[94/109] wizard-work/spark/verify_wordcount.py
|
||||
L15 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=7 tried=1 valid=1 moved_at=15 0.46s
|
||||
[95/109] wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py
|
||||
L49 invalid mutant (1)
|
||||
L88 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=17 tried=2 valid=1 moved_at=88 8.68s
|
||||
[96/109] wizard-work/sphinx/archive-gutenberg/verify_archive.py
|
||||
L25 valid mutant rc=0
|
||||
L63 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=12 tried=2 valid=2 moved_at=63 1.2s
|
||||
[97/109] wizard-work/sphinx/archive-index-2026-10-23/verify_index.py
|
||||
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||
[98/109] wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py
|
||||
L57 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=44 tried=1 valid=1 moved_at=57 0.37s
|
||||
[99/109] wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py
|
||||
-> unprobed-baseline rc=1 sites=29 tried=0 valid=0 moved_at=- 4.96s
|
||||
[100/109] wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py
|
||||
L106 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=35 tried=1 valid=1 moved_at=106 13.06s
|
||||
[101/109] wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py
|
||||
-> unprobed-baseline rc=1 sites=2 tried=0 valid=0 moved_at=- 0.14s
|
||||
[102/109] wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py
|
||||
L61 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=10 tried=1 valid=1 moved_at=61 0.48s
|
||||
[103/109] wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py
|
||||
L44 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=19 tried=1 valid=1 moved_at=44 0.38s
|
||||
[104/109] wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py
|
||||
L93 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=22 tried=1 valid=1 moved_at=93 7.75s
|
||||
[105/109] wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py
|
||||
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||
[106/109] wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py
|
||||
L40 invalid mutant (1)
|
||||
L250 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=69 tried=2 valid=1 moved_at=250 9.260000000000002s
|
||||
[107/109] wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py
|
||||
-> unprobed-baseline rc=1 sites=10 tried=0 valid=0 moved_at=- 40.7s
|
||||
[108/109] wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py
|
||||
L101 valid mutant rc=1
|
||||
-> sensitive rc=0 sites=21 tried=1 valid=1 moved_at=101 0.77s
|
||||
[109/109] wizard-work/sphinx/verify_mm_dossier.py
|
||||
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 3.49s
|
||||
wrote wizard-work/volt/q65_rc_probe.tsv (109 rows)
|
||||
work copy left at /root/.hermes/wizard-outputs/volt/rcprobe-work-c6
|
||||
@@ -0,0 +1,20 @@
|
||||
# The boundary of 'inert': comparison-inert is not 'cannot fail'.
|
||||
# Substrate: spark/verify_hermes_path.py copied unchanged to a scratch dir, then
|
||||
# ONE required path renamed to a directory that does not exist.
|
||||
|
||||
== control: unmodified copy at HEAD (blob 385362b75cfe)
|
||||
OK: /root/.hermes/wizard-outputs/spark
|
||||
OK: /root/troupe-standup/wizard-work/spark
|
||||
OK: /root/.hermes/profiles/spark
|
||||
rc=0
|
||||
|
||||
== falsified subject: /root/.hermes/profiles/spark -> /root/.hermes/profiles/spark-NO-SUCH-DIR-c6
|
||||
OK: /root/.hermes/wizard-outputs/spark
|
||||
OK: /root/troupe-standup/wizard-work/spark
|
||||
MISSING: /root/.hermes/profiles/spark-NO-SUCH-DIR-c6
|
||||
rc=1
|
||||
|
||||
== the other 16 comparison-inert rows cannot fail at all:
|
||||
spark/verify_check.py (13 loc) computes factorial(5) and prints it -- no exit path, no assertion;
|
||||
it carries a runner PASS row with an empty host column, i.e. a graded-clean PASS on a file that cannot fail.
|
||||
15 static no-verdict printers (q65_no_verdict.py --check) + ledger/verify_dates.py (cycle 4: prints impossible dates, rc 0).
|
||||
@@ -0,0 +1,24 @@
|
||||
== wizard-work/forge/verify_chunker_refactor.py (6328 bytes, sha256 fc82c86f32ab)
|
||||
sites=11 sample(spread,3)=[58, 105, 150] baseline rc=0
|
||||
L58 rc=1 invalid <- in sample
|
||||
L61 rc=0 -
|
||||
L64 rc=0 -
|
||||
L68 rc=0 -
|
||||
L101 rc=0 -
|
||||
L105 rc=0 - <- in sample
|
||||
L123 rc=1 MOVED
|
||||
L124 rc=0 -
|
||||
L139 rc=1 MOVED
|
||||
L142 rc=2 MOVED
|
||||
L150 rc=0 - <- in sample
|
||||
moved at 3 of 11 sites: [123, 139, 142]
|
||||
sampled sites that moved: []
|
||||
|
||||
== wizard-work/ledger/verify_dates.py (1172 bytes, sha256 717e81b33a0f)
|
||||
sites=2 sample(spread,3)=[26, 36] baseline rc=0
|
||||
L26 rc=0 - <- in sample
|
||||
L36 rc=0 - <- in sample
|
||||
moved at 0 of 2 sites: []
|
||||
sampled sites that moved: []
|
||||
|
||||
done
|
||||
@@ -0,0 +1,89 @@
|
||||
# provenance: bash wizard-work/volt/CHECK at 6e8f577 (2026-10-25T16:50:03+00:00), 2026-10-25T16:57:38Z
|
||||
ok c1 PASS rc 0 is consistent
|
||||
ok c2 PASS rc 1 is a fault
|
||||
ok c3 TIMEOUT carrying an rc is a fault
|
||||
ok c4 OTHER_RC rc 3 is consistent
|
||||
ok c5 OTHER_RC rc 7 is consistent
|
||||
ok c6 OTHER_RC rc 2 is a fault (that is REFUSED)
|
||||
ok c7 ledger round-trips two rows
|
||||
ok c8 a missing ledger is an error, not an empty pass
|
||||
ok c9 an unknown verdict refuses the ledger
|
||||
ok c10 a committed verifier in neither ledger is UNCLASSED
|
||||
ok c11 a class row covers the same file
|
||||
ok c12 a window row parses to (seconds, why)
|
||||
ok c13 an unlisted path falls back to --timeout
|
||||
ok c14 a malformed window row refuses instead of silently skipping
|
||||
ok c15 a committed path has a blob id
|
||||
ok c16 an absent path has no blob id
|
||||
ok c17 the same path at a new commit is a different blob (DRIFT is detectable)
|
||||
ok c18 a host root in a string VALUE is a host pin
|
||||
ok c19 a host root in a docstring is prose, not a pin
|
||||
ok c20 a PARSED PASS row on a host-independent verifier is graded clean
|
||||
ok c21 a PARSED OTHER_RC row with rc 7 is consistent (not a fault)
|
||||
ok c22 a class-narrowed tally sums to its own population, not the ledger
|
||||
selftest: 22 cells, 22 passed, 0 failed
|
||||
q65_runner -- rev HEAD, 90 population file(s), 90 ledger row(s) [ledger rev 7cd4dd5]
|
||||
rows at emit : 90 (PASS 66, FAIL 24)
|
||||
graded (has rc): 90 of 90
|
||||
graded-clean : 10 of 90 (PASS on a host-independent verifier)
|
||||
host-pinned : 56 PASS row(s) excluded as PASS* -- they read an absolute host root
|
||||
PASS* wizard-work/forge/verify_append_only_owners_dry_refactor.py L41:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_census_row_delta_exit_contract.py L31:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_check_lint_classes_dry.py L44:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_checkin_check_dry_refactor.py L39:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_landing_check_refactor.py L62:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_pdf_refactor.py L41:/root/.hermes
|
||||
PASS* wizard-work/forge/verify_reply_check_dry_refactor.py L33:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_repro_bounds_refactor.py L33:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_roundtrip_refactor.py L21:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_served_drift_summarise_dry.py L48:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_smell_check_monotonic_refactor.py L32:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_smell_scan_refactor.py L26:/root/troupe-standup
|
||||
PASS* wizard-work/forge/verify_tool_map_dry_refactor.py L32:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_arxiv_csai.py L15:/root/.hermes
|
||||
PASS* wizard-work/ledger/verify_arxiv_csai_fresh.py L16:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_arxiv_csv.py L24:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_catalog.py L13:/root/.hermes
|
||||
PASS* wizard-work/ledger/verify_dates.py L6:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_gdp.py L3:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_gdp_per_capita.py L19:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_issues.py L15:/root/.hermes
|
||||
PASS* wizard-work/ledger/verify_stackchain_catalog.py L16:/root/troupe-standup
|
||||
PASS* wizard-work/ledger/verify_users.py L16:/root/troupe-standup
|
||||
PASS* wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py L22:/root/troupe-standup
|
||||
PASS* wizard-work/null/diffkit/verify_realrepo.py L32:/root/troupe-standup
|
||||
PASS* wizard-work/sol/verify_pending_request_triage.py L11:/root/troupe-standup
|
||||
PASS* wizard-work/sol/verify_triage.py L4:/root/troupe-standup
|
||||
PASS* wizard-work/spark/verify_all.py L4:/root/troupe-standup
|
||||
PASS* wizard-work/spark/verify_count.py L5:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_demo.py L20:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_dir.py L23:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_dir_analyzer.py L16:/root/troupe-standup
|
||||
PASS* wizard-work/spark/verify_env.py L9:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_fib.py L24:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_formatter.py L9:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_hermes_cli.py L55:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_hermes_path.py L11:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_json.py L4:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_lines.py L2:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_lwc.py L12:/root/troupe-standup
|
||||
PASS* wizard-work/spark/verify_minimal.py L31:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_push.py L3:/root/troupe-standup
|
||||
PASS* wizard-work/spark/verify_python_env.py L22:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_report.py L10:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_result.py L10:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_scratch_analysis.py L13:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_skills.py L5:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_skills2.py L6:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_skills3.py L6:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_skills_fixed.py L6:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_snippets.py L11:/root/troupe-standup
|
||||
PASS* wizard-work/spark/verify_status.py L7:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_sum.py L6:/root/.hermes
|
||||
PASS* wizard-work/spark/verify_wordcount.py L46:/root/.hermes
|
||||
PASS* wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py L59:/root/.hermes
|
||||
PASS* wizard-work/sphinx/verify_mm_dossier.py L28:/root/.hermes
|
||||
tree audit : 1 unclassed of 110 committed verifier(s) <-- named by neither ledger
|
||||
UNCLASSED wizard-work/forge/verify_probe_forge_refactor.py: committed verify_*.py at HEAD, in no class row in q63_unwired_baseline.tsv and no ledger row
|
||||
q65: 1 fault(s)
|
||||
rc=1
|
||||
@@ -0,0 +1,804 @@
|
||||
#!/usr/bin/env python3
|
||||
"""q65_rc_probe.py -- ask a committed verifier, by execution, whether its exit
|
||||
code can move at all (Q65, volt lane, 2026-10-25, cycle 6).
|
||||
|
||||
WHY THIS EXISTS
|
||||
Cycle 5 answered "does a verifier have a failure path" statically: an AST scan
|
||||
for `raise` / `assert` / non-zero `exit` / `unittest.main` found 15 of 106
|
||||
committed verifiers with none, 14 of them carrying a PASS row in the runner
|
||||
ledger. Static presence is a proxy, and it has two blind spots the tool said
|
||||
so about itself: an exit path inside a `try/except` that swallows it still
|
||||
passes the scan, and a present exit path can be unreachable from any verdict.
|
||||
|
||||
This is the dynamic question instead, and it does not read the source for
|
||||
intent. It perturbs the verifier's OWN decisions and watches the process:
|
||||
flip one comparison operator at a time, re-run the file, and see whether the
|
||||
exit code changes. If a verifier's rc is unchanged after its comparisons have
|
||||
been inverted, its rc is not connected to what it measured -- no input could
|
||||
have made that file say no. That is a stronger statement than "no raise in the
|
||||
AST", because it is made about a process that actually ran.
|
||||
|
||||
The mutation is of the verifier, not of the subject. Conventionally, mutation
|
||||
testing mutates the SUBJECT and asks whether the test still passes (a killed
|
||||
mutant means the test discriminates). Here the subject stays exactly as it is
|
||||
and the ASSERTION is mutated, which inverts the question into the one Q65
|
||||
needs: does an inverted assertion reach the exit code? A verifier whose rc is
|
||||
stable under its own inverted assertions has no verdict channel, whatever its
|
||||
source looks like.
|
||||
|
||||
WHAT IT MEASURES -- one row per probed file, one class per row
|
||||
sensitive baseline rc 0, and >=1 VALID mutant returned a different rc.
|
||||
The rc is wired to a comparison; the file can fail.
|
||||
inert baseline rc 0, >=1 valid mutant, every valid mutant returned
|
||||
the baseline rc, AND every mutable comparison site in the file
|
||||
was tried. The rc does not respond to any of the file's own
|
||||
comparisons. This is the finding.
|
||||
It is a statement about COMPARISONS, not about the world: a
|
||||
verifier whose rc is `all(exists and is_dir ...)` over
|
||||
`os.path.isdir` is comparison-inert and can still fail on a
|
||||
falsified subject (measured: spark/verify_hermes_path.py with
|
||||
one required path renamed -> rc 1). 3 of the 19 comparison-inert
|
||||
rows at 7cd4dd5 are of that kind; the other 16 cannot fail at
|
||||
all, 15 of them static no-verdict printers.
|
||||
inert-partial the same, but the file has more sites than `--exhaustive-cap`
|
||||
allowed, so the negative is bounded: nothing moved in the
|
||||
first N sites, and sites N+1.. were never tried. Explicitly
|
||||
not the same claim as `inert` -- see WHY THE CLASSES ARE SPLIT.
|
||||
no-comparison baseline rc 0 and no single-operator comparison to mutate.
|
||||
Nothing in the file is a decision the rc could carry.
|
||||
unprobed-baseline the unmutated file already returned a non-zero rc: there is
|
||||
no pass state to defend, so the question does not apply.
|
||||
unprobed-timeout the unmutated file did not finish inside its window.
|
||||
unprobed-mutant every mutant tried was invalid (see below), so nothing was
|
||||
measured. Not a finding either way.
|
||||
unprobed-window the verifier's own measurement window (`q65_row_windows.tsv`,
|
||||
shared with the runner) exceeds `--skip-over`, so it was not
|
||||
mutated. A 30-minute live-HTTP verifier is not inverted three
|
||||
times inside a sweep; its rc is the runner's row.
|
||||
unprobed-error the sweep could not touch the file (the work copy lost it
|
||||
mid-run). Recorded as a row with its message, so a pruned
|
||||
scratch directory leaves a readable ledger instead of a
|
||||
traceback halfway through the population.
|
||||
|
||||
A MUTANT IS VALID when its run (a) compiled and (b) finished inside the window
|
||||
and (c) did NOT print a traceback. Flipping `==` to `!=` can turn a type check
|
||||
into a TypeError; an rc that moves because the mutant crashed says nothing
|
||||
about whether a verdict reaches the exit code. Invalid mutants are counted and
|
||||
discarded, and a file left with none is `unprobed-mutant`, not `inert`.
|
||||
|
||||
HOW MUCH IS TRIED -- deterministic spread, so the figure is reproducible
|
||||
`--max-mutants` (default 3) sites are taken from the sorted list of mutable
|
||||
comparison line numbers by even spacing, so a 76-site file is probed near its
|
||||
beginning, middle and end rather than at its first three lines. No randomness:
|
||||
the same rev and the same `--max-mutants` select the same sites.
|
||||
|
||||
If those first sites all return the baseline rc, the sample is NOT the
|
||||
verdict: every remaining site is tried too, up to `--exhaustive-cap`
|
||||
(default 40) per file; 0 means no cap. Only a file whose whole site list was
|
||||
tried can be called `inert`; a file stopped by the cap is `inert-partial`.
|
||||
|
||||
WHY THE CLASSES ARE SPLIT -- a measured false positive, not a hypothetical
|
||||
Cycle 6's first sweep wrote `inert` for `forge/verify_chunker_refactor.py`
|
||||
from a 3-of-11 sample: sites 58, 105, 150 (one of which crashed). Swept
|
||||
exhaustively in a scratch copy, that file's rc moves at 3 of its 11 sites
|
||||
(L123 -> 1, L139 -> 1, L142 -> 2) and the sample happened to hit none of
|
||||
them. The file can fail; the sample said it could not. `inert` is therefore
|
||||
a statement about every comparison in the file or it is not made.
|
||||
|
||||
WHERE IT WRITES
|
||||
`--emit` mutates a COPY, never this worktree. The copy is a detached `git
|
||||
worktree` at `--rev` -- a real checkout, not an export, because a verifier
|
||||
that reads git plumbing returns a non-zero rc in a bare export and that
|
||||
non-zero would be the copy's defect graded as the verifier's baseline. The
|
||||
tool refuses (rc 2) a `--work` equal to the repository root -- a gate that
|
||||
edits the tree it grades is the defect reply 93 §1 already measured once.
|
||||
Every mutant is written, run, and restored inside a `try/finally`; the file
|
||||
hash is re-checked after each file and a mismatch is rc 1, not a silent pass.
|
||||
|
||||
`--emit` writes the TSV (`--tsv`) and appends progress as it goes, so a sweep
|
||||
that is interrupted leaves a readable partial ledger rather than nothing.
|
||||
|
||||
`--check` is the fast half and what the lane's CHECK runs: it re-reads the TSV,
|
||||
binds every row to the blob at `--rev` (a row graded a file that has since
|
||||
changed is STALE), re-derives the class counts from the rows, and faults on a
|
||||
row that contradicts its own class (a `sensitive` row with no moving line, an
|
||||
`inert` row with no valid mutant, a `no-comparison` row that lists sites).
|
||||
It does NOT require population coverage: that is the census's job
|
||||
(`q63_verify_scan.py`) and the runner's, and duplicating it here would make
|
||||
two gates red for one added file. It prints the coverage gap instead.
|
||||
|
||||
rc 0 the ledger is internally consistent and content-bound
|
||||
rc 1 a fault it graded: STALE, MISSING, an unknown class, or a row that
|
||||
contradicts its class
|
||||
rc 2 it could not grade at all (no TSV, not a git repo, --work is the tree)
|
||||
"""
|
||||
import argparse
|
||||
import ast
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
DEFAULT_TSV = os.path.join(HERE, "q65_rc_probe.tsv")
|
||||
DEFAULT_REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||
SCRATCH_ROOT = "/root/.hermes/profiles/volt/cache/scratch"
|
||||
|
||||
CLASSES = ("sensitive", "inert", "inert-partial", "no-comparison",
|
||||
"unprobed-baseline", "unprobed-timeout", "unprobed-mutant",
|
||||
"unprobed-window", "unprobed-error")
|
||||
|
||||
FLIP = {
|
||||
ast.Eq: ast.NotEq, ast.NotEq: ast.Eq,
|
||||
ast.Lt: ast.Gt, ast.Gt: ast.Lt,
|
||||
ast.LtE: ast.GtE, ast.GtE: ast.LtE,
|
||||
ast.In: ast.NotIn, ast.NotIn: ast.In,
|
||||
ast.Is: ast.IsNot, ast.IsNot: ast.Is,
|
||||
}
|
||||
|
||||
TRACEBACK = "Traceback (most recent call last)"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- source side
|
||||
def mutable_lines(src):
|
||||
"""Line numbers of single-operator comparisons that FLIP can invert."""
|
||||
try:
|
||||
tree = ast.parse(src)
|
||||
except SyntaxError:
|
||||
return []
|
||||
lines = set()
|
||||
for node in ast.walk(tree):
|
||||
if (isinstance(node, ast.Compare) and len(node.ops) == 1
|
||||
and type(node.ops[0]) in FLIP):
|
||||
lines.add(node.lineno)
|
||||
return sorted(lines)
|
||||
|
||||
|
||||
def spread(picks, n):
|
||||
"""Even-spaced deterministic subset of a sorted list, n or fewer."""
|
||||
if n <= 0 or not picks:
|
||||
return []
|
||||
if len(picks) <= n:
|
||||
return list(picks)
|
||||
if n == 1:
|
||||
return [picks[0]]
|
||||
step = (len(picks) - 1) / float(n - 1)
|
||||
out = []
|
||||
for i in range(n):
|
||||
idx = int(round(i * step))
|
||||
if picks[idx] not in out:
|
||||
out.append(picks[idx])
|
||||
return out
|
||||
|
||||
|
||||
def mutate(src, line):
|
||||
"""Return source with the comparison at `line` inverted, or None."""
|
||||
tree = ast.parse(src)
|
||||
hits = [n for n in ast.walk(tree)
|
||||
if isinstance(n, ast.Compare) and n.lineno == line
|
||||
and len(n.ops) == 1 and type(n.ops[0]) in FLIP]
|
||||
if not hits:
|
||||
return None
|
||||
hits[0].ops[0] = FLIP[type(hits[0].ops[0])]()
|
||||
return ast.unparse(ast.fix_missing_locations(tree)) + "\n"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- process side
|
||||
def run_file(path, timeout):
|
||||
"""Run one verifier the way the ledger does: cwd is the file's directory."""
|
||||
t0 = time.time()
|
||||
try:
|
||||
p = subprocess.run([sys.executable, os.path.basename(path)],
|
||||
cwd=os.path.dirname(path) or ".",
|
||||
capture_output=True, text=True, timeout=timeout)
|
||||
return {"rc": p.returncode, "traceback": TRACEBACK in p.stderr,
|
||||
"seconds": round(time.time() - t0, 2)}
|
||||
except subprocess.TimeoutExpired:
|
||||
return {"rc": "TIMEOUT", "traceback": False,
|
||||
"seconds": round(time.time() - t0, 2)}
|
||||
|
||||
|
||||
def read_windows(path):
|
||||
"""path -> seconds, from the runner's per-row window table (may be absent)."""
|
||||
table = {}
|
||||
if not path or not os.path.isfile(path):
|
||||
return table
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
line = line.rstrip("\n")
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
cells = line.split("\t")
|
||||
if len(cells) >= 2:
|
||||
try:
|
||||
table[cells[0]] = float(cells[1])
|
||||
except ValueError:
|
||||
continue
|
||||
return table
|
||||
|
||||
|
||||
def probe(path, max_mutants, timeout, log=None, exhaustive_cap=40):
|
||||
"""Baseline, up to max_mutants sample sites, then every remaining site.
|
||||
|
||||
The sample is the cheap question; the exhaustive pass is what licenses the
|
||||
word `inert`. A file stopped by `exhaustive_cap` is inert-partial.
|
||||
"""
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
src = fh.read()
|
||||
before = hashlib.sha256(src.encode()).hexdigest()
|
||||
sites = mutable_lines(src)
|
||||
base = run_file(path, timeout)
|
||||
row = {"path": path, "baseline_rc": base["rc"], "sites": len(sites),
|
||||
"tried": 0, "valid": 0, "moved_at": "-", "seconds": base["seconds"],
|
||||
"cls": None, "note": ""}
|
||||
|
||||
if base["rc"] == "TIMEOUT":
|
||||
row["cls"] = "unprobed-timeout"
|
||||
return row
|
||||
if base["rc"] != 0:
|
||||
row["cls"] = "unprobed-baseline"
|
||||
row["note"] = "baseline rc %s" % base["rc"]
|
||||
return row
|
||||
if not sites:
|
||||
row["cls"] = "no-comparison"
|
||||
return row
|
||||
|
||||
tries = spread(sites, max_mutants)
|
||||
if len(tries) < len(sites):
|
||||
# The negative half of the class needs every site, not the sample.
|
||||
rest = [s for s in sites if s not in tries]
|
||||
cap = len(rest) if exhaustive_cap == 0 else max(0, exhaustive_cap - len(tries))
|
||||
tries += spread(rest, cap)
|
||||
for line in tries:
|
||||
mutant = mutate(src, line)
|
||||
if mutant is None:
|
||||
continue
|
||||
row["tried"] += 1
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write(mutant)
|
||||
try:
|
||||
got = run_file(path, timeout)
|
||||
finally:
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write(src)
|
||||
row["seconds"] += got["seconds"]
|
||||
valid = got["rc"] != "TIMEOUT" and not got["traceback"]
|
||||
if not valid:
|
||||
if log:
|
||||
log(" L%d invalid mutant (%s)" % (line, got["rc"]))
|
||||
continue
|
||||
row["valid"] += 1
|
||||
if log:
|
||||
log(" L%d valid mutant rc=%s" % (line, got["rc"]))
|
||||
if got["rc"] != base["rc"]:
|
||||
row["cls"] = "sensitive"
|
||||
row["moved_at"] = line
|
||||
break
|
||||
|
||||
if row["cls"] != "sensitive":
|
||||
if not row["valid"]:
|
||||
row["cls"] = "unprobed-mutant"
|
||||
elif row["tried"] >= len(sites):
|
||||
row["cls"] = "inert"
|
||||
else:
|
||||
row["cls"] = "inert-partial"
|
||||
row["note"] = "%d of %d sites tried (cap %d)" % (
|
||||
row["tried"], len(sites), exhaustive_cap)
|
||||
after = hashlib.sha256(open(path, encoding="utf-8").read().encode()).hexdigest()
|
||||
if after != before:
|
||||
raise RuntimeError("restore failed for %s" % path)
|
||||
return row
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- git plumbing
|
||||
def tree_blobs(repo, rev):
|
||||
out = subprocess.run(["git", "-C", repo, "ls-tree", "-r", rev],
|
||||
capture_output=True, text=True)
|
||||
if out.returncode != 0:
|
||||
raise SystemExit("q65_rc_probe: cannot read tree at %s: %s"
|
||||
% (rev, out.stderr.strip()))
|
||||
blobs = {}
|
||||
for line in out.stdout.splitlines():
|
||||
meta, path = line.split("\t", 1)
|
||||
parts = meta.split()
|
||||
if len(parts) >= 3 and parts[1] == "blob":
|
||||
blobs[path] = parts[2]
|
||||
return blobs
|
||||
|
||||
|
||||
def population(blobs):
|
||||
return sorted(p for p in blobs
|
||||
if os.path.basename(p).startswith("verify_")
|
||||
and p.endswith(".py")
|
||||
and p.startswith("wizard-work/"))
|
||||
|
||||
|
||||
def strip_basename(path):
|
||||
return path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- modes
|
||||
def read_tsv(path):
|
||||
rows, head = [], {}
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
line = line.rstrip("\n")
|
||||
if not line.strip():
|
||||
continue
|
||||
if line.startswith("#"):
|
||||
for key in ("rev", "rows", "committed"):
|
||||
if line.startswith("# %s " % key):
|
||||
head[key] = line.split(None, 2)[2].split()[0]
|
||||
continue
|
||||
cells = line.split("\t")
|
||||
if len(cells) != 9:
|
||||
raise ValueError("row has %d cells, want 9: %r" % (len(cells), line))
|
||||
rows.append(dict(zip(
|
||||
("cls", "path", "baseline_rc", "sites", "tried", "valid",
|
||||
"moved_at", "blob", "seconds"), cells)))
|
||||
return head, rows
|
||||
|
||||
|
||||
def write_header(fh, rev, rows, committed):
|
||||
fh.write("# Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.\n")
|
||||
fh.write("# Written by q65_rc_probe.py --emit. A row states a measurement, not an\n")
|
||||
fh.write("# exemption: --check faults on a row that contradicts its class or whose\n")
|
||||
fh.write("# file has changed since (STALE).\n")
|
||||
fh.write("# columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds\n")
|
||||
fh.write("# rev %s\n" % rev)
|
||||
fh.write("# rows %d\n" % rows)
|
||||
fh.write("# committed %d\n" % committed)
|
||||
|
||||
|
||||
def do_emit(args):
|
||||
repo = os.path.abspath(args.repo)
|
||||
rev = subprocess.run(["git", "-C", repo, "rev-parse", args.rev],
|
||||
capture_output=True, text=True)
|
||||
if rev.returncode != 0:
|
||||
print("q65_rc_probe: not a git tree at %s" % repo, file=sys.stderr)
|
||||
return 2
|
||||
rev = rev.stdout.strip()
|
||||
blobs = tree_blobs(repo, rev)
|
||||
pop = population(blobs)
|
||||
windows = read_windows(args.windows)
|
||||
prev_rows = []
|
||||
if args.merge and os.path.isfile(args.tsv):
|
||||
try:
|
||||
_, prev_rows = read_tsv(args.tsv)
|
||||
except ValueError:
|
||||
prev_rows = []
|
||||
print("q65_rc_probe: merging into %d existing row(s) from %s"
|
||||
% (len(prev_rows), args.tsv))
|
||||
if args.only:
|
||||
pop = [p for p in pop if args.only in p]
|
||||
if args.limit:
|
||||
pop = pop[:args.limit]
|
||||
|
||||
if args.work:
|
||||
work = os.path.abspath(args.work)
|
||||
if work == os.path.realpath(repo) or work == repo:
|
||||
print("q65_rc_probe: --work is the repository itself; refusing to "
|
||||
"mutate the tree this gate grades", file=sys.stderr)
|
||||
return 2
|
||||
else:
|
||||
work = os.path.join(SCRATCH_ROOT, "rcprobe-%d" % int(time.time()))
|
||||
if not os.path.isdir(work):
|
||||
os.makedirs(work)
|
||||
if not os.path.isdir(os.path.join(work, ".git")) and not os.path.exists(
|
||||
os.path.join(work, ".git")):
|
||||
# A real git worktree, not an export: a verifier that reads git plumbing
|
||||
# (rev-parse, ls-tree, log) returns a non-zero rc in a bare export, and a
|
||||
# non-zero BASELINE is a row with no pass state to defend -- the sweep
|
||||
# would grade its own copy's defect instead of the verifier.
|
||||
add = subprocess.run(["git", "-C", repo, "worktree", "add", "--detach",
|
||||
work, rev], capture_output=True, text=True)
|
||||
if add.returncode != 0:
|
||||
print("q65_rc_probe: cannot add a worktree at %s: %s"
|
||||
% (rev[:12], add.stderr.strip()), file=sys.stderr)
|
||||
return 2
|
||||
|
||||
print("q65_rc_probe -- rev %s, %d population file(s), work copy %s"
|
||||
% (rev[:12], len(pop), work))
|
||||
rows = []
|
||||
for i, rel in enumerate(pop, 1):
|
||||
path = os.path.join(work, rel)
|
||||
if not os.path.isfile(path):
|
||||
rows.append({"cls": "unprobed-mutant", "path": rel, "baseline_rc": "-",
|
||||
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
|
||||
"blob": blobs.get(rel, "-")[:12], "seconds": 0.0,
|
||||
"note": "absent from the work copy"})
|
||||
continue
|
||||
print("[%d/%d] %s" % (i, len(pop), rel))
|
||||
window = windows.get(rel, args.timeout)
|
||||
if window > args.skip_over:
|
||||
row = {"cls": "unprobed-window", "path": rel, "baseline_rc": "-",
|
||||
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
|
||||
"seconds": 0.0, "note": "%ss window" % int(window)}
|
||||
row["blob"] = blobs.get(rel, "-")[:12]
|
||||
rows.append(row)
|
||||
print(" -> %-17s window %ss exceeds --skip-over %ss"
|
||||
% (row["cls"], int(window), int(args.skip_over)))
|
||||
continue
|
||||
try:
|
||||
row = probe(path, args.max_mutants, window, log=lambda m: print(m),
|
||||
exhaustive_cap=getattr(args, "exhaustive_cap", 40))
|
||||
except (OSError, RuntimeError) as exc:
|
||||
# A work copy that lost a file mid-sweep (a scratch pruner, a full
|
||||
# disk) must be a row, not a traceback: the sweep's ledger stays
|
||||
# readable and the row says what happened.
|
||||
row = {"cls": "unprobed-error", "path": rel, "baseline_rc": "-",
|
||||
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
|
||||
"seconds": 0.0, "note": str(exc)[:120]}
|
||||
row["blob"] = blobs.get(rel, "-")[:12]
|
||||
rows.append(row)
|
||||
print(" -> %-17s %s" % (row["cls"], str(exc)[:90]))
|
||||
continue
|
||||
row["blob"] = blobs.get(rel, "-")[:12]
|
||||
row["path"] = rel
|
||||
rows.append(row)
|
||||
print(" -> %-17s rc=%s sites=%s tried=%s valid=%s moved_at=%s %ss"
|
||||
% (row["cls"], row["baseline_rc"], row["sites"], row["tried"],
|
||||
row["valid"], row["moved_at"], row["seconds"]))
|
||||
with open(args.tsv, "w", encoding="utf-8") as fh:
|
||||
write_header(fh, rev, len(rows), len(pop))
|
||||
for r in rows:
|
||||
fh.write("\t".join(str(r.get(k, "-")) for k in
|
||||
("cls", "path", "baseline_rc", "sites", "tried",
|
||||
"valid", "moved_at", "blob", "seconds")) + "\n")
|
||||
|
||||
with open(args.tsv, "w", encoding="utf-8") as fh:
|
||||
merged = rows
|
||||
if args.merge and prev_rows:
|
||||
by_path = {r["path"]: r for r in prev_rows}
|
||||
for r in rows:
|
||||
by_path[r["path"]] = r
|
||||
merged = [by_path[k] for k in sorted(by_path)]
|
||||
write_header(fh, rev, len(merged), len(population(tree_blobs(repo, rev))))
|
||||
for r in merged:
|
||||
fh.write("\t".join(str(r.get(k, "-")) for k in
|
||||
("cls", "path", "baseline_rc", "sites", "tried",
|
||||
"valid", "moved_at", "blob", "seconds")) + "\n")
|
||||
print("wrote %s (%d rows)" % (args.tsv, len(merged)))
|
||||
print("work copy left at %s" % work)
|
||||
return 0
|
||||
|
||||
|
||||
def do_check(args):
|
||||
if not os.path.isfile(args.tsv):
|
||||
print("q65_rc_probe: no ledger at %s -- cannot grade" % args.tsv,
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
head, rows = read_tsv(args.tsv)
|
||||
except ValueError as exc:
|
||||
print("q65_rc_probe: %s" % exc, file=sys.stderr)
|
||||
return 1
|
||||
repo = os.path.abspath(args.repo)
|
||||
try:
|
||||
blobs = tree_blobs(repo, args.rev)
|
||||
except SystemExit as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 2
|
||||
|
||||
faults = []
|
||||
counts = {c: 0 for c in CLASSES}
|
||||
for r in rows:
|
||||
if r["cls"] not in CLASSES:
|
||||
faults.append("UNKNOWN_CLASS %s: %r" % (r["path"], r["cls"]))
|
||||
continue
|
||||
counts[r["cls"]] += 1
|
||||
if r["path"] not in blobs:
|
||||
faults.append("MISSING %s: row in the ledger, absent at %s" % (r["path"], args.rev[:12]))
|
||||
continue
|
||||
if not blobs[r["path"]].startswith(r["blob"]):
|
||||
faults.append("STALE %s: row graded blob %s, tree holds %s"
|
||||
% (r["path"], r["blob"], blobs[r["path"]][:12]))
|
||||
try:
|
||||
moved = int(r["moved_at"]) if r["moved_at"] not in ("-", "") else None
|
||||
tried, valid = int(r["tried"]), int(r["valid"])
|
||||
sites = int(r["sites"]) if r["sites"] not in ("-", "") else None
|
||||
except ValueError:
|
||||
faults.append("BAD_CELL %s: non-numeric tried/valid/moved_at" % r["path"])
|
||||
continue
|
||||
if r["cls"] == "sensitive" and not moved:
|
||||
faults.append("CONTRADICTION %s: sensitive with no moving line" % r["path"])
|
||||
if r["cls"] == "inert" and not (valid >= 1 and sites is not None
|
||||
and tried >= sites and moved is None):
|
||||
faults.append("CONTRADICTION %s: inert with valid=%d tried=%s sites=%s moved=%s"
|
||||
% (r["path"], valid, tried, sites, moved))
|
||||
if r["cls"] == "inert-partial" and not (valid >= 1 and sites is not None
|
||||
and tried < sites and moved is None):
|
||||
faults.append("CONTRADICTION %s: inert-partial with valid=%d tried=%s "
|
||||
"sites=%s moved=%s"
|
||||
% (r["path"], valid, tried, sites, moved))
|
||||
if r["cls"] == "no-comparison" and sites:
|
||||
faults.append("CONTRADICTION %s: no-comparison but lists %d site(s)"
|
||||
% (r["path"], sites))
|
||||
if r["cls"] == "unprobed-mutant" and valid:
|
||||
faults.append("CONTRADICTION %s: unprobed-mutant with %d valid mutant(s)"
|
||||
% (r["path"], valid))
|
||||
|
||||
committed = population(blobs)
|
||||
probed = {r["path"] for r in rows}
|
||||
print("q65_rc_probe -- rev %s, %d row(s), %d committed verifier(s)"
|
||||
% (args.rev[:12], len(rows), len(committed)))
|
||||
for c in CLASSES:
|
||||
print(" %-18s %d" % (c, counts[c]))
|
||||
print(" rc-sensitive %d of %d probed of %d committed"
|
||||
% (counts["sensitive"], counts["sensitive"] + counts["inert"]
|
||||
+ counts["inert-partial"] + counts["no-comparison"], len(committed)))
|
||||
gap = [p for p in committed if p not in probed]
|
||||
if gap:
|
||||
print(" not probed %d (population coverage is the census's leg, "
|
||||
"not this one)" % len(gap))
|
||||
if faults:
|
||||
for f in faults:
|
||||
print(" FAULT %s" % f)
|
||||
print("q65_rc_probe: %d fault(s)" % len(faults))
|
||||
return 1
|
||||
print("q65_rc_probe: 0 fault(s) -- every row content-bound and consistent "
|
||||
"with its class")
|
||||
return 0
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- selftest
|
||||
def selftest():
|
||||
tmp = tempfile.mkdtemp(prefix="rcprobe-selftest-", dir=SCRATCH_ROOT)
|
||||
cells = []
|
||||
|
||||
def cell(name, ok, detail=""):
|
||||
cells.append((name, bool(ok), detail))
|
||||
|
||||
def mk(name, body):
|
||||
d = os.path.join(tmp, name)
|
||||
os.makedirs(d, exist_ok=True)
|
||||
p = os.path.join(d, "verify_x.py")
|
||||
with open(p, "w", encoding="utf-8") as fh:
|
||||
fh.write(body)
|
||||
return p
|
||||
|
||||
# c1 sensitive: the inverted assertion reaches sys.exit
|
||||
p = mk("sensitive", "import sys\nx = 2\nif x != 2:\n sys.exit(0)\n"
|
||||
"if x == 2:\n print('ok')\nelse:\n sys.exit(1)\n")
|
||||
r = probe(p, 3, 20)
|
||||
cell("c1 an inverted assertion that reaches sys.exit is sensitive",
|
||||
r["cls"] == "sensitive" and r["moved_at"], str(r))
|
||||
|
||||
# c2 inert: comparisons whose result is only printed
|
||||
p = mk("inert", "x = 2\ny = 3\nprint('x==2', x == 2)\nprint('y<x', y < x)\n")
|
||||
r = probe(p, 3, 20)
|
||||
cell("c2 comparisons that only print leave the rc unmoved (inert)",
|
||||
r["cls"] == "inert" and r["valid"] >= 1, str(r))
|
||||
|
||||
# c3 no comparison at all
|
||||
p = mk("nocomp", "print('all checks passed')\n")
|
||||
r = probe(p, 3, 20)
|
||||
cell("c3 a file with no comparison is no-comparison",
|
||||
r["cls"] == "no-comparison", str(r))
|
||||
|
||||
# c4 baseline non-zero
|
||||
p = mk("baseline", "import sys\nsys.exit(2)\n")
|
||||
r = probe(p, 3, 20)
|
||||
cell("c4 a non-zero baseline is unprobed-baseline, not a pass",
|
||||
r["cls"] == "unprobed-baseline", str(r))
|
||||
|
||||
# c5 every mutant invalid (the flip reaches an unreachable branch and crashes)
|
||||
p = mk("invalid", "x = None\nif x == None:\n print('none')\nelse:\n"
|
||||
" print(x.attr)\n")
|
||||
r = probe(p, 3, 20)
|
||||
cell("c5 a mutant that crashes is invalid, and no valid mutant is not inert",
|
||||
r["cls"] == "unprobed-mutant" and r["tried"] == 1 and r["valid"] == 0, str(r))
|
||||
cell("c5b the crashed-mutant file was restored byte for byte",
|
||||
open(p).read() == "x = None\nif x == None:\n print('none')\nelse:\n"
|
||||
" print(x.attr)\n")
|
||||
|
||||
# c5c one invalid mutant beside a valid one: counted, discarded, still graded
|
||||
p = mk("mixed", "x = None\ny = 1\nif x == None:\n print('none')\nelse:\n"
|
||||
" print(x.attr)\nif y == 1:\n print('y')\n")
|
||||
r = probe(p, 3, 20)
|
||||
cell("c5c an invalid mutant is counted and discarded, not counted as a test",
|
||||
r["cls"] == "inert" and r["tried"] == 2 and r["valid"] == 1, str(r))
|
||||
|
||||
# c5d the sample is not the verdict: a moving site outside the 3-site
|
||||
# even-spaced sample is found by the exhaustive pass
|
||||
body5d = ("import sys\n"
|
||||
"x1 = 1\n"
|
||||
"x2 = 2\n"
|
||||
"x3 = 3\n"
|
||||
"x4 = 4\n"
|
||||
"x5 = 5\n"
|
||||
"print(x1 == 1)\n"
|
||||
"if x2 == 2:\n"
|
||||
" sys.exit(0)\n"
|
||||
"print(x3 == 3)\n"
|
||||
"print(x4 == 4)\n"
|
||||
"print(x5 == 5)\n"
|
||||
"sys.exit(1)\n")
|
||||
p = mk("outside_sample", body5d)
|
||||
r = probe(p, 3, 20)
|
||||
cell("c5d a moving site the 3-site sample misses is still found",
|
||||
r["cls"] == "sensitive" and r["moved_at"] == 8 and r["sites"] == 5,
|
||||
str(r))
|
||||
cell("c5d2 the sample really did miss it (site 8 is not in spread(sites,3))",
|
||||
8 not in spread(mutable_lines(body5d), 3),
|
||||
str(spread(mutable_lines(body5d), 3)))
|
||||
|
||||
# c5e the same file under a cap that stops before the moving site is
|
||||
# inert-partial, not inert and not sensitive
|
||||
p = mk("capped", body5d)
|
||||
r = probe(p, 3, 20, exhaustive_cap=3)
|
||||
cell("c5e a cap that stops before the moving site is inert-partial",
|
||||
r["cls"] == "inert-partial" and r["tried"] == 3 and r["sites"] == 5,
|
||||
str(r))
|
||||
|
||||
# c6 restore on a sensitive file
|
||||
p = mk("restore", "import sys\nx = 1\nif x == 1:\n sys.exit(0)\n")
|
||||
before = open(p, "rb").read()
|
||||
probe(p, 3, 20)
|
||||
cell("c6 a probed file is restored byte for byte", open(p, "rb").read() == before)
|
||||
|
||||
# c7 spread is deterministic and even
|
||||
picks = list(range(0, 100))
|
||||
cell("c7 spread picks evenly and deterministically",
|
||||
spread(picks, 3) == spread(picks, 3) and len(spread(picks, 3)) == 3
|
||||
and spread(picks, 3)[0] == 0 and spread(picks, 3)[-1] == 99,
|
||||
str(spread(picks, 3)))
|
||||
cell("c7b spread never invents a site", all(s in picks for s in spread(picks, 7)))
|
||||
|
||||
# c8 tsv round-trip
|
||||
tsv = os.path.join(tmp, "led.tsv")
|
||||
rows = [
|
||||
{"cls": "sensitive", "path": "a/verify_a.py", "baseline_rc": 0, "sites": 4,
|
||||
"tried": 2, "valid": 2, "moved_at": 12, "blob": "abc123", "seconds": 0.1},
|
||||
{"cls": "inert", "path": "a/verify_b.py", "baseline_rc": 0, "sites": 1,
|
||||
"tried": 1, "valid": 1, "moved_at": "-", "blob": "def456", "seconds": 0.2},
|
||||
]
|
||||
with open(tsv, "w") as fh:
|
||||
write_header(fh, "f" * 40, len(rows), 2)
|
||||
for r in rows:
|
||||
fh.write("\t".join(str(r[k]) for k in ("cls", "path", "baseline_rc",
|
||||
"sites", "tried", "valid",
|
||||
"moved_at", "blob", "seconds")) + "\n")
|
||||
head, back = read_tsv(tsv)
|
||||
cell("c8 the ledger round-trips its rows and its header",
|
||||
len(back) == 2 and head.get("rows") == "2" and back[0]["moved_at"] == "12",
|
||||
str(head))
|
||||
|
||||
# c9 a short row is refused, not silently skipped
|
||||
bad = os.path.join(tmp, "short.tsv")
|
||||
open(bad, "w").write("# rev " + "f" * 40 + "\n# rows 1\n# committed 1\n"
|
||||
"inert\ta/verify_a.py\t0\t1\n")
|
||||
try:
|
||||
read_tsv(bad)
|
||||
cell("c9 a short row is refused", False)
|
||||
except ValueError:
|
||||
cell("c9 a short row is refused", True)
|
||||
|
||||
# c10 --check refuses a --work that is the tree
|
||||
ns = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=DEFAULT_TSV,
|
||||
only=None, limit=0, work=os.path.realpath(DEFAULT_REPO),
|
||||
max_mutants=1, timeout=1,
|
||||
windows=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||
skip_over=300.0, merge=False)
|
||||
cell("c10 --emit refuses --work equal to the repo root", do_emit(ns) == 2)
|
||||
|
||||
# c12 the window table parses, and a malformed row is skipped not fatal
|
||||
wt = os.path.join(tmp, "win.tsv")
|
||||
open(wt, "w").write("# path\ts\twhy\na/verify_a.py\t1800\tlive HTTP\n"
|
||||
"b/verify_b.py\tnotanumber\tbroken\n")
|
||||
w = read_windows(wt)
|
||||
cell("c12 the window table parses a path and skips a malformed row",
|
||||
w.get("a/verify_a.py") == 1800.0 and "b/verify_b.py" not in w, str(w))
|
||||
|
||||
# c13 a verifier over the skip-over is unprobed-window, and the emit still succeeds
|
||||
one = os.path.join(tmp, "one.tsv")
|
||||
ns3 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=one,
|
||||
only="wizard-work/sphinx/archive-index-2026-10-23/verify_index.py",
|
||||
limit=0, work=os.path.join(tmp, "work"),
|
||||
max_mutants=1, timeout=1,
|
||||
windows=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||
skip_over=1.0, merge=False)
|
||||
rc = do_emit(ns3)
|
||||
head2, rows2 = read_tsv(one)
|
||||
cell("c13 a file whose window exceeds --skip-over is unprobed-window, rc 0",
|
||||
rc == 0 and len(rows2) == 1 and rows2[0]["cls"] == "unprobed-window",
|
||||
str(rows2))
|
||||
|
||||
# c15 --merge keeps a row this run did not probe
|
||||
ns4 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=one,
|
||||
only="wizard-work/spark/verify_sum.py",
|
||||
limit=0, work=os.path.join(tmp, "work"),
|
||||
max_mutants=1, timeout=30,
|
||||
windows=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||
skip_over=300.0, merge=True)
|
||||
rc = do_emit(ns4)
|
||||
head3, rows3 = read_tsv(one)
|
||||
kept = [r for r in rows3 if r["path"].endswith("archive-index-2026-10-23/verify_index.py")]
|
||||
fresh = [r for r in rows3 if r["path"].endswith("spark/verify_sum.py")]
|
||||
cell("c15 --merge keeps the unprobed row beside the new one",
|
||||
rc == 0 and len(rows3) == 2 and kept and fresh, str(rows3))
|
||||
|
||||
# c14 a file the sweep cannot read raises (do_emit turns it into a row)
|
||||
try:
|
||||
probe(os.path.join(tmp, "absent", "verify_x.py"), 1, 5)
|
||||
cell("c14 an unreadable file raises OSError, caught into a row", False)
|
||||
except OSError:
|
||||
cell("c14 an unreadable file raises OSError, caught into a row", True)
|
||||
|
||||
# c16 --check enforces the coverage half of the inert claim
|
||||
blobs_now = tree_blobs(DEFAULT_REPO, "HEAD")
|
||||
real = sorted(p for p in blobs_now
|
||||
if p.startswith("wizard-work/")
|
||||
and os.path.basename(p).startswith("verify_"))[0]
|
||||
b12 = blobs_now[real][:12]
|
||||
|
||||
def _led(cls, tried, sites):
|
||||
f = os.path.join(tmp, "chk-%s-%d-%d.tsv" % (cls, tried, sites))
|
||||
with open(f, "w", encoding="utf-8") as fh:
|
||||
write_header(fh, "HEAD", 1, 1)
|
||||
fh.write("\t".join([cls, real, "0", str(sites), str(tried),
|
||||
"1", "-", b12, "0.1"]) + "\n")
|
||||
return do_check(argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=f))
|
||||
|
||||
cell("c16 inert with sites untried is a contradiction (rc 1)",
|
||||
_led("inert", 2, 5) == 1)
|
||||
cell("c16b inert with every site tried is consistent (rc 0)",
|
||||
_led("inert", 5, 5) == 0)
|
||||
cell("c16c inert-partial with sites untried is consistent (rc 0)",
|
||||
_led("inert-partial", 2, 5) == 0)
|
||||
cell("c16d inert-partial with every site tried is a contradiction (rc 1)",
|
||||
_led("inert-partial", 5, 5) == 1)
|
||||
|
||||
# c11 --check on a missing ledger is rc 2 (cannot grade), not rc 0
|
||||
ns2 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD",
|
||||
tsv=os.path.join(tmp, "nope.tsv"))
|
||||
cell("c11 a missing ledger is rc 2, not a pass", do_check(ns2) == 2)
|
||||
|
||||
ok = sum(1 for _, o, _ in cells if o)
|
||||
for name, o, detail in cells:
|
||||
print("%s %s%s" % ("ok " if o else "FAIL", name,
|
||||
"" if o else " [%s]" % detail))
|
||||
print("selftest: %d cells, %d passed, %d failed" % (len(cells), ok, len(cells) - ok))
|
||||
return 0 if ok == len(cells) else 1
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--repo", default=DEFAULT_REPO)
|
||||
ap.add_argument("--rev", default="HEAD")
|
||||
ap.add_argument("--tsv", default=DEFAULT_TSV)
|
||||
ap.add_argument("--work", help="copy dir to mutate (default: fresh scratch dir)")
|
||||
ap.add_argument("--max-mutants", type=int, default=3)
|
||||
ap.add_argument("--exhaustive-cap", type=int, default=40,
|
||||
help="sites per file tried before an inert verdict is "
|
||||
"downgraded to inert-partial; 0 = no cap")
|
||||
ap.add_argument("--timeout", type=float, default=90.0,
|
||||
help="window for a file the window table does not name")
|
||||
ap.add_argument("--windows", default=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||
help="per-path measurement windows, shared with the runner")
|
||||
ap.add_argument("--skip-over", type=float, default=300.0,
|
||||
help="a verifier whose window exceeds this is not mutated; "
|
||||
"grading it is the runner's row, not this sweep's")
|
||||
ap.add_argument("--only", help="with --emit: only paths containing this substring")
|
||||
ap.add_argument("--limit", type=int, default=0)
|
||||
ap.add_argument("--emit", action="store_true")
|
||||
ap.add_argument("--merge", action="store_true",
|
||||
help="with --emit: keep rows for paths this run did not probe")
|
||||
ap.add_argument("--check", action="store_true")
|
||||
ap.add_argument("--selftest", action="store_true")
|
||||
args = ap.parse_args()
|
||||
if args.selftest:
|
||||
return selftest()
|
||||
if args.check:
|
||||
return do_check(args)
|
||||
if args.emit:
|
||||
return do_emit(args)
|
||||
ap.print_help()
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,118 @@
|
||||
# Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.
|
||||
# Written by q65_rc_probe.py --emit. A row states a measurement, not an
|
||||
# exemption: --check faults on a row that contradicts its class or whose
|
||||
# file has changed since (STALE).
|
||||
# columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds
|
||||
# rev 6e8f577c8cdd17ae7e6d5d2d20513a183ec19d94
|
||||
# rows 110
|
||||
# committed 110
|
||||
sensitive wizard-work/beam/verify_toposort_none.py 0 4 1 1 21 a0df9ceb99a1 0.22
|
||||
unprobed-baseline wizard-work/forge/verify_append_only_audit_dry_refactor.py 1 17 0 0 - 09d40fbd2ca0 2.16
|
||||
sensitive wizard-work/forge/verify_append_only_owners_dry_refactor.py 0 13 1 1 56 0f650718d98b 0.65
|
||||
sensitive wizard-work/forge/verify_census_row_delta_exit_contract.py 0 23 1 1 46 6afe76a665fa 1.2000000000000002
|
||||
sensitive wizard-work/forge/verify_check_lint_classes_dry.py 0 15 1 1 80 62082e082a04 0.68
|
||||
unprobed-baseline wizard-work/forge/verify_check_lint_dry_refactor.py 1 14 0 0 - 06a5c66c3f31 1.22
|
||||
sensitive wizard-work/forge/verify_checkin_check_dry_refactor.py 0 8 1 1 103 d8ba2b221b80 0.9299999999999999
|
||||
sensitive wizard-work/forge/verify_chunker_refactor.py 0 11 8 7 123 c3c7b5631f4a 0.8499999999999999
|
||||
unprobed-timeout wizard-work/forge/verify_citation_hash_class_dry_refactor.py TIMEOUT 12 0 0 - 93abbc78e7f0 90.08
|
||||
unprobed-window wizard-work/forge/verify_citation_hash_class_portable_default.py - - 0 0 - 81c481978bb3 0.0
|
||||
unprobed-baseline wizard-work/forge/verify_citation_resolve_check_dry_refactor.py 1 9 0 0 - cd325ddd210d 15.52
|
||||
unprobed-window wizard-work/forge/verify_doc_census_dry_refactor.py - - 0 0 - 52a9702aa0e0 0.0
|
||||
unprobed-baseline wizard-work/forge/verify_doc_command_report_dry.py 1 8 0 0 - 7a2f42a5b95d 7.96
|
||||
unprobed-baseline wizard-work/forge/verify_gate_blame_dry_refactor.py 1 9 0 0 - 8d01677402c9 10.81
|
||||
sensitive wizard-work/forge/verify_landing_check_refactor.py 0 25 1 1 103 6aa282dade61 3.92
|
||||
unprobed-baseline wizard-work/forge/verify_path_claim_apply_baseline_refactor.py 1 15 0 0 - 999da7274c3c 0.31
|
||||
unprobed-baseline wizard-work/forge/verify_path_claim_dry_refactor.py 1 17 0 0 - ab35c13f9565 0.55
|
||||
sensitive wizard-work/forge/verify_pdf_refactor.py 0 14 1 1 79 44ce76d39b69 7.27
|
||||
sensitive wizard-work/forge/verify_probe_forge_refactor.py 0 21 1 1 67 0b9040ad6834 0.83
|
||||
unprobed-baseline wizard-work/forge/verify_queue_append_audit_dry_refactor.py 1 13 0 0 - 0d5e5ef90a0d 7.81
|
||||
sensitive wizard-work/forge/verify_reply_check_dry_refactor.py 0 11 1 1 72 c94ebf0e965e 3.0
|
||||
sensitive wizard-work/forge/verify_repro_bounds_refactor.py 0 17 1 1 69 2394abeec257 0.94
|
||||
sensitive wizard-work/forge/verify_roundtrip_refactor.py 0 8 1 1 39 0f1d7ea9d5e5 1.26
|
||||
sensitive wizard-work/forge/verify_scan_dry_refactor.py 0 21 1 1 78 6fb7aa14b96e 6.3
|
||||
unprobed-baseline wizard-work/forge/verify_selftest_sweep_dry_refactor.py 1 15 0 0 - 6f5e1621139c 0.25
|
||||
unprobed-window wizard-work/forge/verify_selftest_sweep_refactor.py - - 0 0 - cc8782b4edfc 0.0
|
||||
sensitive wizard-work/forge/verify_served_drift_summarise_dry.py 0 16 1 1 100 de5a49160b2c 12.190000000000001
|
||||
sensitive wizard-work/forge/verify_smell_check_monotonic_refactor.py 0 16 1 1 64 8b8ccea5be0e 1.6800000000000002
|
||||
sensitive wizard-work/forge/verify_smell_scan_refactor.py 0 11 1 1 55 499c8e000359 1.0
|
||||
sensitive wizard-work/forge/verify_tool_map_dry_refactor.py 0 9 1 1 40 de1d9b7a65cf 49.33
|
||||
unprobed-baseline wizard-work/forge/verify_write_probe_dry_refactor.py 1 12 0 0 - ef4d8c00e4a8 23.35
|
||||
sensitive wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py 0 4 2 2 90 77048ae1e3f1 3.45
|
||||
sensitive wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py 0 2 1 1 122 e12a07ca75fc 0.69
|
||||
sensitive wizard-work/ledger/verify_arxiv_csai.py 0 14 1 1 37 872574acb239 0.49
|
||||
sensitive wizard-work/ledger/verify_arxiv_csai_fresh.py 0 18 1 1 42 1a5acdde626e 0.47
|
||||
sensitive wizard-work/ledger/verify_arxiv_csv.py 0 4 2 2 29 cc898af42ea8 0.33999999999999997
|
||||
sensitive wizard-work/ledger/verify_arxiv_fresh.py 0 8 1 1 48 8c6eb40bcf4d 0.19
|
||||
sensitive wizard-work/ledger/verify_catalog.py 0 14 2 1 77 d08c5d389ece 0.39
|
||||
sensitive wizard-work/ledger/verify_commits_check.py 0 2 1 1 122 e84e0c7dd3ed 0.58
|
||||
inert wizard-work/ledger/verify_dates.py 0 2 2 2 - dd3f643441c9 0.48
|
||||
sensitive wizard-work/ledger/verify_gdp.py 0 12 2 2 27 ba577cd8ae05 0.29000000000000004
|
||||
sensitive wizard-work/ledger/verify_gdp_per_capita.py 0 4 1 1 42 7f74b350c60e 1.52
|
||||
unprobed-baseline wizard-work/ledger/verify_gitea_repos.py 1 8 0 0 - 73ef858149be 0.24
|
||||
unprobed-baseline wizard-work/ledger/verify_github_python_repos.py 1 9 0 0 - 6956aa9c1560 2.86
|
||||
sensitive wizard-work/ledger/verify_github_repos.py 0 14 2 2 107 269870d5e95a 5.529999999999999
|
||||
sensitive wizard-work/ledger/verify_issues.py 0 4 1 1 48 176fd33f0522 0.22
|
||||
unprobed-baseline wizard-work/ledger/verify_python_org_repos.py 1 9 0 0 - da60445425c4 2.02
|
||||
sensitive wizard-work/ledger/verify_stackchain_catalog.py 0 8 1 1 105 60aa1853a04b 0.16999999999999998
|
||||
sensitive wizard-work/ledger/verify_users.py 0 6 2 2 40 ef9b73c9cdd3 0.32
|
||||
unprobed-baseline wizard-work/ledger/verify_wizards.py 1 19 0 0 - e08c8675a4e4 15.69
|
||||
unprobed-baseline wizard-work/ledger/verify_workflow.py 1 4 0 0 - 8f9a0ce47f5c 0.22
|
||||
sensitive wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py 0 9 2 2 85 d88e5d606684 1.47
|
||||
unprobed-baseline wizard-work/null/diffkit/verify_realrepo.py 1 12 0 0 - 1e610f00fa50 0.38
|
||||
sensitive wizard-work/sol/verify_pending_request_triage.py 0 4 1 1 27 2d059319d3da 0.19
|
||||
sensitive wizard-work/sol/verify_sol_profile_triage.py 0 5 1 1 37 84386c791a61 1.65
|
||||
unprobed-baseline wizard-work/sol/verify_sol_triage.py 1 11 0 0 - 63792506caf3 0.09
|
||||
sensitive wizard-work/sol/verify_triage.py 0 6 1 1 9 a2ffc04767ef 0.18
|
||||
unprobed-baseline wizard-work/sol/verify_triage_queue.py 1 11 0 0 - 018ca9a2b570 0.13
|
||||
sensitive wizard-work/sol/verify_troupe_sol_health.py 0 8 1 1 43 4b1a409c9b28 0.18
|
||||
no-comparison wizard-work/spark/verify_all.py 0 0 0 0 - 41029ee9976f 0.09
|
||||
sensitive wizard-work/spark/verify_analyzer.py 0 4 1 1 19 af7609cfadc1 0.44
|
||||
inert wizard-work/spark/verify_check.py 0 3 3 3 - 3b7ef86aa2f7 0.31
|
||||
inert wizard-work/spark/verify_count.py 0 1 1 1 - 640a65519f81 0.19
|
||||
sensitive wizard-work/spark/verify_demo.py 0 3 1 1 32 7e4325553fc8 0.2
|
||||
sensitive wizard-work/spark/verify_dir.py 0 3 1 1 15 ec6280d194d4 0.15000000000000002
|
||||
sensitive wizard-work/spark/verify_dir_analyzer.py 0 6 1 1 19 89a49e7684d6 2.7800000000000002
|
||||
no-comparison wizard-work/spark/verify_dirs.py 0 0 0 0 - d2846e746204 0.1
|
||||
no-comparison wizard-work/spark/verify_env.py 0 0 0 0 - d28af1c48c1c 0.08
|
||||
unprobed-baseline wizard-work/spark/verify_environment.py 1 3 0 0 - 4f526b94ecdd 0.07
|
||||
inert wizard-work/spark/verify_fib.py 0 4 4 4 - 163468d71cd7 0.4100000000000001
|
||||
sensitive wizard-work/spark/verify_formatter.py 0 4 1 1 14 0d2ef3a544f5 0.52
|
||||
unprobed-baseline wizard-work/spark/verify_hermes.py 1 5 0 0 - 22c2f9e1ddd8 0.13
|
||||
inert wizard-work/spark/verify_hermes_cli.py 0 9 9 9 - f3793430f6ab 10.519999999999998
|
||||
inert wizard-work/spark/verify_hermes_path.py 0 1 1 1 - 385362b75cfe 0.16
|
||||
unprobed-baseline wizard-work/spark/verify_inventory.py 1 2 0 0 - 02b54c2c98fe 0.1
|
||||
no-comparison wizard-work/spark/verify_json.py 0 0 0 0 - 71b343f0b1f3 0.13
|
||||
inert wizard-work/spark/verify_lines.py 0 1 1 1 - 5be4cf7b5f1e 0.25
|
||||
sensitive wizard-work/spark/verify_loc_report.py 0 8 1 1 14 8e8aa89896b3 0.25
|
||||
sensitive wizard-work/spark/verify_lwc.py 0 3 1 1 18 0ca330355ed9 0.56
|
||||
inert wizard-work/spark/verify_minimal.py 0 3 3 3 - 2b1b18fc7ed7 0.68
|
||||
no-comparison wizard-work/spark/verify_push.py 0 0 0 0 - 707bdcb80646 0.14
|
||||
inert wizard-work/spark/verify_python_env.py 0 1 1 1 - 17c1d5f6ab0b 0.19
|
||||
sensitive wizard-work/spark/verify_report.py 0 3 1 1 11 0b555f165e95 0.2
|
||||
sensitive wizard-work/spark/verify_result.py 0 2 1 1 12 108832bc1239 0.25
|
||||
sensitive wizard-work/spark/verify_scratch_analysis.py 0 6 1 1 17 8f8edfb7d4b3 0.19
|
||||
unprobed-baseline wizard-work/spark/verify_skill_system.py 1 6 0 0 - 6739088b6520 0.16
|
||||
no-comparison wizard-work/spark/verify_skills.py 0 0 0 0 - be69f202a823 0.11
|
||||
no-comparison wizard-work/spark/verify_skills2.py 0 0 0 0 - 9dd92b966a6e 0.09
|
||||
inert wizard-work/spark/verify_skills3.py 0 1 1 1 - 364a78ffbe33 0.24000000000000002
|
||||
inert wizard-work/spark/verify_skills_fixed.py 0 1 1 1 - 4bc11949dc38 0.22
|
||||
sensitive wizard-work/spark/verify_snippets.py 0 3 1 1 25 23d736539363 0.19
|
||||
sensitive wizard-work/spark/verify_status.py 0 5 1 1 10 4c2928cdb2f2 0.63
|
||||
no-comparison wizard-work/spark/verify_sum.py 0 0 0 0 - 8faaade76bc3 0.1
|
||||
sensitive wizard-work/spark/verify_tool.py 0 3 1 1 12 fd90c7f545ad 1.08
|
||||
sensitive wizard-work/spark/verify_wordcount.py 0 7 1 1 15 c4b7e15b56d0 0.46
|
||||
sensitive wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py 0 17 2 1 88 50644ea156a1 8.68
|
||||
sensitive wizard-work/sphinx/archive-gutenberg/verify_archive.py 0 12 2 2 63 36c966d37f08 1.2
|
||||
unprobed-window wizard-work/sphinx/archive-index-2026-10-23/verify_index.py - - 0 0 - 9d4088a373a6 0.0
|
||||
sensitive wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py 0 44 1 1 57 b252edd8547d 0.37
|
||||
unprobed-baseline wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py 1 29 0 0 - a15fab347cf6 4.96
|
||||
sensitive wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py 0 35 1 1 106 3c4b48a678a8 13.06
|
||||
unprobed-baseline wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py 1 2 0 0 - 22ba0b4bbdca 0.14
|
||||
sensitive wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py 0 10 1 1 61 39a5552b8500 0.48
|
||||
sensitive wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py 0 19 1 1 44 534af05a12fa 0.38
|
||||
sensitive wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py 0 22 1 1 93 c1f03701bf9a 7.75
|
||||
unprobed-window wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py - - 0 0 - bed5df5c4a24 0.0
|
||||
sensitive wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py 0 69 2 1 250 95997f60d905 9.260000000000002
|
||||
unprobed-baseline wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 1 10 0 0 - c106248aea18 40.7
|
||||
sensitive wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py 0 21 1 1 101 4b656a6d1d45 0.77
|
||||
unprobed-baseline wizard-work/sphinx/verify_mm_dossier.py 1 12 0 0 - 4b015581816d 3.49
|
||||
|
@@ -15,3 +15,4 @@ wizard-work/forge/verify_doc_census_dry_refactor.py 1800 runs tools/doc_census_c
|
||||
wizard-work/forge/verify_selftest_sweep_refactor.py 1800 runs tools/selftest_sweep.py over every pinned command
|
||||
wizard-work/sphinx/archive-index-2026-10-23/verify_index.py 1800 reads every file at three revisions through git plumbing
|
||||
wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py 1800 live HTTP fetches, 3 retries at a 90s per-request timeout
|
||||
wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 120 live HTTP: 1210 exact id_list lookups plus 3 listing pages; measured 19.87s at 5095e89, 6x headroom
|
||||
|
@@ -17,7 +17,7 @@
|
||||
# non-docstring string constant naming an absolute host root. Such a row
|
||||
# is printed PASS* and excluded from the graded-clean figure.
|
||||
# Regenerate: q65_runner.py --emit
|
||||
# rev: 25b04d0 rows: 88 emitted: q65_runner.py
|
||||
# rev: 6e8f577 rows: 91 emitted: q65_runner.py
|
||||
undeclared wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.32 0f650718d98bf28f07be134d49803cc902bf96b7 L41:/root/troupe-standup
|
||||
undeclared wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 1.23 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 L31:/root/troupe-standup
|
||||
gate-run wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.53 62082e082a0472546b415219a9180836ac88908b L44:/root/troupe-standup
|
||||
@@ -33,6 +33,7 @@ undeclared wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 22.04 8d0
|
||||
undeclared wizard-work/forge/verify_landing_check_refactor.py PASS 0 6.12 6aa282dade614e3e55874adc3e37cf7c01313f3e L62:/root/troupe-standup
|
||||
undeclared wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 0.52 FileNotFoundError: [Errno 2] No such file or directory: '/root/.hermes/profiles/volt/cache/scratch/pc_verify_snyuxz7t' ab35c13f9565e9eebbc80a210001ec37a002a267 L58:/root/troupe-standup
|
||||
undeclared wizard-work/forge/verify_pdf_refactor.py PASS 0 7.75 44ce76d39b69a7d6798af521f958370bf30ad2b1 L41:/root/.hermes
|
||||
undeclared wizard-work/forge/verify_probe_forge_refactor.py PASS 0 0.42 0b9040ad683464475d77b58c744f16dfbdb8438e
|
||||
undeclared wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 16.94 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 L51:/root/troupe-standup
|
||||
undeclared wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 7.31 c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 L33:/root/troupe-standup
|
||||
undeclared wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.35 2394abeec25726726edd91696520d788d72a5d88 L33:/root/troupe-standup
|
||||
@@ -52,6 +53,7 @@ undeclared wizard-work/ledger/verify_arxiv_fresh.py PASS 0 0.09 8c6eb40bcf4d512
|
||||
undeclared wizard-work/ledger/verify_catalog.py PASS 0 0.08 d08c5d389ece6b9906a7315849dfe83f99f28f41 L13:/root/.hermes
|
||||
undeclared wizard-work/ledger/verify_dates.py PASS 0 0.13 dd3f643441c9fd985639c0ac7f250e05a6184aeb L6:/root/troupe-standup
|
||||
undeclared wizard-work/ledger/verify_gdp.py PASS 0 0.10 ba577cd8ae0502f57bfd7b8a7dadd8ef54455dbd L3:/root/troupe-standup
|
||||
undeclared wizard-work/ledger/verify_gdp_per_capita.py PASS 0 0.84 7f74b350c60ee7fd0e361f8d60a2e669ae703a97 L19:/root/troupe-standup
|
||||
undeclared wizard-work/ledger/verify_gitea_repos.py FAIL 1 0.23 73ef858149be5fd7e9ec58056435c3bd31f9419c L16:/root/.hermes
|
||||
undeclared wizard-work/ledger/verify_github_python_repos.py FAIL 1 3.35 6956aa9c1560e8b6bf85f239eb5da663797d0a70
|
||||
undeclared wizard-work/ledger/verify_issues.py PASS 0 0.09 176fd33f05223aaaf789608b662c3d253f685422 L15:/root/.hermes
|
||||
@@ -105,4 +107,5 @@ undeclared wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.
|
||||
undeclared wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py PASS 0 0.21 534af05a12fa9d3285384bae57bce527ed033425
|
||||
wired-invoke wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py PASS 0 2.28 c1f03701bf9af0018e7aac9a567255dd8218488d
|
||||
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py FAIL 1 837.56 bed5df5c4a24c961a3c2bb564b299444a22d5189
|
||||
undeclared wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py PASS 0 19.87 c106248aea1856d29d987497af4fe95dfe6d9768 L59:/root/.hermes
|
||||
undeclared wizard-work/sphinx/verify_mm_dossier.py PASS 0 4.30 4b015581816de5472d11577d661ce4e97f38773a L28:/root/.hermes
|
||||
|
Reference in new issue
Block a user