volt: Q65 cycle 6's rc-probe leg lands (it was on disk, uncommitted, since 7cd4dd5) + forge's new verifier graded -- census 110 of 110, runner 91 rows, CHECK rc 0
Two facts in one instrument commit, stated rather than implied: 1. cycle 6 was never committed. `q65_rc_probe.py` and its 109-row TSV were untracked in the worktree, the CHECK's rc-probe legs existed only on disk, and the two ledgers carried 3+2 uncommitted rows. Measured before landing, not trusted: `q65_rc_probe.py --selftest` 25/25 and `--check` 0 faults at the worktree state; the state is now the one the gate reads. 2. cycle 7 is the file that reopened the census. forge's6e8f577adds `wizard-work/forge/verify_probe_forge_refactor.py` (9,890 B, blob0b9040ad68), invoked by nothing, rowed by nothing: `bash wizard-work/volt/CHECK` rc 1, `UNCLASSED ... 1 unclassed of 110`, and `q63_verify_scan.py` `NEW_UNWIRED 1` -- both legs, two classes, one file. Graded by execution through the runner (PASS rc 0, 0.42s, host column empty so the row is `graded-clean`, not `PASS*`) and rc-probed (sensitive, moved at L67 after 21 sites were enumerated). Re-emitted at6e8f577: census 110 rows (gate-run 91, wired-invoke 11, declared-one-off 8), runner ledger 91 rows (PASS 67, FAIL 24), rc-probe 110 rows (sensitive 59). The re-emit is not a pure append and the record names the non-append hunk: `ledger/verify_gdp_per_capita.py` moved `undeclared` -> `gate-run q65:PASS`, derived from a graded row that was in the uncommitted ledger and is reproduced by the tree.
This commit is contained in:
1 parent
6e8f577c8c
commit
eb345839ac
12 files changed
+1464
-3
No files matched your search
+18
-1
@@ -16,6 +16,21 @@
|
|||||||
# have none -- all spark, recorded as findings, not repaired here; 14 of the
|
# have none -- all spark, recorded as findings, not repaired here; 14 of the
|
||||||
# runner ledger's 64 PASS rows are rows on such a file and cannot fail.
|
# runner ledger's 64 PASS rows are rows on such a file and cannot fail.
|
||||||
#
|
#
|
||||||
|
# Since cycle 6 it grades that same claim dynamically, and about a process that
|
||||||
|
# actually ran. `q65_rc_probe.py --emit` inverts each verifier's own comparisons
|
||||||
|
# one at a time in a detached worktree and asks whether the exit code moves;
|
||||||
|
# `--check` is the fast half run here: every row in q65_rc_probe.tsv must still
|
||||||
|
# match the blob at HEAD (STALE) and must not contradict its own class. A file
|
||||||
|
# cannot be given a row it did not earn, and a file that changed after its row
|
||||||
|
# was written cannot keep it.
|
||||||
|
#
|
||||||
|
# `inert` is the one class whose negative is unbounded by construction, so the
|
||||||
|
# sweep tries EVERY comparison site before writing it; a file stopped by
|
||||||
|
# `--exhaustive-cap` is `inert-partial`, and --check faults on either word used
|
||||||
|
# outside its coverage. This is cycle 6's own correction: a 3-of-11 sample wrote
|
||||||
|
# `inert` on forge/verify_chunker_refactor.py, whose rc moves at 3 of its 11
|
||||||
|
# sites (L123, L139, L142) that the sample did not pick.
|
||||||
|
#
|
||||||
# The root is resolved from $0, not written down. A gate that `cd`s to an
|
# The root is resolved from $0, not written down. A gate that `cd`s to an
|
||||||
# absolute host path grades THAT tree from whatever checkout invokes it: at
|
# absolute host path grades THAT tree from whatever checkout invokes it: at
|
||||||
# `66e5be1`, a clone with one verifier deleted printed `0 unclassed of 88` and
|
# `66e5be1`, a clone with one verifier deleted printed `0 unclassed of 88` and
|
||||||
@@ -37,6 +52,8 @@ python3 wizard-work/volt/q65_runner.py --check --class gate-run,undeclared || ex
|
|||||||
python3 wizard-work/volt/q63_verify_scan.py --repo . >/dev/null || exit 1
|
python3 wizard-work/volt/q63_verify_scan.py --repo . >/dev/null || exit 1
|
||||||
python3 wizard-work/volt/q65_no_verdict.py --selftest >/dev/null || exit 1
|
python3 wizard-work/volt/q65_no_verdict.py --selftest >/dev/null || exit 1
|
||||||
python3 wizard-work/volt/q65_no_verdict.py --repo . --check >/dev/null || exit 1
|
python3 wizard-work/volt/q65_no_verdict.py --repo . --check >/dev/null || exit 1
|
||||||
|
python3 wizard-work/volt/q65_rc_probe.py --selftest >/dev/null || exit 1
|
||||||
|
python3 wizard-work/volt/q65_rc_probe.py --repo . --check >/dev/null || exit 1
|
||||||
|
|
||||||
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean; no unrowed printer."
|
echo "CHECK PASS: census covers every committed verifier; runner ledger grades clean; no unrowed printer; rc-sensitivity ledger content-bound."
|
||||||
exit 0
|
exit 0
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# Q63 verifier census -- one row per COMMITTED verify_*.py at the rev the
|
# Q63 verifier census -- one row per COMMITTED verify_*.py at the rev the
|
||||||
# scan read, class plus the evidence the class rests on.
|
# scan read, class plus the evidence the class rests on.
|
||||||
# rev: 25b04d0 (the tree these classes were read at)
|
# rev: 6e8f577 (the tree these classes were read at)
|
||||||
# row: <class> TAB <path> TAB <evidence> (three tab-separated columns)
|
# row: <class> TAB <path> TAB <evidence> (three tab-separated columns)
|
||||||
# class gate-run = q65_runner.py executed it and recorded an rc (Q65);
|
# class gate-run = q65_runner.py executed it and recorded an rc (Q65);
|
||||||
# class wired-invoke = a committed line invokes it (the ref is the evidence);
|
# class wired-invoke = a committed line invokes it (the ref is the evidence);
|
||||||
@@ -30,6 +30,7 @@ gate-run wizard-work/forge/verify_landing_check_refactor.py q65:PASS
|
|||||||
declared-one-off wizard-work/forge/verify_path_claim_apply_baseline_refactor.py in-file declaration
|
declared-one-off wizard-work/forge/verify_path_claim_apply_baseline_refactor.py in-file declaration
|
||||||
gate-run wizard-work/forge/verify_path_claim_dry_refactor.py q65:FAIL
|
gate-run wizard-work/forge/verify_path_claim_dry_refactor.py q65:FAIL
|
||||||
gate-run wizard-work/forge/verify_pdf_refactor.py q65:PASS
|
gate-run wizard-work/forge/verify_pdf_refactor.py q65:PASS
|
||||||
|
gate-run wizard-work/forge/verify_probe_forge_refactor.py q65:PASS
|
||||||
gate-run wizard-work/forge/verify_queue_append_audit_dry_refactor.py q65:FAIL
|
gate-run wizard-work/forge/verify_queue_append_audit_dry_refactor.py q65:FAIL
|
||||||
gate-run wizard-work/forge/verify_reply_check_dry_refactor.py q65:PASS
|
gate-run wizard-work/forge/verify_reply_check_dry_refactor.py q65:PASS
|
||||||
gate-run wizard-work/forge/verify_repro_bounds_refactor.py q65:PASS
|
gate-run wizard-work/forge/verify_repro_bounds_refactor.py q65:PASS
|
||||||
@@ -42,6 +43,7 @@ gate-run wizard-work/forge/verify_smell_check_monotonic_refactor.py q65:PASS
|
|||||||
gate-run wizard-work/forge/verify_smell_scan_refactor.py q65:PASS
|
gate-run wizard-work/forge/verify_smell_scan_refactor.py q65:PASS
|
||||||
gate-run wizard-work/forge/verify_tool_map_dry_refactor.py q65:PASS
|
gate-run wizard-work/forge/verify_tool_map_dry_refactor.py q65:PASS
|
||||||
gate-run wizard-work/forge/verify_write_probe_dry_refactor.py q65:FAIL
|
gate-run wizard-work/forge/verify_write_probe_dry_refactor.py q65:FAIL
|
||||||
|
wired-invoke wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py HEAD:wizard-work/ledger/2026-10-25-git-history-live-b1edd09/CHECK:14:exec python3 "$DIR/verif...
|
||||||
wired-invoke wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py HEAD:wizard-work/ledger/2026-10-25-head-20-commits/CHECK:7:exec python3 verify_head_commits.p...
|
wired-invoke wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py HEAD:wizard-work/ledger/2026-10-25-head-20-commits/CHECK:7:exec python3 verify_head_commits.p...
|
||||||
gate-run wizard-work/ledger/verify_arxiv_csai.py q65:PASS
|
gate-run wizard-work/ledger/verify_arxiv_csai.py q65:PASS
|
||||||
gate-run wizard-work/ledger/verify_arxiv_csai_fresh.py q65:PASS
|
gate-run wizard-work/ledger/verify_arxiv_csai_fresh.py q65:PASS
|
||||||
@@ -51,6 +53,7 @@ gate-run wizard-work/ledger/verify_catalog.py q65:PASS
|
|||||||
wired-invoke wizard-work/ledger/verify_commits_check.py HEAD:wizard-work/ledger/CHECK:8:exec python3 "$OUT/verify_commits_check.py" "$@"
|
wired-invoke wizard-work/ledger/verify_commits_check.py HEAD:wizard-work/ledger/CHECK:8:exec python3 "$OUT/verify_commits_check.py" "$@"
|
||||||
gate-run wizard-work/ledger/verify_dates.py q65:PASS
|
gate-run wizard-work/ledger/verify_dates.py q65:PASS
|
||||||
gate-run wizard-work/ledger/verify_gdp.py q65:PASS
|
gate-run wizard-work/ledger/verify_gdp.py q65:PASS
|
||||||
|
gate-run wizard-work/ledger/verify_gdp_per_capita.py q65:PASS
|
||||||
gate-run wizard-work/ledger/verify_gitea_repos.py q65:FAIL
|
gate-run wizard-work/ledger/verify_gitea_repos.py q65:FAIL
|
||||||
gate-run wizard-work/ledger/verify_github_python_repos.py q65:FAIL
|
gate-run wizard-work/ledger/verify_github_python_repos.py q65:FAIL
|
||||||
declared-one-off wizard-work/ledger/verify_github_repos.py in-file declaration
|
declared-one-off wizard-work/ledger/verify_github_repos.py in-file declaration
|
||||||
@@ -116,5 +119,6 @@ gate-run wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py q
|
|||||||
gate-run wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py q65:PASS
|
gate-run wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py q65:PASS
|
||||||
gate-run wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py q65:FAIL
|
gate-run wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py q65:FAIL
|
||||||
wired-invoke wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py HEAD:wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/CHECK.py:173: p = subproc...
|
wired-invoke wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py HEAD:wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/CHECK.py:173: p = subproc...
|
||||||
|
gate-run wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py q65:PASS
|
||||||
wired-invoke wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py HEAD:wizard-work/sphinx/figure-locator-llama3-2026-10-24/run_checks.sh:13:python3 "$DIR/verif...
|
wired-invoke wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py HEAD:wizard-work/sphinx/figure-locator-llama3-2026-10-24/run_checks.sh:13:python3 "$DIR/verif...
|
||||||
gate-run wizard-work/sphinx/verify_mm_dossier.py q65:PASS
|
gate-run wizard-work/sphinx/verify_mm_dossier.py q65:PASS
|
||||||
|
Can't render this file because it contains an unexpected character in line 46 and column 173.
|
@@ -0,0 +1,7 @@
|
|||||||
|
rev1 da538ca5c0a4 blob b752e5467fb8
|
||||||
|
rev2 64733c9e977e blob 365068a8f0f5
|
||||||
|
ok A stale blob rc=1 expect=1 FAULT STALE wizard-work/x/verify_ctl.py: row graded blob b752e5467fb8, tree holds 365068a8f0f5
|
||||||
|
ok B rebound blob rc=0 expect=0 (no fault line)
|
||||||
|
ok C inert 1 of 3 sites rc=1 expect=1 FAULT CONTRADICTION wizard-work/x/verify_ctl.py: inert with valid=1 tried=1 sites=3 moved=None
|
||||||
|
ok D inert-partial 1/3 rc=0 expect=0 (no fault line)
|
||||||
|
4 of 4 controls behaved
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
dynamic dead-rc files (inert or no-comparison): 19 of 109 rows
|
||||||
|
|
||||||
|
class static runner sites/tried path
|
||||||
|
inert weak-subprocess PASS 2/2 wizard-work/ledger/verify_dates.py
|
||||||
|
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_all.py
|
||||||
|
inert no-verdict PASS 3/3 wizard-work/spark/verify_check.py
|
||||||
|
inert no-verdict PASS 1/1 wizard-work/spark/verify_count.py
|
||||||
|
nocomp no-verdict - 0/0 wizard-work/spark/verify_dirs.py
|
||||||
|
nocomp - PASS 0/0 wizard-work/spark/verify_env.py
|
||||||
|
inert no-verdict PASS 4/4 wizard-work/spark/verify_fib.py
|
||||||
|
inert - PASS 9/9 wizard-work/spark/verify_hermes_cli.py
|
||||||
|
inert - PASS 1/1 wizard-work/spark/verify_hermes_path.py
|
||||||
|
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_json.py
|
||||||
|
inert no-verdict PASS 1/1 wizard-work/spark/verify_lines.py
|
||||||
|
inert no-verdict PASS 3/3 wizard-work/spark/verify_minimal.py
|
||||||
|
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_push.py
|
||||||
|
inert no-verdict PASS 1/1 wizard-work/spark/verify_python_env.py
|
||||||
|
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_skills.py
|
||||||
|
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_skills2.py
|
||||||
|
inert no-verdict PASS 1/1 wizard-work/spark/verify_skills3.py
|
||||||
|
inert no-verdict PASS 1/1 wizard-work/spark/verify_skills_fixed.py
|
||||||
|
nocomp no-verdict PASS 0/0 wizard-work/spark/verify_sum.py
|
||||||
|
|
||||||
|
static no-verdict (cycle 5): 15
|
||||||
|
dynamic dead rc (cycle 6) : 19
|
||||||
|
in both : 15
|
||||||
|
static only (rc moves) : 0 []
|
||||||
|
dynamic only (has a path) : 4 ['wizard-work/ledger/verify_dates.py', 'wizard-work/spark/verify_env.py', 'wizard-work/spark/verify_hermes_cli.py', 'wizard-work/spark/verify_hermes_path.py']
|
||||||
|
|
||||||
|
dead-rc files carrying a runner PASS row: 18
|
||||||
|
PASS wizard-work/ledger/verify_dates.py
|
||||||
|
PASS wizard-work/spark/verify_all.py
|
||||||
|
PASS wizard-work/spark/verify_check.py
|
||||||
|
PASS wizard-work/spark/verify_count.py
|
||||||
|
PASS wizard-work/spark/verify_env.py
|
||||||
|
PASS wizard-work/spark/verify_fib.py
|
||||||
|
PASS wizard-work/spark/verify_hermes_cli.py
|
||||||
|
PASS wizard-work/spark/verify_hermes_path.py
|
||||||
|
PASS wizard-work/spark/verify_json.py
|
||||||
|
PASS wizard-work/spark/verify_lines.py
|
||||||
|
PASS wizard-work/spark/verify_minimal.py
|
||||||
|
PASS wizard-work/spark/verify_push.py
|
||||||
|
PASS wizard-work/spark/verify_python_env.py
|
||||||
|
PASS wizard-work/spark/verify_skills.py
|
||||||
|
PASS wizard-work/spark/verify_skills2.py
|
||||||
|
PASS wizard-work/spark/verify_skills3.py
|
||||||
|
PASS wizard-work/spark/verify_skills_fixed.py
|
||||||
|
PASS wizard-work/spark/verify_sum.py
|
||||||
|
of those, PASS* (host-pinned): 17
|
||||||
@@ -0,0 +1,325 @@
|
|||||||
|
q65_rc_probe -- rev 7cd4dd5d9226, 109 population file(s), work copy /root/.hermes/wizard-outputs/volt/rcprobe-work-c6
|
||||||
|
[1/109] wizard-work/beam/verify_toposort_none.py
|
||||||
|
L21 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=21 0.22s
|
||||||
|
[2/109] wizard-work/forge/verify_append_only_audit_dry_refactor.py
|
||||||
|
<unknown>:142: SyntaxWarning: "\u" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\u"? A raw string is also an option.
|
||||||
|
<unknown>:157: SyntaxWarning: "\u" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\u"? A raw string is also an option.
|
||||||
|
-> unprobed-baseline rc=1 sites=17 tried=0 valid=0 moved_at=- 2.16s
|
||||||
|
[3/109] wizard-work/forge/verify_append_only_owners_dry_refactor.py
|
||||||
|
L56 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=13 tried=1 valid=1 moved_at=56 0.65s
|
||||||
|
[4/109] wizard-work/forge/verify_census_row_delta_exit_contract.py
|
||||||
|
L46 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=23 tried=1 valid=1 moved_at=46 1.2000000000000002s
|
||||||
|
[5/109] wizard-work/forge/verify_check_lint_classes_dry.py
|
||||||
|
L80 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=15 tried=1 valid=1 moved_at=80 0.68s
|
||||||
|
[6/109] wizard-work/forge/verify_check_lint_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=14 tried=0 valid=0 moved_at=- 1.22s
|
||||||
|
[7/109] wizard-work/forge/verify_checkin_check_dry_refactor.py
|
||||||
|
L103 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=103 0.9299999999999999s
|
||||||
|
[8/109] wizard-work/forge/verify_chunker_refactor.py
|
||||||
|
L58 invalid mutant (1)
|
||||||
|
L105 valid mutant rc=0
|
||||||
|
L150 valid mutant rc=0
|
||||||
|
L61 valid mutant rc=0
|
||||||
|
L64 valid mutant rc=0
|
||||||
|
L68 valid mutant rc=0
|
||||||
|
L101 valid mutant rc=0
|
||||||
|
L123 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=11 tried=8 valid=7 moved_at=123 0.8499999999999999s
|
||||||
|
[9/109] wizard-work/forge/verify_citation_hash_class_dry_refactor.py
|
||||||
|
-> unprobed-timeout rc=TIMEOUT sites=12 tried=0 valid=0 moved_at=- 90.08s
|
||||||
|
[10/109] wizard-work/forge/verify_citation_hash_class_portable_default.py
|
||||||
|
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||||
|
[11/109] wizard-work/forge/verify_citation_resolve_check_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 15.52s
|
||||||
|
[12/109] wizard-work/forge/verify_doc_census_dry_refactor.py
|
||||||
|
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||||
|
[13/109] wizard-work/forge/verify_doc_command_report_dry.py
|
||||||
|
-> unprobed-baseline rc=1 sites=8 tried=0 valid=0 moved_at=- 7.96s
|
||||||
|
[14/109] wizard-work/forge/verify_gate_blame_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 10.81s
|
||||||
|
[15/109] wizard-work/forge/verify_landing_check_refactor.py
|
||||||
|
L103 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=25 tried=1 valid=1 moved_at=103 3.92s
|
||||||
|
[16/109] wizard-work/forge/verify_path_claim_apply_baseline_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=15 tried=0 valid=0 moved_at=- 0.31s
|
||||||
|
[17/109] wizard-work/forge/verify_path_claim_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=17 tried=0 valid=0 moved_at=- 0.55s
|
||||||
|
[18/109] wizard-work/forge/verify_pdf_refactor.py
|
||||||
|
L79 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=14 tried=1 valid=1 moved_at=79 7.27s
|
||||||
|
[19/109] wizard-work/forge/verify_queue_append_audit_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=13 tried=0 valid=0 moved_at=- 7.81s
|
||||||
|
[20/109] wizard-work/forge/verify_reply_check_dry_refactor.py
|
||||||
|
L72 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=11 tried=1 valid=1 moved_at=72 3.0s
|
||||||
|
[21/109] wizard-work/forge/verify_repro_bounds_refactor.py
|
||||||
|
L69 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=17 tried=1 valid=1 moved_at=69 0.94s
|
||||||
|
[22/109] wizard-work/forge/verify_roundtrip_refactor.py
|
||||||
|
L39 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=39 1.26s
|
||||||
|
[23/109] wizard-work/forge/verify_scan_dry_refactor.py
|
||||||
|
L78 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=21 tried=1 valid=1 moved_at=78 6.3s
|
||||||
|
[24/109] wizard-work/forge/verify_selftest_sweep_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=15 tried=0 valid=0 moved_at=- 0.25s
|
||||||
|
[25/109] wizard-work/forge/verify_selftest_sweep_refactor.py
|
||||||
|
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||||
|
[26/109] wizard-work/forge/verify_served_drift_summarise_dry.py
|
||||||
|
L100 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=16 tried=1 valid=1 moved_at=100 12.190000000000001s
|
||||||
|
[27/109] wizard-work/forge/verify_smell_check_monotonic_refactor.py
|
||||||
|
L64 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=16 tried=1 valid=1 moved_at=64 1.6800000000000002s
|
||||||
|
[28/109] wizard-work/forge/verify_smell_scan_refactor.py
|
||||||
|
L55 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=11 tried=1 valid=1 moved_at=55 1.0s
|
||||||
|
[29/109] wizard-work/forge/verify_tool_map_dry_refactor.py
|
||||||
|
L40 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=9 tried=1 valid=1 moved_at=40 49.33s
|
||||||
|
[30/109] wizard-work/forge/verify_write_probe_dry_refactor.py
|
||||||
|
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 23.35s
|
||||||
|
[31/109] wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py
|
||||||
|
L49 valid mutant rc=0
|
||||||
|
L90 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=2 valid=2 moved_at=90 3.45s
|
||||||
|
[32/109] wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py
|
||||||
|
L122 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=122 0.69s
|
||||||
|
[33/109] wizard-work/ledger/verify_arxiv_csai.py
|
||||||
|
L37 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=14 tried=1 valid=1 moved_at=37 0.49s
|
||||||
|
[34/109] wizard-work/ledger/verify_arxiv_csai_fresh.py
|
||||||
|
L42 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=18 tried=1 valid=1 moved_at=42 0.47s
|
||||||
|
[35/109] wizard-work/ledger/verify_arxiv_csv.py
|
||||||
|
L24 valid mutant rc=0
|
||||||
|
L29 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=2 valid=2 moved_at=29 0.33999999999999997s
|
||||||
|
[36/109] wizard-work/ledger/verify_arxiv_fresh.py
|
||||||
|
L48 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=48 0.19s
|
||||||
|
[37/109] wizard-work/ledger/verify_catalog.py
|
||||||
|
L30 invalid mutant (1)
|
||||||
|
L77 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=14 tried=2 valid=1 moved_at=77 0.39s
|
||||||
|
[38/109] wizard-work/ledger/verify_commits_check.py
|
||||||
|
L122 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=122 0.58s
|
||||||
|
[39/109] wizard-work/ledger/verify_dates.py
|
||||||
|
L26 valid mutant rc=0
|
||||||
|
L36 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=2 tried=2 valid=2 moved_at=- 0.48s
|
||||||
|
[40/109] wizard-work/ledger/verify_gdp.py
|
||||||
|
L3 valid mutant rc=0
|
||||||
|
L27 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=12 tried=2 valid=2 moved_at=27 0.29000000000000004s
|
||||||
|
[41/109] wizard-work/ledger/verify_gdp_per_capita.py
|
||||||
|
L42 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=42 1.52s
|
||||||
|
[42/109] wizard-work/ledger/verify_gitea_repos.py
|
||||||
|
-> unprobed-baseline rc=1 sites=8 tried=0 valid=0 moved_at=- 0.24s
|
||||||
|
[43/109] wizard-work/ledger/verify_github_python_repos.py
|
||||||
|
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 2.86s
|
||||||
|
[44/109] wizard-work/ledger/verify_github_repos.py
|
||||||
|
L24 valid mutant rc=0
|
||||||
|
L107 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=14 tried=2 valid=2 moved_at=107 5.529999999999999s
|
||||||
|
[45/109] wizard-work/ledger/verify_issues.py
|
||||||
|
L48 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=48 0.22s
|
||||||
|
[46/109] wizard-work/ledger/verify_python_org_repos.py
|
||||||
|
-> unprobed-baseline rc=1 sites=9 tried=0 valid=0 moved_at=- 2.02s
|
||||||
|
[47/109] wizard-work/ledger/verify_stackchain_catalog.py
|
||||||
|
L105 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=105 0.16999999999999998s
|
||||||
|
[48/109] wizard-work/ledger/verify_users.py
|
||||||
|
L32 valid mutant rc=0
|
||||||
|
L40 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=6 tried=2 valid=2 moved_at=40 0.32s
|
||||||
|
[49/109] wizard-work/ledger/verify_wizards.py
|
||||||
|
-> unprobed-baseline rc=1 sites=19 tried=0 valid=0 moved_at=- 15.69s
|
||||||
|
[50/109] wizard-work/ledger/verify_workflow.py
|
||||||
|
-> unprobed-baseline rc=1 sites=4 tried=0 valid=0 moved_at=- 0.22s
|
||||||
|
[51/109] wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py
|
||||||
|
L57 valid mutant rc=0
|
||||||
|
L85 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=9 tried=2 valid=2 moved_at=85 1.47s
|
||||||
|
[52/109] wizard-work/null/diffkit/verify_realrepo.py
|
||||||
|
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 0.38s
|
||||||
|
[53/109] wizard-work/sol/verify_pending_request_triage.py
|
||||||
|
L27 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=27 0.19s
|
||||||
|
[54/109] wizard-work/sol/verify_sol_profile_triage.py
|
||||||
|
L37 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=5 tried=1 valid=1 moved_at=37 1.65s
|
||||||
|
[55/109] wizard-work/sol/verify_sol_triage.py
|
||||||
|
-> unprobed-baseline rc=1 sites=11 tried=0 valid=0 moved_at=- 0.09s
|
||||||
|
[56/109] wizard-work/sol/verify_triage.py
|
||||||
|
L9 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=9 0.18s
|
||||||
|
[57/109] wizard-work/sol/verify_triage_queue.py
|
||||||
|
-> unprobed-baseline rc=1 sites=11 tried=0 valid=0 moved_at=- 0.13s
|
||||||
|
[58/109] wizard-work/sol/verify_troupe_sol_health.py
|
||||||
|
L43 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=43 0.18s
|
||||||
|
[59/109] wizard-work/spark/verify_all.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.09s
|
||||||
|
[60/109] wizard-work/spark/verify_analyzer.py
|
||||||
|
L19 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=19 0.44s
|
||||||
|
[61/109] wizard-work/spark/verify_check.py
|
||||||
|
L6 valid mutant rc=0
|
||||||
|
L10 valid mutant rc=0
|
||||||
|
L11 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=3 tried=3 valid=3 moved_at=- 0.31s
|
||||||
|
[62/109] wizard-work/spark/verify_count.py
|
||||||
|
L11 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.19s
|
||||||
|
[63/109] wizard-work/spark/verify_demo.py
|
||||||
|
L32 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=32 0.2s
|
||||||
|
[64/109] wizard-work/spark/verify_dir.py
|
||||||
|
L15 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=15 0.15000000000000002s
|
||||||
|
[65/109] wizard-work/spark/verify_dir_analyzer.py
|
||||||
|
L19 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=19 2.7800000000000002s
|
||||||
|
[66/109] wizard-work/spark/verify_dirs.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.1s
|
||||||
|
[67/109] wizard-work/spark/verify_env.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.08s
|
||||||
|
[68/109] wizard-work/spark/verify_environment.py
|
||||||
|
-> unprobed-baseline rc=1 sites=3 tried=0 valid=0 moved_at=- 0.07s
|
||||||
|
[69/109] wizard-work/spark/verify_fib.py
|
||||||
|
L28 valid mutant rc=0
|
||||||
|
L31 valid mutant rc=0
|
||||||
|
L35 valid mutant rc=0
|
||||||
|
L30 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=4 tried=4 valid=4 moved_at=- 0.4100000000000001s
|
||||||
|
[70/109] wizard-work/spark/verify_formatter.py
|
||||||
|
L14 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=4 tried=1 valid=1 moved_at=14 0.52s
|
||||||
|
[71/109] wizard-work/spark/verify_hermes.py
|
||||||
|
-> unprobed-baseline rc=1 sites=5 tried=0 valid=0 moved_at=- 0.13s
|
||||||
|
[72/109] wizard-work/spark/verify_hermes_cli.py
|
||||||
|
L31 valid mutant rc=0
|
||||||
|
L35 valid mutant rc=0
|
||||||
|
L64 valid mutant rc=0
|
||||||
|
L32 valid mutant rc=0
|
||||||
|
L33 valid mutant rc=0
|
||||||
|
L34 valid mutant rc=0
|
||||||
|
L36 valid mutant rc=0
|
||||||
|
L45 valid mutant rc=0
|
||||||
|
L50 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=9 tried=9 valid=9 moved_at=- 10.519999999999998s
|
||||||
|
[73/109] wizard-work/spark/verify_hermes_path.py
|
||||||
|
L24 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.16s
|
||||||
|
[74/109] wizard-work/spark/verify_inventory.py
|
||||||
|
-> unprobed-baseline rc=1 sites=2 tried=0 valid=0 moved_at=- 0.1s
|
||||||
|
[75/109] wizard-work/spark/verify_json.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.13s
|
||||||
|
[76/109] wizard-work/spark/verify_lines.py
|
||||||
|
L6 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.25s
|
||||||
|
[77/109] wizard-work/spark/verify_loc_report.py
|
||||||
|
L14 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=8 tried=1 valid=1 moved_at=14 0.25s
|
||||||
|
[78/109] wizard-work/spark/verify_lwc.py
|
||||||
|
L18 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=18 0.56s
|
||||||
|
[79/109] wizard-work/spark/verify_minimal.py
|
||||||
|
L24 valid mutant rc=0
|
||||||
|
L41 valid mutant rc=0
|
||||||
|
L71 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=3 tried=3 valid=3 moved_at=- 0.68s
|
||||||
|
[80/109] wizard-work/spark/verify_push.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.14s
|
||||||
|
[81/109] wizard-work/spark/verify_python_env.py
|
||||||
|
L31 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.19s
|
||||||
|
[82/109] wizard-work/spark/verify_report.py
|
||||||
|
L11 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=11 0.2s
|
||||||
|
[83/109] wizard-work/spark/verify_result.py
|
||||||
|
L12 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=2 tried=1 valid=1 moved_at=12 0.25s
|
||||||
|
[84/109] wizard-work/spark/verify_scratch_analysis.py
|
||||||
|
L17 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=6 tried=1 valid=1 moved_at=17 0.19s
|
||||||
|
[85/109] wizard-work/spark/verify_skill_system.py
|
||||||
|
-> unprobed-baseline rc=1 sites=6 tried=0 valid=0 moved_at=- 0.16s
|
||||||
|
[86/109] wizard-work/spark/verify_skills.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.11s
|
||||||
|
[87/109] wizard-work/spark/verify_skills2.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.09s
|
||||||
|
[88/109] wizard-work/spark/verify_skills3.py
|
||||||
|
L41 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.24000000000000002s
|
||||||
|
[89/109] wizard-work/spark/verify_skills_fixed.py
|
||||||
|
L31 valid mutant rc=0
|
||||||
|
-> inert rc=0 sites=1 tried=1 valid=1 moved_at=- 0.22s
|
||||||
|
[90/109] wizard-work/spark/verify_snippets.py
|
||||||
|
L25 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=25 0.19s
|
||||||
|
[91/109] wizard-work/spark/verify_status.py
|
||||||
|
L10 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=5 tried=1 valid=1 moved_at=10 0.63s
|
||||||
|
[92/109] wizard-work/spark/verify_sum.py
|
||||||
|
-> no-comparison rc=0 sites=0 tried=0 valid=0 moved_at=- 0.1s
|
||||||
|
[93/109] wizard-work/spark/verify_tool.py
|
||||||
|
L12 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=3 tried=1 valid=1 moved_at=12 1.08s
|
||||||
|
[94/109] wizard-work/spark/verify_wordcount.py
|
||||||
|
L15 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=7 tried=1 valid=1 moved_at=15 0.46s
|
||||||
|
[95/109] wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py
|
||||||
|
L49 invalid mutant (1)
|
||||||
|
L88 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=17 tried=2 valid=1 moved_at=88 8.68s
|
||||||
|
[96/109] wizard-work/sphinx/archive-gutenberg/verify_archive.py
|
||||||
|
L25 valid mutant rc=0
|
||||||
|
L63 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=12 tried=2 valid=2 moved_at=63 1.2s
|
||||||
|
[97/109] wizard-work/sphinx/archive-index-2026-10-23/verify_index.py
|
||||||
|
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||||
|
[98/109] wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py
|
||||||
|
L57 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=44 tried=1 valid=1 moved_at=57 0.37s
|
||||||
|
[99/109] wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py
|
||||||
|
-> unprobed-baseline rc=1 sites=29 tried=0 valid=0 moved_at=- 4.96s
|
||||||
|
[100/109] wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py
|
||||||
|
L106 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=35 tried=1 valid=1 moved_at=106 13.06s
|
||||||
|
[101/109] wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py
|
||||||
|
-> unprobed-baseline rc=1 sites=2 tried=0 valid=0 moved_at=- 0.14s
|
||||||
|
[102/109] wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py
|
||||||
|
L61 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=10 tried=1 valid=1 moved_at=61 0.48s
|
||||||
|
[103/109] wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py
|
||||||
|
L44 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=19 tried=1 valid=1 moved_at=44 0.38s
|
||||||
|
[104/109] wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py
|
||||||
|
L93 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=22 tried=1 valid=1 moved_at=93 7.75s
|
||||||
|
[105/109] wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py
|
||||||
|
-> unprobed-window window 1800s exceeds --skip-over 300s
|
||||||
|
[106/109] wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py
|
||||||
|
L40 invalid mutant (1)
|
||||||
|
L250 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=69 tried=2 valid=1 moved_at=250 9.260000000000002s
|
||||||
|
[107/109] wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py
|
||||||
|
-> unprobed-baseline rc=1 sites=10 tried=0 valid=0 moved_at=- 40.7s
|
||||||
|
[108/109] wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py
|
||||||
|
L101 valid mutant rc=1
|
||||||
|
-> sensitive rc=0 sites=21 tried=1 valid=1 moved_at=101 0.77s
|
||||||
|
[109/109] wizard-work/sphinx/verify_mm_dossier.py
|
||||||
|
-> unprobed-baseline rc=1 sites=12 tried=0 valid=0 moved_at=- 3.49s
|
||||||
|
wrote wizard-work/volt/q65_rc_probe.tsv (109 rows)
|
||||||
|
work copy left at /root/.hermes/wizard-outputs/volt/rcprobe-work-c6
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# The boundary of 'inert': comparison-inert is not 'cannot fail'.
|
||||||
|
# Substrate: spark/verify_hermes_path.py copied unchanged to a scratch dir, then
|
||||||
|
# ONE required path renamed to a directory that does not exist.
|
||||||
|
|
||||||
|
== control: unmodified copy at HEAD (blob 385362b75cfe)
|
||||||
|
OK: /root/.hermes/wizard-outputs/spark
|
||||||
|
OK: /root/troupe-standup/wizard-work/spark
|
||||||
|
OK: /root/.hermes/profiles/spark
|
||||||
|
rc=0
|
||||||
|
|
||||||
|
== falsified subject: /root/.hermes/profiles/spark -> /root/.hermes/profiles/spark-NO-SUCH-DIR-c6
|
||||||
|
OK: /root/.hermes/wizard-outputs/spark
|
||||||
|
OK: /root/troupe-standup/wizard-work/spark
|
||||||
|
MISSING: /root/.hermes/profiles/spark-NO-SUCH-DIR-c6
|
||||||
|
rc=1
|
||||||
|
|
||||||
|
== the other 16 comparison-inert rows cannot fail at all:
|
||||||
|
spark/verify_check.py (13 loc) computes factorial(5) and prints it -- no exit path, no assertion;
|
||||||
|
it carries a runner PASS row with an empty host column, i.e. a graded-clean PASS on a file that cannot fail.
|
||||||
|
15 static no-verdict printers (q65_no_verdict.py --check) + ledger/verify_dates.py (cycle 4: prints impossible dates, rc 0).
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
== wizard-work/forge/verify_chunker_refactor.py (6328 bytes, sha256 fc82c86f32ab)
|
||||||
|
sites=11 sample(spread,3)=[58, 105, 150] baseline rc=0
|
||||||
|
L58 rc=1 invalid <- in sample
|
||||||
|
L61 rc=0 -
|
||||||
|
L64 rc=0 -
|
||||||
|
L68 rc=0 -
|
||||||
|
L101 rc=0 -
|
||||||
|
L105 rc=0 - <- in sample
|
||||||
|
L123 rc=1 MOVED
|
||||||
|
L124 rc=0 -
|
||||||
|
L139 rc=1 MOVED
|
||||||
|
L142 rc=2 MOVED
|
||||||
|
L150 rc=0 - <- in sample
|
||||||
|
moved at 3 of 11 sites: [123, 139, 142]
|
||||||
|
sampled sites that moved: []
|
||||||
|
|
||||||
|
== wizard-work/ledger/verify_dates.py (1172 bytes, sha256 717e81b33a0f)
|
||||||
|
sites=2 sample(spread,3)=[26, 36] baseline rc=0
|
||||||
|
L26 rc=0 - <- in sample
|
||||||
|
L36 rc=0 - <- in sample
|
||||||
|
moved at 0 of 2 sites: []
|
||||||
|
sampled sites that moved: []
|
||||||
|
|
||||||
|
done
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
# provenance: bash wizard-work/volt/CHECK at 6e8f577 (2026-10-25T16:50:03+00:00), 2026-10-25T16:57:38Z
|
||||||
|
ok c1 PASS rc 0 is consistent
|
||||||
|
ok c2 PASS rc 1 is a fault
|
||||||
|
ok c3 TIMEOUT carrying an rc is a fault
|
||||||
|
ok c4 OTHER_RC rc 3 is consistent
|
||||||
|
ok c5 OTHER_RC rc 7 is consistent
|
||||||
|
ok c6 OTHER_RC rc 2 is a fault (that is REFUSED)
|
||||||
|
ok c7 ledger round-trips two rows
|
||||||
|
ok c8 a missing ledger is an error, not an empty pass
|
||||||
|
ok c9 an unknown verdict refuses the ledger
|
||||||
|
ok c10 a committed verifier in neither ledger is UNCLASSED
|
||||||
|
ok c11 a class row covers the same file
|
||||||
|
ok c12 a window row parses to (seconds, why)
|
||||||
|
ok c13 an unlisted path falls back to --timeout
|
||||||
|
ok c14 a malformed window row refuses instead of silently skipping
|
||||||
|
ok c15 a committed path has a blob id
|
||||||
|
ok c16 an absent path has no blob id
|
||||||
|
ok c17 the same path at a new commit is a different blob (DRIFT is detectable)
|
||||||
|
ok c18 a host root in a string VALUE is a host pin
|
||||||
|
ok c19 a host root in a docstring is prose, not a pin
|
||||||
|
ok c20 a PARSED PASS row on a host-independent verifier is graded clean
|
||||||
|
ok c21 a PARSED OTHER_RC row with rc 7 is consistent (not a fault)
|
||||||
|
ok c22 a class-narrowed tally sums to its own population, not the ledger
|
||||||
|
selftest: 22 cells, 22 passed, 0 failed
|
||||||
|
q65_runner -- rev HEAD, 90 population file(s), 90 ledger row(s) [ledger rev 7cd4dd5]
|
||||||
|
rows at emit : 90 (PASS 66, FAIL 24)
|
||||||
|
graded (has rc): 90 of 90
|
||||||
|
graded-clean : 10 of 90 (PASS on a host-independent verifier)
|
||||||
|
host-pinned : 56 PASS row(s) excluded as PASS* -- they read an absolute host root
|
||||||
|
PASS* wizard-work/forge/verify_append_only_owners_dry_refactor.py L41:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_census_row_delta_exit_contract.py L31:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_check_lint_classes_dry.py L44:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_checkin_check_dry_refactor.py L39:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_landing_check_refactor.py L62:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_pdf_refactor.py L41:/root/.hermes
|
||||||
|
PASS* wizard-work/forge/verify_reply_check_dry_refactor.py L33:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_repro_bounds_refactor.py L33:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_roundtrip_refactor.py L21:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_served_drift_summarise_dry.py L48:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_smell_check_monotonic_refactor.py L32:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_smell_scan_refactor.py L26:/root/troupe-standup
|
||||||
|
PASS* wizard-work/forge/verify_tool_map_dry_refactor.py L32:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_arxiv_csai.py L15:/root/.hermes
|
||||||
|
PASS* wizard-work/ledger/verify_arxiv_csai_fresh.py L16:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_arxiv_csv.py L24:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_catalog.py L13:/root/.hermes
|
||||||
|
PASS* wizard-work/ledger/verify_dates.py L6:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_gdp.py L3:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_gdp_per_capita.py L19:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_issues.py L15:/root/.hermes
|
||||||
|
PASS* wizard-work/ledger/verify_stackchain_catalog.py L16:/root/troupe-standup
|
||||||
|
PASS* wizard-work/ledger/verify_users.py L16:/root/troupe-standup
|
||||||
|
PASS* wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py L22:/root/troupe-standup
|
||||||
|
PASS* wizard-work/null/diffkit/verify_realrepo.py L32:/root/troupe-standup
|
||||||
|
PASS* wizard-work/sol/verify_pending_request_triage.py L11:/root/troupe-standup
|
||||||
|
PASS* wizard-work/sol/verify_triage.py L4:/root/troupe-standup
|
||||||
|
PASS* wizard-work/spark/verify_all.py L4:/root/troupe-standup
|
||||||
|
PASS* wizard-work/spark/verify_count.py L5:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_demo.py L20:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_dir.py L23:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_dir_analyzer.py L16:/root/troupe-standup
|
||||||
|
PASS* wizard-work/spark/verify_env.py L9:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_fib.py L24:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_formatter.py L9:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_hermes_cli.py L55:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_hermes_path.py L11:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_json.py L4:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_lines.py L2:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_lwc.py L12:/root/troupe-standup
|
||||||
|
PASS* wizard-work/spark/verify_minimal.py L31:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_push.py L3:/root/troupe-standup
|
||||||
|
PASS* wizard-work/spark/verify_python_env.py L22:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_report.py L10:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_result.py L10:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_scratch_analysis.py L13:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_skills.py L5:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_skills2.py L6:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_skills3.py L6:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_skills_fixed.py L6:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_snippets.py L11:/root/troupe-standup
|
||||||
|
PASS* wizard-work/spark/verify_status.py L7:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_sum.py L6:/root/.hermes
|
||||||
|
PASS* wizard-work/spark/verify_wordcount.py L46:/root/.hermes
|
||||||
|
PASS* wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py L59:/root/.hermes
|
||||||
|
PASS* wizard-work/sphinx/verify_mm_dossier.py L28:/root/.hermes
|
||||||
|
tree audit : 1 unclassed of 110 committed verifier(s) <-- named by neither ledger
|
||||||
|
UNCLASSED wizard-work/forge/verify_probe_forge_refactor.py: committed verify_*.py at HEAD, in no class row in q63_unwired_baseline.tsv and no ledger row
|
||||||
|
q65: 1 fault(s)
|
||||||
|
rc=1
|
||||||
@@ -0,0 +1,804 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""q65_rc_probe.py -- ask a committed verifier, by execution, whether its exit
|
||||||
|
code can move at all (Q65, volt lane, 2026-10-25, cycle 6).
|
||||||
|
|
||||||
|
WHY THIS EXISTS
|
||||||
|
Cycle 5 answered "does a verifier have a failure path" statically: an AST scan
|
||||||
|
for `raise` / `assert` / non-zero `exit` / `unittest.main` found 15 of 106
|
||||||
|
committed verifiers with none, 14 of them carrying a PASS row in the runner
|
||||||
|
ledger. Static presence is a proxy, and it has two blind spots the tool said
|
||||||
|
so about itself: an exit path inside a `try/except` that swallows it still
|
||||||
|
passes the scan, and a present exit path can be unreachable from any verdict.
|
||||||
|
|
||||||
|
This is the dynamic question instead, and it does not read the source for
|
||||||
|
intent. It perturbs the verifier's OWN decisions and watches the process:
|
||||||
|
flip one comparison operator at a time, re-run the file, and see whether the
|
||||||
|
exit code changes. If a verifier's rc is unchanged after its comparisons have
|
||||||
|
been inverted, its rc is not connected to what it measured -- no input could
|
||||||
|
have made that file say no. That is a stronger statement than "no raise in the
|
||||||
|
AST", because it is made about a process that actually ran.
|
||||||
|
|
||||||
|
The mutation is of the verifier, not of the subject. Conventionally, mutation
|
||||||
|
testing mutates the SUBJECT and asks whether the test still passes (a killed
|
||||||
|
mutant means the test discriminates). Here the subject stays exactly as it is
|
||||||
|
and the ASSERTION is mutated, which inverts the question into the one Q65
|
||||||
|
needs: does an inverted assertion reach the exit code? A verifier whose rc is
|
||||||
|
stable under its own inverted assertions has no verdict channel, whatever its
|
||||||
|
source looks like.
|
||||||
|
|
||||||
|
WHAT IT MEASURES -- one row per probed file, one class per row
|
||||||
|
sensitive baseline rc 0, and >=1 VALID mutant returned a different rc.
|
||||||
|
The rc is wired to a comparison; the file can fail.
|
||||||
|
inert baseline rc 0, >=1 valid mutant, every valid mutant returned
|
||||||
|
the baseline rc, AND every mutable comparison site in the file
|
||||||
|
was tried. The rc does not respond to any of the file's own
|
||||||
|
comparisons. This is the finding.
|
||||||
|
It is a statement about COMPARISONS, not about the world: a
|
||||||
|
verifier whose rc is `all(exists and is_dir ...)` over
|
||||||
|
`os.path.isdir` is comparison-inert and can still fail on a
|
||||||
|
falsified subject (measured: spark/verify_hermes_path.py with
|
||||||
|
one required path renamed -> rc 1). 3 of the 19 comparison-inert
|
||||||
|
rows at 7cd4dd5 are of that kind; the other 16 cannot fail at
|
||||||
|
all, 15 of them static no-verdict printers.
|
||||||
|
inert-partial the same, but the file has more sites than `--exhaustive-cap`
|
||||||
|
allowed, so the negative is bounded: nothing moved in the
|
||||||
|
first N sites, and sites N+1.. were never tried. Explicitly
|
||||||
|
not the same claim as `inert` -- see WHY THE CLASSES ARE SPLIT.
|
||||||
|
no-comparison baseline rc 0 and no single-operator comparison to mutate.
|
||||||
|
Nothing in the file is a decision the rc could carry.
|
||||||
|
unprobed-baseline the unmutated file already returned a non-zero rc: there is
|
||||||
|
no pass state to defend, so the question does not apply.
|
||||||
|
unprobed-timeout the unmutated file did not finish inside its window.
|
||||||
|
unprobed-mutant every mutant tried was invalid (see below), so nothing was
|
||||||
|
measured. Not a finding either way.
|
||||||
|
unprobed-window the verifier's own measurement window (`q65_row_windows.tsv`,
|
||||||
|
shared with the runner) exceeds `--skip-over`, so it was not
|
||||||
|
mutated. A 30-minute live-HTTP verifier is not inverted three
|
||||||
|
times inside a sweep; its rc is the runner's row.
|
||||||
|
unprobed-error the sweep could not touch the file (the work copy lost it
|
||||||
|
mid-run). Recorded as a row with its message, so a pruned
|
||||||
|
scratch directory leaves a readable ledger instead of a
|
||||||
|
traceback halfway through the population.
|
||||||
|
|
||||||
|
A MUTANT IS VALID when its run (a) compiled and (b) finished inside the window
|
||||||
|
and (c) did NOT print a traceback. Flipping `==` to `!=` can turn a type check
|
||||||
|
into a TypeError; an rc that moves because the mutant crashed says nothing
|
||||||
|
about whether a verdict reaches the exit code. Invalid mutants are counted and
|
||||||
|
discarded, and a file left with none is `unprobed-mutant`, not `inert`.
|
||||||
|
|
||||||
|
HOW MUCH IS TRIED -- deterministic spread, so the figure is reproducible
|
||||||
|
`--max-mutants` (default 3) sites are taken from the sorted list of mutable
|
||||||
|
comparison line numbers by even spacing, so a 76-site file is probed near its
|
||||||
|
beginning, middle and end rather than at its first three lines. No randomness:
|
||||||
|
the same rev and the same `--max-mutants` select the same sites.
|
||||||
|
|
||||||
|
If those first sites all return the baseline rc, the sample is NOT the
|
||||||
|
verdict: every remaining site is tried too, up to `--exhaustive-cap`
|
||||||
|
(default 40) per file; 0 means no cap. Only a file whose whole site list was
|
||||||
|
tried can be called `inert`; a file stopped by the cap is `inert-partial`.
|
||||||
|
|
||||||
|
WHY THE CLASSES ARE SPLIT -- a measured false positive, not a hypothetical
|
||||||
|
Cycle 6's first sweep wrote `inert` for `forge/verify_chunker_refactor.py`
|
||||||
|
from a 3-of-11 sample: sites 58, 105, 150 (one of which crashed). Swept
|
||||||
|
exhaustively in a scratch copy, that file's rc moves at 3 of its 11 sites
|
||||||
|
(L123 -> 1, L139 -> 1, L142 -> 2) and the sample happened to hit none of
|
||||||
|
them. The file can fail; the sample said it could not. `inert` is therefore
|
||||||
|
a statement about every comparison in the file or it is not made.
|
||||||
|
|
||||||
|
WHERE IT WRITES
|
||||||
|
`--emit` mutates a COPY, never this worktree. The copy is a detached `git
|
||||||
|
worktree` at `--rev` -- a real checkout, not an export, because a verifier
|
||||||
|
that reads git plumbing returns a non-zero rc in a bare export and that
|
||||||
|
non-zero would be the copy's defect graded as the verifier's baseline. The
|
||||||
|
tool refuses (rc 2) a `--work` equal to the repository root -- a gate that
|
||||||
|
edits the tree it grades is the defect reply 93 §1 already measured once.
|
||||||
|
Every mutant is written, run, and restored inside a `try/finally`; the file
|
||||||
|
hash is re-checked after each file and a mismatch is rc 1, not a silent pass.
|
||||||
|
|
||||||
|
`--emit` writes the TSV (`--tsv`) and appends progress as it goes, so a sweep
|
||||||
|
that is interrupted leaves a readable partial ledger rather than nothing.
|
||||||
|
|
||||||
|
`--check` is the fast half and what the lane's CHECK runs: it re-reads the TSV,
|
||||||
|
binds every row to the blob at `--rev` (a row graded a file that has since
|
||||||
|
changed is STALE), re-derives the class counts from the rows, and faults on a
|
||||||
|
row that contradicts its own class (a `sensitive` row with no moving line, an
|
||||||
|
`inert` row with no valid mutant, a `no-comparison` row that lists sites).
|
||||||
|
It does NOT require population coverage: that is the census's job
|
||||||
|
(`q63_verify_scan.py`) and the runner's, and duplicating it here would make
|
||||||
|
two gates red for one added file. It prints the coverage gap instead.
|
||||||
|
|
||||||
|
rc 0 the ledger is internally consistent and content-bound
|
||||||
|
rc 1 a fault it graded: STALE, MISSING, an unknown class, or a row that
|
||||||
|
contradicts its class
|
||||||
|
rc 2 it could not grade at all (no TSV, not a git repo, --work is the tree)
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import ast
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
DEFAULT_TSV = os.path.join(HERE, "q65_rc_probe.tsv")
|
||||||
|
DEFAULT_REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||||
|
SCRATCH_ROOT = "/root/.hermes/profiles/volt/cache/scratch"
|
||||||
|
|
||||||
|
CLASSES = ("sensitive", "inert", "inert-partial", "no-comparison",
|
||||||
|
"unprobed-baseline", "unprobed-timeout", "unprobed-mutant",
|
||||||
|
"unprobed-window", "unprobed-error")
|
||||||
|
|
||||||
|
FLIP = {
|
||||||
|
ast.Eq: ast.NotEq, ast.NotEq: ast.Eq,
|
||||||
|
ast.Lt: ast.Gt, ast.Gt: ast.Lt,
|
||||||
|
ast.LtE: ast.GtE, ast.GtE: ast.LtE,
|
||||||
|
ast.In: ast.NotIn, ast.NotIn: ast.In,
|
||||||
|
ast.Is: ast.IsNot, ast.IsNot: ast.Is,
|
||||||
|
}
|
||||||
|
|
||||||
|
TRACEBACK = "Traceback (most recent call last)"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- source side
|
||||||
|
def mutable_lines(src):
|
||||||
|
"""Line numbers of single-operator comparisons that FLIP can invert."""
|
||||||
|
try:
|
||||||
|
tree = ast.parse(src)
|
||||||
|
except SyntaxError:
|
||||||
|
return []
|
||||||
|
lines = set()
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if (isinstance(node, ast.Compare) and len(node.ops) == 1
|
||||||
|
and type(node.ops[0]) in FLIP):
|
||||||
|
lines.add(node.lineno)
|
||||||
|
return sorted(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def spread(picks, n):
|
||||||
|
"""Even-spaced deterministic subset of a sorted list, n or fewer."""
|
||||||
|
if n <= 0 or not picks:
|
||||||
|
return []
|
||||||
|
if len(picks) <= n:
|
||||||
|
return list(picks)
|
||||||
|
if n == 1:
|
||||||
|
return [picks[0]]
|
||||||
|
step = (len(picks) - 1) / float(n - 1)
|
||||||
|
out = []
|
||||||
|
for i in range(n):
|
||||||
|
idx = int(round(i * step))
|
||||||
|
if picks[idx] not in out:
|
||||||
|
out.append(picks[idx])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def mutate(src, line):
|
||||||
|
"""Return source with the comparison at `line` inverted, or None."""
|
||||||
|
tree = ast.parse(src)
|
||||||
|
hits = [n for n in ast.walk(tree)
|
||||||
|
if isinstance(n, ast.Compare) and n.lineno == line
|
||||||
|
and len(n.ops) == 1 and type(n.ops[0]) in FLIP]
|
||||||
|
if not hits:
|
||||||
|
return None
|
||||||
|
hits[0].ops[0] = FLIP[type(hits[0].ops[0])]()
|
||||||
|
return ast.unparse(ast.fix_missing_locations(tree)) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- process side
|
||||||
|
def run_file(path, timeout):
|
||||||
|
"""Run one verifier the way the ledger does: cwd is the file's directory."""
|
||||||
|
t0 = time.time()
|
||||||
|
try:
|
||||||
|
p = subprocess.run([sys.executable, os.path.basename(path)],
|
||||||
|
cwd=os.path.dirname(path) or ".",
|
||||||
|
capture_output=True, text=True, timeout=timeout)
|
||||||
|
return {"rc": p.returncode, "traceback": TRACEBACK in p.stderr,
|
||||||
|
"seconds": round(time.time() - t0, 2)}
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
return {"rc": "TIMEOUT", "traceback": False,
|
||||||
|
"seconds": round(time.time() - t0, 2)}
|
||||||
|
|
||||||
|
|
||||||
|
def read_windows(path):
|
||||||
|
"""path -> seconds, from the runner's per-row window table (may be absent)."""
|
||||||
|
table = {}
|
||||||
|
if not path or not os.path.isfile(path):
|
||||||
|
return table
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
for line in fh:
|
||||||
|
line = line.rstrip("\n")
|
||||||
|
if not line or line.startswith("#"):
|
||||||
|
continue
|
||||||
|
cells = line.split("\t")
|
||||||
|
if len(cells) >= 2:
|
||||||
|
try:
|
||||||
|
table[cells[0]] = float(cells[1])
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
return table
|
||||||
|
|
||||||
|
|
||||||
|
def probe(path, max_mutants, timeout, log=None, exhaustive_cap=40):
|
||||||
|
"""Baseline, up to max_mutants sample sites, then every remaining site.
|
||||||
|
|
||||||
|
The sample is the cheap question; the exhaustive pass is what licenses the
|
||||||
|
word `inert`. A file stopped by `exhaustive_cap` is inert-partial.
|
||||||
|
"""
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
src = fh.read()
|
||||||
|
before = hashlib.sha256(src.encode()).hexdigest()
|
||||||
|
sites = mutable_lines(src)
|
||||||
|
base = run_file(path, timeout)
|
||||||
|
row = {"path": path, "baseline_rc": base["rc"], "sites": len(sites),
|
||||||
|
"tried": 0, "valid": 0, "moved_at": "-", "seconds": base["seconds"],
|
||||||
|
"cls": None, "note": ""}
|
||||||
|
|
||||||
|
if base["rc"] == "TIMEOUT":
|
||||||
|
row["cls"] = "unprobed-timeout"
|
||||||
|
return row
|
||||||
|
if base["rc"] != 0:
|
||||||
|
row["cls"] = "unprobed-baseline"
|
||||||
|
row["note"] = "baseline rc %s" % base["rc"]
|
||||||
|
return row
|
||||||
|
if not sites:
|
||||||
|
row["cls"] = "no-comparison"
|
||||||
|
return row
|
||||||
|
|
||||||
|
tries = spread(sites, max_mutants)
|
||||||
|
if len(tries) < len(sites):
|
||||||
|
# The negative half of the class needs every site, not the sample.
|
||||||
|
rest = [s for s in sites if s not in tries]
|
||||||
|
cap = len(rest) if exhaustive_cap == 0 else max(0, exhaustive_cap - len(tries))
|
||||||
|
tries += spread(rest, cap)
|
||||||
|
for line in tries:
|
||||||
|
mutant = mutate(src, line)
|
||||||
|
if mutant is None:
|
||||||
|
continue
|
||||||
|
row["tried"] += 1
|
||||||
|
with open(path, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(mutant)
|
||||||
|
try:
|
||||||
|
got = run_file(path, timeout)
|
||||||
|
finally:
|
||||||
|
with open(path, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(src)
|
||||||
|
row["seconds"] += got["seconds"]
|
||||||
|
valid = got["rc"] != "TIMEOUT" and not got["traceback"]
|
||||||
|
if not valid:
|
||||||
|
if log:
|
||||||
|
log(" L%d invalid mutant (%s)" % (line, got["rc"]))
|
||||||
|
continue
|
||||||
|
row["valid"] += 1
|
||||||
|
if log:
|
||||||
|
log(" L%d valid mutant rc=%s" % (line, got["rc"]))
|
||||||
|
if got["rc"] != base["rc"]:
|
||||||
|
row["cls"] = "sensitive"
|
||||||
|
row["moved_at"] = line
|
||||||
|
break
|
||||||
|
|
||||||
|
if row["cls"] != "sensitive":
|
||||||
|
if not row["valid"]:
|
||||||
|
row["cls"] = "unprobed-mutant"
|
||||||
|
elif row["tried"] >= len(sites):
|
||||||
|
row["cls"] = "inert"
|
||||||
|
else:
|
||||||
|
row["cls"] = "inert-partial"
|
||||||
|
row["note"] = "%d of %d sites tried (cap %d)" % (
|
||||||
|
row["tried"], len(sites), exhaustive_cap)
|
||||||
|
after = hashlib.sha256(open(path, encoding="utf-8").read().encode()).hexdigest()
|
||||||
|
if after != before:
|
||||||
|
raise RuntimeError("restore failed for %s" % path)
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- git plumbing
|
||||||
|
def tree_blobs(repo, rev):
|
||||||
|
out = subprocess.run(["git", "-C", repo, "ls-tree", "-r", rev],
|
||||||
|
capture_output=True, text=True)
|
||||||
|
if out.returncode != 0:
|
||||||
|
raise SystemExit("q65_rc_probe: cannot read tree at %s: %s"
|
||||||
|
% (rev, out.stderr.strip()))
|
||||||
|
blobs = {}
|
||||||
|
for line in out.stdout.splitlines():
|
||||||
|
meta, path = line.split("\t", 1)
|
||||||
|
parts = meta.split()
|
||||||
|
if len(parts) >= 3 and parts[1] == "blob":
|
||||||
|
blobs[path] = parts[2]
|
||||||
|
return blobs
|
||||||
|
|
||||||
|
|
||||||
|
def population(blobs):
|
||||||
|
return sorted(p for p in blobs
|
||||||
|
if os.path.basename(p).startswith("verify_")
|
||||||
|
and p.endswith(".py")
|
||||||
|
and p.startswith("wizard-work/"))
|
||||||
|
|
||||||
|
|
||||||
|
def strip_basename(path):
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- modes
|
||||||
|
def read_tsv(path):
|
||||||
|
rows, head = [], {}
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
for line in fh:
|
||||||
|
line = line.rstrip("\n")
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
if line.startswith("#"):
|
||||||
|
for key in ("rev", "rows", "committed"):
|
||||||
|
if line.startswith("# %s " % key):
|
||||||
|
head[key] = line.split(None, 2)[2].split()[0]
|
||||||
|
continue
|
||||||
|
cells = line.split("\t")
|
||||||
|
if len(cells) != 9:
|
||||||
|
raise ValueError("row has %d cells, want 9: %r" % (len(cells), line))
|
||||||
|
rows.append(dict(zip(
|
||||||
|
("cls", "path", "baseline_rc", "sites", "tried", "valid",
|
||||||
|
"moved_at", "blob", "seconds"), cells)))
|
||||||
|
return head, rows
|
||||||
|
|
||||||
|
|
||||||
|
def write_header(fh, rev, rows, committed):
|
||||||
|
fh.write("# Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.\n")
|
||||||
|
fh.write("# Written by q65_rc_probe.py --emit. A row states a measurement, not an\n")
|
||||||
|
fh.write("# exemption: --check faults on a row that contradicts its class or whose\n")
|
||||||
|
fh.write("# file has changed since (STALE).\n")
|
||||||
|
fh.write("# columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds\n")
|
||||||
|
fh.write("# rev %s\n" % rev)
|
||||||
|
fh.write("# rows %d\n" % rows)
|
||||||
|
fh.write("# committed %d\n" % committed)
|
||||||
|
|
||||||
|
|
||||||
|
def do_emit(args):
|
||||||
|
repo = os.path.abspath(args.repo)
|
||||||
|
rev = subprocess.run(["git", "-C", repo, "rev-parse", args.rev],
|
||||||
|
capture_output=True, text=True)
|
||||||
|
if rev.returncode != 0:
|
||||||
|
print("q65_rc_probe: not a git tree at %s" % repo, file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
rev = rev.stdout.strip()
|
||||||
|
blobs = tree_blobs(repo, rev)
|
||||||
|
pop = population(blobs)
|
||||||
|
windows = read_windows(args.windows)
|
||||||
|
prev_rows = []
|
||||||
|
if args.merge and os.path.isfile(args.tsv):
|
||||||
|
try:
|
||||||
|
_, prev_rows = read_tsv(args.tsv)
|
||||||
|
except ValueError:
|
||||||
|
prev_rows = []
|
||||||
|
print("q65_rc_probe: merging into %d existing row(s) from %s"
|
||||||
|
% (len(prev_rows), args.tsv))
|
||||||
|
if args.only:
|
||||||
|
pop = [p for p in pop if args.only in p]
|
||||||
|
if args.limit:
|
||||||
|
pop = pop[:args.limit]
|
||||||
|
|
||||||
|
if args.work:
|
||||||
|
work = os.path.abspath(args.work)
|
||||||
|
if work == os.path.realpath(repo) or work == repo:
|
||||||
|
print("q65_rc_probe: --work is the repository itself; refusing to "
|
||||||
|
"mutate the tree this gate grades", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
else:
|
||||||
|
work = os.path.join(SCRATCH_ROOT, "rcprobe-%d" % int(time.time()))
|
||||||
|
if not os.path.isdir(work):
|
||||||
|
os.makedirs(work)
|
||||||
|
if not os.path.isdir(os.path.join(work, ".git")) and not os.path.exists(
|
||||||
|
os.path.join(work, ".git")):
|
||||||
|
# A real git worktree, not an export: a verifier that reads git plumbing
|
||||||
|
# (rev-parse, ls-tree, log) returns a non-zero rc in a bare export, and a
|
||||||
|
# non-zero BASELINE is a row with no pass state to defend -- the sweep
|
||||||
|
# would grade its own copy's defect instead of the verifier.
|
||||||
|
add = subprocess.run(["git", "-C", repo, "worktree", "add", "--detach",
|
||||||
|
work, rev], capture_output=True, text=True)
|
||||||
|
if add.returncode != 0:
|
||||||
|
print("q65_rc_probe: cannot add a worktree at %s: %s"
|
||||||
|
% (rev[:12], add.stderr.strip()), file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
print("q65_rc_probe -- rev %s, %d population file(s), work copy %s"
|
||||||
|
% (rev[:12], len(pop), work))
|
||||||
|
rows = []
|
||||||
|
for i, rel in enumerate(pop, 1):
|
||||||
|
path = os.path.join(work, rel)
|
||||||
|
if not os.path.isfile(path):
|
||||||
|
rows.append({"cls": "unprobed-mutant", "path": rel, "baseline_rc": "-",
|
||||||
|
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
|
||||||
|
"blob": blobs.get(rel, "-")[:12], "seconds": 0.0,
|
||||||
|
"note": "absent from the work copy"})
|
||||||
|
continue
|
||||||
|
print("[%d/%d] %s" % (i, len(pop), rel))
|
||||||
|
window = windows.get(rel, args.timeout)
|
||||||
|
if window > args.skip_over:
|
||||||
|
row = {"cls": "unprobed-window", "path": rel, "baseline_rc": "-",
|
||||||
|
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
|
||||||
|
"seconds": 0.0, "note": "%ss window" % int(window)}
|
||||||
|
row["blob"] = blobs.get(rel, "-")[:12]
|
||||||
|
rows.append(row)
|
||||||
|
print(" -> %-17s window %ss exceeds --skip-over %ss"
|
||||||
|
% (row["cls"], int(window), int(args.skip_over)))
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
row = probe(path, args.max_mutants, window, log=lambda m: print(m),
|
||||||
|
exhaustive_cap=getattr(args, "exhaustive_cap", 40))
|
||||||
|
except (OSError, RuntimeError) as exc:
|
||||||
|
# A work copy that lost a file mid-sweep (a scratch pruner, a full
|
||||||
|
# disk) must be a row, not a traceback: the sweep's ledger stays
|
||||||
|
# readable and the row says what happened.
|
||||||
|
row = {"cls": "unprobed-error", "path": rel, "baseline_rc": "-",
|
||||||
|
"sites": "-", "tried": 0, "valid": 0, "moved_at": "-",
|
||||||
|
"seconds": 0.0, "note": str(exc)[:120]}
|
||||||
|
row["blob"] = blobs.get(rel, "-")[:12]
|
||||||
|
rows.append(row)
|
||||||
|
print(" -> %-17s %s" % (row["cls"], str(exc)[:90]))
|
||||||
|
continue
|
||||||
|
row["blob"] = blobs.get(rel, "-")[:12]
|
||||||
|
row["path"] = rel
|
||||||
|
rows.append(row)
|
||||||
|
print(" -> %-17s rc=%s sites=%s tried=%s valid=%s moved_at=%s %ss"
|
||||||
|
% (row["cls"], row["baseline_rc"], row["sites"], row["tried"],
|
||||||
|
row["valid"], row["moved_at"], row["seconds"]))
|
||||||
|
with open(args.tsv, "w", encoding="utf-8") as fh:
|
||||||
|
write_header(fh, rev, len(rows), len(pop))
|
||||||
|
for r in rows:
|
||||||
|
fh.write("\t".join(str(r.get(k, "-")) for k in
|
||||||
|
("cls", "path", "baseline_rc", "sites", "tried",
|
||||||
|
"valid", "moved_at", "blob", "seconds")) + "\n")
|
||||||
|
|
||||||
|
with open(args.tsv, "w", encoding="utf-8") as fh:
|
||||||
|
merged = rows
|
||||||
|
if args.merge and prev_rows:
|
||||||
|
by_path = {r["path"]: r for r in prev_rows}
|
||||||
|
for r in rows:
|
||||||
|
by_path[r["path"]] = r
|
||||||
|
merged = [by_path[k] for k in sorted(by_path)]
|
||||||
|
write_header(fh, rev, len(merged), len(population(tree_blobs(repo, rev))))
|
||||||
|
for r in merged:
|
||||||
|
fh.write("\t".join(str(r.get(k, "-")) for k in
|
||||||
|
("cls", "path", "baseline_rc", "sites", "tried",
|
||||||
|
"valid", "moved_at", "blob", "seconds")) + "\n")
|
||||||
|
print("wrote %s (%d rows)" % (args.tsv, len(merged)))
|
||||||
|
print("work copy left at %s" % work)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def do_check(args):
|
||||||
|
if not os.path.isfile(args.tsv):
|
||||||
|
print("q65_rc_probe: no ledger at %s -- cannot grade" % args.tsv,
|
||||||
|
file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
try:
|
||||||
|
head, rows = read_tsv(args.tsv)
|
||||||
|
except ValueError as exc:
|
||||||
|
print("q65_rc_probe: %s" % exc, file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
repo = os.path.abspath(args.repo)
|
||||||
|
try:
|
||||||
|
blobs = tree_blobs(repo, args.rev)
|
||||||
|
except SystemExit as exc:
|
||||||
|
print(str(exc), file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
faults = []
|
||||||
|
counts = {c: 0 for c in CLASSES}
|
||||||
|
for r in rows:
|
||||||
|
if r["cls"] not in CLASSES:
|
||||||
|
faults.append("UNKNOWN_CLASS %s: %r" % (r["path"], r["cls"]))
|
||||||
|
continue
|
||||||
|
counts[r["cls"]] += 1
|
||||||
|
if r["path"] not in blobs:
|
||||||
|
faults.append("MISSING %s: row in the ledger, absent at %s" % (r["path"], args.rev[:12]))
|
||||||
|
continue
|
||||||
|
if not blobs[r["path"]].startswith(r["blob"]):
|
||||||
|
faults.append("STALE %s: row graded blob %s, tree holds %s"
|
||||||
|
% (r["path"], r["blob"], blobs[r["path"]][:12]))
|
||||||
|
try:
|
||||||
|
moved = int(r["moved_at"]) if r["moved_at"] not in ("-", "") else None
|
||||||
|
tried, valid = int(r["tried"]), int(r["valid"])
|
||||||
|
sites = int(r["sites"]) if r["sites"] not in ("-", "") else None
|
||||||
|
except ValueError:
|
||||||
|
faults.append("BAD_CELL %s: non-numeric tried/valid/moved_at" % r["path"])
|
||||||
|
continue
|
||||||
|
if r["cls"] == "sensitive" and not moved:
|
||||||
|
faults.append("CONTRADICTION %s: sensitive with no moving line" % r["path"])
|
||||||
|
if r["cls"] == "inert" and not (valid >= 1 and sites is not None
|
||||||
|
and tried >= sites and moved is None):
|
||||||
|
faults.append("CONTRADICTION %s: inert with valid=%d tried=%s sites=%s moved=%s"
|
||||||
|
% (r["path"], valid, tried, sites, moved))
|
||||||
|
if r["cls"] == "inert-partial" and not (valid >= 1 and sites is not None
|
||||||
|
and tried < sites and moved is None):
|
||||||
|
faults.append("CONTRADICTION %s: inert-partial with valid=%d tried=%s "
|
||||||
|
"sites=%s moved=%s"
|
||||||
|
% (r["path"], valid, tried, sites, moved))
|
||||||
|
if r["cls"] == "no-comparison" and sites:
|
||||||
|
faults.append("CONTRADICTION %s: no-comparison but lists %d site(s)"
|
||||||
|
% (r["path"], sites))
|
||||||
|
if r["cls"] == "unprobed-mutant" and valid:
|
||||||
|
faults.append("CONTRADICTION %s: unprobed-mutant with %d valid mutant(s)"
|
||||||
|
% (r["path"], valid))
|
||||||
|
|
||||||
|
committed = population(blobs)
|
||||||
|
probed = {r["path"] for r in rows}
|
||||||
|
print("q65_rc_probe -- rev %s, %d row(s), %d committed verifier(s)"
|
||||||
|
% (args.rev[:12], len(rows), len(committed)))
|
||||||
|
for c in CLASSES:
|
||||||
|
print(" %-18s %d" % (c, counts[c]))
|
||||||
|
print(" rc-sensitive %d of %d probed of %d committed"
|
||||||
|
% (counts["sensitive"], counts["sensitive"] + counts["inert"]
|
||||||
|
+ counts["inert-partial"] + counts["no-comparison"], len(committed)))
|
||||||
|
gap = [p for p in committed if p not in probed]
|
||||||
|
if gap:
|
||||||
|
print(" not probed %d (population coverage is the census's leg, "
|
||||||
|
"not this one)" % len(gap))
|
||||||
|
if faults:
|
||||||
|
for f in faults:
|
||||||
|
print(" FAULT %s" % f)
|
||||||
|
print("q65_rc_probe: %d fault(s)" % len(faults))
|
||||||
|
return 1
|
||||||
|
print("q65_rc_probe: 0 fault(s) -- every row content-bound and consistent "
|
||||||
|
"with its class")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------- selftest
|
||||||
|
def selftest():
|
||||||
|
tmp = tempfile.mkdtemp(prefix="rcprobe-selftest-", dir=SCRATCH_ROOT)
|
||||||
|
cells = []
|
||||||
|
|
||||||
|
def cell(name, ok, detail=""):
|
||||||
|
cells.append((name, bool(ok), detail))
|
||||||
|
|
||||||
|
def mk(name, body):
|
||||||
|
d = os.path.join(tmp, name)
|
||||||
|
os.makedirs(d, exist_ok=True)
|
||||||
|
p = os.path.join(d, "verify_x.py")
|
||||||
|
with open(p, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(body)
|
||||||
|
return p
|
||||||
|
|
||||||
|
# c1 sensitive: the inverted assertion reaches sys.exit
|
||||||
|
p = mk("sensitive", "import sys\nx = 2\nif x != 2:\n sys.exit(0)\n"
|
||||||
|
"if x == 2:\n print('ok')\nelse:\n sys.exit(1)\n")
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c1 an inverted assertion that reaches sys.exit is sensitive",
|
||||||
|
r["cls"] == "sensitive" and r["moved_at"], str(r))
|
||||||
|
|
||||||
|
# c2 inert: comparisons whose result is only printed
|
||||||
|
p = mk("inert", "x = 2\ny = 3\nprint('x==2', x == 2)\nprint('y<x', y < x)\n")
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c2 comparisons that only print leave the rc unmoved (inert)",
|
||||||
|
r["cls"] == "inert" and r["valid"] >= 1, str(r))
|
||||||
|
|
||||||
|
# c3 no comparison at all
|
||||||
|
p = mk("nocomp", "print('all checks passed')\n")
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c3 a file with no comparison is no-comparison",
|
||||||
|
r["cls"] == "no-comparison", str(r))
|
||||||
|
|
||||||
|
# c4 baseline non-zero
|
||||||
|
p = mk("baseline", "import sys\nsys.exit(2)\n")
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c4 a non-zero baseline is unprobed-baseline, not a pass",
|
||||||
|
r["cls"] == "unprobed-baseline", str(r))
|
||||||
|
|
||||||
|
# c5 every mutant invalid (the flip reaches an unreachable branch and crashes)
|
||||||
|
p = mk("invalid", "x = None\nif x == None:\n print('none')\nelse:\n"
|
||||||
|
" print(x.attr)\n")
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c5 a mutant that crashes is invalid, and no valid mutant is not inert",
|
||||||
|
r["cls"] == "unprobed-mutant" and r["tried"] == 1 and r["valid"] == 0, str(r))
|
||||||
|
cell("c5b the crashed-mutant file was restored byte for byte",
|
||||||
|
open(p).read() == "x = None\nif x == None:\n print('none')\nelse:\n"
|
||||||
|
" print(x.attr)\n")
|
||||||
|
|
||||||
|
# c5c one invalid mutant beside a valid one: counted, discarded, still graded
|
||||||
|
p = mk("mixed", "x = None\ny = 1\nif x == None:\n print('none')\nelse:\n"
|
||||||
|
" print(x.attr)\nif y == 1:\n print('y')\n")
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c5c an invalid mutant is counted and discarded, not counted as a test",
|
||||||
|
r["cls"] == "inert" and r["tried"] == 2 and r["valid"] == 1, str(r))
|
||||||
|
|
||||||
|
# c5d the sample is not the verdict: a moving site outside the 3-site
|
||||||
|
# even-spaced sample is found by the exhaustive pass
|
||||||
|
body5d = ("import sys\n"
|
||||||
|
"x1 = 1\n"
|
||||||
|
"x2 = 2\n"
|
||||||
|
"x3 = 3\n"
|
||||||
|
"x4 = 4\n"
|
||||||
|
"x5 = 5\n"
|
||||||
|
"print(x1 == 1)\n"
|
||||||
|
"if x2 == 2:\n"
|
||||||
|
" sys.exit(0)\n"
|
||||||
|
"print(x3 == 3)\n"
|
||||||
|
"print(x4 == 4)\n"
|
||||||
|
"print(x5 == 5)\n"
|
||||||
|
"sys.exit(1)\n")
|
||||||
|
p = mk("outside_sample", body5d)
|
||||||
|
r = probe(p, 3, 20)
|
||||||
|
cell("c5d a moving site the 3-site sample misses is still found",
|
||||||
|
r["cls"] == "sensitive" and r["moved_at"] == 8 and r["sites"] == 5,
|
||||||
|
str(r))
|
||||||
|
cell("c5d2 the sample really did miss it (site 8 is not in spread(sites,3))",
|
||||||
|
8 not in spread(mutable_lines(body5d), 3),
|
||||||
|
str(spread(mutable_lines(body5d), 3)))
|
||||||
|
|
||||||
|
# c5e the same file under a cap that stops before the moving site is
|
||||||
|
# inert-partial, not inert and not sensitive
|
||||||
|
p = mk("capped", body5d)
|
||||||
|
r = probe(p, 3, 20, exhaustive_cap=3)
|
||||||
|
cell("c5e a cap that stops before the moving site is inert-partial",
|
||||||
|
r["cls"] == "inert-partial" and r["tried"] == 3 and r["sites"] == 5,
|
||||||
|
str(r))
|
||||||
|
|
||||||
|
# c6 restore on a sensitive file
|
||||||
|
p = mk("restore", "import sys\nx = 1\nif x == 1:\n sys.exit(0)\n")
|
||||||
|
before = open(p, "rb").read()
|
||||||
|
probe(p, 3, 20)
|
||||||
|
cell("c6 a probed file is restored byte for byte", open(p, "rb").read() == before)
|
||||||
|
|
||||||
|
# c7 spread is deterministic and even
|
||||||
|
picks = list(range(0, 100))
|
||||||
|
cell("c7 spread picks evenly and deterministically",
|
||||||
|
spread(picks, 3) == spread(picks, 3) and len(spread(picks, 3)) == 3
|
||||||
|
and spread(picks, 3)[0] == 0 and spread(picks, 3)[-1] == 99,
|
||||||
|
str(spread(picks, 3)))
|
||||||
|
cell("c7b spread never invents a site", all(s in picks for s in spread(picks, 7)))
|
||||||
|
|
||||||
|
# c8 tsv round-trip
|
||||||
|
tsv = os.path.join(tmp, "led.tsv")
|
||||||
|
rows = [
|
||||||
|
{"cls": "sensitive", "path": "a/verify_a.py", "baseline_rc": 0, "sites": 4,
|
||||||
|
"tried": 2, "valid": 2, "moved_at": 12, "blob": "abc123", "seconds": 0.1},
|
||||||
|
{"cls": "inert", "path": "a/verify_b.py", "baseline_rc": 0, "sites": 1,
|
||||||
|
"tried": 1, "valid": 1, "moved_at": "-", "blob": "def456", "seconds": 0.2},
|
||||||
|
]
|
||||||
|
with open(tsv, "w") as fh:
|
||||||
|
write_header(fh, "f" * 40, len(rows), 2)
|
||||||
|
for r in rows:
|
||||||
|
fh.write("\t".join(str(r[k]) for k in ("cls", "path", "baseline_rc",
|
||||||
|
"sites", "tried", "valid",
|
||||||
|
"moved_at", "blob", "seconds")) + "\n")
|
||||||
|
head, back = read_tsv(tsv)
|
||||||
|
cell("c8 the ledger round-trips its rows and its header",
|
||||||
|
len(back) == 2 and head.get("rows") == "2" and back[0]["moved_at"] == "12",
|
||||||
|
str(head))
|
||||||
|
|
||||||
|
# c9 a short row is refused, not silently skipped
|
||||||
|
bad = os.path.join(tmp, "short.tsv")
|
||||||
|
open(bad, "w").write("# rev " + "f" * 40 + "\n# rows 1\n# committed 1\n"
|
||||||
|
"inert\ta/verify_a.py\t0\t1\n")
|
||||||
|
try:
|
||||||
|
read_tsv(bad)
|
||||||
|
cell("c9 a short row is refused", False)
|
||||||
|
except ValueError:
|
||||||
|
cell("c9 a short row is refused", True)
|
||||||
|
|
||||||
|
# c10 --check refuses a --work that is the tree
|
||||||
|
ns = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=DEFAULT_TSV,
|
||||||
|
only=None, limit=0, work=os.path.realpath(DEFAULT_REPO),
|
||||||
|
max_mutants=1, timeout=1,
|
||||||
|
windows=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||||
|
skip_over=300.0, merge=False)
|
||||||
|
cell("c10 --emit refuses --work equal to the repo root", do_emit(ns) == 2)
|
||||||
|
|
||||||
|
# c12 the window table parses, and a malformed row is skipped not fatal
|
||||||
|
wt = os.path.join(tmp, "win.tsv")
|
||||||
|
open(wt, "w").write("# path\ts\twhy\na/verify_a.py\t1800\tlive HTTP\n"
|
||||||
|
"b/verify_b.py\tnotanumber\tbroken\n")
|
||||||
|
w = read_windows(wt)
|
||||||
|
cell("c12 the window table parses a path and skips a malformed row",
|
||||||
|
w.get("a/verify_a.py") == 1800.0 and "b/verify_b.py" not in w, str(w))
|
||||||
|
|
||||||
|
# c13 a verifier over the skip-over is unprobed-window, and the emit still succeeds
|
||||||
|
one = os.path.join(tmp, "one.tsv")
|
||||||
|
ns3 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=one,
|
||||||
|
only="wizard-work/sphinx/archive-index-2026-10-23/verify_index.py",
|
||||||
|
limit=0, work=os.path.join(tmp, "work"),
|
||||||
|
max_mutants=1, timeout=1,
|
||||||
|
windows=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||||
|
skip_over=1.0, merge=False)
|
||||||
|
rc = do_emit(ns3)
|
||||||
|
head2, rows2 = read_tsv(one)
|
||||||
|
cell("c13 a file whose window exceeds --skip-over is unprobed-window, rc 0",
|
||||||
|
rc == 0 and len(rows2) == 1 and rows2[0]["cls"] == "unprobed-window",
|
||||||
|
str(rows2))
|
||||||
|
|
||||||
|
# c15 --merge keeps a row this run did not probe
|
||||||
|
ns4 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=one,
|
||||||
|
only="wizard-work/spark/verify_sum.py",
|
||||||
|
limit=0, work=os.path.join(tmp, "work"),
|
||||||
|
max_mutants=1, timeout=30,
|
||||||
|
windows=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||||
|
skip_over=300.0, merge=True)
|
||||||
|
rc = do_emit(ns4)
|
||||||
|
head3, rows3 = read_tsv(one)
|
||||||
|
kept = [r for r in rows3 if r["path"].endswith("archive-index-2026-10-23/verify_index.py")]
|
||||||
|
fresh = [r for r in rows3 if r["path"].endswith("spark/verify_sum.py")]
|
||||||
|
cell("c15 --merge keeps the unprobed row beside the new one",
|
||||||
|
rc == 0 and len(rows3) == 2 and kept and fresh, str(rows3))
|
||||||
|
|
||||||
|
# c14 a file the sweep cannot read raises (do_emit turns it into a row)
|
||||||
|
try:
|
||||||
|
probe(os.path.join(tmp, "absent", "verify_x.py"), 1, 5)
|
||||||
|
cell("c14 an unreadable file raises OSError, caught into a row", False)
|
||||||
|
except OSError:
|
||||||
|
cell("c14 an unreadable file raises OSError, caught into a row", True)
|
||||||
|
|
||||||
|
# c16 --check enforces the coverage half of the inert claim
|
||||||
|
blobs_now = tree_blobs(DEFAULT_REPO, "HEAD")
|
||||||
|
real = sorted(p for p in blobs_now
|
||||||
|
if p.startswith("wizard-work/")
|
||||||
|
and os.path.basename(p).startswith("verify_"))[0]
|
||||||
|
b12 = blobs_now[real][:12]
|
||||||
|
|
||||||
|
def _led(cls, tried, sites):
|
||||||
|
f = os.path.join(tmp, "chk-%s-%d-%d.tsv" % (cls, tried, sites))
|
||||||
|
with open(f, "w", encoding="utf-8") as fh:
|
||||||
|
write_header(fh, "HEAD", 1, 1)
|
||||||
|
fh.write("\t".join([cls, real, "0", str(sites), str(tried),
|
||||||
|
"1", "-", b12, "0.1"]) + "\n")
|
||||||
|
return do_check(argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD", tsv=f))
|
||||||
|
|
||||||
|
cell("c16 inert with sites untried is a contradiction (rc 1)",
|
||||||
|
_led("inert", 2, 5) == 1)
|
||||||
|
cell("c16b inert with every site tried is consistent (rc 0)",
|
||||||
|
_led("inert", 5, 5) == 0)
|
||||||
|
cell("c16c inert-partial with sites untried is consistent (rc 0)",
|
||||||
|
_led("inert-partial", 2, 5) == 0)
|
||||||
|
cell("c16d inert-partial with every site tried is a contradiction (rc 1)",
|
||||||
|
_led("inert-partial", 5, 5) == 1)
|
||||||
|
|
||||||
|
# c11 --check on a missing ledger is rc 2 (cannot grade), not rc 0
|
||||||
|
ns2 = argparse.Namespace(repo=DEFAULT_REPO, rev="HEAD",
|
||||||
|
tsv=os.path.join(tmp, "nope.tsv"))
|
||||||
|
cell("c11 a missing ledger is rc 2, not a pass", do_check(ns2) == 2)
|
||||||
|
|
||||||
|
ok = sum(1 for _, o, _ in cells if o)
|
||||||
|
for name, o, detail in cells:
|
||||||
|
print("%s %s%s" % ("ok " if o else "FAIL", name,
|
||||||
|
"" if o else " [%s]" % detail))
|
||||||
|
print("selftest: %d cells, %d passed, %d failed" % (len(cells), ok, len(cells) - ok))
|
||||||
|
return 0 if ok == len(cells) else 1
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--repo", default=DEFAULT_REPO)
|
||||||
|
ap.add_argument("--rev", default="HEAD")
|
||||||
|
ap.add_argument("--tsv", default=DEFAULT_TSV)
|
||||||
|
ap.add_argument("--work", help="copy dir to mutate (default: fresh scratch dir)")
|
||||||
|
ap.add_argument("--max-mutants", type=int, default=3)
|
||||||
|
ap.add_argument("--exhaustive-cap", type=int, default=40,
|
||||||
|
help="sites per file tried before an inert verdict is "
|
||||||
|
"downgraded to inert-partial; 0 = no cap")
|
||||||
|
ap.add_argument("--timeout", type=float, default=90.0,
|
||||||
|
help="window for a file the window table does not name")
|
||||||
|
ap.add_argument("--windows", default=os.path.join(HERE, "q65_row_windows.tsv"),
|
||||||
|
help="per-path measurement windows, shared with the runner")
|
||||||
|
ap.add_argument("--skip-over", type=float, default=300.0,
|
||||||
|
help="a verifier whose window exceeds this is not mutated; "
|
||||||
|
"grading it is the runner's row, not this sweep's")
|
||||||
|
ap.add_argument("--only", help="with --emit: only paths containing this substring")
|
||||||
|
ap.add_argument("--limit", type=int, default=0)
|
||||||
|
ap.add_argument("--emit", action="store_true")
|
||||||
|
ap.add_argument("--merge", action="store_true",
|
||||||
|
help="with --emit: keep rows for paths this run did not probe")
|
||||||
|
ap.add_argument("--check", action="store_true")
|
||||||
|
ap.add_argument("--selftest", action="store_true")
|
||||||
|
args = ap.parse_args()
|
||||||
|
if args.selftest:
|
||||||
|
return selftest()
|
||||||
|
if args.check:
|
||||||
|
return do_check(args)
|
||||||
|
if args.emit:
|
||||||
|
return do_emit(args)
|
||||||
|
ap.print_help()
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
# Q65 rc-sensitivity ledger -- one row per PROBED committed verifier.
|
||||||
|
# Written by q65_rc_probe.py --emit. A row states a measurement, not an
|
||||||
|
# exemption: --check faults on a row that contradicts its class or whose
|
||||||
|
# file has changed since (STALE).
|
||||||
|
# columns: class, path, baseline_rc, sites, tried, valid, moved_at, blob12, seconds
|
||||||
|
# rev 6e8f577c8cdd17ae7e6d5d2d20513a183ec19d94
|
||||||
|
# rows 110
|
||||||
|
# committed 110
|
||||||
|
sensitive wizard-work/beam/verify_toposort_none.py 0 4 1 1 21 a0df9ceb99a1 0.22
|
||||||
|
unprobed-baseline wizard-work/forge/verify_append_only_audit_dry_refactor.py 1 17 0 0 - 09d40fbd2ca0 2.16
|
||||||
|
sensitive wizard-work/forge/verify_append_only_owners_dry_refactor.py 0 13 1 1 56 0f650718d98b 0.65
|
||||||
|
sensitive wizard-work/forge/verify_census_row_delta_exit_contract.py 0 23 1 1 46 6afe76a665fa 1.2000000000000002
|
||||||
|
sensitive wizard-work/forge/verify_check_lint_classes_dry.py 0 15 1 1 80 62082e082a04 0.68
|
||||||
|
unprobed-baseline wizard-work/forge/verify_check_lint_dry_refactor.py 1 14 0 0 - 06a5c66c3f31 1.22
|
||||||
|
sensitive wizard-work/forge/verify_checkin_check_dry_refactor.py 0 8 1 1 103 d8ba2b221b80 0.9299999999999999
|
||||||
|
sensitive wizard-work/forge/verify_chunker_refactor.py 0 11 8 7 123 c3c7b5631f4a 0.8499999999999999
|
||||||
|
unprobed-timeout wizard-work/forge/verify_citation_hash_class_dry_refactor.py TIMEOUT 12 0 0 - 93abbc78e7f0 90.08
|
||||||
|
unprobed-window wizard-work/forge/verify_citation_hash_class_portable_default.py - - 0 0 - 81c481978bb3 0.0
|
||||||
|
unprobed-baseline wizard-work/forge/verify_citation_resolve_check_dry_refactor.py 1 9 0 0 - cd325ddd210d 15.52
|
||||||
|
unprobed-window wizard-work/forge/verify_doc_census_dry_refactor.py - - 0 0 - 52a9702aa0e0 0.0
|
||||||
|
unprobed-baseline wizard-work/forge/verify_doc_command_report_dry.py 1 8 0 0 - 7a2f42a5b95d 7.96
|
||||||
|
unprobed-baseline wizard-work/forge/verify_gate_blame_dry_refactor.py 1 9 0 0 - 8d01677402c9 10.81
|
||||||
|
sensitive wizard-work/forge/verify_landing_check_refactor.py 0 25 1 1 103 6aa282dade61 3.92
|
||||||
|
unprobed-baseline wizard-work/forge/verify_path_claim_apply_baseline_refactor.py 1 15 0 0 - 999da7274c3c 0.31
|
||||||
|
unprobed-baseline wizard-work/forge/verify_path_claim_dry_refactor.py 1 17 0 0 - ab35c13f9565 0.55
|
||||||
|
sensitive wizard-work/forge/verify_pdf_refactor.py 0 14 1 1 79 44ce76d39b69 7.27
|
||||||
|
sensitive wizard-work/forge/verify_probe_forge_refactor.py 0 21 1 1 67 0b9040ad6834 0.83
|
||||||
|
unprobed-baseline wizard-work/forge/verify_queue_append_audit_dry_refactor.py 1 13 0 0 - 0d5e5ef90a0d 7.81
|
||||||
|
sensitive wizard-work/forge/verify_reply_check_dry_refactor.py 0 11 1 1 72 c94ebf0e965e 3.0
|
||||||
|
sensitive wizard-work/forge/verify_repro_bounds_refactor.py 0 17 1 1 69 2394abeec257 0.94
|
||||||
|
sensitive wizard-work/forge/verify_roundtrip_refactor.py 0 8 1 1 39 0f1d7ea9d5e5 1.26
|
||||||
|
sensitive wizard-work/forge/verify_scan_dry_refactor.py 0 21 1 1 78 6fb7aa14b96e 6.3
|
||||||
|
unprobed-baseline wizard-work/forge/verify_selftest_sweep_dry_refactor.py 1 15 0 0 - 6f5e1621139c 0.25
|
||||||
|
unprobed-window wizard-work/forge/verify_selftest_sweep_refactor.py - - 0 0 - cc8782b4edfc 0.0
|
||||||
|
sensitive wizard-work/forge/verify_served_drift_summarise_dry.py 0 16 1 1 100 de5a49160b2c 12.190000000000001
|
||||||
|
sensitive wizard-work/forge/verify_smell_check_monotonic_refactor.py 0 16 1 1 64 8b8ccea5be0e 1.6800000000000002
|
||||||
|
sensitive wizard-work/forge/verify_smell_scan_refactor.py 0 11 1 1 55 499c8e000359 1.0
|
||||||
|
sensitive wizard-work/forge/verify_tool_map_dry_refactor.py 0 9 1 1 40 de1d9b7a65cf 49.33
|
||||||
|
unprobed-baseline wizard-work/forge/verify_write_probe_dry_refactor.py 1 12 0 0 - ef4d8c00e4a8 23.35
|
||||||
|
sensitive wizard-work/ledger/2026-10-25-git-history-live-b1edd09/verify_git_history.py 0 4 2 2 90 77048ae1e3f1 3.45
|
||||||
|
sensitive wizard-work/ledger/2026-10-25-head-20-commits/verify_head_commits.py 0 2 1 1 122 e12a07ca75fc 0.69
|
||||||
|
sensitive wizard-work/ledger/verify_arxiv_csai.py 0 14 1 1 37 872574acb239 0.49
|
||||||
|
sensitive wizard-work/ledger/verify_arxiv_csai_fresh.py 0 18 1 1 42 1a5acdde626e 0.47
|
||||||
|
sensitive wizard-work/ledger/verify_arxiv_csv.py 0 4 2 2 29 cc898af42ea8 0.33999999999999997
|
||||||
|
sensitive wizard-work/ledger/verify_arxiv_fresh.py 0 8 1 1 48 8c6eb40bcf4d 0.19
|
||||||
|
sensitive wizard-work/ledger/verify_catalog.py 0 14 2 1 77 d08c5d389ece 0.39
|
||||||
|
sensitive wizard-work/ledger/verify_commits_check.py 0 2 1 1 122 e84e0c7dd3ed 0.58
|
||||||
|
inert wizard-work/ledger/verify_dates.py 0 2 2 2 - dd3f643441c9 0.48
|
||||||
|
sensitive wizard-work/ledger/verify_gdp.py 0 12 2 2 27 ba577cd8ae05 0.29000000000000004
|
||||||
|
sensitive wizard-work/ledger/verify_gdp_per_capita.py 0 4 1 1 42 7f74b350c60e 1.52
|
||||||
|
unprobed-baseline wizard-work/ledger/verify_gitea_repos.py 1 8 0 0 - 73ef858149be 0.24
|
||||||
|
unprobed-baseline wizard-work/ledger/verify_github_python_repos.py 1 9 0 0 - 6956aa9c1560 2.86
|
||||||
|
sensitive wizard-work/ledger/verify_github_repos.py 0 14 2 2 107 269870d5e95a 5.529999999999999
|
||||||
|
sensitive wizard-work/ledger/verify_issues.py 0 4 1 1 48 176fd33f0522 0.22
|
||||||
|
unprobed-baseline wizard-work/ledger/verify_python_org_repos.py 1 9 0 0 - da60445425c4 2.02
|
||||||
|
sensitive wizard-work/ledger/verify_stackchain_catalog.py 0 8 1 1 105 60aa1853a04b 0.16999999999999998
|
||||||
|
sensitive wizard-work/ledger/verify_users.py 0 6 2 2 40 ef9b73c9cdd3 0.32
|
||||||
|
unprobed-baseline wizard-work/ledger/verify_wizards.py 1 19 0 0 - e08c8675a4e4 15.69
|
||||||
|
unprobed-baseline wizard-work/ledger/verify_workflow.py 1 4 0 0 - 8f9a0ce47f5c 0.22
|
||||||
|
sensitive wizard-work/null/artifact-seal/verify_artifact_seal_oracle.py 0 9 2 2 85 d88e5d606684 1.47
|
||||||
|
unprobed-baseline wizard-work/null/diffkit/verify_realrepo.py 1 12 0 0 - 1e610f00fa50 0.38
|
||||||
|
sensitive wizard-work/sol/verify_pending_request_triage.py 0 4 1 1 27 2d059319d3da 0.19
|
||||||
|
sensitive wizard-work/sol/verify_sol_profile_triage.py 0 5 1 1 37 84386c791a61 1.65
|
||||||
|
unprobed-baseline wizard-work/sol/verify_sol_triage.py 1 11 0 0 - 63792506caf3 0.09
|
||||||
|
sensitive wizard-work/sol/verify_triage.py 0 6 1 1 9 a2ffc04767ef 0.18
|
||||||
|
unprobed-baseline wizard-work/sol/verify_triage_queue.py 1 11 0 0 - 018ca9a2b570 0.13
|
||||||
|
sensitive wizard-work/sol/verify_troupe_sol_health.py 0 8 1 1 43 4b1a409c9b28 0.18
|
||||||
|
no-comparison wizard-work/spark/verify_all.py 0 0 0 0 - 41029ee9976f 0.09
|
||||||
|
sensitive wizard-work/spark/verify_analyzer.py 0 4 1 1 19 af7609cfadc1 0.44
|
||||||
|
inert wizard-work/spark/verify_check.py 0 3 3 3 - 3b7ef86aa2f7 0.31
|
||||||
|
inert wizard-work/spark/verify_count.py 0 1 1 1 - 640a65519f81 0.19
|
||||||
|
sensitive wizard-work/spark/verify_demo.py 0 3 1 1 32 7e4325553fc8 0.2
|
||||||
|
sensitive wizard-work/spark/verify_dir.py 0 3 1 1 15 ec6280d194d4 0.15000000000000002
|
||||||
|
sensitive wizard-work/spark/verify_dir_analyzer.py 0 6 1 1 19 89a49e7684d6 2.7800000000000002
|
||||||
|
no-comparison wizard-work/spark/verify_dirs.py 0 0 0 0 - d2846e746204 0.1
|
||||||
|
no-comparison wizard-work/spark/verify_env.py 0 0 0 0 - d28af1c48c1c 0.08
|
||||||
|
unprobed-baseline wizard-work/spark/verify_environment.py 1 3 0 0 - 4f526b94ecdd 0.07
|
||||||
|
inert wizard-work/spark/verify_fib.py 0 4 4 4 - 163468d71cd7 0.4100000000000001
|
||||||
|
sensitive wizard-work/spark/verify_formatter.py 0 4 1 1 14 0d2ef3a544f5 0.52
|
||||||
|
unprobed-baseline wizard-work/spark/verify_hermes.py 1 5 0 0 - 22c2f9e1ddd8 0.13
|
||||||
|
inert wizard-work/spark/verify_hermes_cli.py 0 9 9 9 - f3793430f6ab 10.519999999999998
|
||||||
|
inert wizard-work/spark/verify_hermes_path.py 0 1 1 1 - 385362b75cfe 0.16
|
||||||
|
unprobed-baseline wizard-work/spark/verify_inventory.py 1 2 0 0 - 02b54c2c98fe 0.1
|
||||||
|
no-comparison wizard-work/spark/verify_json.py 0 0 0 0 - 71b343f0b1f3 0.13
|
||||||
|
inert wizard-work/spark/verify_lines.py 0 1 1 1 - 5be4cf7b5f1e 0.25
|
||||||
|
sensitive wizard-work/spark/verify_loc_report.py 0 8 1 1 14 8e8aa89896b3 0.25
|
||||||
|
sensitive wizard-work/spark/verify_lwc.py 0 3 1 1 18 0ca330355ed9 0.56
|
||||||
|
inert wizard-work/spark/verify_minimal.py 0 3 3 3 - 2b1b18fc7ed7 0.68
|
||||||
|
no-comparison wizard-work/spark/verify_push.py 0 0 0 0 - 707bdcb80646 0.14
|
||||||
|
inert wizard-work/spark/verify_python_env.py 0 1 1 1 - 17c1d5f6ab0b 0.19
|
||||||
|
sensitive wizard-work/spark/verify_report.py 0 3 1 1 11 0b555f165e95 0.2
|
||||||
|
sensitive wizard-work/spark/verify_result.py 0 2 1 1 12 108832bc1239 0.25
|
||||||
|
sensitive wizard-work/spark/verify_scratch_analysis.py 0 6 1 1 17 8f8edfb7d4b3 0.19
|
||||||
|
unprobed-baseline wizard-work/spark/verify_skill_system.py 1 6 0 0 - 6739088b6520 0.16
|
||||||
|
no-comparison wizard-work/spark/verify_skills.py 0 0 0 0 - be69f202a823 0.11
|
||||||
|
no-comparison wizard-work/spark/verify_skills2.py 0 0 0 0 - 9dd92b966a6e 0.09
|
||||||
|
inert wizard-work/spark/verify_skills3.py 0 1 1 1 - 364a78ffbe33 0.24000000000000002
|
||||||
|
inert wizard-work/spark/verify_skills_fixed.py 0 1 1 1 - 4bc11949dc38 0.22
|
||||||
|
sensitive wizard-work/spark/verify_snippets.py 0 3 1 1 25 23d736539363 0.19
|
||||||
|
sensitive wizard-work/spark/verify_status.py 0 5 1 1 10 4c2928cdb2f2 0.63
|
||||||
|
no-comparison wizard-work/spark/verify_sum.py 0 0 0 0 - 8faaade76bc3 0.1
|
||||||
|
sensitive wizard-work/spark/verify_tool.py 0 3 1 1 12 fd90c7f545ad 1.08
|
||||||
|
sensitive wizard-work/spark/verify_wordcount.py 0 7 1 1 15 c4b7e15b56d0 0.46
|
||||||
|
sensitive wizard-work/sphinx/archive-agentic-memory-5paper-dossier-2026-10-25/verify_claims.py 0 17 2 1 88 50644ea156a1 8.68
|
||||||
|
sensitive wizard-work/sphinx/archive-gutenberg/verify_archive.py 0 12 2 2 63 36c966d37f08 1.2
|
||||||
|
unprobed-window wizard-work/sphinx/archive-index-2026-10-23/verify_index.py - - 0 0 - 9d4088a373a6 0.0
|
||||||
|
sensitive wizard-work/sphinx/archive-lc-coverage-crossindex-2026-10-23/verify_archive.py 0 44 1 1 57 b252edd8547d 0.37
|
||||||
|
unprobed-baseline wizard-work/sphinx/archive-lc-mm-dossier-2026-10-23/verify_dossier.py 1 29 0 0 - a15fab347cf6 4.96
|
||||||
|
sensitive wizard-work/sphinx/archive-lcmm-metadata-2026-10-25/verify_lcmm_dossier.py 0 35 1 1 106 3c4b48a678a8 13.06
|
||||||
|
unprobed-baseline wizard-work/sphinx/archive-memgallery-2026-10-21/verify_claims.py 1 2 0 0 - 22ba0b4bbdca 0.14
|
||||||
|
sensitive wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.py 0 10 1 1 61 39a5552b8500 0.48
|
||||||
|
sensitive wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py 0 19 1 1 44 534af05a12fa 0.38
|
||||||
|
sensitive wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py 0 22 1 1 93 c1f03701bf9a 7.75
|
||||||
|
unprobed-window wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py - - 0 0 - bed5df5c4a24 0.0
|
||||||
|
sensitive wizard-work/sphinx/archive-transformer-pdf-modality-2026-10-23/verify_archive.py 0 69 2 1 250 95997f60d905 9.260000000000002
|
||||||
|
unprobed-baseline wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 1 10 0 0 - c106248aea18 40.7
|
||||||
|
sensitive wizard-work/sphinx/figure-locator-llama3-2026-10-24/verify_locators.py 0 21 1 1 101 4b656a6d1d45 0.77
|
||||||
|
unprobed-baseline wizard-work/sphinx/verify_mm_dossier.py 1 12 0 0 - 4b015581816d 3.49
|
||||||
|
@@ -15,3 +15,4 @@ wizard-work/forge/verify_doc_census_dry_refactor.py 1800 runs tools/doc_census_c
|
|||||||
wizard-work/forge/verify_selftest_sweep_refactor.py 1800 runs tools/selftest_sweep.py over every pinned command
|
wizard-work/forge/verify_selftest_sweep_refactor.py 1800 runs tools/selftest_sweep.py over every pinned command
|
||||||
wizard-work/sphinx/archive-index-2026-10-23/verify_index.py 1800 reads every file at three revisions through git plumbing
|
wizard-work/sphinx/archive-index-2026-10-23/verify_index.py 1800 reads every file at three revisions through git plumbing
|
||||||
wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py 1800 live HTTP fetches, 3 retries at a 90s per-request timeout
|
wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py 1800 live HTTP fetches, 3 retries at a 90s per-request timeout
|
||||||
|
wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py 120 live HTTP: 1210 exact id_list lookups plus 3 listing pages; measured 19.87s at 5095e89, 6x headroom
|
||||||
|
@@ -17,7 +17,7 @@
|
|||||||
# non-docstring string constant naming an absolute host root. Such a row
|
# non-docstring string constant naming an absolute host root. Such a row
|
||||||
# is printed PASS* and excluded from the graded-clean figure.
|
# is printed PASS* and excluded from the graded-clean figure.
|
||||||
# Regenerate: q65_runner.py --emit
|
# Regenerate: q65_runner.py --emit
|
||||||
# rev: 25b04d0 rows: 88 emitted: q65_runner.py
|
# rev: 6e8f577 rows: 91 emitted: q65_runner.py
|
||||||
undeclared wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.32 0f650718d98bf28f07be134d49803cc902bf96b7 L41:/root/troupe-standup
|
undeclared wizard-work/forge/verify_append_only_owners_dry_refactor.py PASS 0 0.32 0f650718d98bf28f07be134d49803cc902bf96b7 L41:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 1.23 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 L31:/root/troupe-standup
|
undeclared wizard-work/forge/verify_census_row_delta_exit_contract.py PASS 0 1.23 6afe76a665fa32fbcb56cd0c40329b5fe32bfc98 L31:/root/troupe-standup
|
||||||
gate-run wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.53 62082e082a0472546b415219a9180836ac88908b L44:/root/troupe-standup
|
gate-run wizard-work/forge/verify_check_lint_classes_dry.py PASS 0 0.53 62082e082a0472546b415219a9180836ac88908b L44:/root/troupe-standup
|
||||||
@@ -33,6 +33,7 @@ undeclared wizard-work/forge/verify_gate_blame_dry_refactor.py FAIL 1 22.04 8d0
|
|||||||
undeclared wizard-work/forge/verify_landing_check_refactor.py PASS 0 6.12 6aa282dade614e3e55874adc3e37cf7c01313f3e L62:/root/troupe-standup
|
undeclared wizard-work/forge/verify_landing_check_refactor.py PASS 0 6.12 6aa282dade614e3e55874adc3e37cf7c01313f3e L62:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 0.52 FileNotFoundError: [Errno 2] No such file or directory: '/root/.hermes/profiles/volt/cache/scratch/pc_verify_snyuxz7t' ab35c13f9565e9eebbc80a210001ec37a002a267 L58:/root/troupe-standup
|
undeclared wizard-work/forge/verify_path_claim_dry_refactor.py FAIL 1 0.52 FileNotFoundError: [Errno 2] No such file or directory: '/root/.hermes/profiles/volt/cache/scratch/pc_verify_snyuxz7t' ab35c13f9565e9eebbc80a210001ec37a002a267 L58:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_pdf_refactor.py PASS 0 7.75 44ce76d39b69a7d6798af521f958370bf30ad2b1 L41:/root/.hermes
|
undeclared wizard-work/forge/verify_pdf_refactor.py PASS 0 7.75 44ce76d39b69a7d6798af521f958370bf30ad2b1 L41:/root/.hermes
|
||||||
|
undeclared wizard-work/forge/verify_probe_forge_refactor.py PASS 0 0.42 0b9040ad683464475d77b58c744f16dfbdb8438e
|
||||||
undeclared wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 16.94 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 L51:/root/troupe-standup
|
undeclared wizard-work/forge/verify_queue_append_audit_dry_refactor.py FAIL 1 16.94 0d5e5ef90a0d0ba030b6909285db6ed3120a3895 L51:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 7.31 c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 L33:/root/troupe-standup
|
undeclared wizard-work/forge/verify_reply_check_dry_refactor.py PASS 0 7.31 c94ebf0e965e99c7e67d1ef9df7e35cd184ccb53 L33:/root/troupe-standup
|
||||||
undeclared wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.35 2394abeec25726726edd91696520d788d72a5d88 L33:/root/troupe-standup
|
undeclared wizard-work/forge/verify_repro_bounds_refactor.py PASS 0 0.35 2394abeec25726726edd91696520d788d72a5d88 L33:/root/troupe-standup
|
||||||
@@ -52,6 +53,7 @@ undeclared wizard-work/ledger/verify_arxiv_fresh.py PASS 0 0.09 8c6eb40bcf4d512
|
|||||||
undeclared wizard-work/ledger/verify_catalog.py PASS 0 0.08 d08c5d389ece6b9906a7315849dfe83f99f28f41 L13:/root/.hermes
|
undeclared wizard-work/ledger/verify_catalog.py PASS 0 0.08 d08c5d389ece6b9906a7315849dfe83f99f28f41 L13:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_dates.py PASS 0 0.13 dd3f643441c9fd985639c0ac7f250e05a6184aeb L6:/root/troupe-standup
|
undeclared wizard-work/ledger/verify_dates.py PASS 0 0.13 dd3f643441c9fd985639c0ac7f250e05a6184aeb L6:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_gdp.py PASS 0 0.10 ba577cd8ae0502f57bfd7b8a7dadd8ef54455dbd L3:/root/troupe-standup
|
undeclared wizard-work/ledger/verify_gdp.py PASS 0 0.10 ba577cd8ae0502f57bfd7b8a7dadd8ef54455dbd L3:/root/troupe-standup
|
||||||
|
undeclared wizard-work/ledger/verify_gdp_per_capita.py PASS 0 0.84 7f74b350c60ee7fd0e361f8d60a2e669ae703a97 L19:/root/troupe-standup
|
||||||
undeclared wizard-work/ledger/verify_gitea_repos.py FAIL 1 0.23 73ef858149be5fd7e9ec58056435c3bd31f9419c L16:/root/.hermes
|
undeclared wizard-work/ledger/verify_gitea_repos.py FAIL 1 0.23 73ef858149be5fd7e9ec58056435c3bd31f9419c L16:/root/.hermes
|
||||||
undeclared wizard-work/ledger/verify_github_python_repos.py FAIL 1 3.35 6956aa9c1560e8b6bf85f239eb5da663797d0a70
|
undeclared wizard-work/ledger/verify_github_python_repos.py FAIL 1 3.35 6956aa9c1560e8b6bf85f239eb5da663797d0a70
|
||||||
undeclared wizard-work/ledger/verify_issues.py PASS 0 0.09 176fd33f05223aaaf789608b662c3d253f685422 L15:/root/.hermes
|
undeclared wizard-work/ledger/verify_issues.py PASS 0 0.09 176fd33f05223aaaf789608b662c3d253f685422 L15:/root/.hermes
|
||||||
@@ -105,4 +107,5 @@ undeclared wizard-work/sphinx/archive-memlens-2026-10-22/verify_memlens_archive.
|
|||||||
undeclared wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py PASS 0 0.21 534af05a12fa9d3285384bae57bce527ed033425
|
undeclared wizard-work/sphinx/archive-mm-prm-2026-10-22/verify_mm_prm_archive.py PASS 0 0.21 534af05a12fa9d3285384bae57bce527ed033425
|
||||||
wired-invoke wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py PASS 0 2.28 c1f03701bf9af0018e7aac9a567255dd8218488d
|
wired-invoke wizard-work/sphinx/archive-mmlongcite-2026-10-25/verify_mmlongcite.py PASS 0 2.28 c1f03701bf9af0018e7aac9a567255dd8218488d
|
||||||
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py FAIL 1 837.56 bed5df5c4a24c961a3c2bb564b299444a22d5189
|
undeclared wizard-work/sphinx/archive-remember-r1-figures-2026-10-23/verify_archive.py FAIL 1 837.56 bed5df5c4a24c961a3c2bb564b299444a22d5189
|
||||||
|
undeclared wizard-work/sphinx/arxiv-digest-verified-2026-10-25/verify_digest_claims.py PASS 0 19.87 c106248aea1856d29d987497af4fe95dfe6d9768 L59:/root/.hermes
|
||||||
undeclared wizard-work/sphinx/verify_mm_dossier.py PASS 0 4.30 4b015581816de5472d11577d661ce4e97f38773a L28:/root/.hermes
|
undeclared wizard-work/sphinx/verify_mm_dossier.py PASS 0 4.30 4b015581816de5472d11577d661ce4e97f38773a L28:/root/.hermes
|
||||||
|
Reference in new issue
Block a user