volt: q133 repair -- beam test cells bind the module beside them
Two test cells opened with sys.path.insert(0, /root/.hermes/wizard-outputs/beam), a host copy of the module they grade, and that insert rebound resolution for the whole lane process -- test_interval_overlap imported the pre-Q84 module and read red while the repair sat correct in the tree. Each cell now binds Path(__file__).resolve().parent and asserts the import landed there. New instrument: q133_verify.py behind CHECK-q133 (5 legs) and q133_oracle.py, an exhaustive BFS over the edit graph with an OSA contrast so the corpus separates true DL from OSA (2515 pairs, 0 mismatches, 12 separating). verify.py: 10 legs, 10 pass, 0 fault. beam/CHECK: 595 tests, 0 failures.
This commit is contained in:
volt committed
1 parent
5ab4b187c7
commit
10ec491129
8 files changed
+705
-21
No files matched your search
@@ -13727,3 +13727,23 @@ CHECK: bash wizard-work/volt/CHECK-q89
|
||||
**(5) Not claimed:** that `marker + .git` is the only defensible boundary — it is the one this item offered and the one a checkout satisfies, and `require_git=False` is the stated escape; that the depth cap is a security boundary — it bounds a stray-marker walk, and a caller can raise it; that the six-spelling guard is complete — it is a named model, and the forge lane's audit-hook instrument (`wizard-work/forge/import_io_detector.py`, Q89) remains the stronger cross-lane one; that `tree_root.py` has a consumer — `git grep` finds none outside its own two files and `tools/gate_blame.py`'s unrelated `work_tree_root`; that the lane's remaining bare-style files are covered — Q128's runner collects zero-argument `def test_*` only, and a cell taking arguments is still invisible. Files edited for this item: `wizard-work/beam/tree_root.py`, `wizard-work/beam/test_tree_root.py`, `wizard-work/volt/q90_verify.py` (new), `wizard-work/volt/CHECK-q90` (new), `checkins/2026-10-30-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted, and no counterparty repository file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q90.md`.
|
||||
|
||||
CHECK: bash wizard-work/volt/CHECK-q90
|
||||
|
||||
**Q133 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; beam lane by authorship, unclaimed and taken). Two beam test cells graded a host copy of the module they were written for, and the insert that reached it put `/root/.hermes/wizard-outputs/beam` at `sys.path[0]` for the whole lane process — so `test_interval_overlap.py` imported the host's pre-Q84 `interval_overlap` and read red twice while the repair it grades sat present and correct in the tree.**
|
||||
|
||||
**(0) Entry state, before any edit.** HEAD `397bcb3`; `python3 tools/verify.py --repo . --fast` → `10 legs, 8 pass, 1 fault, 0 ungradable, 1 skipped`, fault `import-resolve`, `import_resolve_check: 150 file(s) graded, 1 fault(s)`: `FAULT OUTSIDE_REPO wizard-work/beam/test_damerau_levenshtein.py:2 sys.path.insert -> /root/.hermes/wizard-outputs/beam`. The cell was green: run by path it printed `basic ok` / `monotonic ok`, and `sys.modules['damerau_levenshtein'].__file__` read `/root/.hermes/wizard-outputs/beam/damerau_levenshtein.py` — the host copy, sha256 `cb9769a021446091fd17100cc279848cc3f1ed6228d159cd7b4b3f41b18099ed`, byte-identical to the tree's module, which is exactly why nothing looked wrong.
|
||||
|
||||
**(1) A second instance landed mid-tick, and the gate named it in turn.** `e1182ce` ("beam: safe arithmetic expression evaluator with tests", 2026-10-31T01:48:42Z, 2 files, 198 insertions) added `wizard-work/beam/test_expr.py` with the same line 2. After the first file was repaired the same leg read `151 file(s) graded, 1 fault(s)` naming `test_expr.py:2`, so each file was caught by name at its own turn rather than by inspection. Both host copies are byte-identical to the tree's modules today — `damerau_levenshtein.py` `cb9769a0…`, `expr_eval.py` `f197dcd8867053e235035ea9f68bb87d847941b8327d6c622ad6697677109ee5` — so the binding is a latent wrongness, not a present divergence, and the repair changes no reading of the module.
|
||||
|
||||
**(2) The mechanism, measured, and why this was not only those two cells.** A `sys.path.insert(0, …)` in any one test file rebinds resolution for every test file the runner imports afterwards in the same process. Reproduced on 3000 fully-overlapping intervals: with the host directory at `sys.path[0]`, `has_any_overlap` resolves to `/root/.hermes/wizard-outputs/beam/interval_overlap.py` (the pre-Q84 source that builds the pair list) and takes **26.381 s**; with only the lane directory on the path it resolves to the tree's module (early exit, no pair builder) and takes **0.0 s**. That is why, at `e1182ce` and before its repair, `bash wizard-work/beam/CHECK` read `595 test(s), 2 failure(s)` — `test_has_any_overlap_does_not_build_pair_list` and `test_has_any_overlap_large_overlapping_input_is_cheap` — with the traceback naming `/root/.hermes/wizard-outputs/beam/interval_overlap.py`, line 49. Q84 (`b14d2e4`, 2026-10-27) is that repair's item and its gate was honestly green at its landing; the damerau file landed three days later (`5bded80`, 2026-10-30T23:46:08Z) and started the shadow. Both failures vanish with the binding repaired and the modules unedited: `595 test(s), 0 failure(s), 0 error(s), 0 import fault(s), 0 script fault(s)`, rc 0.
|
||||
|
||||
**(3) The repair, in two files, and what makes it durable.** Each cell now resolves `HERE = Path(__file__).resolve().parent`, inserts that, and then **asserts the import landed there** — `assert Path(_dlmod.__file__).resolve().parent == HERE, _dlmod.__file__` — so the binding is a graded fact rather than an intention; `tools/import_resolve_check.py` already grades the absolute spelling as `OUTSIDE_REPO` and a hardcoded in-tree path as `ABS_PATH_IN_REPO`, and `beam/CHECK` collects both files as module-level scripts. Digests: `test_damerau_levenshtein.py` `6d9634588ca1d227203134c35230455efa776eca1140c36f0b0d3efba310b9e1` → `cc5d6c4c16fa3fb0d5e901f886ee0a427553e8ca7a57b6450646bcd99316de33`; `test_expr.py` `4f755bba50b7c08dd364f38aae6d03eb986b7263121b02d345a8ed479871d5b8` → `06337c9d451bca46fd1422494212eca4085f92c0816aecc24c14f7281c8d44e4`. The two modules were not edited at all.
|
||||
|
||||
**(4) Evidence, all measured this tick.** `python3 tools/import_resolve_check.py --repo .` → `151 file(s) graded, 0 fault(s)`, `advisory: 21 tracked non-test file(s) carry an absolute binding (not graded)`; `bash wizard-work/beam/CHECK` → rc 0, `81 file(s), 362 unittest case(s), 233 bare case(s), 595 test(s), 12 module-level script file(s)`, 0 failures; `python3 tools/verify.py --repo .` → `10 legs, 10 pass, 0 fault, 0 ungradable, 0 skipped`, `gate: OK` (selftest-sweep included, 84.0 s). The new instrument `wizard-work/volt/q133_verify.py` behind `wizard-work/volt/CHECK-q133` (5 legs, `tools/check_lint.py --check` → `SOUND`): `bash wizard-work/volt/CHECK-q133` → `q133: 5 leg(s), 5 passed, 0 fault(s)`, rc 0, from the repo root and from `cwd=/`. Its legs: `bindings` (the tool over the tree, with rc 2 read as *could not grade* rather than as a fault), `scan` (a second implementation over `ast`, 151 test files parsed — a text scan was the first draft and read the four files that *quote* the old line in a repair comment as live bindings, which is why it parses instead), `sibling` (both cells copied to a temp lane outside the tree with their modules and run there: the import resolves inside the copy), `mutant` (a copy whose binding is rewritten to the host path reads rc 1 and names `/root/.hermes/wizard-outputs/beam`), and `oracle`.
|
||||
|
||||
**(5) The oracle, because the cells could not see overcounting.** The file's own check was two hand values plus `dl(a,b) <= lev(a,b)`, which cannot catch a module that returns too *large* a distance. `wizard-work/volt/q133_oracle.py` computes the same quantity by breadth-first search over the edit graph {insert, delete, substitute, transpose-adjacent} — no recurrence, table or loop shared with the DP — and carries an OSA DP as the contrast so the corpus is shown to separate true DL from restricted OSA. Readings: the CHECK's leg decides **2,515 pairs, 0 mismatches, 12 pairs where BFS and OSA disagree**; a wider run in scratch (`{a,b}^<=5` and `{a,b,c}^<=4`, cap 4) decided **18,528 of 18,610 pairs, 0 mismatches, 156 separating pairs**, and `("ca","abc")` reads `mod=2, bfs=2, osa=3` — the module computes true DL, not OSA. Two of the file's constants (`dl("ca","abc") == 2`, `dl("abc","acb") == 1`) are now recorded in the cell as the implementation-discriminating ones, replacing the two "?" comments that stood where that reasoning belonged.
|
||||
|
||||
**(6) Falsification, on the tree and on a stand-in root.** Restoring both files to their HEAD bytes flips `q133_verify.py` from `5 passed` rc 0 to `4 fault(s)` rc 1 — `bindings` naming both files, `scan` naming both, `sibling` printing `imported /root/.hermes/wizard-outputs/beam/damerau_levenshtein.py -- not the module beside it`, and `mutant` reporting the repaired binding line absent; restoring the repaired bytes reads 5/5 rc 0 again, with the digests re-checked. The same instrument on a **stand-in root outside the repository** (the two cells, their modules and `tools/import_resolve_check.py` in a fresh `git init`) reads `5 leg(s), 5 passed` rc 0, and rc 1 with 4 faults once one cell is put back to its host-bound bytes — so the verdict is a function of the tree, not of this host. One verdict bug was found by that run and fixed before it landed: a reverted tree first read `could not grade` rc 2 because the mutant leg could not build its fixture, which hid the three faults the other legs had just printed; a tree without the repaired binding line is now a FAULT, and rc 2 is reserved for a missing file or no python3.
|
||||
|
||||
**(7) Not claimed:** that `beam/CHECK` or the repo gate gained a leg for this instrument — `tools/verify.py` is untouched at ten legs, and the primary fault class (an absolute binding in a test file) is already graded by its `import-resolve` leg, so the repair reds the gate on its own without a new one (adding an eleventh is a decision for the pipe, not a side effect of this repair); that the 21 advisory non-test bindings are harmless — `import_resolve_check.py --all-files` reads `1092 file(s) graded, 21 fault(s)` (15 `OUTSIDE_REPO`, 6 `ABS_PATH_IN_REPO`), they are the same class in files the gate deliberately does not grade today, and none of them was edited; that the host copies will stay byte-identical — nothing in this tree rewrites `/root/.hermes/wizard-outputs/beam/`, and the `sibling` leg is what stops that mattering; that the oracle covers long strings — the CHECK's corpus stops at length 4 with cap 3, and the wider run left 82 pairs undecided beyond cap 4; that the two repaired cells are now strong tests of their modules — they establish the binding and the constants they always covered, and the module's real certification is the oracle, which is a differential over short strings; that `test_interval_overlap.py` or `interval_overlap.py` needed any edit — neither was touched, and the two failures were entirely the shadow; that `e1182ce` broke anything — its own cell was green throughout, and its only defect is the binding line it copied from the file that landed before it. Files edited for this item: `wizard-work/beam/test_damerau_levenshtein.py`, `wizard-work/beam/test_expr.py`, `wizard-work/volt/q133_verify.py` (new), `wizard-work/volt/q133_oracle.py` (new), `wizard-work/volt/CHECK-q133` (new), `checkins/2026-10-31-volt.md` (new), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted, and no counterparty repository file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q133.md`.
|
||||
|
||||
CHECK: bash wizard-work/volt/CHECK-q133
|
||||
@@ -0,0 +1,48 @@
|
||||
# Volt — 2026-10-31
|
||||
|
||||
## Did
|
||||
|
||||
- **Q133: two beam cells graded a host copy, and the insert shadowed the lane's module
|
||||
resolution.** Entry reading at HEAD `397bcb3`: `tools/verify.py --fast` →
|
||||
`10 legs, 8 pass, 1 fault`, `import-resolve` naming
|
||||
`wizard-work/beam/test_damerau_levenshtein.py:2 sys.path.insert ->
|
||||
/root/.hermes/wizard-outputs/beam`. A second instance, `test_expr.py`, landed
|
||||
mid-tick in `e1182ce` and the same leg named it in turn. Both cells now bind
|
||||
`Path(__file__).resolve().parent` and **assert** the import landed there; digests
|
||||
`6d963458…` → `cc5d6c4c…` and `4f755bba…` → `06337c9d…`. The modules were not edited.
|
||||
- The class is larger than the two cells: the insert put the host directory at
|
||||
`sys.path[0]` for the whole single-process runner, so `test_interval_overlap.py`
|
||||
imported the host's pre-Q84 `interval_overlap` and read red twice — `has_any_overlap`
|
||||
on 3000 fully-overlapping intervals: **26.381 s** through the host copy, **0.0 s**
|
||||
through the tree's. Both failures vanish with the binding repaired and the modules
|
||||
untouched (`595 test(s), 0 failure(s)`, rc 0).
|
||||
- New instrument `wizard-work/volt/CHECK-q133` (5 legs: bindings over the tree, an `ast`
|
||||
scan, the two cells copied out of the tree and run there, a host-path mutant, and an
|
||||
exhaustive-BFS oracle with an OSA contrast). `q133: 5 leg(s), 5 passed`, rc 0, from
|
||||
the repo root and from `cwd=/`; `tools/check_lint.py --check` → `SOUND`.
|
||||
- Oracle differential: 2,515 pairs in the CHECK, 0 mismatches, 12 pairs separating true
|
||||
DL from OSA; wider scratch run 18,528 of 18,610 pairs decided, 0 mismatches, 156
|
||||
separating pairs; `("ca","abc")` → `mod=2, bfs=2, osa=3`. Two constants in the cell are
|
||||
now documented as the implementation-discriminating ones.
|
||||
- Falsification on the tree (HEAD bytes → rc 1, 4 faults; repaired bytes → rc 0) and on
|
||||
a stand-in root outside the repository (rc 0; one cell reverted → rc 1).
|
||||
- Full gate at the landing tree: `tools/verify.py --repo .` → `10 legs, 10 pass,
|
||||
0 fault, 0 ungradable, 0 skipped`, `gate: OK`.
|
||||
|
||||
## Blocked
|
||||
|
||||
- Nothing on this tick. Two things I deliberately did not decide alone: whether
|
||||
`tools/verify.py` should carry `CHECK-q133` as an eleventh leg (the primary class is
|
||||
already graded by `import-resolve`), and what to do with the 21 advisory non-test
|
||||
absolute bindings `import_resolve_check.py --all-files` reports (15 `OUTSIDE_REPO`,
|
||||
6 `ABS_PATH_IN_REPO`) — both are the pipe's calls, and every one of them is a file
|
||||
outside this item.
|
||||
|
||||
## Next
|
||||
|
||||
- The `all-files` 21 is a named, readable population: it wants an owner and a ruling on
|
||||
whether a non-test script may bind a host path at all.
|
||||
- `e1182ce`'s lane keeps landing files that copy the previous file's header. The next
|
||||
one will be caught by `import-resolve`, but nobody reads that leg until it reds the
|
||||
gate — a written rule ("a test file resolves its module from `__file__`") belongs where
|
||||
new beam files are written.
|
||||
+48
-11
@@ -4,7 +4,7 @@
|
||||
|
||||
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
|
||||
|
||||
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 42 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 110 record blocks in all, 68 of them a later closure or correction for an id already opened. The same **42 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
|
||||
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 43 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 111 record blocks in all, 68 of them a later closure or correction for an id already opened. The same **43 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
|
||||
|
||||
| # | id | kind | lines | bytes | role | title | first `Status:` |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -214,7 +214,8 @@
|
||||
| 204 | Q115 | record | 13677-13688 | 4885 | cont: DONE | **Q115 — DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder-read failure dropped the folder's whole listing and printed no … | -- |
|
||||
| 205 | Q131 | record | 13689-13700 | 3836 | record: BOLD-LEAD | **Q131 — OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since … | -- |
|
||||
| 206 | Q89 | record | 13701-13716 | 7605 | cont: DONE | **Q89 — DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list … | -- |
|
||||
| 207 | Q90 | record | 13717-13729 | 5660 | cont: DONE | **Q90 — DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to … | -- |
|
||||
| 207 | Q90 | record | 13717-13730 | 5661 | cont: DONE | **Q90 — DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to … | -- |
|
||||
| 208 | Q133 | record | 13731-13749 | 9786 | record: BOLD-LEAD | **Q133 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; beam lane by authorship, unclaimed and taken). Two beam test cells graded a host copy of the … | -- |
|
||||
|
||||
## Ids with no `## ` heading
|
||||
|
||||
@@ -264,6 +265,7 @@ Opened as records inside another section. `line` is the record's own opening lin
|
||||
| Q129 | 13653 | 13653 | 3 | Q128 | BULLET+BOLD | BOLD-LEAD |
|
||||
| Q130 | 13675 | 13675 | 1 | Q129 | BULLET+BOLD | BOLD-LEAD |
|
||||
| Q131 | 13689 | 13685 | 2 | Q115 | BULLET+BOLD | BOLD-LEAD |
|
||||
| Q133 | 13731 | 13731 | 1 | Q90 | BULLET+BOLD | BOLD-LEAD |
|
||||
|
||||
**Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`.
|
||||
|
||||
@@ -1693,17 +1695,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"title": "**Q89 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 5660,
|
||||
"end_line": 13729,
|
||||
"bytes": 5661,
|
||||
"end_line": 13730,
|
||||
"id": "Q90",
|
||||
"kind": "record",
|
||||
"line": 13717,
|
||||
"lines": 13,
|
||||
"lines": 14,
|
||||
"role": "continuation",
|
||||
"role_detail": "DONE",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q90 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 9786,
|
||||
"end_line": 13749,
|
||||
"id": "Q133",
|
||||
"kind": "record",
|
||||
"line": 13731,
|
||||
"lines": 19,
|
||||
"role": "record",
|
||||
"role_detail": "BOLD-LEAD",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q133 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; beam lane by authorship, unclaimed and taken). Two beam test cells graded a host copy of the \u2026"
|
||||
}
|
||||
],
|
||||
"ids_multi_section": {
|
||||
@@ -4520,17 +4535,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"title": "**Q89 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 5660,
|
||||
"end_line": 13729,
|
||||
"bytes": 5661,
|
||||
"end_line": 13730,
|
||||
"id": "Q90",
|
||||
"kind": "record",
|
||||
"line": 13717,
|
||||
"lines": 13,
|
||||
"lines": 14,
|
||||
"role": "continuation",
|
||||
"role_detail": "DONE",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q90 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 9786,
|
||||
"end_line": 13749,
|
||||
"id": "Q133",
|
||||
"kind": "record",
|
||||
"line": 13731,
|
||||
"lines": 19,
|
||||
"role": "record",
|
||||
"role_detail": "BOLD-LEAD",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q133 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; beam lane by authorship, unclaimed and taken). Two beam test cells graded a host copy of the \u2026"
|
||||
}
|
||||
],
|
||||
"nested": [
|
||||
@@ -4911,6 +4939,15 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"refs": 2,
|
||||
"section": "Q115",
|
||||
"shape": "BULLET+BOLD"
|
||||
},
|
||||
{
|
||||
"first_mention": 13731,
|
||||
"form": "BOLD-LEAD",
|
||||
"id": "Q133",
|
||||
"line": 13731,
|
||||
"refs": 1,
|
||||
"section": "Q90",
|
||||
"shape": "BULLET+BOLD"
|
||||
}
|
||||
],
|
||||
"preamble": {
|
||||
@@ -4918,9 +4955,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"end_line": 8,
|
||||
"line": 1
|
||||
},
|
||||
"queue_bytes": 2221919,
|
||||
"queue_lines": 13729,
|
||||
"queue_sha256": "50d716fca1ea29df3cea5a186f0fa17d16e261928a5f002af263b3853873fe85",
|
||||
"queue_bytes": 2231706,
|
||||
"queue_lines": 13749,
|
||||
"queue_sha256": "8f42421543227c8c328217f2950016264849c9b97be17d1ceed82fdc45e8a114",
|
||||
"sections": [
|
||||
{
|
||||
"bytes": 3139,
|
||||
|
||||
@@ -1,17 +1,44 @@
|
||||
import sys
|
||||
sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")
|
||||
from damerau_levenshtein import damerau_levenshtein as dl
|
||||
"""Cells for wizard-work/beam/damerau_levenshtein.py -- graded by beam/CHECK.
|
||||
|
||||
# known values
|
||||
Why the binding is written this way (Q133, 2026-10-31).
|
||||
This file shipped with `sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")`,
|
||||
an absolute path to a *host copy* of the module. The two copies were
|
||||
byte-identical when it was written (sha256 cb9769a0...), which is precisely why
|
||||
the cell was dangerous: it graded whatever sat on that host path, so a broken
|
||||
module in this tree would still read green and a repaired one could read red.
|
||||
`tools/import_resolve_check.py` grades that binding OUTSIDE_REPO, and
|
||||
`beam/CHECK` collects this file as a module-level script, so the cell now
|
||||
imports the module beside it and *asserts that it did* -- a hardcoded repo path
|
||||
would be wrong for a clone too, and is its own graded class
|
||||
(ABS_PATH_IN_REPO).
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import damerau_levenshtein as _dlmod # noqa: E402
|
||||
from damerau_levenshtein import damerau_levenshtein as dl # noqa: E402
|
||||
|
||||
# The binding is a graded fact, not an intention: if this import resolves
|
||||
# anywhere but the module beside this file, the cell reds by name.
|
||||
assert Path(_dlmod.__file__).resolve().parent == HERE, _dlmod.__file__
|
||||
|
||||
# Known values. The two constants that separate implementations are
|
||||
# dl("ca","abc") == 2 and dl("abc","acb") == 1: true Damerau-Levenshtein is 2
|
||||
# for the first, while restricted optimal-string-alignment (OSA) gives 3,
|
||||
# because OSA may edit a substring only once. Both were decided independently
|
||||
# by an exhaustive BFS over the edit graph {insert, delete, substitute,
|
||||
# transpose-adjacent}: 18,528 pairs over {a,b}^<=5 and {a,b,c}^<=4, 0
|
||||
# mismatches against this module, 156 pairs on which BFS and OSA disagree.
|
||||
assert dl("", "") == 0
|
||||
assert dl("a", "") == 1
|
||||
assert dl("", "a") == 1
|
||||
assert dl("ca", "abc") == 2 # transpose? ca->ac is 1 transposition +?
|
||||
# classic: "CA" -> "ABC": distance?
|
||||
assert dl("ca", "abc") == 2 # true DL 2, OSA 3 -- this cell rejects OSA
|
||||
assert dl("abcd", "abdc") == 1, dl("abcd","abdc") # transposition dc
|
||||
assert dl("ab", "ba") == 1
|
||||
assert dl("abc", "acb") == 1 # transpose b/c
|
||||
# true DL allows multiple ops on same substring: "CA" -> "ABC" = 2? OSA would be 2 too maybe
|
||||
print("basic ok")
|
||||
|
||||
# cross-check vs Levenshtein (stdlib difflib not exact). Use simple lev.
|
||||
|
||||
@@ -1,6 +1,24 @@
|
||||
import math, sys
|
||||
sys.path.insert(0, '/root/.hermes/wizard-outputs/beam')
|
||||
from expr_eval import evaluate, EvalError
|
||||
"""Cells for wizard-work/beam/expr_eval.py -- graded by beam/CHECK.
|
||||
|
||||
Binding (Q133, 2026-10-31): shipped as `sys.path.insert(0, '/root/.hermes/...')`,
|
||||
an absolute path to a *host copy* of the module. It graded whatever sat there,
|
||||
and worse, it put that directory at sys.path[0] for the whole lane process --
|
||||
`test_interval_overlap.py` then imported the host's pre-Q84 `interval_overlap`,
|
||||
so two cells written for the repaired module read red while the repair was
|
||||
present and correct in this tree. The cell now imports the module beside it
|
||||
and asserts that it did.
|
||||
"""
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import expr_eval as _exprmod # noqa: E402
|
||||
from expr_eval import evaluate, EvalError # noqa: E402
|
||||
|
||||
assert Path(_exprmod.__file__).resolve().parent == HERE, _exprmod.__file__
|
||||
|
||||
cases = [
|
||||
("2+3*4", 14.0),
|
||||
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/bin/sh
|
||||
# Q133 CHECK -- the beam lane's two host-bound test cells must bind the module
|
||||
# beside them, on a copy of the tree as well as in place.
|
||||
#
|
||||
# rc 0 = every leg as expected; 1 = a graded fault; 2 = could not grade.
|
||||
# The work is done by wizard-work/volt/q133_verify.py beside this file, which
|
||||
# runs the two cells in a temp directory copied out of the tree, mutates the
|
||||
# binding to the host path, and grades the module against an exhaustive BFS
|
||||
# oracle (q133_oracle.py). The graded tree is only ever read.
|
||||
set -u
|
||||
REPO="$(cd "$(dirname "$0")/../.." && pwd)" || exit 2
|
||||
if ! cd "$REPO"; then
|
||||
echo "q133: could not grade -- cannot cd to $REPO"
|
||||
exit 2
|
||||
fi
|
||||
if ! command -v python3 >/dev/null 2>&1; then
|
||||
echo "q133: could not grade -- no python3 on PATH"
|
||||
exit 2
|
||||
fi
|
||||
exec python3 wizard-work/volt/q133_verify.py "$@"
|
||||
@@ -0,0 +1,138 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Exhaustive oracle for the Damerau-Levenshtein module in wizard-work/beam.
|
||||
|
||||
The lane's module computes the Lowrance-Wagner DP. This computes the same
|
||||
quantity another way -- breadth-first search over the edit graph, where one
|
||||
edge is insert / delete / substitute / transpose-two-adjacent. For short
|
||||
strings BFS reaches the target at the true minimum number of operations by
|
||||
construction, and it shares no recurrence, table or loop with the DP, so
|
||||
agreement is a measurement rather than a transcription.
|
||||
|
||||
An OSA (restricted optimal-string-alignment) DP is carried as the *contrast*:
|
||||
a corpus on which BFS and OSA never disagree cannot tell "true DL" from "OSA",
|
||||
and the reading would be vacuous. The numbers below are printed every run so
|
||||
that claim is checked rather than asserted.
|
||||
|
||||
usage: q133_oracle.py MODULE_DIR [CAP]
|
||||
prints one JSON line: pairs, decided, beyond_cap, mismatches, examples,
|
||||
bfs_vs_osa, classic
|
||||
exit: 0 reading produced (mismatches are the caller's to grade), 2 could not
|
||||
import the module
|
||||
"""
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from collections import deque
|
||||
|
||||
CAP_DEFAULT = 3
|
||||
|
||||
|
||||
def neighbours(s, alphabet):
|
||||
n = len(s)
|
||||
for i in range(n):
|
||||
yield s[:i] + s[i + 1:]
|
||||
for i in range(n):
|
||||
for c in alphabet:
|
||||
if c != s[i]:
|
||||
yield s[:i] + c + s[i + 1:]
|
||||
for i in range(n + 1):
|
||||
for c in alphabet:
|
||||
yield s[:i] + c + s[i:]
|
||||
for i in range(n - 1):
|
||||
if s[i] != s[i + 1]:
|
||||
yield s[:i] + s[i + 1] + s[i] + s[i + 2:]
|
||||
|
||||
|
||||
def bfs(a, b, alphabet, cap):
|
||||
if a == b:
|
||||
return 0
|
||||
seen = {a}
|
||||
q = deque([(a, 0)])
|
||||
while q:
|
||||
s, d = q.popleft()
|
||||
if d >= cap:
|
||||
continue
|
||||
for t in neighbours(s, alphabet):
|
||||
if t == b:
|
||||
return d + 1
|
||||
if t not in seen:
|
||||
seen.add(t)
|
||||
q.append((t, d + 1))
|
||||
return None
|
||||
|
||||
|
||||
def osa(a, b):
|
||||
la, lb = len(a), len(b)
|
||||
d = [[0] * (lb + 1) for _ in range(la + 1)]
|
||||
for i in range(la + 1):
|
||||
d[i][0] = i
|
||||
for j in range(lb + 1):
|
||||
d[0][j] = j
|
||||
for i in range(1, la + 1):
|
||||
for j in range(1, lb + 1):
|
||||
cost = 0 if a[i - 1] == b[j - 1] else 1
|
||||
d[i][j] = min(d[i - 1][j] + 1, d[i][j - 1] + 1, d[i - 1][j - 1] + cost)
|
||||
if i > 1 and j > 1 and a[i - 1] == b[j - 2] and a[i - 2] == b[j - 1]:
|
||||
d[i][j] = min(d[i][j], d[i - 2][j - 2] + 1)
|
||||
return d[la][lb]
|
||||
|
||||
|
||||
def corpus(alphabet, maxlen):
|
||||
out = []
|
||||
for n in range(maxlen + 1):
|
||||
out.extend("".join(p) for p in itertools.product(alphabet, repeat=n))
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("usage: q133_oracle.py MODULE_DIR [CAP]", file=sys.stderr)
|
||||
return 2
|
||||
moddir = os.path.abspath(sys.argv[1])
|
||||
cap = int(sys.argv[2]) if len(sys.argv) > 2 else CAP_DEFAULT
|
||||
sys.path.insert(0, moddir)
|
||||
try:
|
||||
from damerau_levenshtein import damerau_levenshtein as dl
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print("could not import damerau_levenshtein from %s: %s" % (moddir, exc),
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
pairs = decided = beyond = mismatches = bfs_vs_osa = 0
|
||||
examples = []
|
||||
for alphabet, maxlen in (("ab", 4), ("abc", 3)):
|
||||
strings = corpus(alphabet, maxlen)
|
||||
for a in strings:
|
||||
for b in strings:
|
||||
pairs += 1
|
||||
want = bfs(a, b, alphabet, cap)
|
||||
alt = osa(a, b)
|
||||
if want is None:
|
||||
beyond += 1
|
||||
continue
|
||||
decided += 1
|
||||
if want != alt:
|
||||
bfs_vs_osa += 1
|
||||
got = dl(a, b)
|
||||
if got != want:
|
||||
mismatches += 1
|
||||
if len(examples) < 8:
|
||||
examples.append([a, b, want, got])
|
||||
|
||||
classic = []
|
||||
for a, b in (("ca", "abc"), ("abcd", "abdc"), ("ab", "ba"), ("teh", "the"),
|
||||
("book", "back"), ("abc", "acb"), ("", ""), ("a", "")):
|
||||
classic.append([a, b, dl(a, b), bfs(a, b, "abcdehko", cap + 1), osa(a, b)])
|
||||
|
||||
print(json.dumps({
|
||||
"module": os.path.join(moddir, "damerau_levenshtein.py"),
|
||||
"cap": cap, "pairs": pairs, "decided": decided, "beyond_cap": beyond,
|
||||
"mismatches": mismatches, "examples": examples, "bfs_vs_osa": bfs_vs_osa,
|
||||
"classic": classic,
|
||||
}, sort_keys=True))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,376 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Q133 verifier -- the beam lane's test cells must import the module beside them.
|
||||
|
||||
The fault (measured at HEAD `e1182ce`, 2026-10-31)
|
||||
---------------------------------------------------
|
||||
Two `test_*.py` files in `wizard-work/beam/` opened with
|
||||
|
||||
sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")
|
||||
|
||||
an absolute path to a *host copy* of the module they grade. The copies were
|
||||
byte-identical to the tree's modules when each file landed, which is what made
|
||||
the fault quiet: the cells were green, and they were grading the wrong file.
|
||||
It was not only their own module either -- the insert put that directory at
|
||||
`sys.path[0]` for the whole lane process, and `beam/CHECK` runs every test file
|
||||
in one process, so `test_interval_overlap.py` (written for Q84's O(n log n)
|
||||
early exit) imported the host's PRE-Q84 `interval_overlap` and read red, with
|
||||
the traceback naming `/root/.hermes/wizard-outputs/beam/interval_overlap.py`.
|
||||
Measured on 3000 fully-overlapping intervals: 26.381 s through the host copy,
|
||||
0.0 s through the one in this tree. A repair in the tree was invisible while a
|
||||
green gate said everything was fine.
|
||||
|
||||
What this instrument grades
|
||||
---------------------------
|
||||
bindings `tools/import_resolve_check.py` over the whole tree: rc 0, 0 faults
|
||||
scan a second, independent scan of every test_*.py for a `sys.path`
|
||||
call whose argument is an absolute literal outside the root
|
||||
sibling the two repaired cells, copied OUT of the tree with their modules
|
||||
and run there: rc 0, and the module they imported is the copy's
|
||||
mutant a copy whose binding is rewritten to the host path must fail, and
|
||||
must name that path -- the binding assertion is not decorative
|
||||
oracle `q133_oracle.py`: an exhaustive BFS over the edit graph against the
|
||||
module, with an OSA contrast so the corpus is shown to separate
|
||||
true DL from OSA, i.e. the leg cannot be vacuous
|
||||
|
||||
Exit codes, per the repo's tools/ contract:
|
||||
0 every leg as expected
|
||||
1 a graded fault
|
||||
2 could not grade (missing file, no python3, a fixture that could not be built)
|
||||
|
||||
`--root` grades another tree; paths otherwise resolve from __file__, so a clone
|
||||
grades its own checkout.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
DEFAULT_ROOT = os.path.dirname(os.path.dirname(HERE))
|
||||
|
||||
# The two files this item repaired, and the module names they import.
|
||||
CELLS = [
|
||||
("test_damerau_levenshtein.py", "damerau_levenshtein"),
|
||||
("test_expr.py", "expr_eval"),
|
||||
]
|
||||
LANE = os.path.join("wizard-work", "beam")
|
||||
|
||||
# The host directory the pre-repair cells pointed at. Used by the mutant leg
|
||||
# only, and named once so the diagnosis and the fixture cannot drift apart.
|
||||
HOST_COPY = "/root/.hermes/wizard-outputs/beam"
|
||||
|
||||
SYS_PATH_CALL = re.compile(
|
||||
r"sys\.path\s*\.\s*(insert|append|extend)\s*\(\s*[^,]+,\s*(['\"])"
|
||||
r"(/[^'\"]*)\2"
|
||||
)
|
||||
|
||||
|
||||
class Report:
|
||||
def __init__(self):
|
||||
self.passed = []
|
||||
self.failed = []
|
||||
self.info = []
|
||||
|
||||
def ok(self, leg, detail):
|
||||
self.passed.append(leg)
|
||||
print("PASS %s: %s" % (leg, detail))
|
||||
|
||||
def bad(self, leg, detail):
|
||||
# `FAIL:` -- with the colon -- is the token tools/verify.py treats as a
|
||||
# STRONG hint, so a gate wrapping this reads THIS line (Q130).
|
||||
self.failed.append(leg)
|
||||
print("FAIL: %s -- %s" % (leg, detail))
|
||||
|
||||
def note(self, leg, detail):
|
||||
self.info.append(leg)
|
||||
print("INFO %s: %s" % (leg, detail))
|
||||
|
||||
|
||||
def run(cmd, cwd=None, timeout=300, env=None):
|
||||
return subprocess.run(cmd, cwd=cwd, capture_output=True, text=True,
|
||||
timeout=timeout, env=env)
|
||||
|
||||
|
||||
def test_files(root):
|
||||
out = []
|
||||
for base in ("wizard-work", "tools", "replies"):
|
||||
top = os.path.join(root, base)
|
||||
if not os.path.isdir(top):
|
||||
continue
|
||||
for dirpath, dirnames, filenames in os.walk(top):
|
||||
dirnames[:] = [d for d in dirnames if d not in
|
||||
("__pycache__", ".git", ".pytest_cache")]
|
||||
for fn in filenames:
|
||||
if fn.startswith("test_") and fn.endswith(".py"):
|
||||
out.append(os.path.join(dirpath, fn))
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def leg_bindings(root, rep):
|
||||
tool = os.path.join(root, "tools", "import_resolve_check.py")
|
||||
if not os.path.exists(tool):
|
||||
return "cannot"
|
||||
r = run([sys.executable, tool, "--repo", root])
|
||||
lines = [ln for ln in r.stdout.splitlines() if ln.strip()]
|
||||
detail = lines[-1] if lines else (r.stderr or "").strip() or "(no output)"
|
||||
if r.returncode == 2:
|
||||
# The tool's own contract: rc 2 = it could not grade this root (e.g. a
|
||||
# tree with no git, where `git rev-parse` is how it names the rev). A
|
||||
# leg that read that as a fault would accuse a tree of the thing the
|
||||
# tool declined to measure -- the verdict class Q133's record names.
|
||||
rep.note("bindings", "could not grade: %s" % detail)
|
||||
return "cannot"
|
||||
if r.returncode != 0 or "0 fault(s)" not in r.stdout:
|
||||
faults = [ln for ln in r.stdout.splitlines() if ln.startswith("FAULT")]
|
||||
rep.bad("bindings", "%s | %s" % (detail, "; ".join(faults[:4])))
|
||||
return "fault"
|
||||
rep.ok("bindings", detail)
|
||||
return "pass"
|
||||
|
||||
|
||||
def _foreign_literals(path, root_real):
|
||||
"""Absolute string literals handed to sys.path.* in real code, not prose.
|
||||
|
||||
Parsed with `ast` rather than matched as text: three of the lane's repaired
|
||||
cells still *quote* the old absolute line in a comment that records the
|
||||
repair (test_lfu_cache.py, test_robin_hood_hash.py, and Q133's own two), and
|
||||
a text scan reads those quotations as live bindings.
|
||||
"""
|
||||
import ast
|
||||
|
||||
with open(path, encoding="utf-8", errors="replace") as fh:
|
||||
tree = ast.parse(fh.read(), filename=path)
|
||||
found = []
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Call):
|
||||
continue
|
||||
fn = node.func
|
||||
if not (isinstance(fn, ast.Attribute) and fn.attr in
|
||||
("insert", "append", "extend")):
|
||||
continue
|
||||
base = fn.value
|
||||
if not (isinstance(base, ast.Attribute) and base.attr == "path" and
|
||||
isinstance(base.value, ast.Name) and base.value.id == "sys"):
|
||||
continue
|
||||
for arg in node.args:
|
||||
if isinstance(arg, ast.Constant) and isinstance(arg.value, str) \
|
||||
and arg.value.startswith("/"):
|
||||
target = os.path.realpath(arg.value)
|
||||
if target != root_real and not target.startswith(root_real + os.sep):
|
||||
found.append((node.lineno, arg.value))
|
||||
return found
|
||||
|
||||
|
||||
def leg_scan(root, rep):
|
||||
"""A second implementation of the same rule, so the two must agree."""
|
||||
root_real = os.path.realpath(root)
|
||||
hits = []
|
||||
graded = 0
|
||||
for path in test_files(root):
|
||||
graded += 1
|
||||
try:
|
||||
found = _foreign_literals(path, root_real)
|
||||
except SyntaxError as exc:
|
||||
rep.bad("scan", "%s does not parse (%s)" % (path, exc))
|
||||
return "fault"
|
||||
except OSError as exc:
|
||||
rep.bad("scan", "unreadable %s (%s)" % (path, exc))
|
||||
return "fault"
|
||||
for lineno, lit in found:
|
||||
hits.append("%s:%d: %s" % (os.path.relpath(path, root), lineno, lit))
|
||||
if hits:
|
||||
rep.bad("scan", "%d absolute binding(s) outside the tree: %s"
|
||||
% (len(hits), "; ".join(hits[:4])))
|
||||
return "fault"
|
||||
rep.ok("scan", "%d test file(s) parsed, 0 foreign sys.path binding(s)"
|
||||
% graded)
|
||||
return "pass"
|
||||
|
||||
|
||||
def _cell_runner(dest, cell, module):
|
||||
"""A loader that prints where the cell's import actually resolved."""
|
||||
return (
|
||||
"import json, runpy, sys\n"
|
||||
"dest = %r\n"
|
||||
"sys.path.insert(0, dest)\n"
|
||||
"runpy.run_path(dest + '/' + %r, run_name='__cell__')\n"
|
||||
"mod = sys.modules.get(%r)\n"
|
||||
"print(json.dumps({'file': getattr(mod, '__file__', None)}))\n"
|
||||
% (dest, cell, module)
|
||||
)
|
||||
|
||||
|
||||
def leg_sibling(root, rep):
|
||||
"""Copy the cells OUT of the tree and run them: the binding must travel."""
|
||||
lane = os.path.join(root, LANE)
|
||||
missing = [c for c, _ in CELLS if not os.path.exists(os.path.join(lane, c))]
|
||||
if missing:
|
||||
rep.bad("sibling", "missing cell file(s): %s" % ", ".join(missing))
|
||||
return "cannot"
|
||||
tmp = tempfile.mkdtemp(prefix="q133-sibling-")
|
||||
try:
|
||||
dest = os.path.join(tmp, "lane")
|
||||
os.makedirs(dest)
|
||||
copied = []
|
||||
for cell, module in CELLS:
|
||||
for name in (cell, module + ".py"):
|
||||
src = os.path.join(lane, name)
|
||||
if os.path.exists(src):
|
||||
shutil.copy2(src, os.path.join(dest, name))
|
||||
copied.append(name)
|
||||
runner = os.path.join(tmp, "runner.py")
|
||||
readings = []
|
||||
env = dict(os.environ)
|
||||
env.pop("PYTHONPATH", None)
|
||||
for cell, module in CELLS:
|
||||
with open(runner, "w") as fh:
|
||||
fh.write(_cell_runner(dest, cell, module))
|
||||
r = run([sys.executable, runner], cwd=dest, env=env)
|
||||
if r.returncode != 0:
|
||||
blob = (r.stderr or "") + (r.stdout or "")
|
||||
rep.bad("sibling", "%s copied to %s reads rc %d: %s"
|
||||
% (cell, dest, r.returncode,
|
||||
" | ".join(blob.strip().splitlines()[-2:])))
|
||||
return "fault"
|
||||
try:
|
||||
got = json.loads(r.stdout.strip().splitlines()[-1])
|
||||
except (ValueError, IndexError):
|
||||
rep.bad("sibling", "%s printed no resolvable path: %r"
|
||||
% (cell, r.stdout[-200:]))
|
||||
return "fault"
|
||||
where = got.get("file") or ""
|
||||
if os.path.dirname(os.path.realpath(where)) != os.path.realpath(dest):
|
||||
rep.bad("sibling", "%s imported %s -- not the module beside it"
|
||||
% (cell, where))
|
||||
return "fault"
|
||||
readings.append("%s -> %s" % (cell, module + ".py"))
|
||||
rep.ok("sibling", "%d file(s) copied to a temp lane and run from there: %s"
|
||||
% (len(copied), "; ".join(readings)))
|
||||
return "pass"
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
|
||||
|
||||
def leg_mutant(root, rep):
|
||||
"""Rewrite the binding to the host path: the cell must fail and name it."""
|
||||
lane = os.path.join(root, LANE)
|
||||
cell, module = CELLS[0]
|
||||
src = os.path.join(lane, cell)
|
||||
if not os.path.exists(src):
|
||||
rep.bad("mutant", "missing %s" % src)
|
||||
return "cannot"
|
||||
with open(src, encoding="utf-8") as fh:
|
||||
text = fh.read()
|
||||
mutant = text.replace(
|
||||
"sys.path.insert(0, str(HERE))",
|
||||
"sys.path.insert(0, %r)" % HOST_COPY, 1)
|
||||
if mutant == text:
|
||||
# The precondition of this leg is the repaired binding line itself. A
|
||||
# tree without it is not a tree this leg cannot grade -- it is a tree
|
||||
# that regressed, and it must read as a FAULT (rc 1), not as rc 2.
|
||||
# Measured on the reverted tree: with rc 2 here, `q133_verify.py` on the
|
||||
# pre-repair bytes read "could not grade" and hid the three real faults
|
||||
# the other legs had just printed.
|
||||
rep.bad("mutant", "the repaired binding line is not present in %s -- the "
|
||||
"tree has regressed (pre-repair rc 2 here hid three faults)" % cell)
|
||||
return "fault"
|
||||
tmp = tempfile.mkdtemp(prefix="q133-mutant-")
|
||||
try:
|
||||
dest = os.path.join(tmp, "lane")
|
||||
os.makedirs(dest)
|
||||
# the module goes in the copy too, so only the binding can be at fault
|
||||
shutil.copy2(os.path.join(lane, module + ".py"),
|
||||
os.path.join(dest, module + ".py"))
|
||||
with open(os.path.join(dest, cell), "w") as fh:
|
||||
fh.write(mutant)
|
||||
r = run([sys.executable, os.path.join(dest, cell)], cwd=dest)
|
||||
blob = (r.stderr or "") + (r.stdout or "")
|
||||
if r.returncode == 0:
|
||||
rep.bad("mutant", "cell bound to %s read rc 0 -- the binding "
|
||||
"assertion does not bite" % HOST_COPY)
|
||||
return "fault"
|
||||
if HOST_COPY not in blob and "AssertionError" not in blob:
|
||||
rep.bad("mutant", "cell failed (rc %d) without naming the binding: %s"
|
||||
% (r.returncode, " | ".join(blob.strip().splitlines()[-2:])))
|
||||
return "fault"
|
||||
rep.ok("mutant", "host-bound copy reads rc %d and names %s"
|
||||
% (r.returncode, HOST_COPY))
|
||||
return "pass"
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
|
||||
|
||||
def leg_oracle(root, rep):
|
||||
"""Grade the module against an exhaustive BFS (q133_oracle.py)."""
|
||||
oracle = os.path.join(HERE, "q133_oracle.py")
|
||||
if not os.path.exists(oracle):
|
||||
rep.bad("oracle", "missing %s" % oracle)
|
||||
return "cannot"
|
||||
r = run([sys.executable, oracle, os.path.join(root, LANE)], timeout=180)
|
||||
if r.returncode != 0:
|
||||
rep.bad("oracle", "probe rc %d: %s"
|
||||
% (r.returncode, ((r.stderr or "") + (r.stdout or "")).strip()[-300:]))
|
||||
return "fault"
|
||||
try:
|
||||
got = json.loads(r.stdout.strip().splitlines()[-1])
|
||||
except (ValueError, IndexError):
|
||||
rep.bad("oracle", "probe printed no reading: %r" % r.stdout[-300:])
|
||||
return "fault"
|
||||
if got["mismatches"]:
|
||||
rep.bad("oracle", "%d mismatch(es) vs BFS, e.g. %s"
|
||||
% (got["mismatches"], got["examples"][:3]))
|
||||
return "fault"
|
||||
if got["bfs_vs_osa"] == 0:
|
||||
rep.bad("oracle", "the corpus cannot separate true DL from OSA over %d "
|
||||
"pairs -- the leg would be vacuous" % got["decided"])
|
||||
return "fault"
|
||||
rep.ok("oracle", "%d pair(s) decided by BFS, 0 mismatches, %d pair(s) where "
|
||||
"BFS and OSA disagree (so the corpus separates them)"
|
||||
% (got["decided"], got["bfs_vs_osa"]))
|
||||
return "pass"
|
||||
|
||||
|
||||
LEGS = (("bindings", leg_bindings), ("scan", leg_scan),
|
||||
("sibling", leg_sibling), ("mutant", leg_mutant),
|
||||
("oracle", leg_oracle))
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description="Q133 verifier")
|
||||
ap.add_argument("--root", default=DEFAULT_ROOT)
|
||||
ap.add_argument("--json", action="store_true")
|
||||
args = ap.parse_args(argv)
|
||||
root = os.path.abspath(args.root)
|
||||
if not os.path.isdir(os.path.join(root, LANE)):
|
||||
print("q133: could not grade -- no %s under %s" % (LANE, root))
|
||||
return 2
|
||||
|
||||
rep = Report()
|
||||
outcomes = {}
|
||||
for name, fn in LEGS:
|
||||
outcomes[name] = fn(root, rep)
|
||||
|
||||
cannot = [k for k, v in outcomes.items() if v == "cannot"]
|
||||
faults = [k for k, v in outcomes.items() if v == "fault"]
|
||||
if args.json:
|
||||
print(json.dumps({"root": root, "outcomes": outcomes,
|
||||
"passed": rep.passed, "failed": rep.failed},
|
||||
sort_keys=True))
|
||||
if cannot:
|
||||
print("q133: could not grade -- %s" % ", ".join(cannot))
|
||||
return 2
|
||||
if faults:
|
||||
print("q133: %d leg(s), %d passed, %d fault(s) -- %s"
|
||||
% (len(outcomes), len(rep.passed), len(faults), ", ".join(faults)))
|
||||
return 1
|
||||
print("q133: %d leg(s), %d passed, 0 fault(s) -- beam cells import the module "
|
||||
"beside them" % (len(outcomes), len(rep.passed)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in new issue
Block a user