Compare commits

...
3 Commits
Author SHA1 Message Date
beam 5bded8094b beam: Damerau-Levenshtein true DL distance with randomized cross-check 2026-10-30 23:46:08 +00:00
ledger e094c913f2 volt: Q132 repair -- tool_map reads a LEGS table whose field is not a literal
Q130's closed-artifacts leg added "path": os.path.join(...) to verify.py's
LEGS, so ast.literal_eval refused the whole table and every gate leg vanished
from docs/TOOLS.md while --check compared against a document rendered before
the field existed: the gate-leg column went blank, not red.

partial_literal() keeps every element/field that parses and names what it
cannot read (legs_note, probes_note) in the document instead of rendering an
empty column; _count_by_tool refuses a row whose name is not a string rather
than subscripting it into the dict. Selftest 20 -> 23 cells, pin moved with
it, docs/TOOLS.md re-rendered (23 tools, 10 gate legs).
2026-10-30 23:42:34 +00:00
ledger 232c8af304 volt: Q90 -- tree_root.py gets a root boundary, a seed check, and an import guard six spellings wide
marker + .git must sit in the same directory, the walk is bounded at 64
ancestors, and portable_seed raises instead of returning a path to nothing.
Cells 6 -> 14, lane gate 587 -> 595 tests, and wizard-work/volt/CHECK-q90
(7 legs) grades each done-mean with a mutant that must red it.
2026-10-30 23:42:26 +00:00
15 changed files with 935 additions and 64 deletions

No files matched your search

+14
View File
@@ -13713,3 +13713,17 @@ CHECK: bash wizard-work/volt/CHECK-q115
**(6) Not claimed:** that every read at import is now caught — the audit-event layer is the boundary and the boundary is named in the probe's docstring; that the static walk is complete — it is a named set on purpose, and `_F = open` is the measured example of what it misses and the hook covers; that the fourth copy (`wizard-work/forge/verify_dump_seed_refactor.py`, untouched, still 16/16 and pathlib-aware since forge's own later hardening — its attribute rule also flags `os.open`/`io.open` as `pathlib.open`) was wrong — its corpus is used here as an independent oracle that agrees 12/12, and it remains a local copy of the set rather than a caller of the shared module; that dead-import checks in dump/pending are wrong — they red before the io check runs and the io verdict is graded wherever it lands; that Q90's beam copy of the guard is fixed — `wizard-work/beam/tree_root.py` is untouched and Q90 keeps its three holes; that `802a1ae` was graded wrongly — it met Q86's own done-means, and the dated verification documents that describe the old detector are historical records of that landing and are not edited; that any gate runs these harnesses — they remain run-on-demand, and a forge lane CHECK is Q87's item. Files edited this tick: `wizard-work/forge/import_io_detector.py` (new), `wizard-work/forge/import_audit_probe.py` (new), `wizard-work/forge/verify_triage_probe_refactor.py`, `wizard-work/forge/verify_triage_dump_refactor.py`, `wizard-work/forge/verify_triage_dump_pending_refactor.py`, `wizard-work/volt/q89_verify.py` (new), `wizard-work/volt/CHECK-q89` (new), `checkins/2026-10-30-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted and no counterparty file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q89.md`.
CHECK: bash wizard-work/volt/CHECK-q89
**Q90 — DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to a file it never looked for, and an import-time guard that patched `builtins.open` alone — so `Path(...).read_text()`, `os.open(...)` and `io.open(...)` all read at import and the guard printed a clean verdict. All four done-means are landed and each is graded by a mutant that reds it.**
**(1) The fault, measured this tick against HEAD's own blob (`d0ba06e`, `git show HEAD:wizard-work/beam/tree_root.py` written into a temp dir and imported by path, never the shared tree).** Decoy: a root carrying `WORK-QUEUE.md` *and* `.git`, with a nested `vendor/drop` that carries a marker only — `find_tree_root(vendor/drop/n/x/m.py)` resolved **`…/real/vendor/drop`**, the decoy, not the repo. Lone marker: a directory with only `WORK-QUEUE.md` was returned as a root rather than refused. Seed: `portable_seed` over a root whose `inbox/` is empty returned `…/root2/inbox/seed-2026-09-26.json`, **`exists=False`**, no exception. Guard: replicating HEAD's own `test_import_no_io` over four spellings, `open()` was caught (1 call) and `Path.read_text()`, `os.open()` and `io.open()` each **survived with 0 recorded calls** — each reached the filesystem and raised `FileNotFoundError` on the probe's `m.py`, which is the survival, not a catch. So reply 125's three findings reproduce at HEAD on this host.
**(2) The repair — `wizard-work/beam/tree_root.py` (+49/−6, blob `7d78e0d6…`, sha256 `95a1f08adcd8923ee9ed95506b4dc9bef8577eae0ba877f92c3fb5f2658ac26a`).** (a) **Bounded root:** a candidate directory counts only when the marker *and* a `.git` entry sit in it — the pair a checkout carries and a decoy does not — and the walk examines at most `max_depth` ancestors (default 64), raising `FileNotFoundError` when none qualifies instead of falling back to whatever marker it last saw. A marker-only tree is still reachable, but only through the explicit `require_git=False` the caller passes and owns. (b) **`portable_seed` raises** `FileNotFoundError` naming the root and the path when the seed is absent, after the existing escape check, so a caller cannot inherit a promise the filesystem does not keep. (c) **The guard names what it models:** six read entry points — `builtins.open`, `io.open`, `os.open`, `Path.open`, `Path.read_text`, `Path.read_bytes` — armed at once, each recording its own label.
**(3) Cells and the gate — `wizard-work/beam/test_tree_root.py` (+165/−26, blob `ac0d6218…`, sha256 `6e62a5a3105b0c3a713ec0755abf0c4e20835d43e265d0f6571a39e02a585c81`).** Six cells became fourteen: decoy-ignored, lone-marker-refused, depth-bounded, `require_git=False` opt-out, seed-missing-raises, seed-escape-raises, plus the import-read cell and two guard cells — one asserting each modelled spelling is caught *under its own label*, one asserting the spelling table equals the model list, so narrowing either reds the file instead of passing it. **Measured:** `bash wizard-work/beam/CHECK` → rc **0**, `79 file(s), 362 unittest case(s), 233 bare case(s), 595 test(s), 0 failure(s), 0 error(s)`, against **587 / 225** before the edit — the delta is exactly these eight cells, with the gate unedited, which is done-mean 3. And the lane gate reaches them: in a full copy of `wizard-work/beam/` with only the `.git` conjunct removed, `bash CHECK` reads **rc 1** and names `test_find_root_ignores_decoy_marker`.
**(4) The instrument — `wizard-work/volt/q90_verify.py` (new, 7 legs) behind `wizard-work/volt/CHECK-q90` (new), `bash wizard-work/volt/CHECK-q90` → `q90: 7 leg(s), 0 fault -- OK`, rc 0, 27 s; `tools/check_lint.py --check` → `SOUND`.** Every leg runs in a temp copy and the shared tree is never mutated: control (rc 0, printed bare count 14 == the 14 `def test_*()` in the file), the full-lane gate leg above, and five mutants — decoy accepted again → rc 1; depth test removed → rc 1; `isfile` raise removed → rc 1; a real module-scope `Path(__file__).read_text()` appended to the module → rc 1 on `test_import_no_io`; one spelling dropped from the test's model table → rc 1 on the set-equality cell. Each mutant fails the cell it targets and nothing else in the run.
**(5) Not claimed:** that `marker + .git` is the only defensible boundary — it is the one this item offered and the one a checkout satisfies, and `require_git=False` is the stated escape; that the depth cap is a security boundary — it bounds a stray-marker walk, and a caller can raise it; that the six-spelling guard is complete — it is a named model, and the forge lane's audit-hook instrument (`wizard-work/forge/import_io_detector.py`, Q89) remains the stronger cross-lane one; that `tree_root.py` has a consumer — `git grep` finds none outside its own two files and `tools/gate_blame.py`'s unrelated `work_tree_root`; that the lane's remaining bare-style files are covered — Q128's runner collects zero-argument `def test_*` only, and a cell taking arguments is still invisible. Files edited for this item: `wizard-work/beam/tree_root.py`, `wizard-work/beam/test_tree_root.py`, `wizard-work/volt/q90_verify.py` (new), `wizard-work/volt/CHECK-q90` (new), `checkins/2026-10-30-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted, and no counterparty repository file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q90.md`.
CHECK: bash wizard-work/volt/CHECK-q90
+27
View File
@@ -63,3 +63,30 @@
harder copy, 16/16, pathlib-aware; its 12-spelling corpus is used as an
independent oracle and agrees 12/12) and `wizard-work/beam/tree_root.py`
(Q90's three holes). Neither harness is run by any gate; that is Q87.
## Q90 — `tree_root.py`: the root had no boundary, the seed was never checked, and the import guard modelled one spelling
- Took Q90 (beam lane by authorship, unclaimed). Reproduced all four faults
this tick against HEAD's own blob (`d0ba06e`), in a temp dir, never the
shared tree: a nested `vendor/drop` carrying only `WORK-QUEUE.md` **won**
the walk over the real root above it; a lone marker was returned as a root;
`portable_seed` returned a path whose `exists=False`; and replicating HEAD's
own guard, `open()` was caught while `Path.read_text()`, `os.open()` and
`io.open()` each survived with 0 recorded calls and reached the filesystem
(`FileNotFoundError` on the probe's `m.py`).
- Repair: marker **and** `.git` in the same directory, a 64-ancestor depth
bound, `FileNotFoundError` instead of a fallback, `require_git=False` as the
documented opt-out; `portable_seed` raises when the seed is absent; the
guard arms six named read entry points and records which one fired.
- Cells 6 → 14 and the lane gate moves: `bash wizard-work/beam/CHECK` rc 0,
`587 test(s)` before, `595` after (bare cases 225 → 233), gate unedited.
In a full copy with only the `.git` conjunct removed, the lane CHECK reads
rc 1 and names `test_find_root_ignores_decoy_marker`.
- Instrument `wizard-work/volt/CHECK-q90` (over `q90_verify.py`): 7 legs,
`0 fault -- OK`, rc 0, 27 s, `check_lint` SOUND — control, the full-lane
gate leg, and five mutants (decoy accepted, depth removed, seed check
removed, a real module-scope `Path.read_text()`, one spelling dropped from
the model table), each red on the cell it targets.
- Left alone: the forge lane's stronger audit-hook detector (Q89) and
`tools/gate_blame.py`'s unrelated `work_tree_root`. No consumer of
`tree_root.py` exists outside its own two files.
+2 -1
View File
@@ -32,7 +32,7 @@ module docstring, flags from the tool's own `--help`, probe rows from
| `selftest_sweep.py` | every instrument proves itself, in one command, and the number it advertises is the number it prints. | `--repo` `--tools-dir` `--timeout` `--json` `--baseline` `--no-baseline` `--write-baseline` `--quiet` `--verbose` `--selftest` | 1 / 0 / 1 | `selftest-sweep` |
| `served_drift_check.py` | does the host actually serve the tree at this revision? | `--root` `--rev` `--base-url` `--prefix` `--exclude` `--max-bytes` `--timeout` `--json` `--quiet` `--selftest` | 1 / 0 / 1 | - |
| `tool_map.py` | the instruments get a map, and the map is generated from the tree. | `--repo` `--rev` `--write` `--check` `--json` `--selftest` | 1 / 0 / 1 | - |
| `verify.py` | the HOME the index gate did not have: one command, nine legs, one exit code, and a row per leg so a partial run cannot look like a clean one. | `--repo` `--queue` `--index` `--checkin-day` `--fast` `--timeout` `--json` `--quiet` `--selftest` | 1 / 0 / 1 | - |
| `verify.py` | the HOME the index gate did not have: one command, ten legs, one exit code, and a row per leg so a partial run cannot look like a clean one. | `--repo` `--queue` `--index` `--checkin-day` `--fast` `--timeout` `--json` `--quiet` `--selftest` | 1 / 0 / 1 | - |
| `write_probe.py` | Measure the WRITE end of every tool in tools/: `--json PATH` at a target the host cannot write. | `--repo` `--only` `--timeout` `--json` `--selftest` `--verbose` | 1 / 0 / 1 | `write-axis` |
## The gate
@@ -52,6 +52,7 @@ dropped by `--fast`.
| `selftest-sweep` | `selftest_sweep.py` | yes | every tool's --selftest and both read-refusal axes |
| `write-axis` | `write_probe.py` | no | a --json target this host cannot write: rc 2, one line naming it, never a traceback -- and the same rule for every caller-path write flag (--write, --save-vocab) |
| `import-resolve` | `import_resolve_check.py` | no | every committed test file binds the module it imports to the repo, not to a host path (static sys.path scan plus --import-probe) |
| `closed-artifacts` | `closed_artifact_guard.py` | no | every repair a closed item landed is still the bytes that landed (sha256 pins for Q109/Q120 and the seven Q126 cells) and its cell still runs green |
A tool that appears in no leg is still graded by `selftest-sweep`, which runs
every tracked tool's `--selftest` and both read-refusal axes -- so the
+40 -13
View File
@@ -4,7 +4,7 @@
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 42 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 109 record blocks in all, 67 of them a later closure or correction for an id already opened. The same **42 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 42 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 110 record blocks in all, 68 of them a later closure or correction for an id already opened. The same **42 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
| # | id | kind | lines | bytes | role | title | first `Status:` |
|---|---|---|---|---|---|---|---|
@@ -213,7 +213,8 @@
| 203 | Q130 | record | 13675-13676 | 5143 | record: BOLD-LEAD | **Q130 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). The third sweep of the day put ten repaired blobs … | -- |
| 204 | Q115 | record | 13677-13688 | 4885 | cont: DONE | **Q115 — DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder-read failure dropped the folder's whole listing and printed no … | -- |
| 205 | Q131 | record | 13689-13700 | 3836 | record: BOLD-LEAD | **Q131 — OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since … | -- |
| 206 | Q89 | record | 13701-13715 | 7604 | cont: DONE | **Q89 — DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list … | -- |
| 206 | Q89 | record | 13701-13716 | 7605 | cont: DONE | **Q89 — DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list … | -- |
| 207 | Q90 | record | 13717-13729 | 5660 | cont: DONE | **Q90 — DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to … | -- |
## Ids with no `## ` heading
@@ -259,7 +260,7 @@ Opened as records inside another section. `line` is the record's own opening lin
| Q124 | 13350 | 13350 | 4 | Q95 | BULLET+BOLD | BOLD-LEAD |
| Q126 | 13545 | 13545 | 8 | Q113 | BULLET+BOLD | BOLD-LEAD |
| Q127 | 13577 | 13577 | 4 | Q110 | BULLET+BOLD | BOLD-LEAD |
| Q128 | 13635 | 13635 | 1 | Q116 | BULLET+BOLD | BOLD-LEAD |
| Q128 | 13635 | 13635 | 2 | Q116 | BULLET+BOLD | BOLD-LEAD |
| Q129 | 13653 | 13653 | 3 | Q128 | BULLET+BOLD | BOLD-LEAD |
| Q130 | 13675 | 13675 | 1 | Q129 | BULLET+BOLD | BOLD-LEAD |
| Q131 | 13689 | 13685 | 2 | Q115 | BULLET+BOLD | BOLD-LEAD |
@@ -1679,17 +1680,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q131 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since \u2026"
},
{
"bytes": 7604,
"end_line": 13715,
"bytes": 7605,
"end_line": 13716,
"id": "Q89",
"kind": "record",
"line": 13701,
"lines": 15,
"lines": 16,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q89 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list \u2026"
},
{
"bytes": 5660,
"end_line": 13729,
"id": "Q90",
"kind": "record",
"line": 13717,
"lines": 13,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q90 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to \u2026"
}
],
"ids_multi_section": {
@@ -4493,17 +4507,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q131 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since \u2026"
},
{
"bytes": 7604,
"end_line": 13715,
"bytes": 7605,
"end_line": 13716,
"id": "Q89",
"kind": "record",
"line": 13701,
"lines": 15,
"lines": 16,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q89 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list \u2026"
},
{
"bytes": 5660,
"end_line": 13729,
"id": "Q90",
"kind": "record",
"line": 13717,
"lines": 13,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q90 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/tree_root.py` had no root boundary, a `portable_seed` that returned a path to \u2026"
}
],
"nested": [
@@ -4854,7 +4881,7 @@ The block below is the parse this document renders, byte for byte. `--check` com
"form": "BOLD-LEAD",
"id": "Q128",
"line": 13635,
"refs": 1,
"refs": 2,
"section": "Q116",
"shape": "BULLET+BOLD"
},
@@ -4891,9 +4918,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
"end_line": 8,
"line": 1
},
"queue_bytes": 2216258,
"queue_lines": 13715,
"queue_sha256": "c68b8208ec31bfab2df4abdcb19d7e2e90405155dbda8e8698b06af249ef1771",
"queue_bytes": 2221919,
"queue_lines": 13729,
"queue_sha256": "50d716fca1ea29df3cea5a186f0fa17d16e261928a5f002af263b3853873fe85",
"sections": [
{
"bytes": 3139,
+1 -1
View File
@@ -29,6 +29,6 @@ queue_index.py 80 cells
reply_check.py 49 cells
selftest_sweep.py 49 cells
served_drift_check.py 25 cells
tool_map.py 20 cells
tool_map.py 23 cells
verify.py 59 cells
write_probe.py 15 cells
1 # selftest_baseline.tsv -- the cell count each tool's selftest prints, pinned.
29 reply_check.py
30 selftest_sweep.py
31 served_drift_check.py
32 tool_map.py
33 verify.py
34 write_probe.py
+145 -17
View File
@@ -7,7 +7,7 @@ WHY THIS EXISTS
queue block, in a reply -- and nothing states the whole set: which tool grades
what, which flags each one actually accepts, which of them the two probe axes
cover, and which leg of the operator's gate a failure shows up in. The closest
thing is `verify.py`'s `LEGS` table, which names six of them.
thing is `verify.py`'s `LEGS` table, which names ten of them.
A hand-written map would be a fourth notation of the defect this suite keeps
grading: prose about the tree, one step from the thing it describes, with no
@@ -106,7 +106,7 @@ USAGE
python3 tool_map.py --repo . --check docs/TOOLS.md # grade drift, rc 1 on drift
python3 tool_map.py --repo . --rev WORKTREE --check docs/TOOLS.md # ...as it will be committed
python3 tool_map.py --repo . --json out.json
python3 tool_map.py --selftest # 20 cells
python3 tool_map.py --selftest # 23 cells
"""
from __future__ import annotations
@@ -171,8 +171,51 @@ def tracked_tools(repo, rev):
return sorted(names)
_UNREADABLE = object()
def partial_literal(node):
"""`ast.literal_eval`, but a container survives a field that is not literal.
`literal_eval` refuses the WHOLE table when one value is a call, and the
caller used to drop that table silently. `verify.py`'s `LEGS` gained
`"path": os.path.join(...)` and every gate leg vanished from the map while
`--check` still read rc 0 against a document rendered before the field
existed. A dict here keeps every key whose value parses, a tuple or list
keeps every element that survives, and `_UNREADABLE` is returned only when
the node itself is not a container of literals -- the caller decides
whether that is fatal, and `collect()` says so in the document.
"""
try:
return ast.literal_eval(node)
except (ValueError, SyntaxError):
pass
if isinstance(node, (ast.Tuple, ast.List)):
out = [v for v in (partial_literal(e) for e in node.elts)
if v is not _UNREADABLE]
if node.elts and not out:
return _UNREADABLE # nothing survived: not an empty container
return out
if isinstance(node, ast.Dict):
out = {}
for k, v in zip(node.keys, node.values):
if k is None: # {**mapping}
continue
kv, vv = partial_literal(k), partial_literal(v)
if kv is _UNREADABLE or vv is _UNREADABLE:
continue
out[kv] = vv
return out
return _UNREADABLE
def literal_assignments(repo, rev, path, names):
"""{name: value} for the module-level assignments whose value is a literal."""
"""{name: value} for the module-level assignments whose value is a literal.
A value that is a container of literals is read even when one field of it
is not (see `partial_literal`); a value that cannot be read at all is
absent from the result, and `collect()` names that absence in the render.
"""
src = read_blob(repo, rev, path)
if src is None:
return {}
@@ -187,10 +230,9 @@ def literal_assignments(repo, rev, path, names):
tgt = node.targets[0]
if not isinstance(tgt, ast.Name) or tgt.id not in names:
continue
try:
found[tgt.id] = ast.literal_eval(node.value)
except (ValueError, SyntaxError):
continue
val = partial_literal(node.value)
if val is not _UNREADABLE:
found[tgt.id] = val
return found
@@ -284,19 +326,25 @@ def flags_of(repo, rev, name):
def _count_by_tool(entries):
"""Count probes per tool name (first element of each entry tuple).
"""Count probes per tool name: ({tool: rows}, rows that were not readable).
All three probe tables share the same shape for tallying: each entry's
first element is the tool name, and the rest is payload the tally ignores.
READ_PROBES / NOT_UTF8_PROBES entries are (tool, argv);
MALFORMED_PROBES entries are (tool, label, ...). This helper replaces the
three identical loops that previously counted each table by hand.
MALFORMED_PROBES entries are (tool, label, ...). A row whose first element
is not a string -- an entry built from a name the AST could not resolve --
is counted as dropped instead of raising out of the renderer, and
`collect()` names the drop in the document so a short count is never read
as a table that has no more rows.
"""
counts = {}
counts, dropped = {}, 0
for entry in entries:
name = entry[0]
name = entry[0] if isinstance(entry, (list, tuple)) and entry else None
if not isinstance(name, str):
dropped += 1
continue
counts[name] = counts.get(name, 0) + 1
return counts
return counts, dropped
def collect(repo, rev):
@@ -307,10 +355,27 @@ def collect(repo, rev):
sweep = literal_assignments(repo, rev, "tools/selftest_sweep.py",
{"READ_PROBES", "NOT_UTF8_PROBES",
"MALFORMED_PROBES"})
read_n = _count_by_tool(sweep.get("READ_PROBES", ()))
utf8_n = _count_by_tool(sweep.get("NOT_UTF8_PROBES", ()))
mal_n = _count_by_tool(sweep.get("MALFORMED_PROBES", ()))
read_n, read_d = _count_by_tool(sweep.get("READ_PROBES", ()))
utf8_n, utf8_d = _count_by_tool(sweep.get("NOT_UTF8_PROBES", ()))
mal_n, mal_d = _count_by_tool(sweep.get("MALFORMED_PROBES", ()))
probes_dropped = read_d + utf8_d + mal_d
probes_note = ""
if probes_dropped:
probes_note = ("%d probe row(s) in `tools/selftest_sweep.py` could not be "
"read as data -- the probe counts above are lower than the "
"tables that carry them" % probes_dropped)
leg_src = read_blob(repo, rev, "tools/verify.py") or ""
legs = literal_assignments(repo, rev, "tools/verify.py", {"LEGS"}).get("LEGS", ())
legs_note = ""
if not legs and re.search(r"^LEGS\s*=", leg_src, re.M):
# An empty column and an unread table render identically, so say which
# one this is in the document itself. A `LEGS` that is a call, an
# import, or a comprehension cannot be read as data; dropping it
# silently is how the gate column went blank while `--check` stayed
# green against a document rendered before the table changed.
legs_note = ("the `LEGS` table in `tools/verify.py` could not be read as "
"data -- the gate column above is empty because the map "
"failed to read it, not because no tool has a leg")
leg_of = {}
for leg in legs:
if isinstance(leg, dict) and "tool" in leg and "key" in leg:
@@ -330,7 +395,8 @@ def collect(repo, rev):
"malformed": mal_n.get(name, 0),
"legs": leg_of.get(name, []),
})
return {"rev": rev, "rows": rows,
return {"rev": rev, "rows": rows, "legs_note": legs_note,
"probes_note": probes_note,
"legs": [{"key": l.get("key"), "tool": l.get("tool"),
"slow": bool(l.get("slow")), "what": l.get("what", "")}
for l in legs if isinstance(l, dict)]}
@@ -367,6 +433,8 @@ def render(data):
if (r["read"] or r["not_utf8"] or r["malformed"]) else "-")
gate = ", ".join(f"`{k}`" for k in r["legs"]) or "-"
lines.append(f"| `{r['tool']}` | {purpose} | {flags} | {probes} | {gate} |")
if data.get("probes_note"):
lines += ["", "*" + data["probes_note"] + "*"]
lines += [
"",
"## The gate",
@@ -381,6 +449,8 @@ def render(data):
for leg in legs:
lines.append(f"| `{leg['key']}` | `{leg['tool']}` | "
f"{'yes' if leg['slow'] else 'no'} | {leg['what']} |")
if data.get("legs_note"):
lines += ["", "*" + data["legs_note"] + "*"]
lines += [
"",
"A tool that appears in no leg is still graded by `selftest-sweep`, which runs",
@@ -528,6 +598,64 @@ def selftest(repo):
cell("WORKTREE outside a git work tree: rc 2, not a crash",
rc_wt_norepo == 2, f"rc {rc_wt_norepo}")
# -- a leg field that is not a literal must not erase the leg --------
# Q130 gave the closed-artifacts leg `"path": os.path.join(...)`. That
# one call made `ast.literal_eval` refuse the whole LEGS table, so every
# leg vanished from the map while `--check` stayed green against a
# document rendered before the field existed. Both directions are
# pinned here: a non-literal FIELD keeps its leg, and a LEGS that is not
# data at all is named in the rendered document instead of rendering an
# empty column that reads as "no tool has a leg".
vpath = os.path.join(fixture, "tools", "verify.py")
with open(vpath, "w") as fh:
fh.write('import os\n'
'LEGS = ({"key": "fake", "tool": "fake_tool.py",\n'
' "slow": False,\n'
' "path": os.path.join("tools", "fake_tool.py"),\n'
' "what": "grade the fake contract"},)\n')
commit("non-literal leg field")
d2 = collect(fixture, "HEAD")
by2 = {r["tool"]: r for r in d2["rows"]}
cell("collect: a leg field that is not a literal does not erase the leg",
by2["fake_tool.py"]["legs"] == ["fake"]
and [l["key"] for l in d2["legs"]] == ["fake"]
and d2["legs_note"] == "",
str((by2["fake_tool.py"]["legs"], d2["legs"], d2["legs_note"])))
with open(vpath, "w") as fh:
fh.write('LEGS = build_legs()\n')
commit("unreadable LEGS")
d3 = collect(fixture, "HEAD")
txt3 = render(d3)
cell("render: an unreadable LEGS table is named, not silently empty",
d3["legs"] == [] and d3["legs_note"] != ""
and "could not be read as data" in txt3
and {r["tool"]: r["legs"] for r in d3["rows"]}["fake_tool.py"] == [],
str((d3["legs"], d3["legs_note"])))
# A probe row whose element is not a literal must not become an empty
# element the counter then subscripts: `_count_by_tool` reads entry[0],
# so a dropped name raised IndexError out of the renderer.
with open(os.path.join(fixture, "tools", "verify.py"), "w") as fh:
fh.write('LEGS = ({"key": "fake", "tool": "fake_tool.py",\n'
' "slow": False, "what": "grade the fake contract"},)\n')
with open(os.path.join(fixture, "tools", "selftest_sweep.py"), "w") as fh:
fh.write('READ_PROBES = ((TOOL_NAME, ["--repo", "{nope}"]),)\n'
'NOT_UTF8_PROBES = ()\nMALFORMED_PROBES = ()\n')
commit("non-literal probe element")
try:
d4 = collect(fixture, "HEAD")
crash = ""
read_n4 = {r["tool"]: r["read"] for r in d4["rows"]}["fake_tool.py"]
note4 = d4["probes_note"]
txt4 = render(d4)
except Exception as exc: # noqa: BLE001 - graded
crash, read_n4, note4, txt4 = "%s: %s" % (type(exc).__name__, exc), None, "", ""
cell("collect: a probe row that is not a literal is dropped, not indexed",
crash == "" and read_n4 == 0 and "could not be read as data" in note4
and note4 in txt4,
crash or f"read={read_n4} note={note4!r}")
# -- the currency cell: the reason this document stays true ----------
here = os.path.join(repo, "docs", "TOOLS.md")
try:
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -e
cd "$(dirname "$(readlink -f "$0")")"
python3 test_damerau_levenshtein.py
+49
View File
@@ -0,0 +1,49 @@
"""Damerau-Levenshtein edit distance (true DL, not OSA)."""
from __future__ import annotations
def damerau_levenshtein(a: str, b: str) -> int:
la, lb = len(a), len(b)
if la == 0:
return lb
if lb == 0:
return la
last = {}
INF = la + lb
d = [[0] * (lb + 2) for _ in range(la + 2)]
d[0][0] = INF
for i in range(1, la + 2):
d[i][0] = INF
d[i][1] = i - 1
for j in range(1, lb + 2):
d[0][j] = INF
d[1][j] = j - 1
for i in range(1, la + 1):
db = 0
for j in range(1, lb + 1):
i1 = last.get(b[j - 1], 0)
j1 = db
cost = 1
if a[i - 1] == b[j - 1]:
cost = 0
db = j
d[i + 1][j + 1] = min(
d[i][j] + cost,
d[i + 1][j] + 1,
d[i][j + 1] + 1,
)
if i1 > 0 and j1 > 0:
d[i + 1][j + 1] = min(
d[i + 1][j + 1],
d[i1][j1] + (i - i1 - 1) + 1 + (j - j1 - 1),
)
last[a[i - 1]] = i
return d[la + 1][lb + 1]
if __name__ == "__main__":
import sys
print(damerau_levenshtein(sys.argv[1], sys.argv[2]))
@@ -0,0 +1,38 @@
import sys
sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")
from damerau_levenshtein import damerau_levenshtein as dl
# known values
assert dl("", "") == 0
assert dl("a", "") == 1
assert dl("", "a") == 1
assert dl("ca", "abc") == 2 # transpose? ca->ac is 1 transposition +?
# classic: "CA" -> "ABC": distance?
assert dl("abcd", "abdc") == 1, dl("abcd","abdc") # transposition dc
assert dl("ab", "ba") == 1
assert dl("abc", "acb") == 1 # transpose b/c
# true DL allows multiple ops on same substring: "CA" -> "ABC" = 2? OSA would be 2 too maybe
print("basic ok")
# cross-check vs Levenshtein (stdlib difflib not exact). Use simple lev.
def lev(a,b):
if not a: return len(b)
if not b: return len(a)
prev=list(range(len(b)+1))
for i,ca in enumerate(a,1):
cur=[i]
for j,cb in enumerate(b,1):
cur.append(min(prev[j]+1, cur[-1]+1, prev[j-1]+(ca!=cb)))
prev=cur
return prev[-1]
import random
random.seed(42)
for _ in range(2000):
n=random.randint(0,8)
m=random.randint(0,8)
a=''.join(random.choice('ab') for _ in range(n))
b=''.join(random.choice('ab') for _ in range(m))
# DL <= lev always (more ops allowed)
assert dl(a,b) <= lev(a,b), (a,b,dl(a,b),lev(a,b))
print("monotonic ok")
+165 -26
View File
@@ -1,3 +1,17 @@
"""Cells for tree_root.py -- root boundary, seed existence, import-time reads.
Q90, 2026-10-30. Three of these cells exist because the module could not
fail that way before: a stray ancestor marker was accepted as a root, a
missing seed was returned as a path, and the import-time guard patched
``builtins.open`` only, so ``Path(...).read_text()``, ``os.open`` and
``io.open`` all read at import while it printed a clean verdict.
The guard below models six read entry points by name and arms all six. It is
self-checking in both directions: a clean module must record zero calls, and
each modelled spelling executed at module scope must be recorded under its
own label -- so deleting an arm, or narrowing the model, reds this file
instead of silently passing it.
"""
import os
import sys
import tempfile
@@ -6,12 +20,86 @@ from beam_assert import fail, raises
sys.path.insert(0, os.path.dirname(__file__))
from tree_root import find_tree_root, assert_inside_tree, portable_seed
_HERE = os.path.dirname(os.path.abspath(__file__))
# The read entry points this lane models. Asserted against _SPELLINGS so the
# model cannot be narrowed without a red cell.
_READ_MODEL = [
"builtins.open",
"io.open",
"os.open",
"Path.open",
"Path.read_text",
"Path.read_bytes",
]
# One import-time statement per modelled entry point. Each must be recorded
# under exactly its own label.
_SPELLINGS = {
"builtins.open": "open(__file__)\n",
"io.open": "import io\nio.open(__file__)\n",
"os.open": "import os\nos.open(__file__, os.O_RDONLY)\n",
"Path.open": "from pathlib import Path\nPath(__file__).open()\n",
"Path.read_text": "from pathlib import Path\nPath(__file__).read_text()\n",
"Path.read_bytes": "from pathlib import Path\nPath(__file__).read_bytes()\n",
}
def _guarded_import(src, name="guarded_module.py"):
"""Exec ``src`` as a module with every modelled read entry point armed.
Returns the labels of the reads attempted during the exec. The spy raises
RuntimeError with a marker string instead of touching the disk, so an
attempted read is recorded and stopped rather than performed.
"""
import builtins
import io
from pathlib import Path
calls = []
armed = []
def arm(obj, attr, label):
orig = getattr(obj, attr)
armed.append((obj, attr, orig))
def spy(*a, **k):
calls.append(label)
raise RuntimeError("import-time I/O detected: %s" % label)
setattr(obj, attr, spy)
arm(builtins, "open", "builtins.open")
arm(io, "open", "io.open")
arm(os, "open", "os.open")
for attr in ("open", "read_text", "read_bytes"):
arm(Path, attr, "Path." + attr)
try:
try:
exec(compile(src, name, "exec"),
{"__name__": "guarded_" + os.path.basename(name),
"__file__": name})
except RuntimeError as e:
if "import-time I/O detected" not in str(e):
raise
finally:
for obj, attr, orig in reversed(armed):
setattr(obj, attr, orig)
return calls
def _tree(marker=True, git=True):
"""A temp root with the marker and, optionally, a .git entry."""
d = tempfile.mkdtemp()
if marker:
open(os.path.join(d, "WORK-QUEUE.md"), "w").write("x")
if git:
os.makedirs(os.path.join(d, ".git"))
return d
def test_find_root_walks_up():
d = tempfile.mkdtemp()
root = os.path.join(d, "repo")
os.makedirs(os.path.join(root, "inbox"))
open(os.path.join(root, "WORK-QUEUE.md"), "w").write("x")
root = _tree()
deep = os.path.join(root, "a", "b", "c")
os.makedirs(deep)
f = os.path.join(deep, "m.py")
@@ -25,11 +113,46 @@ def test_find_root_missing_raises():
find_tree_root(d)
def test_find_root_ignores_decoy_marker():
"""A marker without .git is not a root, even when it is nearer than one."""
root = _tree()
decoy = os.path.join(root, "vendor", "drop")
os.makedirs(os.path.join(decoy, "n", "x"))
open(os.path.join(decoy, "WORK-QUEUE.md"), "w").write("x")
start = os.path.join(decoy, "n", "x", "m.py")
open(start, "w").write("pass")
assert find_tree_root(start) == os.path.realpath(root)
def test_find_root_rejects_lone_marker():
"""With no real root above it, a lone marker raises instead of winning."""
d = _tree(marker=True, git=False)
with raises(FileNotFoundError):
find_tree_root(d)
def test_find_root_respects_depth_bound():
root = _tree()
deep = root
for i in range(6):
deep = os.path.join(deep, "d%d" % i)
os.makedirs(deep)
assert find_tree_root(deep) == os.path.realpath(root)
with raises(FileNotFoundError):
find_tree_root(deep, max_depth=3)
def test_find_root_require_git_opt_out():
"""A marker-only tree is reachable, but only when the caller asks for it."""
d = _tree(marker=True, git=False)
assert find_tree_root(d, require_git=False) == os.path.realpath(d)
def test_assert_inside_ok():
root = tempfile.mkdtemp()
p = os.path.join(root, "inbox", "seed.json")
os.makedirs(os.path.dirname(p), exist_ok=True)
open(p, "w").write("{}")
open(p, "w").write("{}\n")
assert assert_inside_tree(p, root) == os.path.realpath(p)
@@ -41,28 +164,44 @@ def test_assert_inside_escape_raises():
def test_portable_seed_inside():
root = tempfile.mkdtemp()
os.makedirs(os.path.join(root, "inbox"))
rel = os.path.join("inbox", "seed-2026-09-26.json")
os.makedirs(os.path.dirname(os.path.join(root, rel)))
open(os.path.join(root, rel), "w").write("{}\n")
p = portable_seed(root)
assert p.startswith(os.path.realpath(root) + os.sep)
assert p == os.path.realpath(os.path.join(root, rel))
def test_portable_seed_missing_raises():
"""The seed that is not there must raise, not be returned as a path."""
root = tempfile.mkdtemp()
os.makedirs(os.path.join(root, "inbox"))
with raises(FileNotFoundError):
portable_seed(root)
def test_portable_seed_escape_raises():
root = tempfile.mkdtemp()
with raises(ValueError):
portable_seed(root, rel="../outside.json")
def test_import_no_io():
src = open(os.path.join(os.path.dirname(__file__), "tree_root.py")).read()
import builtins
orig = builtins.open
calls = []
def guard(*a, **k):
calls.append(a)
raise RuntimeError("import-time I/O detected")
ns = {}
builtins.open = guard
try:
try:
exec(compile(src, "tree_root.py", "exec"), ns)
assert not calls, "import-time open calls: %r" % calls
except RuntimeError as e:
if "import-time I/O" in str(e):
fail("module performed import-time I/O")
raise
finally:
builtins.open = orig
src = open(os.path.join(_HERE, "tree_root.py")).read()
calls = _guarded_import(src, "tree_root.py")
assert calls == [], "import-time reads in tree_root.py: %r" % (calls,)
def test_import_io_model_catches_each_spelling():
"""Every modelled spelling must be caught, under its own label."""
misses = []
for label, src in _SPELLINGS.items():
calls = _guarded_import(src, "spelling.py")
if calls != [label]:
misses.append("%s -> %r" % (label, calls))
assert not misses, "read spelling(s) survived the guard: %s" % ("; ".join(misses))
def test_import_io_model_is_exactly_the_listed_set():
assert sorted(_SPELLINGS) == sorted(_READ_MODEL), (
"guard model drifted: %r vs %r" % (sorted(_SPELLINGS), sorted(_READ_MODEL))
)
+49 -6
View File
@@ -1,19 +1,51 @@
"""tree_root.py - derive a repo root from __file__ instead of a host-absolute path."""
"""tree_root.py - derive a repo root from __file__ instead of a host-absolute path.
Boundary rule (Q90, 2026-10-30). A marker file alone is not a repository: a
stray ``WORK-QUEUE.md`` in a parent directory -- a download, an unpacked
tarball, a second checkout, a vendor drop -- satisfies the marker test and
wins the walk, so the root a caller resolves can be a directory this tree
does not own, and every path derived from it lands in the wrong tree. A
candidate therefore counts only when the marker *and* a ``.git`` entry sit in
the same directory: the pair a real checkout carries and a decoy does not.
The walk is also bounded -- 64 ancestors by default, or ``max_depth`` -- so a
pathological tree cannot walk to ``/`` and adopt a foreign marker there.
Callers that genuinely grade a marker-only tree (a fixture, an export with no
VCS) pass ``require_git=False``; that is an explicit weakening, not the
default, and the caller owns it.
"""
from __future__ import annotations
import os
DEFAULT_MARKER = "WORK-QUEUE.md"
DEFAULT_MAX_DEPTH = 64
def find_tree_root(start, marker="WORK-QUEUE.md"):
def find_tree_root(start, marker=DEFAULT_MARKER, require_git=True,
max_depth=DEFAULT_MAX_DEPTH):
"""Return the repository root at or above ``start``.
A candidate directory must contain ``marker`` and, when ``require_git`` is
true (the default), a ``.git`` entry as well. At most ``max_depth``
ancestors are examined. Raises ``FileNotFoundError`` when no candidate
qualifies -- never falls back to the topmost directory it happened to see.
"""
cur = os.path.realpath(start)
if os.path.isfile(cur):
cur = os.path.dirname(cur)
want = "marker %r%s" % (marker, " and .git" if require_git else "")
depth = 0
while True:
if os.path.isfile(os.path.join(cur, marker)):
return cur
if not require_git or os.path.exists(os.path.join(cur, ".git")):
return cur
parent = os.path.dirname(cur)
if parent == cur:
raise FileNotFoundError("marker %r not found in any ancestor of %s" % (marker, start))
depth += 1
if parent == cur or depth > max_depth:
raise FileNotFoundError(
"%s not found in %d ancestor(s) of %s" % (want, max_depth, start)
)
cur = parent
@@ -26,7 +58,18 @@ def assert_inside_tree(path, root):
def portable_seed(root, rel="inbox/seed-2026-09-26.json"):
return assert_inside_tree(os.path.join(root, rel), root)
"""Return the absolute seed path inside ``root``, or raise.
Raises ``FileNotFoundError`` when the seed does not exist: returning a
path to a file that is not there is the failure this function exists to
prevent, and a caller that trusts the return value would read it later,
far from the root that never had it. Raises ``ValueError`` (via
``assert_inside_tree``) when ``rel`` escapes ``root``.
"""
p = assert_inside_tree(os.path.join(root, rel), root)
if not os.path.isfile(p):
raise FileNotFoundError("seed not found inside tree root %s: %s" % (root, p))
return p
if __name__ == "__main__":
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
# Q132 CHECK -- tool_map.py must read a LEGS table whose field is not a
# literal, must name a table it cannot read, and must stay the render of
# this tree.
#
# rc 0 = every leg as expected; 1 = a graded fault; 2 = could not grade.
# The work is done by wizard-work/volt/q132_verify.py beside this file;
# every fixture leg runs in a throwaway git repo, so the graded tree is
# only ever read. Run it from any checkout.
set -u
REPO="$(cd "$(dirname "$0")/../.." && pwd)" || exit 2
if ! cd "$REPO"; then
echo "q132: could not grade -- cannot cd to $REPO"
exit 2
fi
if ! command -v python3 >/dev/null 2>&1; then
echo "q132: could not grade -- no python3 on PATH"
exit 2
fi
exec python3 wizard-work/volt/q132_verify.py
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
# Q90 CHECK -- tree_root.py's root boundary, its seed existence, and the
# import-time-read model, each graded by a mutant that must red it.
#
# rc 0 = every leg as expected; 1 = a graded fault; 2 = could not grade.
# The work is done by wizard-work/volt/q90_verify.py beside this file; every
# leg runs in a temp copy of the lane, so the tree this instrument ships in is
# never mutated. Run it from any checkout.
set -u
REPO="$(cd "$(dirname "$0")/../.." && pwd)" || exit 2
if ! cd "$REPO"; then
echo "q90: could not grade -- cannot cd to $REPO"
exit 2
fi
if ! command -v python3 >/dev/null 2>&1; then
echo "q90: could not grade -- no python3 on PATH"
exit 2
fi
exec python3 wizard-work/volt/q90_verify.py
+195
View File
@@ -0,0 +1,195 @@
#!/usr/bin/env python3
"""Q132 instrument: `tool_map` must read a `LEGS` table whose field is not a literal.
The fault, measured at `d0ba06e` before any edit: `tools/verify.py`'s `LEGS`
gained `"path": os.path.join("wizard-work", "spark",
"closed_artifact_guard.py")` when Q130 added the `closed-artifacts` leg. That
one call made `ast.literal_eval` refuse the whole table, so
`literal_assignments()` returned nothing for `LEGS`, every row's gate column
rendered `-`, and the render no longer matched `docs/TOOLS.md`. The gate-leg
column did not go red; it went BLANK.
Legs
1. control, fixed parser a fixture whose `LEGS` carries a call-valued
field renders the leg by name
2. mutant, pre-fix parser the same fixture and document rendered by
`ast.literal_eval`-only: the leg is gone, the
document drifts, and `--check` reads rc 1
3. unreadable table a `LEGS` that is not data at all is named in
the rendered document, not rendered as `-`
4. real tree is current `--check docs/TOOLS.md` rc 0 at HEAD and at
WORKTREE, and the render names 10 gate legs
5. seller's own cells `--selftest` rc 0 and 22/22 cells printed
Every fixture leg runs in its own throwaway git repo under a temp directory;
the graded tree is only read. Exit codes follow the repo contract: 0 every
leg as expected, 1 a leg disagreed, 2 could not grade.
"""
import os
import re
import shutil
import subprocess
import sys
import tempfile
HERE = os.path.dirname(os.path.abspath(__file__))
REPO = os.path.dirname(os.path.dirname(HERE))
TOOL_MAP = os.path.join(REPO, "tools", "tool_map.py")
FAIL = []
COUNT = [0]
MUT_OLD = """ val = partial_literal(node.value)
if val is not _UNREADABLE:
found[tgt.id] = val
"""
MUT_NEW = """ try:
found[tgt.id] = ast.literal_eval(node.value)
except (ValueError, SyntaxError):
continue
"""
FAKE_TOOL = ('#!/usr/bin/env python3\n'
'"""fake_tool.py -- grades the fake contract, which nothing measured."""\n'
'import sys\nprint("ran")\n')
SWEEP = ('READ_PROBES = ()\nNOT_UTF8_PROBES = ()\nMALFORMED_PROBES = ()\n')
VERIFY_LITERAL_FIELD = (
'import os\n'
'LEGS = ({"key": "fake", "tool": "fake_tool.py", "slow": False,\n'
' "path": os.path.join("tools", "fake_tool.py"),\n'
' "what": "grade the fake contract"},)\n')
def leg(name, ok, detail=""):
COUNT[0] += 1
print("%s %-26s %s" % ("PASS" if ok else "FAIL", name, detail))
if not ok:
FAIL.append(name)
def run(argv, cwd=None):
p = subprocess.run(argv, cwd=cwd, capture_output=True, text=True)
return p.returncode, p.stdout, p.stderr
def git(repo, *args):
env = dict(os.environ, GIT_AUTHOR_NAME="t", GIT_AUTHOR_EMAIL="t@e",
GIT_COMMITTER_NAME="t", GIT_COMMITTER_EMAIL="t@e")
return run(["git", "-C", repo] + list(args))[0], env
def build_fixture(root, verify_src=VERIFY_LITERAL_FIELD):
"""A three-tool git work tree whose LEGS table is the grading surface."""
os.makedirs(os.path.join(root, "tools"))
os.makedirs(os.path.join(root, "docs"))
for name, body in (("fake_tool.py", FAKE_TOOL),
("selftest_sweep.py", SWEEP),
("verify.py", verify_src)):
with open(os.path.join(root, "tools", name), "w") as fh:
fh.write(body)
env = dict(os.environ, GIT_AUTHOR_NAME="t", GIT_AUTHOR_EMAIL="t@e",
GIT_COMMITTER_NAME="t", GIT_COMMITTER_EMAIL="t@e")
run(["git", "init", "-q", root], cwd=None)
run(["git", "-C", root, "add", "-A"])
run(["git", "-C", root, "-c", "user.name=t", "-c", "user.email=t@e",
"commit", "-qm", "fixture"])
return env
def gate_cell(doc_text, tool="fake_tool.py"):
"""The last table cell of the row for `tool`, or None."""
for line in doc_text.splitlines():
if line.startswith("| `%s`" % tool):
return line.rstrip().rstrip("|").split("|")[-1].strip()
return None
def main():
if not os.path.isfile(TOOL_MAP):
print("q132: could not grade -- no tools/tool_map.py at %s" % REPO)
return 2
td = tempfile.mkdtemp(prefix="q132.")
fixed = os.path.join(td, "tool_map_fixed.py")
shutil.copyfile(TOOL_MAP, fixed)
mutant = os.path.join(td, "tool_map_mutant.py")
with open(TOOL_MAP, encoding="utf-8") as fh:
src = fh.read()
if src.count(MUT_OLD) != 1:
print("q132: could not grade -- the mutant block appears %d time(s)"
% src.count(MUT_OLD))
return 2
with open(mutant, "w", encoding="utf-8") as fh:
fh.write(src.replace(MUT_OLD, MUT_NEW))
# -- 1. control: the fixed parser reads a call-valued field --------------
fx = os.path.join(td, "fx")
build_fixture(fx)
doc = os.path.join(fx, "docs", "TOOLS.md")
rc, out, err = run([sys.executable, fixed, "--repo", fx, "--write", doc])
with open(doc, encoding="utf-8") as fh:
text = fh.read()
cell = gate_cell(text)
leg("control-render", rc == 0 and cell == "`fake`",
"rc=%d gate=%r" % (rc, cell))
# -- 2. mutant: the pre-fix parser loses the leg and drifts -------------
mdoc = os.path.join(fx, "docs", "MUTANT.md")
rc_m, out_m, err_m = run([sys.executable, mutant, "--repo", fx,
"--write", mdoc])
with open(mdoc, encoding="utf-8") as fh:
mtext = fh.read()
mcell = gate_cell(mtext)
rc_c, out_c, err_c = run([sys.executable, mutant, "--repo", fx,
"--check", doc])
leg("mutant-loses-leg", rc_m == 0 and mcell == "-",
"rc=%d gate=%r" % (rc_m, mcell))
leg("mutant-doc-drifts", rc_c == 1 and "DRIFT" in out_c,
"rc=%d first=%r" % (rc_c, (out_c or err_c).splitlines()[:1]))
# -- 3. a LEGS that is not data is named, not rendered as `-` -----------
fx2 = os.path.join(td, "fx2")
build_fixture(fx2, verify_src="LEGS = build_legs()\n")
doc2 = os.path.join(fx2, "docs", "TOOLS.md")
rc2, out2, err2 = run([sys.executable, fixed, "--repo", fx2,
"--write", doc2])
with open(doc2, encoding="utf-8") as fh:
text2 = fh.read()
named = "could not be read as data" in text2
rc2c, out2c, _ = run([sys.executable, fixed, "--repo", fx2,
"--check", doc2])
leg("unreadable-named", rc2 == 0 and named and gate_cell(text2) == "-",
"rc=%d named=%s gate=%r" % (rc2, named, gate_cell(text2)))
leg("unreadable-still-checks", rc2c == 0,
"rc=%d (the document states the failure, so it is current)" % rc2c)
# -- 4. the real tree: the column is populated and the map is current ---
rdoc = os.path.join(td, "TOOLS.real.md")
rc3, out3, err3 = run([sys.executable, TOOL_MAP, "--repo", REPO,
"--rev", "WORKTREE", "--write", rdoc])
told = re.search(r"(\d+) gate legs", out3 or "")
legs_n = int(told.group(1)) if told else -1
rc_h, out_h, err_h = run([sys.executable, TOOL_MAP, "--repo", REPO,
"--check", os.path.join(REPO, "docs", "TOOLS.md")])
rc_w, out_w, err_w = run([sys.executable, TOOL_MAP, "--repo", REPO,
"--rev", "WORKTREE", "--check",
os.path.join(REPO, "docs", "TOOLS.md")])
leg("real-tree-legs", rc3 == 0 and legs_n == 10, "rc=%d legs=%d" % (rc3, legs_n))
leg("real-tree-current", rc_h == 0 and rc_w == 0,
"HEAD rc=%d WORKTREE rc=%d %s" % (rc_h, rc_w, (out_h or err_h).strip()[:80]))
# -- 5. the tool's own cells -------------------------------------------
rc4, out4, err4 = run([sys.executable, TOOL_MAP, "--repo", REPO, "--selftest"])
m = re.search(r"selftest: (\d+) cells, (\d+) passed, (\d+) failed", out4 or "")
ok4 = bool(m) and m.group(1) == m.group(2) == "23" and m.group(3) == "0"
leg("tool-selftest", rc4 == 0 and ok4,
"rc=%d %s" % (rc4, m.group(0) if m else (err4 or out4).strip()[:90]))
print("q132: %d leg(s), %d fault -- %s"
% (COUNT[0], len(FAIL), "OK" if not FAIL else "FAILED"))
return 0 if not FAIL else 1
if __name__ == "__main__":
sys.exit(main())
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
"""Q90 instrument: grade tree_root.py's four done-means, and prove each cell bites.
The shared tree is never mutated. Every leg runs in a temp copy of the lane
(or of the four files a cell needs), and every mutant is written into a copy.
Legs
1. control small tree -- tree_root.py's 14 cells all pass under
run_lane_tests.py, rc 0, printed count == defs
2. gate reaches cells -- the *lane's own* CHECK, run in a full copy of
wizard-work/beam/ with the decoy fixed module
swapped for the accepting one, exits 1 and names
the decoy cell (Q128's runner collects it, so a
green lane gate cannot hide this file)
3. decoy marker -- mutant: marker alone accepted again -> rc 1
4. depth bound -- mutant: depth test removed -> rc 1
5. seed existence -- mutant: isfile raise removed -> rc 1
6. import-time read -- mutant: a real module-scope Path.read_text()
appended to tree_root.py -> rc 1
7. guard model narrowed -- mutant: one spelling dropped from the test's
model table -> rc 1 on the set-equality cell
Exit codes follow the repo's tools/ contract: 0 every leg as expected,
1 a leg disagreed, 2 could not grade (missing lane files, runner crashed).
"""
import os
import re
import shutil
import subprocess
import sys
import tempfile
HERE = os.path.dirname(os.path.abspath(__file__))
REPO = os.path.dirname(os.path.dirname(HERE))
LANE = os.path.join(REPO, "wizard-work", "beam")
SMALL = ("tree_root.py", "test_tree_root.py", "beam_assert.py", "run_lane_tests.py")
FAILURES = []
LEGS = [0]
def leg(name, ok, detail):
LEGS[0] += 1
status = "PASS" if ok else "FAULT"
print("%s %-24s %s" % (status, name, detail))
if not ok:
FAILURES.append(name)
def run(cwd, argv, timeout=300):
p = subprocess.run(argv, cwd=cwd, capture_output=True, text=True,
timeout=timeout)
return p.returncode, p.stdout + p.stderr
def small_copy(mutate=None):
"""Copy the four files a cell needs; apply mutate(dict path->text)."""
d = tempfile.mkdtemp(prefix="q90-")
for n in SMALL:
shutil.copyfile(os.path.join(LANE, n), os.path.join(d, n))
if mutate:
for name, fn in mutate.items():
path = os.path.join(d, name)
with open(path) as fh:
text = fh.read()
out = fn(text)
if out == text:
raise SystemExit("mutant for %s did not change the file" % (name,))
with open(path, "w") as fh:
fh.write(out)
return d
def lane_copy(mutate=None):
d = tempfile.mkdtemp(prefix="q90-lane-")
dest = os.path.join(d, "beam")
shutil.copytree(LANE, dest, ignore=shutil.ignore_patterns("__pycache__"))
if mutate:
for name, fn in mutate.items():
path = os.path.join(dest, name)
with open(path) as fh:
text = fh.read()
with open(path, "w") as fh:
fh.write(fn(text))
return dest
def sub(text, old, new):
if old not in text:
raise SystemExit("anchor not found: %r" % (old,))
return text.replace(old, new, 1)
def defs(path):
with open(path) as fh:
return re.findall(r"^def (test_\w+)\(\):", fh.read(), re.M)
def main():
for n in SMALL:
if not os.path.isfile(os.path.join(LANE, n)):
raise SystemExit("cannot grade: missing wizard-work/beam/%s" % n)
want = len(defs(os.path.join(LANE, "test_tree_root.py")))
if want < 10:
raise SystemExit("cannot grade: test_tree_root.py exposes %d cells" % want)
# 1. control
d = small_copy()
rc, out = run(d, [sys.executable, "run_lane_tests.py"])
m = re.search(r"(\d+) bare case\(s\)", out)
got = int(m.group(1)) if m else -1
leg("control", rc == 0 and got == want,
"rc=%d bare=%d want=%d" % (rc, got, want))
# 2. the lane's own gate must reach these cells
d = lane_copy({"tree_root.py": lambda t: sub(
t,
'if not require_git or os.path.exists(os.path.join(cur, ".git")):',
"if True:")})
rc, out = run(d, ["bash", "CHECK"])
leg("gate-reaches-cells",
rc == 1 and "test_find_root_ignores_decoy_marker" in out,
"lane CHECK rc=%d decoy cell named=%s"
% (rc, "test_find_root_ignores_decoy_marker" in out))
mutants = [
("decoy-marker", "tree_root.py", lambda t: sub(
t,
'if not require_git or os.path.exists(os.path.join(cur, ".git")):',
"if True:"), "test_find_root_ignores_decoy_marker"),
("depth-bound", "tree_root.py", lambda t: sub(
t, "if parent == cur or depth > max_depth:", "if parent == cur:"),
"test_find_root_respects_depth_bound"),
("seed-existence", "tree_root.py", lambda t: sub(
t, "if not os.path.isfile(p):", "if False:"),
"test_portable_seed_missing_raises"),
("import-read", "tree_root.py", lambda t: (
t + "\nfrom pathlib import Path as _ProbePath\n"
"_ProbePath(__file__).read_text()\n"), "test_import_no_io"),
("model-narrowed", "test_tree_root.py", lambda t: sub(
t, ' "Path.read_text": "from pathlib import Path\\n'
'Path(__file__).read_text()\\n",\n', ""),
"test_import_io_model_is_exactly_the_listed_set"),
]
for name, fname, fn, cell in mutants:
d = small_copy({fname: fn})
rc, out = run(d, [sys.executable, "run_lane_tests.py"])
leg(name, rc == 1 and cell in out,
"rc=%d red=%s" % (rc, cell if cell in out else "not-named"))
if LEGS[0] != 7:
raise SystemExit("cannot grade: ran %d legs, expected 7" % LEGS[0])
if FAILURES:
print("q90: %d leg(s), %d fault -- %s"
% (LEGS[0], len(FAILURES), ", ".join(FAILURES)))
return 1
print("q90: %d leg(s), 0 fault -- OK" % LEGS[0])
return 0
if __name__ == "__main__":
try:
sys.exit(main())
except subprocess.TimeoutExpired:
print("q90: could not grade -- a leg timed out")
sys.exit(2)