stackchain/troupe-standup
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

volt: Q89 closure -- the import-time-I/O detector is a shared instrument now, not three copies of a name list

Browse Source 
The three refactor harnesses (probe, dump, pending) each carried the same
_IO_CALLS name list plus a builtins.open spy. Measured at bb4a85e in a clean
clone: Path(__file__).read_text(), os.open(__file__, O_RDONLY), io.open(__file__)
and with Path(__file__).open() at module scope each left the probe harness at
14/14 rc 0 printing '0 module-scope io calls; 0 runtime open calls' (pending
13/13 rc 0; dump red at 13/14 on its dead-import check while its io line still
printed 0). The name-list walk resolves Path(p).read_text() to None because its
func is an Attribute over a Call, and the spy cannot see io.open/pathlib.

Now: wizard-work/forge/import_io_detector.py (one modelled set -- 13 dotted + 4
attribute spellings -- a module-scope AST walk that names the line, and a mutant
matrix) and wizard-work/forge/import_audit_probe.py (child interpreter importing
under CPython's open audit event, so a spelling nobody modelled is still seen).
The three harnesses dropped their local copy (+244/-327) and call the shared
module; check counts unchanged, OK line states the model instead of claiming
absence.

Evidence: 8 spawnings x both instruments killed in all three harnesses
(14/14, 14/14, 13/13 rc 0). Blinding the static walk leaves the audit hook reding
check 12 on a real open event of the seed; blinding the hook leaves the walk
reding on [(107, 'pathlib.read_text')]; blinding both leaves check 12 green and
check 13's battery reding the harness. _F = open / _F(DEFAULT_PATH) -- a spelling
no list names -- is caught by the hook alone.

Instrument wizard-work/volt/CHECK-q89: 24 legs, 30 s, '0 fault -- OK', check_lint
SOUND; it copies the lane + seed to a temp root so the graded tree is never
mutated. Fourth copy (verify_dump_seed_refactor.py) untouched at 16/16 and used
as an independent oracle: agrees 12/12. Q90's beam guard untouched.
verify.py --fast: 10 legs, 9 pass, 0 fault, 1 skipped, gate OK.
This commit is contained in:
ledger committed 2026-10-30 23:13:20 +00:00
1 parent bb4a85e6be
commit d0ba06e7de
10 files changed
+919 -338
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

WORK-QUEUE.md
+16
View File
@@ -13697,3 +13697,19 @@ CHECK: bash wizard-work/volt/CHECK-q115
**(4) Not claimed:** that the harness was ever green after 2026-10-12 (it was not re-run; the red is measured at HEAD only); that the seven original authorship expectations are the last word (they are the Q15-era table, unchanged except for the format fix — their heads and listings were re-derived from the synthetic worlds and all seven pass); that the harness is wired into any cron or gate (it is still run by hand, which is why a red went five weeks unnoticed — wiring it is a cron decision, not this item's); that `s4`'s digest check proves the module never writes elsewhere (it proves these two paths are untouched). Files edited for this item: `/root/.hermes/scripts/troupe_pipe_monitor_harness.py`, `wizard-work/volt/CHECK-q115` (new), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered). The two `.bak-20261030T2100Z` copies are the pre-repair bytes. Nothing from `inbox/` is quoted.
CHECK: bash wizard-work/volt/CHECK-q115
**Q89 — DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list copied three times, so `Path(...).read_text()`, `os.open(...)`, `io.open(...)` and `Path(...).open()` all read a file at import while the harness printed `0 module-scope io calls; 0 runtime open calls` and exited 0. The modelled set now lives in one shared module with two instruments — an AST walk that names the site and a child interpreter that imports under CPython's `open` audit hook — and each of the eight modelled spellings is a spawning that both instruments must kill.**
**(1) The fault, measured at `bb4a85e` before any edit.** In a clean clone, one statement appended at module scope to each module, harness run with `cwd` = the clone's `wizard-work/forge` (`/root/.hermes/profiles/volt/cache/scratch/q89/q89_repro.py`): probe — `Path(__file__).read_text()`, `os.open(__file__, O_RDONLY)`, `io.open(__file__)` and `with Path(__file__).open()` each read **14/14 checks passed, rc 0**, printing `OK: no import-time I/O (0 module-scope io calls; 0 runtime open calls)`; `codecs.open(__file__)` was the one spelling the `builtins.open` spy caught (`12/14`, `runtime import-time open()`). pending — the same four evasions, **13/13 rc 0**. dump — the same four red at `13/14`, but on its unreferenced-import check (the appended `from pathlib import Path`), **while its io line still printed 0 calls**: no cell in that harness attributed the read either. The old `_IO_CALLS` was byte-identical in all three and its `_call_name` returns `None` for a `func` that is an `ast.Attribute` over an `ast.Call`, which is exactly `Path(p).read_text()`.
**(2) What replaced it — one instrument, two instruments deep.** New `wizard-work/forge/import_io_detector.py` (192 lines, sha256 `9cbf61137dc6…`) holds the modelled set (13 dotted spellings — `open`, `io.open`, `os.open`, `os.read`, `codecs.open`, `json.load`, `json.loads`, `pickle.load`, `pickle.loads`, `csv.reader`, `csv.DictReader`, `urllib.request.urlopen`, `urllib.urlopen` — and 4 attribute spellings whatever the receiver: `read_text`, `read_bytes`, `read`, `open`), the module-scope AST walk that returns `(lineno, spelling)`, the call into the audit probe, and `mutant_matrix()`. New `wizard-work/forge/import_audit_probe.py` (112 lines, sha256 `89df7ab8d487…`) is a child interpreter that installs `sys.addaudithook` on the `open` event and imports the module by path, printing the reads as JSON; it filters the target's own source/bytecode, `.py`/`.pyc`/`.so` opens under `sys.path` or the interpreter prefix, and write-only opens, and its docstring names what it cannot see (`os.read` on an inherited descriptor, `mmap`, `sqlite3.connect`, a C extension's own `PyInit_*`, a subprocess's I/O). The three harnesses lost their local `_IO_CALLS`/`_call_name`/`_calls_at_module_scope` block and now call the shared module (`+244/−327` over the three; blobs probe `65c86cede763…`, dump `ea39b3ddc089…`, pending `f40478c8645c…`). Check counts are unchanged and the OK line states the model instead of claiming absence: `OK: no import-time read (static 0 over 17 modelled spellings; audit 0 open events outside the import machinery)`.
**(3) The mutant battery, both instruments, in the harness.** Check 13 now spawns every modelled spelling at module scope in a copy of the source and requires **both** a static hit and a real `open` audit event. Measured in all three harnesses: `open` static 1 / audit 1, `json.load` 2 / 1, `io.open` 1 / 1, `os.open` 1 / 1, `codecs.open` 1 / 1, `pathlib.read_text` 1 / 1, `pathlib.read_bytes` 1 / 1, `pathlib.open` 1 / 1 → `8/8 import-time-read spellings killed by both instruments`; harness totals probe **14/14**, dump **14/14**, pending **13/13**, rc 0.
**(4) Which instrument is load-bearing — three mutants of the detector itself**, run on a copy of the tree (`/root/.hermes/profiles/volt/cache/scratch/q89/falsify_blinded.py`). With the static sets emptied, check 12 reds on the **audit hook alone**: `1 open event(s) during import: [{'path': '…/inbox/seed-2026-09-26.json', 'mode': 'r', 'flags': 524288}]`. With `audit_reads()` replaced by a zero-report, check 12 reds on the **static walk alone**: `static read-spelling(s) at module scope: [(107, 'pathlib.read_text')]`. With **both** blinded, check 12 prints `OK` — and check 13 still reds the harness, `8/8 spelling mutant(s) survived an instrument`, so a silently emptied detector cannot pass. The class reading: `_F = open` / `_F(DEFAULT_PATH)` resolves to a name no set contains, the static walk reads zero by construction, and the audit hook catches the real open event (`FAIL: 1 open event(s) during import`).
**(5) The instrument, `wizard-work/volt/CHECK-q89` (a 19-line `sh` wrapper over `wizard-work/volt/q89_verify.py`, 272 lines, sha256 `38a2940338e3…`), 24 legs, 30 s.** It copies `wizard-work/forge` and `inbox/seed-2026-09-26.json` into a temp root and never mutates the tree it grades. Measured: `q89: 24 leg line(s), 0 fault -- OK`, rc **0** — 3 clean legs (14/14, 14/14, 13/13), 12 evasion legs (4 spellings × 3 harnesses, every one `rc=1` naming `static read-spelling(s) at module scope: [(107, 'pathlib.read_text')]` and its siblings), 3 negative-control legs (`len()` at module scope stays green), 3 blinded-instrument legs, 1 no-list leg, the fourth copy at `16/16`, and 12/12 agreement with that harness's own `_IO_SPELLINGS` corpus (negative control unflagged). `tools/check_lint.py --check "bash wizard-work/volt/CHECK-q89"` → `SOUND`.
**(6) Not claimed:** that every read at import is now caught — the audit-event layer is the boundary and the boundary is named in the probe's docstring; that the static walk is complete — it is a named set on purpose, and `_F = open` is the measured example of what it misses and the hook covers; that the fourth copy (`wizard-work/forge/verify_dump_seed_refactor.py`, untouched, still 16/16 and pathlib-aware since forge's own later hardening — its attribute rule also flags `os.open`/`io.open` as `pathlib.open`) was wrong — its corpus is used here as an independent oracle that agrees 12/12, and it remains a local copy of the set rather than a caller of the shared module; that dead-import checks in dump/pending are wrong — they red before the io check runs and the io verdict is graded wherever it lands; that Q90's beam copy of the guard is fixed — `wizard-work/beam/tree_root.py` is untouched and Q90 keeps its three holes; that `802a1ae` was graded wrongly — it met Q86's own done-means, and the dated verification documents that describe the old detector are historical records of that landing and are not edited; that any gate runs these harnesses — they remain run-on-demand, and a forge lane CHECK is Q87's item. Files edited this tick: `wizard-work/forge/import_io_detector.py` (new), `wizard-work/forge/import_audit_probe.py` (new), `wizard-work/forge/verify_triage_probe_refactor.py`, `wizard-work/forge/verify_triage_dump_refactor.py`, `wizard-work/forge/verify_triage_dump_pending_refactor.py`, `wizard-work/volt/q89_verify.py` (new), `wizard-work/volt/CHECK-q89` (new), `checkins/2026-10-30-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted and no counterparty file was edited. Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q89.md`.
CHECK: bash wizard-work/volt/CHECK-q89
checkins/2026-10-30-volt.md
+26
View File
@@ -37,3 +37,29 @@
## Ask
- Nothing.
## Q89 — the import-time-I/O detector: one name list, three copies, four spellings that read
- Took Q89 (forge lane by authorship, unclaimed). Measured at `bb4a85e` in a clean clone:
`Path(__file__).read_text()`, `os.open(...)`, `io.open(...)` and
`with Path(...).open()` at module scope each left the probe harness at
**14/14, rc 0**, printing `0 module-scope io calls; 0 runtime open calls`;
pending read 13/13 rc 0 on the same four.
- Replacement: `wizard-work/forge/import_io_detector.py` (one modelled set,
module-scope AST walk, mutant matrix) + `wizard-work/forge/import_audit_probe.py`
(child interpreter, `open` audit hook). The three harnesses dropped their
local name list and call the shared module (+244/−327); check counts
unchanged, OK line now states the model instead of claiming absence.
- Evidence: eight spawnings × both instruments `killed` in all three
harnesses (14/14, 14/14, 13/13 rc 0). Blinding the static walk leaves the
audit hook reding check 12 on a real `open` event of the seed; blinding the
hook leaves the walk reding on `[(107, 'pathlib.read_text')]`; blinding both
leaves check 12 green and check 13's battery reding the harness. A spelling
no list names (`_F = open`) is caught by the hook alone.
- Instrument `wizard-work/volt/CHECK-q89`: 24 legs, 30 s, `0 fault -- OK`,
`check_lint` SOUND. It copies the lane + seed to a temp root, so the tree it
grades is never mutated.
- Left alone: `wizard-work/forge/verify_dump_seed_refactor.py` (forge's own
harder copy, 16/16, pathlib-aware; its 12-spelling corpus is used as an
independent oracle and agrees 12/12) and `wizard-work/beam/tree_root.py`
(Q90's three holes). Neither harness is run by any gate; that is Q87.
docs/queue-index.md
+38 -11
View File
@@ -4,7 +4,7 @@
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 42 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 108 record blocks in all, 66 of them a later closure or correction for an id already opened. The same **42 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 42 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 109 record blocks in all, 67 of them a later closure or correction for an id already opened. The same **42 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
| # | id | kind | lines | bytes | role | title | first `Status:` |
|---|---|---|---|---|---|---|---|
@@ -212,7 +212,8 @@
| 202 | Q129 | record | 13671-13674 | 1515 | cont: DONE | **Q129 — DONE at `590c7d9` (the rename) and the commit this line lands in (the instrument, this record, the re-rendered index); opened and repaired in the same tick, so no opening record was ever … | -- |
| 203 | Q130 | record | 13675-13676 | 5143 | record: BOLD-LEAD | **Q130 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). The third sweep of the day put ten repaired blobs … | -- |
| 204 | Q115 | record | 13677-13688 | 4885 | cont: DONE | **Q115 — DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder-read failure dropped the folder's whole listing and printed no … | -- |
| 205 | Q131 | record | 13689-13699 | 3835 | record: BOLD-LEAD | **Q131 — OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since … | -- |
| 205 | Q131 | record | 13689-13700 | 3836 | record: BOLD-LEAD | **Q131 — OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since … | -- |
| 206 | Q89 | record | 13701-13715 | 7604 | cont: DONE | **Q89 — DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list … | -- |
## Ids with no `## ` heading
@@ -1665,17 +1666,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q115 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder-read failure dropped the folder's whole listing and printed no \u2026"
},
{
"bytes": 3835,
"end_line": 13699,
"bytes": 3836,
"end_line": 13700,
"id": "Q131",
"kind": "record",
"line": 13689,
"lines": 11,
"lines": 12,
"role": "record",
"role_detail": "BOLD-LEAD",
"status": null,
"status_line": null,
"title": "**Q131 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since \u2026"
},
{
"bytes": 7604,
"end_line": 13715,
"id": "Q89",
"kind": "record",
"line": 13701,
"lines": 15,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q89 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list \u2026"
}
],
"ids_multi_section": {
@@ -4466,17 +4480,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q115 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder-read failure dropped the folder's whole listing and printed no \u2026"
},
{
"bytes": 3835,
"end_line": 13699,
"bytes": 3836,
"end_line": 13700,
"id": "Q131",
"kind": "record",
"line": 13689,
"lines": 11,
"lines": 12,
"role": "record",
"role_detail": "BOLD-LEAD",
"status": null,
"status_line": null,
"title": "**Q131 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30; the pipe's monitor harness by authorship, unclaimed and taken). The instrument that certifies the pipe's monitor has been 0/7 since \u2026"
},
{
"bytes": 7604,
"end_line": 13715,
"id": "Q89",
"kind": "record",
"line": 13701,
"lines": 15,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q89 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; forge lane by authorship, unclaimed and taken). The import-time-I/O detector in the three refactor harnesses was one name list \u2026"
}
],
"nested": [
@@ -4864,9 +4891,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
"end_line": 8,
"line": 1
},
"queue_bytes": 2208653,
"queue_lines": 13699,
"queue_sha256": "3b4dfd291f68a5ede31141e3aa8b36aff44318d0a413620d1e5e7567a4835d0c",
"queue_bytes": 2216258,
"queue_lines": 13715,
"queue_sha256": "c68b8208ec31bfab2df4abdcb19d7e2e90405155dbda8e8698b06af249ef1771",
"sections": [
{
"bytes": 3139,
wizard-work/forge/import_audit_probe.py
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""Child interpreter for ``import_io_detector.audit_reads``.
Import ``MODULE_PATH`` (argv[1]) with an audit hook on CPython's ``open`` event
and print ONE json object on stdout::
{"target": "<realpath>", "events": [{"path": ..., "mode": ..., "flags": ...}],
"import_error": null | "Type: message"}
Exit is 0 whenever that json was printed -- an import that raised is *reported*
in ``import_error``, not treated as a probe failure, because a mutant may raise
after the read and the read is what is being graded.
Why a child process: ``sys.addaudithook`` cannot be uninstalled, so an
in-process hook would keep watching the harness for the rest of its life.
What is filtered, and why (each rule is a claim, stated so it can be argued
with):
* the target module's own source and bytecode -- the import itself
* ``.py``/``.pyc``/``.pyo``/``.pyd``/``.so`` open from a directory on
``sys.path`` or under the interpreter's prefix -- the import
machinery loading code, not the module reading data
* opens whose mode/flags are write-only -- this grades reads;
``sys.dont_write_bytecode`` is set so a ``__pycache__`` write would not
even be attempted
Everything else is reported: a data file, a config, a seed, whatever spelling
of ``open`` produced it.
"""
from __future__ import annotations
import importlib.util
import json
import os
import sys
sys.dont_write_bytecode = True
TARGET = os.path.realpath(sys.argv[1])
CODE_EXT = (".py", ".pyc", ".pyo", ".pyd", ".so")
_DIRS = sorted({
os.path.realpath(d)
for d in [*sys.path, sys.base_prefix, sys.prefix]
if d and os.path.isdir(d)
})
def _is_code(path: str) -> bool:
"""True when this open is the import machinery loading code."""
real = os.path.realpath(path)
if real in (TARGET, TARGET + "c"):
return True
if real.endswith(CODE_EXT):
return any(real == d or real.startswith(d + os.sep) for d in _DIRS)
return False
def _is_write(mode, flags) -> bool:
"""True when the open cannot be a read (so it is out of this probe's scope)."""
if isinstance(mode, str) and mode:
return any(c in mode for c in "wax")
try:
value = int(flags)
except (TypeError, ValueError):
return False
return bool(value & (os.O_WRONLY | os.O_CREAT | os.O_TRUNC | os.O_APPEND))
def main() -> int:
events: list = []
def hook(event, args):
if event != "open":
return
path = args[0] if args else None
mode = args[1] if len(args) > 1 else None
flags = args[2] if len(args) > 2 else None
if path is None or _is_write(mode, flags):
return
try:
spelling = os.fspath(path)
except TypeError:
spelling = str(path)
if isinstance(spelling, bytes):
spelling = spelling.decode("utf-8", "replace")
if _is_code(spelling):
return
events.append({
"path": spelling,
"mode": mode if isinstance(mode, str) else None,
"flags": flags if isinstance(flags, int) else None,
})
sys.addaudithook(hook)
error = None
try:
spec = importlib.util.spec_from_file_location("_audited_import", TARGET)
module = importlib.util.module_from_spec(spec)
sys.modules["_audited_import"] = module
spec.loader.exec_module(module)
except BaseException as exc: # report, do not grade, the error
error = "%s: %s" % (type(exc).__name__, exc)
json.dump({"target": TARGET, "events": events, "import_error": error},
sys.stdout)
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
sys.exit(main())
wizard-work/forge/import_io_detector.py
+192
View File
@@ -0,0 +1,192 @@
#!/usr/bin/env python3
"""Did the module perform a READ while it was being imported?
Two instruments, deliberately different in kind, because Q89's fault was a
detector that modelled ONE spelling (a dotted-name list plus a ``builtins.open``
spy) and therefore read zero on ``Path(p).read_text()``, ``os.open(...)`` and
``io.open(...)`` while the module read a file at import.
``static_reads(src)`` -- an AST walk of the MODULE-SCOPE statements (a call
nested inside a ``def``/``async def``/``class`` body runs lazily and is not
counted; that is the I/O the refactor is allowed to keep, and it lives under
``main()``). This instrument matches spellings BY NAME, and the set is stated
here rather than discovered at runtime:
* dotted names -- ``open``, ``io.open``, ``os.open``, ``os.read``,
``codecs.open``, ``json.load``, ``json.loads``, ``pickle.load``,
``pickle.loads``, ``csv.reader``, ``csv.DictReader``,
``urllib.request.urlopen``, ``urllib.urlopen``;
* attribute names, whatever the receiver -- ``.read_text``, ``.read_bytes``,
``.read``, ``.open``. This is what reaches ``Path(p).read_text()``: its
``func`` is an ``ast.Attribute`` over an ``ast.Call``, which a walk that
only joins ``Name``/``Attribute`` chains resolves to ``None``.
It is the *diagnostic*: it names the line number, so a red cell can quote the
site instead of saying "something read something".
``audit_reads(module_path)`` -- the *class* answer. A child interpreter
(``import_audit_probe.py``) installs an audit hook on CPython's ``open`` event
and imports the module by path. That event is raised by ``builtins.open``, by
``io.open``, by ``os.open``/``os.fdopen`` and therefore by ``pathlib`` too, so
every spelling of a real open is seen whatever the source looks like --
including spellings no list here names. It is the *enforcement*.
NOT modelled, stated rather than papered over: reads that never cross the
``open`` audit event -- ``os.read`` on a descriptor opened elsewhere, ``mmap``
(``mmap.__new__``), ``sqlite3.connect``, a C extension reading inside its own
``PyInit_*``, and any I/O a subprocess started at import performs. The static
walk catches ``os.read`` by name; the audit hook cannot see it. Writes are out
of scope for both instruments: this grades reads at import, nothing else.
Neither instrument claims "no I/O" -- both report a count and a site list, and
zero of both is what a clean module must produce.
"""
from __future__ import annotations
import ast
import json
import os
import subprocess
import sys
import tempfile
PROBE = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"import_audit_probe.py")
#: Dotted spellings this instrument models, exactly as written in source.
MODELLED_NAMES = frozenset({
"open", "io.open", "os.open", "os.read", "codecs.open",
"json.load", "json.loads", "pickle.load", "pickle.loads",
"csv.reader", "csv.DictReader",
"urllib.request.urlopen", "urllib.urlopen",
})
#: Attribute spellings this instrument models, whatever the receiver.
MODELLED_ATTRS = frozenset({"read_text", "read_bytes", "read", "open"})
#: One spawning per spelling. Each is appended at module scope to a copy of the
#: module under test and must be killed by BOTH instruments -- a spelling the
#: static walk misses is caught by the audit hook, and a spelling the audit hook
#: cannot see (nothing here, by construction) is caught by the walk. Every
#: snippet reads ``DEFAULT_PATH``, the constant all three refactored triage
#: modules define, because a snippet that read ``__file__`` would be reading
#: code -- which the audit probe filters as import machinery, by design.
SPELLING_MUTANTS = (
("open", "_q = open(DEFAULT_PATH)"),
("json.load", "_bootstrap = json.load(open(DEFAULT_PATH))"),
("io.open", "import io\n_h = io.open(DEFAULT_PATH)\n_h.close()"),
("os.open", "import os\n_fd = os.open(DEFAULT_PATH, os.O_RDONLY)\n"
"os.close(_fd)"),
("codecs.open", "import codecs\n_h = codecs.open(DEFAULT_PATH)\n_h.close()"),
("pathlib.read_text",
"from pathlib import Path\n_q = Path(DEFAULT_PATH).read_text()"),
("pathlib.read_bytes",
"from pathlib import Path\n_q = Path(DEFAULT_PATH).read_bytes()"),
("pathlib.open",
"from pathlib import Path\n_h = Path(DEFAULT_PATH).open()\n_h.close()"),
)
def _dotted(call: ast.Call):
"""``json.load`` for a Name/Attribute chain, else None."""
parts: list = []
cur = call.func
while isinstance(cur, ast.Attribute):
parts.append(cur.attr)
cur = cur.value
if isinstance(cur, ast.Name):
parts.append(cur.id)
return ".".join(reversed(parts))
return None
def static_reads(src: str) -> list:
"""``[(lineno, spelling)]`` for read-spellings at module scope."""
tree = ast.parse(src)
found: list = []
def scan(node) -> None:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
return # lazy -- runs at call time
if isinstance(node, ast.Call):
name = _dotted(node)
if name in MODELLED_NAMES:
found.append((node.lineno, name))
elif isinstance(node.func, ast.Attribute) and node.func.attr in MODELLED_ATTRS:
found.append((node.lineno, "pathlib." + node.func.attr))
for child in ast.iter_child_nodes(node):
scan(child)
for statement in tree.body:
scan(statement)
return found
def audit_reads(module_path: str, timeout: int = 120) -> dict:
"""Import ``module_path`` in a child interpreter under an ``open`` audit hook.
Returns ``{"rc", "events", "import_error", "error"}``. ``rc`` is 0 when the
probe answered; a non-zero ``rc`` means the probe itself could not run and
``error`` says why (a harness must FAIL on it, never read it as cleanliness).
"""
try:
proc = subprocess.run([sys.executable, PROBE, module_path],
capture_output=True, text=True, timeout=timeout)
except (OSError, subprocess.SubprocessError) as exc:
return {"rc": 2, "events": [], "import_error": None,
"error": "%s: %s" % (type(exc).__name__, exc)}
if proc.returncode != 0:
return {"rc": proc.returncode, "events": [], "import_error": None,
"error": (proc.stderr or proc.stdout or "").strip()[-600:]}
lines = [l for l in (proc.stdout or "").splitlines() if l.strip()]
try:
data = json.loads(lines[-1])
except (ValueError, IndexError):
return {"rc": 2, "events": [], "import_error": None,
"error": "unparseable probe output: %r" % (proc.stdout or "")[:300]}
data["rc"] = 0
data.setdefault("error", None)
data.setdefault("import_error", None)
return data
def mutant_matrix(module_path: str, mutations=None, timeout: int = 120) -> list:
"""Grade every modelled spelling: each spawning must be killed by both.
Each row: ``{"label", "static", "audit", "import_error", "rc", "ok"}``.
The spawning is written beside the module (a ``.mutant-*`` file, removed in
``finally``) so that a ``__file__``-relative ``DEFAULT_PATH`` resolves to the
same tree it does in the module itself.
"""
mutations = list(mutations if mutations is not None else SPELLING_MUTANTS)
with open(module_path, encoding="utf-8") as handle:
source = handle.read()
directory = os.path.dirname(os.path.abspath(module_path))
rows: list = []
for label, snippet in mutations:
mutated = source + "\n" + snippet + "\n"
static = static_reads(mutated)
fd, temp = tempfile.mkstemp(prefix=".mutant-", suffix=".py", dir=directory)
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
handle.write(mutated)
probe = audit_reads(temp, timeout=timeout)
finally:
try:
os.unlink(temp)
except OSError:
pass
rows.append({
"label": label,
"static": static,
"audit": probe.get("events") or [],
"import_error": probe.get("import_error"),
"rc": probe.get("rc"),
"error": probe.get("error"),
"ok": bool(static) and bool(probe.get("events")) and probe.get("rc") == 0,
})
return rows
def modelled_spelling_count() -> int:
"""How many spellings this instrument names, for a printing harness to state."""
return len(MODELLED_NAMES) + len(MODELLED_ATTRS)
wizard-work/forge/verify_triage_dump_pending_refactor.py
+82 -109
View File
@@ -25,14 +25,22 @@ probe itself (same JSON input, same output). Checks:
prints the custom file's contents. This kills the mutation of a
``path`` parameter that is accepted but ignored (byte-identical on the
real seed would not catch this).
10. No import-time I/O (M6 killer): a static AST walk of the top-level
statements plus a runtime ``builtins.open`` spy both require zero calls.
Mutant M6 (``_bootstrap = json.load(open(DEFAULT_PATH))`` at module scope,
no print) survives check 5 (rc 0, zero output) by being *silent*, not
clean -- the read happened but was invisible to the silence test.
11. M6 is detected by check 10's io-detector (non-vacuity): the detector fires
on the measured mutant applied to a temp copy, so it is sensitive rather
than theater.
10. No import-time read (M6 killer): two instruments, both must read zero on
the clean module -- a static AST walk over the spellings
``import_io_detector`` names, which includes the ``Path(...).read_text()``,
``os.open(...)`` and ``io.open(...)`` forms a dotted-name list resolves to
None, and a child interpreter that imports the module under an ``open``
audit hook, so every spelling of a real open is seen whatever the source
says. Mutant M6 (``_bootstrap = json.load(open(DEFAULT_PATH))`` at module
scope, no print) survives check 5 (rc 0, zero output) by being *silent*,
not clean -- the read happened but was invisible to the silence test.
11. Every modelled spelling is killed by BOTH instruments (non-vacuity): eight
spawnings at module scope -- ``open``, ``json.load``, ``io.open``,
``os.open``, ``codecs.open``, ``pathlib.read_text``, ``pathlib.read_bytes``,
``pathlib.open`` -- each must be named by the static walk AND produce a
real ``open`` audit event, so a detector that matches nothing cannot pass.
Q89's reading, before this: four of those eight returned rc 0 with "0
module-scope io calls; 0 runtime open calls" printed.
12. The seed path is tree-portable: no host-absolute literal in the refactored
source, and ``DEFAULT_PATH`` resolves inside the tree. Closes the Q86
"a clone grades the host's inbox" gap -- on this host the resolution is
@@ -53,6 +61,12 @@ import tempfile
# Resolve paths from this file's location (wizard-work/forge/).
WORK_DIR = os.path.dirname(os.path.abspath(__file__))
# Q89: the import-time-read detector is shared, not copied three times. Three
# byte-identical copies of a name list is how the evasion survived `802a1ae`;
# one module with one modelled set is what the three harnesses now grade.
if WORK_DIR not in sys.path:
sys.path.insert(0, WORK_DIR)
import import_io_detector as io_det # noqa: E402 (sibling, imported after WORK_DIR)
ORIGINAL = "_triage_dump_pending.py"
REFACTORED = "_triage_dump_pending_refactored.py"
MOD_NAME = REFACTORED[:-3] # strip .py
@@ -371,123 +385,82 @@ def check_path_override_honoured() -> bool:
# ----------------------------------------------------------------------
# Q86: the refactor-lane harness proved SILENCE at import, not ABSENCE of I/O,
# and its seed path was host-absolute. The two fixes below close both gaps.
# Q86/Q89: the refactor-lane harness proved SILENCE at import, not ABSENCE of
# I/O, and its seed path was host-absolute. Both gaps are closed below.
#
# A Call to any of these at MODULE SCOPE runs while the module is imported --
# the M6 mutant (`_bootstrap = json.load(open(DEFAULT_PATH))`, no print) hides
# from check 5 because it emits no stdout, so it needs its own detector.
_IO_CALLS = frozenset({
"open", "json.load", "json.loads", "pickle.load", "pickle.loads",
"csv.reader", "csv.DictReader", "urllib.request.urlopen", "codecs.open",
})
def _call_name(call: ast.Call):
"""Dotted name of a Call's function (``json.load``, ``open``, ...), or
None for a callable expression this detector does not model."""
parts = []
cur = call.func
while isinstance(cur, ast.Attribute):
parts.append(cur.attr)
cur = cur.value
if isinstance(cur, ast.Name):
parts.append(cur.id)
return ".".join(reversed(parts))
return None
def _calls_at_module_scope(src: str) -> list:
"""``[(lineno, name)]`` for io-reading Call nodes at module scope.
A Call counts as "module scope" only when it is NOT nested inside a
FunctionDef / AsyncFunctionDef / ClassDef body -- those run lazily, at call
time, and are exactly the I/O the refactor is allowed to keep (it lives
under ``main()``). The ``if __name__ == "__main__"`` guard's ``main()``
call resolves to ``main`` (not in the io set), so it is correctly ignored.
"""
tree = ast.parse(src)
found: list = []
def scan(node):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
return # do not descend -- calls inside a def body are lazy
if isinstance(node, ast.Call):
name = _call_name(node)
if name in _IO_CALLS:
found.append((node.lineno, name))
for child in ast.iter_child_nodes(node):
scan(child)
for stmt in tree.body:
scan(stmt)
return found
# Q89: a detector that models ONE spelling is a name list, not a detector. The
# dotted-name walk plus a `builtins.open` spy installed by `802a1ae` read zero
# on `Path(p).read_text()`, `os.open(...)` and `io.open(...)` while the module
# read a file at import. The modelled set and both instruments now live in
# `import_io_detector.py` beside this file: an AST walk that names the site, and
# a child interpreter importing under CPython's `open` audit hook, which fires
# for every spelling of a real open. The clean module is graded at check 12; the
# mutant battery at check 13 grades the instrument on each spelling it names.
# ----------------------------------------------------------------------
def check_no_import_time_io() -> bool:
"""Check 5 proves the import is SILENT, not that no I/O happens. M6
(`_bootstrap = json.load(open(DEFAULT_PATH))` at module scope, no print)
survives check 5 (rc 0, zero output) while reading the seed at import.
Two independent detectors, both must read zero on the clean module:
(a) static: AST-walk the top-level statements for an open/json.load call;
(b) dynamic: patch builtins.open to a recorder, re-import the module, and
require zero calls. The import machinery reads .py source via
_io.FileIO (not builtins.open), so this does NOT false-positive on a
clean module -- measured separately (0 calls on a fresh import)."""
static = _calls_at_module_scope(_read(REFACTORED))
Two instruments, both must read zero on the clean module:
(a) static -- an AST walk of the module-scope statements over the
spellings ``import_io_detector.MODELLED_NAMES`` /
``MODELLED_ATTRS`` name. This is what makes ``Path(p).read_text()``,
``os.open(...)`` and ``io.open(...)`` visible at all; it reports the
line number, so a red cell can quote the site.
(b) dynamic -- ``import_audit_probe.py`` imports the module in a child
interpreter under an audit hook on CPython's ``open`` event, so a
spelling nobody modelled is still caught. The probe's own docstring
names what it filters (the import machinery's code reads) and what it
cannot see (reads below the audit layer).
Neither instrument claims "no I/O": each reports a count, and zero of both
is the pass condition."""
static = io_det.static_reads(_read(REFACTORED))
if static:
print("FAIL: static io-call(s) at module scope: %s" % static)
print("FAIL: static read-spelling(s) at module scope: %s" % static)
return False
import builtins
calls: list = []
real_open = builtins.open
def _spy_open(*args, **kwargs):
calls.append(args[0] if args else kwargs.get("file"))
raise RuntimeError("__check_no_import_time_io__ import-time open()")
builtins.open = _spy_open
try:
_mod, exc, _out, _err = _safe_import()
finally:
builtins.open = real_open
if exc is not None:
# A RuntimeError carrying our marker is the very failure we reject;
# any other exception is a real import error and is ungradable here.
if isinstance(exc, RuntimeError) and "__check_no_import_time_io__" in str(exc):
print("FAIL: runtime import-time open() on %r (rc 2)" % calls)
return False
print("FAIL: import raised %s: %r (stdout=%r stderr=%r)"
% (type(exc).__name__, exc, _out, _err))
probe = io_det.audit_reads(os.path.join(WORK_DIR, REFACTORED))
if probe["rc"] != 0:
print("FAIL: the import-audit probe could not grade the module: %s"
% probe["error"])
return False
if calls:
print("FAIL: import made %d open() call(s): %r" % (len(calls), calls))
if probe["import_error"]:
print("FAIL: import raised %s" % probe["import_error"])
return False
print("OK: no import-time I/O (0 module-scope io calls; 0 runtime open calls)")
if probe["events"]:
print("FAIL: %d open event(s) during import: %s"
% (len(probe["events"]), probe["events"]))
return False
print("OK: no import-time read (static 0 over %d modelled spellings; "
"audit 0 open events outside the import machinery)"
% io_det.modelled_spelling_count())
return True
def check_m6_mutant_killed() -> bool:
"""Non-vacuity for check 10: a detector that matches nothing is theater.
Apply the measured M6 mutation -- a silent module-scope
``json.load(open(DEFAULT_PATH))`` inserted before the first def -- to a temp
copy of the source and confirm the static io-detector fires on it. The
clean module has zero such calls (check 10); the mutant has two, so the
detector is sensitive rather than vacuous. (M6 is the mutant measured for
Q86: it returns 10/10 under the old check 5.)"""
src = _read(REFACTORED)
anchor = "def load_data(path: str) -> dict[str, Any]:"
m6_expr = "_bootstrap = json.load(open(DEFAULT_PATH))\n"
mutated = src.replace(anchor, m6_expr + anchor, 1)
if m6_expr not in mutated:
# guard against the insertion anchor drifting in the source
print("FAIL: M6 mutation could not be applied (anchor %r not found)" % anchor)
"""Non-vacuity for check 12: a detector that matches nothing is theater.
Every spelling ``import_io_detector`` names is spawned at module scope in a
copy of the refactored source, and each spawning must be killed by BOTH
instruments -- named by the static walk and caught as a real ``open`` event
by the audit hook. M6 (the measured Q86 mutant, ``json.load(open(...))``) is
the first row; ``io.open``, ``os.open`` and the three ``pathlib`` rows are
the Q89 spellings the old detector read zero on."""
rows = io_det.mutant_matrix(os.path.join(WORK_DIR, REFACTORED))
missed = [row for row in rows if not row["ok"]]
for row in rows:
print(" %-18s static=%-2d audit=%-2d %s"
% (row["label"], len(row["static"]), len(row["audit"]),
"killed" if row["ok"] else
"SURVIVED (rc=%s %r)" % (row["rc"], row["error"])))
if missed:
print("FAIL: %d/%d spelling mutant(s) survived an instrument: %s"
% (len(missed), len(rows), [row["label"] for row in missed]))
return False
sites = _calls_at_module_scope(mutated)
if not sites:
print("FAIL: M6 mutant NOT detected by the io-detector (check 10 is vacuous?)")
return False
print("OK: M6 mutant detected at module scope: %s" % sites)
print("OK: %d/%d import-time-read spellings killed by both instruments "
"(static AST + open-audit hook)" % (len(rows), len(rows)))
return True
wizard-work/forge/verify_triage_dump_refactor.py
+81 -110
View File
@@ -38,14 +38,22 @@ Checks:
prints the custom file's data and not the real seed path. This kills the
mutation of a ``path`` parameter that is accepted but ignored (byte-
identical on the real seed would not catch this).
12. No import-time I/O (M6 killer): a static AST walk of the top-level
statements plus a runtime ``builtins.open`` spy both require zero calls.
Mutant M6 (``_bootstrap = json.load(open(DEFAULT_PATH))`` at module scope,
no print) survived check 5 (rc 0, zero output) by being *silent*, not
clean -- the read happened but was invisible to the silence test.
13. M6 is detected by check 12's io-detector (non-vacuity): the detector fires
on the measured mutant applied to a temp copy, so it is sensitive rather
than theater.
12. No import-time read (M6 killer): two instruments, both must read zero on
the clean module -- a static AST walk over the spellings
``import_io_detector`` names, which includes the ``Path(...).read_text()``,
``os.open(...)`` and ``io.open(...)`` forms a dotted-name list resolves to
None, and a child interpreter that imports the module under an ``open``
audit hook, so every spelling of a real open is seen whatever the source
says. Mutant M6 (``_bootstrap = json.load(open(DEFAULT_PATH))`` at module
scope, no print) survives check 5 (rc 0, zero output) by being *silent*,
not clean -- the read happened but was invisible to the silence test.
13. Every modelled spelling is killed by BOTH instruments (non-vacuity): eight
spawnings at module scope -- ``open``, ``json.load``, ``io.open``,
``os.open``, ``codecs.open``, ``pathlib.read_text``, ``pathlib.read_bytes``,
``pathlib.open`` -- each must be named by the static walk AND produce a
real ``open`` audit event, so a detector that matches nothing cannot pass.
Q89's reading, before this: four of those eight returned rc 0 with "0
module-scope io calls; 0 runtime open calls" printed.
14. The seed path is tree-portable: no host-absolute literal in the refactored
source, and ``DEFAULT_PATH`` resolves inside the tree. Closes the Q86
"a clone grades the host's inbox" gap -- on this host the resolution is
@@ -67,6 +75,12 @@ import tempfile
# Resolve paths from this file's location (wizard-work/forge/).
WORK_DIR = os.path.dirname(os.path.abspath(__file__))
# Q89: the import-time-read detector is shared, not copied three times. Three
# byte-identical copies of a name list is how the evasion survived `802a1ae`;
# one module with one modelled set is what the three harnesses now grade.
if WORK_DIR not in sys.path:
sys.path.insert(0, WORK_DIR)
import import_io_detector as io_det # noqa: E402 (sibling, imported after WORK_DIR)
ORIGINAL = "_triage_dump.py"
REFACTORED = "_triage_dump_refactored.py"
MOD_NAME = REFACTORED[:-3] # strip .py -> _triage_dump_refactored
@@ -409,125 +423,82 @@ def check_path_override_honoured() -> bool:
# ----------------------------------------------------------------------
# Q86: the refactor-lane harness proved SILENCE at import, not ABSENCE of I/O,
# and its seed path was host-absolute. The two fixes below close both gaps.
# Q86/Q89: the refactor-lane harness proved SILENCE at import, not ABSENCE of
# I/O, and its seed path was host-absolute. Both gaps are closed below.
#
# A Call to any of these at MODULE SCOPE runs while the module is imported --
# the M6 mutant (`_bootstrap = json.load(open(DEFAULT_PATH))`, no print) hides
# from check 5 because it emits no stdout, so it needs its own detector.
_IO_CALLS = frozenset({
"open", "json.load", "json.loads", "pickle.load", "pickle.loads",
"csv.reader", "csv.DictReader", "urllib.request.urlopen", "codecs.open",
})
def _call_name(call: ast.Call):
"""Dotted name of a Call's function (``json.load``, ``open``, ...), or
None for a callable expression this detector does not model."""
parts = []
cur = call.func
while isinstance(cur, ast.Attribute):
parts.append(cur.attr)
cur = cur.value
if isinstance(cur, ast.Name):
parts.append(cur.id)
return ".".join(reversed(parts))
return None
def _calls_at_module_scope(src: str) -> list:
"""``[(lineno, name)]`` for io-reading Call nodes at module scope.
A Call counts as "module scope" only when it is NOT nested inside a
FunctionDef / AsyncFunctionDef / ClassDef body -- those run lazily, at call
time, and are exactly the I/O the refactor is allowed to keep (it lives
under ``main()``). The ``if __name__ == "__main__"`` guard's ``main()``
call resolves to ``main`` (not in the io set), so it is correctly ignored.
"""
tree = ast.parse(src)
found: list = []
def scan(node):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
return # do not descend -- calls inside a def body are lazy
if isinstance(node, ast.Call):
name = _call_name(node)
if name in _IO_CALLS:
found.append((node.lineno, name))
for child in ast.iter_child_nodes(node):
scan(child)
for stmt in tree.body:
scan(stmt)
return found
# Q89: a detector that models ONE spelling is a name list, not a detector. The
# dotted-name walk plus a `builtins.open` spy installed by `802a1ae` read zero
# on `Path(p).read_text()`, `os.open(...)` and `io.open(...)` while the module
# read a file at import. The modelled set and both instruments now live in
# `import_io_detector.py` beside this file: an AST walk that names the site, and
# a child interpreter importing under CPython's `open` audit hook, which fires
# for every spelling of a real open. The clean module is graded at check 12; the
# mutant battery at check 13 grades the instrument on each spelling it names.
# ----------------------------------------------------------------------
def check_no_import_time_io() -> bool:
"""Check 5 proves the import is SILENT, not that no I/O happens. M6
(`_bootstrap = json.load(open(DEFAULT_PATH))` at module scope, no print)
survives check 5 (rc 0, zero output) while reading the seed at import.
Two independent detectors, both must read zero on the clean module:
(a) static: AST-walk the top-level statements for an open/json.load call;
(b) dynamic: patch builtins.open to a recorder, re-import the module, and
require zero calls. The import machinery reads .py source via
_io.FileIO (not builtins.open), so this does NOT false-positive on a
clean module -- measured separately (0 calls on a fresh import)."""
src = _read(REFACTORED)
static = _calls_at_module_scope(src)
Two instruments, both must read zero on the clean module:
(a) static -- an AST walk of the module-scope statements over the
spellings ``import_io_detector.MODELLED_NAMES`` /
``MODELLED_ATTRS`` name. This is what makes ``Path(p).read_text()``,
``os.open(...)`` and ``io.open(...)`` visible at all; it reports the
line number, so a red cell can quote the site.
(b) dynamic -- ``import_audit_probe.py`` imports the module in a child
interpreter under an audit hook on CPython's ``open`` event, so a
spelling nobody modelled is still caught. The probe's own docstring
names what it filters (the import machinery's code reads) and what it
cannot see (reads below the audit layer).
Neither instrument claims "no I/O": each reports a count, and zero of both
is the pass condition."""
static = io_det.static_reads(_read(REFACTORED))
if static:
print("FAIL: static io-call(s) at module scope: %s" % static)
print("FAIL: static read-spelling(s) at module scope: %s" % static)
return False
# Dynamic backstop: require zero open() calls during a fresh import.
import builtins
calls: list = []
real_open = builtins.open
def _spy_open(*args, **kwargs):
calls.append(args[0] if args else kwargs.get("file"))
raise RuntimeError("__check_no_import_time_io__ import-time open()")
builtins.open = _spy_open
try:
_mod, exc, _out, _err = _safe_import()
finally:
builtins.open = real_open
if exc is not None:
# A RuntimeError carrying our marker is the very failure we reject;
# any other exception is a real import error and is ungradable here.
if isinstance(exc, RuntimeError) and "__check_no_import_time_io__" in str(exc):
print("FAIL: runtime import-time open() on %r (rc 2)" % calls)
return False
print("FAIL: import raised %s: %r (stdout=%r stderr=%r)"
% (type(exc).__name__, exc, _out, _err))
probe = io_det.audit_reads(os.path.join(WORK_DIR, REFACTORED))
if probe["rc"] != 0:
print("FAIL: the import-audit probe could not grade the module: %s"
% probe["error"])
return False
if calls:
print("FAIL: import made %d open() call(s): %r" % (len(calls), calls))
if probe["import_error"]:
print("FAIL: import raised %s" % probe["import_error"])
return False
print("OK: no import-time I/O (0 module-scope io calls; 0 runtime open calls)")
if probe["events"]:
print("FAIL: %d open event(s) during import: %s"
% (len(probe["events"]), probe["events"]))
return False
print("OK: no import-time read (static 0 over %d modelled spellings; "
"audit 0 open events outside the import machinery)"
% io_det.modelled_spelling_count())
return True
def check_m6_mutant_killed() -> bool:
"""Non-vacuity for check 12: a detector that matches nothing is theater.
Apply the measured M6 mutation -- a silent module-scope
``json.load(open(DEFAULT_PATH))`` inserted before the first def -- to a temp
copy of the source and confirm the static io-detector fires on it. The
clean module has zero such calls (check 12); the mutant has two, so the
detector is sensitive rather than vacuous. (M6 is the mutant measured for
Q86: it returns 11/11 under the old check 5.)"""
src = _read(REFACTORED)
m6_expr = "_bootstrap = json.load(open(DEFAULT_PATH))\n"
mutated = src.replace("def load_data(path):",
m6_expr + "def load_data(path):", 1)
if m6_expr not in mutated:
# guard against the insertion anchor drifting in the source
print("FAIL: M6 mutation could not be applied (anchor 'def load_data' not found)")
Every spelling ``import_io_detector`` names is spawned at module scope in a
copy of the refactored source, and each spawning must be killed by BOTH
instruments -- named by the static walk and caught as a real ``open`` event
by the audit hook. M6 (the measured Q86 mutant, ``json.load(open(...))``) is
the first row; ``io.open``, ``os.open`` and the three ``pathlib`` rows are
the Q89 spellings the old detector read zero on."""
rows = io_det.mutant_matrix(os.path.join(WORK_DIR, REFACTORED))
missed = [row for row in rows if not row["ok"]]
for row in rows:
print(" %-18s static=%-2d audit=%-2d %s"
% (row["label"], len(row["static"]), len(row["audit"]),
"killed" if row["ok"] else
"SURVIVED (rc=%s %r)" % (row["rc"], row["error"])))
if missed:
print("FAIL: %d/%d spelling mutant(s) survived an instrument: %s"
% (len(missed), len(rows), [row["label"] for row in missed]))
return False
sites = _calls_at_module_scope(mutated)
if not sites:
print("FAIL: M6 mutant NOT detected by the io-detector (check 12 is vacuous?)")
return False
print("OK: M6 mutant detected at module scope: %s" % sites)
print("OK: %d/%d import-time-read spellings killed by both instruments "
"(static AST + open-audit hook)" % (len(rows), len(rows)))
return True
wizard-work/forge/verify_triage_probe_refactor.py
+81 -108
View File
@@ -36,14 +36,22 @@ Checks:
prints the custom file's contents. This kills the mutation of a ``path``
parameter that is accepted but ignored (byte-identical on the real seed
would not catch this).
12. No import-time I/O (M6 killer): a static AST walk of the top-level
statements plus a runtime ``builtins.open`` spy both require zero calls.
Mutant M6 (``_bootstrap = json.load(open(DEFAULT_PATH))`` at module scope,
no print) survives check 5 (rc 0, zero output) by being *silent*, not
clean -- the read happened but was invisible to the silence test.
13. M6 is detected by check 12's io-detector (non-vacuity): the detector fires
on the measured mutant applied to a temp copy, so it is sensitive rather
than theater.
12. No import-time read (M6 killer): two instruments, both must read zero on
the clean module -- a static AST walk over the spellings
``import_io_detector`` names, which includes the ``Path(...).read_text()``,
``os.open(...)`` and ``io.open(...)`` forms a dotted-name list resolves to
None, and a child interpreter that imports the module under an ``open``
audit hook, so every spelling of a real open is seen whatever the source
says. Mutant M6 (``_bootstrap = json.load(open(DEFAULT_PATH))`` at module
scope, no print) survives check 5 (rc 0, zero output) by being *silent*,
not clean -- the read happened but was invisible to the silence test.
13. Every modelled spelling is killed by BOTH instruments (non-vacuity): eight
spawnings at module scope -- ``open``, ``json.load``, ``io.open``,
``os.open``, ``codecs.open``, ``pathlib.read_text``, ``pathlib.read_bytes``,
``pathlib.open`` -- each must be named by the static walk AND produce a
real ``open`` audit event, so a detector that matches nothing cannot pass.
Q89's reading, before this: four of those eight returned rc 0 with "0
module-scope io calls; 0 runtime open calls" printed.
14. The seed path is tree-portable: no host-absolute literal in the refactored
source, and ``DEFAULT_PATH`` resolves inside the tree. Closes the Q86
"a clone grades the host's inbox" gap -- on this host the resolution is
@@ -65,6 +73,12 @@ import tempfile
# Resolve paths from this file's location (wizard-work/forge/).
WORK_DIR = os.path.dirname(os.path.abspath(__file__))
# Q89: the import-time-read detector is shared, not copied three times. Three
# byte-identical copies of a name list is how the evasion survived `802a1ae`;
# one module with one modelled set is what the three harnesses now grade.
if WORK_DIR not in sys.path:
sys.path.insert(0, WORK_DIR)
import import_io_detector as io_det # noqa: E402 (sibling, imported after WORK_DIR)
ORIGINAL = "_triage_probe.py"
REFACTORED = "_triage_probe_refactored.py"
MOD_NAME = REFACTORED[:-3] # strip .py -> _triage_probe_refactored
@@ -395,123 +409,82 @@ def check_path_override_honoured() -> bool:
# ----------------------------------------------------------------------
# Q86: the refactor-lane harness proved SILENCE at import, not ABSENCE of I/O,
# and its seed path was host-absolute. The two fixes below close both gaps.
# Q86/Q89: the refactor-lane harness proved SILENCE at import, not ABSENCE of
# I/O, and its seed path was host-absolute. Both gaps are closed below.
#
# A Call to any of these at MODULE SCOPE runs while the module is imported --
# the M6 mutant (`_bootstrap = json.load(open(DEFAULT_PATH))`, no print) hides
# from check 5 because it emits no stdout, so it needs its own detector.
_IO_CALLS = frozenset({
"open", "json.load", "json.loads", "pickle.load", "pickle.loads",
"csv.reader", "csv.DictReader", "urllib.request.urlopen", "codecs.open",
})
def _call_name(call: ast.Call):
"""Dotted name of a Call's function (``json.load``, ``open``, ...), or
None for a callable expression this detector does not model."""
parts = []
cur = call.func
while isinstance(cur, ast.Attribute):
parts.append(cur.attr)
cur = cur.value
if isinstance(cur, ast.Name):
parts.append(cur.id)
return ".".join(reversed(parts))
return None
def _calls_at_module_scope(src: str) -> list:
"""``[(lineno, name)]`` for io-reading Call nodes at module scope.
A Call counts as "module scope" only when it is NOT nested inside a
FunctionDef / AsyncFunctionDef / ClassDef body -- those run lazily, at call
time, and are exactly the I/O the refactor is allowed to keep (it lives
under ``main()``). The ``if __name__ == "__main__"`` guard's ``main()``
call resolves to ``main`` (not in the io set), so it is correctly ignored.
"""
tree = ast.parse(src)
found: list = []
def scan(node):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
return # do not descend -- calls inside a def body are lazy
if isinstance(node, ast.Call):
name = _call_name(node)
if name in _IO_CALLS:
found.append((node.lineno, name))
for child in ast.iter_child_nodes(node):
scan(child)
for stmt in tree.body:
scan(stmt)
return found
# Q89: a detector that models ONE spelling is a name list, not a detector. The
# dotted-name walk plus a `builtins.open` spy installed by `802a1ae` read zero
# on `Path(p).read_text()`, `os.open(...)` and `io.open(...)` while the module
# read a file at import. The modelled set and both instruments now live in
# `import_io_detector.py` beside this file: an AST walk that names the site, and
# a child interpreter importing under CPython's `open` audit hook, which fires
# for every spelling of a real open. The clean module is graded at check 12; the
# mutant battery at check 13 grades the instrument on each spelling it names.
# ----------------------------------------------------------------------
def check_no_import_time_io() -> bool:
"""Check 5 proves the import is SILENT, not that no I/O happens. M6
(`_bootstrap = json.load(open(DEFAULT_PATH))` at module scope, no print)
survives check 5 (rc 0, zero output) while reading the seed at import.
Two independent detectors, both must read zero on the clean module:
(a) static: AST-walk the top-level statements for an open/json.load call;
(b) dynamic: patch builtins.open to a recorder, re-import the module, and
require zero calls. The import machinery reads .py source via
_io.FileIO (not builtins.open), so this does NOT false-positive on a
clean module -- measured separately (0 calls on a fresh import)."""
static = _calls_at_module_scope(_read(REFACTORED))
Two instruments, both must read zero on the clean module:
(a) static -- an AST walk of the module-scope statements over the
spellings ``import_io_detector.MODELLED_NAMES`` /
``MODELLED_ATTRS`` name. This is what makes ``Path(p).read_text()``,
``os.open(...)`` and ``io.open(...)`` visible at all; it reports the
line number, so a red cell can quote the site.
(b) dynamic -- ``import_audit_probe.py`` imports the module in a child
interpreter under an audit hook on CPython's ``open`` event, so a
spelling nobody modelled is still caught. The probe's own docstring
names what it filters (the import machinery's code reads) and what it
cannot see (reads below the audit layer).
Neither instrument claims "no I/O": each reports a count, and zero of both
is the pass condition."""
static = io_det.static_reads(_read(REFACTORED))
if static:
print("FAIL: static io-call(s) at module scope: %s" % static)
print("FAIL: static read-spelling(s) at module scope: %s" % static)
return False
import builtins
calls: list = []
real_open = builtins.open
def _spy_open(*args, **kwargs):
calls.append(args[0] if args else kwargs.get("file"))
raise RuntimeError("__check_no_import_time_io__ import-time open()")
builtins.open = _spy_open
try:
_mod, exc, _out, _err = _safe_import()
finally:
builtins.open = real_open
if exc is not None:
# A RuntimeError carrying our marker is the very failure we reject;
# any other exception is a real import error and is ungradable here.
if isinstance(exc, RuntimeError) and "__check_no_import_time_io__" in str(exc):
print("FAIL: runtime import-time open() on %r (rc 2)" % calls)
return False
print("FAIL: import raised %s: %r (stdout=%r stderr=%r)"
% (type(exc).__name__, exc, _out, _err))
probe = io_det.audit_reads(os.path.join(WORK_DIR, REFACTORED))
if probe["rc"] != 0:
print("FAIL: the import-audit probe could not grade the module: %s"
% probe["error"])
return False
if calls:
print("FAIL: import made %d open() call(s): %r" % (len(calls), calls))
if probe["import_error"]:
print("FAIL: import raised %s" % probe["import_error"])
return False
print("OK: no import-time I/O (0 module-scope io calls; 0 runtime open calls)")
if probe["events"]:
print("FAIL: %d open event(s) during import: %s"
% (len(probe["events"]), probe["events"]))
return False
print("OK: no import-time read (static 0 over %d modelled spellings; "
"audit 0 open events outside the import machinery)"
% io_det.modelled_spelling_count())
return True
def check_m6_mutant_killed() -> bool:
"""Non-vacuity for check 12: a detector that matches nothing is theater.
Apply the measured M6 mutation -- a silent module-scope
``json.load(open(DEFAULT_PATH))`` inserted before the first def -- to a temp
copy of the source and confirm the static io-detector fires on it. The
clean module has zero such calls (check 12); the mutant has two, so the
detector is sensitive rather than vacuous. (M6 is the mutant measured for
Q86: it returns 11/11 under the old check 5.)"""
src = _read(REFACTORED)
anchor = "def load_data(path):"
m6_expr = "_bootstrap = json.load(open(DEFAULT_PATH))\n"
mutated = src.replace(anchor, m6_expr + anchor, 1)
if m6_expr not in mutated:
# guard against the insertion anchor drifting in the source
print("FAIL: M6 mutation could not be applied (anchor %r not found)" % anchor)
Every spelling ``import_io_detector`` names is spawned at module scope in a
copy of the refactored source, and each spawning must be killed by BOTH
instruments -- named by the static walk and caught as a real ``open`` event
by the audit hook. M6 (the measured Q86 mutant, ``json.load(open(...))``) is
the first row; ``io.open``, ``os.open`` and the three ``pathlib`` rows are
the Q89 spellings the old detector read zero on."""
rows = io_det.mutant_matrix(os.path.join(WORK_DIR, REFACTORED))
missed = [row for row in rows if not row["ok"]]
for row in rows:
print(" %-18s static=%-2d audit=%-2d %s"
% (row["label"], len(row["static"]), len(row["audit"]),
"killed" if row["ok"] else
"SURVIVED (rc=%s %r)" % (row["rc"], row["error"])))
if missed:
print("FAIL: %d/%d spelling mutant(s) survived an instrument: %s"
% (len(missed), len(rows), [row["label"] for row in missed]))
return False
sites = _calls_at_module_scope(mutated)
if not sites:
print("FAIL: M6 mutant NOT detected by the io-detector (check 12 is vacuous?)")
return False
print("OK: M6 mutant detected at module scope: %s" % sites)
print("OK: %d/%d import-time-read spellings killed by both instruments "
"(static AST + open-audit hook)" % (len(rows), len(rows)))
return True
wizard-work/volt/CHECK-q89
Executable
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
# Q89 CHECK -- the import-time-read detector, graded on the spellings it names
# and on one no list can name.
#
# rc 0 = every leg as expected; 1 = a graded fault; 2 = could not grade.
# The work is done by wizard-work/volt/q89_verify.py beside this file; it copies
# the forge lane and the seed it reads into a temp root, so the tree this
# instrument ships in is never mutated. Run it from any checkout.
set -u
REPO="$(cd "$(dirname "$0")/../.." && pwd)" || exit 2
if ! cd "$REPO"; then
echo "q89: could not grade -- cannot cd to $REPO"
exit 2
fi
if ! command -v python3 >/dev/null 2>&1; then
echo "q89: could not grade -- no python3 on PATH"
exit 2
fi
exec python3 wizard-work/volt/q89_verify.py
wizard-work/volt/q89_verify.py
+272
View File
@@ -0,0 +1,272 @@
#!/usr/bin/env python3
"""Q89: grade the import-time-read detector on every spelling it names, and on
one spelling no list can name.
What it grades, in legs:
1. clean the three refactor harnesses are green on the tree as committed
and print their own totals (14/14, 14/14, 13/13 today).
2. evasion the four Q89 spellings -- ``Path(...).read_text()``,
``os.open(...)``, ``io.open(...)``, ``Path(...).open()`` -- are
appended at module scope to each harness's module in turn and the
harness must go red (rc 1) naming the site. Before the repair all
four read rc 0 with "0 module-scope io calls; 0 runtime open
calls" printed.
3. control a module-scope ``len(DEFAULT_PATH)`` is not a read: it must stay
green, so the detector is not a constant-red.
4. blinding the two instruments are disabled in turn. With the static walk
blinded the audit hook must be the one that reds check 12; with
the audit hook blinded the static walk must. With both blinded
check 12 prints OK and the mutant battery (check 13) must still
red the harness -- a detector cannot be emptied quietly.
5. no-list ``_F = open`` / ``_F(DEFAULT_PATH)`` resolves to a name no set
contains; the audit hook must catch the real open event anyway.
This is the difference between a longer list and a closed class.
6. fourth the fourth copy of this detector (``verify_dump_seed_refactor.py``,
forge's own later hardening) still passes 16/16 and the shared
walk agrees with its 12-spelling corpus, so this module is a
superset rather than a third convention.
Everything runs in a temporary copy of ``wizard-work/forge`` beside a copy of
``inbox/seed-2026-09-26.json``; the tree the instrument ships in is never
mutated, which is what lets it grade in one checkout and be run in another.
rc 0 = every leg as expected; 1 = a graded fault; 2 = could not grade.
"""
from __future__ import annotations
import importlib.util
import json
import os
import shutil
import subprocess
import sys
import tempfile
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(
os.path.abspath(__file__))))
SEED = os.path.join("inbox", "seed-2026-09-26.json")
SEED_SHA_PIN = None # the seed is copied, not pinned: its bytes are not graded
HARNESSES = (
("probe", "verify_triage_probe_refactor.py", "_triage_probe_refactored.py"),
("dump", "verify_triage_dump_refactor.py", "_triage_dump_refactored.py"),
("pending", "verify_triage_dump_pending_refactor.py",
"_triage_dump_pending_refactored.py"),
)
FOURTH = ("verify_dump_seed_refactor.py", "_dump_seed_refactored.py")
EVASIONS = {
"pathlib.read_text":
"\nfrom pathlib import Path as _P\n_evade = _P(DEFAULT_PATH).read_text()\n",
"os.open":
"\nimport os as _o\n_fd = _o.open(DEFAULT_PATH, _o.O_RDONLY)\n_o.close(_fd)\n",
"io.open":
"\nimport io as _io\n_h = _io.open(DEFAULT_PATH)\n_h.close()\n",
"Path.open":
"\nfrom pathlib import Path as _P\nwith _P(DEFAULT_PATH).open() as _fh:\n"
" _evade = _fh.read()\n",
}
CONTROL = "\n_neg = len(DEFAULT_PATH)\n"
NO_LIST = "_F = open\n_h = _F(DEFAULT_PATH)\n_h.close()\n"
BLIND_STATIC = "\nMODELLED_NAMES = frozenset()\nMODELLED_ATTRS = frozenset()\n"
BLIND_AUDIT = ('\n\ndef audit_reads(module_path, timeout=120):\n'
' return {"rc": 0, "events": [], "import_error": None,\n'
' "error": None}\n')
class Report:
def __init__(self) -> None:
self.faults = 0
self.legs = 0
def ok(self, leg: str, note: str) -> None:
self.legs += 1
print("ok %-22s %s" % (leg, note))
def fail(self, leg: str, note: str) -> None:
self.legs += 1
self.faults += 1
print("FAIL %-20s %s" % (leg, note))
def run(forge: str, harness: str):
proc = subprocess.run(["python3", harness], cwd=forge, capture_output=True,
text=True, timeout=300)
out = proc.stdout or ""
total = next((l.strip() for l in out.splitlines()
if l.startswith("=== Verification")), "")
first_fail = next((l.strip() for l in out.splitlines()
if l.strip().startswith("FAIL")), "")
# The io detector's own verdict, wherever it lands: a harness whose earlier
# checks red first (the dump lane's dead-import check shadows it) must still
# be graded on whether the io detector fired, not on check order.
io_fail = next((l.strip() for l in out.splitlines()
if l.strip().startswith("FAIL")
and ("static read-spelling" in l or "open event(s)" in l)), "")
return proc.returncode, total, first_fail, io_fail, out
def load(name: str, path: str):
spec = importlib.util.spec_from_file_location(name, path)
if spec is None or spec.loader is None:
raise SystemExit("cannot load %s" % path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def build_copy(base: str) -> str:
"""Copy the forge lane and the seed the modules read; return the copy's forge."""
work = os.path.join(base, "tree")
forge = os.path.join(work, "wizard-work", "forge")
os.makedirs(os.path.dirname(forge))
shutil.copytree(os.path.join(ROOT, "wizard-work", "forge"), forge,
ignore=shutil.ignore_patterns("__pycache__", ".pytest_cache"))
os.makedirs(os.path.join(work, "inbox"))
shutil.copy2(os.path.join(ROOT, SEED), os.path.join(work, SEED))
return forge
def with_suffix(forge: str, name: str, suffix: str, fn):
"""Run ``fn`` with ``suffix`` appended to one module, then restore the bytes."""
path = os.path.join(forge, name)
with open(path, encoding="utf-8") as handle:
original = handle.read()
try:
with open(path, "w", encoding="utf-8") as handle:
handle.write(original + suffix)
return fn()
finally:
with open(path, "w", encoding="utf-8") as handle:
handle.write(original)
with open(path, encoding="utf-8") as handle:
if handle.read() != original:
raise SystemExit(2)
def main() -> int:
if not os.path.isdir(os.path.join(ROOT, "wizard-work", "forge")):
print("FAIL tree: %s is not this instrument's repo" % ROOT)
return 2
report = Report()
base = tempfile.mkdtemp(prefix="q89-")
try:
forge = build_copy(base)
# (1) clean
totals = {}
for label, harness, _module in HARNESSES:
rc, total, fail, _io_fail, _out = run(forge, harness)
totals[label] = total
if rc == 0 and total:
report.ok("clean " + label, total.replace("=== Verification: ", ""))
else:
report.fail("clean " + label, "rc=%d %s %s" % (rc, total, fail))
# (2) evasion: every spelling must red every harness
for label, harness, module in HARNESSES:
for spelling, body in EVASIONS.items():
rc, _total, fail, io_fail, _out = with_suffix(
forge, module, body, lambda: run(forge, harness))
if rc == 1 and io_fail:
report.ok("%s %s" % (label, spelling), "rc=1, %s" % io_fail[:60])
else:
report.fail("%s %s" % (label, spelling),
"rc=%d, io detector said %r (first FAIL %r)"
% (rc, io_fail[:50], fail[:50]))
# (3) the negative control is not a read
for label, harness, module in HARNESSES:
rc, _total, fail, _io_fail, _out = with_suffix(
forge, module, CONTROL, lambda: run(forge, harness))
if rc == 0:
report.ok("control " + label, "len() at module scope stays green")
else:
report.fail("control " + label, "rc=%d %r" % (rc, fail[:80]))
# (4) each instrument is load-bearing, and the battery is the tripwire
probe_module = HARNESSES[0][2]
probe_harness = HARNESSES[0][1]
def _blinded(static_patch: str, audit_patch: str):
detector = os.path.join(forge, "import_io_detector.py")
with open(detector, encoding="utf-8") as handle:
saved = handle.read()
def go():
with open(detector, "w", encoding="utf-8") as handle:
handle.write(saved + static_patch + audit_patch)
return run(forge, probe_harness)
try:
return with_suffix(forge, probe_module, EVASIONS["pathlib.read_text"], go)
finally:
with open(detector, "w", encoding="utf-8") as handle:
handle.write(saved)
rc, _t, fail, io_fail, out = _blinded(BLIND_STATIC, "")
if rc == 1 and io_fail and "open event(s) during import" in io_fail:
report.ok("blinded static", "audit hook alone reds check 12")
else:
report.fail("blinded static", "rc=%d %r" % (rc, io_fail[:90]))
rc, _t, fail, io_fail, out = _blinded("", BLIND_AUDIT)
if rc == 1 and io_fail and "static read-spelling" in io_fail:
report.ok("blinded audit", "static walk alone reds check 12")
else:
report.fail("blinded audit", "rc=%d %r" % (rc, io_fail[:90]))
rc, _t, fail, io_fail, out = _blinded(BLIND_STATIC, BLIND_AUDIT)
check12_clean = ("OK: no import-time read" in out
and "modelled spellings" in out)
battery_red = "spelling mutant(s) survived" in out
if rc == 1 and check12_clean and battery_red:
report.ok("blinded both", "check 12 green, check 13 battery reds it")
else:
report.fail("blinded both",
"rc=%d check12_clean=%s battery_red=%s"
% (rc, check12_clean, battery_red))
# (5) a spelling no list names
rc, _t, fail, io_fail, out = with_suffix(
forge, probe_module, NO_LIST, lambda: run(forge, probe_harness))
if rc == 1 and io_fail and "open event(s) during import" in io_fail:
report.ok("no-list spelling", "_F = open caught by the audit hook")
else:
report.fail("no-list spelling", "rc=%d %r" % (rc, io_fail[:100]))
# (6) the fourth copy still passes, and the shared walk is a superset
rc, total, fail, _io_fail, _out = run(forge, FOURTH[0])
if rc == 0 and "16/16" in total:
report.ok("fourth copy", total.replace("=== Verification: ", ""))
else:
report.fail("fourth copy", "rc=%d %s %s" % (rc, total, fail))
detector = load("q89_detector",
os.path.join(forge, "import_io_detector.py"))
fourth = load("q89_fourth", os.path.join(forge, FOURTH[0]))
with open(os.path.join(forge, fourth.REFACTORED), encoding="utf-8") as handle:
source = handle.read()
missed = [label for label, snippet in fourth._IO_SPELLINGS
if not detector.static_reads(source + "\n" + snippet + "\n")]
if not missed:
report.ok("corpus agreement",
"%d/%d of the fourth's spellings flagged"
% (len(fourth._IO_SPELLINGS), len(fourth._IO_SPELLINGS)))
else:
report.fail("corpus agreement", "shared walk missed %s" % missed)
except (OSError, subprocess.SubprocessError, SystemExit) as exc:
print("FAIL instrument: %s: %s" % (type(exc).__name__, exc))
return 2
finally:
shutil.rmtree(base, ignore_errors=True)
if report.faults:
print("q89: FAULT -- %d leg line(s) red of %d graded" % (report.faults, report.legs))
return 1
print("q89: %d leg line(s), 0 fault -- OK" % report.legs)
return 0
if __name__ == "__main__":
sys.exit(main())