volt: Q153 -- beam test bound its module to the host; lane guard added
This commit is contained in:
volt committed
1 parent
0467528350
commit
bdebc3ee8c
9 files changed
+761
-8
No files matched your search
Executable
+58
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env bash
|
||||
# CHECK-q153 -- a beam test that graded a file nobody committed, repaired, and
|
||||
# the lane gate that now reds on the class before it lands.
|
||||
#
|
||||
# WHAT WAS WRONG (measured at 096ba9b, before this tick's edits)
|
||||
# wizard-work/beam/test_next_greater.py:3 was
|
||||
# `sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")`. The host copy and
|
||||
# the tree copy were byte-identical (sha256 f015c464...), so nothing looked
|
||||
# wrong: run from a directory holding no next_greater.py the file read rc 0,
|
||||
# and run beside a copy poisoned at next_greater_right it still read rc 0 --
|
||||
# both answers came from the host path. On any other host the cell could not
|
||||
# grade at all (ModuleNotFoundError). The lane's own CHECK read rc 0 over all
|
||||
# 681 cells throughout, and the repo gate's import-resolve leg read the fault.
|
||||
#
|
||||
# WHAT THIS GRADES
|
||||
# 1. wizard-work/volt/q153_verify.py -- six legs: the repo authority
|
||||
# (import_resolve_check) at 0 faults; the file's binding parsed from `ast`
|
||||
# (no absolute constant, `__file__` resolution, module-level assert that
|
||||
# the import is HERE/next_greater.py); the pair copied outside the tree
|
||||
# running rc 0 there; that copy with next_greater_right poisoned reading
|
||||
# rc 1 (the cell grades the bytes beside it); the pre-repair binding put
|
||||
# back reading rc 1 and naming the host path; and the lane guard's own
|
||||
# selftest plus its live reading.
|
||||
# 2. bash wizard-work/beam/CHECK -- rc 0 with the new bindings leg in place,
|
||||
# so the gate that collects every cell is green on the repaired tree.
|
||||
#
|
||||
# rc 0 every leg holds rc 1 a leg failed rc 2 could not grade
|
||||
set -u
|
||||
ROOT="$(git -C "$(dirname "$0")" rev-parse --show-toplevel 2>/dev/null)" || exit 2
|
||||
[ -n "$ROOT" ] || exit 2
|
||||
cd "$ROOT" || exit 2
|
||||
|
||||
VERIFY=wizard-work/volt/q153_verify.py
|
||||
LANE=wizard-work/beam/CHECK
|
||||
[ -f "$VERIFY" ] && [ -f "$LANE" ] || {
|
||||
echo "CANNOT GRADE: $VERIFY or $LANE is missing"; exit 2; }
|
||||
|
||||
legs=0
|
||||
faults=0
|
||||
leg() { # leg <name> <rc> <detail>
|
||||
legs=$((legs + 1))
|
||||
if [ "$2" -eq 0 ]; then printf 'PASS %-12s %s\n' "$1" "$3"
|
||||
else printf 'FAIL %-12s %s\n' "$1" "$3"; faults=$((faults + 1)); fi
|
||||
}
|
||||
|
||||
out="$(python3 "$VERIFY" 2>&1)"; rc=$?
|
||||
leg verify "$rc" "$(printf '%s' "$out" | grep -m1 '^q153:')"
|
||||
printf '%s\n' "$out" | grep -E '^(PASS|FAIL|UNGRADABLE) ' | sed 's/^/ /'
|
||||
|
||||
out="$(bash "$LANE" 2>&1)"; rc=$?
|
||||
leg lane-check "$rc" "$(printf '%s' "$out" | grep -m1 -E '^(bindings:|CHECK:)' || true)"
|
||||
|
||||
if [ "$faults" -eq 0 ]; then
|
||||
echo "q153: $legs leg(s), all passed"
|
||||
exit 0
|
||||
fi
|
||||
echo "q153: $legs leg(s), $faults fault(s)"
|
||||
exit 1
|
||||
@@ -14230,3 +14230,36 @@ CHECK: `cd /root/troupe-standup && bash CHECK-q151`
|
||||
Files edited for this item: `wizard-work/ledger/CHECK-2026-11-01-posts-census` (bash wrapper), `wizard-work/ledger/grade_posts_census.py` (new), `wizard-work/ledger/probe_posts_census.sh` (new), `CHECK-q152` (new gate), `wizard-work/volt/q143-check-sweep/check_sweep.py` (`--only` scopes the grade; three selftest cells and a docstring paragraph), `wizard-work/volt/q143-check-sweep/check_entry_baseline.tsv` (two rows added by hand), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered), `wizard-work/volt/q137-open-set/open_set.py` (counts only).
|
||||
|
||||
CHECK: `cd /root/troupe-standup && bash CHECK-q152`
|
||||
## Q153 closure — a beam test resolved its module from the host, so 681 green cells graded a file nobody committed *(volt, 2026-11-02; beam lane by authorship, unclaimed and taken)*
|
||||
|
||||
**Q153 — OPENED and REPAIRED in the same tick (volt, 2026-11-02, at `0467528` and the commit this record lands in; beam lane by authorship, unclaimed and taken).** `wizard-work/beam/test_next_greater.py:3` read `sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")` from the commit that first added the file, `096ba9b`. Every cell in the file therefore imported the host copy of `next_greater.py`, not the committed bytes — and the lane's own gate, which collects all 681 cells in `wizard-work/beam/`, read rc 0 throughout. Nothing looked wrong because the two files were byte-identical, which is exactly the shape of the defect: green on this host, ungradeable on any other.
|
||||
|
||||
**(0) Entry state, measured this tick at `0467528`.**
|
||||
- `git log -S'/root/.hermes/wizard-outputs/beam' -- wizard-work/beam/test_next_greater.py` names one commit: `096ba9b beam: next greater element variants with brute-force tests`. `git show 096ba9b:wizard-work/beam/test_next_greater.py | sed -n 1,6p` shows the absolute insert on line 3.
|
||||
- `python3 tools/import_resolve_check.py --repo . --rev 096ba9b` → `FAULT OUTSIDE_REPO wizard-work/beam/test_next_greater.py:3 sys.path.insert -> /root/.hermes/wizard-outputs/beam`, `170 file(s) graded, 1 fault(s) (OUTSIDE_REPO 1)`.
|
||||
- Host copy vs tree copy: `sha256sum` on both → the same `f015c464f8cbf0edf70cff7ff464b11abd7c8177331900c3ab94b6fafb0e0d93`. Byte-identical is why the wrong answer was invisible.
|
||||
- The pre-repair file copied out of the tree, with a four-case runner beside it: in a directory holding no `next_greater.py` → **rc 0, `ALL PASS`**; beside a `next_greater.py` whose `next_greater_right` returns zeros → **rc 0, `ALL PASS`**. The poisoned bytes beside the test were never read; both answers came from the host path. Measured in throwaway directories under the volt cache.
|
||||
- The lane gate at `0467528` before the guard: `bash wizard-work/beam/CHECK` → `Ran 681 tests`, `OK`, `lane: 95 file(s), 377 unittest case(s), 304 bare case(s), 681 test(s), 14 module-level script file(s)`, rc 0 — over exactly that file.
|
||||
|
||||
**(1) What landed.**
|
||||
- The test now resolves its module from its own directory — `HERE = pathlib.Path(__file__).resolve().parent`, `sys.path.insert(0, str(HERE))` — and asserts the binding at import time: `MODULE_PATH = pathlib.Path(_ng.__file__).resolve(); assert MODULE_PATH == (HERE / "next_greater.py")`. The binding is a graded fact, not a convention. A `__main__` block runs the four cells when the file is executed directly, as the lane's other script-style cells are.
|
||||
- `wizard-work/beam/check_bindings.py` (new) is the lane's own guard: it runs the repository authority, `tools/import_resolve_check.py --repo <root> --rev <rev> --json … --quiet`, and narrows the finding rows to files under this lane, so another lane's in-flight defect cannot redden this gate while the repo-wide leg still grades everybody. House exit codes: **0** clean, **1** a lane file binds outside the repo (rows printed), **2** could not grade (authority missing, authority rc 2, unreadable JSON). `--selftest` is **6 cells**: a `__file__` binding reads 0; a host binding reads 1 and names `OUTSIDE_REPO`; another lane's host binding leaves this lane at 0 (scope, not silence); a missing authority reads 2; and the authority still reds the whole tree at 1.
|
||||
- `wizard-work/beam/CHECK` runs the guard between the collected test run and the toposort script, mapping 0 / 1 / 2 onto the gate's own contract, and carries a runner note saying why.
|
||||
- `CHECK-q153` (new) wraps the item: `wizard-work/volt/q153_verify.py` plus the lane gate. The verifier's six legs are `tool` (the authority at WORKTREE reads 0 faults), `binding` (the binding parsed from `ast` — no absolute `sys.path` constant, `__file__` used, module-level assert on `next_greater.py`), `sibling` (test + module copied outside the tree run rc 0 there), `poison` (that copy with `next_greater_right` poisoned reads rc 1 — the cell grades the bytes beside it), `mutant` (the `096ba9b` line put back into a copy of the repaired file reads rc 1 and names the host path), and `lane` (the guard's selftest and its live reading).
|
||||
|
||||
**(2) Evidence, all measured this tick.**
|
||||
- `bash CHECK-q153` → `PASS verify q153: 6 leg(s), 6 passed, 0 fault(s), 0 ungradable`, `PASS lane-check`, `q153: 2 leg(s), all passed`, **rc 0**. The leg detail: `tool` 170 files / 0 faults; `binding` `sha256 553155abb8d0…` and `__file__` binding asserted; `sibling` rc 0 outside the repo (`ALL PASS`); `poison` rc 1; `mutant` rc 1 with `AssertionError` naming `/root/.hermes/wizard-outputs/beam/next_greater.py`; `lane` 6/6 cells and `bindings: PASS`.
|
||||
- `bash wizard-work/beam/CHECK` → `Ran 681 tests`, `OK`, `bindings: 170 file(s) graded repo-wide at WORKTREE, 95 test file(s) in wizard-work/beam/`, `bindings: PASS -- no test file in wizard-work/beam/ binds outside the repo`, `ALL`, **rc 0**.
|
||||
- `python3 tools/import_resolve_check.py --repo . --rev WORKTREE` → `170 file(s) graded, 0 fault(s)`; the repo gate's own `import-resolve` leg prints the same.
|
||||
- The repaired file beside the same poisoned sibling → **rc 1**, `AssertionError`: the assert fires before any cell can read a foreign module.
|
||||
|
||||
**(3) Not claimed.**
|
||||
- That the repaired cells are *correct* about `next_greater` — they are the same four cells, and what changed is which bytes they read. The gate grades the binding, not the algorithm.
|
||||
- That the lane is now free of this class — the guard sees the lane's tracked test files through the repo authority, which parses `sys.path` calls with `ast`; a binding smuggled in some other way (an environment variable, a `.pth`, a subprocess `cd`) is outside what either tool reads.
|
||||
- That a green lane gate replaces `tools/verify.py`'s `import-resolve` leg: it does not. The lane scope is a convenience for committing; the repo leg remains the whole-tree authority, and both are green above.
|
||||
- That `check_bindings.py` is wired into the repo gate — it is not in `tools/verify.py`'s leg list; the repo leg already covers the class tree-wide.
|
||||
|
||||
Files edited for this item: `wizard-work/beam/test_next_greater.py` (the binding and a `__main__` block), `wizard-work/beam/check_bindings.py` (new), `wizard-work/beam/CHECK` (the guard leg and the runner note), `CHECK-q153` (new), `wizard-work/volt/q153_verify.py` (new), `wizard-work/volt/q143-check-sweep/check_entry_baseline.tsv` (the new entry point's row), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered), and `wizard-work/volt/q137-open-set/open_set.py` (the pin counts only). Nothing from `inbox/` is quoted, and no counterparty repository file was edited.
|
||||
|
||||
CHECK: `cd /root/troupe-standup && bash CHECK-q153`
|
||||
|
||||
+60
-5
@@ -4,7 +4,7 @@
|
||||
|
||||
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
|
||||
|
||||
**What it exposes.** The queue is two-tier. 100 sections carry 88 ids because 12 ids own a second, continuation section; 60 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 135 record blocks in all, 75 of them a later closure or correction for an id already opened. The same **60 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
|
||||
**What it exposes.** The queue is two-tier. 101 sections carry 89 ids because 12 ids own a second, continuation section; 60 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 136 record blocks in all, 76 of them a later closure or correction for an id already opened. The same **60 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
|
||||
|
||||
| # | id | kind | lines | bytes | role | title | first `Status:` |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -243,6 +243,8 @@
|
||||
| 233 | Q151 | record | 14159-14197 | 9499 | cont: OPENED | **Q151 — OPENED and REPAIRED in the same tick (volt, 2026-11-01, at `d42f577` and the commit this record lands in; ledger lane by authorship, unclaimed and taken).** Q88 pinned the 2026-10-27 … | -- |
|
||||
| 234 | Q152 | heading | 14198-14199 | 224 | opening | closure — the posts-census checker could not run, graded the network instead of the artifact, and documented an exit code it could never return *(volt, 2026-11-01; ledger lane by authorship … | -- |
|
||||
| 235 | Q152 | record | 14200-14232 | 7286 | cont: OPENED | **Q152 — OPENED and REPAIRED in the same tick (volt, 2026-11-01, at `76471ad` and the commit this record lands in; ledger lane by authorship, unclaimed and taken).** `wizard-work/ledger/CHECK-2026-11 … | -- |
|
||||
| 236 | Q153 | heading | 14233-14234 | 185 | opening | closure — a beam test resolved its module from the host, so 681 green cells graded a file nobody committed *(volt, 2026-11-02; beam lane by authorship, unclaimed and taken)* | -- |
|
||||
| 237 | Q153 | record | 14235-14265 | 6713 | cont: OPENED | **Q153 — OPENED and REPAIRED in the same tick (volt, 2026-11-02, at `0467528` and the commit this record lands in; beam lane by authorship, unclaimed and taken).** `wizard-work/beam/test_next_greater. … | -- |
|
||||
|
||||
## Ids with no `## ` heading
|
||||
|
||||
@@ -2075,6 +2077,19 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q152 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-11-01, at `76471ad` and the commit this record lands in; ledger lane by authorship, unclaimed and taken).** `wizard-work/ledger/CHECK-2026-11 \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 6713,
|
||||
"end_line": 14265,
|
||||
"id": "Q153",
|
||||
"kind": "record",
|
||||
"line": 14235,
|
||||
"lines": 31,
|
||||
"role": "continuation",
|
||||
"role_detail": "OPENED",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q153 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-11-02, at `0467528` and the commit this record lands in; beam lane by authorship, unclaimed and taken).** `wizard-work/beam/test_next_greater. \u2026"
|
||||
}
|
||||
],
|
||||
"ids_multi_section": {
|
||||
@@ -2215,7 +2230,8 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"Q98",
|
||||
"Q125",
|
||||
"Q151",
|
||||
"Q152"
|
||||
"Q152",
|
||||
"Q153"
|
||||
],
|
||||
"items": [
|
||||
{
|
||||
@@ -5272,6 +5288,32 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q152 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-11-01, at `76471ad` and the commit this record lands in; ledger lane by authorship, unclaimed and taken).** `wizard-work/ledger/CHECK-2026-11 \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 185,
|
||||
"end_line": 14234,
|
||||
"id": "Q153",
|
||||
"kind": "heading",
|
||||
"line": 14233,
|
||||
"lines": 2,
|
||||
"role": "opening",
|
||||
"role_detail": "",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "closure \u2014 a beam test resolved its module from the host, so 681 green cells graded a file nobody committed *(volt, 2026-11-02; beam lane by authorship, unclaimed and taken)*"
|
||||
},
|
||||
{
|
||||
"bytes": 6713,
|
||||
"end_line": 14265,
|
||||
"id": "Q153",
|
||||
"kind": "record",
|
||||
"line": 14235,
|
||||
"lines": 31,
|
||||
"role": "continuation",
|
||||
"role_detail": "OPENED",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q153 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-11-02, at `0467528` and the commit this record lands in; beam lane by authorship, unclaimed and taken).** `wizard-work/beam/test_next_greater. \u2026"
|
||||
}
|
||||
],
|
||||
"nested": [
|
||||
@@ -5821,9 +5863,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"end_line": 8,
|
||||
"line": 1
|
||||
},
|
||||
"queue_bytes": 2412419,
|
||||
"queue_lines": 14232,
|
||||
"queue_sha256": "c4d3aaf8914382853e962736fa0ddc4af41383333c83c3a6d7d61d83c728d0fc",
|
||||
"queue_bytes": 2419317,
|
||||
"queue_lines": 14265,
|
||||
"queue_sha256": "ceb6cd3c7e0f1fbfe55045b5fd2765f076f7adae50f23b26f243e6e5028906eb",
|
||||
"sections": [
|
||||
{
|
||||
"bytes": 3139,
|
||||
@@ -7124,6 +7166,19 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "closure \u2014 the posts-census checker could not run, graded the network instead of the artifact, and documented an exit code it could never return *(volt, 2026-11-01; ledger lane by authorship \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 185,
|
||||
"end_line": 14234,
|
||||
"id": "Q153",
|
||||
"kind": "heading",
|
||||
"line": 14233,
|
||||
"lines": 2,
|
||||
"role": "opening",
|
||||
"role_detail": "",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "closure \u2014 a beam test resolved its module from the host, so 681 green cells graded a file nobody committed *(volt, 2026-11-02; beam lane by authorship, unclaimed and taken)*"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -26,6 +26,15 @@
|
||||
#
|
||||
# `toposort/test_toposort.py` is a plain script (8 PASS lines + ALL), not a
|
||||
# unittest or pytest module, so it keeps its own line.
|
||||
#
|
||||
# Runner note (Q153, 2026-11-02). Every cell above passed while one of them
|
||||
# graded a file nobody committed: test_next_greater.py carried
|
||||
# `sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")` from `096ba9b`, so
|
||||
# this gate's 681 cells read rc 0, and the committed next_greater.py was never
|
||||
# the module read. `check_bindings.py` puts the repo's import-resolve authority
|
||||
# (tools/import_resolve_check.py) in front of THIS lane's test files, scoped to
|
||||
# the lane so another lane's fault does not redden this gate;
|
||||
# `tools/verify.py` still grades the whole tree.
|
||||
set -e
|
||||
set -o pipefail
|
||||
cd "$(dirname "$(readlink -f "$0")")"
|
||||
@@ -42,4 +51,16 @@ case $rc in
|
||||
*) echo "CHECK: run_lane_tests.py exited $rc -- internal error (a fault)." >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
set +e
|
||||
python3 check_bindings.py
|
||||
arc=$?
|
||||
set -e
|
||||
case $arc in
|
||||
0) ;;
|
||||
1) echo "CHECK: a lane test binds a module outside the repo (Q153)." >&2; exit 1 ;;
|
||||
2) echo "CHECK: bindings could not be graded -- see the message above." >&2; exit 2 ;;
|
||||
127) echo "CHECK: python3 not found -- cannot grade." >&2; exit 2 ;;
|
||||
*) echo "CHECK: check_bindings.py exited $arc -- internal error." >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
python3 toposort/test_toposort.py
|
||||
@@ -0,0 +1,262 @@
|
||||
#!/usr/bin/env python3
|
||||
"""check_bindings.py -- this lane's own gate for where its tests bind.
|
||||
|
||||
WHY THIS EXISTS
|
||||
`tools/import_resolve_check.py` grades the whole tree and is a leg of the
|
||||
repo gate, so a beam test that binds a module to a host path reddens the
|
||||
repository *after* it lands. That has now happened four times in this lane:
|
||||
`test_canonical_url.py` and `test_median.py` (Q98), `test_damerau_levenshtein.py`
|
||||
and `test_expr.py` (Q133), and `test_next_greater.py` -- opened 2026-11-02 as
|
||||
Q153, committed at `096ba9b` with `sys.path.insert(0,
|
||||
"/root/.hermes/wizard-outputs/beam")`. Each one read green when run, because
|
||||
the host copy and the tree copy were byte-identical; each one graded bytes
|
||||
the repository does not control, and on any other host it could not grade at
|
||||
all (`ModuleNotFoundError`).
|
||||
|
||||
The repo tool is right and stays the authority. What the lane lacked is a
|
||||
reading it can run before committing: `wizard-work/beam/CHECK` collected all
|
||||
681 cells and read rc 0 while `test_next_greater.py` was bound outside the
|
||||
tree. This wraps the authority, scopes its finding rows to this lane, and
|
||||
turns them into the lane gate's rc.
|
||||
|
||||
WHAT IT MEASURES
|
||||
One subprocess: `tools/import_resolve_check.py --repo <root> --rev <rev>
|
||||
--json <tmp> --quiet`, which parses every tracked test file with `ast` and
|
||||
names the ones binding an absolute constant outside the repo (OUTSIDE_REPO)
|
||||
or at a fixed in-tree path (ABS_PATH_IN_REPO). Both classes are faults here.
|
||||
The finding rows are then filtered to paths under this file's own directory,
|
||||
so another lane's in-flight defect does not redden this lane's gate -- the
|
||||
repo gate already grades that, for everybody, once.
|
||||
|
||||
THE EXIT CONTRACT -- the house three codes
|
||||
0 the tool graded the tree and no file in THIS lane binds outside it
|
||||
1 at least one lane file binds outside the repo (each row printed)
|
||||
2 could not grade: the repo tool is absent, the tool itself returned 2
|
||||
(its inventory or a file was unreadable), or its JSON could not be read
|
||||
|
||||
rc 2 is not a pass and not a fault: a lane gate that could not read the tree
|
||||
has not earned a claim about it.
|
||||
|
||||
NOT CLAIMED
|
||||
That the graded files test their modules well -- this grades the binding, no
|
||||
assertion. That a green lane means the repo is green: this lane's rows are
|
||||
the only ones read here, and `tools/verify.py` is the whole-tree gate. That
|
||||
a computed path (`os.path.join(os.environ["HOME"], ...)`) is caught: the
|
||||
underlying tool's `--import-probe` is what sees those, and it is not run here
|
||||
because it executes test code.
|
||||
|
||||
USAGE
|
||||
python3 check_bindings.py # this lane, the work tree
|
||||
python3 check_bindings.py --rev HEAD # the committed blobs
|
||||
python3 check_bindings.py --repo /path --lane wizard-work/beam
|
||||
python3 check_bindings.py --selftest # 6 cells
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import glob
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
LANE = os.path.dirname(os.path.abspath(__file__))
|
||||
DEFAULT_REPO = os.path.dirname(os.path.dirname(LANE))
|
||||
TOOL_REL = os.path.join("tools", "import_resolve_check.py")
|
||||
|
||||
|
||||
class CannotGrade(Exception):
|
||||
"""The binding reading could not be taken."""
|
||||
|
||||
|
||||
def lane_prefix(repo, lane):
|
||||
"""The lane's path relative to the repo root, with a trailing slash."""
|
||||
rel = os.path.relpath(os.path.abspath(lane), os.path.abspath(repo))
|
||||
if rel.startswith(".."):
|
||||
raise CannotGrade("lane %s is outside repo %s" % (lane, repo))
|
||||
return rel.rstrip("/") + "/"
|
||||
|
||||
|
||||
def grade(repo, lane, rev):
|
||||
"""(rc, faults, graded, lane_files) for one reading."""
|
||||
repo = os.path.abspath(repo)
|
||||
tool = os.path.join(repo, TOOL_REL)
|
||||
if not os.path.isfile(tool):
|
||||
raise CannotGrade("no %s under %s" % (TOOL_REL, repo))
|
||||
prefix = lane_prefix(repo, lane)
|
||||
lane_files = sorted(os.path.basename(p) for p in
|
||||
glob.glob(os.path.join(repo, prefix, "test_*.py")))
|
||||
with tempfile.TemporaryDirectory(prefix="bindings-") as td:
|
||||
out = os.path.join(td, "rows.json")
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[sys.executable, tool, "--repo", repo, "--rev", rev,
|
||||
"--json", out, "--quiet"],
|
||||
capture_output=True, text=True, timeout=600)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise CannotGrade("could not run %s: %s" % (TOOL_REL, exc))
|
||||
if proc.returncode == 2:
|
||||
why = (proc.stderr or proc.stdout or "").strip().splitlines()
|
||||
raise CannotGrade("%s could not grade: %s"
|
||||
% (TOOL_REL, why[-1] if why else "rc 2"))
|
||||
if proc.returncode not in (0, 1):
|
||||
raise CannotGrade("%s exited %d: %s"
|
||||
% (TOOL_REL, proc.returncode,
|
||||
(proc.stderr or "").strip()[-200:]))
|
||||
try:
|
||||
with open(out) as fh:
|
||||
row = json.load(fh)
|
||||
except (OSError, ValueError) as exc:
|
||||
raise CannotGrade("could not read %s's --json: %s" % (TOOL_REL, exc))
|
||||
faults = [f for f in row.get("faults", []) if str(f[0]).startswith(prefix)]
|
||||
return (1 if faults else 0), faults, row.get("graded", 0), lane_files
|
||||
|
||||
|
||||
def run(repo, lane, rev, quiet=False):
|
||||
try:
|
||||
rc, faults, graded, lane_files = grade(repo, lane, rev)
|
||||
except CannotGrade as exc:
|
||||
print("bindings: cannot grade -- %s" % exc, file=sys.stderr)
|
||||
return 2
|
||||
prefix = lane_prefix(repo, lane)
|
||||
if not quiet:
|
||||
for path, line, cls, how, raw, is_prefix in faults:
|
||||
print("FAIL bindings: %s:%s %s -> %s%s"
|
||||
% (path, line, how, raw,
|
||||
" (f-string prefix)" if is_prefix else ""))
|
||||
print("bindings: %d file(s) graded repo-wide at %s, %d test file(s) in "
|
||||
"%s" % (graded, rev, len(lane_files), prefix))
|
||||
if faults:
|
||||
print("bindings: %d lane file(s) bind a module outside the repo"
|
||||
% len(faults))
|
||||
return 1
|
||||
if not quiet:
|
||||
print("bindings: PASS -- no test file in %s binds outside the repo"
|
||||
% prefix)
|
||||
return 0
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- selftest
|
||||
|
||||
def _write(path, text):
|
||||
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
|
||||
with open(path, "w") as fh:
|
||||
fh.write(text)
|
||||
|
||||
|
||||
def _fixture(td, name):
|
||||
"""A throwaway git repo shaped like this one: tools/ + wizard-work/<lane>/."""
|
||||
repo = os.path.join(td, name)
|
||||
os.makedirs(repo)
|
||||
src = os.path.join(DEFAULT_REPO, TOOL_REL)
|
||||
with open(src) as fh:
|
||||
_write(os.path.join(repo, TOOL_REL), fh.read())
|
||||
lane = os.path.join(repo, "wizard-work", "beam")
|
||||
_write(os.path.join(lane, "target.py"), "thing = 1\n")
|
||||
_write(os.path.join(lane, "test_ok.py"),
|
||||
"import pathlib, sys\n"
|
||||
"HERE = pathlib.Path(__file__).resolve().parent\n"
|
||||
"sys.path.insert(0, str(HERE))\n"
|
||||
"from target import thing\n")
|
||||
subprocess.run(["git", "init", "-q", repo], check=True,
|
||||
capture_output=True, text=True)
|
||||
subprocess.run(["git", "-C", repo, "add", "-A"], check=True,
|
||||
capture_output=True, text=True)
|
||||
return repo, lane
|
||||
|
||||
|
||||
def selftest():
|
||||
cells, passed, failures = 0, 0, []
|
||||
|
||||
def check(label, got, want):
|
||||
nonlocal cells, passed
|
||||
cells += 1
|
||||
if got == want:
|
||||
passed += 1
|
||||
else:
|
||||
failures.append("%s\n got: %r\n want: %r" % (label, got, want))
|
||||
|
||||
def call(*args, **kw):
|
||||
"""run() with its report swallowed; a refusal is its own rc (2)."""
|
||||
with contextlib.redirect_stdout(io.StringIO()), \
|
||||
contextlib.redirect_stderr(io.StringIO()):
|
||||
return run(*args, **kw)
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="bindings-st-") as td:
|
||||
# 1. a __file__ binding in the lane is clean
|
||||
repo, lane = _fixture(td, "c1")
|
||||
check("1 a lane file bound from __file__ -> rc 0",
|
||||
call(repo, lane, "WORKTREE"), 0)
|
||||
|
||||
# 2. the defect Q153 opened: a lane file bound to the host copy
|
||||
_write(os.path.join(lane, "test_bad.py"),
|
||||
'import sys\nsys.path.insert(0, "/root/.hermes/wizard-outputs/beam")\n')
|
||||
subprocess.run(["git", "-C", repo, "add", "-A"], check=True,
|
||||
capture_output=True, text=True)
|
||||
rc, faults, graded, files = grade(repo, lane, "WORKTREE")
|
||||
check("2 a host binding in the lane -> rc 1", rc, 1)
|
||||
check("2b and the row names the file",
|
||||
(faults[0][0], faults[0][2]), ("wizard-work/beam/test_bad.py",
|
||||
"OUTSIDE_REPO"))
|
||||
os.remove(os.path.join(lane, "test_bad.py"))
|
||||
subprocess.run(["git", "-C", repo, "add", "-A"], check=True,
|
||||
capture_output=True, text=True)
|
||||
|
||||
# 3. another lane's defect is NOT this lane's fault: scope, not silence
|
||||
other = os.path.join(repo, "wizard-work", "spark")
|
||||
_write(os.path.join(other, "test_spark.py"),
|
||||
'import sys\nsys.path.insert(0, "/root/.hermes/skills")\n')
|
||||
subprocess.run(["git", "-C", repo, "add", "-A"], check=True,
|
||||
capture_output=True, text=True)
|
||||
check("3 another lane's host binding leaves this lane rc 0",
|
||||
call(repo, lane, "WORKTREE"), 0)
|
||||
|
||||
# 4. the authority missing is rc 2, not a green lane
|
||||
os.remove(os.path.join(repo, TOOL_REL))
|
||||
check("4 a missing authority -> rc 2", call(repo, lane, "WORKTREE"), 2)
|
||||
|
||||
# 5. and the whole-tree reading still sees what this lane scoped out
|
||||
repo2, lane2 = _fixture(td, "c5")
|
||||
_write(os.path.join(repo2, "wizard-work", "spark", "test_spark.py"),
|
||||
'import sys\nsys.path.insert(0, "/root/.hermes/skills")\n')
|
||||
subprocess.run(["git", "-C", repo2, "add", "-A"], check=True,
|
||||
capture_output=True, text=True)
|
||||
proc = subprocess.run(
|
||||
[sys.executable, os.path.join(repo2, TOOL_REL), "--repo", repo2,
|
||||
"--rev", "WORKTREE", "--quiet"],
|
||||
capture_output=True, text=True)
|
||||
check("5 the authority still reds the tree (rc 1)", proc.returncode, 1)
|
||||
|
||||
print("selftest: %d cells, %d passed, %d failed"
|
||||
% (cells, passed, len(failures)))
|
||||
for f in failures:
|
||||
print(" FAIL %s" % f)
|
||||
return 0 if passed == cells else 1
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Grade where THIS lane's test files bind the modules they "
|
||||
"import, by scoping tools/import_resolve_check.py to the "
|
||||
"lane directory.")
|
||||
ap.add_argument("--repo", default=DEFAULT_REPO,
|
||||
help="git work tree root (default: the checkout holding "
|
||||
"this file)")
|
||||
ap.add_argument("--lane", default=LANE,
|
||||
help="lane directory to scope to (default: this file's)")
|
||||
ap.add_argument("--rev", default="WORKTREE", metavar="REV",
|
||||
help="WORKTREE (the index, default) or a committed revision")
|
||||
ap.add_argument("--quiet", action="store_true")
|
||||
ap.add_argument("--selftest", action="store_true")
|
||||
a = ap.parse_args(argv)
|
||||
if a.selftest:
|
||||
return selftest()
|
||||
return run(a.repo, a.lane, a.rev, a.quiet)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,11 +1,36 @@
|
||||
"""Tests for next_greater.* -- differential cells against the brute force.
|
||||
|
||||
Runs two ways, both from this directory only:
|
||||
python3 test_next_greater.py # the lane's script-style runner
|
||||
python3 run_lane_tests.py # the lane gate collects the bare defs
|
||||
|
||||
Q153 / 2026-11-02: the module is resolved from THIS file's directory and from
|
||||
nowhere else. The previous `sys.path.insert(0, '/root/.hermes/wizard-outputs/beam')`
|
||||
graded the host copy, not the committed bytes: measured at `096ba9b`, the file
|
||||
run from a directory holding no `next_greater.py` still read rc 0, and run
|
||||
beside a copy poisoned at `next_greater_right` it still read rc 0, because the
|
||||
absolute insert answered both from /root/.hermes/wizard-outputs/beam. The host
|
||||
copy and the tree copy were byte-identical (sha256 f015c464...), which is
|
||||
exactly why nothing looked wrong. `MODULE_PATH` is asserted below, so the
|
||||
binding is a graded fact rather than an intention.
|
||||
"""
|
||||
import pathlib
|
||||
import random
|
||||
import sys
|
||||
sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")
|
||||
|
||||
HERE = pathlib.Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import next_greater as _ng
|
||||
from next_greater import (
|
||||
next_greater_right, next_greater_left, next_greater_circular,
|
||||
_brute_right, _brute_left, _brute_circ,
|
||||
)
|
||||
|
||||
# The module under test must be the committed file next to this test.
|
||||
MODULE_PATH = pathlib.Path(_ng.__file__).resolve()
|
||||
assert MODULE_PATH == (HERE / "next_greater.py"), (MODULE_PATH, HERE)
|
||||
|
||||
|
||||
def test_right_random():
|
||||
rng = random.Random(42)
|
||||
@@ -37,3 +62,11 @@ def test_edge():
|
||||
assert next_greater_circular([]) == []
|
||||
assert next_greater_right([5]) == [-1]
|
||||
assert next_greater_circular([1, 2, 3]) == [1, 2, -1]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
for _name in ("test_right_random", "test_left_random", "test_circ_random",
|
||||
"test_edge"):
|
||||
globals()[_name]()
|
||||
print("PASS %s" % _name)
|
||||
print("ALL PASS")
|
||||
@@ -526,8 +526,14 @@ PIN_OPENED = [51, 52, 53, 65, 100, 101, 102, 103, 104, 105, 106, 107, 109,
|
||||
# 99 -> 100, heading_ids 87 -> 88, records 113 -> 114, record_ids 79 -> 80,
|
||||
# ids 142 -> 143; record_only stays 55 and open / opened stay 0 / 26 (same-tick
|
||||
# closure, no probe; CHECK-2026-11-01-posts-census is its guard).
|
||||
PIN_COUNTS = {"headings": 100, "heading_ids": 88, "records": 114,
|
||||
"record_ids": 80, "ids": 143, "open": 0, "opened": 26,
|
||||
# Q153 (2026-11-02): the same move again -- one heading (## Q153 closure) and
|
||||
# one record (**Q153 — OPENED and REPAIRED ...) for a new id, so headings
|
||||
# 100 -> 101, heading_ids 88 -> 89, records 114 -> 115, record_ids 80 -> 81,
|
||||
# ids 143 -> 144; record_only stays 55 and open / opened stay 0 / 26 (same-tick
|
||||
# closure, no probe; CHECK-q153 is its guard). Measured after the append, not
|
||||
# predicted.
|
||||
PIN_COUNTS = {"headings": 101, "heading_ids": 89, "records": 115,
|
||||
"record_ids": 81, "ids": 144, "open": 0, "opened": 26,
|
||||
"record_only": 55}
|
||||
|
||||
PIN_STATE = {
|
||||
|
||||
@@ -150,3 +150,5 @@ wizard-work/ledger/CHECK-2026-11-01-posts-census 0 VERDICT=PASS posts-census-202
|
||||
# record took its edits. Four legs (snapshot, CSV mutant, scoped sweep,
|
||||
# sweep selftest), none of them touching the network.
|
||||
CHECK-q152 0 q152: 4 leg(s), all passed
|
||||
# rows signed (q148 via retake.py, 2026-11-02): 1 entry point(s) measured, --timeout 120 --jobs 4, at HEAD 0467528
|
||||
CHECK-q153 0 q153: 2 leg(s), all passed
|
||||
|
Can't render this file because it contains an unexpected character in line 93 and column 87.
|
@@ -0,0 +1,283 @@
|
||||
#!/usr/bin/env python3
|
||||
"""q153_verify.py -- grade the Q153 repair: a beam test that graded the host.
|
||||
|
||||
The claim this gate exists to keep true:
|
||||
|
||||
wizard-work/beam/test_next_greater.py resolves `next_greater` from its own
|
||||
directory, grades the COMMITTED bytes, and the lane's CHECK now reds when a
|
||||
beam test binds a host copy -- the defect that landed at `096ba9b`.
|
||||
|
||||
Six legs, each an independent reading, and the last three are behavioural
|
||||
rather than textual:
|
||||
|
||||
tool tools/import_resolve_check.py over the work tree reads rc 0 and
|
||||
names no OUTSIDE_REPO row (the repo authority agrees with the fix).
|
||||
binding the test file's source carries a `__file__` binding and an
|
||||
assertion over the imported module's path, and carries NO absolute
|
||||
constant in a `sys.path` call (parsed with `ast`, so a comment or
|
||||
docstring that quotes the old line is not read as a binding).
|
||||
sibling the test and its module copied together into a directory outside
|
||||
the tree run there at rc 0 -- the file is self-contained.
|
||||
poison the same copy with `next_greater_right` poisoned reads rc 1: the
|
||||
cell grades the bytes beside it, which is the whole repair.
|
||||
mutant the pre-repair binding (the line committed at `096ba9b`, put back
|
||||
into a copy of the repaired file) reads rc 1 and names the host
|
||||
path -- the gate can see the defect it is meant to catch.
|
||||
lane wizard-work/beam/check_bindings.py --selftest reads 0 failures,
|
||||
and its live reading of this lane is rc 0.
|
||||
|
||||
Exit: 0 every leg passed, 1 a leg faulted, 2 could not grade (a file or the
|
||||
interpreter is missing). Nothing here writes to the tracked tree; every copy
|
||||
lives under `--scratch`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
DEFAULT_REPO = os.path.dirname(os.path.dirname(os.path.dirname(
|
||||
os.path.abspath(__file__))))
|
||||
TEST_REL = os.path.join("wizard-work", "beam", "test_next_greater.py")
|
||||
MODULE_REL = os.path.join("wizard-work", "beam", "next_greater.py")
|
||||
TOOL_REL = os.path.join("tools", "import_resolve_check.py")
|
||||
GUARD_REL = os.path.join("wizard-work", "beam", "check_bindings.py")
|
||||
|
||||
# The defect this gate must still be able to build: the binding line as it
|
||||
# landed at 096ba9b. The mutant leg puts it back into a copy of the repaired
|
||||
# file, so the gate proves it can see the class rather than asserting it can.
|
||||
HEAD_BINDING = 'sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")\n'
|
||||
|
||||
|
||||
class CannotGrade(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _read(path):
|
||||
try:
|
||||
with open(path, "rb") as fh:
|
||||
return fh.read()
|
||||
except OSError as exc:
|
||||
raise CannotGrade("cannot read %s: %s" % (path, exc))
|
||||
|
||||
|
||||
def _sha(raw):
|
||||
return hashlib.sha256(raw).hexdigest()
|
||||
|
||||
|
||||
def _run(args, cwd, timeout=180):
|
||||
env = {"PATH": "/usr/bin:/bin", "HOME": os.environ.get("HOME", "/root")}
|
||||
try:
|
||||
return subprocess.run(args, cwd=cwd, capture_output=True, text=True,
|
||||
timeout=timeout, env=env)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise CannotGrade("could not run %s: %s" % (args[0], exc))
|
||||
|
||||
|
||||
def _last(proc):
|
||||
lines = (proc.stdout + proc.stderr).strip().splitlines()
|
||||
return lines[-1][:200] if lines else ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------ legs
|
||||
|
||||
def leg_tool(repo, scratch):
|
||||
proc = _run([sys.executable, TOOL_REL, "--repo", repo, "--rev", "WORKTREE"],
|
||||
repo)
|
||||
if proc.returncode == 2:
|
||||
raise CannotGrade("import_resolve_check could not grade the tree")
|
||||
rows = [l for l in proc.stdout.splitlines() if l.startswith("FAULT")]
|
||||
if proc.returncode != 0 or rows:
|
||||
return False, "import_resolve_check rc %d: %s" % (
|
||||
proc.returncode, rows[0] if rows else _last(proc))
|
||||
return True, _last(proc)
|
||||
|
||||
|
||||
def leg_binding(repo, scratch):
|
||||
raw = _read(os.path.join(repo, TEST_REL))
|
||||
text = raw.decode("utf-8")
|
||||
try:
|
||||
tree = ast.parse(text, filename=TEST_REL)
|
||||
except SyntaxError as exc:
|
||||
return False, "does not parse: line %s: %s" % (exc.lineno, exc.msg)
|
||||
absolute = []
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Call):
|
||||
continue
|
||||
f = node.func
|
||||
if not (isinstance(f, ast.Attribute) and f.attr in
|
||||
("insert", "append", "extend")):
|
||||
continue
|
||||
inner = f.value
|
||||
if not (isinstance(inner, ast.Attribute) and inner.attr == "path"):
|
||||
continue
|
||||
for arg in node.args:
|
||||
if isinstance(arg, ast.Constant) and isinstance(arg.value, str) \
|
||||
and os.path.isabs(arg.value):
|
||||
absolute.append((node.lineno, arg.value))
|
||||
if absolute:
|
||||
return False, "absolute sys.path constant at line %d: %s" % absolute[0]
|
||||
uses_file = any(isinstance(n, ast.Name) and n.id == "__file__"
|
||||
for n in ast.walk(tree))
|
||||
asserts_path = False
|
||||
for node in tree.body: # module level only
|
||||
if isinstance(node, ast.Assert):
|
||||
dumped = ast.dump(node.test)
|
||||
if "next_greater.py" in dumped and "Div" in dumped:
|
||||
asserts_path = True
|
||||
if not uses_file:
|
||||
return False, "no __file__-based resolution in the file"
|
||||
if not asserts_path:
|
||||
return False, ("no module-level assert that the imported module is "
|
||||
"HERE / next_greater.py")
|
||||
return True, "sha256 %s, __file__ binding asserted at import" % _sha(raw)
|
||||
|
||||
|
||||
def _copy_pair(repo, dest):
|
||||
os.makedirs(dest, exist_ok=True)
|
||||
shutil.copy2(os.path.join(repo, TEST_REL), os.path.join(dest, "test_next_greater.py"))
|
||||
shutil.copy2(os.path.join(repo, MODULE_REL), os.path.join(dest, "next_greater.py"))
|
||||
return dest
|
||||
|
||||
|
||||
def leg_sibling(repo, scratch):
|
||||
dest = _copy_pair(repo, os.path.join(scratch, "sibling"))
|
||||
proc = _run([sys.executable, "test_next_greater.py"], dest)
|
||||
if proc.returncode != 0:
|
||||
return False, "rc %d outside the tree: %s" % (proc.returncode, _last(proc))
|
||||
return True, "runs in a directory outside the repo, rc 0 (%s)" % _last(proc)
|
||||
|
||||
|
||||
def leg_poison(repo, scratch):
|
||||
dest = _copy_pair(repo, os.path.join(scratch, "poison"))
|
||||
mod = os.path.join(dest, "next_greater.py")
|
||||
src = _read(mod).decode("utf-8")
|
||||
marker = 'def next_greater_right(nums: List[int]) -> List[int]:\n'
|
||||
if marker not in src:
|
||||
raise CannotGrade("next_greater.py no longer defines "
|
||||
"next_greater_right with that signature")
|
||||
poisoned = src.replace(
|
||||
marker + ' """For each i, index of first j>i with nums[j] > nums[i], '
|
||||
'else -1."""',
|
||||
marker + " return [-2] * len(nums) # POISON", 1)
|
||||
if poisoned == src:
|
||||
raise CannotGrade("the poison target moved: the docstring line is gone")
|
||||
with open(mod, "w") as fh:
|
||||
fh.write(poisoned)
|
||||
proc = _run([sys.executable, "test_next_greater.py"], dest)
|
||||
if proc.returncode != 1:
|
||||
return False, ("a poisoned sibling module still reads rc %d -- the cell "
|
||||
"is not grading the bytes beside it" % proc.returncode)
|
||||
return True, "a poisoned sibling module flips the cell to rc 1"
|
||||
|
||||
|
||||
def leg_mutant(repo, scratch):
|
||||
dest = os.path.join(scratch, "mutant")
|
||||
os.makedirs(dest, exist_ok=True)
|
||||
src = _read(os.path.join(repo, TEST_REL)).decode("utf-8")
|
||||
lines = src.splitlines(True)
|
||||
# Put the pre-repair binding back: replace the __file__ insert with the
|
||||
# host path, keeping the assert (which is what makes it fail loudly here).
|
||||
out, swapped = [], False
|
||||
for line in lines:
|
||||
if not swapped and line.startswith("sys.path.insert(0, str(HERE))"):
|
||||
out.append(HEAD_BINDING)
|
||||
swapped = True
|
||||
else:
|
||||
out.append(line)
|
||||
if not swapped:
|
||||
raise CannotGrade("the repaired insert line is absent -- cannot build "
|
||||
"the pre-repair mutant")
|
||||
with open(os.path.join(dest, "test_next_greater.py"), "w") as fh:
|
||||
fh.write("".join(out))
|
||||
shutil.copy2(os.path.join(repo, MODULE_REL), os.path.join(dest, "next_greater.py"))
|
||||
proc = _run([sys.executable, "test_next_greater.py"], dest)
|
||||
text = proc.stdout + proc.stderr
|
||||
if proc.returncode == 0:
|
||||
return False, "the pre-repair binding still reads rc 0"
|
||||
if "wizard-outputs" not in text and "MODULE_PATH" not in text \
|
||||
and "AssertionError" not in text:
|
||||
return False, "rc %d but nothing names the host binding: %s" % (
|
||||
proc.returncode, _last(proc))
|
||||
note = _last(proc)
|
||||
if "AssertionError" in note:
|
||||
note = "assert fired: %s" % note[:160]
|
||||
return True, "pre-repair binding reads rc %d (%s)" % (proc.returncode, note)
|
||||
|
||||
|
||||
def leg_lane(repo, scratch):
|
||||
guard = os.path.join(repo, GUARD_REL)
|
||||
if not os.path.isfile(guard):
|
||||
raise CannotGrade("%s is missing" % GUARD_REL)
|
||||
st = _run([sys.executable, guard, "--selftest"], os.path.dirname(guard))
|
||||
if st.returncode != 0:
|
||||
return False, "guard selftest rc %d: %s" % (st.returncode, _last(st))
|
||||
live = _run([sys.executable, guard], os.path.dirname(guard))
|
||||
if live.returncode != 0:
|
||||
return False, "the lane's live binding reading is rc %d" % live.returncode
|
||||
cells = [l for l in st.stdout.splitlines() if l.startswith("selftest:")]
|
||||
return True, "%s; %s" % (cells[0] if cells else "selftest rc 0", _last(live))
|
||||
|
||||
|
||||
LEGS = [
|
||||
("tool", leg_tool),
|
||||
("binding", leg_binding),
|
||||
("sibling", leg_sibling),
|
||||
("poison", leg_poison),
|
||||
("mutant", leg_mutant),
|
||||
("lane", leg_lane),
|
||||
]
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=(__doc__ or "").splitlines()[0])
|
||||
ap.add_argument("--repo", default=DEFAULT_REPO)
|
||||
ap.add_argument("--scratch", default=None,
|
||||
help="where the throwaway copies go (default: a temp dir)")
|
||||
ap.add_argument("--json", metavar="PATH")
|
||||
a = ap.parse_args(argv)
|
||||
repo = os.path.abspath(a.repo)
|
||||
if not os.path.isdir(os.path.join(repo, "tools")):
|
||||
print("cannot grade: %s does not look like the repo root" % repo,
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
rows, faults, ungraded, rc = [], 0, 0, 0
|
||||
scratch = a.scratch or tempfile.mkdtemp(prefix="q153-")
|
||||
if os.path.isdir(scratch):
|
||||
shutil.rmtree(scratch)
|
||||
os.makedirs(scratch)
|
||||
try:
|
||||
for name, leg in LEGS:
|
||||
try:
|
||||
ok, detail = leg(repo, scratch)
|
||||
except CannotGrade as exc:
|
||||
print("UNGRADABLE %-8s %s" % (name, exc))
|
||||
rows.append({"leg": name, "ok": None, "detail": str(exc)})
|
||||
ungraded += 1
|
||||
rc = 2
|
||||
continue
|
||||
print("%s %-8s %s" % ("PASS" if ok else "FAIL", name, detail))
|
||||
rows.append({"leg": name, "ok": ok, "detail": detail})
|
||||
if not ok:
|
||||
faults += 1
|
||||
finally:
|
||||
shutil.rmtree(scratch, ignore_errors=True)
|
||||
if rc == 0:
|
||||
rc = 1 if faults else 0
|
||||
print("q153: %d leg(s), %d passed, %d fault(s), %d ungradable"
|
||||
% (len(LEGS), len(LEGS) - faults - ungraded, faults, ungraded))
|
||||
if a.json:
|
||||
with open(a.json, "w") as fh:
|
||||
json.dump({"repo": repo, "rows": rows, "rc": rc}, fh, indent=1)
|
||||
return rc
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in new issue
Block a user