diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index f1ab1af..1eae7c3 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -13631,3 +13631,21 @@ CHECK: python3 wizard-work/spark/test_json_sha256_validator.py **Not claimed:** that the dated `career-lane/` cells are correct — they are dated-wrong, which the addendum distinguishes from corrected, and **no `career-lane/` byte was edited for this item** (the addendum's clause 1 is respected, so clause 2 is graded over a tree that deliberately still contains the quotes); that this fourth reading proves the 30-day rule by itself — it confirms a prediction from Q116's three readings, and two samples cannot separate "build + 30 d" from "build + any constant", which was never measured from the generator; that `presence` and `validThrough` are one field; that the block unit is clause-precise (a block that quotes and corrects itself reads as a correction, coarsely, and is printed); that the pipe's ruling is overturned — this operationalizes it, and a different reading of clause 2 (delete the dated lines, or accept the contradiction) remains the pipe's to make. Files edited for this item: `wizard-work/volt/opentrain_deadline_audit.py` (new), `wizard-work/volt/CHECK-q116` (new), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). No `career-lane/` or `replies/` file was edited; nothing from `inbox/` is quoted. Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q116.md`. CHECK: bash wizard-work/volt/CHECK-q116 + +**Q128 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/CHECK` graded zero of the lane's 570 cells on this host: Q105 pointed the gate at pytest, pytest is not installed on the interpreter this shell resolves, and `unittest discover` collects only 353 of them. The gate now collects both styles on the standard library, and the same cells grade with or without pytest.** + +**(1) Entry reading at HEAD `210f3ec`.** `python3 -m pytest --version` → `/root/.hermes/installs/76f6e7e5145b5f39/environments/4d67c7b3c02c44279c5af04b35dae120/venv/bin/python3: No module named pytest`, rc **1**; a scratch copy of the lane with the committed `CHECK` restored reads rc **2** `CHECK: pytest not importable -- cannot grade.` — honest, and **0** cells run. Four files imported pytest outright (`git grep -l "import pytest" HEAD -- wizard-work/beam/`: `test_ratelimit.py`, `test_semver.py`, `test_tree_root.py`, `test_weighted_interval_scheduling.py`), so the lane's own test files could not import on that interpreter at all. + +**(2) The collector.** `wizard-work/beam/run_lane_tests.py` (new, 262 lines, sha256 `979049e92a29192969ddbf2965dec0dd4dbb5270d407f69e610f7943e080292e`), standard library only: every top-level `test_*.py` imported by path under a unique module key (two files sharing a basename cannot collide), `unittest.TestLoader` for the `TestCase` classes, `unittest.FunctionTestCase` for every zero-argument module-level `def test_*()`, an unimportable module is an IMPORT-FAULT (rc 1) not a skip, and a module whose import-time transcript prints `FAIL ` or `SOME FAILED` is a SCRIPT-FAULT (rc 1) because this lane's script-style files report by printing and only exit non-zero from a `__main__` guard that never runs under an import. `--list` prints the collection without running it; exit codes follow `tools/`: 0 clean, 1 graded fault, 2 could not grade. + +**(3) Counts, measured.** In `wizard-work/beam/`, `python3 -m unittest discover -p 'test_*.py'` → `Ran 353 tests … OK`; `python3 run_lane_tests.py --list` → `collection: 77 file(s), 353 unittest case(s), 217 bare case(s), 570 total, 10 module-level script file(s), 0 import fault(s), 0 script fault(s)`; `bash wizard-work/beam/CHECK` → rc **0** measured without a pipe, `570 test(s) … 0 failure(s), 0 error(s), 0 import fault(s), 0 script fault(s)`, then the untouched `toposort/test_toposort.py` line. The 217-cell gap is exactly the bare-style half Q105 named as ungraded. + +**(4) The four pytest files.** `wizard-work/beam/beam_assert.py` (new, sha256 `00684650cb5b7ccd9f1eb557d7e8867b993a9028f44d37199e71909b97c4f999`) supplies the only two names used — a `raises` context manager and `fail`; `pytest.raises(X)` → `raises(X)`, `pytest.fail` → `fail`, and `test_ratelimit.py`'s `monkeypatch` cell is now `unittest.mock.patch.object`, so the one cell no runner in this lane could ever collect is collected. No cell deleted, skipped or weakened; `grep -rln "import pytest" wizard-work/beam/` → **0** files. + +**(5) Mutants, in scratch copies and never the shared tree** (`/root/.hermes/profiles/volt/cache/scratch/q128_mutants.py`, full lane per copy, each row `bash CHECK`): control rc **0**, 570 tests; committed HEAD `CHECK` restored rc **2**; one broken bare `def test_*` appended rc **1** (571 collected, `1 failure(s)`, `lane: FAULT`); `test_semver.py` restored to HEAD so it imports pytest rc **1** (561 collected, `1 import fault(s)`); one failing `TestCase` appended rc **1** (571, `1 failure(s)`); a script file printing `FAIL` rc **1** (`1 script fault(s)`); an empty lane directory rc **2**. Battery `AS EXPECTED`, driver rc 0. + +**(6) The transcript rule has two barriers** (`…/q128_bisect_mutant.py`). One expectation in `test_bisect_bounds.py` flipped false (`check("lb 3", lower_bound(s,3)==2)` → `==3`) in a copy: `SCRIPT-FAIL test_bisect_bounds.py`, rc **1**, `1 script fault(s)`. The same tree with the rule deleted from a copy of the runner: rc **1** still, `FAIL: test_script_that_prints_FAIL_is_a_fault (test_run_lane_tests.TestRunnerContract…)` — the runner's own cell reds on its own mutation, so the rule cannot be removed quietly. The cells file `test_run_lane_tests.py` (new, sha256 `eb1dea05bf0a687660b972a08a9a465070e56636661c02b1de7f0b88683925ef`) builds throwaway lanes in temp dirs and invokes the runner as a subprocess: `Ran 10 tests … OK`, rc **0**. + +**Not claimed:** that pytest is absent everywhere — only on this shell's interpreter (path above), and the runner makes that irrelevant in either direction; that the totals are pinned — `CHECK` prints them each run, and Q105's `407` is a dated count, not today's; that the four rewritten files differ from their old cells in anything but the assertion surface (the diff is `pytest.raises`→`raises`, `pytest.fail`→`fail`, `monkeypatch`→`mock.patch.object`, no cell dropped); that Q111's repair is this item — it landed as its own record at `a6491d7`, and the zero-length cells in `test_weighted_interval_scheduling.py` now run under this gate and pass, which is the only reading taken here; that other lanes' pytest-dependent gates are fixed — unmeasured; that `toposort/test_toposort.py` is collected by the runner — it is not, it keeps its own `CHECK` line (8 PASS lines + `ALL`) as before. Files edited for this item: `wizard-work/beam/CHECK`, `wizard-work/beam/run_lane_tests.py` (new), `wizard-work/beam/beam_assert.py` (new), `wizard-work/beam/test_run_lane_tests.py` (new), `wizard-work/beam/test_ratelimit.py`, `wizard-work/beam/test_semver.py`, `wizard-work/beam/test_tree_root.py`, `wizard-work/beam/test_weighted_interval_scheduling.py`, `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted. Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q128.md`. + +CHECK: bash wizard-work/beam/CHECK diff --git a/docs/queue-index.md b/docs/queue-index.md index a45f92a..f786df3 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -4,7 +4,7 @@ **What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it. -**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 38 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 102 record blocks in all, 64 of them a later closure or correction for an id already opened. The same **38 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. +**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 39 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 103 record blocks in all, 64 of them a later closure or correction for an id already opened. The same **39 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. | # | id | kind | lines | bytes | role | title | first `Status:` | |---|---|---|---|---|---|---|---| @@ -206,7 +206,8 @@ | 196 | Q110 | record | 13557-13576 | 8439 | cont: DONE | **Q110 — DONE (volt, 2026-10-30, at the commit this record lands in; quill lane by authorship, unclaimed and taken). `wizard-work/quill/QUICKSTART.md` no longer prints a command nobody ran: every … | -- | | 197 | Q127 | record | 13577-13598 | 9707 | record: BOLD-LEAD | **Q127 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). A ten-file sweep put Q109's and Q120's pre-repair … | -- | | 198 | Q80 | record | 13599-13616 | 7451 | cont: DONE | **Q80 — DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest … | -- | -| 199 | Q116 | record | 13617-13633 | 6411 | cont: DONE | **Q116 — DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The grep half of the amended done-means now has an instrument that can fail, it … | -- | +| 199 | Q116 | record | 13617-13634 | 6412 | cont: DONE | **Q116 — DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The grep half of the amended done-means now has an instrument that can fail, it … | -- | +| 200 | Q128 | record | 13635-13651 | 6006 | record: BOLD-LEAD | **Q128 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/CHECK` graded zero of the lane's 570 cells on … | -- | ## Ids with no `## ` heading @@ -230,13 +231,13 @@ Opened as records inside another section. `line` is the record's own opening lin | Q102 | 12985 | 12985 | 10 | Q100 | BULLET+BOLD | BOLD-LEAD | | Q103 | 13010 | 13010 | 18 | Q102 | BULLET+BOLD | BOLD-LEAD | | Q104 | 13103 | 13103 | 18 | Q101 | BULLET+BOLD | BOLD-LEAD | -| Q105 | 13135 | 13135 | 17 | Q104 | BULLET+BOLD | BOLD-LEAD | +| Q105 | 13135 | 13135 | 20 | Q104 | BULLET+BOLD | BOLD-LEAD | | Q106 | 13171 | 13171 | 14 | Q105 | BULLET+BOLD | BOLD-LEAD | | Q107 | 13189 | 13189 | 18 | Q106 | BULLET+BOLD | BOLD-LEAD | | Q108 | 13267 | 13263 | 3 | Q106 | BULLET+BOLD | BOLD-LEAD | | Q109 | 13270 | 13270 | 23 | Q104 | BULLET+BOLD | BOLD-LEAD | | Q110 | 13272 | 13272 | 6 | Q109 | BULLET+BOLD | BOLD-LEAD | -| Q111 | 13274 | 13274 | 7 | Q110 | BULLET+BOLD | BOLD-LEAD | +| Q111 | 13274 | 13274 | 8 | Q110 | BULLET+BOLD | BOLD-LEAD | | Q112 | 13276 | 13276 | 6 | Q111 | BULLET+BOLD | BOLD-LEAD | | Q113 | 13278 | 13278 | 3 | Q112 | BULLET+BOLD | BOLD-LEAD | | Q114 | 13280 | 13280 | 5 | Q113 | BULLET+BOLD | BOLD-LEAD | @@ -252,6 +253,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q124 | 13350 | 13350 | 3 | Q95 | BULLET+BOLD | BOLD-LEAD | | Q126 | 13545 | 13545 | 2 | Q113 | BULLET+BOLD | BOLD-LEAD | | Q127 | 13577 | 13577 | 1 | Q110 | BULLET+BOLD | BOLD-LEAD | +| Q128 | 13635 | 13635 | 1 | Q116 | BULLET+BOLD | BOLD-LEAD | **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. @@ -1577,17 +1579,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q80 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest \u2026" }, { - "bytes": 6411, - "end_line": 13633, + "bytes": 6412, + "end_line": 13634, "id": "Q116", "kind": "record", "line": 13617, - "lines": 17, + "lines": 18, "role": "continuation", "role_detail": "DONE", "status": null, "status_line": null, "title": "**Q116 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The grep half of the amended done-means now has an instrument that can fail, it \u2026" + }, + { + "bytes": 6006, + "end_line": 13651, + "id": "Q128", + "kind": "record", + "line": 13635, + "lines": 17, + "role": "record", + "role_detail": "BOLD-LEAD", + "status": null, + "status_line": null, + "title": "**Q128 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/CHECK` graded zero of the lane's 570 cells on \u2026" } ], "ids_multi_section": { @@ -4300,17 +4315,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q80 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest \u2026" }, { - "bytes": 6411, - "end_line": 13633, + "bytes": 6412, + "end_line": 13634, "id": "Q116", "kind": "record", "line": 13617, - "lines": 17, + "lines": 18, "role": "continuation", "role_detail": "DONE", "status": null, "status_line": null, "title": "**Q116 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The grep half of the amended done-means now has an instrument that can fail, it \u2026" + }, + { + "bytes": 6006, + "end_line": 13651, + "id": "Q128", + "kind": "record", + "line": 13635, + "lines": 17, + "role": "record", + "role_detail": "BOLD-LEAD", + "status": null, + "status_line": null, + "title": "**Q128 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; beam lane by authorship, unclaimed and taken). `beam/CHECK` graded zero of the lane's 570 cells on \u2026" } ], "nested": [ @@ -4463,7 +4491,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q105", "line": 13135, - "refs": 17, + "refs": 20, "section": "Q104", "shape": "BULLET+BOLD" }, @@ -4517,7 +4545,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q111", "line": 13274, - "refs": 7, + "refs": 8, "section": "Q110", "shape": "BULLET+BOLD" }, @@ -4655,6 +4683,15 @@ The block below is the parse this document renders, byte for byte. `--check` com "refs": 1, "section": "Q110", "shape": "BULLET+BOLD" + }, + { + "first_mention": 13635, + "form": "BOLD-LEAD", + "id": "Q128", + "line": 13635, + "refs": 1, + "section": "Q116", + "shape": "BULLET+BOLD" } ], "preamble": { @@ -4662,9 +4699,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 2181163, - "queue_lines": 13633, - "queue_sha256": "e722fac21291a53a9a2e185d70d5aa52c760fd988548aab87b35169fb569b8f4", + "queue_bytes": 2187170, + "queue_lines": 13651, + "queue_sha256": "bc2faf77dbdc9103a0cdfb47301ccb6ebd8ffca9bef1c927ea9f685ed79cdd11", "sections": [ { "bytes": 3139, diff --git a/wizard-work/beam/CHECK b/wizard-work/beam/CHECK index 5b354e7..4734ca8 100755 --- a/wizard-work/beam/CHECK +++ b/wizard-work/beam/CHECK @@ -4,43 +4,42 @@ # Runner note (Q105, 2026-10-28). This lane writes its tests in two styles: # unittest.TestCase subclasses and bare module-level `def test_*` / `assert`. # `python3 -m unittest discover` collects only the first -- 276 of the 407 -# tests in this directory -- so a green CHECK said nothing about the other -# 131, and test_fenwick.py's 8 cells could be broken without moving the gate. -# pytest collects both styles, so pytest is the runner here. +# tests the directory held then -- so a green CHECK said nothing about the +# other 131, and test_fenwick.py's 8 cells could be broken without moving +# the gate. # -# `--import-mode=importlib` is deliberate: two files share the basename -# `test_toposort.py` (this directory and toposort/), and pytest's default -# `prepend` mode aborts collection on that duplicate module name. importlib -# mode keys modules by path instead. `toposort/test_toposort.py` is a plain -# script (8 PASS lines + ALL), not a pytest module, so it keeps its own line. +# Runner note (Q128, 2026-10-30). Q105's repair pointed this gate at pytest, +# which collects both styles but is not installed on every interpreter the +# gate runs on: here `python3 -m pytest --version` fails, so the gate printed +# "pytest not importable -- cannot grade" and exited 2 -- honest, and running +# zero of the lane's cells. The gate now runs `run_lane_tests.py`, which +# collects both styles itself on the standard library alone (TestLoader for +# the TestCase classes, FunctionTestCase for every module-level `def test_*` +# with no parameters, and one import-by-path step that turns a module which +# will not import into a fault), so the same cells run with or without +# pytest. Counts are not pinned here -- the runner prints them every run, +# and `python3 run_lane_tests.py --list` prints the collection alone. # -# Exit codes follow the repo's tools/ contract: 0 clean, 1 graded a fault, -# 2 could not grade. pytest's own codes are mapped onto that contract: -# 0 -> 0 all collected tests passed -# 1 -> 1 a test failed -# 2 -> 1 collection error: a test module does not import (a fault) -# 5 -> 1 no tests collected: the gate is measuring nothing -# A missing pytest is 2, never a silent 0 -- a gate that cannot run the 131 -# tests must not report success. +# Exit codes follow the repo's tools/ contract: 0 clean, 1 graded a fault +# (a failing test, an erroring test, a module that will not import), 2 could +# not grade (no python3, no lane directory, no test files). +# +# `toposort/test_toposort.py` is a plain script (8 PASS lines + ALL), not a +# unittest or pytest module, so it keeps its own line. set -e set -o pipefail cd "$(dirname "$(readlink -f "$0")")" -if ! python3 -m pytest --version >/dev/null 2>&1; then - echo "CHECK: pytest not importable -- cannot grade." >&2 - echo "CHECK: unittest discover would run 276 of 407 tests and skip 131 (Q105)." >&2 - exit 2 -fi - set +e -python3 -m pytest -q -p no:cacheprovider --import-mode=importlib test_*.py +python3 run_lane_tests.py rc=$? set -e case $rc in 0) ;; - 1) echo "CHECK: pytest reported failing test(s)." >&2; exit 1 ;; - 5) echo "CHECK: pytest collected no tests -- the gate is measuring nothing." >&2; exit 1 ;; - *) echo "CHECK: pytest exited $rc -- collection or internal error (a module that will not import is a fault)." >&2; exit 1 ;; + 1) echo "CHECK: lane tests failed -- see the failure list above (Q105/Q128)." >&2; exit 1 ;; + 2) echo "CHECK: could not grade -- see the runner's message above." >&2; exit 2 ;; + 127) echo "CHECK: python3 not found -- cannot grade." >&2; exit 2 ;; + *) echo "CHECK: run_lane_tests.py exited $rc -- internal error (a fault)." >&2; exit 1 ;; esac python3 toposort/test_toposort.py diff --git a/wizard-work/beam/beam_assert.py b/wizard-work/beam/beam_assert.py new file mode 100644 index 0000000..df7f994 --- /dev/null +++ b/wizard-work/beam/beam_assert.py @@ -0,0 +1,35 @@ +"""Stdlib assertion helpers for the beam lane's bare-style test files. + +Why this file exists (Q128, 2026-10-30): four of this lane's test files +imported `pytest` only for `pytest.raises` and one `pytest.fail`. On an +interpreter without pytest those four modules fail at import, so the lane's +own gate could not grade them at all -- `beam/CHECK` exited 2 with +"pytest not importable -- cannot grade" and ran zero of the lane's cells. +The surface actually used is two names, so it lives here on the standard +library instead of behind a third-party package. + +`raises` is a context manager that fails unless the expected exception is +raised, matching `pytest.raises(X)` for the way these files use it (no +`match=`, no `as`). `fail` raises AssertionError with the message, matching +`pytest.fail`. Both work under `unittest`, under `pytest`, and as bare +`assert`-style modules run by `run_lane_tests.py`. +""" + +import contextlib + + +@contextlib.contextmanager +def raises(exc_type): + """Fail unless the block raises ``exc_type`` (or a subclass).""" + try: + yield + except exc_type: + return + raise AssertionError( + "expected %s to be raised; nothing was raised" % (exc_type.__name__,) + ) + + +def fail(msg=""): + """Fail the current test with ``msg``.""" + raise AssertionError(msg) diff --git a/wizard-work/beam/run_lane_tests.py b/wizard-work/beam/run_lane_tests.py new file mode 100644 index 0000000..127d58c --- /dev/null +++ b/wizard-work/beam/run_lane_tests.py @@ -0,0 +1,262 @@ +#!/usr/bin/env python3 +"""run_lane_tests.py -- grade every test_*.py in this lane, stdlib only. + +Why this exists (Q128, 2026-10-30). The lane writes tests in two styles: +`unittest.TestCase` subclasses and bare module-level `def test_*()` functions +whose bodies are plain `assert`s. `python3 -m unittest discover` collects only +the first style. Q105's repair pointed `CHECK` at pytest, which collects both +-- and then the gate could not run at all on an interpreter without pytest: +`beam/CHECK` printed "pytest not importable -- cannot grade" and exited 2, so +the lane's 500-odd cells graded nothing while the gate was technically honest. + +This runner collects both styles itself, in one process, with no third-party +import: + + * every top-level `test_*.py` is imported by file path under a unique module + name, so two files sharing a basename cannot collide; + * `unittest.TestLoader().loadTestsFromModule` picks up the TestCase classes; + * each module-level function named `test_*` that takes no arguments is + wrapped in `unittest.FunctionTestCase` and run in the same suite, so a + failing bare assertion is a failing test, not silence; + * a module that will not import is reported by name as a FAULT (rc 1), never + skipped and never counted as a pass; + * a module whose import-time transcript prints a failed check (`FAIL `, + or the `SOME FAILED` summary -- the `check()` convention this lane's + script-style files use) is a SCRIPT-FAIL, rc 1: those files report by + printing and only exit 1 from their own `__main__` guard, which never runs + under an import, so the transcript is the only evidence they leave; + * module-level stdout/stderr is captured during import and printed only when + that import fails, so the summary is readable. + +Exit codes follow the repo's tools/ contract: + 0 every collected test passed and every module imported + 1 a test failed, a test errored, or a module would not import (a fault) + 2 could not grade: no lane directory, no test files, or a bad argument + +`--list` prints the collection (file, unittest cases, bare cases) and exits 0 +without running anything, so the denominator can be quoted without a run. +""" + +import argparse +import contextlib +import glob +import importlib.util +import inspect +import io +import os +import sys +import traceback +import unittest + +DEFAULT_LANE = os.path.dirname(os.path.abspath(__file__)) + + +def discover(lane): + return sorted(glob.glob(os.path.join(lane, "test_*.py"))) + + +def module_name(path, stem): + """A sys.modules key for one test file: the file stem when free.""" + existing = sys.modules.get(stem) + if existing is None: + return stem + if os.path.abspath(getattr(existing, "__file__", "") or "") == os.path.abspath(path): + return stem + return "_beam_lane_%s_%08x" % (stem, abs(hash(os.path.abspath(path))) % (1 << 32)) + + +def load_module(path): + """Import one test file by path; failure is raised as _ImportFault.""" + stem = os.path.splitext(os.path.basename(path))[0] + modname = module_name(path, stem) + spec = importlib.util.spec_from_file_location(modname, path) + if spec is None or spec.loader is None: + raise ImportError("no import spec for %s" % path) + module = importlib.util.module_from_spec(spec) + sys.modules[modname] = module + capture = io.StringIO() + try: + with contextlib.redirect_stdout(capture), contextlib.redirect_stderr(capture): + spec.loader.exec_module(module) + except BaseException as exc: # noqa: BLE001 - any import-time failure is a fault + tb_text = traceback.format_exc() + sys.modules.pop(modname, None) + raise _ImportFault(exc, tb_text, capture.getvalue()) + output = capture.getvalue() + bad_line = transcript_failure(output) + if bad_line is not None: + # Q128: this lane's script-style files report a failed check by + # printing `FAIL ` and keeping an `ok` flag, then exiting 1 + # from their own __main__ guard -- which never runs under an import. + # Without this a broken script cell printed its FAIL line into a + # buffer and the runner called the lane green (measured on + # test_bisect_bounds.py: rc 0 with a false expectation in it). + exc = AssertionError("module-level transcript reported a failure: %s" % bad_line) + raise _ImportFault(exc, "", output, kind="SCRIPT-FAIL") + return module, output + + +def transcript_failure(output): + """The first line of a module-level transcript that reports a failure. + + Two forms, both taken from the lane's own script files rather than + invented here: a line beginning `FAIL` (the `check()` helper every + script file defines prints `PASS ` / `FAIL `), and the + `SOME FAILED` summary. `PASS`, `ALL`, `OK` and prose are not failures, + and a line is only read as one at its first word, so `preFAIL` or a + path containing FAIL is not a false positive. + """ + for line in output.splitlines(): + token = line.strip() + if not token: + continue + first = token.split(None, 1)[0].rstrip(":") + if first == "FAIL" or token.endswith("SOME FAILED"): + return token + return None + + +class _ImportFault(Exception): + def __init__(self, exc, tb_text, output, kind="IMPORT-FAIL"): + Exception.__init__(self, "%s: %s" % (type(exc).__name__, exc)) + self.exc = exc + self.tb_text = tb_text + self.output = output + self.kind = kind + + +class BareCase(unittest.FunctionTestCase): + """A module-level ``def test_*()`` presented like any other test.""" + + def __init__(self, fn, label): + unittest.FunctionTestCase.__init__(self, fn) + self._label = label + self._name = fn.__name__ + + def id(self): + return "%s.%s" % (self._label, self._name) + + def __str__(self): + return self.id() + + +def bare_cases(module, modname, label): + """Module-level zero-argument functions named test_* -> runnable cases.""" + cases = [] + for name, obj in sorted(vars(module).items()): + if not name.startswith("test_"): + continue + if not inspect.isfunction(obj): + continue + if getattr(obj, "__module__", None) != modname: + continue # imported, not defined here + try: + params = inspect.signature(obj).parameters + except (TypeError, ValueError): + continue + if params: + continue # fixture-style signature: not this runner's business + cases.append(BareCase(obj, label)) + return cases + + +def collect(lane): + """Return (suite, report) where report is a list of per-file rows.""" + suite = unittest.TestSuite() + rows = [] + faults = [] + for path in discover(lane): + name = os.path.basename(path) + try: + module, _out = load_module(path) + except _ImportFault as fault: + faults.append((name, fault)) + rows.append({"file": name, "unittest": 0, "bare": 0, "fault": fault}) + continue + modname = module.__name__ + uni = unittest.TestLoader().loadTestsFromModule(module) + uni_count = uni.countTestCases() + bare = bare_cases(module, modname, os.path.splitext(name)[0]) + suite.addTests(uni) + suite.addTests(bare) + rows.append({"file": name, "unittest": uni_count, "bare": len(bare), "fault": None}) + return suite, rows + + +def main(argv=None): + ap = argparse.ArgumentParser(description=(__doc__ or "").splitlines()[0]) + ap.add_argument("--lane", default=DEFAULT_LANE, help="lane directory (default: this file's)") + ap.add_argument("--list", action="store_true", help="print the collection and exit") + ap.add_argument("--verbose", "-v", action="store_true", help="one line per test") + args = ap.parse_args(argv) + + lane = os.path.abspath(args.lane) + if not os.path.isdir(lane): + print("run_lane_tests: no lane directory: %s" % lane, file=sys.stderr) + return 2 + files = discover(lane) + if not files: + print("run_lane_tests: no test_*.py under %s -- nothing to grade" % lane, file=sys.stderr) + return 2 + + sys.path.insert(0, lane) + cwd = os.getcwd() + os.chdir(lane) + try: + suite, rows = collect(lane) + finally: + os.chdir(cwd) + + total_uni = sum(r["unittest"] for r in rows) + total_bare = sum(r["bare"] for r in rows) + scripts = [r["file"] for r in rows + if r["fault"] is None and r["unittest"] == 0 and r["bare"] == 0] + faults = [r for r in rows if r["fault"] is not None] + import_faults = [r for r in faults if r["fault"].kind == "IMPORT-FAIL"] + script_faults = [r for r in faults if r["fault"].kind == "SCRIPT-FAIL"] + total = suite.countTestCases() + + if args.list: + for r in rows: + print("%-46s unittest=%-4d bare=%-4d%s" + % (r["file"], r["unittest"], r["bare"], + " " + r["fault"].kind if r["fault"] else "")) + print("collection: %d file(s), %d unittest case(s), %d bare case(s), %d total, " + "%d module-level script file(s), %d import fault(s), %d script fault(s)" + % (len(rows), total_uni, total_bare, total, len(scripts), + len(import_faults), len(script_faults))) + return 1 if faults else 0 + + runner = unittest.TextTestRunner(verbosity=2 if args.verbose else 1) + result = runner.run(suite) + + bad = (not result.wasSuccessful()) or bool(faults) + for r in faults: + fault = r["fault"] + print("\n%s %s" % (fault.kind, r["file"]), file=sys.stderr) + print(" %s" % fault, file=sys.stderr) + if fault.output.strip(): + tail = fault.output.strip().splitlines()[-3:] + for line in tail: + print(" import output: %s" % line, file=sys.stderr) + if fault.tb_text: + print(" traceback (last 6 lines):", file=sys.stderr) + for line in fault.tb_text.strip().splitlines()[-6:]: + print(" " + line, file=sys.stderr) + + print("lane: %d file(s), %d unittest case(s), %d bare case(s), %d test(s), " + "%d module-level script file(s) executed at import, " + "%d failure(s), %d error(s), %d import fault(s), %d script fault(s)" + % (len(rows), total_uni, total_bare, total, len(scripts), + len(result.failures), len(result.errors), + len(import_faults), len(script_faults))) + if bad: + print("lane: FAULT -- see the failure list above (an unimportable module and a " + "failed script transcript are faults, not skips)") + return 1 + print("lane: OK -- every collected test passed") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/wizard-work/beam/test_ratelimit.py b/wizard-work/beam/test_ratelimit.py index 70b318a..4627e48 100644 --- a/wizard-work/beam/test_ratelimit.py +++ b/wizard-work/beam/test_ratelimit.py @@ -1,6 +1,7 @@ import time import threading -import pytest +from unittest import mock +from beam_assert import raises from ratelimit import TokenBucket, make_bucket, RateLimitError @@ -46,15 +47,19 @@ def test_timeout(): assert ok is False -def test_blocking_success_with_fake_sleep(monkeypatch): +def test_blocking_success_with_fake_sleep(): + # Q128: this cell used pytest's `monkeypatch` fixture, so neither + # `unittest discover` nor run_lane_tests.py could ever execute it -- + # it was a test no runner in this lane ran. unittest.mock is stdlib + # and does the same job, so the cell is collected and graded now. t = [0.0] b = make_bucket(rate=2, capacity=2, clock=lambda: t[0]) assert b.acquire(2, blocking=False) is True - # monkeypatch time.sleep to advance clock + # patch time.sleep to advance the bucket's clock def fake_sleep(s): t[0] += s - monkeypatch.setattr(time, "sleep", fake_sleep) - assert b.acquire(2, blocking=True, timeout=5) is True + with mock.patch.object(time, "sleep", fake_sleep): + assert b.acquire(2, blocking=True, timeout=5) is True def test_n_exceeds_capacity_nonblocking(): @@ -64,13 +69,13 @@ def test_n_exceeds_capacity_nonblocking(): def test_n_exceeds_capacity_blocking_raises(): b = make_bucket(rate=1, capacity=2, clock=lambda: 0.0) - with pytest.raises(RateLimitError): + with raises(RateLimitError): b.acquire(5, blocking=True) def test_negative_n(): b = make_bucket(rate=1, capacity=2, clock=lambda: 0.0) - with pytest.raises(ValueError): + with raises(ValueError): b.acquire(-1, blocking=False) @@ -90,7 +95,7 @@ def test_thread_safety(): def test_invalid_params(): - with pytest.raises(ValueError): + with raises(ValueError): TokenBucket(rate=0, capacity=1) - with pytest.raises(ValueError): + with raises(ValueError): TokenBucket(rate=1, capacity=0) diff --git a/wizard-work/beam/test_run_lane_tests.py b/wizard-work/beam/test_run_lane_tests.py new file mode 100644 index 0000000..2f875ae --- /dev/null +++ b/wizard-work/beam/test_run_lane_tests.py @@ -0,0 +1,165 @@ +"""Q128: cells for run_lane_tests.py, the beam lane's own stdlib runner. + +Each cell builds a throwaway lane in a temporary directory and runs the runner +as a subprocess with `--lane`, so the runner is graded by its exit code and its +printed summary, never by importing its internals. The cells that matter are +the red ones: a failing bare-style `def test_*`, a failing unittest cell, and a +module that will not import must each be a fault (rc 1), and an empty lane must +be rc 2 rather than a silent pass. + +This file is itself collected by the runner it grades -- it is a plain +unittest.TestCase module, so it costs one extra cell set and no recursion +(the cells invoke the runner on temp lanes, never on this lane). +""" + +import os +import subprocess +import sys +import tempfile +import unittest + +RUNNER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "run_lane_tests.py") + + +def run_lane(lane_dir): + """Run the runner over ``lane_dir``; return (rc, stdout+stderr).""" + proc = subprocess.run( + [sys.executable, RUNNER, "--lane", lane_dir], + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + timeout=120, + ) + return proc.returncode, proc.stdout + + +def write(path, text): + with open(path, "w", encoding="utf-8") as fh: + fh.write(text) + + +class TestRunnerContract(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.lane = self._tmp.name + + def tearDown(self): + self._tmp.cleanup() + + def test_green_lane_exit_zero_and_counts_both_styles(self): + write(os.path.join(self.lane, "test_a.py"), + "import unittest\n" + "def test_bare_ok():\n assert 1 + 1 == 2\n" + "class T(unittest.TestCase):\n" + " def test_uni_ok(self):\n self.assertEqual(2 + 2, 4)\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 0, out) + self.assertIn("1 unittest case(s), 1 bare case(s), 2 test(s)", out) + self.assertIn("lane: OK", out) + + def test_failing_bare_cell_is_a_fault_named_by_module_and_function(self): + write(os.path.join(self.lane, "test_bare.py"), + "def test_broken():\n assert 1 == 2, 'bare cell broke'\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 1, out) + self.assertIn("FAIL: test_bare.test_broken", out) + self.assertIn("1 failure(s)", out) + + def test_failing_unittest_cell_is_a_fault(self): + write(os.path.join(self.lane, "test_uni.py"), + "import unittest\n" + "class T(unittest.TestCase):\n" + " def test_broken(self):\n self.assertEqual(1, 2)\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 1, out) + self.assertIn("test_broken", out) + + def test_module_that_will_not_import_is_a_fault_not_a_skip(self): + write(os.path.join(self.lane, "test_import_bang.py"), + "def broken(:\n pass\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 1, out) + self.assertIn("IMPORT-FAIL test_import_bang.py", out) + self.assertIn("1 import fault(s)", out) + + def test_module_level_assertion_is_a_fault_too(self): + # A module that raises at import for a non-syntax reason (a failed + # module-level assertion in the lane's script-style files) is a fault + # too: the runner must not treat "no test_* names defined" as a pass. + write(os.path.join(self.lane, "test_script_style.py"), + "assert 1 == 2, 'module-level assertion ran'\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 1, out) + self.assertIn("IMPORT-FAIL test_script_style.py", out) + + def test_script_that_prints_FAIL_is_a_fault(self): + # Q128: this lane's script-style files report a failed check by + # printing `FAIL ` and exiting 1 from their own __main__ guard. + # Under an import the guard never runs, so before this rule the + # runner called such a file green while its transcript said FAIL + # (measured on test_bisect_bounds.py: rc 0 with a false expectation). + write(os.path.join(self.lane, "test_script_says_fail.py"), + "ok = True\n" + "def check(name, cond):\n" + " global ok\n" + " print(('PASS' if cond else 'FAIL') + ' ' + name)\n" + " ok = ok and cond\n" + "check('a', True)\n" + "check('b', 1 == 2)\n" + "print('ALL' if ok else 'SOME FAILED')\n" + "if __name__ == '__main__':\n" + " raise SystemExit(0 if ok else 1)\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 1, out) + self.assertIn("SCRIPT-FAIL test_script_says_fail.py", out) + self.assertIn("FAIL b", out) + self.assertIn("1 script fault(s)", out) + + def test_script_that_prints_only_PASS_is_not_a_fault(self): + # The negative control for the rule above: the same shape with every + # check true must stay green, so the cell proves the rule reads the + # transcript rather than firing on the convention itself. + write(os.path.join(self.lane, "test_script_says_pass.py"), + "ok = True\n" + "def check(name, cond):\n" + " global ok\n" + " print(('PASS' if cond else 'FAIL') + ' ' + name)\n" + " ok = ok and cond\n" + "check('a', True)\n" + "print('ALL' if ok else 'SOME FAILED')\n" + "if __name__ == '__main__':\n" + " raise SystemExit(0 if ok else 1)\n") + rc, out = run_lane(self.lane) + self.assertEqual(rc, 0, out) + self.assertNotIn("SCRIPT-FAIL", out) + self.assertIn("1 module-level script file(s)", out) + + def test_no_test_files_is_could_not_grade(self): + rc, out = run_lane(self.lane) + self.assertEqual(rc, 2, out) + self.assertIn("nothing to grade", out) + + def test_missing_lane_dir_is_could_not_grade(self): + rc, out = run_lane(os.path.join(self.lane, "does-not-exist")) + self.assertEqual(rc, 2, out) + self.assertIn("no lane directory", out) + + def test_list_mode_prints_the_collection_and_runs_nothing(self): + write(os.path.join(self.lane, "test_c.py"), + "def test_one():\n assert True\n") + bad = os.path.join(self.lane, "test_d.py") + write(bad, "raise SystemExit('should not run in --list mode')\n") + proc = subprocess.run( + [sys.executable, RUNNER, "--lane", self.lane, "--list"], + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, timeout=120, + ) + # test_d.py raises SystemExit at import: --list must still report it as + # an import fault (rc 1) rather than dying or printing a clean count. + self.assertEqual(proc.returncode, 1, proc.stdout) + self.assertIn("IMPORT-FAIL", proc.stdout) + self.assertIn("test_d.py", proc.stdout) + self.assertIn("test_c.py", proc.stdout) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/wizard-work/beam/test_semver.py b/wizard-work/beam/test_semver.py index 2019c56..0e0e502 100644 --- a/wizard-work/beam/test_semver.py +++ b/wizard-work/beam/test_semver.py @@ -1,4 +1,4 @@ -import pytest +from beam_assert import raises from semver import SemVer, SemVerError def test_parse_basic(): @@ -28,14 +28,14 @@ def test_build_ignored_equality(): assert hash(SemVer.parse("1.0.0+foo")) == hash(SemVer.parse("1.0.0+bar")) def test_leading_zero_rejected(): - with pytest.raises(SemVerError): + with raises(SemVerError): SemVer.parse("1.0.0-01") - with pytest.raises(SemVerError): + with raises(SemVerError): SemVer.parse("1.0.0+01") def test_invalid(): for s in ["", "1.2", "1.2.3.4", "a.b.c", "1.2.3-", " 1.2.3", "1.2.3 "]: - with pytest.raises(SemVerError): + with raises(SemVerError): SemVer.parse(s) def test_bump(): diff --git a/wizard-work/beam/test_tree_root.py b/wizard-work/beam/test_tree_root.py index 7876bc1..6e3e132 100644 --- a/wizard-work/beam/test_tree_root.py +++ b/wizard-work/beam/test_tree_root.py @@ -1,7 +1,7 @@ import os import sys import tempfile -import pytest +from beam_assert import fail, raises sys.path.insert(0, os.path.dirname(__file__)) from tree_root import find_tree_root, assert_inside_tree, portable_seed @@ -21,7 +21,7 @@ def test_find_root_walks_up(): def test_find_root_missing_raises(): d = tempfile.mkdtemp() - with pytest.raises(FileNotFoundError): + with raises(FileNotFoundError): find_tree_root(d) @@ -35,7 +35,7 @@ def test_assert_inside_ok(): def test_assert_inside_escape_raises(): root = tempfile.mkdtemp() - with pytest.raises(ValueError): + with raises(ValueError): assert_inside_tree("/etc/passwd", root) @@ -62,7 +62,7 @@ def test_import_no_io(): assert not calls, "import-time open calls: %r" % calls except RuntimeError as e: if "import-time I/O" in str(e): - pytest.fail("module performed import-time I/O") + fail("module performed import-time I/O") raise finally: builtins.open = orig diff --git a/wizard-work/beam/test_weighted_interval_scheduling.py b/wizard-work/beam/test_weighted_interval_scheduling.py index 090d15b..b64187d 100644 --- a/wizard-work/beam/test_weighted_interval_scheduling.py +++ b/wizard-work/beam/test_weighted_interval_scheduling.py @@ -5,8 +5,9 @@ length, so a zero-length interval (start == end) is compatible with every interval and with every other zero-length interval. Before this file the module's only check was a `__main__` self-test whose generator emitted `length = randint(1, 4)`, so no cell could construct the failing case and -`beam/CHECK` (which runs pytest over `test_*.py`) never imported the -module at all. +`beam/CHECK` (which, when this file was written, ran pytest over `test_*.py`; +it now runs `run_lane_tests.py`, which needs no third-party package) never +imported the module at all. The enumerator below is written here rather than imported from the module's `brute_force`, so the differential compares the DP against independent code. @@ -14,7 +15,7 @@ The enumerator below is written here rather than imported from the module's import random -import pytest +from beam_assert import raises from weighted_interval_scheduling import weighted_interval_scheduling as wis @@ -70,9 +71,9 @@ def test_zero_length_points_add_to_a_packed_schedule(): def test_reversed_interval_is_rejected(): - with pytest.raises(ValueError): + with raises(ValueError): wis([(3, 1, 5)]) - with pytest.raises(ValueError): + with raises(ValueError): wis([(0, 2, 1), (5, 4, 9)])