volt: Q105 -- beam/CHECK runs pytest: the gate could not see 131 of its 407 tests (unittest discover 276, rc 0 over a broken test_fenwick cell; now 407+2007 subtests, rc 1 on the same mutation, rc 2 without pytest)
This commit is contained in:
1 parent
e49a7844f6
commit
a2c59f324d
1 file changed
+41
-1
+41
-1
@@ -1,6 +1,46 @@
|
||||
#!/bin/bash
|
||||
# CHECK: verify all beam reliability primitives
|
||||
#
|
||||
# Runner note (Q105, 2026-10-28). This lane writes its tests in two styles:
|
||||
# unittest.TestCase subclasses and bare module-level `def test_*` / `assert`.
|
||||
# `python3 -m unittest discover` collects only the first -- 276 of the 407
|
||||
# tests in this directory -- so a green CHECK said nothing about the other
|
||||
# 131, and test_fenwick.py's 8 cells could be broken without moving the gate.
|
||||
# pytest collects both styles, so pytest is the runner here.
|
||||
#
|
||||
# `--import-mode=importlib` is deliberate: two files share the basename
|
||||
# `test_toposort.py` (this directory and toposort/), and pytest's default
|
||||
# `prepend` mode aborts collection on that duplicate module name. importlib
|
||||
# mode keys modules by path instead. `toposort/test_toposort.py` is a plain
|
||||
# script (8 PASS lines + ALL), not a pytest module, so it keeps its own line.
|
||||
#
|
||||
# Exit codes follow the repo's tools/ contract: 0 clean, 1 graded a fault,
|
||||
# 2 could not grade. pytest's own codes are mapped onto that contract:
|
||||
# 0 -> 0 all collected tests passed
|
||||
# 1 -> 1 a test failed
|
||||
# 2 -> 1 collection error: a test module does not import (a fault)
|
||||
# 5 -> 1 no tests collected: the gate is measuring nothing
|
||||
# A missing pytest is 2, never a silent 0 -- a gate that cannot run the 131
|
||||
# tests must not report success.
|
||||
set -e
|
||||
set -o pipefail
|
||||
cd "$(dirname "$(readlink -f "$0")")"
|
||||
python3 -m unittest discover -p 'test_*.py' -v
|
||||
|
||||
if ! python3 -m pytest --version >/dev/null 2>&1; then
|
||||
echo "CHECK: pytest not importable -- cannot grade." >&2
|
||||
echo "CHECK: unittest discover would run 276 of 407 tests and skip 131 (Q105)." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
set +e
|
||||
python3 -m pytest -q -p no:cacheprovider --import-mode=importlib test_*.py
|
||||
rc=$?
|
||||
set -e
|
||||
case $rc in
|
||||
0) ;;
|
||||
1) echo "CHECK: pytest reported failing test(s)." >&2; exit 1 ;;
|
||||
5) echo "CHECK: pytest collected no tests -- the gate is measuring nothing." >&2; exit 1 ;;
|
||||
*) echo "CHECK: pytest exited $rc -- collection or internal error (a module that will not import is a fault)." >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
python3 toposort/test_toposort.py
|
||||
Reference in new issue
Block a user