beam: waive the committed runner, derive the census guard's module set, bind the bounds test beside itself (Q165, Q166)

This commit is contained in:
ledger committed 2026-11-03 19:54:35 +00:00
1 parent 2829ff0b84
commit 9be1896c46
5 files changed
+229 -43

No files matched your search

+67 -25
View File
@@ -1,22 +1,33 @@
#!/usr/bin/env bash
# CHECK-q164 -- the beam coverage census graded the directory instead of the
# tree, so a scratch file on this host moved its verdict (volt, 2026-11-03).
# CHECK-q164 -- the beam coverage census grades the tree, not this host's
# directory, so a scratch file beside the lane cannot move its verdict.
#
# The defect: `check_coverage.py` listed the lane with
# The defect (Q164, 2026-11-03): `check_coverage.py` listed the lane with
# `glob(lane/*.py)`. At `a96da08` the worktree read 125 module(s) / 3 fault(s)
# and a `git archive HEAD` export of the SAME COMMIT read 124 / 2 -- the
# difference was `_run_regex.py`, untracked, which no clone of that commit
# contains. Same class as Q140/Q141 (a census that describes its host).
#
# Six legs:
# tree the census reads the tree: rc 0, 124 modules, and the untracked
# scratch file is named in a note rather than graded
# Why the legs below count nothing by hand (Q165, 2026-11-03). `2829ff0` added
# `binary_search_bounds.py` and committed `_run_regex.py`; this guard's literal
# `124 module(s)` and its "names _run_regex in a note" expectation then both
# went red on a census that was reading the tree correctly. The expected module
# set is now DERIVED (`git ls-files`) and compared row for row, so a lane module
# added or removed is graded by the comparison rather than by a number someone
# has to remember to edit into this file.
#
# Seven legs:
# tree the census reads rc 0 with 0 faults and its rows name EXACTLY the
# tracked lane modules: the set `git ls-files` reports, no more (it
# graded a host file) and no fewer (it dropped a tree module)
# scratch a new untracked *.py dropped into the lane does NOT move the
# verdict (rc 0), which is the item: the reading is of the tree
# verdict (rc 0), is named in a note, and is ABSENT from the rows
# dir --enumerate dir on the same directory DOES grade it (rc 1), so
# the green above is the source of the list, not a lost tooth
# export a `git archive HEAD` export of the same commit reads rc 0 with
# the same module count: no clone can disagree with this worktree
# runner the committed hand runner `_run_regex` reads as a stated waiver
# naming the collected test it duplicates, not as UNCOVERED
# smoke the lane gate executes Q163's smoke cells (grep the CHECK line,
# run it, and read its coverage row as covered-gate)
# selftest the census's own fixture corpus, 17 cells, 0 failed
@@ -27,7 +38,8 @@ set -o pipefail
cd "$(dirname "${BASH_SOURCE[0]}")" || exit 2
ROOT="$(pwd)"
LANE="$ROOT/wizard-work/beam"
REL="wizard-work/beam"
LANE="$ROOT/$REL"
CENSUS="$LANE/check_coverage.py"
GATE="$LANE/CHECK"
SMOKE="$LANE/_smoke_intervals.py"
@@ -56,30 +68,51 @@ failed=0
row() { printf ' %-10s %s\n' "$1" "$2"; }
verdict() { if [ "$2" -eq 0 ]; then passed=$((passed + 1)); else failed=$((failed + 1)); fi; row "$1" "$3"; }
# --- leg 1: the census reads the tree ---------------------------------------
# The expected set: the lane's tracked *.py minus the collected tests. Read
# from the tree, so neither leg below can rot on a literal count.
git -C "$ROOT" ls-files -z -- "$REL" \
| tr '\0' '\n' \
| grep "^$REL/[^/]*\.py\$" \
| sed 's#.*/##; s/\.py$//' \
| grep -v '^test_' \
| sort > "$WORK/want" || exit 2
want="$(grep -c . "$WORK/want")"
# One module per line: the census's own state rows only (notes and summary
# lines share the stream), first occurrence per module (--list repeats the
# non-covered rows when it prints the report), waiver-table rows dropped
# because they grade the table rather than the lane's module list.
"$PY" "$CENSUS" --list 2>/dev/null \
| awk '$1 ~ /^(covered-|exempt$|UNCOVERED$)/ && !seen[$2]++ {print $2}' \
| sort > "$WORK/got"
# --- leg 1: the census reads the tree, and the tree is the truth ------------
out="$("$PY" "$CENSUS" 2>&1)"; trc=$?
last="$(printf '%s' "$out" | grep '^coverage: [0-9]' | tail -1)"
note="$(printf '%s' "$out" | grep 'untracked, not graded' | head -1)"
n_last="$(printf '%s' "$last" | sed -n 's/^coverage: \([0-9][0-9]*\) module(s).*/\1/p')"
extra="$(comm -13 "$WORK/want" "$WORK/got" | tr '\n' ' ')"
gone="$(comm -23 "$WORK/want" "$WORK/got" | tr '\n' ' ')"
badrow="$("$PY" "$CENSUS" --list 2>/dev/null | awk '$1 ~ /^(UNCOVERED|WAIVER-)/ {c++} END {print c+0}')"
verdict tree \
"$([ "$trc" -eq 0 ] && printf '%s' "$last" | grep -q '124 module(s)' \
&& printf '%s' "$last" | grep -q '0 fault(s)' \
&& printf '%s' "$note" | grep -q '_run_regex' && echo 0 || echo 1)" \
"rc=$trc ${last:-<no summary>} ${note:-<no untracked note>}"
"$([ "$trc" -eq 0 ] && printf '%s' "$last" | grep -q ' 0 fault(s)' \
&& [ "$n_last" = "$want" ] && [ -z "$extra" ] && [ -z "$gone" ] \
&& [ "$badrow" -eq 0 ] && echo 0 || echo 1)" \
"rc=$trc ${last:-<no summary>} rows=$want extra=${extra:-none} missing=${gone:-none} faultRows=$badrow"
# --- leg 2: a host artifact cannot move the verdict -------------------------
printf 'print("scratch probe, never committed")\n' > "$PROBE"
out="$("$PY" "$CENSUS" 2>&1)"; src=$?
last="$(printf '%s' "$out" | grep '^coverage: [0-9]' | tail -1)"
note="$(printf '%s' "$out" | grep 'untracked, not graded: zz_q164_scratch_probe' | head -1)"
inrows="$("$PY" "$CENSUS" --list 2>/dev/null | awk '$2 == "zz_q164_scratch_probe" {c++} END {print c+0}')"
verdict scratch \
"$([ "$src" -eq 0 ] && printf '%s' "$last" | grep -q '0 fault(s)' \
&& printf '%s' "$last" | grep -q '124 module(s)' \
&& [ -n "$note" ] && echo 0 || echo 1)" \
"rc=$src ${last:-<no summary>} ${note:-<probe not named>}"
"$([ "$src" -eq 0 ] && printf '%s' "$last" | grep -q ' 0 fault(s)' \
&& [ -n "$note" ] && [ "$inrows" -eq 0 ] && echo 0 || echo 1)" \
"rc=$src ${last:-<no summary>} ${note:-<probe not named>} probeRows=$inrows"
# --- leg 3: the directory reading still has teeth ---------------------------
out="$("$PY" "$CENSUS" --enumerate dir 2>&1)"; drc=$?
faultline="$(printf '%s' "$out" | grep 'UNCOVERED' | tr '\n' ';')"
faultline="$(printf '%s' "$out" | awk '$1 ~ /^(UNCOVERED|WAIVER-)/' | tr '\n' ';')"
verdict dir \
"$([ "$drc" -eq 1 ] && printf '%s' "$faultline" | grep -q 'zz_q164_scratch_probe' && echo 0 || echo 1)" \
"rc=$drc (want 1) ${faultline:-<no fault row>}"
@@ -87,17 +120,26 @@ rm -f "$PROBE"
# --- leg 4: the same commit, read from a clean export -----------------------
if git -C "$ROOT" archive HEAD 2>/dev/null | tar -x -C "$WORK" 2>/dev/null; then
out="$("$PY" "$WORK/wizard-work/beam/check_coverage.py" 2>&1)"; erc=$?
out="$("$PY" "$WORK/$REL/check_coverage.py" 2>&1)"; erc=$?
last="$(printf '%s' "$out" | grep '^coverage: [0-9]' | tail -1)"
verdict export \
"$([ "$erc" -eq 0 ] && printf '%s' "$last" | grep -q '124 module(s)' \
&& printf '%s' "$last" | grep -q '0 fault(s)' && echo 0 || echo 1)" \
"rc=$erc ${last:-<no summary>}"
"$([ "$erc" -eq 0 ] && printf '%s' "$last" | grep -q ' 0 fault(s)' \
&& printf '%s' "$last" | grep -q " $want module(s)" && echo 0 || echo 1)" \
"rc=$erc ${last:-<no summary>} want=$want module(s)"
else
verdict export 1 "git archive HEAD would not export -- cannot read the commit"
fi
# --- leg 5: the lane gate runs the smoke ------------------------------------
# --- leg 5: the committed hand runner is a waiver, not a fault --------------
tracked_regex="$(git -C "$ROOT" ls-files -- "$REL/_run_regex.py")"
rrow="$("$PY" "$CENSUS" --list 2>/dev/null | awk '$2 == "_run_regex" {print; exit}')"
rstate="$(printf '%s' "$rrow" | awk '{print $1}')"
verdict runner \
"$([ -n "$tracked_regex" ] && [ "$rstate" = "exempt" ] \
&& printf '%s' "$rrow" | grep -q 'test_regex_match' && echo 0 || echo 1)" \
"tracked=${tracked_regex:-no} state=${rstate:-<no row>}"
# --- leg 6: the lane gate runs the smoke ------------------------------------
line="$(grep -c '^python3 _smoke_intervals\.py$' "$GATE")"
out="$(cd "$LANE" && "$PY" _smoke_intervals.py 2>&1)"; mrc=$?
last="$(printf '%s' "$out" | tail -1)"
@@ -107,7 +149,7 @@ verdict smoke \
&& [ -n "$rowline" ] && echo 0 || echo 1)" \
"check-line=$line rc=$mrc ${last:-<no output>} ${rowline:-<not covered-gate>}"
# --- leg 6: the census's fixture corpus -------------------------------------
# --- leg 7: the census's fixture corpus -------------------------------------
out="$("$PY" "$CENSUS" --selftest 2>&1)"; crc=$?
last="$(printf '%s' "$out" | tail -1)"
verdict selftest \
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
# CHECK-q166 -- the binary-search bounds test binds the module BESIDE itself,
# not a directory of the host that ran it.
#
# The defect (Q166, 2026-11-03): `test_binary_search_bounds.py` landed at
# `2829ff0` with `sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")` --
# a directory outside the repository. `tools/import_resolve_check.py` read
# `FAULT OUTSIDE_REPO wizard-work/beam/test_binary_search_bounds.py:2` at this
# tree, and the repo gate's import-resolve leg went red on it. The file also
# shipped with no `test_*` function, so the lane runner read it as
# `unittest=0 bare=0` and a wrong bound showed up as an import error against
# the file rather than as a named failing cell.
#
# Four legs:
# bind the test names no host directory and does bind its own: the
# `HERE` binding and the resolved-module assertion are both present
# tool `tools/import_resolve_check.py` reports 0 fault(s) and names this
# file nowhere in its faults array -- the leg that caught it
# cells a scratch lane holding this test and its module reads rc 0 with
# 2 bare case(s): the checks are attributed cells now
# poison the same lane with `binary_search_bounds.py` replaced by a
# stand-in that answers 0/0/True reads rc 1 and names a bounds
# cell, so the green above is the module beside the test
#
# Exit codes: 0 all legs passed, 1 a fault, 2 could not grade.
set -u
set -o pipefail
cd "$(dirname "${BASH_SOURCE[0]}")" || exit 2
ROOT="$(pwd)"
LANE="$ROOT/wizard-work/beam"
TEST="$LANE/test_binary_search_bounds.py"
MOD="$LANE/binary_search_bounds.py"
RUNNER="$LANE/run_lane_tests.py"
TOOL="$ROOT/tools/import_resolve_check.py"
PY="${PYTHON:-python3}"
for f in "$TEST" "$MOD" "$RUNNER" "$TOOL"; do
if [ ! -f "$f" ]; then
printf 'CANNOT GRADE: missing %s\n' "$f"
exit 2
fi
done
WORK="$(mktemp -d "${TMPDIR:-/tmp}/check-q166.XXXXXX")" || exit 2
trap 'rm -rf "$WORK"' EXIT
passed=0
failed=0
row() { printf ' %-8s %s\n' "$1" "$2"; }
verdict() { if [ "$2" -eq 0 ]; then passed=$((passed + 1)); else failed=$((failed + 1)); fi; row "$1" "$3"; }
# --- leg 1: the binding is beside the test, not on the host -----------------
# Only the code line counts: a quoted path inside the file's docstring is the
# history of the defect, and `^sys.path.insert(0, "` matches no such prose.
quotedbind="$(grep -c '^sys\.path\.insert(0, "' "$TEST")"
hereline="$(grep -c 'sys.path.insert(0, HERE)' "$TEST")"
assertline="$(grep -c 'not the module beside this file' "$TEST")"
verdict bind \
"$([ "$quotedbind" -eq 0 ] && [ "$hereline" -ge 1 ] && [ "$assertline" -ge 1 ] && echo 0 || echo 1)" \
"quotedBinds=$quotedbind HERE-bind=$hereline resolve-assert=$assertline"
# --- leg 2: the tool that caught it reads the file clean --------------------
"$PY" "$TOOL" --json "$WORK/rows.json" --quiet > "$WORK/tool.log" 2>&1; trc=$?
named="$(grep -c 'test_binary_search_bounds' "$WORK/rows.json")"
verdict tool \
"$([ "$trc" -eq 0 ] && grep -q '"faults": \[\]' "$WORK/rows.json" \
&& [ "$named" -eq 0 ] && echo 0 || echo 1)" \
"rc=$trc faults-named=$(printf '%s' "$named") $(tail -1 "$WORK/tool.log")"
# --- leg 3: the cells are collected and green beside the module -------------
mkdir -p "$WORK/lane" || exit 2
cp "$TEST" "$MOD" "$WORK/lane/" || exit 2
out="$("$PY" "$RUNNER" --lane "$WORK/lane" 2>&1)"; crc=$?
last="$(printf '%s' "$out" | tail -1)"
cells="$(printf '%s' "$out" | grep -c '2 bare case(s)')"
verdict cells \
"$([ "$crc" -eq 0 ] && [ "$cells" -ge 1 ] && printf '%s' "$last" | grep -q 'lane: OK' && echo 0 || echo 1)" \
"rc=$crc ${last:-<no summary>}"
# --- leg 4: a stand-in beside it reds, so the green is this file ------------
mkdir -p "$WORK/poison" || exit 2
cp "$TEST" "$WORK/poison/" || exit 2
printf 'def bisect_left(a, x):\n return 0\n\n\ndef bisect_right(a, x):\n return 0\n\n\ndef contains(a, x):\n return True\n' > "$WORK/poison/binary_search_bounds.py" || exit 2
out="$("$PY" "$RUNNER" --lane "$WORK/poison" 2>&1)"; prc=$?
last="$(printf '%s' "$out" | tail -1)"
namedcell="$(printf '%s' "$out" | grep -c 'test_bounds_')"
verdict poison \
"$([ "$prc" -eq 1 ] && printf '%s' "$last" | grep -q 'lane: FAULT' \
&& [ "$namedcell" -ge 1 ] && echo 0 || echo 1)" \
"rc=$prc (want 1) cells-named=$namedcell ${last:-<no summary>}"
if [ "$failed" -eq 0 ]; then
echo "q166: $passed leg(s), 0 fault(s), rc 0"
exit 0
fi
echo "q166: $passed passed, $failed fault(s), rc 1"
exit 1
+1
View File
@@ -297,3 +297,4 @@ verify_wordcount_test.py EXAMPLE quoted at WORK-QUEUE.md:14386 (Q157 record) as
wizard-work/null/.hermes-tmp.RqZp3z EXAMPLE quoted at WORK-QUEUE.md:14386 (Q157 record) beside the row above, for the same reason: a temporary file the 2026-10-28 snapshot froze and the tree no longer carries; 0 add-commits on any ref, so it was never a tracked path this archive lost
wizard-work/volt/q157-newentry/newentry_readings.tsv NOT_ADDED cited at WORK-QUEUE.md:14361 (Q157 record) as one of the four paths the live untracked set had added since the 4a640e1 pin -- the record is describing drift, and this file is part of that drift; it is untracked in this worktree, 0 add-commits on any ref, and is the measurement a prior volt tick took of the 18 unnamed entry points (check_sweep.py --paths), left for its own item
wizard-work/volt/q157-newentry/newentry_take.py NOT_ADDED cited at WORK-QUEUE.md:14361 beside the row above and for the same reason: the runner that took that measurement; untracked in this worktree, 0 add-commits on any ref
wizard-work/beam/_run_regex.py NOT_ADDED cited at WORK-QUEUE.md:14553 (Q164 record) as the untracked file whose presence moved the beam coverage census -- 159 B, mtime 2026-11-03 10:50, a six-line hand runner over test_regex_match of the same shape as _run_tests.py -- and it is the item that it must NOT be graded: it is untracked in this worktree, git log --all --diff-filter=A -- wizard-work/beam/_run_regex.py -> 0 commits, and check_coverage.py now names it in a note instead of counting it. Tracking it would make it a lane module with no reader, which is the verdict this item removes
Can't render this file because it contains an unexpected character in line 15 and column 3.
+9
View File
@@ -102,6 +102,15 @@ EXEMPT = {
"run inside `bash CHECK` (run_lane_tests.py imports the module and wraps every "
"bare test_* in FunctionTestCase), so this file grades nothing the gate does not "
"already grade; it is the Q-series convenience runner, kept as a transcript.",
"_run_regex.py":
"six-line hand runner of the same shape, waived by the same reading: it imports "
"test_regex_match and calls its five test_* cells (cases_recursive, cases_dp, "
"cross_check_random, type_errors, match_alias), printing ALL PASS. That test file "
"is collected and its cells run inside `bash CHECK` through run_lane_tests.py, so "
"this script grades nothing the gate does not already grade. Q164 met it as an "
"untracked scratch file and named it in a note; `2829ff0` committed it, which "
"turned the note into an UNCOVERED row, and this waiver is the honest disposition "
"rather than a deletion of another lane's committed file.",
"conftest.py":
"pytest collection hook (two lines: put the lane on sys.path), read by the "
"collector rather than imported by a test. The lane's gate does not use pytest "
+54 -18
View File
@@ -1,19 +1,55 @@
import sys, random
sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")
from binary_search_bounds import bisect_left, bisect_right, contains
"""Bounds cells for `binary_search_bounds.py`, the module BESIDE this file.
# brute checks
for a in [[], [1], [1,2,3], [1,1,2,2,3]]:
for x in [0,1,2,3,4]:
assert bisect_left(a,x)==sum(1 for v in a if v<x), (a,x)
assert bisect_right(a,x)==sum(1 for v in a if v<=x), (a,x)
# random
random.seed(42)
for _ in range(1000):
n=random.randint(0,20)
a=sorted(random.randint(-5,5) for _ in range(n))
x=random.randint(-6,6)
assert bisect_left(a,x)==sum(1 for v in a if v<x)
assert bisect_right(a,x)==sum(1 for v in a if v<=x)
assert contains(a,x)==(x in a)
print("ALL TESTS PASSED")
Q166. This file landed at `2829ff0` with
`sys.path.insert(0, "/root/.hermes/wizard-outputs/beam")`, a directory outside
the repository, so it graded whatever that host directory held and no clone
could reproduce its reading: `tools/import_resolve_check.py` read
`FAULT OUTSIDE_REPO wizard-work/beam/test_binary_search_bounds.py:2`. It now
binds its own directory and asserts which file it read, the same repair Q160
and Q162 made in this lane.
The same edit makes the checks attributable. The file shipped with its
assertions at module level and no `test_*` function, so the lane's runner read
it as `unittest=0 bare=0`: a wrong bound surfaced as an import error against
the file, never as a named failing cell.
"""
import os
import random
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import binary_search_bounds # noqa: E402
from binary_search_bounds import bisect_left, bisect_right, contains # noqa: E402
assert os.path.realpath(binary_search_bounds.__file__) == \
os.path.realpath(os.path.join(HERE, "binary_search_bounds.py")), \
"graded %s, not the module beside this file" % binary_search_bounds.__file__
def _brute_left(a, x):
return sum(1 for v in a if v < x)
def _brute_right(a, x):
return sum(1 for v in a if v <= x)
def test_bounds_brute_cases():
for a in ([], [1], [1, 2, 3], [1, 1, 2, 2, 3]):
for x in (0, 1, 2, 3, 4):
assert bisect_left(a, x) == _brute_left(a, x), (a, x)
assert bisect_right(a, x) == _brute_right(a, x), (a, x)
assert contains(a, x) == (x in a), (a, x)
def test_bounds_random_1000():
random.seed(42)
for _ in range(1000):
n = random.randint(0, 20)
a = sorted(random.randint(-5, 5) for _ in range(n))
x = random.randint(-6, 6)
assert bisect_left(a, x) == _brute_left(a, x), (a, x)
assert bisect_right(a, x) == _brute_right(a, x), (a, x)
assert contains(a, x) == (x in a), (a, x)