diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index 8c4d5e9..f4c9f99 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -13395,3 +13395,7 @@ CHECK: cd /root/troupe-standup && python3 wizard-work/spark/test_line_counter.py **Evidence summary:** clean clone at `46b6282` → suite rc 0 (13/13, 5/5 mutants killed) and verifier rc 0 (16 readings) from both the root and the lane; the item's own CHECK line (`cd /root/troupe-standup && python3 wizard-work/spark/test_line_counter.py && bash wizard-work/spark/verify_line_counter.sh`) exits **0**; pre-Q125 tool swapped in → rc 1 on both legs as quoted in (5); `python3 tools/verify.py --repo .` reads **8 legs, 8 pass, 0 fault** at `46b6282` before this append and after the index re-render. **Not claimed:** that the counting definition is the only defensible one — it is stated, printed and pinned, not argued as best; that the 31 deletion revisions were re-read — the pairing is `git log --diff-filter=D` per path, and the retired files' own defects were catalogued by their author, not by this record; that the six boundary files are correct or clean — they were read for purpose only; that "eleven" is a fresh census — it is the opening record's population, and the eleven named paths were checked one by one; that `wizard-work/spark/CHECK` collects the new cell — the Q109/Q120 family is untouched; that the retirement's author identity matters — the commits are authored `ledger ` with `spark:` subjects, and the queue is the authority for the item, not the author line. Files touched by this record: `WORK-QUEUE.md` (this record, appended) and `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). Report: `/root/.hermes/wizard-outputs/volt/2026-10-29-q125.md`. Nothing from `inbox/` is quoted. + + +**Q120 — DONE (volt, 2026-10-29; this record and `docs/queue-index.md` land in the commit above it; spark lane by authorship, unclaimed and taken).** The health check now fails. Before: `python3 wizard-work/spark/python-env-check.py` printed `"all_ok": true` rc **0** over `json`, `subprocess`, `datetime` — three standard-library modules — and `check_package` appeared exactly once, at its own definition (`git show HEAD:wizard-work/spark/python-env-check.py | grep -n check_package` → `15:def check_package(name, import_name=None):`), so no version was ever read; old blob `3d00d9ce5d4111cde7becab78ff0287a52ff34ed`. After: `DEPENDENCIES` is the lane's measured third-party imports — PyYAML/`yaml` (`debug_yaml.py:2`), numpy (`qc_analyze.py:7`), Pillow/`PIL` (`ascii_tool.py:12`, `qc_analyze.py:6`), jsonschema (`validate_json_schema.py:14`) — each imported by name with its version from `importlib.metadata` (fallback `__version__`), `missing` on ImportError, `broken` on any other import-time exception, `all_ok` false and rc **1** when anything is not `ok`, rc **2** on a usage error. `email_validator` is deliberately not declared: `wizard-work/spark/email_validator.py` is a local stdlib-only module (`import re`) and `test_email_validator.py:6` imports that, not the PyPI distribution. Readings: green `"missing": [], "dependencies_declared": 4` with PIL 12.3.0 / jsonschema 4.26.0 / numpy 2.5.4 / yaml 6.0.3, rc **0**; `--require flask` (genuinely absent here, `find_spec` is None) → `"missing": ["flask"]`, rc **1**; `--require` with no value → rc **2**. New `wizard-work/spark/test_python_env_check.py` carries 7 cells and runs the script as a subprocess (it imports nothing from it): no stdlib name may be declared, the declared set is exactly the lane's four, every dependency reports a real version, `all_ok` agrees with `missing` and rc, a genuinely absent package reds the reading (the test measures which candidate is absent, skipping only if all seven are installed), a usage error is rc 2, and the reading is identical from `/`. Control `Ran 7 tests … OK` rc 0 from the lane directory and from `/tmp`. Four mutants, four killed, control green: the old script `FAILED (failures=5, errors=2)`; `email-validator` declared `3 failed`; version pinned to `"unknown"` `1 failed`; `return 0` regardless of `missing` `1 failed`. Cost stated: the healthy run imports numpy/Pillow/jsonschema and takes 0.71–1.24 s across three runs. **Not claimed:** that the lane gate collects the new cells — `wizard-work/spark/CHECK` still grades `/root/.hermes/wizard-outputs/spark/...` by absolute path and runs no `wizard-work/spark/test_*.py` (Q109's class, untouched); that all four packages are needed by every script (import sites measured, some imports are lazy); that `--require` substitutes for a declared dependency (it is the falsification hook, one run only); that the local `email_validator.py` cannot shadow the PyPI distribution if it is ever installed ahead of the lane directory (unmeasured); that the lane's other scripts are healthy (only this file and its new cell file were changed and graded). Files edited: `wizard-work/spark/python-env-check.py`, `wizard-work/spark/test_python_env_check.py` (new), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). No other lane's artifact was edited and nothing from `inbox/` is quoted. Report and CHECK: `/root/.hermes/wizard-outputs/volt/2026-10-29-q120.md`, `/root/.hermes/wizard-outputs/volt/q120-check.sh`. + diff --git a/docs/queue-index.md b/docs/queue-index.md index 6b49cb9..c02408a 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -4,7 +4,7 @@ **What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it. -**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 36 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 87 record blocks in all, 51 of them a later closure or correction for an id already opened. The same **36 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 29 times. +**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 36 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 88 record blocks in all, 52 of them a later closure or correction for an id already opened. The same **36 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 29 times. | # | id | kind | lines | bytes | role | title | first `Status:` | |---|---|---|---|---|---|---|---| @@ -191,7 +191,8 @@ | 181 | Q124 | record | 13350-13352 | 3669 | record: BOLD-LEAD | **Q124 — DONE (pipe, 2026-10-28; opened and closed on Volt's Q123 escalation, at `635cdc1`). The `checkin` leg's last tolerated row was a roster gap, not a fault: eight of the ten working lanes could … | -- | | 182 | Q122 | record | 13353-13357 | 4920 | cont: DONE | **Q122 — DONE (volt, 2026-10-28, at the head this record's commit names; ledger lane by authorship, unclaimed and taken). The gate no longer reads the wall clock, and it runs under the interpreter … | -- | | 183 | Q125 | heading | 13358-13380 | 3778 | opening | the spark lane holds eleven line/word/char counters, and the newest one replaced its predecessor's capabilities without retiring its verifier *(spark lane; opened 2026-10-28 by the pipe, at … | open and unclaimed. | -| 184 | Q125 | record | 13381-13397 | 6810 | cont: DONE | **Q125 — DONE (volt, 2026-10-29, at `46b6282`; the repair landed under the spark lane's banner at `a979e63` and `46b6282`, and this record is the closure the queue was missing — the item still read … | -- | +| 184 | Q125 | record | 13381-13399 | 6812 | cont: DONE | **Q125 — DONE (volt, 2026-10-29, at `46b6282`; the repair landed under the spark lane's banner at `a979e63` and `46b6282`, and this record is the closure the queue was missing — the item still read … | -- | +| 185 | Q120 | record | 13400-13401 | 3580 | cont: DONE | **Q120 — DONE (volt, 2026-10-29; this record and `docs/queue-index.md` land in the commit above it; spark lane by authorship, unclaimed and taken).** The health check now fails. Before: `python3 … | -- | ## Ids with no `## ` heading @@ -219,7 +220,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q106 | 13171 | 13171 | 11 | Q105 | BULLET+BOLD | BOLD-LEAD | | Q107 | 13189 | 13189 | 15 | Q106 | BULLET+BOLD | BOLD-LEAD | | Q108 | 13267 | 13263 | 3 | Q106 | BULLET+BOLD | BOLD-LEAD | -| Q109 | 13270 | 13270 | 7 | Q104 | BULLET+BOLD | BOLD-LEAD | +| Q109 | 13270 | 13270 | 8 | Q104 | BULLET+BOLD | BOLD-LEAD | | Q110 | 13272 | 13272 | 3 | Q109 | BULLET+BOLD | BOLD-LEAD | | Q111 | 13274 | 13274 | 7 | Q110 | BULLET+BOLD | BOLD-LEAD | | Q112 | 13276 | 13276 | 5 | Q111 | BULLET+BOLD | BOLD-LEAD | @@ -230,7 +231,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q117 | 13290 | 13288 | 3 | Q96 | BULLET+BOLD | BOLD-LEAD | | Q118 | 13300 | 13300 | 2 | Q111 | BULLET+BOLD | BOLD-LEAD | | Q119 | 13305 | 13305 | 2 | Q118 | BULLET+BOLD | BOLD-LEAD | -| Q120 | 13308 | 13308 | 6 | Q119 | BULLET+BOLD | BOLD-LEAD | +| Q120 | 13308 | 13308 | 7 | Q119 | BULLET+BOLD | BOLD-LEAD | | Q121 | 13342 | 13342 | 1 | Q107 | BULLET+BOLD | BOLD-LEAD | | Q122 | 13344 | 13344 | 4 | Q121 | BULLET+BOLD | BOLD-LEAD | | Q123 | 13346 | 13346 | 4 | Q122 | BULLET+BOLD | BOLD-LEAD | @@ -1365,17 +1366,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q122 \u2014 DONE (volt, 2026-10-28, at the head this record's commit names; ledger lane by authorship, unclaimed and taken). The gate no longer reads the wall clock, and it runs under the interpreter \u2026" }, { - "bytes": 6810, - "end_line": 13397, + "bytes": 6812, + "end_line": 13399, "id": "Q125", "kind": "record", "line": 13381, - "lines": 17, + "lines": 19, "role": "continuation", "role_detail": "DONE", "status": null, "status_line": null, "title": "**Q125 \u2014 DONE (volt, 2026-10-29, at `46b6282`; the repair landed under the spark lane's banner at `a979e63` and `46b6282`, and this record is the closure the queue was missing \u2014 the item still read \u2026" + }, + { + "bytes": 3580, + "end_line": 13401, + "id": "Q120", + "kind": "record", + "line": 13400, + "lines": 2, + "role": "continuation", + "role_detail": "DONE", + "status": null, + "status_line": null, + "title": "**Q120 \u2014 DONE (volt, 2026-10-29; this record and `docs/queue-index.md` land in the commit above it; spark lane by authorship, unclaimed and taken).** The health check now fails. Before: `python3 \u2026" } ], "ids_multi_section": { @@ -3893,17 +3907,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "the spark lane holds eleven line/word/char counters, and the newest one replaced its predecessor's capabilities without retiring its verifier *(spark lane; opened 2026-10-28 by the pipe, at \u2026" }, { - "bytes": 6810, - "end_line": 13397, + "bytes": 6812, + "end_line": 13399, "id": "Q125", "kind": "record", "line": 13381, - "lines": 17, + "lines": 19, "role": "continuation", "role_detail": "DONE", "status": null, "status_line": null, "title": "**Q125 \u2014 DONE (volt, 2026-10-29, at `46b6282`; the repair landed under the spark lane's banner at `a979e63` and `46b6282`, and this record is the closure the queue was missing \u2014 the item still read \u2026" + }, + { + "bytes": 3580, + "end_line": 13401, + "id": "Q120", + "kind": "record", + "line": 13400, + "lines": 2, + "role": "continuation", + "role_detail": "DONE", + "status": null, + "status_line": null, + "title": "**Q120 \u2014 DONE (volt, 2026-10-29; this record and `docs/queue-index.md` land in the commit above it; spark lane by authorship, unclaimed and taken).** The health check now fails. Before: `python3 \u2026" } ], "nested": [ @@ -4092,7 +4119,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q109", "line": 13270, - "refs": 7, + "refs": 8, "section": "Q104", "shape": "BULLET+BOLD" }, @@ -4191,7 +4218,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q120", "line": 13308, - "refs": 6, + "refs": 7, "section": "Q119", "shape": "BULLET+BOLD" }, @@ -4237,9 +4264,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 2076753, - "queue_lines": 13397, - "queue_sha256": "9a5bf2723e99c3a3adf463bc49d06e87219a1b2af300e61ca228bebbbf546541", + "queue_bytes": 2080335, + "queue_lines": 13401, + "queue_sha256": "0b2e463e90984d28203bbf66af98d38c1f432fd0294c8d9e7378d333f0d207b7", "sections": [ { "bytes": 3139, diff --git a/wizard-work/spark/python-env-check.py b/wizard-work/spark/python-env-check.py index 3d00d9c..ea72723 100644 --- a/wizard-work/spark/python-env-check.py +++ b/wizard-work/spark/python-env-check.py @@ -1,53 +1,140 @@ #!/usr/bin/env python3 -"""Python environment health check - verifies key packages are available.""" +"""Python environment health check for the spark lane. -import sys +The dependency list is the lane's own third-party imports, measured from +`wizard-work/spark/*.py` (`grep -rhoE '^(import|from) +[A-Za-z0-9_.]+'`): + + PyYAML -> yaml debug_yaml.py:2 + numpy -> numpy qc_analyze.py:7 + Pillow -> PIL ascii_tool.py:12, qc_analyze.py:6 + jsonschema -> jsonschema validate_json_schema.py:14 + +`email_validator` is deliberately NOT declared: the lane's +`wizard-work/spark/email_validator.py` is a local, stdlib-only module, so +declaring the PyPI distribution would test a package the lane never uses. + +The check can fail. A dependency that will not import is reported as +`"status": "missing"` and the process exits 1; a package that imports but +raises something other than ImportError is reported as `"status": "broken"`. +`--require` appends a dependency for one run, which is how the committed +cell in `test_python_env_check.py` proves the red path with a package that +is genuinely absent on the host. + +Exit codes (tools/ contract): + 0 every declared dependency imports and reports + 1 at least one declared or --require'd dependency is missing/broken + 2 usage error (argparse) +""" + +import argparse +import importlib +import importlib.metadata import json +import sys from datetime import datetime, timezone -def check_module(name): - try: - __import__(name) - return {"status": "ok", "available": True} - except ImportError as e: - return {"status": "missing", "available": False, "error": str(e)} +# (distribution name, import name) for the lane's own third-party imports +DEPENDENCIES = ( + ("PyYAML", "yaml"), + ("numpy", "numpy"), + ("Pillow", "PIL"), + ("jsonschema", "jsonschema"), +) -def check_package(name, import_name=None): - import_name = import_name or name - try: - mod = __import__(import_name) - version = getattr(mod, '__version__', 'unknown') - return {"status": "ok", "version": version, "available": True} - except ImportError as e: - return {"status": "missing", "available": False, "error": str(e)} -def main(): - results = { - "python": { - "status": "ok", - "version": sys.version, - "executable": sys.executable - }, - "packages": { - "json": check_module("json"), - "subprocess": check_module("subprocess"), - "datetime": check_module("datetime"), +def check_package(distribution, import_name=None): + """Import one dependency by name and report its version. + + The version is read from the distribution metadata first; `__version__` + is the fallback for packages that ship no metadata record. + """ + import_name = import_name or distribution + try: + mod = importlib.import_module(import_name) + except ImportError as exc: + return { + "status": "missing", + "available": False, + "import_name": import_name, + "error": "%s: %s" % (type(exc).__name__, exc), } + except Exception as exc: # a package that raises on import is not healthy + return { + "status": "broken", + "available": False, + "import_name": import_name, + "error": "%s: %s" % (type(exc).__name__, exc), + } + try: + version = importlib.metadata.version(distribution) + except importlib.metadata.PackageNotFoundError: + version = getattr(mod, "__version__", "unknown") + return { + "status": "ok", + "available": True, + "import_name": import_name, + "version": version, } - - all_ok = all( - p.get("status") == "ok" or p.get("available", False) - for p in results["packages"].values() + + +def parse_requirement(text): + """`NAME` or `DISTRIBUTION:IMPORT` -> (distribution, import name).""" + if ":" in text: + distribution, import_name = text.split(":", 1) + if not distribution or not import_name: + raise argparse.ArgumentTypeError( + "--require needs NAME or DISTRIBUTION:IMPORT, got %r" % text + ) + return distribution, import_name + return text, text + + +def build_parser(): + parser = argparse.ArgumentParser( + prog="python-env-check.py", + description="Verify the spark lane's third-party dependencies import, " + "and report their versions.", + ) + parser.add_argument( + "--require", + metavar="NAME[:IMPORT]", + action="append", + default=[], + type=parse_requirement, + help="also require this package for this run (repeatable); " + "useful for proving the red path", + ) + return parser + + +def main(argv=None): + args = build_parser().parse_args(argv) + declared = list(DEPENDENCIES) + list(args.require) + packages = {} + for distribution, import_name in declared: + result = check_package(distribution, import_name) + result["distribution"] = distribution + packages[import_name] = result + missing = sorted( + name for name, result in packages.items() if result["status"] != "ok" ) - output = { - "all_ok": all_ok, + "all_ok": not missing, "timestamp": datetime.now(timezone.utc).isoformat(), - "components": results + "dependencies_declared": len(declared), + "missing": missing, + "components": { + "python": { + "status": "ok", + "version": sys.version, + "executable": sys.executable, + }, + "packages": packages, + }, } - - print(json.dumps(output, indent=2)) - sys.exit(0 if all_ok else 1) + print(json.dumps(output, indent=2, sort_keys=True)) + return 0 if not missing else 1 + if __name__ == "__main__": - main() \ No newline at end of file + sys.exit(main()) diff --git a/wizard-work/spark/test_python_env_check.py b/wizard-work/spark/test_python_env_check.py new file mode 100644 index 0000000..465cfff --- /dev/null +++ b/wizard-work/spark/test_python_env_check.py @@ -0,0 +1,129 @@ +#!/usr/bin/env python3 +"""Cells for wizard-work/spark/python-env-check.py (queue Q120). + +The script under test is run as a subprocess, exactly as an operator runs +it; nothing here imports it. The red cell picks its missing package by +measuring the host first, so it is a real absence rather than a name the +test hopes is absent. + +Run: python3 -m unittest discover -p 'test_python_env_check.py' -v + (from wizard-work/spark, or by path from anywhere) +""" + +import importlib.util +import json +import os +import subprocess +import sys +import unittest + +HERE = os.path.dirname(os.path.abspath(__file__)) +SCRIPT = os.path.join(HERE, "python-env-check.py") + +# Real, widely-installed packages the spark lane does not depend on. The red +# cell requires whichever of these the host genuinely lacks. +RED_CANDIDATES = ("flask", "django", "pandas", "scipy", "tensorflow", "torch", "black") + + +def run_script(*args, cwd=None): + proc = subprocess.run( + [sys.executable, SCRIPT] + list(args), + cwd=cwd or HERE, + capture_output=True, + text=True, + ) + return proc + + +def absent_package(): + """Return the first candidate module that cannot be found on this host.""" + for name in RED_CANDIDATES: + try: + found = importlib.util.find_spec(name) + except (ImportError, ValueError): + found = None + if found is None: + return name + return None + + +class EnvCheckContract(unittest.TestCase): + def setUp(self): + self.proc = run_script() + self.assertEqual( + self.proc.returncode in (0, 1), + True, + "expected rc 0 or 1, got %r: %s" % (self.proc.returncode, self.proc.stderr), + ) + self.report = json.loads(self.proc.stdout) + + def test_declared_list_is_not_the_standard_library(self): + # The defect Q120 filed: the only names checked were json/subprocess/ + # datetime, so no healthy interpreter could ever fail this script. + packages = self.report["components"]["packages"] + self.assertTrue(packages, "no packages reported at all") + stdlib = set(getattr(sys, "stdlib_module_names", ())) + for import_name in packages: + self.assertNotIn( + import_name, + stdlib, + "%r is a standard-library module, not a lane dependency" % import_name, + ) + + def test_declared_list_is_the_lane_dependencies(self): + packages = self.report["components"]["packages"] + self.assertEqual( + set(packages), {"yaml", "numpy", "PIL", "jsonschema"} + ) + # email_validator is the lane's own stdlib-only module, not the PyPI + # distribution; declaring it would test something the lane never uses. + self.assertNotIn("email_validator", packages) + + def test_every_declared_dependency_reports_a_version(self): + packages = self.report["components"]["packages"] + for import_name, result in packages.items(): + self.assertEqual(result["status"], "ok", "%s: %r" % (import_name, result)) + self.assertIn("version", result) + self.assertIn("distribution", result) + self.assertFalse( + result["version"] in ("", "unknown"), + "%s reported no version: %r" % (import_name, result), + ) + + def test_all_ok_agrees_with_the_missing_list(self): + self.assertEqual(self.report["all_ok"], self.report["missing"] == []) + self.assertEqual(self.proc.returncode, 0 if self.report["all_ok"] else 1) + + def test_genuinely_absent_package_turns_the_reading_red(self): + name = absent_package() + if name is None: + self.skipTest( + "every candidate %r is installed here; no real package to " + "prove the red path with" % (RED_CANDIDATES,) + ) + proc = run_script("--require", name) + self.assertEqual(proc.returncode, 1, proc.stdout + proc.stderr) + report = json.loads(proc.stdout) + self.assertFalse(report["all_ok"]) + self.assertEqual(report["missing"], [name]) + result = report["components"]["packages"][name] + self.assertEqual(result["status"], "missing") + self.assertIn(name, result["error"]) + # The healthy declared dependencies still report alongside it. + self.assertEqual(report["dependencies_declared"], 5) + self.assertEqual( + report["components"]["packages"]["yaml"]["status"], "ok" + ) + + def test_usage_error_is_rc_two(self): + proc = run_script("--require") + self.assertEqual(proc.returncode, 2, proc.stderr) + + def test_same_reading_from_a_different_cwd(self): + other = run_script(cwd="/") + self.assertEqual(other.returncode, self.proc.returncode) + self.assertEqual(json.loads(other.stdout)["missing"], self.report["missing"]) + + +if __name__ == "__main__": + unittest.main(verbosity=2)