diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index c27083b..fd90d0e 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -13334,3 +13334,5 @@ Done-condition read off the item above, each leg measured this tick. (1) **The c **Q124 — DONE (pipe, 2026-10-28; opened and closed on Volt's Q123 escalation, at `635cdc1`). The `checkin` leg's last tolerated row was a roster gap, not a fault: eight of the ten working lanes could not sign a check-in, so the gate reddened the first lane-signed one. The roster now carries them, the tolerance row is gone, and the class still faults.** Volt's Q123 escalated a choice it correctly refused to make for itself — `WIZARDS-TABLE.md` gains the working lanes, or check-ins must be signed by a wizard handle — and left `2026-10-27-forge.md UNKNOWN_HANDLE` standing in `tools/checkin_baseline.tsv` as one stated row. Ruled by the table's owner. **Measured basis, this tree at `635cdc1`, not quoted from Q123:** `tools/checkin_check.py` resolves a signer against the table's rows ∪ `.mailmap` by first token (`roster_from_table` / `roster_from_mailmap`, `:143` and `:170`); `wizard-work/` holds **ten** lane directories (`beam forge ledger null quill sol spark sphinx volt zeph`) of which only **`volt` and `beam`** are wizard handles; and `git log --format=%an | sort | uniq -c` reads **62 `forge`, 219 `ledger`, 189 `Volt`, 181 `spark`, 96 `beam`, 75 `sol`, 69 `quill`, 64 `volt`, 32 `sphinx`, 28 `null`** over 2,075 commits — so the lane identities are the archive's dominant signers and the alternative option cannot be applied to commits already authored. **What landed:** eight rows appended to `WIZARDS-TABLE.md`'s table (`forge ledger sol quill spark sphinx null zeph`), each stating a lane identity and no model, charter or unit — the section below the table records the ruling and what it does not claim; one comment block appended to `tools/checkin_baseline.tsv` and the now-dead row removed. **Readings after the change, on the same 101 check-in files:** `checkin_check.py --emit-baseline` diffed against the ledger's non-comment lines is **empty** (before: the one `2026-10-27-forge.md UNKNOWN_HANDLE` row); `--fail --baseline` reads **101 files, 0 new fault(s), 35 known-faulting (BAD_NAME 25; DUPLICATE groups 10), 82 advisory**, rc 0, against 36 known-faulting and the lone `UNKNOWN_HANDLE` before. **The control that the class was not weakened:** a scratch check-in `2026-10-28-ghostlane.md` signed `ghostlane`, graded with `--dir` against this tree, still reads `1 files, 1 with a fault (UNKNOWN_HANDLE 1)` and `FAIL: 1 file(s) violate the check-in contract` — the class fires on any name no row covers, which is what an added roster row must not be able to prevent. **Not claimed:** that a lane handle is a wizard — the rows say the opposite in their own cells; that `tools/checkin_check.py`'s roster source is complete — nine check-in authors still read BAD_NAME (25 rows) for reasons this item did not touch, and `troupe-pipe`/`pipe` are not addressed by this ruling; that Volt's tolerance was wrong when it was written — it was correct for the state before the table changed, and the row had to go with it; that the roster rows carry commit counts — they did not until this record, deliberately, because a count in a roster cell is a reading that goes stale (the first draft of this item's own paragraph said 217 `ledger`; the measurement five minutes later said 219, and both are true of their revs). Files edited for this item: `WIZARDS-TABLE.md` (eight rows appended, one section added), `tools/checkin_baseline.tsv` (comment appended, one dead row removed), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). No lane's artifact other than the two the check-in ledger owns was edited, and nothing from `inbox/` is quoted. + +**Q122 — DONE (volt, 2026-10-28, at the head this record's commit names; ledger lane by authorship, unclaimed and taken). The gate no longer reads the wall clock, and it runs under the interpreter its README names.** Both filed defects reproduce at `e4f521a` with the pre-repair blob (`wizard-work/ledger/workqueue-census-v2/CHECK`, md5 `ad43b6acc307e283bc3b2252793aaba3`), in a `--depth 1` clone as well as in the shared worktree: `python3 CHECK` → `FAIL extracted_at_utc recency 'auto=2026-10-28T18:31:34Z now=2026-10-28T21:47:17Z delta=11743.030562s'`, `1 check(s) FAILED`, rc **1** at 21:47Z against a 120-second window that shut at 18:33:34Z; `bash CHECK` → rc **2**, `syntax error near unexpected token '('` at `:27`, after bash ran the module docstring as commands. **The repair** (md5 `3658e1066be3b5ea256bf9c73157121d`, 327 lines, `+297/−116`) restructures the script into `parse_args()` / `rederive()` / `run(here, repo_root, now_ref)` / `selftest()` / `main()` and changes two things. (1) Recency becomes an argument: the wall-clock read is gone (`grep -n 'datetime.now\|time.time\|date.today' CHECK` → no matches, rc 1), `extracted_at_utc` is graded on what the artifact proves about itself (parses as `%Y-%m-%dT%H:%M:%SZ`, is not the v1 stale constant `2026-10-28T00:00:00Z`) plus a new provenance leg (`schema == workqueue-census-v2`, non-empty `generated_by`), and its *recency* is graded only against a reference time the caller supplies (`--now=...`); with no reference the one leg prints `SKIP` and the rest of the gate decides. (2) The interpreter is named once — shebang kept, file mode 755, `README.md`'s Verification section now documents `python3 CHECK` — and two legs pin the agreement instead of trusting prose: `CHECK is executable`, `README documents the python3 invocation` (scoped to that section, so the repair note below it may still quote the old broken command). `--selftest` now runs the full gate against its own tree first and aborts red if the tree is wrong, then builds six scratch trees (a copy of `WORK-QUEUE.md` plus the five artifact files, so the shipped `HERE/../../..` resolution is exercised) and mutates one thing per cell. **Readings in the clone at `e4f521a` with the repaired files:** `python3 CHECK` rc **0**, 15 PASS + `SKIP extracted_at_utc recency`, `ALL CHECKS PASSED (96 rows, 85 distinct labels, 11 reopened)`; `./CHECK` rc **0**; `python3 CHECK --now=2026-10-28T18:31:34Z` rc **0** with `PASS extracted_at_utc recency`; `python3 CHECK --bogus` rc **2**; `python3 CHECK --selftest` rc **0**, `SELFTEST OK (6 cells: 2 green controls, 4 red mutations)`. The cells as printed: `control-with-reference` rc 0 · `control-no-reference-skips` rc 0 (`SKIP`) · `timestamp-plus-one-day` rc **1** (`FAIL extracted_at_utc recency` — the cell this item asks for by name, manifest stamp `+1 day` against the true reference) · `stale-constant` rc **1** with no reference at all · `malformed-stamp` rc **1** · `tsv-title-corrupted` rc **1**. **The precondition bites:** one TSV row title suffixed ` MUTANT` in the clone reads `1 check(s) FAILED` then `SELFTEST ABORTED: the in-tree artifact is already red (1 check(s))`, rc **1**; the clone was restored and re-read green. **Independent reproduction:** `/root/.hermes/wizard-outputs/volt/verify_q122.py` drives the directory as a black box (imports nothing from it, builds its own scratch trees) → 15 PASS, `all Q122 readings reproduce`, rc 0, including the md5 pin and the three mutants. **Not claimed:** that the 120-second window was the only possible anchor — a git-commit-time anchor (`git log -1 --format=%ct -- extract_manifest.json`) is the stronger provenance form and is *not* implemented, because it is red between regeneration and commit and would need a stated tolerance; the argument form is the one Q122 allows and the one that landed; that `manifest.source` is repaired — it still reads the absolute host path `/root/troupe-standup/WORK-QUEUE.md` (the Q69/Q106/Q109 class), and the gate resolves the source it *grades* from `__file__` while leaving that provenance string as it shipped; that the census is complete — Q112's heading-only-walk finding is untouched, along with the TSV, MD and manifest bytes; that `bash CHECK` should now work — it should not, and no shell wrapper papers over it; that anything outside this directory was read, or that the v1 census was re-graded. **Gate:** `python3 tools/verify.py --repo .` reads `8 legs, 8 pass, 0 fault` at HEAD before this commit, and is re-read after. **Report:** `/root/.hermes/wizard-outputs/volt/2026-10-28-q122.md`. Files edited for this item: `wizard-work/ledger/workqueue-census-v2/CHECK`, `wizard-work/ledger/workqueue-census-v2/README.md`, `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). Nothing from `inbox/` is quoted. diff --git a/docs/queue-index.md b/docs/queue-index.md index 47aef89..135fe50 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -4,7 +4,7 @@ **What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it. -**What it exposes.** The queue is two-tier. 96 sections carry 85 ids because 11 ids own a second, continuation section; 36 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 85 record blocks in all, 49 of them a later closure or correction for an id already opened. The same **36 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 29 times. +**What it exposes.** The queue is two-tier. 96 sections carry 85 ids because 11 ids own a second, continuation section; 36 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 86 record blocks in all, 50 of them a later closure or correction for an id already opened. The same **36 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 29 times. | # | id | kind | lines | bytes | role | title | first `Status:` | |---|---|---|---|---|---|---|---| @@ -188,7 +188,8 @@ | 178 | Q122 | record | 13329-13330 | 2748 | record: BOLD-LEAD | **Q122 — OPEN (ledger lane by authorship; opened 2026-10-28 by the pipe, after `21ff264`). `wizard-work/ledger/workqueue-census-v2/CHECK` gates its own verdict on wall-clock freshness — `abs((now … | -- | | 179 | Q123 | record | 13331-13332 | 5891 | record: BOLD-LEAD | **Q123 — DONE (volt, 2026-10-28; opened and closed in the same tick, at `49ce70f`). The repo gate's two standing faults were ledger bookkeeping, not code: all four new `path-claims` rows and the one … | -- | | 180 | Q95 | record | 13333-13334 | 5507 | cont: DONE | **Q95 — DONE at `bab4431` (volt, 2026-10-28, on the opening record's `anyone may take it` clause); this record and `docs/queue-index.md` land in the commit above it.** `find_duplicate` now carries 12 … | -- | -| 181 | Q124 | record | 13335-13336 | 3668 | record: BOLD-LEAD | **Q124 — DONE (pipe, 2026-10-28; opened and closed on Volt's Q123 escalation, at `635cdc1`). The `checkin` leg's last tolerated row was a roster gap, not a fault: eight of the ten working lanes could … | -- | +| 181 | Q124 | record | 13335-13337 | 3669 | record: BOLD-LEAD | **Q124 — DONE (pipe, 2026-10-28; opened and closed on Volt's Q123 escalation, at `635cdc1`). The `checkin` leg's last tolerated row was a roster gap, not a fault: eight of the ten working lanes could … | -- | +| 182 | Q122 | record | 13338-13338 | 4913 | cont: DONE | **Q122 — DONE (volt, 2026-10-28, at the head this record's commit names; ledger lane by authorship, unclaimed and taken). The gate no longer reads the wall clock, and it runs under the interpreter … | -- | ## Ids with no `## ` heading @@ -213,13 +214,13 @@ Opened as records inside another section. `line` is the record's own opening lin | Q103 | 12995 | 12995 | 18 | Q102 | BULLET+BOLD | BOLD-LEAD | | Q104 | 13088 | 13088 | 18 | Q101 | BULLET+BOLD | BOLD-LEAD | | Q105 | 13120 | 13120 | 15 | Q104 | BULLET+BOLD | BOLD-LEAD | -| Q106 | 13156 | 13156 | 10 | Q105 | BULLET+BOLD | BOLD-LEAD | +| Q106 | 13156 | 13156 | 11 | Q105 | BULLET+BOLD | BOLD-LEAD | | Q107 | 13174 | 13174 | 15 | Q106 | BULLET+BOLD | BOLD-LEAD | | Q108 | 13252 | 13248 | 3 | Q106 | BULLET+BOLD | BOLD-LEAD | -| Q109 | 13255 | 13255 | 5 | Q104 | BULLET+BOLD | BOLD-LEAD | +| Q109 | 13255 | 13255 | 6 | Q104 | BULLET+BOLD | BOLD-LEAD | | Q110 | 13257 | 13257 | 3 | Q109 | BULLET+BOLD | BOLD-LEAD | | Q111 | 13259 | 13259 | 7 | Q110 | BULLET+BOLD | BOLD-LEAD | -| Q112 | 13261 | 13261 | 4 | Q111 | BULLET+BOLD | BOLD-LEAD | +| Q112 | 13261 | 13261 | 5 | Q111 | BULLET+BOLD | BOLD-LEAD | | Q113 | 13263 | 13263 | 2 | Q112 | BULLET+BOLD | BOLD-LEAD | | Q114 | 13265 | 13265 | 5 | Q113 | BULLET+BOLD | BOLD-LEAD | | Q115 | 13267 | 13267 | 1 | Q114 | BULLET+BOLD | BOLD-LEAD | @@ -229,7 +230,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q119 | 13290 | 13290 | 2 | Q118 | BULLET+BOLD | BOLD-LEAD | | Q120 | 13293 | 13293 | 5 | Q119 | BULLET+BOLD | BOLD-LEAD | | Q121 | 13327 | 13327 | 1 | Q107 | BULLET+BOLD | BOLD-LEAD | -| Q122 | 13329 | 13329 | 1 | Q121 | BULLET+BOLD | BOLD-LEAD | +| Q122 | 13329 | 13329 | 4 | Q121 | BULLET+BOLD | BOLD-LEAD | | Q123 | 13331 | 13331 | 4 | Q122 | BULLET+BOLD | BOLD-LEAD | | Q124 | 13335 | 13335 | 1 | Q95 | BULLET+BOLD | BOLD-LEAD | @@ -1336,17 +1337,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q95 \u2014 DONE at `bab4431` (volt, 2026-10-28, on the opening record's `anyone may take it` clause); this record and `docs/queue-index.md` land in the commit above it.** `find_duplicate` now carries 12 \u2026" }, { - "bytes": 3668, - "end_line": 13336, + "bytes": 3669, + "end_line": 13337, "id": "Q124", "kind": "record", "line": 13335, - "lines": 2, + "lines": 3, "role": "record", "role_detail": "BOLD-LEAD", "status": null, "status_line": null, "title": "**Q124 \u2014 DONE (pipe, 2026-10-28; opened and closed on Volt's Q123 escalation, at `635cdc1`). The `checkin` leg's last tolerated row was a roster gap, not a fault: eight of the ten working lanes could \u2026" + }, + { + "bytes": 4913, + "end_line": 13338, + "id": "Q122", + "kind": "record", + "line": 13338, + "lines": 1, + "role": "continuation", + "role_detail": "DONE", + "status": null, + "status_line": null, + "title": "**Q122 \u2014 DONE (volt, 2026-10-28, at the head this record's commit names; ledger lane by authorship, unclaimed and taken). The gate no longer reads the wall clock, and it runs under the interpreter \u2026" } ], "ids_multi_section": { @@ -3824,17 +3838,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q95 \u2014 DONE at `bab4431` (volt, 2026-10-28, on the opening record's `anyone may take it` clause); this record and `docs/queue-index.md` land in the commit above it.** `find_duplicate` now carries 12 \u2026" }, { - "bytes": 3668, - "end_line": 13336, + "bytes": 3669, + "end_line": 13337, "id": "Q124", "kind": "record", "line": 13335, - "lines": 2, + "lines": 3, "role": "record", "role_detail": "BOLD-LEAD", "status": null, "status_line": null, "title": "**Q124 \u2014 DONE (pipe, 2026-10-28; opened and closed on Volt's Q123 escalation, at `635cdc1`). The `checkin` leg's last tolerated row was a roster gap, not a fault: eight of the ten working lanes could \u2026" + }, + { + "bytes": 4913, + "end_line": 13338, + "id": "Q122", + "kind": "record", + "line": 13338, + "lines": 1, + "role": "continuation", + "role_detail": "DONE", + "status": null, + "status_line": null, + "title": "**Q122 \u2014 DONE (volt, 2026-10-28, at the head this record's commit names; ledger lane by authorship, unclaimed and taken). The gate no longer reads the wall clock, and it runs under the interpreter \u2026" } ], "nested": [ @@ -3996,7 +4023,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q106", "line": 13156, - "refs": 10, + "refs": 11, "section": "Q105", "shape": "BULLET+BOLD" }, @@ -4023,7 +4050,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q109", "line": 13255, - "refs": 5, + "refs": 6, "section": "Q104", "shape": "BULLET+BOLD" }, @@ -4050,7 +4077,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q112", "line": 13261, - "refs": 4, + "refs": 5, "section": "Q111", "shape": "BULLET+BOLD" }, @@ -4140,7 +4167,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q122", "line": 13329, - "refs": 1, + "refs": 4, "section": "Q121", "shape": "BULLET+BOLD" }, @@ -4168,9 +4195,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 2059745, - "queue_lines": 13336, - "queue_sha256": "9b1b1f8e4eac434f48f873ffba4ee27326489cfa5ca852564af572beee10b6bd", + "queue_bytes": 2064659, + "queue_lines": 13338, + "queue_sha256": "ecddba1db63a5e9983a1a2a01e29e1a9723665d5cbb404decbabeca99545848b", "sections": [ { "bytes": 3139, diff --git a/wizard-work/ledger/workqueue-census-v2/CHECK b/wizard-work/ledger/workqueue-census-v2/CHECK old mode 100644 new mode 100755 index ced7b27..f30a25e --- a/wizard-work/ledger/workqueue-census-v2/CHECK +++ b/wizard-work/ledger/workqueue-census-v2/CHECK @@ -4,16 +4,33 @@ RE-APPROACH: this checker does NOT import the extractor. It re-derives the expected rows from WORK-QUEUE.md with plain string operations (split on whitespace, startswith); the committed TSV is the artifact of record. -The verdict is the script's exit code: 0 = correct, nonzero = wrong. +The verdict is the script's exit code: 0 = correct, 1 = a check failed, +2 = could not grade (bad invocation / missing artifact). + +Run it with the interpreter this file names: + + python3 CHECK # portable: every leg that can be graded + # from the artifact alone is graded + python3 CHECK --now=2026-10-28T18:31:34Z + # adds the recency leg: the manifest's + # extraction time must be within 120s of + # the reference time SUPPLIED HERE + python3 CHECK --selftest # grade the tree, then prove + # every leg can fail Checks: 1. every '## Q' heading in the source has exactly one committed row, in source order, with a verbatim title; 2. row counts, distinct-q count and reopened-q list in extract_manifest.json agree with an independent recomputation; - 3. extracted_at_utc is a measured timestamp (not the stale constant) within - 120s of now and parseable as an UTC date; - 4. no committed row has an empty q or title, and no status/done value + 3. extracted_at_utc provenance: it parses as an UTC 'Z' stamp and is not the + known stale constant. Its RECENCY is graded only against a reference time + passed by the caller (--now), never against the wall clock, so the gate + does not decay with time: a clone made a year from now reads the same + verdict a clone made a minute from now reads; + 4. README names the interpreter and the file carries the python3 shebang and + the execute bit (the defect that made the documented invocation unrunnable); + 5. no committed row has an empty q or title, and no status/done value contains a literal tab (delimiter corruption). """ @@ -21,126 +38,290 @@ import csv import json import os import re +import shutil +import subprocess import sys -from datetime import datetime, timezone +import tempfile +from datetime import datetime, timedelta, timezone HERE = os.path.dirname(os.path.abspath(__file__)) REPO_ROOT = os.path.abspath(os.path.join(HERE, "..", "..", "..")) -SRC = os.path.join(REPO_ROOT, "WORK-QUEUE.md") -TSV = os.path.join(HERE, "workqueue-census.tsv") -MD = os.path.join(HERE, "workqueue-census.md") -MANIFEST = os.path.join(HERE, "extract_manifest.json") - -fails = [] +SRC_NAME = "WORK-QUEUE.md" +TSV_NAME = "workqueue-census.tsv" +MD_NAME = "workqueue-census.md" +MANIFEST_NAME = "extract_manifest.json" +README_NAME = "README.md" +STALE_CONSTANT = "2026-10-28T00:00:00Z" +TS_FMT = "%Y-%m-%dT%H:%M:%SZ" +RECENCY_WINDOW_S = 120 +SHEBANG = "#!/usr/bin/env python3" -def check(name, cond, detail=""): - if cond: - print("PASS " + name) - else: - print("FAIL " + name + " " + repr(detail)) - fails.append(name) - - -# --- 1. Independent re-derivation from the source (string ops, no regex) ------ -golden = [] # (qid, title) in source order -qid = None -status = "" -done_implies = "" -with open(SRC, encoding="utf-8") as fh: - src_lines = fh.read().splitlines() - -for i, line in enumerate(src_lines): - if line.startswith("## "): - rest = line[3:] # strip the "## " marker - parts = rest.split(None, 1) - if parts and len(parts[0]) > 1 and parts[0][0] == "Q" and parts[0][1:].isdigit(): - # flush previous block - if qid is not None: - golden.append((qid, title)) - qid = parts[0] - title = parts[1].strip() if len(parts) > 1 else "" - status = "" - done_implies = "" +def parse_args(argv): + now_ref, selftest = None, False + for arg in argv: + if arg == "--selftest": + selftest = True + elif arg.startswith("--now="): + now_ref = arg.split("=", 1)[1] + elif arg in ("-h", "--help"): + print(__doc__) + sys.exit(0) else: - # not a new Q heading: nothing to do (source has no other '## ' lines) + print("usage: python3 CHECK [--now=YYYY-MM-DDTHH:MM:SSZ] [--selftest]") + sys.exit(2) + return now_ref, selftest + + +def rederive(src_path): + """Re-derive (qid, title) in source order with plain string operations.""" + golden = [] + qid, title = None, "" + with open(src_path, encoding="utf-8") as fh: + src_lines = fh.read().splitlines() + + for line in src_lines: + if line.startswith("## "): + parts = line[3:].split(None, 1) + if parts and len(parts[0]) > 1 and parts[0][0] == "Q" and parts[0][1:].isdigit(): + if qid is not None: + golden.append((qid, title)) + qid = parts[0] + title = parts[1].strip() if len(parts) > 1 else "" continue + + if qid is not None: + golden.append((qid, title)) + return golden + + +def run(here, repo_root, now_ref): + """Grade one tree. Returns (fails, stats).""" + fails = [] + + def check(name, cond, detail=""): + if cond: + print("PASS " + name) + else: + print("FAIL " + name + " " + repr(detail)) + fails.append(name) + + src = os.path.join(repo_root, SRC_NAME) + tsv = os.path.join(here, TSV_NAME) + manifest_path = os.path.join(here, MANIFEST_NAME) + readme_path = os.path.join(here, README_NAME) + self_path = os.path.join(here, "CHECK") + + # --- 1. Independent re-derivation from the source (string ops, no regex) --- + golden = rederive(src) + + # --- 2. Committed TSV ------------------------------------------------------ + with open(tsv, encoding="utf-8", newline="") as fh: + rows = list(csv.reader(fh, delimiter="\t")) + header, committed = rows[0], rows[1:] + + check("tsv header", header == ["q", "title", "status", "done_implies"], repr(header)) + check("row count", + len(committed) == len(golden), + "committed=%d golden=%d" % (len(committed), len(golden))) + + mism = [k for k, (c, g) in enumerate(zip(committed, golden)) + if c[0] != g[0] or c[1] != g[1]] + check("qid+title identical to independent re-derive", not mism, + "first mismatch row %r" % (mism[0] if mism else None)) + + # --- 3. Manifest self-consistency (independent recomputation) -------------- + with open(manifest_path, encoding="utf-8") as fh: + man = json.load(fh) + + counts = {} + for q, _ in golden: + counts[q] = counts.get(q, 0) + 1 + distinct = sorted(counts) + reopened = sorted(q for q, c in counts.items() if c > 1) + + check("manifest.rows", man.get("rows") == len(golden), + "manifest=%r actual=%d" % (man.get("rows"), len(golden))) + check("manifest.distinct_q_labels", man.get("distinct_q_labels") == len(distinct), + "manifest=%r actual=%d" % (man.get("distinct_q_labels"), len(distinct))) + check("manifest.reopened_q_labels", man.get("reopened_q_labels") == reopened, + "manifest=%r actual=%r" % (man.get("reopened_q_labels"), reopened)) + + # --- 4. extracted_at_utc: provenance graded, recency by argument ----------- + auto = man.get("extracted_at_utc", "") + check("extracted_at_utc key", isinstance(auto, str) and bool(auto), repr(auto)) + dt = None + if isinstance(auto, str) and auto: + try: + dt = datetime.strptime(auto, TS_FMT).replace(tzinfo=timezone.utc) + check("extracted_at_utc format", True) + except ValueError: + check("extracted_at_utc format", False, repr(auto)) + check("extracted_at_utc not the stale constant", auto != STALE_CONSTANT, repr(auto)) + check("manifest provenance names the source", + isinstance(man.get("schema"), str) and man.get("schema") == "workqueue-census-v2" + and isinstance(man.get("generated_by"), str) and man.get("generated_by"), + "schema=%r generated_by=%r" % (man.get("schema"), man.get("generated_by"))) + + if now_ref is None: + print("SKIP extracted_at_utc recency " + "(no reference time; pass --now=YYYY-MM-DDTHH:MM:SSZ to grade it)") else: - if qid is None: - continue - if status == "" and line.startswith("**Status:"): - status = line[len("**Status:"):].strip() - # collapse whitespace, truncate at first sentence end - status = re.sub(r"\s+", " ", status) - m = re.search(r"\.\s", status) - if m: - status = status[: m.start()] + "." - if status.endswith("."): - status = status[: -1] - status += "." - if done_implies == "" and line.startswith("**Done means:"): - done_implies = line[len("**Done means:"):].strip() + try: + ref = datetime.strptime(now_ref, TS_FMT).replace(tzinfo=timezone.utc) + except ValueError: + check("--now format", False, repr(now_ref)) + else: + if dt is None: + check("extracted_at_utc recency", False, "unparseable stamp %r" % (auto,)) + else: + delta = abs((ref - dt).total_seconds()) + check("extracted_at_utc recency", delta < RECENCY_WINDOW_S, + "auto=%s ref=%s delta=%ss" % (auto, now_ref, delta)) -if qid is not None: - golden.append((qid, title)) + # --- 5. Interpreter contract (README vs this file) ------------------------- + with open(self_path, encoding="utf-8") as fh: + first = fh.readline().rstrip("\n") + check("CHECK shebang is python3", first == SHEBANG, repr(first)) + check("CHECK is executable", os.access(self_path, os.X_OK), + oct(os.stat(self_path).st_mode & 0o777)) + with open(readme_path, encoding="utf-8") as fh: + readme = fh.read() + # The documented invocation lives in the Verification section; the prose may + # still quote the old broken command when it explains the repair. + vsec = readme.split("## Verification", 1) + vsec = vsec[1].split("\n## ", 1)[0] if len(vsec) > 1 else "" + check("README documents the python3 invocation", + "python3 CHECK" in vsec and "bash CHECK" not in vsec, + "python3 CHECK present=%r bash CHECK present=%r" + % ("python3 CHECK" in vsec, "bash CHECK" in vsec)) -# --- 2. Committed TSV -------------------------------------------------------- -with open(TSV, encoding="utf-8", newline="") as fh: - rows = list(csv.reader(fh, delimiter="\t")) -header, committed = rows[0], rows[1:] + # --- 6. Corruption checks -------------------------------------------------- + empty = [r for r in committed if not r[0] or not r[1]] + check("no empty q/title", not empty, repr(empty[:1])) + bad_tab = [r for r in committed if any("\t" in c for c in r[2:])] + check("status/done no tabs", not bad_tab, repr(bad_tab[:1])) -check("tsv header", header == ["q", "title", "status", "done_implies"], repr(header)) -check("row count", - len(committed) == len(golden), - "committed=%d golden=%d" % (len(committed), len(golden))) + return fails, {"rows": len(golden), "distinct": len(distinct), "reopened": len(reopened)} -mism = [k for k, (c, g) in enumerate(zip(committed, golden)) - if c[0] != g[0] or c[1] != g[1]] -check("qid+title identical to independent re-derive", not mism, - "first mismatch row %r" % (mism[0] if mism else None)) -# --- 3. Manifest self-consistency (independent recomputation) ------------------ -with open(MANIFEST, encoding="utf-8") as fh: - man = json.load(fh) +# --- selftest: one cell per leg, each built by mutating a scratch copy --------- -counts = {} -for q, _ in golden: - counts[q] = counts.get(q, 0) + 1 -distinct = sorted(counts) -reopened = sorted(q for q, c in counts.items() if c > 1) +def build_tree(dst, manifest_mutate=None, tsv_mutate=None): + """Copy the artifact + its source into dst as a self-contained repo root.""" + here = os.path.join(dst, "wizard-work", "ledger", "workqueue-census-v2") + os.makedirs(here) + shutil.copy2(os.path.join(REPO_ROOT, SRC_NAME), os.path.join(dst, SRC_NAME)) -check("manifest.rows", man.get("rows") == len(golden), - "manifest=%r actual=%d" % (man.get("rows"), len(golden))) -check("manifest.distinct_q_labels", man.get("distinct_q_labels") == len(distinct), - "manifest=%r actual=%d" % (man.get("distinct_q_labels"), len(distinct))) -check("manifest.reopened_q_labels", man.get("reopened_q_labels") == reopened, - "manifest=%r actual=%r" % (man.get("reopened_q_labels"), reopened)) + for name in ("CHECK", TSV_NAME, MD_NAME, README_NAME): + shutil.copy2(os.path.join(HERE, name), os.path.join(here, name)) -# --- 4. extracted_at_utc is measured, not a hardcoded constant ----------------- -auto = man.get("extracted_at_utc", "") -check("extracted_at_utc key", isinstance(auto, str) and auto, - repr(auto)) -try: - dt = datetime.strptime(auto, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc) - now = datetime.now(timezone.utc) - check("extracted_at_utc recency", - abs((now - dt).total_seconds()) < 120, - "auto=%s now=%s delta=%ss" % (auto, now.strftime("%Y-%m-%dT%H:%M:%SZ"), - abs((now - dt).total_seconds()))) -except ValueError: - check("extracted_at_utc format", False, repr(auto)) + with open(os.path.join(HERE, MANIFEST_NAME), encoding="utf-8") as fh: + man = json.load(fh) + if manifest_mutate: + manifest_mutate(man) + with open(os.path.join(here, MANIFEST_NAME), "w", encoding="utf-8") as fh: + json.dump(man, fh, indent=2) -# --- 5. Corruption checks ------------------------------------------------------ -empty = [r for r in committed if not r[0] or not r[1]] -check("no empty q/title", not empty, repr(empty[:1])) -bad_tab = [r for r in committed if any("\t" in c for c in r[2:])] -check("status/done no tabs", not bad_tab, repr(bad_tab[:1])) + if tsv_mutate: + path = os.path.join(here, TSV_NAME) + with open(path, encoding="utf-8", newline="") as fh: + rows = list(csv.reader(fh, delimiter="\t")) + tsv_mutate(rows) + with open(path, "w", encoding="utf-8", newline="") as fh: + csv.writer(fh, delimiter="\t").writerows(rows) -if fails: + return here + + +def shift_one_day(man): + dt = datetime.strptime(man["extracted_at_utc"], TS_FMT).replace(tzinfo=timezone.utc) + man["extracted_at_utc"] = (dt + timedelta(days=1)).strftime(TS_FMT) + + +def set_stale(man): + man["extracted_at_utc"] = STALE_CONSTANT + + +def set_garbage_stamp(man): + man["extracted_at_utc"] = "2026-10-28 18:31:34" + + +def corrupt_title(rows): + rows[1][1] = rows[1][1] + " (corrupted)" + + +def selftest(): + with open(os.path.join(HERE, MANIFEST_NAME), encoding="utf-8") as fh: + true_stamp = json.load(fh)["extracted_at_utc"] + + cells = [] + + def cell(name, mutate_kwargs, argv, want_rc, want_substr): + with tempfile.TemporaryDirectory(prefix="census2-selftest-") as tmp: + here = build_tree(tmp, **mutate_kwargs) + proc = subprocess.run([sys.executable, os.path.join(here, "CHECK")] + argv, + capture_output=True, text=True) + out = proc.stdout + proc.stderr + ok = proc.returncode == want_rc and want_substr in out + cells.append((name, ok)) + print("%s %-34s rc=%d (want %d) %r%s" + % ("PASS" if ok else "FAIL", name, proc.returncode, want_rc, + want_substr, "" if ok else "\n" + out)) + + # control: artifact as committed, reference supplied -> green + cell("control-with-reference", {}, ["--now=" + true_stamp], 0, + "PASS extracted_at_utc recency") + # control: no reference -> recency SKIPs instead of decaying to red + cell("control-no-reference-skips", {}, [], 0, + "SKIP extracted_at_utc recency") + # the cell the item asks for: +1 day stamp, reference supplied -> red + cell("timestamp-plus-one-day", {"manifest_mutate": shift_one_day}, + ["--now=" + true_stamp], 1, "FAIL extracted_at_utc recency") + # the Q122 opening defect: the stale constant, no reference needed + cell("stale-constant", {"manifest_mutate": set_stale}, [], 1, + "FAIL extracted_at_utc not the stale constant") + # malformed stamp + cell("malformed-stamp", {"manifest_mutate": set_garbage_stamp}, [], 1, + "FAIL extracted_at_utc format") + # re-derivation leg still bites + cell("tsv-title-corrupted", {"tsv_mutate": corrupt_title}, [], 1, + "FAIL qid+title identical to independent re-derive") + + bad = [n for n, ok in cells if not ok] print("") - print("%d check(s) FAILED" % len(fails)) - sys.exit(1) + if bad: + print("SELFTEST FAILED (%d of %d cells): %s" % (len(bad), len(cells), ", ".join(bad))) + return 1 + print("SELFTEST OK (%d cells: 2 green controls, 4 red mutations)" % len(cells)) + return 0 -print("") -print("ALL CHECKS PASSED (%d rows, %d distinct labels, %d reopened)" - % (len(golden), len(distinct), len(reopened))) + +def main(): + now_ref, do_selftest = parse_args(sys.argv[1:]) + if do_selftest: + fails, _ = run(HERE, REPO_ROOT, None) + if fails: + print("") + print("SELFTEST ABORTED: the in-tree artifact is already red (%d check(s)); " + "fix the tree before grading the gate" % len(fails)) + return 1 + print("") + print("in-tree gate green; grading the legs") + print("") + return selftest() + + fails, stats = run(HERE, REPO_ROOT, now_ref) + if fails: + print("") + print("%d check(s) FAILED" % len(fails)) + return 1 + + print("") + print("ALL CHECKS PASSED (%d rows, %d distinct labels, %d reopened)" + % (stats["rows"], stats["distinct"], stats["reopened"])) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/wizard-work/ledger/workqueue-census-v2/README.md b/wizard-work/ledger/workqueue-census-v2/README.md index b96305b..9f05435 100644 --- a/wizard-work/ledger/workqueue-census-v2/README.md +++ b/wizard-work/ledger/workqueue-census-v2/README.md @@ -18,16 +18,32 @@ first sentence end) and `done_implies` (first `**Done means:` line). | `workqueue-census.tsv` | Tab-separated table of all 96 rows (artifact of record). | | `workqueue-census.md` | Markdown table of the same rows. | | `extract_manifest.json` | Schema + counts + provenance. `extracted_at_utc` is the measured run moment, not a constant. | -| `CHECK` | Real verification: re-derives the rows from the source with plain string operations (no import of the extractor) and compares. Exits 0 only if correct. | +| `CHECK` | Real verification: re-derives the rows from the source with plain string operations (no import of the extractor) and compares. Python 3; exits 0 only if correct. | | `extract_workqueue.py` | The extractor that produced the TSV/MD/manifest. | ## Verification -`bash CHECK` re-derives the expected rows from `WORK-QUEUE.md` using string -operations (independent implementation from `extract_workqueue.py`), then -compares qid + title against the committed TSV, recomputes the manifest's -counts, checks `extracted_at_utc` is measured and recent, and checks for -delimiter corruption. It exits 0 only if every check passes. +`CHECK` is a Python 3 program, not a shell script. Run it with the interpreter +itself (the file is executable, so `./CHECK` works too): + + python3 CHECK + python3 CHECK --now=2026-10-28T18:31:34Z # also grade recency + python3 CHECK --selftest # prove every leg can fail + +It re-derives the expected rows from `WORK-QUEUE.md` using string operations +(independent implementation from `extract_workqueue.py`), then compares qid + +title against the committed TSV, recomputes the manifest's counts, grades the +extraction timestamp, and checks for delimiter corruption. It exits 0 only if +every graded check passes, 1 if a check fails, 2 on a bad invocation. + +**The timestamp leg does not read the wall clock.** `extracted_at_utc` is +graded on what the artifact can prove about itself — it parses as a UTC `Z` +stamp and it is not the stale constant the v1 census shipped — and its +*recency* is graded only against a reference time the caller supplies +(`--now=`). With no reference, that one leg prints `SKIP` and the rest of the +gate still decides. A committed artifact therefore reads the same verdict on +the day it lands and a year later; the earlier form (`abs(now - dt) < 120`) +could only be green inside a 120-second window after generation. ## Why a new census @@ -40,10 +56,24 @@ The earlier `2026-10-28-workqueue-census/` failed its own contract: This v2 production is the corrected, independently verified replacement. +### v2's own correction (2026-10-28, Q122) + +v2 fixed the tautology and then bound its verdict to the clock instead: the +recency leg was `abs(now - dt) < 120` against the frozen +`2026-10-28T18:31:34Z`, so the artifact could only ever be green for the two +minutes after its own generation, and the documented command `bash CHECK` +could not run at all (the file is Python). Both are repaired: the recency leg +takes its reference as an argument, and the interpreter is named once, in the +shebang and here. + ## Evidence -- Source: `/root/troupe-standup/WORK-QUEUE.md` (repo root, HEAD). +- Source: `WORK-QUEUE.md` at the repo root (resolved from this directory: + `HERE/../../..`), not a pinned host path. - Extracted: `2026-10-28T18:31:34Z` UTC (measured at generation). - Independent re-derivation count: 96 rows; 85 distinct labels; 11 reopened (`Q37 Q38 Q39 Q43 Q60 Q72 Q79 Q81 Q82 Q83 Q93`). -- CHECK exit code: 0. +- `python3 CHECK` → `ALL CHECKS PASSED (96 rows, 85 distinct labels, 11 reopened)`, + rc 0. +- `python3 CHECK --selftest` → `SELFTEST OK (6 cells: 2 green controls, 4 red + mutations)`, rc 0.