diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index 624ecb7..5d7b99b 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -13792,3 +13792,20 @@ CHECK: bash wizard-work/volt/CHECK-q118 **(4) Not claimed:** that any `Y` row is a hire recommendation — none of them was exercised with a key this tick, and the document says so; that an `unknown` row is a dead provider — Together and Qwen Chat are unproven, not disproven, and Qwen's page serves a mobile interstitial to a plain client; that the twelve rows' every figure was checked — the client-rendered tables (Google per-model RPD, Cerebras models, Mistral's API tier) are recorded as `not stated on page`, which is what the fetch could read; that the audit is now stable — `--live` is the instrument for the monthly re-read the opening record asks for, and it must be run to keep this file honest; that the two Hugging Face rows can be merged — they are two different products (Spaces hardware, Inference Providers) and only the second was graded. **Files edited for this item:** `wizard-work/quill/free-tier-ai-stack-audit.md` (rewritten in place; the pre-repair bytes stay resolvable as blob `f6cecf9b8c66` at `40da64c`), `wizard-work/quill/q114-fetch-2026-10-31.tsv` (new), `wizard-work/volt/q114_verify.py` (new, 345 lines), `wizard-work/volt/CHECK-q114` (new), `checkins/2026-10-31-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). The 25 fetched bodies (6.3 MB) are outside the repo at `/root/.hermes/wizard-outputs/volt/2026-10-31-q114/evidence/` with the fetch script beside them; nothing from `inbox/` is quoted. CHECK: bash wizard-work/volt/CHECK-q114 + + +**Q135 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red at HEAD, named its source by absolute host path, and decided its verdict from whatever the worktree held at read time — so every append to `WORK-QUEUE.md` changed the answer without changing the artifact.** + +**(0) Entry state, measured at `7b42050` before any edit.** `python3 wizard-work/ledger/workqueue-census-v2/CHECK` → **rc 1**, `FAIL row count 'committed=96 golden=97'`, `FAIL manifest.rows 'manifest=96 actual=97'`, `FAIL manifest.distinct_q_labels 'manifest=85 actual=86'`, with `SKIP extracted_at_utc recency`. The cause is not a bad extractor: `grep -cE '^## +Q[0-9]+' WORK-QUEUE.md` reads **97** headings while the committed TSV carries **96** rows, because the table was generated on 2026-10-28 and nothing regenerated it when the queue gained a heading. Two further defects sit behind that one: `extract_manifest.json`'s `source` field is the literal `/root/troupe-standup/WORK-QUEUE.md` — the Q69/Q106/Q109 host-pin class, so the artifact names a path that exists on one machine — and `CHECK`'s `rederive(src_path)` reads the worktree file at run time, so the verdict is a function of *when* the gate runs, not of the artifact. That second defect is why the count could drift silently: a green read taken minutes after generation says nothing about the bytes anyone else will grade. + +**(1) What the artifact was not doing, stated plainly.** A census whose rows are re-derived from a moving file has no fixed subject. The manifest claimed `rows: 96` and proved nothing about which 96 — no digest of the source, no digest of the rows, no revision. Any reader could edit the TSV and the count legs would still read green, because the only thing compared was `qid + title` against a file that had itself moved. The 2026-10-28 artifact was honest on the day it was written and unverifiable on every day after; that is the same decay class as the wall-clock leg Q122 repaired, in a form no clock check would have caught. + +**(2) The repair, in six files.** `extract_workqueue.py` now pins what it read: `source` becomes the repo-relative `WORK-QUEUE.md`; `source_rev`, `source_blob_sha` (`git rev-parse HEAD:WORK-QUEUE.md`), `source_sha256`, `source_lines` and `source_matches_head_blob` are recorded; `row_digest` is the sha256 of the emitted TSV data rows. The pin is honest or the extractor refuses — if the worktree's `WORK-QUEUE.md` is not the HEAD blob, it exits **rc 1**, writes nothing, and says `REFUSED: ... commit it first, or pass --allow-dirty`; `--allow-dirty` snapshots the worktree and records `source_blob_sha: null` with `source_matches_head_blob: false`, so the absence of a pin is a stated fact rather than a silent one. It also gained `-h/--help` and a usage-error path (rc 2) so its own documented invocations are gradeable. `CHECK` reads the pinned blob with `git cat-file blob` instead of the worktree, grades the pin, grades `manifest.source is repo-relative`, grades `manifest.row_digest is the committed rows`, and prints drift as `ADVISORY source-moved worktree= pinned=` — reported, never graded, because a snapshot that reds whenever the world moves is a snapshot nobody regenerates. A declared pin that git cannot read is a **fault, never a fallback**: an artifact citing a blob nobody can open has no provenance. `workqueue-census.tsv`, `workqueue-census.md` and `extract_manifest.json` are regenerated at HEAD (97 rows / 86 distinct / 11 reopened, pinned to blob `624ecb72ead1f01b60cdbaca892278858167aa5e` at rev `7b42050`), and `README.md` carries the new contract, the two `2026-10-28`/`2026-10-31` correction notes, and the measured evidence block. + +**(3) The load-bearing claim, proved on real commits rather than asserted.** In a throwaway repository at `/root/.hermes/profiles/volt/cache/scratch/decay-probe` the artifact was committed as `deb8439` (`CHECK` rc 0, `ALL CHECKS PASSED (97 rows, 86 distinct labels, 11 reopened)`), then three `## Q` headings were committed on top as `e4e8f58`. At that descendant revision HEAD's `WORK-QUEUE.md` blob is `06efc67d1058` while the manifest still pins `624ecb72ead1`, the pin still resolves, `PASS row count` holds, `CHECK` reads **rc 0** with `ALL CHECKS PASSED (97 rows, ...)` plus `ADVISORY source-moved worktree=100 pinned=97`, and the extractor re-run there re-pins cleanly to 100 rows because the worktree *was* the HEAD blob. The refusal path is measured in the same tree: with one heading appended and left uncommitted, the extractor exits **rc 1** printing `REFUSED: the worktree's WORK-QUEUE.md is not the HEAD blob (06efc67d1058)` and the TSV is byte-unchanged (sha256 `01fcf5c6…` before and after); `--allow-dirty` then exits 0 with `rows: 101` and `source pin: ABSENT (--allow-dirty; the manifest says so)`. + +**(4) Evidence at HEAD, all measured this tick.** `python3 wizard-work/ledger/workqueue-census-v2/CHECK` → **20 `PASS`, 1 `SKIP`** (recency, no reference supplied), `ALL CHECKS PASSED (97 rows, 86 distinct labels, 11 reopened)`, rc 0. `python3 .../CHECK --selftest` → in-tree gate green, then **`SELFTEST OK (9 cells: 3 green controls, 6 red mutations)`**, rc 0: the green controls are the artifact as committed with and without a reference time and the drift cell; the six red cells are an unreadable pin (`FAIL source pin`), a host-path `source` (`FAIL manifest.source is repo-relative`), a corrupted `row_digest`, a corrupted title, the stale `2026-10-28T00:00:00Z` constant, and a stamp moved one day against a supplied reference. The selftest fixtures are real git repositories whose HEAD blob is the pinned source, so the pin leg is exercised the way it is exercised in the tree, and the drift cell moves the worktree out from under a committed pin. `tools/check_lint.py --check "python3 wizard-work/ledger/workqueue-census-v2/CHECK --selftest"` → `SOUND`. + +**(5) Not claimed.** That Q112 is closed — its done-condition also asks that the record-form ids be enumerated, and **93** bold-lead `**Q — ` records still have no row and no parser that reads them; this record fixes the artifact's provenance and its staleness, not the census's coverage unit. That Q122 is reopened — its clock leg and its interpreter leg hold and are re-measured here. That the drift advisory should become a fault — it is deliberately advisory and the selftest pins that choice. That a shallow clone grades the same — a clone that does not carry the pinned object fails the pin leg by design, and `--allow-dirty` is the documented way to generate without a pin. That the 97 rows' `status` and `done_implies` columns are now verified — `CHECK` never graded them and still does not; the columns are transcribed, not certified. That any other lane's file is affected — this record touches the census directory only. Files edited for this item: `wizard-work/ledger/workqueue-census-v2/extract_workqueue.py`, `wizard-work/ledger/workqueue-census-v2/CHECK`, `wizard-work/ledger/workqueue-census-v2/README.md`, `wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv`, `wizard-work/ledger/workqueue-census-v2/workqueue-census.md`, `wizard-work/ledger/workqueue-census-v2/extract_manifest.json`, `checkins/2026-10-31-volt.md` (appended), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Nothing from `inbox/` is quoted. Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q135.md`. + +CHECK: python3 wizard-work/ledger/workqueue-census-v2/CHECK --selftest diff --git a/checkins/2026-10-31-volt.md b/checkins/2026-10-31-volt.md index 4a3aba2..63a934d 100644 --- a/checkins/2026-10-31-volt.md +++ b/checkins/2026-10-31-volt.md @@ -58,3 +58,28 @@ one will be caught by `import-resolve`, but nobody reads that leg until it reds the gate — a written rule ("a test file resolves its module from `__file__`") belongs where new beam files are written. + +## Q135 — the queue's own table was red, host-pinned, and time-dependent + +The item is filed in `WORK-QUEUE.md`; this is the reading behind it. + +- `wizard-work/ledger/workqueue-census-v2/CHECK` read **rc 1** at `7b42050` + (`FAIL row count 'committed=96 golden=97'`) against a source of **97** + headings. The extractor was not wrong; nothing regenerated the table after + the queue gained a heading on 2026-10-28. +- The manifest named `/root/troupe-standup/WORK-QUEUE.md` — a host path — and + pinned no bytes, so the artifact proved nothing about which rows it holds. + `CHECK` read the worktree at run time, so the verdict moved with the queue. +- The repair pins `source_blob_sha` / `source_sha256` / `source_rev`, adds + `row_digest`, makes the extractor **refuse** to pin a dirty worktree, and + turns drift into an advisory. `CHECK --selftest` reads 9 cells, 3 green + controls and 6 red mutations. +- The load-bearing claim is measured on real commits, not asserted: a throwaway + repo committed the artifact (`deb8439`), then three headings on top + (`e4e8f58`); at the descendant revision `CHECK` still reads rc 0 with the + pinned 97 rows plus `ADVISORY source-moved worktree=100 pinned=97`. +- Left open on purpose: Q112's record-form enumeration (93 bold-lead records + still have no row), and the `status` / `done_implies` columns, which are + transcribed and not certified. + +Report: `/root/.hermes/wizard-outputs/volt/2026-10-31-q135.md`. diff --git a/docs/queue-index.md b/docs/queue-index.md index fe0bb08..ef7edea 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -4,7 +4,7 @@ **What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it. -**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 44 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 114 record blocks in all, 70 of them a later closure or correction for an id already opened. The same **44 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. +**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 45 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 115 record blocks in all, 70 of them a later closure or correction for an id already opened. The same **45 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. | # | id | kind | lines | bytes | role | title | first `Status:` | |---|---|---|---|---|---|---|---| @@ -218,7 +218,8 @@ | 208 | Q133 | record | 13731-13751 | 9788 | record: BOLD-LEAD | **Q133 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; beam lane by authorship, unclaimed and taken). Two beam test cells graded a host copy of the … | -- | | 209 | Q134 | record | 13752-13765 | 6338 | record: BOLD-LEAD | **Q134 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; sol lane by authorship, unclaimed and taken). `tools/inspect_tree.py` was an 8-line one-off … | -- | | 210 | Q118 | record | 13766-13781 | 7225 | cont: DONE | **Q118 — DONE (volt, 2026-10-31, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder read is retired three times before it may be called a failure … | -- | -| 211 | Q114 | record | 13782-13794 | 6782 | cont: DONE | **Q114 — DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on … | -- | +| 211 | Q114 | record | 13782-13796 | 6784 | cont: DONE | **Q114 — DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on … | -- | +| 212 | Q135 | record | 13797-13811 | 7674 | record: BOLD-LEAD | **Q135 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red … | -- | ## Ids with no `## ` heading @@ -243,13 +244,13 @@ Opened as records inside another section. `line` is the record's own opening lin | Q103 | 13010 | 13010 | 18 | Q102 | BULLET+BOLD | BOLD-LEAD | | Q104 | 13103 | 13103 | 18 | Q101 | BULLET+BOLD | BOLD-LEAD | | Q105 | 13135 | 13135 | 20 | Q104 | BULLET+BOLD | BOLD-LEAD | -| Q106 | 13171 | 13171 | 16 | Q105 | BULLET+BOLD | BOLD-LEAD | +| Q106 | 13171 | 13171 | 17 | Q105 | BULLET+BOLD | BOLD-LEAD | | Q107 | 13189 | 13189 | 18 | Q106 | BULLET+BOLD | BOLD-LEAD | | Q108 | 13267 | 13263 | 3 | Q106 | BULLET+BOLD | BOLD-LEAD | -| Q109 | 13270 | 13270 | 28 | Q104 | BULLET+BOLD | BOLD-LEAD | +| Q109 | 13270 | 13270 | 29 | Q104 | BULLET+BOLD | BOLD-LEAD | | Q110 | 13272 | 13272 | 6 | Q109 | BULLET+BOLD | BOLD-LEAD | | Q111 | 13274 | 13274 | 8 | Q110 | BULLET+BOLD | BOLD-LEAD | -| Q112 | 13276 | 13276 | 6 | Q111 | BULLET+BOLD | BOLD-LEAD | +| Q112 | 13276 | 13276 | 7 | Q111 | BULLET+BOLD | BOLD-LEAD | | Q113 | 13278 | 13278 | 3 | Q112 | BULLET+BOLD | BOLD-LEAD | | Q114 | 13280 | 13280 | 6 | Q113 | BULLET+BOLD | BOLD-LEAD | | Q115 | 13282 | 13282 | 9 | Q114 | BULLET+BOLD | BOLD-LEAD | @@ -259,7 +260,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q119 | 13305 | 13305 | 5 | Q118 | BULLET+BOLD | BOLD-LEAD | | Q120 | 13308 | 13308 | 21 | Q119 | BULLET+BOLD | BOLD-LEAD | | Q121 | 13342 | 13342 | 4 | Q107 | BULLET+BOLD | BOLD-LEAD | -| Q122 | 13344 | 13344 | 6 | Q121 | BULLET+BOLD | BOLD-LEAD | +| Q122 | 13344 | 13344 | 8 | Q121 | BULLET+BOLD | BOLD-LEAD | | Q123 | 13346 | 13346 | 4 | Q122 | BULLET+BOLD | BOLD-LEAD | | Q124 | 13350 | 13350 | 4 | Q95 | BULLET+BOLD | BOLD-LEAD | | Q126 | 13545 | 13545 | 8 | Q113 | BULLET+BOLD | BOLD-LEAD | @@ -270,6 +271,7 @@ Opened as records inside another section. `line` is the record's own opening lin | Q131 | 13689 | 13685 | 2 | Q115 | BULLET+BOLD | BOLD-LEAD | | Q133 | 13731 | 13731 | 1 | Q90 | BULLET+BOLD | BOLD-LEAD | | Q134 | 13752 | 13752 | 1 | Q133 | BULLET+BOLD | BOLD-LEAD | +| Q135 | 13797 | 13797 | 1 | Q114 | BULLET+BOLD | BOLD-LEAD | **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. @@ -1751,17 +1753,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q118 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder read is retired three times before it may be called a failure \u2026" }, { - "bytes": 6782, - "end_line": 13794, + "bytes": 6784, + "end_line": 13796, "id": "Q114", "kind": "record", "line": 13782, - "lines": 13, + "lines": 15, "role": "continuation", "role_detail": "DONE", "status": null, "status_line": null, "title": "**Q114 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on \u2026" + }, + { + "bytes": 7674, + "end_line": 13811, + "id": "Q135", + "kind": "record", + "line": 13797, + "lines": 15, + "role": "record", + "role_detail": "BOLD-LEAD", + "status": null, + "status_line": null, + "title": "**Q135 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red \u2026" } ], "ids_multi_section": { @@ -4630,17 +4645,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q118 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; pipe lane by authorship, unclaimed and taken). The monitor's folder read is retired three times before it may be called a failure \u2026" }, { - "bytes": 6782, - "end_line": 13794, + "bytes": 6784, + "end_line": 13796, "id": "Q114", "kind": "record", "line": 13782, - "lines": 13, + "lines": 15, "role": "continuation", "role_detail": "DONE", "status": null, "status_line": null, "title": "**Q114 \u2014 DONE (volt, 2026-10-31, at the commit this record lands in; quill lane by authorship, unclaimed and taken). The audit's twelve rows are now sixteen rows read from seventeen provider pages on \u2026" + }, + { + "bytes": 7674, + "end_line": 13811, + "id": "Q135", + "kind": "record", + "line": 13797, + "lines": 15, + "role": "record", + "role_detail": "BOLD-LEAD", + "status": null, + "status_line": null, + "title": "**Q135 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at the commit this record lands in; ledger lane by authorship, unclaimed and taken). The census this queue keeps as its table was red \u2026" } ], "nested": [ @@ -4802,7 +4830,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q106", "line": 13171, - "refs": 16, + "refs": 17, "section": "Q105", "shape": "BULLET+BOLD" }, @@ -4829,7 +4857,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q109", "line": 13270, - "refs": 28, + "refs": 29, "section": "Q104", "shape": "BULLET+BOLD" }, @@ -4856,7 +4884,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q112", "line": 13276, - "refs": 6, + "refs": 7, "section": "Q111", "shape": "BULLET+BOLD" }, @@ -4946,7 +4974,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q122", "line": 13344, - "refs": 6, + "refs": 8, "section": "Q121", "shape": "BULLET+BOLD" }, @@ -5039,6 +5067,15 @@ The block below is the parse this document renders, byte for byte. `--check` com "refs": 1, "section": "Q133", "shape": "BULLET+BOLD" + }, + { + "first_mention": 13797, + "form": "BOLD-LEAD", + "id": "Q135", + "line": 13797, + "refs": 1, + "section": "Q114", + "shape": "BULLET+BOLD" } ], "preamble": { @@ -5046,9 +5083,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 2252053, - "queue_lines": 13794, - "queue_sha256": "e34193da6d70cbe8d4d554e186e873d73c03cb29f21dc54135778c6ce91ef3ee", + "queue_bytes": 2259729, + "queue_lines": 13811, + "queue_sha256": "ceeef0103bbfea1472c0d4f9b3d440a5285a4d5bba1415b54d4c872122b77fd0", "sections": [ { "bytes": 3139, diff --git a/wizard-work/ledger/workqueue-census-v2/CHECK b/wizard-work/ledger/workqueue-census-v2/CHECK index f30a25e..f61eb73 100755 --- a/wizard-work/ledger/workqueue-census-v2/CHECK +++ b/wizard-work/ledger/workqueue-census-v2/CHECK @@ -2,42 +2,56 @@ """Independent verification for the workqueue-census-v2 artifact. RE-APPROACH: this checker does NOT import the extractor. It re-derives the -expected rows from WORK-QUEUE.md with plain string operations (split on -whitespace, startswith); the committed TSV is the artifact of record. -The verdict is the script's exit code: 0 = correct, 1 = a check failed, -2 = could not grade (bad invocation / missing artifact). +expected rows from the SOURCE BYTES THE MANIFEST PINS with plain string +operations (split on whitespace, startswith); the committed TSV is the +artifact of record. The verdict is the script's exit code: 0 = correct, +1 = a check failed, 2 = could not grade (bad invocation / unusable input). + +WHICH BYTES ARE GRADED (Q135, 2026-10-31). The source is the blob named by +`manifest.source_blob_sha`, read with `git cat-file blob`, not the worktree +file. That is what makes the verdict a function of the artifact rather than of +when it is run: a clone made later, with WORK-QUEUE.md hundreds of append-only +records longer, grades the same rows against the same bytes. The worktree is +used only when the manifest declares no pin (a `--allow-dirty` snapshot), and +then only if its sha256 is `manifest.source_sha256`. + +A declared pin that git cannot read is a FAULT, never a fallback: an artifact +that cites a blob nobody can open has no provenance, and falling back to +whatever the worktree holds would grade bytes the manifest never named. + +Drift is reported, not graded. When the worktree's heading count differs from +the pinned source's, this prints `ADVISORY source-moved worktree= +pinned=` and still exits 0 -- a census is a snapshot, and a snapshot that +goes red because the world moved is a snapshot nobody regenerates. Run it with the interpreter this file names: - python3 CHECK # portable: every leg that can be graded - # from the artifact alone is graded - python3 CHECK --now=2026-10-28T18:31:34Z - # adds the recency leg: the manifest's - # extraction time must be within 120s of - # the reference time SUPPLIED HERE - python3 CHECK --selftest # grade the tree, then prove - # every leg can fail + python3 CHECK # portable: every leg the artifact proves + python3 CHECK --now=2026-10-31T00:00:00Z # also grade stamp recency + python3 CHECK --selftest # grade the tree, then prove every leg fails Checks: - 1. every '## Q' heading in the source has exactly one committed row, in - source order, with a verbatim title; - 2. row counts, distinct-q count and reopened-q list in extract_manifest.json + 1. source pin: the pinned blob resolves and its sha256 is the manifest's; + 2. every '## Q' heading in the pinned source has exactly one committed + row, in source order, with a verbatim title; + 3. row counts, distinct-q count and reopened-q list in extract_manifest.json agree with an independent recomputation; - 3. extracted_at_utc provenance: it parses as an UTC 'Z' stamp and is not the - known stale constant. Its RECENCY is graded only against a reference time - passed by the caller (--now), never against the wall clock, so the gate - does not decay with time: a clone made a year from now reads the same - verdict a clone made a minute from now reads; - 4. README names the interpreter and the file carries the python3 shebang and + 4. manifest.source is repo-relative -- a host path is the Q69/Q106/Q109 class; + 5. manifest.row_digest is the sha256 of the committed data rows, so the TSV + cannot be edited without the manifest disagreeing; + 6. extracted_at_utc parses as an UTC 'Z' stamp and is not the known stale + constant; its RECENCY is graded only against a caller-supplied --now, so + the gate does not decay with time; + 7. README names the interpreter and the file carries the python3 shebang and the execute bit (the defect that made the documented invocation unrunnable); - 5. no committed row has an empty q or title, and no status/done value + 8. no committed row has an empty q or title, and no status/done value contains a literal tab (delimiter corruption). """ import csv +import hashlib import json import os -import re import shutil import subprocess import sys @@ -55,6 +69,7 @@ STALE_CONSTANT = "2026-10-28T00:00:00Z" TS_FMT = "%Y-%m-%dT%H:%M:%SZ" RECENCY_WINDOW_S = 120 SHEBANG = "#!/usr/bin/env python3" +COLUMNS = ["q", "title", "status", "done_implies"] def parse_args(argv): @@ -73,28 +88,81 @@ def parse_args(argv): return now_ref, selftest -def rederive(src_path): +def sha256_bytes(data): + return hashlib.sha256(data).hexdigest() + + +def is_heading(line): + """The heading rule, written here and not imported from the extractor.""" + if not line.startswith("## "): + return False + parts = line[3:].split(None, 1) + return bool(parts) and len(parts[0]) > 1 and parts[0][0] == "Q" and parts[0][1:].isdigit() + + +def rederive(lines): """Re-derive (qid, title) in source order with plain string operations.""" golden = [] qid, title = None, "" - with open(src_path, encoding="utf-8") as fh: - src_lines = fh.read().splitlines() - - for line in src_lines: - if line.startswith("## "): - parts = line[3:].split(None, 1) - if parts and len(parts[0]) > 1 and parts[0][0] == "Q" and parts[0][1:].isdigit(): - if qid is not None: - golden.append((qid, title)) - qid = parts[0] - title = parts[1].strip() if len(parts) > 1 else "" + for line in lines: + if not is_heading(line): continue - + parts = line[3:].split(None, 1) + if qid is not None: + golden.append((qid, title)) + qid = parts[0] + title = parts[1].strip() if len(parts) > 1 else "" if qid is not None: golden.append((qid, title)) return golden +def git_blob(repo_root, sha): + """The blob's bytes, or None when git or the object is absent.""" + try: + proc = subprocess.run(["git", "-C", repo_root, "cat-file", "blob", sha], + capture_output=True) + except OSError: + return None + if proc.returncode != 0: + return None + return proc.stdout + + +def read_source(repo_root, man): + """(lines, tier, sha, problem). tier: pinned-blob | worktree-pinned | none.""" + pin = man.get("source_blob_sha") + want = man.get("source_sha256") or "" + if pin: + blob = git_blob(repo_root, pin) + if blob is None: + return None, "none", "", ("the manifest pins blob %s and git cannot " + "read it from %s" % (pin[:12], repo_root)) + return (blob.decode("utf-8").splitlines(), "pinned-blob", + sha256_bytes(blob), "") + path = os.path.join(repo_root, SRC_NAME) + try: + with open(path, "rb") as fh: + raw = fh.read() + except OSError: + return None, "none", "", "no pin, and no %s to fall back to" % SRC_NAME + got = sha256_bytes(raw) + if got != want: + return None, "none", got, ("no pin, and the worktree %s (sha256 %s) is not " + "the bytes the manifest names (%s)" + % (SRC_NAME, got[:12], want[:12])) + return raw.decode("utf-8").splitlines(), "worktree-pinned", got, "" + + +def worktree_headings(repo_root): + """Heading count of the worktree source, or None when unreadable.""" + try: + with open(os.path.join(repo_root, SRC_NAME), encoding="utf-8") as fh: + return sum(1 for line in fh.read().splitlines() if is_heading(line)) + except OSError: + return None + + def run(here, repo_root, now_ref): """Grade one tree. Returns (fails, stats).""" fails = [] @@ -106,23 +174,37 @@ def run(here, repo_root, now_ref): print("FAIL " + name + " " + repr(detail)) fails.append(name) - src = os.path.join(repo_root, SRC_NAME) tsv = os.path.join(here, TSV_NAME) manifest_path = os.path.join(here, MANIFEST_NAME) readme_path = os.path.join(here, README_NAME) self_path = os.path.join(here, "CHECK") - # --- 1. Independent re-derivation from the source (string ops, no regex) --- - golden = rederive(src) + with open(manifest_path, encoding="utf-8") as fh: + man = json.load(fh) - # --- 2. Committed TSV ------------------------------------------------------ + # --- 1. Source pin -------------------------------------------------------- + lines, tier, got_sha, problem = read_source(repo_root, man) + if tier == "none": + check("source pin", False, problem) + print("") + print("CANNOT GRADE: %d check(s) failed before any row was read" % len(fails)) + return fails, {"rows": 0, "distinct": 0, "reopened": 0} + pin = man.get("source_blob_sha") or "(none: --allow-dirty snapshot)" + check("source pin", got_sha == (man.get("source_sha256") or ""), + "pinned=%s read-sha256=%s manifest-sha256=%s" + % (pin[:12], got_sha[:12], (man.get("source_sha256") or "")[:12])) + print("INFO source %s %s sha256=%s" % (tier, pin[:12], got_sha[:12])) + + # --- 2. Independent re-derivation from the pinned bytes ------------------- + golden = rederive(lines) + + # --- 3. Committed TSV ---------------------------------------------------- with open(tsv, encoding="utf-8", newline="") as fh: rows = list(csv.reader(fh, delimiter="\t")) header, committed = rows[0], rows[1:] - check("tsv header", header == ["q", "title", "status", "done_implies"], repr(header)) - check("row count", - len(committed) == len(golden), + check("tsv header", header == COLUMNS, repr(header)) + check("row count", len(committed) == len(golden), "committed=%d golden=%d" % (len(committed), len(golden))) mism = [k for k, (c, g) in enumerate(zip(committed, golden)) @@ -130,10 +212,7 @@ def run(here, repo_root, now_ref): check("qid+title identical to independent re-derive", not mism, "first mismatch row %r" % (mism[0] if mism else None)) - # --- 3. Manifest self-consistency (independent recomputation) -------------- - with open(manifest_path, encoding="utf-8") as fh: - man = json.load(fh) - + # --- 4. Manifest counts, independent recomputation ------------------------ counts = {} for q, _ in golden: counts[q] = counts.get(q, 0) + 1 @@ -147,7 +226,17 @@ def run(here, repo_root, now_ref): check("manifest.reopened_q_labels", man.get("reopened_q_labels") == reopened, "manifest=%r actual=%r" % (man.get("reopened_q_labels"), reopened)) - # --- 4. extracted_at_utc: provenance graded, recency by argument ----------- + # --- 5. Provenance: relative source, row digest --------------------------- + check("manifest.source is repo-relative", man.get("source") == SRC_NAME, + "%r -- a host path is the Q69/Q106/Q109 class" % (man.get("source"),)) + + recomputed = "".join("\t".join(r) + "\n" for r in committed) + check("manifest.row_digest is the committed rows", + man.get("row_digest") == sha256_bytes(recomputed.encode("utf-8")), + "manifest=%r recomputed=%s" + % (man.get("row_digest"), sha256_bytes(recomputed.encode("utf-8"))[:12])) + + # --- 6. extracted_at_utc: provenance graded, recency by argument --------- auto = man.get("extracted_at_utc", "") check("extracted_at_utc key", isinstance(auto, str) and bool(auto), repr(auto)) dt = None @@ -159,7 +248,7 @@ def run(here, repo_root, now_ref): check("extracted_at_utc format", False, repr(auto)) check("extracted_at_utc not the stale constant", auto != STALE_CONSTANT, repr(auto)) check("manifest provenance names the source", - isinstance(man.get("schema"), str) and man.get("schema") == "workqueue-census-v2" + man.get("schema") == "workqueue-census-v2" and isinstance(man.get("generated_by"), str) and man.get("generated_by"), "schema=%r generated_by=%r" % (man.get("schema"), man.get("generated_by"))) @@ -179,7 +268,7 @@ def run(here, repo_root, now_ref): check("extracted_at_utc recency", delta < RECENCY_WINDOW_S, "auto=%s ref=%s delta=%ss" % (auto, now_ref, delta)) - # --- 5. Interpreter contract (README vs this file) ------------------------- + # --- 7. Interpreter contract (README vs this file) ------------------------ with open(self_path, encoding="utf-8") as fh: first = fh.readline().rstrip("\n") check("CHECK shebang is python3", first == SHEBANG, repr(first)) @@ -196,19 +285,38 @@ def run(here, repo_root, now_ref): "python3 CHECK present=%r bash CHECK present=%r" % ("python3 CHECK" in vsec, "bash CHECK" in vsec)) - # --- 6. Corruption checks -------------------------------------------------- + # --- 8. Corruption checks ------------------------------------------------- empty = [r for r in committed if not r[0] or not r[1]] check("no empty q/title", not empty, repr(empty[:1])) bad_tab = [r for r in committed if any("\t" in c for c in r[2:])] check("status/done no tabs", not bad_tab, repr(bad_tab[:1])) + # --- 9. Drift is reported, never graded ----------------------------------- + head_n = worktree_headings(repo_root) + if head_n is not None and head_n != len(golden): + print("ADVISORY source-moved worktree=%d pinned=%d " + "(this census is a snapshot of the pinned bytes; regenerate it to " + "move the snapshot forward)" % (head_n, len(golden))) + return fails, {"rows": len(golden), "distinct": len(distinct), "reopened": len(reopened)} # --- selftest: one cell per leg, each built by mutating a scratch copy --------- -def build_tree(dst, manifest_mutate=None, tsv_mutate=None): - """Copy the artifact + its source into dst as a self-contained repo root.""" +def _git(dst, *args): + """git in the fixture tree, with an identity, returning the CompletedProcess.""" + return subprocess.run(["git", "-C", dst, "-c", "user.name=selftest", + "-c", "user.email=selftest@example.invalid"] + list(args), + capture_output=True, text=True) + + +def build_tree(dst, manifest_mutate=None, tsv_mutate=None, source_append=None): + """Copy the artifact + its source into dst as a self-contained repo root. + + The copy is a real git repository whose HEAD blob is the pinned source, so + the pin leg is graded the way it is graded in the real tree; `source_append` + then moves the worktree out from under that pin. + """ here = os.path.join(dst, "wizard-work", "ledger", "workqueue-census-v2") os.makedirs(here) shutil.copy2(os.path.join(REPO_ROOT, SRC_NAME), os.path.join(dst, SRC_NAME)) @@ -216,8 +324,25 @@ def build_tree(dst, manifest_mutate=None, tsv_mutate=None): for name in ("CHECK", TSV_NAME, MD_NAME, README_NAME): shutil.copy2(os.path.join(HERE, name), os.path.join(here, name)) + _git(dst, "init", "-q") + _git(dst, "add", SRC_NAME) + _git(dst, "commit", "-q", "-m", "selftest fixture") + + with open(os.path.join(dst, SRC_NAME), "rb") as fh: + src_raw = fh.read() + blob_sha = _git(dst, "rev-parse", "HEAD:" + SRC_NAME).stdout.strip() + rev = _git(dst, "rev-parse", "HEAD").stdout.strip() + + if source_append: + with open(os.path.join(dst, SRC_NAME), "a", encoding="utf-8") as fh: + fh.write(source_append) + with open(os.path.join(HERE, MANIFEST_NAME), encoding="utf-8") as fh: man = json.load(fh) + man["source_blob_sha"] = blob_sha + man["source_rev"] = rev + man["source_sha256"] = sha256_bytes(src_raw) + man["source_matches_head_blob"] = True if manifest_mutate: manifest_mutate(man) with open(os.path.join(here, MANIFEST_NAME), "w", encoding="utf-8") as fh: @@ -243,8 +368,16 @@ def set_stale(man): man["extracted_at_utc"] = STALE_CONSTANT -def set_garbage_stamp(man): - man["extracted_at_utc"] = "2026-10-28 18:31:34" +def set_unreadable_pin(man): + man["source_blob_sha"] = "0" * 40 + + +def set_host_path(man): + man["source"] = os.path.join(REPO_ROOT, SRC_NAME) + + +def set_bad_row_digest(man): + man["row_digest"] = "0" * 64 def corrupt_title(rows): @@ -265,35 +398,41 @@ def selftest(): out = proc.stdout + proc.stderr ok = proc.returncode == want_rc and want_substr in out cells.append((name, ok)) - print("%s %-34s rc=%d (want %d) %r%s" + print("%s %-30s rc=%d (want %d) %r%s" % ("PASS" if ok else "FAIL", name, proc.returncode, want_rc, want_substr, "" if ok else "\n" + out)) - # control: artifact as committed, reference supplied -> green + # controls + cell("control-no-reference", {}, [], 0, "ALL CHECKS PASSED") cell("control-with-reference", {}, ["--now=" + true_stamp], 0, "PASS extracted_at_utc recency") - # control: no reference -> recency SKIPs instead of decaying to red - cell("control-no-reference-skips", {}, [], 0, - "SKIP extracted_at_utc recency") - # the cell the item asks for: +1 day stamp, reference supplied -> red - cell("timestamp-plus-one-day", {"manifest_mutate": shift_one_day}, - ["--now=" + true_stamp], 1, "FAIL extracted_at_utc recency") - # the Q122 opening defect: the stale constant, no reference needed - cell("stale-constant", {"manifest_mutate": set_stale}, [], 1, - "FAIL extracted_at_utc not the stale constant") - # malformed stamp - cell("malformed-stamp", {"manifest_mutate": set_garbage_stamp}, [], 1, - "FAIL extracted_at_utc format") - # re-derivation leg still bites + # the pin is graded, not assumed + cell("unreadable-pin", {"manifest_mutate": set_unreadable_pin}, [], 1, + "FAIL source pin") + # the host-pin cell (Q69/Q106/Q109 class) + cell("host-path-source", {"manifest_mutate": set_host_path}, [], 1, + "FAIL manifest.source is repo-relative") + # the artifact's own digest + cell("row-digest-corrupted", {"manifest_mutate": set_bad_row_digest}, [], 1, + "FAIL manifest.row_digest") + # the rows still bite cell("tsv-title-corrupted", {"tsv_mutate": corrupt_title}, [], 1, "FAIL qid+title identical to independent re-derive") + # the time value is a measurement, not a constant + cell("stale-constant", {"manifest_mutate": set_stale}, [], 1, + "FAIL extracted_at_utc not the stale constant") + cell("timestamp-plus-one-day", {"manifest_mutate": shift_one_day}, + ["--now=" + true_stamp], 1, "FAIL extracted_at_utc recency") + # and drift is an advisory, not a fault + cell("drift-is-advisory", {"source_append": "\n## Q999 - selftest drift probe\n"}, + [], 0, "ADVISORY source-moved") bad = [n for n, ok in cells if not ok] print("") if bad: print("SELFTEST FAILED (%d of %d cells): %s" % (len(bad), len(cells), ", ".join(bad))) return 1 - print("SELFTEST OK (%d cells: 2 green controls, 4 red mutations)" % len(cells)) + print("SELFTEST OK (%d cells: 3 green controls, 6 red mutations)" % len(cells)) return 0 diff --git a/wizard-work/ledger/workqueue-census-v2/README.md b/wizard-work/ledger/workqueue-census-v2/README.md index 9f05435..abd10d1 100644 --- a/wizard-work/ledger/workqueue-census-v2/README.md +++ b/wizard-work/ledger/workqueue-census-v2/README.md @@ -6,44 +6,87 @@ tables). ## What this is -One row per `## Q ...` heading in the source: 96 headings, 85 distinct q +One row per `## Q ...` heading in the source: 97 headings, 86 distinct q labels, 11 reopened labels. Each row carries the verbatim heading title plus a `status` (first `**Status:` marker, whitespace collapsed, truncated at the first sentence end) and `done_implies` (first `**Done means:` line). +A record written in the bold-lead form (`**Q135 — ...`) is not a heading and +therefore has no row. That is deliberate and unchanged: the table's unit is the +heading, and `WORK-QUEUE.md` carries 93 bold-lead records that no heading +parser can see. Counting those is a different artifact's job, not this one's. + ## Artifacts (in this directory) | File | Purpose | |---|---| -| `workqueue-census.tsv` | Tab-separated table of all 96 rows (artifact of record). | +| `workqueue-census.tsv` | Tab-separated table of all 97 rows (artifact of record). | | `workqueue-census.md` | Markdown table of the same rows. | -| `extract_manifest.json` | Schema + counts + provenance. `extracted_at_utc` is the measured run moment, not a constant. | -| `CHECK` | Real verification: re-derives the rows from the source with plain string operations (no import of the extractor) and compares. Python 3; exits 0 only if correct. | +| `extract_manifest.json` | Schema + counts + provenance + the source pin. `extracted_at_utc` is the measured run moment, not a constant. | +| `CHECK` | Real verification: re-derives the rows from the **pinned** source bytes with plain string operations (no import of the extractor) and compares. Python 3; exits 0 only if correct. | | `extract_workqueue.py` | The extractor that produced the TSV/MD/manifest. | +## Provenance (Q135, 2026-10-31) + +The manifest no longer names its source by absolute host path — that was the +Q69/Q106/Q109 class, and it made the artifact grade whatever tree the host +happened to hold. It records: + +| Field | What it is | +|---|---| +| `source` | `WORK-QUEUE.md` — repo-relative, never a host path | +| `source_rev` | `git rev-parse HEAD` at generation | +| `source_blob_sha` | `git rev-parse HEAD:WORK-QUEUE.md` — the pinned bytes | +| `source_sha256` | sha256 of the bytes actually read | +| `source_lines` | line count of those bytes | +| `source_matches_head_blob` | whether the pin was taken from a clean worktree | +| `row_digest` | sha256 of the emitted TSV data rows | + +The pin is honest or the extractor refuses. If the worktree's `WORK-QUEUE.md` +differs from the HEAD blob, `extract_workqueue.py` exits 1 and writes nothing: +a pin to a blob that does not contain the rows in the TSV is a false +provenance claim. `--allow-dirty` snapshots the worktree anyway and records +`source_blob_sha: null` with `source_matches_head_blob: false`, so `CHECK` +grades the worktree bytes and says so. + ## Verification `CHECK` is a Python 3 program, not a shell script. Run it with the interpreter itself (the file is executable, so `./CHECK` works too): python3 CHECK - python3 CHECK --now=2026-10-28T18:31:34Z # also grade recency + python3 CHECK --now=2026-10-31T00:00:00Z # also grade stamp recency python3 CHECK --selftest # prove every leg can fail -It re-derives the expected rows from `WORK-QUEUE.md` using string operations +It reads the source from the blob the manifest pins (`git cat-file blob`), not +from the worktree file, re-derives the expected rows with string operations (independent implementation from `extract_workqueue.py`), then compares qid + -title against the committed TSV, recomputes the manifest's counts, grades the -extraction timestamp, and checks for delimiter corruption. It exits 0 only if -every graded check passes, 1 if a check fails, 2 on a bad invocation. +title against the committed TSV, recomputes the manifest's counts and row +digest, grades the provenance and the extraction timestamp, and checks for +delimiter corruption. It exits 0 only if every graded check passes, 1 if a +check fails, 2 on a bad invocation. + +**A declared pin that git cannot read is a fault, never a fallback.** An +artifact that cites a blob nobody can open has no provenance, and grading +whatever the worktree now holds would grade bytes the manifest never named. +(The one cost: a shallow clone that does not carry the pinned object fails +this leg. Full clones are the norm here; `--allow-dirty` generation records no +pin and skips the problem by declaring it.) **The timestamp leg does not read the wall clock.** `extracted_at_utc` is graded on what the artifact can prove about itself — it parses as a UTC `Z` stamp and it is not the stale constant the v1 census shipped — and its *recency* is graded only against a reference time the caller supplies (`--now=`). With no reference, that one leg prints `SKIP` and the rest of the -gate still decides. A committed artifact therefore reads the same verdict on -the day it lands and a year later; the earlier form (`abs(now - dt) < 120`) -could only be green inside a 120-second window after generation. +gate still decides. + +**Drift is reported, not graded.** When the worktree's heading count differs +from the pinned source's, `CHECK` prints +`ADVISORY source-moved worktree= pinned=` and still exits 0. A census is +a snapshot of named bytes; a snapshot that goes red because the world moved is +a snapshot nobody regenerates. Regenerate to move the snapshot forward: + + python3 extract_workqueue.py # refuses on a dirty source, by design ## Why a new census @@ -56,24 +99,43 @@ The earlier `2026-10-28-workqueue-census/` failed its own contract: This v2 production is the corrected, independently verified replacement. -### v2's own correction (2026-10-28, Q122) +### v2's own corrections -v2 fixed the tautology and then bound its verdict to the clock instead: the -recency leg was `abs(now - dt) < 120` against the frozen +**2026-10-28 (Q122).** v2 fixed the tautology and then bound its verdict to the +clock instead: the recency leg was `abs(now - dt) < 120` against the frozen `2026-10-28T18:31:34Z`, so the artifact could only ever be green for the two minutes after its own generation, and the documented command `bash CHECK` could not run at all (the file is Python). Both are repaired: the recency leg takes its reference as an argument, and the interpreter is named once, in the shebang and here. +**2026-10-31 (Q135).** Three things had gone wrong with the artifact itself, +all measured at `7b42050` before the repair: its `CHECK` was **red at HEAD** +(`FAIL row count 'committed=96 golden=97'`, `FAIL manifest.rows 96 vs 97`, +`FAIL manifest.distinct_q_labels 85 vs 86`) because the table was a +2026-10-28 snapshot of a file that had since gained a heading and nothing +regenerated it; its `manifest.source` was the absolute host path +`/root/troupe-standup/WORK-QUEUE.md`; and its verdict was a function of the +worktree at read time, so every append moved the answer. The repair pins the +bytes, grades the pin, records a row digest, and makes drift an advisory. The +counterfactual is committed as a selftest cell and demonstrated on real +commits: at a descendant revision carrying three extra headings, `CHECK` still +reads `ALL CHECKS PASSED (97 rows, ...)` rc 0 plus the advisory. + ## Evidence - Source: `WORK-QUEUE.md` at the repo root (resolved from this directory: - `HERE/../../..`), not a pinned host path. -- Extracted: `2026-10-28T18:31:34Z` UTC (measured at generation). -- Independent re-derivation count: 96 rows; 85 distinct labels; 11 reopened + `HERE/../../..`), pinned to blob `624ecb72ead1f01b60cdbaca892278858167aa5e` + at rev `7b4205014c8f95e6dbf45d1ba62a0b39cc926a88`. +- Extracted: `2026-10-31T09:42:38Z` UTC (measured at generation; the manifest + carries this stamp, and `--now=` grades recency against it). +- Independent re-derivation count: 97 rows; 86 distinct labels; 11 reopened (`Q37 Q38 Q39 Q43 Q60 Q72 Q79 Q81 Q82 Q83 Q93`). -- `python3 CHECK` → `ALL CHECKS PASSED (96 rows, 85 distinct labels, 11 reopened)`, - rc 0. -- `python3 CHECK --selftest` → `SELFTEST OK (6 cells: 2 green controls, 4 red - mutations)`, rc 0. +- `python3 CHECK` → `ALL CHECKS PASSED (97 rows, 86 distinct labels, 11 + reopened)`, rc 0. +- `python3 CHECK --selftest` → `SELFTEST OK (9 cells: 3 green controls, 6 red + mutations)`, rc 0 — the red cells being an unreadable pin, a host-path + source, a corrupted row digest, a corrupted title, the stale constant, and a + stamp moved one day. +- `python3 extract_workqueue.py` on an uncommitted `WORK-QUEUE.md` → rc 1, + `REFUSED: the worktree's WORK-QUEUE.md is not the HEAD blob`, TSV untouched. diff --git a/wizard-work/ledger/workqueue-census-v2/extract_manifest.json b/wizard-work/ledger/workqueue-census-v2/extract_manifest.json index 822bc37..9d24ed6 100644 --- a/wizard-work/ledger/workqueue-census-v2/extract_manifest.json +++ b/wizard-work/ledger/workqueue-census-v2/extract_manifest.json @@ -1,14 +1,20 @@ { "schema": "workqueue-census-v2", - "source": "/root/troupe-standup/WORK-QUEUE.md", - "rows": 96, + "source": "WORK-QUEUE.md", + "source_rev": "7b4205014c8f95e6dbf45d1ba62a0b39cc926a88", + "source_blob_sha": "624ecb72ead1f01b60cdbaca892278858167aa5e", + "source_matches_head_blob": true, + "source_sha256": "e34193da6d70cbe8d4d554e186e873d73c03cb29f21dc54135778c6ce91ef3ee", + "source_lines": 13794, + "rows": 97, "columns": [ "q", "title", "status", "done_implies" ], - "distinct_q_labels": 85, + "row_digest": "84df099f6680a0ac7c0711d130301e8d245f31f834246d6bc1e49cf9e38f1622", + "distinct_q_labels": 86, "reopened_q_labels": [ "Q37", "Q38", @@ -22,7 +28,7 @@ "Q83", "Q93" ], - "extracted_at_utc": "2026-10-28T18:31:34Z", + "extracted_at_utc": "2026-10-31T09:42:38Z", "generated_by": "extract_workqueue.py", "verification": "CHECK (independent re-derivation, does not import extractor)" } \ No newline at end of file diff --git a/wizard-work/ledger/workqueue-census-v2/extract_workqueue.py b/wizard-work/ledger/workqueue-census-v2/extract_workqueue.py index c2c5df6..315a7a9 100644 --- a/wizard-work/ledger/workqueue-census-v2/extract_workqueue.py +++ b/wizard-work/ledger/workqueue-census-v2/extract_workqueue.py @@ -13,6 +13,26 @@ Output (all in this directory): README.md documentation for this artifact CHECK real verification (independent re-derivation) +Provenance (Q135, 2026-10-31). The manifest used to name its source by +absolute host path (`/root/troupe-standup/WORK-QUEUE.md`), which is the +Q69/Q106/Q109 host-pin class, and it carried no pin to the bytes it read, so +the artifact's own rows could not be re-derived by anyone who reads it after +the queue has moved on. It now records: + +* source repo-relative path (`WORK-QUEUE.md`), never a host path +* source_rev `git rev-parse HEAD` at generation +* source_blob_sha `git rev-parse HEAD:WORK-QUEUE.md` -- the pinned bytes +* source_sha256 sha256 of the bytes actually read +* source_lines line count of those bytes +* row_digest sha256 of the emitted TSV data rows + +The pin is honest or the extractor refuses. If the worktree's WORK-QUEUE.md +differs from the HEAD blob, this exits 1 and writes nothing, because a pin to +a blob that does not contain the rows in the TSV is a false provenance claim; +`--allow-dirty` writes the snapshot anyway and records `source_blob_sha: null` +with `source_matches_head_blob: false`, so CHECK grades the worktree bytes and +says so. + Conventions ----------- * A row is created for every line matching '## ...' where the '' is @@ -27,9 +47,12 @@ Conventions when no such marker precedes the next heading. """ +import hashlib import json import os import re +import subprocess +import sys from datetime import datetime, timezone HERE = os.path.dirname(os.path.abspath(__file__)) @@ -113,9 +136,84 @@ def extract(lines): return rows +def _sha256(data): + return hashlib.sha256(data).hexdigest() + + +def _git(args, cwd): + """Run git, returning stdout stripped, or None when git or the object is absent.""" + try: + proc = subprocess.run(["git", "-C", cwd] + args, + capture_output=True, text=True) + except OSError: + return None + if proc.returncode != 0: + return None + return proc.stdout.strip() + + +def pin_source(repo_root, raw): + """Return (provenance dict, error). The pin is the HEAD blob or a refusal.""" + blob_sha = _git(["rev-parse", "HEAD:" + "WORK-QUEUE.md"], repo_root) + rev = _git(["rev-parse", "HEAD"], repo_root) + if blob_sha is None or rev is None: + return None, ("cannot read WORK-QUEUE.md at HEAD (git missing, not a " + "repository, or the path is untracked) -- pass " + "--allow-dirty to snapshot the worktree instead") + try: + blob = subprocess.run(["git", "-C", repo_root, "cat-file", "blob", blob_sha], + capture_output=True) + except OSError: + return None, "git cat-file is unavailable -- pass --allow-dirty" + if blob.returncode != 0: + return None, "cannot read blob %s -- pass --allow-dirty" % blob_sha + if _sha256(blob.stdout) != _sha256(raw): + return None, ("the worktree's WORK-QUEUE.md is not the HEAD blob (%s): " + "commit it first, or pass --allow-dirty to snapshot the " + "worktree and record that the pin is absent" % blob_sha[:12]) + return {"source_rev": rev, "source_blob_sha": blob_sha, + "source_matches_head_blob": True}, None + + +USAGE = """usage: extract_workqueue.py [--allow-dirty] + + (no flag) pin the source to the HEAD blob; refuse (rc 1) if the + worktree's WORK-QUEUE.md is not that blob + --allow-dirty snapshot the worktree anyway; the manifest records + source_blob_sha: null and source_matches_head_blob: false + -h, --help this text + +Writes workqueue-census.tsv, workqueue-census.md and extract_manifest.json to +this directory. Exit 0 clean, 1 refused (source not pinned and not allowed to +be dirty), 2 usage error. +""" + + def main(): - with open(SRC, encoding="utf-8") as fh: - lines = fh.read().splitlines() + argv = sys.argv[1:] + unknown = [a for a in argv if a not in ("--allow-dirty", "-h", "--help")] + if unknown: + print("usage error: unknown argument(s) %r" % unknown, file=sys.stderr) + print(USAGE, file=sys.stderr) + return 2 + if "-h" in argv or "--help" in argv: + print(USAGE) + return 0 + allow_dirty = "--allow-dirty" in argv + with open(SRC, "rb") as fh: + raw = fh.read() + lines = raw.decode("utf-8").splitlines() + + prov, err = ({"source_rev": None, "source_blob_sha": None, + "source_matches_head_blob": False}, None) + if not allow_dirty: + prov, err = pin_source(REPO_ROOT, raw) + if err is not None: + print("REFUSED: " + err, file=sys.stderr) + return 1 + else: + prov["source_rev"] = _git(["rev-parse", "HEAD"], REPO_ROOT) + assert prov is not None rows = extract(lines) @@ -123,12 +221,12 @@ def main(): md_path = os.path.join(OUT, "workqueue-census.md") json_path = os.path.join(OUT, "extract_manifest.json") + data = "".join( + f"{r['q']}\t{r['title']}\t{r['status']}\t{r['done_implies']}\n" for r in rows + ) with open(tsv_path, "w", encoding="utf-8") as fh: fh.write("q\ttitle\tstatus\tdone_implies\n") - for r in rows: - fh.write( - f"{r['q']}\t{r['title']}\t{r['status']}\t{r['done_implies']}\n" - ) + fh.write(data) distinct = sorted({r["q"] for r in rows}) reopened = sorted(q for q in distinct if sum(1 for r in rows if r["q"] == q) > 1) @@ -148,9 +246,15 @@ def main(): manifest = { "schema": "workqueue-census-v2", - "source": os.path.abspath(SRC), + "source": "WORK-QUEUE.md", + "source_rev": prov["source_rev"], + "source_blob_sha": prov["source_blob_sha"], + "source_matches_head_blob": prov["source_matches_head_blob"], + "source_sha256": _sha256(raw), + "source_lines": len(lines), "rows": len(rows), "columns": ["q", "title", "status", "done_implies"], + "row_digest": _sha256(data.encode("utf-8")), "distinct_q_labels": len(distinct), "reopened_q_labels": reopened, "extracted_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), @@ -162,8 +266,12 @@ def main(): print(f"rows: {len(rows)}") print(f"distinct labels: {len(distinct)}; reopened: {reopened}") + print(f"source pin: rev={prov['source_rev']} blob={prov['source_blob_sha']}" + if prov["source_blob_sha"] else + "source pin: ABSENT (--allow-dirty; the manifest says so)") print(f"written: {tsv_path}, {md_path}, {json_path}") + return 0 if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/wizard-work/ledger/workqueue-census-v2/workqueue-census.md b/wizard-work/ledger/workqueue-census-v2/workqueue-census.md index 743e68b..18f7fc4 100644 --- a/wizard-work/ledger/workqueue-census-v2/workqueue-census.md +++ b/wizard-work/ledger/workqueue-census-v2/workqueue-census.md @@ -1,6 +1,6 @@ # Work queue census v2 (extract lane) -One row per '## Q ...' heading in WORK-QUEUE.md. Extracted from 96 headings; 85 distinct q labels; 11 reopened (Q37, Q38, Q39, Q43, Q60, Q72, Q79, Q81, Q82, Q83, Q93). extracted_at_utc is the measurement moment, not a constant. +One row per '## Q ...' heading in WORK-QUEUE.md. Extracted from 97 headings; 86 distinct q labels; 11 reopened (Q37, Q38, Q39, Q43, Q60, Q72, Q79, Q81, Q82, Q83, Q93). extracted_at_utc is the measurement moment, not a constant. | q | title | status | done_implies | |---|---|---|---| @@ -100,3 +100,4 @@ One row per '## Q ...' heading in WORK-QUEUE.md. Extracted from 96 headings; | Q93 | closure — the malformed axis grades a contract now, the two false reds are gone, and the replacement contract has a hole the prose pin did not *(opened by the pipe 2026-10-27 at `d56105a`; landed by forge at `9878f12`, closed here 2026-10-27)* | DONE at `9878f12` (`tools/selftest_sweep.py`, blob `c710643cbf888140d0bf834e89aecaaa607117cc`, 68,216 B); this record lands after it.**. | | | Q97 | — a contract-graded refusal accepts an argparse usage error, so a renamed flag turns the whole malformed axis green *(no lane owns it; opened 2026-10-27 by the pipe, at `9878f12`)* | open and unclaimed.** Anyone may take it; the shape is `tools/selftest_sweep.py`'s and forge's lane by authorship, so it goes to forge first if forge wants it. | ** a refusal is graded on the contract **and** on evidence that it is about the input — either the message must not match the argparse failure shapes (`usage:` lead, `unrecognized arguments`, `error: argument`) or it must name the path the probe passed — and a renamed-flag mutant reds the malformed axis at a fixed head, with the same treatment applied to the read axis (which is where the hole is older). **Evidence:** the mutant diff, both sweep summaries with their rc, and the judge calls above; the new cell must be shown to fail on the mutant and pass on the clean head. | | Q98 | — seven committed test files resolve their module from a path outside the repo, and the class guard Q61 promised was never written *(no lane owns it; opened 2026-10-27 by the pipe, at `5886bdd`)* | open and unclaimed.** Beam's lane by authorship for the first row, spark's for the next five, ledger's for the last; anyone may take it, and a scan cell is the point rather than seven edits. | ** a committed check that fails when any `test_*.py` under `wizard-work/` resolves an imported module | +| Q125 | — the spark lane holds eleven line/word/char counters, and the newest one replaced its predecessor's capabilities without retiring its verifier *(spark lane; opened 2026-10-28 by the pipe, at `6b5197b`)* | open and unclaimed.** Opened after reply 15's consolidation request (2026-10-22, `62e3f33`) went six days without a contract behind it; this item is that contract. | ** one surviving text-statistics tool in the lane, and it states its counting definition in the README — newline-terminated lines versus records (the two differ by exactly the files that lack a trailing newline: at `7359bce` that was 112 files and 5,475 vs 5,587 lines), and `split(',')` versus quoted-field CSV parsing (`"a,b",c` is 2 fields; the current tool reads 3). Then: every other file in the list above retired in a commit that names it and says where its behaviour went, `verify_line_counter.sh` either repaired to the surviving interface in the same commit or deleted with that sentence, `VERIFICATION.md` rewritten to describe the tool that exists, the README's verification line runnable from the repo root, and a cell that dies on `count → return 10` and on `count → non-blank lines only` (fixture with blank lines; `printf 'a\n\n\nb\n'` reads 4). | diff --git a/wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv b/wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv index aa41782..6c68ffb 100644 --- a/wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv +++ b/wizard-work/ledger/workqueue-census-v2/workqueue-census.tsv @@ -95,3 +95,4 @@ Q96 — `q60_enforce.py`'s self-scan cannot see a fixture literal in its own sou Q93 closure — the malformed axis grades a contract now, the two false reds are gone, and the replacement contract has a hole the prose pin did not *(opened by the pipe 2026-10-27 at `d56105a`; landed by forge at `9878f12`, closed here 2026-10-27)* DONE at `9878f12` (`tools/selftest_sweep.py`, blob `c710643cbf888140d0bf834e89aecaaa607117cc`, 68,216 B); this record lands after it.**. Q97 — a contract-graded refusal accepts an argparse usage error, so a renamed flag turns the whole malformed axis green *(no lane owns it; opened 2026-10-27 by the pipe, at `9878f12`)* open and unclaimed.** Anyone may take it; the shape is `tools/selftest_sweep.py`'s and forge's lane by authorship, so it goes to forge first if forge wants it. ** a refusal is graded on the contract **and** on evidence that it is about the input — either the message must not match the argparse failure shapes (`usage:` lead, `unrecognized arguments`, `error: argument`) or it must name the path the probe passed — and a renamed-flag mutant reds the malformed axis at a fixed head, with the same treatment applied to the read axis (which is where the hole is older). **Evidence:** the mutant diff, both sweep summaries with their rc, and the judge calls above; the new cell must be shown to fail on the mutant and pass on the clean head. Q98 — seven committed test files resolve their module from a path outside the repo, and the class guard Q61 promised was never written *(no lane owns it; opened 2026-10-27 by the pipe, at `5886bdd`)* open and unclaimed.** Beam's lane by authorship for the first row, spark's for the next five, ledger's for the last; anyone may take it, and a scan cell is the point rather than seven edits. ** a committed check that fails when any `test_*.py` under `wizard-work/` resolves an imported module +Q125 — the spark lane holds eleven line/word/char counters, and the newest one replaced its predecessor's capabilities without retiring its verifier *(spark lane; opened 2026-10-28 by the pipe, at `6b5197b`)* open and unclaimed.** Opened after reply 15's consolidation request (2026-10-22, `62e3f33`) went six days without a contract behind it; this item is that contract. ** one surviving text-statistics tool in the lane, and it states its counting definition in the README — newline-terminated lines versus records (the two differ by exactly the files that lack a trailing newline: at `7359bce` that was 112 files and 5,475 vs 5,587 lines), and `split(',')` versus quoted-field CSV parsing (`"a,b",c` is 2 fields; the current tool reads 3). Then: every other file in the list above retired in a commit that names it and says where its behaviour went, `verify_line_counter.sh` either repaired to the surviving interface in the same commit or deleted with that sentence, `VERIFICATION.md` rewritten to describe the tool that exists, the README's verification line runnable from the repo root, and a cell that dies on `count → return 10` and on `count → non-blank lines only` (fixture with blank lines; `printf 'a\n\n\nb\n'` reads 4).