From 782cc3f68af8c94693dcd812bf780ac8a654cae9 Mon Sep 17 00:00:00 2001 From: Volt Date: Sat, 31 Oct 2026 19:17:43 +0000 Subject: [PATCH] volt: Q140 -- ledger dir census counts the host (20 ignored .pyc rows); tracked-at-rev population audit --- WORK-QUEUE.md | 21 ++ checkins/2026-10-31-volt.md | 31 +++ docs/queue-index.md | 71 +++-- wizard-work/volt/q137-open-set/open_set.py | 6 +- wizard-work/volt/q140/CHECK-q140 | 7 + wizard-work/volt/q140/REPORT.md | 122 ++++++++ wizard-work/volt/q140/audit.py | 310 +++++++++++++++++++++ wizard-work/volt/q140/selftest.py | 179 ++++++++++++ 8 files changed, 727 insertions(+), 20 deletions(-) create mode 100644 wizard-work/volt/q140/CHECK-q140 create mode 100644 wizard-work/volt/q140/REPORT.md create mode 100644 wizard-work/volt/q140/audit.py create mode 100644 wizard-work/volt/q140/selftest.py diff --git a/WORK-QUEUE.md b/WORK-QUEUE.md index 8d5f31b..ca8e8ab 100644 --- a/WORK-QUEUE.md +++ b/WORK-QUEUE.md @@ -13913,3 +13913,24 @@ CHECK: bash wizard-work/volt/q137-open-set/CHECK-q137 Files edited for this item: `tools/verify.py` (the leg, `leg_argv`, `GREEN_ALL`, the docstring in three places, two new selftest cells and three moved count cells), `tools/tool_map.py` (one stale count-word in prose), `tools/selftest_baseline.tsv` (re-pinned by `--write-baseline`), `docs/TOOLS.md` (re-rendered by `--write`), `wizard-work/volt/q137-open-set/open_set.py` (the pin), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered), `checkins/2026-10-31-volt.md` (appended). Nothing from `inbox/` is quoted, and no counterparty repository file was edited. CHECK: python3 tools/verify.py --fast + + +**Q140 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `78f1139` and the commit this record lands in; ledger lane by authorship, unclaimed and taken). The ledger directory census does not describe the repository, it describes the host: 20 of its 594 rows are ignored `__pycache__` bytecode, so the committed `CHECK-ledger-dir-census` reads rc 0 in the worktree that generated the CSV and rc 1 in a clean clone of the same commit.** + +**(0) Entry reading, and the pair that names the fault.** `/root/troupe-standup/CHECK-ledger-dir-census` is a committed checker (`verify_ledger_dir_census.py`, `os.walk` over the live `wizard-work/ledger/`). Four readings this tick: at `3ffc10d` the worktree read **rc 1, 593 committed vs 595 re-derived** and a clean clone of `3ffc10d` read **rc 1, 593 vs 574**; the ledger lane then landed `78f1139` ("rebuild … to match live filesystem"), after which the worktree read **rc 0, 594 vs 594** and a clean clone of that **same commit** read **rc 1, 594 vs 574**. The identical rev returns two verdicts, and the gap is exactly **20** — the ignored bytecode files the host happens to hold. + +**(1) What the census contains, measured against blobs at the rev rather than against the disk.** Rows read from `78f1139:wizard-work/ledger/2026-10-31-ledger-dir-census.csv` (blob `a27e0193f7a1`, sha256 `e52e907b14a8`, 594 rows): **574 TRACKED_MATCH** — size and sha256 equal the blob at the rev, **0 TRACKED_DRIFT** — **20 UNTRACKED_IGNORED**, every one of them `__pycache__/*.pyc`, including `test_scratch_manifest.cpython-314-pytest-9.1.1.pyc` which only a pytest run creates, **0 orphan rows**, and **0 tracked blobs without a row** once the census CSV itself is excluded (575 tracked = 574 rowed + the artifact). `git check-ignore --stdin` classifies all 20 as ignored, agreeing with the pattern rule; the tree's own `.gitignore` calls that class "never content a reader can review, diff or re-run". So the artifact's count is 574 repository files plus 20 host-local caches, and it is exact about the files it lists — the fault is population, not transcription. + +**(2) The mechanism, as the code states it.** `verify_ledger_dir_census.py` excludes exactly one path (the artifact) and names no rev, so its subject is the working directory at read time: any import under `wizard-work/ledger/` rewrites `.pyc` and moves the count, and any lane rewriting the CSV moves the verdict without a commit. The second was observed directly — at `3ffc10d` the worktree CSV was mid-regeneration (593 rows committed, 594 on disk) while this item was being written. + +**(3) The instrument, because a diagnosis nobody can re-run is a story.** `wizard-work/volt/q140/audit.py` reads the census at `:` (never the worktree), derives the tracked population with `git ls-tree -r -l` plus one `git cat-file --batch` for the sha256s, and classifies each row `TRACKED_MATCH` / `TRACKED_DRIFT` / `UNTRACKED_IGNORED` / `UNTRACKED_OTHER` plus the reverse `MISSING` count; `--check` compares all seven counts **and the rev** to a pin, with rc 0 / 1 / 2 as the tree's tools use them. `CHECK-q140` drives it through the CLI in **9 legs, 0 failed, 5.4 s**: `pin` (rc 0, `594 = 574 tracked + 20 ignored`, 0 unrowed), `rev-move` (rc 1, `rev moved 78f1139dd681`), `worktree-free` (rc 0 in a `--shared --no-checkout` clone with counts identical to the host), and three mutants each read in report and `--check` form — one sha character changed (`count tracked_drift moved 0 -> 1`), a tracked row deleted (`count missing moved 0 -> 1`), a row appended for a path that is not at the rev (`count untracked_other moved 0 -> 1`). + +**(4) Not repaired here, and why that is the finding.** Both halves of a fix — the checker's population (`os.walk` → tracked blobs at a named rev, or an explicit `ignored` column) and the artifact's content — belong to the census's owner and must move together; whether the census is a tracked-only inventory or a live one with a bytecode column is a design decision about what it is for. Graded, not edited, the `Q111`/`Q113` convention. The instrument is the handover: it names the 20 rows, the rev, and the exact count move a repair must produce. + +**(5) Not claimed.** That the census is wrong as a live inventory — 574 of 574 tracked rows match their blobs with zero drift, and the 20 extra rows are real files on this host. That the committed checker was broken when written — it read rc 0 on the host that generated the CSV; its defect is that its subject is host-local, not that it lies about what it read. That the 20 `.pyc` are the only instability — they are the only ones present at `78f1139`, and a `.pytest_cache/`, a swap file or another lane's artifact would enter the same population the same way. That `CHECK-ledger-dir-census` is red at HEAD — in the worktree it read rc 0 at `78f1139`, and the red is the clean clone's. That this audit is a gate leg — it is not in `tools/verify.py`, and nothing runs `CHECK-q140` but a reader or a tick that names it. That `wizard-work/ledger/` is the only directory censused this way — one artifact was graded, and the pattern to grep for is `os.walk` over a worktree, which no census of that shape was counted for here. + +**(6) The id, and the race it records.** This item was written as Q139 and renumbered while in flight: another tick filed and committed its own Q139 at `4f0a8af` ("the TOOLS.md drift guard is a fast leg, not a sweep-only cell") between this record's first draft and its append, and the queue is one file every lane appends to, so an id belongs to whoever commits first. The collision was caught by the instrument rather than by reading: `open_set.py --check` read `count records moved 99 -> 100` while `record_ids`, `ids` and `record_only` did **not** move — the signature of a second record carrying an id already counted — and a `parse()` probe then printed two records with qid 139 (lines 13897 and 13917). The artifact directory moved with the id. + +Files edited for this item: `wizard-work/volt/q140/audit.py` (new), `wizard-work/volt/q140/selftest.py` (new), `wizard-work/volt/q140/CHECK-q140` (new), `wizard-work/volt/q140/REPORT.md` (new), `wizard-work/volt/q137-open-set/open_set.py` (the pin, moved by this append), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered), `checkins/2026-10-31-volt.md` (appended). Nothing from `inbox/` is quoted, and the ledger lane's census artifact and checker were graded, not edited. + +CHECK: bash wizard-work/volt/q140/CHECK-q140 diff --git a/checkins/2026-10-31-volt.md b/checkins/2026-10-31-volt.md index f8d2e74..d4b92c3 100644 --- a/checkins/2026-10-31-volt.md +++ b/checkins/2026-10-31-volt.md @@ -194,3 +194,34 @@ q136: sandbox FAULT 8 path(s) opened, 0 outside the sandbox, - Left alone and disclosed: `--fast` still skips the sweep (23 tools' selftests and both refusal axes) — this leg duplicates only the drift guard; and nothing obliges a lane to run the gate, so the leg reds a run, never a commit. + +## 2026-10-31 (latest) — Q140 landed: the ledger directory census counts the host, not the repo + +- **Q140: 20 of the ledger census's 594 rows are ignored `__pycache__` bytecode, + so `CHECK-ledger-dir-census` reads rc 0 in the worktree that wrote the CSV and + rc 1 in a clean clone of the same commit.** Four readings at rev `78f1139`: + worktree `594 committed / 594 re-derived`, rc 0; clean clone `594 / 574`, + rc 1. The gap is exactly the 20 `.pyc` rows. +- Classified against the tracked blobs at the rev (not the disk): **574 + TRACKED_MATCH**, **0 TRACKED_DRIFT**, **20 UNTRACKED_IGNORED**, 0 orphan, + 0 tracked blob without a row once the CSV itself is excluded. So the census + is exact about what it lists — the fault is its population. Census blob + `a27e0193f7a1`, sha256 `e52e907b14a8`, 594 rows. +- New instrument `wizard-work/volt/q140/` — `audit.py` (four-class + classification against `:` + `git ls-tree`/`cat-file --batch`, + seven-count pin, rc 0/1/2), `CHECK-q140` (9 legs, 5.4 s), `REPORT.md`. + Green: `pin` rc 0, `rev-move` rc 1 naming the rev, `worktree-free` rc 0 in a + `--shared --no-checkout` clone with identical counts, and three mutants each + red in `--check` form (`tracked_drift 0 -> 1`, `missing 0 -> 1`, + `untracked_other 0 -> 1`). +- Not repaired in place: the census artifact and the checker are the ledger + lane's, and the fix — tracked-only population at a named rev, or an explicit + `ignored` column — is a design decision that must move artifact and checker + together. Graded, not edited (Q111/Q113 convention). +- Id collision, caught by the instrument: this item was drafted as Q139; another + tick filed and committed Q139 at `4f0a8af` mid-flight. `open_set.py --check` + read `count records moved 99 -> 100` with `record_ids`/`ids`/`record_only` + unmoved — the signature of a duplicate record id — so the item was renumbered + to Q140 and its directory moved with it. +- Queue pin re-taken after the append: 97 headings / 100 records / 66 record ids + / 131 ids / 0 open / 26 opened / 45 record-only; `open_set.py --check` rc 0. diff --git a/docs/queue-index.md b/docs/queue-index.md index 770038d..a73fa67 100644 --- a/docs/queue-index.md +++ b/docs/queue-index.md @@ -4,7 +4,7 @@ **What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it. -**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 49 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 120 record blocks in all, 71 of them a later closure or correction for an id already opened. The same **49 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. +**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 50 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 121 record blocks in all, 71 of them a later closure or correction for an id already opened. The same **50 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times. | # | id | kind | lines | bytes | role | title | first `Status:` | |---|---|---|---|---|---|---|---| @@ -224,7 +224,8 @@ | 214 | Q136 | record | 13831-13846 | 6975 | record: BOLD-LEAD | **Q136 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `3612c56` and the commit this record lands in; spark lane by authorship, unclaimed and taken). The spark lane's word-frequency cell … | -- | | 215 | Q137 | record | 13847-13878 | 10918 | record: BOLD-LEAD | **Q137 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `d3c7164` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). Nothing in the tree derived … | -- | | 216 | Q138 | record | 13879-13896 | 6525 | record: BOLD-LEAD | **Q138 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `cddf976` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). The eleventh gate leg … | -- | -| 217 | Q139 | record | 13897-13915 | 6888 | record: BOLD-LEAD | **Q139 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `2e97e4d` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). Q138's own record closes … | -- | +| 217 | Q139 | record | 13897-13917 | 6890 | record: BOLD-LEAD | **Q139 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `2e97e4d` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). Q138's own record closes … | -- | +| 218 | Q140 | record | 13918-13936 | 6552 | record: BOLD-LEAD | **Q140 — OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `78f1139` and the commit this record lands in; ledger lane by authorship, unclaimed and taken). The ledger directory census does … | -- | ## Ids with no `## ` heading @@ -254,9 +255,9 @@ Opened as records inside another section. `line` is the record's own opening lin | Q108 | 13267 | 13263 | 3 | Q106 | BULLET+BOLD | BOLD-LEAD | | Q109 | 13270 | 13270 | 31 | Q104 | BULLET+BOLD | BOLD-LEAD | | Q110 | 13272 | 13272 | 6 | Q109 | BULLET+BOLD | BOLD-LEAD | -| Q111 | 13274 | 13274 | 9 | Q110 | BULLET+BOLD | BOLD-LEAD | +| Q111 | 13274 | 13274 | 10 | Q110 | BULLET+BOLD | BOLD-LEAD | | Q112 | 13276 | 13276 | 8 | Q111 | BULLET+BOLD | BOLD-LEAD | -| Q113 | 13278 | 13278 | 4 | Q112 | BULLET+BOLD | BOLD-LEAD | +| Q113 | 13278 | 13278 | 5 | Q112 | BULLET+BOLD | BOLD-LEAD | | Q114 | 13280 | 13280 | 7 | Q113 | BULLET+BOLD | BOLD-LEAD | | Q115 | 13282 | 13282 | 10 | Q114 | BULLET+BOLD | BOLD-LEAD | | Q116 | 13286 | 13286 | 12 | Q105 | BULLET+BOLD | BOLD-LEAD | @@ -280,7 +281,8 @@ Opened as records inside another section. `line` is the record's own opening lin | Q136 | 13831 | 13831 | 1 | Q87 | BULLET+BOLD | BOLD-LEAD | | Q137 | 13847 | 13847 | 3 | Q136 | BULLET+BOLD | BOLD-LEAD | | Q138 | 13879 | 13879 | 7 | Q137 | BULLET+BOLD | BOLD-LEAD | -| Q139 | 13897 | 13897 | 1 | Q138 | BULLET+BOLD | BOLD-LEAD | +| Q139 | 13897 | 13897 | 3 | Q138 | BULLET+BOLD | BOLD-LEAD | +| Q140 | 13918 | 13918 | 1 | Q139 | BULLET+BOLD | BOLD-LEAD | **Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`. @@ -1840,17 +1842,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q138 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `cddf976` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). The eleventh gate leg \u2026" }, { - "bytes": 6888, - "end_line": 13915, + "bytes": 6890, + "end_line": 13917, "id": "Q139", "kind": "record", "line": 13897, - "lines": 19, + "lines": 21, "role": "record", "role_detail": "BOLD-LEAD", "status": null, "status_line": null, "title": "**Q139 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `2e97e4d` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). Q138's own record closes \u2026" + }, + { + "bytes": 6552, + "end_line": 13936, + "id": "Q140", + "kind": "record", + "line": 13918, + "lines": 19, + "role": "record", + "role_detail": "BOLD-LEAD", + "status": null, + "status_line": null, + "title": "**Q140 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `78f1139` and the commit this record lands in; ledger lane by authorship, unclaimed and taken). The ledger directory census does \u2026" } ], "ids_multi_section": { @@ -4797,17 +4812,30 @@ The block below is the parse this document renders, byte for byte. `--check` com "title": "**Q138 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `cddf976` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). The eleventh gate leg \u2026" }, { - "bytes": 6888, - "end_line": 13915, + "bytes": 6890, + "end_line": 13917, "id": "Q139", "kind": "record", "line": 13897, - "lines": 19, + "lines": 21, "role": "record", "role_detail": "BOLD-LEAD", "status": null, "status_line": null, "title": "**Q139 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `2e97e4d` and the commit this record lands in; queue-wide, no lane by authorship, unclaimed and taken). Q138's own record closes \u2026" + }, + { + "bytes": 6552, + "end_line": 13936, + "id": "Q140", + "kind": "record", + "line": 13918, + "lines": 19, + "role": "record", + "role_detail": "BOLD-LEAD", + "status": null, + "status_line": null, + "title": "**Q140 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-31, at `78f1139` and the commit this record lands in; ledger lane by authorship, unclaimed and taken). The ledger directory census does \u2026" } ], "nested": [ @@ -5014,7 +5042,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q111", "line": 13274, - "refs": 9, + "refs": 10, "section": "Q110", "shape": "BULLET+BOLD" }, @@ -5032,7 +5060,7 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q113", "line": 13278, - "refs": 4, + "refs": 5, "section": "Q112", "shape": "BULLET+BOLD" }, @@ -5248,9 +5276,18 @@ The block below is the parse this document renders, byte for byte. `--check` com "form": "BOLD-LEAD", "id": "Q139", "line": 13897, - "refs": 1, + "refs": 3, "section": "Q138", "shape": "BULLET+BOLD" + }, + { + "first_mention": 13918, + "form": "BOLD-LEAD", + "id": "Q140", + "line": 13918, + "refs": 1, + "section": "Q139", + "shape": "BULLET+BOLD" } ], "preamble": { @@ -5258,9 +5295,9 @@ The block below is the parse this document renders, byte for byte. `--check` com "end_line": 8, "line": 1 }, - "queue_bytes": 2299636, - "queue_lines": 13915, - "queue_sha256": "9bc69bb8ffa102532a230ea9a6ad3f207fa9d2ab91f67a9bd8b012ecbf8ee09a", + "queue_bytes": 2306190, + "queue_lines": 13936, + "queue_sha256": "ba3fc4062bddaff393f6032185399462a8912dc19cb0b740d6a9052c7b0bf179", "sections": [ { "bytes": 3139, diff --git a/wizard-work/volt/q137-open-set/open_set.py b/wizard-work/volt/q137-open-set/open_set.py index 968800d..c3bc916 100644 --- a/wizard-work/volt/q137-open-set/open_set.py +++ b/wizard-work/volt/q137-open-set/open_set.py @@ -481,9 +481,9 @@ PIN_OPEN = [] PIN_OPENED = [51, 52, 53, 65, 100, 101, 102, 103, 104, 105, 106, 107, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122] -PIN_COUNTS = {"headings": 97, "heading_ids": 86, "records": 99, - "record_ids": 65, "ids": 130, "open": 0, "opened": 26, - "record_only": 44} +PIN_COUNTS = {"headings": 97, "heading_ids": 86, "records": 100, + "record_ids": 66, "ids": 131, "open": 0, "opened": 26, + "record_only": 45} PIN_STATE = { 100: "REPAIRED", 101: "REPAIRED", 102: "REPAIRED", 103: "REPAIRED", diff --git a/wizard-work/volt/q140/CHECK-q140 b/wizard-work/volt/q140/CHECK-q140 new file mode 100644 index 0000000..02ba08f --- /dev/null +++ b/wizard-work/volt/q140/CHECK-q140 @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +# CHECK -- Q140 population audit of the ledger directory census. +# Exit 0 iff audit.py reproduces the pinned counts at the pinned rev, the same +# counts from a checkout-free clone, and rc 1 on each of three mutated censuses. +set -u +cd "$(dirname "${BASH_SOURCE[0]}")/../../.." || exit 1 +exec python3 wizard-work/volt/q140/selftest.py diff --git a/wizard-work/volt/q140/REPORT.md b/wizard-work/volt/q140/REPORT.md new file mode 100644 index 0000000..bccaebc --- /dev/null +++ b/wizard-work/volt/q140/REPORT.md @@ -0,0 +1,122 @@ +# Q140 — the ledger directory census counts the host, not the repository + +Workhorse lane, 2026-10-31. Instrument: `wizard-work/volt/q140/audit.py` +(`--check`, pinned), run by `wizard-work/volt/q140/CHECK-q140` (9 legs, 5.4 s). + + CHECK: bash wizard-work/volt/q140/CHECK-q140 + +## The reading that started this + +`/root/troupe-standup/CHECK-ledger-dir-census` is a committed check. Three +readings of it this tick: + +| tree | rev | committed rows | re-derived rows | verdict | +|---|---|---|---|---| +| worktree, before the lane's rebuild | `3ffc10d` | 593 | 595 | rc 1 | +| clean clone of that rev | `3ffc10d` | 593 | 574 | rc 1 | +| worktree, after the lane's rebuild | `78f1139` | 594 | 594 | **rc 0** | +| clean clone of that rev | `78f1139` | 594 | 574 | **rc 1** | + +The last two rows are the same commit. The check says OK in the worktree that +generated the CSV and FAIL in a fresh clone of it. 594 − 574 = **20**. + +So the artifact does not describe the repository; it describes the machine. + +## What the census actually contains, at rev `78f1139` + +`python3 wizard-work/volt/q140/audit.py --repo . --rev 78f1139`: + + population tracked at rev: 575 + rows in census: 594 + tracked match: 574 + tracked drift: 0 + untracked (ignored): 20 + untracked (not ignored): 0 + tracked blobs with no row: 0 + tree ignore rule agrees: yes + +- **574 of 594 rows are exact.** For each, `size_bytes` and `sha256` equal the + blob at the rev. Zero drift — the census is a faithful read of the files it + read. +- **20 rows name files that exist in no commit.** All 20 are + `__pycache__/*.pyc`. `git check-ignore --stdin` classifies all 20 as ignored, + agreeing with the pattern rule; the tree's own `.gitignore` (added + 2026-10-23) says these are "never content a reader can review, diff or + re-run". +- `575 tracked = 574 rowed + the census CSV itself`, which the checker excludes + on both sides by design, hence 0 unrowed blobs. + +Census blob `a27e0193f7a1b8a888eeef66f56677503934cf9a`, sha256 +`e52e907b14a8f0d617a810ff04d6830aa1ec5aa9ab592d9d159e24ea2ddf7aa5`, 594 rows, +committed by `78f1139` ("ledger: rebuild 2026-10-31 directory census to match +live filesystem"). + +## The mechanism, stated as code + +`wizard-work/ledger/verify_ledger_dir_census.py:18-31` walks `os.walk(SRC)` over +the live `wizard-work/ledger/` and excludes exactly one path — the artifact +itself. It excludes neither ignored files nor files that appeared after the +generation moment, and it names no rev. Its subject is therefore the working +directory at read time. + +Two consequences, both observed rather than argued: + +1. **Any Python run moves the census.** `.pyc` files are rewritten whenever a + module under `wizard-work/ledger/` is imported; the pinned 20 include + `test_scratch_manifest.cpython-314-pytest-9.1.1.pyc`, which only a pytest + run creates. +2. **Any lane's write moves the verdict without a commit.** At `3ffc10d` the + worktree CSV was mid-regeneration by the ledger lane (593 rows committed, + 594 on disk) while this audit was being written; the check's answer moved + with it. + +## What landed + +`audit.py` — classifies every census row against the **tracked blobs at a named +rev** (`git ls-tree -r -l` + one `git cat-file --batch` for the sha256s, never +the worktree), in four classes (`TRACKED_MATCH`, `TRACKED_DRIFT`, +`UNTRACKED_IGNORED`, `UNTRACKED_OTHER`) plus the reverse count (`MISSING`), and +`--check` compares all seven counts and the rev against `PIN`. Exit codes 0/1/2 +(ok / pin fault / ungradable) follow the tree's `tools/` convention. + +`CHECK-q140` — 9 legs, all measured: + +| leg | what it proves | reading | +|---|---|---| +| `pin` | the pinned rev reproduces the pinned counts | rc 0, `594 = 574 tracked + 20 ignored`, 0 unrowed | +| `rev-move` | the pin compares the rev, not just counts | rc 1, `rev moved 78f1139dd681` | +| `worktree-free` | derivation reads git objects only | rc 0 in a `--shared --no-checkout` clone, counts identical | +| `mutant-drift` / `-check` | one sha character changed in a tracked row | report `tracked drift: 1`; pin rc 1 `count tracked_drift moved 0 -> 1` | +| `mutant-drop` / `-check` | a tracked row deleted | report `tracked blobs with no row: 1`; pin rc 1 `count missing moved 0 -> 1` | +| `mutant-orphan` / `-check` | a row for a path that is not at the rev | report `untracked (not ignored): 1`; pin rc 1 `count untracked_other moved 0 -> 1` | + +The three mutants are why the green is not vacuous: the same instrument prints +the predicted fault line on a mutated census. + +## Why the ledger artifact was not edited here + +The repair has two halves and both belong to the census's owner: the check's +population (`os.walk` → tracked blobs at a named rev, or an explicit +bytecode column) and the artifact's content have to move together, and the +choice between "tracked-only census" and "live inventory with an `ignored` +column" is a design decision about what the census is *for*. Graded here, not +edited — the `Q111`/`Q113` convention. The instrument above is the handover: it +names the 20 rows, the rev, and the exact count move any fix must produce. + +## Not claimed + +- That the census is wrong as a *live inventory*. It is exact: 574 of 574 + tracked rows match their blobs at the rev with zero drift. +- That the committed check was broken when it was written. It read rc 0 on the + host that generated the CSV; the defect is that its subject is host-local, + not that it misreports what it reads. +- That the 20 `.pyc` rows are the only source of instability. They are the only + ones present at `78f1139`; a `.pytest_cache/`, an editor swap file, or any + lane's new artifact would enter the same population the same way. +- That `CHECK-ledger-dir-census` is red at HEAD. In the worktree it read rc 0 + at `78f1139`; in a clean clone of that rev it read rc 1. +- That this audit is a gate leg. It is not in `tools/verify.py`; nothing runs + `CHECK-q140` but a reader or a tick that names it. +- That `wizard-work/ledger/` is the only directory censused this way. This item + graded one artifact; the `os.walk`-over-the-worktree pattern is what to grep + for, and no census of that pattern was counted here. diff --git a/wizard-work/volt/q140/audit.py b/wizard-work/volt/q140/audit.py new file mode 100644 index 0000000..b6dba5b --- /dev/null +++ b/wizard-work/volt/q140/audit.py @@ -0,0 +1,310 @@ +#!/usr/bin/env python3 +"""audit.py -- Q140: population audit of a directory census against a pinned rev. + +The ledger lane's `2026-10-31-ledger-dir-census.csv` records +(rel_path, size_bytes, sha256) for every file it finds under +wizard-work/ledger, and `verify_ledger_dir_census.py` re-derives that list by +walking the *live worktree* and comparing row for row. That population is a +function of the host: it contains ignored `__pycache__/*.pyc` files (which the +tree's own .gitignore calls "never content a reader can review, diff or +re-run") and it changes whenever any Python run recompiles, or any lane writes +the CSV again. + +This tool answers the question the census never asked, deterministically: + + of the rows in the census at rev R, which ones name a blob that actually + exists in the repository at R, and which name a file that exists only on + the machine that generated it? + +For every census row it reads the CSV at `:` (never the worktree) and +classifies it: + + TRACKED_MATCH path is a blob at rev R, same size and same sha256 + TRACKED_DRIFT path is a blob at rev R, size or sha256 differs + UNTRACKED_IGNORED path is not at R and is covered by the tree's ignore rule + UNTRACKED_OTHER path is not at R and is not ignored (an orphan row) + +and it counts the reverse direction: + + MISSING a blob at rev R under the censused directory with no row + +`--check` compares every count against PIN (below) and exits 1 on any move, so +a stale pin and a drifted census are both visible. Exit codes: 0 ok, 1 pin +fault, 2 could not grade. + +Usage: + python3 audit.py --repo . --rev --check + python3 audit.py --repo . --rev # report only + python3 audit.py --repo . --rev --json + python3 audit.py --repo . --rev --csv-file /tmp/mutant.csv # controls +""" +import argparse +import csv +import hashlib +import io +import json +import os +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +DEFAULT_REPO = os.path.abspath(os.path.join(HERE, "..", "..", "..")) +DEFAULT_CSV = "wizard-work/ledger/2026-10-31-ledger-dir-census.csv" +CENSUS_DIR = "wizard-work/ledger" + +# The pin: every count this tool prints for the rev it was measured at. +# Moving it is a deliberate edit made with the commit that moved the census. +PIN = { + "rev": "78f1139dd681f1b41c69463103a6c05a13490282", + "csv_blob": "a27e0193f7a1b8a888eeef66f56677503934cf9a", + "csv_sha256": "e52e907b14a8f0d617a810ff04d6830aa1ec5aa9ab592d9d159e24ea2ddf7aa5", + "rows": 594, + "tracked_in_population": 575, + "tracked_match": 574, + "tracked_drift": 0, + "untracked_ignored": 20, + "untracked_other": 0, + "missing": 0, # the census CSV itself is excluded on both sides +} + + +class Ungradable(Exception): + """Raised when the rev or the CSV cannot be read: exit 2, never exit 1.""" + + +def git(repo, *args, stdin=None, check=True): + proc = subprocess.run(["git", "-C", repo, *args], input=stdin, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + if check and proc.returncode != 0: + raise Ungradable("git %s -> rc %d: %s" % ( + " ".join(args), proc.returncode, + proc.stderr.decode("utf-8", "replace").strip()[:400])) + return proc + + +def resolve_rev(repo, rev): + proc = git(repo, "rev-parse", "--verify", rev + "^{commit}", check=False) + if proc.returncode != 0: + raise Ungradable("rev %r is not a commit in %s" % (rev, repo)) + return proc.stdout.decode().strip() + + +def read_csv_at_rev(repo, rev, rel): + proc = git(repo, "rev-parse", "--verify", "%s:%s" % (rev, rel), check=False) + if proc.returncode != 0: + raise Ungradable("%s has no %s" % (rev, rel)) + blob = proc.stdout.decode().strip() + text = git(repo, "cat-file", "blob", blob).stdout.decode("utf-8") + return blob, text + + +def read_csv_file(path): + with open(path, encoding="utf-8", newline="") as f: + return f.read() + + +def parse_rows(text): + r = csv.reader(io.StringIO(text, newline="")) + try: + header = next(r) + except StopIteration: + raise Ungradable("census csv is empty") + if header != ["rel_path", "size_bytes", "sha256"]: + raise Ungradable("census header is %r, expected rel_path,size_bytes,sha256" + % (header,)) + rows = [] + for lineno, row in enumerate(r, start=2): + if not row: + continue + if len(row) != 3: + raise Ungradable("census line %d has %d fields" % (lineno, len(row))) + rows.append(tuple(row)) + return rows + + +def tracked_population(repo, rev, directory): + """path -> (size, blob_oid, sha256) for every blob at rev under directory.""" + out = git(repo, "ls-tree", "-r", "-l", "--full-tree", rev, "--", directory).stdout + entries = [] + for line in out.decode("utf-8").splitlines(): + if not line.strip(): + continue + meta, path = line.split("\t", 1) + parts = meta.split() + if len(parts) != 4: + raise Ungradable("unexpected ls-tree row: %r" % (line,)) + mode, typ, oid, size = parts + if typ != "blob": + continue + entries.append((path, int(size), oid)) + + # one cat-file --batch for the whole population, not one subprocess per blob + stdin = "".join(oid + "\n" for _, _, oid in entries).encode() + proc = subprocess.run(["git", "-C", repo, "cat-file", "--batch"], + input=stdin, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + if proc.returncode != 0: + raise Ungradable("git cat-file --batch -> rc %d: %s" % ( + proc.returncode, proc.stderr.decode("utf-8", "replace")[:400])) + buf = proc.stdout + pos = 0 + digests = [] + for _ in entries: + nl = buf.index(b"\n", pos) + header = buf[pos:nl].decode("utf-8", "replace").split() + pos = nl + 1 + if len(header) < 3 or header[1] != "blob": + raise Ungradable("unexpected batch header: %r" % (header,)) + size = int(header[2]) + content = buf[pos:pos + size] + pos += size + 1 # trailing newline + digests.append(hashlib.sha256(content).hexdigest()) + pop = {} + for (path, size, _oid), digest in zip(entries, digests): + pop[path] = (size, digest) + return pop + + +def ignored_set(repo, paths): + """Measured with the tree's own rule; pattern-classified if git cannot read it.""" + if not paths: + return set() + proc = git(repo, "check-ignore", "--stdin", check=False, stdin=( + "".join(p + "\n" for p in paths)).encode()) + if proc.returncode not in (0, 1): + return None + return set(l for l in proc.stdout.decode("utf-8", "replace").splitlines() if l) + + +def classify(rows, pop, csv_rel): + counts = {"rows": len(rows), "tracked_in_population": len(pop), + "tracked_match": 0, "tracked_drift": 0, + "untracked_ignored": 0, "untracked_other": 0, "missing": 0} + drift, untracked_ignored, untracked_other = [], [], [] + seen = set() + for path, size, sha in rows: + seen.add(path) + if path in pop: + tsize, tdigest = pop[path] + if int(size) == tsize and sha.strip().lower() == tdigest: + counts["tracked_match"] += 1 + else: + counts["tracked_drift"] += 1 + drift.append({"path": path, "census": [int(size), sha], + "rev": [tsize, tdigest]}) + elif "__pycache__/" in path or path.endswith((".pyc", ".pyo")): + counts["untracked_ignored"] += 1 + untracked_ignored.append(path) + else: + counts["untracked_other"] += 1 + untracked_other.append(path) + missing = sorted(p for p in pop if p not in seen and p != csv_rel) + counts["missing"] = len(missing) + return counts, {"drift": drift, "untracked_ignored": untracked_ignored, + "untracked_other": untracked_other, "missing": missing} + + +def faults(counts, expected): + out = [] + for key in ("rows", "tracked_in_population", "tracked_match", + "tracked_drift", "untracked_ignored", "untracked_other", + "missing"): + if counts[key] != expected[key]: + out.append("count %s moved %d -> %d" % (key, expected[key], counts[key])) + return out + + +def main(argv=None): + ap = argparse.ArgumentParser(add_help=True, + description="population audit of a directory census") + ap.add_argument("--repo", default=DEFAULT_REPO) + ap.add_argument("--rev", default="HEAD") + ap.add_argument("--csv", default=DEFAULT_CSV, help="census path relative to repo") + ap.add_argument("--csv-file", default=None, + help="read rows from this file instead of : (controls)") + ap.add_argument("--dir", default=CENSUS_DIR, help="population directory at rev") + ap.add_argument("--check", action="store_true", help="compare counts against PIN") + ap.add_argument("--json", action="store_true", help="emit machine-readable result") + args = ap.parse_args(argv) + + repo = os.path.abspath(args.repo) + try: + rev = resolve_rev(repo, args.rev) + if args.csv_file: + blob, text = "(worktree)", read_csv_file(args.csv_file) + else: + blob, text = read_csv_at_rev(repo, rev, args.csv) + rows = parse_rows(text) + pop = tracked_population(repo, rev, args.dir) + counts, detail = classify(rows, pop, args.csv) + except Ungradable as exc: + print("UNGRADABLE: %s" % exc) + return 2 + + ignored = ignored_set(repo, detail["untracked_ignored"] + detail["untracked_other"]) + rules_agree = None + if ignored is not None: + by_pattern = set(detail["untracked_ignored"]) + rules_agree = by_pattern == ignored + + result = {"repo": repo, "rev": rev, "csv": args.csv, "csv_blob": blob, + "csv_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(), + "counts": counts, "detail": detail, + "ignore_rule_agrees": rules_agree, + "population_is_tracked_only": counts["rows"] == counts["tracked_in_population"] - 1 + and counts["untracked_ignored"] == 0 and counts["untracked_other"] == 0} + + if args.json: + print(json.dumps(result, indent=2, sort_keys=True)) + else: + print("rev %s csv %s" % (rev[:12], args.csv)) + print(" population tracked at rev: %d" % counts["tracked_in_population"]) + print(" rows in census: %d" % counts["rows"]) + print(" tracked match: %d" % counts["tracked_match"]) + print(" tracked drift: %d" % counts["tracked_drift"]) + print(" untracked (ignored): %d" % counts["untracked_ignored"]) + print(" untracked (not ignored): %d" % counts["untracked_other"]) + print(" tracked blobs with no row: %d" % counts["missing"]) + if rules_agree is not None: + print(" tree ignore rule agrees: %s" % ("yes" if rules_agree else "NO")) + if detail["untracked_other"]: + for p in detail["untracked_other"][:10]: + print(" orphan row: %s" % p) + if detail["untracked_ignored"]: + print(" ignored rows (first 3):") + for p in detail["untracked_ignored"][:3]: + print(" %s" % p) + if detail["missing"]: + print(" unrowed blobs (first 3):") + for p in detail["missing"][:3]: + print(" %s" % p) + print(" population is tracked-only: %s" % ( + "yes" if result["population_is_tracked_only"] else "NO")) + + if not args.check: + return 0 + + expected = dict(PIN) + expected_rev = expected.pop("rev") + faults_found = faults(counts, expected) + if not args.csv_file: + if rev != expected_rev: + faults_found.insert(0, "rev moved %s -> %s" % (expected_rev[:12], rev[:12])) + if blob != expected.pop("csv_blob"): + faults_found.insert(1, "csv blob moved %s -> %s" % ( + PIN["csv_blob"][:12], blob[:12])) + if faults_found: + print("FAULT: %d" % len(faults_found)) + for f in faults_found: + print(" " + f) + return 1 + print("CHECK OK: %d rows = %d tracked + %d ignored (%d orphan), %d unrowed, " + "counts match the pin" % ( + counts["rows"], counts["tracked_match"], + counts["untracked_ignored"], counts["untracked_other"], + counts["missing"])) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/wizard-work/volt/q140/selftest.py b/wizard-work/volt/q140/selftest.py new file mode 100644 index 0000000..bdce66a --- /dev/null +++ b/wizard-work/volt/q140/selftest.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +"""selftest.py -- Q140: nine legs over audit.py, run by CHECK-q140. + +Every leg drives the instrument through its command line, so a leg cannot pass +because it shares the instrument's bugs: + + 1 pin the pinned rev reads rc 0 with the pinned counts + 2 rev-move a different rev reads rc 1 and names the rev, not just counts + 3 worktree-free a --shared --no-checkout clone of the repo reads identical counts + 4 mutant-drift one sha character changed in a tracked row -> rc 1 + 5 mutant-drop one tracked row deleted -> rc 1 + 6 mutant-orphan one row appended for a path that is not at the rev -> rc 1 + +Legs 4-6 are why the pin is not vacuous: the same instrument that prints OK on +the real census prints FAULT with the predicted count move on a mutated one. +""" +import json +import os +import shutil +import subprocess +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +AUDIT = os.path.join(HERE, "audit.py") +REPO = os.path.abspath(os.path.join(HERE, "..", "..", "..")) +sys.path.insert(0, HERE) +import audit # noqa: E402 (pin and csv path only; no derivation is reused) + +CSV = audit.DEFAULT_CSV +PIN_REV = audit.PIN["rev"] +OTHER_REV = "2e97e4d46242224cf56f3d24f3771054bad22eac" # census commit before the rebuild +FAULTS = [] +LEGS = 0 + + +def run(args): + proc = subprocess.run([sys.executable, AUDIT] + args, cwd=REPO, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + return proc.returncode, proc.stdout.decode("utf-8", "replace") + + +def leg(name, rc, out, want_rc, must=()): + global LEGS + LEGS += 1 + bad = [] + if rc != want_rc: + bad.append("rc %d, wanted %d" % (rc, want_rc)) + for needle in must: + if needle not in out: + bad.append("missing %r" % needle) + if bad: + FAULTS.append(name) + print("FAIL %-16s %s" % (name, "; ".join(bad))) + for line in out.strip().splitlines()[-6:]: + print(" | " + line) + return False + print("PASS %-16s rc=%d %s" % (name, rc, must[-1] if must else "")) + return True + + +def git(*args, stdin=None): + proc = subprocess.run(["git", "-C", REPO, *args], input=stdin, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + if proc.returncode != 0: + raise SystemExit("git %s rc %d: %s" % (" ".join(args), proc.returncode, + proc.stderr.decode()[:300])) + return proc.stdout + + +def census_text(): + blob = git("rev-parse", "%s:%s" % (PIN_REV, CSV)).decode().strip() + return git("cat-file", "blob", blob).decode("utf-8") + + +def is_tracked(path): + return subprocess.run(["git", "-C", REPO, "cat-file", "-e", + "%s:%s" % (PIN_REV, path)], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL).returncode == 0 + + +def first_tracked_row(lines): + for i, line in enumerate(lines): + if not line.strip(): + continue + path = line.split(",", 1)[0] + if is_tracked(path): + return i + raise SystemExit("no tracked row in the census") + + +def write_mutant(dirpath, text): + path = os.path.join(dirpath, "mutant-census.csv") + with open(path, "w", encoding="utf-8", newline="") as f: + f.write(text) + return path + + +def main(): + scratch = tempfile.mkdtemp(prefix="q140-", dir=os.environ.get("TMPDIR") or "/tmp") + try: + # 1 -- the pinned reading + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--check"]) + leg("pin", rc, out, 0, ["CHECK OK: 594 rows = 574 tracked + 20 ignored", + "0 unrowed", "counts match the pin"]) + + # 2 -- the pin compares the rev, so a moved rev is a named fault + rc, out = run(["--repo", REPO, "--rev", OTHER_REV, "--check"]) + leg("rev-move", rc, out, 1, ["rev moved %s" % PIN_REV[:12]]) + + # 3 -- the derivation is a function of git objects, not of the worktree + global LEGS + LEGS += 1 + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--json"]) + host = json.loads(out)["counts"] + clone = os.path.join(scratch, "clone.git") + subprocess.run(["git", "clone", "--quiet", "--shared", "--no-checkout", + REPO, clone], check=True, + stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) + rc2, out2 = run(["--repo", clone, "--rev", PIN_REV, "--json"]) + ok = rc2 == 0 + try: + theirs = json.loads(out2)["counts"] + except Exception: + theirs = None + same = theirs == host + if not (rc == 0 and ok and same): + FAULTS.append("worktree-free") + print("FAIL worktree-free rc host=%d clone=%d same_counts=%s" + % (rc, rc2, same)) + for line in out2.strip().splitlines()[-4:]: + print(" | " + line) + else: + print("PASS worktree-free rc=0 clone counts identical to host: %s" + % json.dumps(host, sort_keys=True)) + + # 4 -- mutation: one sha character in a tracked row + lines = census_text().splitlines(keepends=True) + i = first_tracked_row(lines) + parts = lines[i].rstrip("\r\n").split(",") + parts[2] = ("0" if parts[2][0] != "0" else "1") + parts[2][1:] + lines[i] = ",".join(parts) + "\n" + path = write_mutant(scratch, "".join(lines)) + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--csv-file", path]) + leg("mutant-drift", rc, out, 0, ["tracked drift: 1"]) + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--csv-file", path, "--check"]) + leg("mutant-drift-check", rc, out, 1, ["count tracked_drift moved 0 -> 1"]) + + # 5 -- mutation: a tracked row removed + lines = census_text().splitlines(keepends=True) + i = first_tracked_row(lines) + del lines[i] + path = write_mutant(scratch, "".join(lines)) + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--csv-file", path]) + leg("mutant-drop", rc, out, 0, ["tracked blobs with no row: 1"]) + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--csv-file", path, "--check"]) + leg("mutant-drop-check", rc, out, 1, ["count missing moved 0 -> 1"]) + + # 6 -- mutation: a row for a path that is not at the rev + ghost = "wizard-work/ledger/ghost-q140.md,3," + "0" * 64 + "\n" + path = write_mutant(scratch, census_text().rstrip("\r\n") + "\n" + ghost) + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--csv-file", path]) + leg("mutant-orphan", rc, out, 0, ["untracked (not ignored): 1"]) + rc, out = run(["--repo", REPO, "--rev", PIN_REV, "--csv-file", path, "--check"]) + leg("mutant-orphan-check", rc, out, 1, ["count untracked_other moved 0 -> 1"]) + finally: + shutil.rmtree(scratch, ignore_errors=True) + + print("\n%d leg(s), %d failed" % (LEGS, len(FAULTS))) + if FAULTS: + print("FAILED: " + ", ".join(FAULTS)) + return 1 + print("Q140 OK") + return 0 + + +if __name__ == "__main__": + sys.exit(main())