troupe-standup/recoveries/helm-read-path-guard.md

Q47 — the read path: a subject the interpreter can run must neither crash the gate nor be falsely refused

Merlin, 2026-10-13. Helm lane. Scratch tree /root/crucible/q47/; nothing deployed, no cp into /root/.helm/bin/, decisions/ and inbox/ untouched, no another-party file edited, no secret. Item opened by the pipe on reply 30's C1 (2026-10-13 01:58Z), measured there on the shipped Q46 gate.

The defect. Two directions, one cause. (a) A subject with # -*- coding: latin-1 -*- and one latin-1 byte — bytes-compile OK, python3 <fixture> rc 0, so the interpreter runs it — crashed the gate at COLLECTOR.read_text(): UTF-8 decode raises UnicodeDecodeError, rc 1, a traceback, no dated refusal. (b) The real collector with a UTF-8 BOM — bytes-compile OK, because CPython strips a BOM when it reads a source file — was falsely refused by the Q46 guard: the decoded text still carries U+FEFF and compile() calls it an invalid non-printable character. The cause in both cases: the gate reads a decoded str, the interpreter compiles the file's bytes.

1. The pair

Collector byte-identical to Q46's (4e45021d80395d3f…, 720 lines / 29,954 B) — measured by sha256, not asserted. Every change is in the gate.

	Q46	Q47
sha256	ae69d5dd735b6f3cfe82b3682b02acf4557b2939d104b9f15b81cffaeef016d4	76f2292b9a6cad08505956787cf05e65e3edaf7e0f23f73f4aed2e59a62866c1
bytes	110,509	115,182
lines	2,032	2,124

out/DIFF-vs-q46.txt (fresh diff -u): 130 lines = 2 file headers + 4 hunk headers + 96 added + 4 removed + 24 context, and 2,032 + 96 − 4 = 2,124 exactly. The decomposition convention is the lane rule: added = ^+ lines minus the +++ header, removed = ^- minus ---.

2. What changed — one reader, four read sites, one refusal

1. _q47_subject_text(path) reads the subject the way CPython reads a source file: the file's bytes through tokenize.detect_encoding (BOM first, then the PEP 263 cookie, then utf-8), decoding with utf-8-sig when a BOM is present so the BOM is stripped exactly as the interpreter strips it. A non-file path returns ("", "utf-8"), the shape Q35's marker check expects. An unreadable cookie or bytes that do not decode raise _Q47ReadError.
2. The dated exit-2 refusal sits at the same point as Q46's guard — after Q35's source precondition, before the admission probe, before any leg, before any sandbox is created — with the live snapshot read before and after and the delta printed. A file that cannot be read the CPython way cannot be run by the interpreter either, so refusal is the honest shape.
3. All four subject-source reads are routed through the one reader: the Q35 precondition's _q35_src (line 210), the admission's _q39_src_text (was line 361), the G8 source read (was 1704), and the G8 source-guard check (was 1971). No read_text() on the subject survives.
4. One Q47 NOTE on stderr, never stdout, and only when the subject's encoding is not plain utf-8 — so a green pair's stdout shape is unchanged and the positive side of a read-path fix is visible rather than inferred (the Q38 precedent).
5. The Q46 parse guard, the Q45 redirect, every check() and the suite count are untouched: the guard compiles the same _q35_src it always did — it now compiles what the interpreter would compile. Green stays 169 passed / 0 failed; the guard registers no check.

3. The fixtures, and the interpreter's own verdict (measured, not assumed)

make_fixtures.py builds the three from the real collector (digest asserted against 4e45021d…), each construction asserting its own counts:

fixture	construction	python3 <fixture> --skip-tests (the ground truth)
collector-latin1.py (30,028 B / 6ce8bf90292d83f1…)	cookie on line 1, one 0xe9 in a trailing comment	rc 2 — the destination rule refuses; the file runs
collector-bom.py (29,957 B / d14906c868e24997…)	UTF-8 BOM prepended, no cookie	rc 2 — the destination rule refuses; the file runs
collector-undecodable.py (30,006 B / 47fc3580fdfe8308…)	one 0xe9 in a trailing comment, no cookie	rc 1 — SyntaxError: Non-UTF-8 code starting with '\xe9'

4. The discriminator — fifteen invocations, five gates

run_negatives_q47.py → out/Q47-NEGATIVES.json (57,247 B, 21 rows, generated_at 2026-10-13T03:46:55.002706+00:00); verdict in out/Q47-VERDICT.json (recomputed from the rows by recompute_verdict.py — see §7 defect 1). Legacy gates digest-verified against the Q46 record (9b09f328…, c22601ec…, 18580c10…, 2bbca218…).

fixture	Q39	Q43	Q44	Q45	Q47 (shipped)
collector-latin1.py	rc 1 — UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 30026 (0.16 s, no sandbox)	rc 1 — same (0.13 s)	rc 1 — same (0.15 s)	rc 1 — same (0.14 s)	rc 0, 169 passed / 0 failed, VERDICT: ALL GATES PASS, 13,312 B stdout, 12.44 s, Q47 NOTE … (iso-8859-1), no leftover
collector-bom.py	rc 0, 164 passed, ALL GATES PASS, 11,398 B, 10.12 s — Q39 runs the BOM subject green (see §6)	rc 1 — SyntaxError: invalid non-printable character U+FEFF at the sandbox copy's import (\ufeff"""The Helm — P0 collector.), 575 B, legacy/q43/helm-legs-_sja2yko left (0.61 s)	rc 1 — AttributeError: 'NoneType' object has no attribute 'body' (_q39_tree.body on a BOM'd parse), 0 B, legacy/q44/helm-legs-sl1v6yx5 left (0.22 s)	rc 1 — SyntaxError: … U+FEFF at the sandbox copy, 575 B, legacy/q45/helm-legs-8qmmr11u left (0.66 s)	rc 0, 169 passed / 0 failed, ALL GATES PASS, 13,312 B, 12.22 s, Q47 NOTE … (utf-8-sig), no leftover
collector-undecodable.py	rc 1 — UnicodeDecodeError … position 30004 (0.14 s)	rc 1 — same (0.14 s)	rc 1 — same (0.19 s)	rc 1 — same (0.14 s)	rc 2, stdout 0 B (e3b0c44298fc1c149…), dated Q47 REFUSAL: … cannot be read the way CPython reads a source file - 'utf-8' codec can't decode byte 0xe9 in position 30004: invalid continuation byte - refused at 2026-10-13T03:47:32.685519+00:00, before the admission probe, before any leg, and before any sandbox is created. — no traceback, no leftover (0.13 s)

The item's four done-criteria, as measured: the latin-1 subject proceeds past the guard and runs the whole suite green; the BOM subject is not refused and runs green; a genuinely undecodable subject is a dated exit-2 refusal; the four pre-Q46 gates' rcs are measured on both fixtures above (rc 1 crashes — frozen controls, not patched); green still 169 passed / 0 failed; no sandbox left. live_delta = 0 and live_held_delta = 0 on all 21 rows.

5. Regression controls, all still refused

• q47-q46reg-noparse: the Q46 class re-measured under the Q47 gate — rc 2, the dated Q46 REFUSAL unchanged (invalid syntax (noparse-with-markers.py, line 13) (line 13, offset 12)). The read-path fix did not loosen the parse guard.
• q47-neg-n1…n5: all five Q39 negative fixtures rc 2, live delta 0 — the admission machinery is untouched. (n5's refusal line is the Q39 NOTE plus the leg's own refusing to write the live snapshot … without --deploy, rc 2, 575 B.)

6. The green pair, and the stdout identity

run_green_q47.py → out/Q47-GREEN.json (2,725 B), logs out/q47-green-{1..5}.{out,err}: five consecutive rc 0, 169 passed / 0 failed, VERDICT: ALL GATES PASS, 13,312 B stdout / 425 B stderr each, 12.03–12.58 s, live delta 0, held delta 0, no leftover. Green stdout digests: 21920260cade855a… / 580021754ed55d3d… / 49025d65cc3082e5… / c8155f076d214d24… / 1d2c7d69a8531d39….

out/q47-green-1.out vs out/../q46/out/q46-green-1.out: 184 lines each, exactly 4 differ, and all four are the randomized names — line 1 (/tmp/tmpkhag_ezo/ vs /tmp/tmpl46pgy2a/), line 10 (the q31-g8-deploy temp dir) and lines 183–184 (the two sandbox-path lines, which name the tree). So the Q47 gate's green output is byte-identical to Q46's but for the random names — the read path changed nothing the suite prints.

The Q39-bom row is a finding, not noise. The oldest gate runs the BOM subject green with 164 passed while Q43/Q44/Q45 crash on it. The mechanism is visible in its own report: parse line -: the subject does not parse: invalid non-printable character U+FEFF — Q39 records the parse failure and continues, because Q39 never compiles the subject's text; it only imports the file (where CPython strips the BOM) and runs the probe copy as a subprocess (whose SyntaxError the Q46-era crash detector cannot see). The read-path class is therefore a property of the gates that compile the subject's text — Q43/Q45's sandbox-copy import, Q44's ast — not of subject handling generally. The BOM subject is runnable by the interpreter and green on the oldest gate; it is Q43–Q46's text compilation that made it dangerous.

7. Limits and defects — mine first, logged not deleted

1. Harness verdict bug (mine). row_for() was passed READPATH_FIXTURES (which carry .py) while the discriminator loop names rows with the suffix stripped, so the verdict's generator raised StopIteration → RuntimeError after the rows JSON was written. The rows are all on disk; the verdict was recomputed from them by recompute_verdict.py, and run_negatives_q47.py is patched (the strip) so the file on disk is right. No gate invocation was re-run.
2. Vacuous fixture-dest census (mine). The negative fixtures' destination literals still name the q46 tree — cp -a inherited fixtures already retargeted by q46's setup_fixtures.py — so this run's fixture_dest_census() (which watches q47/fixtures) was empty on every row. Evidence no cross-tree write happened: ls /root/crucible/q46/fixtures shows no n1-dest…n5-dest/dep-dest directories, and each negative row's stderr shows the leg refusing its own destination. The right fix was to re-run setup_fixtures.py retargeted into q47/fixtures.
3. The copies are UTF-8 by construction (recorded limit). The probe copy and the legs' sandbox copy prepend their own header lines, which push the subject's cookie past PEP 263's line-2 limit, so a cookie subject's non-ASCII bytes in executable strings would be re-encoded as UTF-8 in the copies. The fixture therefore carries its single latin-1 byte in a comment only. A copy faithful in the subject's own encoding would need the cookie hoisted — not built.
4. The guard is static (carried from Q46): runtime-generated code, or a source produced by an import hook, is out of every static guard's reach.
5. tokenize.detect_encoding reads the first two lines only — faithful, not a limit: CPython does not honour a cookie past line 2 either.
6. Byte counts, hashes and timings are dated readings (2026-10-13 ~03:30–04:00Z).

8. Live state, and the §4 consequence

Censused on (bytes, sha256, mtime_ns, ctime_ns) before and after all 26 invocations (21 harness rows + 5 green runs), delta 0 on every field: snapshot /root/.helm/data/snapshot.json 14,039 B / aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d / mtime_ns 1791479579475780303 / ctime_ns 1791789490146606356; held pin /root/.helm/data/held/stackchain-home.html.sha256 120 B / 5b6d70716153c3f4… / mtime_ns 1791478687099553071 / ctime_ns 1791846636217687791 (that ctime is the Q45 restore's trace, as recorded there). Deployed bin untouched: helm-selftest.py 33,647 B / 7ed1bca8f9da624f…, helm-collect.py ee0b36a82f23fcf3…, helm-render.py 5c67473d89283c3d…, helm-validate.py 12570786ba703787…, summarize.py 7386999867789694…. Final tree: 0 sandboxes.

The adoption pair is now collector 4e45021d… + gate 76f2292b… (superseding ae69d5dd…). Q40's finding is unchanged and is why the cp matters: the deployed gate 7ed1bca8… reads HELM_COLLECTOR zero times, so it has no subject-selection path and none of Q31–Q47's protection is in production until the pair lands. The cp stays Alexander's.

9. Q7, measured

After this item the Helm lane's executable set is empty again: the adoption cp, Q9 (checksum handover), Q13/Q19 and Q42 are human- or counterparty-gated. Merlin's Q45 §9 follow-on — resolve every path the subject declares at runtime, not only the literals it declares statically — stays open and unclaimed beside the adoption. No new item opened.