troupe-standup/recoveries/helm-pin-pill-staleness.md

The pin pill that outlived its reading

Helm lane · Run 29 · 2026-10-14 · Merlin. Scratch tree /root/crucible/q59/; nothing deployed, no cp into /root/.helm/bin/, no write to the live document root, no --deploy anywhere, decisions/ and inbox/ untouched, no another-party file edited, no secret printed. Clock: R29-MEASURE.json generated_at 2026-10-14T15:53:07+00:00, R29-GATE.json generated_at 2026-10-14T15:59:55+00:00 — both read from the artifacts, not approximated.

Unit taken. No unclaimed queue item was executable and none was invented: Q42 is zephyrus-sparkborn's (closed), Q52 Mercury's (closed), Q53 Mercury's (open), Q9/Q13/Q19 gated on Alexander, and the queue's own hygiene ledger records the Helm executable set empty. The unit is the half of Run 28's finding the fix does not reach — named by the control plane's own re-derivation of G10 (WORK-QUEUE.md, 2026-10-14, commit 7ec2172: "the match/drift pill still reads the frozen manifest_match field (renderer :323/:326), so a re-rendered page would show a green 'match' beside 'as of … not re-checked since' while the pin provably no longer matches the live body — internally inconsistent, and the false claim survives"). Re-measured here before any edit; confirmed.

---

1. The finding, measured

/root/crucible/q59/measure_r29.py → out/R29-MEASURE.json (generated_at 2026-10-14T15:53:07+00:00). Read-only; the only writes are its own scratch dir.

The pin, and the body it is pinned against, today.

object	value
held pin /root/.helm/data/held/stackchain-home.html.sha256	120 B, sha256 5b6d70716153c3f4bd82045d5d71edf88f793a240ce1275c3dfca5759d0ee297, mtime_ns 1791478687099553071, ctime_ns 1791957230761646950
pin text	f27326e8b13d1dec9eb158fe40e5091f79dcaf73b47312c4e7d3958ba1d95d18 stackchain-home.html pinned 2026-10-08T16:58:07.100Z
live body https://www.stackchainmagazine.net/ (fetched this run)	HTTP 200 / 217,163 B / sha256 c34ae2c245117ca73ba5720b2c22b6fab689840070ff68cda7bffe3e723986d4
pin_matches_live_now	false

What the renderer is handed. Snapshot /root/.helm/data/snapshot.json 14,039 B / sha256 aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d / mtime_ns 1791479579475780303, generated_at 2026-10-08T17:12:59.475Z; its stackchain_site block reads manifest_match: true, live_body_sha256: f27326e8b13d1dec…, evidence.observed_at: 2026-10-08T17:12:59.458Z, http_status: 200, rtt_seconds: 0.545. The collector's own threshold for this metric is 3600 s (STALENESS_THRESHOLDS_S["stackchain_site"], read from the collector module, not assumed).

The renderer's own output, at two pinned clocks, before any edit. The deployed renderer /root/.helm/bin/helm-render.py 21,766 B / sha256 5c67473d89283c3d4434928dc96d45ff48a41d8f443ecc3c98e9236be8b597f9 / mtime_ns 1791717163756240575, run through its own CLI:

$ python3 /root/.helm/bin/helm-render.py --out <scratch> --as-of 2026-10-14T16:00:00.000Z
rendered -> … (11806 bytes)
render clock 2026-10-14T16:00:00+00:00  snapshot age 5d 22h 47m
verdict at render: 0 green / 6 not green  (5 stale, 1 unreachable)
  stackchain_site    state=OK           freshness@render=STALE

and the emitted cell, verbatim:

<div class="kv">live body sha256 <span class="mono">f27326e8b13d1dec…</span></div>
<div class="kv">held copy <span style="color:#22c55e">match</span> (pinned vs live)</div>
<div class="kv dim">as of 2026-10-08T17:12:59.458Z · 5d 22h 47m ago · not re-checked since</div>

So a render today prints green match beside not re-checked since, in a page whose own verdict pill reads 0 green / 6 not green — while the pin is, measured this run, false. The frozen manifest_match field is not merely old; it asserts an equality the live body no longer satisfies.

The control plane's line numbers check out and are cited with their content, not from memory: helm-render.py:323 is match_color = "#22c55e" if site.get("manifest_match") else "#ef4444" and :326 is match_label = "match" if site.get("manifest_match") else "drift" — both read the frozen field, while :327-329 builds the qualifier from evidence.observed_at and the freshness re-derived at render. The comment above them states the rule ("Say so, with its clock, rather than presenting it as a live tamper signal — the renderer cannot re-fetch") and the colour asserts the opposite of it.

The deployed page /var/www/alexander-whitestone/helm/index.html (9,865 B / sha256 427b7b54c69b18d270288910fc810732463129f48ec4bdd7dbc75cdb22997ce4 / mtime 2026-10-11T08:57:14.017Z, mtime_ns 1791709034017837233) carries neither the qualifier nor the pill's condition — as of false, not re-checked since false — which is Run 28's G10 row and stays red. The defect measured here is the one a re-render would publish, i.e. the one that stands between G10's remedy and an honest page.

2. Why the self-test could not see it

G10's four checks read the qualifier; none reads the pill. G5's manifest checks run on a synthetic fixture at a pinned clock. Nothing in the lineage asserted that the pill's verdict is a function of the reading's freshness — the exact shape of the Q34/Q40 law this lane keeps meeting: a gate that tests the producer cannot vouch for the claim in the artifact.

3. The fix

/root/crucible/q59/patch_render_r29.py → helm-render-g11.py. The deployed renderer is copied byte-for-byte (sha256sum asserted equal before the edit; the plain copy helm-render-plain.py is kept for the discriminator), then one edit: the pill's colour and label become a function of site_fresh alongside the frozen field.

    if match_ok and site_fresh == "FRESH":
        match_color, match_label = "#22c55e", "match"
    elif match_ok:
        match_color, match_label = STATE_COLORS["STALE"], "match (unverified)"
    else:
        match_color, match_label = "#ef4444", "drift"

	bytes	sha256	lines
deployed renderer (pre-image)	21,766	5c67473d89283c3d4434928dc96d45ff48a41d8f443ecc3c98e9236be8b597f9	497
q59/helm-render-plain.py (control)	21,766	identical	497
q59/helm-render-g11.py	22,735	1e82a1818c57cc9796855ac8ab9de006a3647748596126482ced5df3b5ad3f7a	513

Delta +969 B / +16 lines (the edit and its comment). Asserted by the patcher: the anchor line occurs exactly once; compile() on the source before any write; the trailing newline survives; SNAPSHOT, OUT_DEFAULT, if is_prod and not deploy: and STATE_COLORS["STALE"] each occur exactly once afterwards, so the destination gate and the snapshot binding are untouched.

The emitted bytes, both branches (out/R29-FINAL.json):

render	renderer	clock	bytes	sha256
stale branch	unpatched	--as-of 2026-10-14T16:00:00.000Z	11,806	3f48643d8e7bac77020674198e6be78a35f27a0fec71b400d718b52b88ca4ad7
stale branch	fixed	same	11,819	951a8a7645fe7ef42247de97d37ab10c99f89a9c05bff3fe8f36177456936485
fresh branch	unpatched	--as-of 2026-10-08T18:00:00.000Z	11,624	1eec213e9e363d36135123892d385e5961d170aa80fd7f60ed6aadca28360e3d
fresh branch	fixed	same	11,624	identical

The stale branch moves +13 B in exactly one span — first differing character at offset 8910, held copy <span style="color:#22c55e">match</span> → held copy <span style="color:#f59e0b">match (unverified)</span> (q59/diffcheck.py). The fresh branch is byte-identical, so the edit conditions the pill and does not touch the page's legitimate case.

4. The gate

/root/crucible/q59/make_gate_q59.py → helm-selftest-q59.py, four checks on Q45S10 + G10's gate (q58/helm-selftest-q58.py 153,469 B / 3b98b62fa5c7… / 2,916 lines / 131 registrations, digest re-asserted before the copy):

gate	bytes	sha256	lines	registrations
q58 (pre-image)	153,469	3b98b62fa5c7ea626091f6d0f63566de79818ae414bcffe323dd1ddd53956158	2,916	131
q59/helm-selftest-q59.py (G11)	161,122	125754b270bf6b0f55911f407fa9c36c6161b45a71e646fa36a70db096d7de93	3,053	135
q59/helm-selftest-q59-twin.py	161,124	2b0d38f3d7a9a2117ca7540eb21ba92cbfccfcb55190318638bf2135f4c7847b	3,053	135

+7,653 B / +137 lines / +4 registrations. The four rows:

1. The contract, on real data — the real snapshot through the fixed renderer's own CLI at a pinned stale clock: rc 0, the unqualified green match absent, the amber match (unverified) present, not re-checked since present.
2. The fresh branch is not over-refused — the same renderer at --as-of 2026-10-08T18:00:00.000Z (site FRESH: threshold 3600 s, reading 47 m old): the pill is still the green match, not re-checked since absent.
3. The discriminator, at renderer level — the unpatched renderer, the same snapshot, the same pinned stale clock, still prints the unqualified green match. Rows 1–2 therefore measure the edit, not the clock.
4. Read-only — bytes + sha256 + mtime_ns on four live objects (deployed page, snapshot, held pin, deployed renderer) around all three legs. Unlike G10's row this is four, because the renderer reads three of them; the live paths in this block are census targets, and row 4 is the assertion that none moved.

No check() was removed or renamed, so G10's four rows and every pre-existing row are untouched; 131 → 135 is purely additive.

5. The matrix

/root/crucible/q59/run_gate_q59.py → out/R29-GATE.json (generated_at 2026-10-14T15:59:55+00:00), every invocation with HELM_COLLECTOR=/root/crucible/q48/helm-collect.py (subject 4e45021d…, 29,954 B / 720 lines) and a live census before and after.

Prediction, stated before the run: shipped rc 1 / 184 passed / 1 failed, the one failure being G10's deployed-page row; twin rc 1 / 183 passed / 2 failed, that row plus G11 row 1. Met exactly.

cell	rc	suite	stdout	the G11 rows
shipped ×1	1	184 passed / 1 failed	15,667 B	4 × PASS
shipped ×2	1	184 passed / 1 failed	15,667 B	4 × PASS, identical to run 1
twin	1	183 passed / 2 failed	15,786 B	row 1 FAIL (rc=0 out_bytes=11806 out_sha256=3f48643d8e7bac77 unqualified_green=…), rows 2–4 PASS

The twin's only edit is asserted to be one line (make_twin_q59.py: differing line 2888, _g11_renderer repointed at the plain copy; both files 3,053 lines). The G11 rows are byte-identical across the two shipped runs; the two stdouts differ on 4 lines, all of them per-run mkdtemp tokens, and after normalising the three shapes actually present (/tmp/tmp*, /tmp/q31-g8-deploy-*, helm-legs-*) they are byte-identical — sha256 ea12512ef4f653ab85d75d64d626cace849fac3e0313ad4c6a0466b08aa95a16, residual differing lines 0 (q59/normstdout.py).

Live delta {} on bytes / sha256 / mtime_ns and ctime_ns for all four objects across all three invocations. Sandboxes 0/0 both ways (shallow glob and recursive rglob) before and after each.

6. Live, deployed, sandboxes, disk

Unchanged and re-read at the end of the run (out/R29-FINAL.json): snapshot 14,039 B / aa842b0fd88037f8… / mtime_ns 1791479579475780303 / ctime_ns 1791957230760646867; held pin 120 B / 5b6d70716153c3f4… / mtime_ns 1791478687099553071 / ctime_ns 1791957230761646950; deployed page 9,865 B / 427b7b54c69b18d2… / mtime_ns 1791709034017837233; deployed renderer 21,766 B / 5c67473d89283c3d…. Deployed /root/.helm/bin/ untouched — five digests re-read: helm-collect.py ee0b36a82f23…, helm-render.py 5c67473d8928…, helm-selftest.py 7ed1bca8f9da…, helm-validate.py 12570786ba70…, summarize.py 738699986778… — and grep -c HELM_COLLECTOR /root/.helm/bin/helm-selftest.py = 0, so Q40 stands and neither G10 nor G11 is in production. Sandboxes 0 in the q59 tree. Disk 45 G free of 154 (72 %), df -h /.

7. Limits, stated

• The renderer still cannot re-fetch (its own docstring: "zero external requests"), so the fix marks a stale reading unverified rather than re-measuring it. The pill's honesty is bounded by the collector's clock; only a re-collect can make the claim current again.
• The snapshot's frozen manifest_match: true is still wrong about today. Nothing in the renderer can correct a field; that is the collector's, and it is the re-collect in the sequence below.
• The deployed page remains the old generation, so G10's red row is unchanged and the public page still carries neither the qualifier nor the conditioned pill.
• The label match (unverified) is the renderer's own vocabulary — it adds no new term beyond the not re-checked since line already on the page.

8. Defects of mine, logged first

1. The stdout normalisation in run_gate_q59.py used two of the three mkdtemp shapes and reported stdout_identical_after_sandbox_normalisation: false with sha d9adcf2b…; the missed shape was /tmp/q31-g8-deploy-*. Corrected in final_r29.py, which re-writes the two fields in out/R29-GATE.json in place and records the correction beside them: true, sha ea12512ef4f653ab85d75d64d626cace849fac3e0313ad4c6a0466b08aa95a16, residual 0. The lane's own pattern-written-from-memory class, reproduced on a fresh artifact.
2. Nothing else. No figure, digest, line count, registration count or verdict in §1–§7 depends on the defect above; the prediction in §5 was written before the run and met.

9. Adoption — Alexander's, and now a sequence rather than a decision

The §4 cp pair has moved: collector 4e45021d80395d3f… + gate 125754b270bf6b0f55911f407fa9c36c6161b45a71e646fa36a70db096d7de93 (161,122 B / 3,053 lines / 135 registrations), superseding 3b98b62fa5c7… (G10), c56230e5a075… (Q45S10), cd43284ff009… (Q45S9), 78b290ecf2d9… (Q52), e4967da9… (Q51), a951dbe8… (Q50). And the renderer must be adopted before any re-render: helm-render-g11.py 1e82a1818c57… (22,735 B) replacing the deployed 5c67473d8928… (21,766 B). Without it, the re-render Run 28 asked for would deploy the internally inconsistent page this record measures. The order is therefore: adopt renderer → adopt collector + gate → re-collect → re-render → deploy, with the manifest_match pin refresh falling out of the re-collect. Beside it, unchanged: the Q13 publication call, Q9's checksum handover, Q19's onboarding.