troupe-standup/recoveries/helm-pin-pill-staleness.md

# The pin pill that outlived its reading

**Helm lane · Run 29 · 2026-10-14 · Merlin.** Scratch tree `/root/crucible/q59/`; nothing
deployed, no `cp` into `/root/.helm/bin/`, no write to the live document root, no
`--deploy` anywhere, `decisions/` and `inbox/` untouched, no another-party file
edited, no secret printed. Clock: `R29-MEASURE.json` `generated_at
2026-10-14T15:53:07+00:00`, `R29-GATE.json` `generated_at 2026-10-14T15:59:55+00:00`
— both read from the artifacts, not approximated.

**Unit taken.** No unclaimed queue item was executable and none was invented: Q42 is
zephyrus-sparkborn's (closed), Q52 Mercury's (closed), Q53 Mercury's (open), Q9/Q13/Q19
gated on Alexander, and the queue's own hygiene ledger records the Helm executable set
empty. The unit is the half of Run 28's finding the fix does not reach — named by the
control plane's own re-derivation of G10 (`WORK-QUEUE.md`, 2026-10-14, commit
`7ec2172`: *"the match/drift pill still reads the frozen `manifest_match` field
(renderer :323/:326), so a re-rendered page would show a green 'match' beside 'as of
… not re-checked since' while the pin provably no longer matches the live body —
internally inconsistent, and the false claim survives"*). Re-measured here before any
edit; confirmed.

---

## 1. The finding, measured

`/root/crucible/q59/measure_r29.py` → `out/R29-MEASURE.json`
(`generated_at 2026-10-14T15:53:07+00:00`). Read-only; the only writes are its own
scratch dir.

**The pin, and the body it is pinned against, today.**

| object | value |
|---|---|
| held pin `/root/.helm/data/held/stackchain-home.html.sha256` | 120 B, sha256 `5b6d70716153c3f4bd82045d5d71edf88f793a240ce1275c3dfca5759d0ee297`, mtime_ns `1791478687099553071`, ctime_ns `1791957230761646950` |
| pin text | `f27326e8b13d1dec9eb158fe40e5091f79dcaf73b47312c4e7d3958ba1d95d18  stackchain-home.html  pinned 2026-10-08T16:58:07.100Z` |
| live body `https://www.stackchainmagazine.net/` (fetched this run) | HTTP 200 / 217,163 B / sha256 `c34ae2c245117ca73ba5720b2c22b6fab689840070ff68cda7bffe3e723986d4` |
| **`pin_matches_live_now`** | **`false`** |

**What the renderer is handed.** Snapshot `/root/.helm/data/snapshot.json` 14,039 B /
sha256 `aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d` /
mtime_ns `1791479579475780303`, `generated_at 2026-10-08T17:12:59.475Z`; its
`stackchain_site` block reads `manifest_match: true`,
`live_body_sha256: f27326e8b13d1dec…`, `evidence.observed_at:
2026-10-08T17:12:59.458Z`, `http_status: 200`, `rtt_seconds: 0.545`. The collector's
own threshold for this metric is **3600 s** (`STALENESS_THRESHOLDS_S["stackchain_site"]`,
read from the collector module, not assumed).

**The renderer's own output, at two pinned clocks, before any edit.** The deployed
renderer `/root/.helm/bin/helm-render.py` 21,766 B / sha256
`5c67473d89283c3d4434928dc96d45ff48a41d8f443ecc3c98e9236be8b597f9` /
mtime_ns `1791717163756240575`, run through its own CLI:

```
$ python3 /root/.helm/bin/helm-render.py --out <scratch> --as-of 2026-10-14T16:00:00.000Z
rendered -> … (11806 bytes)
render clock 2026-10-14T16:00:00+00:00  snapshot age 5d 22h 47m
verdict at render: 0 green / 6 not green  (5 stale, 1 unreachable)
  stackchain_site    state=OK           freshness@render=STALE
```

and the emitted cell, verbatim:

```html
<div class="kv">live body sha256 <span class="mono">f27326e8b13d1dec…</span></div>
<div class="kv">held copy <span style="color:#22c55e">match</span> (pinned vs live)</div>
<div class="kv dim">as of 2026-10-08T17:12:59.458Z · 5d 22h 47m ago · not re-checked since</div>
```

So a render today prints **green `match`** beside **`not re-checked since`**, in a page
whose own verdict pill reads **`0 green / 6 not green`** — while the pin is, measured
this run, **false**. The frozen `manifest_match` field is not merely old; it asserts an
equality the live body no longer satisfies.

The control plane's line numbers check out and are cited with their content, not from
memory: `helm-render.py:323` is
`    match_color = "#22c55e" if site.get("manifest_match") else "#ef4444"` and `:326` is
`    match_label = "match" if site.get("manifest_match") else "drift"` — both read the
frozen field, while `:327-329` builds the qualifier from `evidence.observed_at` and the
freshness re-derived at render. The comment above them states the rule
(*"Say so, with its clock, rather than presenting it as a live tamper signal — the
renderer cannot re-fetch"*) and the colour asserts the opposite of it.

The deployed page `/var/www/alexander-whitestone/helm/index.html` (9,865 B / sha256
`427b7b54c69b18d270288910fc810732463129f48ec4bdd7dbc75cdb22997ce4` / mtime
2026-10-11T08:57:14.017Z, mtime_ns `1791709034017837233`) carries neither the qualifier
nor the pill's condition — `as of` **false**, `not re-checked since` **false** — which is
Run 28's G10 row and stays red. **The defect measured here is the one a re-render would
publish**, i.e. the one that stands between G10's remedy and an honest page.

## 2. Why the self-test could not see it

G10's four checks read the **qualifier**; none reads the **pill**. G5's manifest checks
run on a synthetic fixture at a pinned clock. Nothing in the lineage asserted that the
pill's verdict is a function of the reading's freshness — the exact shape of the Q34/Q40
law this lane keeps meeting: *a gate that tests the producer cannot vouch for the
claim in the artifact.*

## 3. The fix

`/root/crucible/q59/patch_render_r29.py` → `helm-render-g11.py`. The deployed renderer
is copied byte-for-byte (`sha256sum` asserted equal before the edit; the plain copy
`helm-render-plain.py` is kept for the discriminator), then one edit: the pill's colour
and label become a function of `site_fresh` alongside the frozen field.

```python
    if match_ok and site_fresh == "FRESH":
        match_color, match_label = "#22c55e", "match"
    elif match_ok:
        match_color, match_label = STATE_COLORS["STALE"], "match (unverified)"
    else:
        match_color, match_label = "#ef4444", "drift"
```

| | bytes | sha256 | lines |
|---|---|---|---|
| deployed renderer (pre-image) | 21,766 | `5c67473d89283c3d4434928dc96d45ff48a41d8f443ecc3c98e9236be8b597f9` | 497 |
| `q59/helm-render-plain.py` (control) | 21,766 | identical | 497 |
| **`q59/helm-render-g11.py`** | **22,735** | **`1e82a1818c57cc9796855ac8ab9de006a3647748596126482ced5df3b5ad3f7a`** | **513** |

Delta **+969 B / +16 lines** (the edit and its comment). Asserted by the patcher: the
anchor line occurs exactly once; `compile()` on the source before any write; the
trailing newline survives; `SNAPSHOT`, `OUT_DEFAULT`, `if is_prod and not deploy:` and
`STATE_COLORS["STALE"]` each occur exactly once afterwards, so the destination gate and
the snapshot binding are untouched.

**The emitted bytes, both branches** (`out/R29-FINAL.json`):

| render | renderer | clock | bytes | sha256 |
|---|---|---|---|---|
| stale branch | unpatched | `--as-of 2026-10-14T16:00:00.000Z` | 11,806 | `3f48643d8e7bac77020674198e6be78a35f27a0fec71b400d718b52b88ca4ad7` |
| stale branch | **fixed** | same | **11,819** | **`951a8a7645fe7ef42247de97d37ab10c99f89a9c05bff3fe8f36177456936485`** |
| fresh branch | unpatched | `--as-of 2026-10-08T18:00:00.000Z` | 11,624 | `1eec213e9e363d36135123892d385e5961d170aa80fd7f60ed6aadca28360e3d` |
| fresh branch | **fixed** | same | 11,624 | **identical** |

The stale branch moves **+13 B in exactly one span** — first differing character at
offset 8910, `held copy <span style="color:#22c55e">match</span>` →
`held copy <span style="color:#f59e0b">match (unverified)</span>`
(`q59/diffcheck.py`). The fresh branch is **byte-identical**, so the edit conditions the
pill and does not touch the page's legitimate case.

## 4. The gate

`/root/crucible/q59/make_gate_q59.py` → `helm-selftest-q59.py`, four checks on Q45S10 +
G10's gate (`q58/helm-selftest-q58.py` 153,469 B / `3b98b62fa5c7…` / 2,916 lines / **131**
registrations, digest re-asserted before the copy):

| gate | bytes | sha256 | lines | registrations |
|---|---|---|---|---|
| q58 (pre-image) | 153,469 | `3b98b62fa5c7ea626091f6d0f63566de79818ae414bcffe323dd1ddd53956158` | 2,916 | 131 |
| **`q59/helm-selftest-q59.py` (G11)** | **161,122** | **`125754b270bf6b0f55911f407fa9c36c6161b45a71e646fa36a70db096d7de93`** | **3,053** | **135** |
| `q59/helm-selftest-q59-twin.py` | 161,124 | `2b0d38f3d7a9a2117ca7540eb21ba92cbfccfcb55190318638bf2135f4c7847b` | 3,053 | 135 |

**+7,653 B / +137 lines / +4 registrations.** The four rows:

1. **The contract, on real data** — the real snapshot through the *fixed* renderer's own
   CLI at a pinned stale clock: rc 0, the unqualified green match **absent**, the
   amber `match (unverified)` **present**, `not re-checked since` present.
2. **The fresh branch is not over-refused** — the same renderer at
   `--as-of 2026-10-08T18:00:00.000Z` (site FRESH: threshold 3600 s, reading 47 m old):
   the pill is still the green match, `not re-checked since` absent.
3. **The discriminator, at renderer level** — the *unpatched* renderer, the same
   snapshot, the same pinned stale clock, still prints the unqualified green match. Rows
   1–2 therefore measure the edit, not the clock.
4. **Read-only** — bytes + sha256 + mtime_ns on **four** live objects (deployed page,
   snapshot, held pin, deployed renderer) around all three legs. Unlike G10's row this
   is four, because the renderer reads three of them; the live paths in this block are
   census targets, and row 4 is the assertion that none moved.

No `check()` was removed or renamed, so G10's four rows and every pre-existing row are
untouched; 131 → 135 is purely additive.

## 5. The matrix

`/root/crucible/q59/run_gate_q59.py` → `out/R29-GATE.json`
(`generated_at 2026-10-14T15:59:55+00:00`), every invocation with
`HELM_COLLECTOR=/root/crucible/q48/helm-collect.py` (subject `4e45021d…`, 29,954 B /
720 lines) and a live census before and after.

**Prediction, stated before the run:** *shipped rc 1 / 184 passed / 1 failed, the one
failure being G10's deployed-page row; twin rc 1 / 183 passed / 2 failed, that row plus
G11 row 1.* Met exactly.

| cell | rc | suite | stdout | the G11 rows |
|---|---|---|---|---|
| shipped ×1 | 1 | **184 passed / 1 failed** | 15,667 B | 4 × PASS |
| shipped ×2 | 1 | **184 passed / 1 failed** | 15,667 B | 4 × PASS, identical to run 1 |
| **twin** | 1 | **183 passed / 2 failed** | 15,786 B | row 1 **FAIL** (`rc=0 out_bytes=11806 out_sha256=3f48643d8e7bac77 unqualified_green=…`), rows 2–4 PASS |

The twin's only edit is asserted to be **one line** (`make_twin_q59.py`: differing line
2888, `_g11_renderer` repointed at the plain copy; both files 3,053 lines). The G11
rows are byte-identical across the two shipped runs; the two stdouts differ on **4
lines**, all of them per-run `mkdtemp` tokens, and after normalising the three shapes
actually present (`/tmp/tmp*`, `/tmp/q31-g8-deploy-*`, `helm-legs-*`) they are
byte-identical — sha256 **`ea12512ef4f653ab85d75d64d626cace849fac3e0313ad4c6a0466b08aa95a16`**,
**residual differing lines 0** (`q59/normstdout.py`).

Live delta **`{}`** on bytes / sha256 / mtime_ns **and ctime_ns** for all four objects
across all three invocations. Sandboxes **0/0 both ways** (shallow glob and recursive
`rglob`) before and after each.

## 6. Live, deployed, sandboxes, disk

Unchanged and re-read at the end of the run (`out/R29-FINAL.json`): snapshot 14,039 B /
`aa842b0fd88037f8…` / mtime_ns `1791479579475780303` / ctime_ns
`1791957230760646867`; held pin 120 B / `5b6d70716153c3f4…` / mtime_ns
`1791478687099553071` / ctime_ns `1791957230761646950`; deployed page 9,865 B /
`427b7b54c69b18d2…` / mtime_ns `1791709034017837233`; deployed renderer 21,766 B /
`5c67473d89283c3d…`. Deployed `/root/.helm/bin/` untouched — five digests re-read:
`helm-collect.py` `ee0b36a82f23…`, `helm-render.py` `5c67473d8928…`,
`helm-selftest.py` `7ed1bca8f9da…`, `helm-validate.py` `12570786ba70…`,
`summarize.py` `738699986778…` — and `grep -c HELM_COLLECTOR
/root/.helm/bin/helm-selftest.py` = **0**, so **Q40 stands and neither G10 nor G11 is in
production**. Sandboxes 0 in the q59 tree. Disk **45 G free of 154 (72 %)**, `df -h /`.

## 7. Limits, stated

- The renderer still cannot re-fetch (its own docstring: *"zero external requests"*), so
  the fix marks a stale reading **unverified** rather than re-measuring it. The pill's
  honesty is bounded by the collector's clock; only a re-collect can make the claim
  current again.
- The snapshot's frozen `manifest_match: true` is still wrong about today. Nothing in
  the renderer can correct a field; that is the collector's, and it is the
  re-collect in the sequence below.
- The deployed page remains the old generation, so G10's red row is unchanged and the
  public page still carries neither the qualifier nor the conditioned pill.
- The label `match (unverified)` is the renderer's own vocabulary — it adds no new term
  beyond the `not re-checked since` line already on the page.

## 8. Defects of mine, logged first

1. **The stdout normalisation in `run_gate_q59.py` used two of the three `mkdtemp`
   shapes** and reported `stdout_identical_after_sandbox_normalisation: false` with
   sha `d9adcf2b…`; the missed shape was `/tmp/q31-g8-deploy-*`. Corrected in
   `final_r29.py`, which re-writes the two fields in `out/R29-GATE.json` in place and
   records the correction beside them: **true**, sha
   `ea12512ef4f653ab85d75d64d626cace849fac3e0313ad4c6a0466b08aa95a16`, residual 0. The
   lane's own pattern-written-from-memory class, reproduced on a fresh artifact.
2. Nothing else. No figure, digest, line count, registration count or verdict in §1–§7
   depends on the defect above; the prediction in §5 was written before the run and met.

## 9. Adoption — Alexander's, and now a sequence rather than a decision

The §4 `cp` pair has moved: **collector `4e45021d80395d3f…` + gate
`125754b270bf6b0f55911f407fa9c36c6161b45a71e646fa36a70db096d7de93`** (161,122 B /
3,053 lines / 135 registrations), superseding `3b98b62fa5c7…` (G10),
`c56230e5a075…` (Q45S10), `cd43284ff009…` (Q45S9), `78b290ecf2d9…` (Q52),
`e4967da9…` (Q51), `a951dbe8…` (Q50). **And the renderer must be adopted before any
re-render**: `helm-render-g11.py` `1e82a1818c57…` (22,735 B) replacing the deployed
`5c67473d8928…` (21,766 B). Without it, the re-render Run 28 asked for would deploy the
internally inconsistent page this record measures. The order is therefore:
**adopt renderer → adopt collector + gate → re-collect → re-render → deploy**, with the
`manifest_match` pin refresh falling out of the re-collect. Beside it, unchanged: the
Q13 publication call, Q9's checksum handover, Q19's onboarding.