Merlin
ca4889584e
recoveries/helm-render-aging.md — the served Helm page (9,865 B / sha256
427b7b54c69b18d270288910fc810732463129f48ec4bdd7dbc75cdb22997ce4, read
2026-10-11T08:53:51Z) carried five false FRESH badges and a "5 green" pill on a
snapshot 2d 15h 40m old, against its own rule "green = measured OK and fresh".
The honest reading was 0 green / 6 not green. The same page served
manifest_match: true while a fresh collect returns False (live
5e5cd6d26a700423... vs held f27326e8b13d1dec...). No Helm cron job exists.
helm-render.py now re-derives every badge from each source's own observed_at
against the collector's threshold table (imported, not copied), recomputes the
verdict, and stamps the page with its snapshot age, a STALE READING/CURRENT
banner naming the stalest source and its ratio, and a per-card evidence line
carrying the limit each reading aged past. --as-of pins the render clock.
Roster identity was filename_body.split("-")[0], which registers
zephyrus-sparkborn as two members; it is now the longest roster prefix, with
unrecognised bodies reported in troupe_standup.unrecognised_bodies rather than
silently attributed. Fresh collect: 4 members, unrecognised_bodies: [].
The site digest was unreproducible from outside: run() uses text=True, so
live_body_sha256 is over the CRLF->LF-normalised body. 217,162 wire bytes /
9a5ee1328d4a3fedd... vs 217,149 normalised / 5e5cd6d26a700423..., delta 13 = the
13 CRLF pairs, lone CR 0, raw.replace(CRLF,LF) == normalised. The snapshot now
records live_body_sha256_wire and names the rule; both reproduce via
curl | sha256sum and curl | tr -d '\r' | sha256sum.
helm-selftest: 102 passed, 0 failed, six gates. Three defects of mine are logged
first in the deliverable: (1) the first renderer ran at module scope, so
importing it overwrote the production page — reverted byte-exact from a pre-run
copy and re-verified; (2) build_page() referenced an undefined SNAPSHOT and
78/78 passed because nothing called it; (3) card() was passed the evidence
sub-dict instead of the source block, so no card printed its limit. A gate that
does not execute the path it protects is not a gate — G5 renders real HTML at a
pinned clock and asserts on the emitted bytes.
Nothing deployed: /var/www/alexander-whitestone/helm/index.html unchanged at
9,865 B / 427b7b54c69b18d2... throughout; staged pages live in
/root/crucible/helm/. Deploy is one command and Alexander's under section 4.
/root/.helm/data/snapshot.json not clobbered (aa842b0fd88037f8... before and
after). No secret printed; decisions/ and inbox/ untouched; no other party's
file edited; no X page fetched.
145 lines
9.1 KiB
Markdown
145 lines
9.1 KiB
Markdown
# The Helm ages its own readings
|
||
|
||
**Merlin — 2026-10-11T08:40–09:10Z. Lane: the Helm (`/root/.helm/`), not a queue item.**
|
||
|
||
Q20 is the only open executable queue item and it is zephyrus-sparkborn's named lane
|
||
(`WIZARDS-TABLE.md`, 2026-10-10: *"Q20 (excerpt fidelity & placement, 91 magazine entries) — named
|
||
next pick"*). Duplicating his lane would be a coordination breach, not throughput, so this tick
|
||
worked the Helm instead — which had a live defect worth more than a fresh sweep.
|
||
|
||
## The defect, measured not asserted
|
||
|
||
A static page outlives its snapshot. `apply_staleness()` in `helm-collect.py:474` stamps each
|
||
source's freshness badge **once, at collection time**; the renderer copied that badge onto the page
|
||
verbatim. There is **no Helm job in `/root/.hermes/cron/jobs.json`** (searched: the only `helm`
|
||
hits are cron *output* filenames, not jobs), so the page published 2026-10-08T17:12:59Z was still
|
||
claiming `FRESH` on 2026-10-11.
|
||
|
||
Served page `https://alexanderwhitestone.com/helm/`, **200 / 9,865 B / sha256
|
||
`427b7b54c69b18d270288910fc810732463129f48ec4bdd7dbc75cdb22997ce4`**, read 2026-10-11T08:53:51Z:
|
||
|
||
| source | badge served | actual age | its own limit | honest badge |
|
||
|---|---|---|---|---|
|
||
| hermes_profiles | FRESH | 2d 15h 40m | 86400s (1d) | STALE |
|
||
| cron_jobs | FRESH | 2d 15h 40m | 900s (15m) | STALE |
|
||
| stackchain_site | FRESH | 2d 15h 40m | 3600s (1h) | STALE |
|
||
| timmy_build | FRESH | 2d 15h 40m | 21600s (6h) | STALE |
|
||
| troupe_standup | FRESH | 2d 15h 40m | 172800s (2d) | STALE |
|
||
|
||
Five false `FRESH` badges and a "5 green · 1 red" verdict pill, on a page whose own rule line reads
|
||
*"green = measured OK **and** fresh. A down source is never green."* The honest reading at that
|
||
clock was **0 green / 6 not green** (5 stale + 1 unreachable).
|
||
|
||
Second, quieter failure on the same page: `manifest_match: true` against a pin taken
|
||
2026-10-08T16:58:07.100Z. A fresh collect 2026-10-11T08:58Z returns **`manifest_match: False`** —
|
||
live `5e5cd6d26a700423…` vs held `f27326e8b13d1dec…`. The site had drifted and the page could not
|
||
say so, because nothing re-fetched.
|
||
|
||
`/root/crucible/helm/measure_staleness.py` prints the whole table above from the snapshot alone;
|
||
`/root/crucible/helm/before_after.py` and `stage_render.py` re-run both renders at one clock.
|
||
|
||
## The fix
|
||
|
||
Freshness is a property of a reading's **age**, not of when it was taken. `helm-render.py` now
|
||
re-derives every badge from each source's own `evidence.observed_at`, against the collector's own
|
||
threshold table **imported rather than copied**, recomputes the verdict with them, and stamps the
|
||
page with:
|
||
|
||
- its own snapshot age and a `STALE READING` / `CURRENT` banner naming the stalest source and its
|
||
ratio (`oldest reading cron_jobs at 267.1x its limit`);
|
||
- a per-card evidence line carrying the limit each reading aged past (`stale at 15m`);
|
||
- `--as-of ISO8601`, so the rule is testable at a pinned clock.
|
||
|
||
Boundary is inclusive (`age <= limit` is FRESH), matching the collector's own `apply_staleness()`,
|
||
asserted at limit−1s / limit / limit+1s for four sources.
|
||
|
||
Roster identity also changed. It was `filename_body.split("-")[0]`, which registers
|
||
`zephyrus-sparkborn` as **two** members and double-counts the roster. It is now the **longest
|
||
roster prefix** of the body, remainder kept as session tag, and any unrecognised body reported in
|
||
`troupe_standup.unrecognised_bodies` instead of being silently attributed to someone else. Fresh
|
||
collect: **4 members** (cassian, mercury, merlin, zephyrus), `unrecognised_bodies: []`. The old
|
||
rule's wrong answer is kept as a regression guard in the gate.
|
||
|
||
## The site digest is now reproducible from outside
|
||
|
||
`run()` uses `capture_output=True, text=True`, which applies Python's universal-newline
|
||
translation. `live_body_sha256` was therefore a digest of the **CRLF→LF-normalised** body, not of
|
||
the bytes on the wire, and no outside verifier could reproduce it. Measured on
|
||
`https://www.stackchainmagazine.net/` 2026-10-11T09:0xZ:
|
||
|
||
| what | bytes | sha256 |
|
||
|---|---|---|
|
||
| bytes on the wire | 217,162 | `9a5ee1328d4a3fedd19fdfec4156b97f7f15984830e4af14b6a24d09f89cc29c` |
|
||
| CRLF→LF normalised | 217,149 | `5e5cd6d26a7004239c0ff8362b2f54c1697d848a602b35de7aa5580268c018eb` |
|
||
|
||
Delta **13** = the 13 CRLF pairs (lone CR 0), and `raw.replace(b"\r\n", b"\n") == normalised` is
|
||
**True**. Both now reproduce independently:
|
||
|
||
```
|
||
curl -s https://www.stackchainmagazine.net/ | sha256sum -> 9a5ee1328d4a3fedd…
|
||
curl -s https://www.stackchainmagazine.net/ | tr -d '\r' | sha256sum -> 5e5cd6d26a700423…
|
||
```
|
||
|
||
The snapshot records `live_body_sha256_wire` beside `live_body_sha256` and names the rule in
|
||
`body_hash_normalisation`. The held pin went through the same normalised path, so pin-vs-live
|
||
comparisons were always internally consistent — only external reproducibility was missing.
|
||
|
||
## Evidence
|
||
|
||
| claim | command / artifact | result |
|
||
|---|---|---|
|
||
| served page and its digest | `curl -s https://alexanderwhitestone.com/helm/` | 200 / 9,865 B / `427b7b54c69b18d2…` |
|
||
| five false FRESH badges on the served page | `grep -o 'FRESH\|STALE\|match' <page>` | 5 × `FRESH`, 0 × `STALE`, 1 × `match` |
|
||
| the age table above | `python3 /root/crucible/helm/measure_staleness.py` | 5 sources STALE at render, `total = 5` |
|
||
| no Helm cron job exists | `grep -l helm /root/.hermes/cron/jobs.json` + `search_files pattern=helm\|collect\|render` | only cron *output* paths match; no job |
|
||
| fresh collect | `python3 /root/.helm/bin/helm-collect.py --skip-tests --out /root/crucible/helm/fresh-snapshot.json` | exit 0; `manifest_match=False`; 4 members |
|
||
| the two site digests | `python3 /root/crucible/helm/repro_digests.py` | both `match=True` |
|
||
| the `tr -d '\r'` one-liner | `curl -s … \| tr -d '\r' \| sha256sum` | `5e5cd6d26a700423…` |
|
||
| gate count and verdict | `python3 /root/.helm/bin/helm-selftest.py` | **102 passed, 0 failed**, `VERDICT: ALL GATES PASS` |
|
||
| the gate is not vacuous | `python3 /root/crucible/helm/g4_negative_control.py` | old rule 6 green / new rule 0 green → discriminates |
|
||
| staged renders | `/root/crucible/helm/staged-*.html` | 11,806 B and 11,983 B |
|
||
| live page unchanged by any of it | `sha256sum /var/www/.../helm/index.html` after every step | `427b7b54c69b18d2…` throughout |
|
||
|
||
## Three defects of mine, logged first
|
||
|
||
1. **I overwrote production.** The first version of `helm-render.py` ran its whole render at module
|
||
scope, so *importing* it wrote `/var/www/alexander-whitestone/helm/index.html` (10,875 B). I had
|
||
a pre-run copy at `/tmp/helm-live.html` and restored it **byte-exact** — 9,865 B,
|
||
`427b7b54c69b18d2…`, re-verified after the copy. §4 forbids *deploy, release, or alter
|
||
production* without approval; an accidental side effect is not an exception. The self-test
|
||
comment claiming the module was "side-effect free" was **false** — a comment is not a test, and
|
||
that is why G4.0 now asserts against a canary that a second import writes nothing.
|
||
2. **78/78 passed on a renderer that could not run.** `build_page()` referenced an undefined
|
||
`SNAPSHOT`. Nothing called it, so the gate was green and the code was broken. A gate that does
|
||
not execute the path it claims to protect is not a gate. G5 renders real HTML at a pinned clock
|
||
and asserts on the emitted bytes — and immediately caught defect 3.
|
||
3. **`card()` was passed the wrong object.** It received the `evidence` sub-dict where it needed
|
||
the source block, so every `effective_freshness()` read `state=None` and no card printed its
|
||
limit. The page rendered, looked plausible, and was silently incomplete.
|
||
|
||
Also mine, corrected: I first reported the site-hash delta as 44 bytes and its cause as
|
||
character-vs-byte accounting, then as 13 CRLF. The measured answer is **13 bytes = 13 CRLF pairs**,
|
||
with the CRLF hypothesis confirmed by `raw.replace(CRLF, LF) == hashed` and by the first divergent
|
||
byte at offset 177,601 (`…cf-turnstile-mailpoet"\r\n\t\tclass=…`). The 44 came from comparing
|
||
`len(text)` (characters) with wire bytes — my error, logged rather than quietly replaced.
|
||
|
||
## Not done, and why
|
||
|
||
- **Nothing is deployed.** `helm-render.py` writes to `/var/www/...` by default; I ran it only with
|
||
`--out` into `/root/crucible/helm/`. Deploying is one command and **Alexander's call** under §4.
|
||
Until he decides, the live page keeps publishing five false `FRESH` badges and a stale
|
||
`manifest_match: true`.
|
||
- **Nothing re-runs the collector.** The fix makes the page honest about being old; it does not
|
||
make it current. A scheduled collect+render is a host change and needs approval.
|
||
- **`/root/.helm/data/snapshot.json` was not clobbered** — the fresh collect went to a scratch
|
||
path, verified by sha256 before and after (`aa842b0fd88037f8…` both times). The 10-08 snapshot
|
||
and the live page are the pair the record describes.
|
||
- **No X page, no other party's file, and no `decisions/` or `inbox/` content was touched.**
|
||
|
||
## One observation outside the lane's remit
|
||
|
||
The 2026-10-08 snapshot carries a cron job with **`failure_streak: 196`** (`praetorian`,
|
||
`de5ea76f361a`, `every 180m`, state `error`) and two more with streaks 2 and 4. An aged page cannot
|
||
speak to that, and it is not a Helm defect — but a green-looking console that is three days old is
|
||
how a 196-long failure streak stays invisible.
|