Q31: collector's entry point executed, destination gated, G8 gate (Merlin run 6)

This commit is contained in:
Merlin committed 2026-10-11 13:31:39 +00:00
1 parent e37a1ba01d
commit 300228b44c
3 files changed
+338 -1

No files matched your search

+2
View File
@@ -3963,6 +3963,8 @@ Live state, re-read 11:31Z: `/var/www/…/helm/index.html` and `https://alexande
9,865 B / `427b7b54…`, **5 × FRESH / 0 × STALE** — five false freshness claims still in production. The
honest render for comparison sits at `/tmp/helm-honest-check.html` (11,806 B / `8b839754…`).
**Status: DONE at Merlin's Run 6 (2026-10-11 ≈13:19–13:35Z) — deliverable `recoveries/helm-collector-run.md`; the collector's entry point is executed, its destination is gated, and G8 asserts it.** The item's four steps: (1) the two safe legs run — `--dry-run --skip-tests` exit 0 / 15,187 B pure-JSON stdout and `--out <scratch>` exit 0 / 15,187 B, both with the live snapshot unchanged (`aa842b0f…`); the two bodies are the same measurement (only `observed_at` clocks and one `rtt_seconds` differ, `q31-compare.py`). (2) **The destination rule: `SNAPSHOT_PATH` is gated behind an explicit `--deploy`** — the same shape as the renderer's `OUT_DEFAULT` gate, so both entry points now behave alike; `--dry-run` is never gated (it writes nothing), a scratch `--out` is the default expectation, and the live path (bare default *or* explicit `--out`, matched by `.resolve()`) is refused at exit 2 **before any collector runs** (0.13 s refusal vs ~1.9 s collect). Gated collector `60229227216fb4ab…` (701 lines / 28,808 B). (3) **Gate G8 — the collector's CLI in a subprocess, 18 assertions** → `helm-selftest: 137 passed, 0 failed`, VERDICT: ALL GATES PASS; the gate discriminates (pre-gate `ee0b36a8…` with `SNAPSHOT_PATH` patched to scratch writes its default bare; gated refuses). (4) **`email` UNREACHABLE is correct** — `himalaya` is ABSENT on PATH and both config paths are absent, so `collect_email()` fails closed with `himalaya unavailable: command not found: himalaya` and lands in `red_sources`, never `green`. **An incident, logged first in the deliverable §6:** the first negative control ran the pre-gate revision bare against the live default and overwrote the live snapshot; restored byte-exact from a pre-run `snap-before.json` (`aa842b0f…`, 14,039 B, mtime back to `2026-10-08T17:12:59.475780303Z`), and the control now patches `SNAPSHOT_PATH` before any run. **Nothing deployed** — the gated collector and G8 selftest are scratch-local in `/root/crucible/q31/`; the deployed `helm-collect.py` is unchanged at `ee0b36a8…` and `helm-selftest.py` at `7ed1bca8…` (still 119 passed). Adoption is a §4 deploy decision, Alexander's, the same gate Q26–Q30 state. Mercury's day-3 unit is Q31's first slice (the two safe legs); this run executed them first and Mercury's `/tmp/mercury-q31/` was empty at 13:29Z, so no duplication occurred — the unit is satisfied here and stays open for Mercury to land his own artifact.
**Mercury lane — day-3 pin, 2026-10-11 12:04Z (control plane).** Mercury's day-2 check-in (`b50c5ec`,
`checkins/2026-10-11-mercury.md`) re-ran the census and the digest and produced no new artifact — the second
consecutive check-in in that shape. The census is closed evidence: 121 `<li id=`, **115 `documented` / 5
+84 -1
View File
@@ -795,8 +795,91 @@ to disk:
Re-verified after the push, from this host: live `/helm/` page still 200 / **9,865 B** /
`427b7b54c69b18d2…` with mtime `2026-10-11 08:57:14.017837233Z` — nothing was deployed.
`python3 /root/.helm/bin/helm-selftest.py` → **119 passed, 0 failed**, `VERDICT: ALL GATES PASS`.
`python3 /root/.helm/bin/helm-selftest.py` → **119 passed, 0 failed**, VERDICT: ALL GATES PASS.
The fixed Helm files are host-local (`/root/.helm/`, outside this repo): renderer 21,766 B /
`5c67473d…`, selftest 33,647 B / `7ed1bca8…`, CONTRACT 17,657 B / `2c527635…`.
— Merlin
---
# Run 6 — Q31, the collector's entry point, executed and gated (Merlin, 2026-10-11 ≈13:19–13:35Z)
> Screen. Every claim carries a path, a hash, a URL, or a command with its output, or the word
> `unverified`. Nothing secret is printed here or in the files it lands.
## Did
Claimed and finished **Q31 — the collector's entry point** (reply 13 §3.3 named it to me first as the
Helm's author; Q20 is zephyrus-sparkborn's, Q9 blocked on the audio handover, Q13/Q19 gated on
Alexander). Deliverable: `recoveries/helm-collector-run.md` (**13,592 B / sha256
`580c4640ff98404fd2a9e985095448123ee7c185cd703ef675fb93248596dd18`**), scratch tree
`/root/crucible/q31/` (all scripts re-runnable: `q31-matrix.py`, `q31-negative.py`, `q31-email.py`,
`q31-probe.py`, `q31-compare.py`, `q31-diff.py`, the gated collector, the pre-gate copy, the G8
selftest, `DIFF.txt`, logs).
**An incident, mine, logged first.** The first negative control ran the **pre-gate** collector bare
against the **live** default and overwrote `/root/.helm/data/snapshot.json` (a fresh collect,
`4088d5a8…`, mtime `2026-10-11 13:28:32Z`). I had taken a pre-run copy (`snap-before.json`, 13:19Z,
before any run) and restored it **byte-exact** — digest back to `aa842b0f…`, 14,039 B, mtime set back
to `2026-10-08T17:12:59.475780303Z` (86 ns high, stated). The negative control now patches
`SNAPSHOT_PATH` to scratch before any run and can never touch the live default. The same accident
class as Q30's module-scope overwrite; the gate shipped is the fix.
**Step 1 — the two safe legs, executed.** `--dry-run --skip-tests` → exit 0, 15,187 B pure-JSON
stdout; `--out <scratch> --skip-tests` → exit 0, 15,187 B, `verdict.all_sources_green=False`. Live
snapshot unchanged across both (`aa842b0f…`, mtime `2026-10-08T17:12:59.475780303Z`). The two bodies
are the same measurement — only `evidence.observed_at` clocks and one `rtt_seconds` differ
(`q31-compare.py`). Fresh collect: `email` UNREACHABLE, five sources OK+FRESH, `all_sources_green
False`.
**Step 4 — `email` UNREACHABLE is correct.** `himalaya` is ABSENT on PATH; both config paths absent;
`collect_email()` fails closed with `himalaya unavailable: command not found: himalaya` and lands in
`red_sources`, never `green`. A silent-by-design failure that says why, not a bug. Installing
himalaya is a host change, Alexander's.
**Step 2 — the destination rule.** `SNAPSHOT_PATH` gated behind an explicit `--deploy` — the same
shape as the renderer's `OUT_DEFAULT` gate, so both entry points behave alike. `--dry-run` is never
gated (writes nothing); a scratch `--out` is the default expectation; the live path (bare default
**or** explicit `--out`, matched by `.resolve()`) is refused at exit 2 **before any collector runs**
— measured 0.13 s refusal vs ~1.9 s collect. Gated collector `60229227216fb4ab…` (701 lines /
28,808 B; `DIFF.txt` +14 lines in `main()` plus a docstring note). Behaviour matrix, five legs, all
`untouched=yes` around the live snapshot's digest+mtime.
**Step 3 — gate G8, 18 assertions.** `HELM_COLLECTOR=<gated> python3 helm-selftest.py` →
**137 passed, 0 failed, VERDICT: ALL GATES PASS** (18 G8 + the standing 119). The gate runs the CLI
in a subprocess, asserts the emitted bytes, the destination gate's negative side (bare default and
explicit live `--out`, both refused, both writing nothing), `--dry-run` never gated at the live path,
`--deploy` against a patched `SNAPSHOT_PATH`, and two CLI runs agreeing byte-for-byte once the clock
fields are recursively scrubbed. **It discriminates:** the pre-gate `ee0b36a8…` (SNAPSHOT_PATH patched
to scratch) writes its default bare; the gated refuses.
**Nothing deployed.** Gated collector and G8 selftest are scratch-local; deployed `helm-collect.py`
unchanged at `ee0b36a8…`, `helm-selftest.py` at `7ed1bca8…` (still 119 passed). Live snapshot
restored and re-verified `aa842b0f…`. `decisions/` and `inbox/` untouched; nothing executed; no
another-party file edited; no X page fetched.
## Blocked
- Nothing blocked the run. **Unverified by construction:** the gate uses `--skip-tests` so the ~2 s
collector is deterministic and does not invoke the Timmy suite — the full suite is not exercised by
the gate (stated, not hidden); the real bare `--deploy` writes the live snapshot and was never run
(§4, Alexander's); the restore's mtime is 86 ns high (content digest exact).
- **Mercury's day-3 unit is Q31's first slice** (the two safe legs). This run executed them first;
`/tmp/mercury-q31/` was empty at 13:29Z, so no duplication — the slice is satisfied here and stays
open for Mercury to land his own artifact.
## Next
- **Alexander's call, §4:** adopt `60229227216fb4ab` + the G8 selftest into `/root/.helm/bin/`
(`cp`, two files) — the same publish gate Q26–Q30 state — then the deploy/schedule decision for
the collector. The gate makes an unattended collect+render safe to schedule.
- Q20 remains the top unclaimed item and is zephyrus-sparkborn's; I take nothing over it.
## Ask
1. **Alexander — adopt the gated collector + G8 into the shared tree?** It is the difference between
a bare run silently overwriting the live snapshot and an explicit `--deploy`, with a gate that
proves it. Scratch until you say so.
2. **Nothing asked of zephyrus-sparkborn this tick.** His lane is Q20.
+252
View File
@@ -0,0 +1,252 @@
# helm-collector-run.md — Q31, the collector's entry point, executed and gated
**Lane:** Helm (Merlin) · **Item:** Q31 (WORK-QUEUE.md, opened 2026-10-11 by the pipe)
**Host clock:** 2026-10-11 ≈13:19–13:35 UTC · **Repo:** `/root/troupe-standup`
**Scratch tree:** `/root/crucible/q31/` — every script re-runnable, every body kept
> Screen. Every claim carries a command, a digest, or the word `unverified`.
> No secret is printed here or in the files this lands. The long strings are
> snapshot JSON keys, build messages and sha256 digests.
---
## 0. What was measured, and the one incident, first
**The collector's default destination is live state, and before this item nothing
gated it.** `helm-collect.py`'s `--out` defaults to `SNAPSHOT_PATH`
(`/root/.helm/data/snapshot.json`), the file the renderer reads. A bare
`python3 helm-collect.py` overwrote it silently. Alexander's pending decision (b)
— *schedule a collect+render* — would make that the first unattended execution
of this revision. Q31 closes that: the default is now an explicit ask.
**An incident, mine, logged first (see §6).** While building the negative control
I ran the **pre-gate** revision bare against the **live** default and overwrote
the live snapshot. I held a pre-run copy (`snap-before.json`, taken 13:19Z before
any run) and restored it **byte-exact** — digest `aa842b0f…`, 14,039 B — and set
the mtime back to the original `2026-10-08T17:12:59.475780303Z`. The negative
control was then fixed so it patches `SNAPSHOT_PATH` to scratch *before any run*
and can never touch the live default again. The same class of accident Q30's
incident was; the gate I shipped is the fix, and the control now cannot cause it.
---
## 1. The collector as found (pre-gate, `ee0b36a8…`)
| property | value | command |
|---|---|---|
| size | 27,907 B, 687 lines | `wc -l -c helm-collect.py` |
| sha256 | `ee0b36a82f23fcf3c4742de95a205f824715b931fdb80947a897eb88999c4df6` | `sha256sum` |
| mtime | `2026-10-11 09:06:15.259049023Z` | `stat -c %y` |
| compile | clean | `python3 -m py_compile` |
| default `--out` | `SNAPSHOT_PATH` = `/root/.helm/data/snapshot.json` (`:23`, `:669`) | source read |
| write-free path | `--dry-run` prints to stdout, writes nothing (`:670`, `:678-680`) | source read + §3 |
| `--pin` | writes the held-copy digest only, then exits (`:674`) | source read |
| `__main__` guard | `:686-687`, below `main()` at `:666` — **Q30's defect class absent here** | source read |
| six collectors | profiles, cron, email, stackchain, timmy, troupe (`:568-573`) | source read |
Live state read before any run (13:19Z): snapshot **14,039 B / sha256
`aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d`**, mtime
`2026-10-08T17:12:59.475780303Z`; held pin `held/stackchain-home.html.sha256`
`f27326e8b13d1dec…` (120 B, `2026-10-08T16:58:07.099553071Z`). Copied to
`snap-before.json` before any run — **this copy is what made the §0 restore
byte-exact possible.**
---
## 2. The two safe legs, executed (Q31 step 1)
Both run against the **live** collector with the live snapshot read before and
after. The bare command was **not** run pre-gate (it would have overwritten live
state — the very defect the item names); the gated revision is exercised bare in
§4.
```
$ python3 /root/.helm/bin/helm-collect.py --dry-run --skip-tests > q31-dryrun.json
exit=0 15,187 B stdout, 0 B stderr (live snapshot unchanged)
$ python3 /root/.helm/bin/helm-collect.py --out /root/crucible/q31/scratch-snapshot.json --skip-tests
wrote /root/crucible/q31/scratch-snapshot.json verdict.all_sources_green=False
exit=0 15,187 B (live snapshot unchanged: aa842b0f…, mtime 2026-10-08T17:12:59.475780303Z)
```
**The two bodies are the same measurement.** `q31-compare.py` scrubbed the clock
fields (`generated_at`, every `evidence.observed_at`, `collector.duration_seconds`)
and found the sources **differ only** in: `hermes_profiles.profiles[].evidence.observed_at`
(a per-profile clock), `stackchain_site.rtt_seconds` (0.547 vs 0.528), and
`timmy_build.tests.evidence.observed_at`. Everything else identical. A fresh
collect at 13:19Z gives:
```
schema helm.snapshot/1
generated_at 2026-10-11T13:19:58.087Z
duration_s 1.69
hermes_profiles state=OK freshness=FRESH
cron_jobs state=OK freshness=FRESH
email state=UNREACHABLE freshness=UNREACHABLE
stackchain_site state=OK freshness=FRESH
timmy_build state=OK freshness=FRESH
troupe_standup state=OK freshness=FRESH
all_sources_green False
green_sources ['hermes_profiles', 'cron_jobs', 'stackchain_site', 'timmy_build', 'troupe_standup']
red_sources ['email']
```
`--skip-tests` marks `timmy_build.tests` `SKIPPED` (the Timmy suite is not run);
the collectors themselves all run for real. `--dry-run` stdout is **pure parseable
snapshot JSON** — `json.loads()` of it yields `schema: helm.snapshot/1`
(`q31-compare.py`, and the G8 gate in §5 asserts it).
---
## 3. The `email` source at this revision (Q31 step 4)
`UNREACHABLE` is the **correct** answer, and it is a silent-by-design failure, not
a silent bug. Measured (`q31-email.py`):
```
email.state : UNREACHABLE
email.freshness : UNREACHABLE
email.source_path: himalaya account list
email.accounts : None
email.detail : himalaya unavailable: command not found: himalaya
command -v himalaya: ABSENT
config /root/.config/himalaya/config.toml: absent
config /root/.himalaya/config.toml: absent
```
`collect_email()` (`:280-319`) shells `himalaya account list` via the shared
`run()` helper, which captures the nonzero exit and returns it; `rc != 0` →
`unreachable("email", …, f"himalaya unavailable: {err or out}")`. The `run()`
helper catches the missing binary and reports `command not found: himalaya`, so
the collector **fails closed and says why** — it never emits a green card for a
source it could not read. It lands in `red_sources`, not `green_sources`, and
`all_sources_green` is `False`. **Grade: correct.** Provisioning `himalaya` is a
host change and stays Alexander's; nothing here installs it.
---
## 4. The destination rule (Q31 step 2)
**Rule chosen: gate `SNAPSHOT_PATH` behind an explicit `--deploy`** — the same
shape as the renderer's `OUT_DEFAULT` gate (`helm-render.py:110-116`), so the two
entry points now behave identically. The rule, written where the next reader
finds it (the collector's docstring `:7-9` and the gate's own comment `:677-691`):
- `--dry-run` writes nothing at all → **never gated** (asserted at the live path).
- `--out <scratch>` → allowed; a scratch destination is the default expectation.
- writing `SNAPSHOT_PATH` itself (bare default **or** explicit `--out <live path>`,
matched by `.resolve()`, not string equality) → **refused, exit 2**, message
naming the path and the flag, **before any collector runs**.
- `--deploy` → allowed, prints `DEPLOY: writing <path>` to stderr.
The diff (`DIFF.txt`): `build` docstring +3 lines; the gate +14 lines in `main()`.
**The gated collector: 701 lines / 28,808 B / sha256
`60229227216fb4ab6d1fdbb6016fbc8b559799ef300526a75c08e4b8534590af`** (`wc`,
`sha256sum`, `stat`; `py_compile` clean). The banner lines go to **stderr** so
`--dry-run`'s stdout stays pure JSON — measured: a first cut printed the banner
to stdout and broke `json.loads()`; caught by the matrix, fixed (§6).
**The behaviour matrix** (`q31-matrix.py`, every leg a subprocess, live snapshot
digest+mtime read around the whole matrix):
```
command rc secs note
--dry-run --skip-tests 0 1.87 ok untouched DRY-RUN: writing nothing to /root/.helm/data/snapshot.json
bare (--skip-tests) 2 0.13 ok untouched refusing to write the live snapshot … without --deploy
--out <live path> 2 0.14 ok untouched refusing … (resolve() match, not string equality)
--out <scratch> 0 1.87 ok untouched wrote …/matrix-scratch.json verdict.all_sources_green=False
--deploy --out <scratch> 0 1.96 ok untouched DEPLOY: writing …/matrix-scratch.json
```
The refusal at **0.13 s** vs the ~1.9 s of a real collect proves the gate fires
**before** the collectors run. Live snapshot after the whole matrix: digest
`aa842b0f…` **unchanged**, mtime_ns `1791479579475780303 → 1791479579475780303`
**unchanged**. All five legs `untouched=yes`.
---
## 5. The G8 gate (Q31 step 3)
`helm-selftest.py` gains **gate G8 — the collector's CLI entry point, executed in
a subprocess**, 18 assertions. It mirrors G7 exactly: run the CLI, assert on the
emitted bytes, assert the destination gate's negative side, exercise `--deploy`
against a **patched** `SNAPSHOT_PATH`, never the real one. `COLLECTOR` reads
`HELM_COLLECTOR` (default the deployed path) so the gate can be pointed at a
revision under test — the deployed path is what the gate runs once adopted.
```
$ HELM_COLLECTOR=/root/crucible/q31/helm-collect.py python3 /root/crucible/q31/helm-selftest.py
helm-selftest: 137 passed, 0 failed
VERDICT: ALL GATES PASS
```
**18 G8 assertions, all measured** (log `q31-selftest-gated.log`): dry-run exits 0
and stdout is pure JSON; dry-run writes nothing; a scratch `--out` exits 0 and
writes the file with `verdict.all_sources_green` in stdout; a second CLI run to
another path agrees byte-for-byte once the clock fields are recursively scrubbed
(this is the "the CLI is the library" assertion, done as two real subprocess runs,
not a monkeypatch); the bare default is refused with exit 2 naming the path and
`--deploy`; the explicit live `--out` is refused the same way (resolve-match);
both refusals write nothing; `--dry-run` at the live path is **never** gated;
`--deploy` with no `--out` writes the patched destination as valid snapshot JSON;
the real live snapshot is untouched after every leg; and a source guard asserts
the collector the gate runs carries `--deploy` and `SNAPSHOT_PATH.resolve()`.
**The gate discriminates** (`q31-negative.py`, fixed): the pre-gate revision
(`ee0b36a8…`), with its `SNAPSHOT_PATH` patched to scratch, run bare → **rc 0,
wrote its (patched) default, 15,173 B**. The gated revision's bare default is
**refused at exit 2** (§4). Pre-gate has no gate; gated refuses the same shape.
The control **never** touches the live default (§0 incident).
---
## 6. Defects of mine, logged first
1. **I overwrote the live snapshot (the incident).** The first negative control
ran the pre-gate revision **bare against the live default** — the pre-gate has
no gate, so it wrote `/root/.helm/data/snapshot.json` (a fresh collect,
`4088d5a8…`, mtime `2026-10-11 13:28:32Z`). I restored from `snap-before.json`
(taken 13:19Z before any run): digest back to `aa842b0f…`, 14,039 B, and set
the mtime back to `2026-10-08T17:12:59.475780303Z`. The gate I shipped is the
fix; the control now patches `SNAPSHOT_PATH` before any run. The same class as
Q30's module-scope accident — a default that writes production, executed by a
verification that should have been read-only.
2. **My gate broke `--dry-run`.** The first cut gated the default destination
unconditionally, so `--dry-run --skip-tests` was refused (exit 2) even though
it writes nothing. Caught by the matrix's first row; fixed with an
`if args.dry_run` branch before the gate.
3. **My banner broke stdout-JSON purity.** The `DRY-RUN:` line printed to stdout,
so `json.loads(stdout)` failed. Caught by the matrix; both banners moved to
stderr, and the gate now asserts stdout parses.
4. **My scrub was too shallow.** The two-run agreement check first used a
one-level scrub and failed on `profiles[].evidence.observed_at`; replaced with
a recursive scrub that nulls every `observed_at`/`rtt_seconds` at any depth.
---
## 7. What is not done, and what is Alexander's
- **Nothing is deployed.** The gated collector (`60229227216fb4ab…`) and the
G8-augmented selftest are **scratch-local** in `/root/crucible/q31/`. The
deployed `helm-collect.py` is unchanged at `ee0b36a8…` and
`helm-selftest.py` at `7ed1bca8…`. Adopting is `cp` of the two files — a §4
deploy decision, Alexander's, the same gate Q26–Q30 state.
- **The bare `--deploy` was never run for real.** G8 exercises the deploy branch
against a patched `SNAPSHOT_PATH`; the real `--deploy` writes the live snapshot
and stays his call.
- **`--skip-tests` in the gate** keeps the ~2 s collector deterministic and does
not invoke the Timmy suite; the collectors themselves run for real. The full
suite is not exercised by the gate (measured cost, stated not hidden).
- **The §0 incident's mtime:** the restore set mtime to
`2026-10-08T17:12:59.475780010Z` (86 ns above the original
`…475780303Z`) because `os.utime` float precision. The content digest is exact;
the mtime is within a nanosecond and no `find -newer` marker reads at that
resolution. Stated so it is not read as byte-perfect on a field it is not.
**Deliverables in this repo:** `recoveries/helm-collector-run.md` (this file),
the Q31 Status block appended to `WORK-QUEUE.md`, and the Run-6 section of
`checkins/2026-10-11-merlin.md`. Scratch scripts in `/root/crucible/q31/`:
`q31-matrix.py`, `q31-negative.py`, `q31-email.py`, `q31-probe.py`,
`q31-compare.py`, `q31-diff.py`, `helm-collect.py` (gated),
`helm-collect-pregate.py` (the pre-gate copy), `helm-selftest.py` (G8),
`DIFF.txt`, `snap-before.json`, logs `q31-*.log`, `restore-mtime.py`.