0 Fork 0

Files

T

timmy 1c0d660c2a Q31 Run 7: dead digest corrected to the artifact that exists (8a477376), restore-mtime root-caused and fixed, gate re-run 137/137 (Merlin)

2026-10-11 15:39:52 +00:00

19 KiB

Raw Blame History

helm-collector-run.md — Q31, the collector's entry point, executed and gated

Lane: Helm (Merlin) · Item: Q31 (WORK-QUEUE.md, opened 2026-10-11 by the pipe) Host clock: 2026-10-11 ≈13:19–13:35 UTC · Repo: /root/troupe-standup Scratch tree: /root/crucible/q31/ — every script re-runnable, every body kept

Screen. Every claim carries a command, a digest, or the word unverified. No secret is printed here or in the files this lands. The long strings are snapshot JSON keys, build messages and sha256 digests.

0. What was measured, and the one incident, first

The collector's default destination is live state, and before this item nothing gated it. helm-collect.py's --out defaults to SNAPSHOT_PATH (/root/.helm/data/snapshot.json), the file the renderer reads. A bare python3 helm-collect.py overwrote it silently. Alexander's pending decision (b) — schedule a collect+render — would make that the first unattended execution of this revision. Q31 closes that: the default is now an explicit ask.

An incident, mine, logged first (see §6). While building the negative control I ran the pre-gate revision bare against the live default and overwrote the live snapshot. I held a pre-run copy (snap-before.json, taken 13:19Z before any run) and restored it byte-exact — digest aa842b0f…, 14,039 B — and set the mtime back to the original 2026-10-08T17:12:59.475780303Z. The negative control was then fixed so it patches SNAPSHOT_PATH to scratch before any run and can never touch the live default again. The same class of accident Q30's incident was; the gate I shipped is the fix, and the control now cannot cause it.

1. The collector as found (pre-gate, `ee0b36a8…`)

property	value	command
size	27,907 B, 687 lines	`wc -l -c helm-collect.py`
sha256	`ee0b36a82f23fcf3c4742de95a205f824715b931fdb80947a897eb88999c4df6`	`sha256sum`
mtime	`2026-10-11 09:06:15.259049023Z`	`stat -c %y`
compile	clean	`python3 -m py_compile`
default `--out`	`SNAPSHOT_PATH` = `/root/.helm/data/snapshot.json` (`:23`, `:669`)	source read
write-free path	`--dry-run` prints to stdout, writes nothing (`:670`, `:678-680`)	source read + §3
`--pin`	writes the held-copy digest only, then exits (`:674`)	source read
`__main__` guard	`:686-687`, below `main()` at `:666` — Q30's defect class absent here	source read
six collectors	profiles, cron, email, stackchain, timmy, troupe (`:568-573`)	source read

Live state read before any run (13:19Z): snapshot 14,039 B / sha256 aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d, mtime 2026-10-08T17:12:59.475780303Z; held pin held/stackchain-home.html.sha256 f27326e8b13d1dec… (120 B, 2026-10-08T16:58:07.099553071Z). Copied to snap-before.json before any run — this copy is what made the §0 restore byte-exact possible.

2. The two safe legs, executed (Q31 step 1)

Both run against the live collector with the live snapshot read before and after. The bare command was not run pre-gate (it would have overwritten live state — the very defect the item names); the gated revision is exercised bare in §4.

$ python3 /root/.helm/bin/helm-collect.py --dry-run --skip-tests > q31-dryrun.json
exit=0   15,187 B stdout, 0 B stderr        (live snapshot unchanged)

$ python3 /root/.helm/bin/helm-collect.py --out /root/crucible/q31/scratch-snapshot.json --skip-tests
wrote /root/crucible/q31/scratch-snapshot.json  verdict.all_sources_green=False
exit=0   15,187 B        (live snapshot unchanged: aa842b0f…, mtime 2026-10-08T17:12:59.475780303Z)

The two bodies are the same measurement. q31-compare.py scrubbed the clock fields (generated_at, every evidence.observed_at, collector.duration_seconds) and found the sources differ only in: hermes_profiles.profiles[].evidence.observed_at (a per-profile clock), stackchain_site.rtt_seconds (0.547 vs 0.528), and timmy_build.tests.evidence.observed_at. Everything else identical. A fresh collect at 13:19Z gives:

schema          helm.snapshot/1
generated_at    2026-10-11T13:19:58.087Z
duration_s      1.69
hermes_profiles    state=OK           freshness=FRESH
cron_jobs          state=OK           freshness=FRESH
email              state=UNREACHABLE  freshness=UNREACHABLE
stackchain_site    state=OK           freshness=FRESH
timmy_build        state=OK           freshness=FRESH
troupe_standup     state=OK           freshness=FRESH
all_sources_green False
green_sources    ['hermes_profiles', 'cron_jobs', 'stackchain_site', 'timmy_build', 'troupe_standup']
red_sources      ['email']

--skip-tests marks timmy_build.tests SKIPPED (the Timmy suite is not run); the collectors themselves all run for real. --dry-run stdout is pure parseable snapshot JSON — json.loads() of it yields schema: helm.snapshot/1 (q31-compare.py, and the G8 gate in §5 asserts it).

3. The `email` source at this revision (Q31 step 4)

UNREACHABLE is the correct answer, and it is a silent-by-design failure, not a silent bug. Measured (q31-email.py):

email.state      : UNREACHABLE
email.freshness  : UNREACHABLE
email.source_path: himalaya account list
email.accounts   : None
email.detail     : himalaya unavailable: command not found: himalaya
command -v himalaya: ABSENT
config /root/.config/himalaya/config.toml: absent
config /root/.himalaya/config.toml: absent

collect_email() (:280-319) shells himalaya account list via the shared run() helper, which captures the nonzero exit and returns it; rc != 0 → unreachable("email", …, f"himalaya unavailable: {err or out}"). The run() helper catches the missing binary and reports command not found: himalaya, so the collector fails closed and says why — it never emits a green card for a source it could not read. It lands in red_sources, not green_sources, and all_sources_green is False. Grade: correct. Provisioning himalaya is a host change and stays Alexander's; nothing here installs it.

4. The destination rule (Q31 step 2)

Rule chosen: gate SNAPSHOT_PATH behind an explicit --deploy — the same shape as the renderer's OUT_DEFAULT gate (helm-render.py:110-116), so the two entry points now behave identically. The rule, written where the next reader finds it (the collector's docstring :7-9 and the gate's own comment :677-691):

--dry-run writes nothing at all → never gated (asserted at the live path).
--out <scratch> → allowed; a scratch destination is the default expectation.
writing SNAPSHOT_PATH itself (bare default or explicit --out <live path>, matched by .resolve(), not string equality) → refused, exit 2, message naming the path and the flag, before any collector runs.
--deploy → allowed, prints DEPLOY: writing <path> to stderr.

The diff (DIFF.txt): build docstring +3 lines; the gate +14 lines in main(). The gated collector: 701 lines / 28,808 B / sha256 60229227216fb4ab6d1fdbb6016fbc8b559799ef300526a75c08e4b8534590af (wc, sha256sum, stat; py_compile clean). The banner lines go to stderr so --dry-run's stdout stays pure JSON — measured: a first cut printed the banner to stdout and broke json.loads(); caught by the matrix, fixed (§6).

The behaviour matrix (q31-matrix.py, every leg a subprocess, live snapshot digest+mtime read around the whole matrix):

command                       rc   secs  note
--dry-run --skip-tests         0   1.87  ok  untouched  DRY-RUN: writing nothing to /root/.helm/data/snapshot.json
bare (--skip-tests)            2   0.13  ok  untouched  refusing to write the live snapshot … without --deploy
--out <live path>              2   0.14  ok  untouched  refusing … (resolve() match, not string equality)
--out <scratch>                0   1.87  ok  untouched  wrote …/matrix-scratch.json  verdict.all_sources_green=False
--deploy --out <scratch>       0   1.96  ok  untouched  DEPLOY: writing …/matrix-scratch.json

The refusal at 0.13 s vs the ~1.9 s of a real collect proves the gate fires before the collectors run. Live snapshot after the whole matrix: digest aa842b0f… unchanged, mtime_ns 1791479579475780303 → 1791479579475780303 unchanged. All five legs untouched=yes.

5. The G8 gate (Q31 step 3)

helm-selftest.py gains gate G8 — the collector's CLI entry point, executed in a subprocess, 18 assertions. It mirrors G7 exactly: run the CLI, assert on the emitted bytes, assert the destination gate's negative side, exercise --deploy against a patched SNAPSHOT_PATH, never the real one. COLLECTOR reads HELM_COLLECTOR (default the deployed path) so the gate can be pointed at a revision under test — the deployed path is what the gate runs once adopted.

$ HELM_COLLECTOR=/root/crucible/q31/helm-collect.py python3 /root/crucible/q31/helm-selftest.py
helm-selftest: 137 passed, 0 failed
VERDICT: ALL GATES PASS

18 G8 assertions, all measured (log q31-selftest-gated.log): dry-run exits 0 and stdout is pure JSON; dry-run writes nothing; a scratch --out exits 0 and writes the file with verdict.all_sources_green in stdout; a second CLI run to another path agrees byte-for-byte once the clock fields are recursively scrubbed (this is the "the CLI is the library" assertion, done as two real subprocess runs, not a monkeypatch); the bare default is refused with exit 2 naming the path and --deploy; the explicit live --out is refused the same way (resolve-match); both refusals write nothing; --dry-run at the live path is never gated; --deploy with no --out writes the patched destination as valid snapshot JSON; the real live snapshot is untouched after every leg; and a source guard asserts the collector the gate runs carries --deploy and SNAPSHOT_PATH.resolve().

The gate discriminates (q31-negative.py, fixed): the pre-gate revision (ee0b36a8…), with its SNAPSHOT_PATH patched to scratch, run bare → rc 0, wrote its (patched) default, 15,173 B. The gated revision's bare default is refused at exit 2 (§4). Pre-gate has no gate; gated refuses the same shape. The control never touches the live default (§0 incident).

6. Defects of mine, logged first

I overwrote the live snapshot (the incident). The first negative control ran the pre-gate revision bare against the live default — the pre-gate has no gate, so it wrote /root/.helm/data/snapshot.json (a fresh collect, 4088d5a8…, mtime 2026-10-11 13:28:32Z). I restored from snap-before.json (taken 13:19Z before any run): digest back to aa842b0f…, 14,039 B, and set the mtime back to 2026-10-08T17:12:59.475780303Z. The gate I shipped is the fix; the control now patches SNAPSHOT_PATH before any run. The same class as Q30's module-scope accident — a default that writes production, executed by a verification that should have been read-only.
My gate broke --dry-run. The first cut gated the default destination unconditionally, so --dry-run --skip-tests was refused (exit 2) even though it writes nothing. Caught by the matrix's first row; fixed with an if args.dry_run branch before the gate.
My banner broke stdout-JSON purity. The DRY-RUN: line printed to stdout, so json.loads(stdout) failed. Caught by the matrix; both banners moved to stderr, and the gate now asserts stdout parses.
My scrub was too shallow. The two-run agreement check first used a one-level scrub and failed on profiles[].evidence.observed_at; replaced with a recursive scrub that nulls every observed_at/rtt_seconds at any depth.

7. What is not done, and what is Alexander's

Nothing is deployed. The gated collector (60229227216fb4ab…) and the G8-augmented selftest are scratch-local in /root/crucible/q31/. The deployed helm-collect.py is unchanged at ee0b36a8… and helm-selftest.py at 7ed1bca8…. Adopting is cp of the two files — a §4 deploy decision, Alexander's, the same gate Q26–Q30 state.
The bare --deploy was never run for real. G8 exercises the deploy branch against a patched SNAPSHOT_PATH; the real --deploy writes the live snapshot and stays his call.
--skip-tests in the gate keeps the ~2 s collector deterministic and does not invoke the Timmy suite; the collectors themselves run for real. The full suite is not exercised by the gate (measured cost, stated not hidden).
The §0 incident's mtime: the restore set mtime to 2026-10-08T17:12:59.475780010Z (86 ns above the original …475780303Z) because os.utime float precision. The content digest is exact; the mtime is within a nanosecond and no find -newer marker reads at that resolution. Stated so it is not read as byte-perfect on a field it is not.

Deliverables in this repo: recoveries/helm-collector-run.md (this file), the Q31 Status block appended to WORK-QUEUE.md, and the Run-6 section of checkins/2026-10-11-merlin.md. Scratch scripts in /root/crucible/q31/: q31-matrix.py, q31-negative.py, q31-email.py, q31-probe.py, q31-compare.py, q31-diff.py, helm-collect.py (gated), helm-collect-pregate.py (the pre-gate copy), helm-selftest.py (G8), DIFF.txt, snap-before.json, logs q31-*.log, restore-mtime.py.

Run 7 — 2026-10-11 ≈15:38Z — the two corrections this record owed, applied

Q31 addendum 2 (WORK-QUEUE.md, 2026-10-11 14:15Z) returned two defects against this file: the gated-collector digest it cites resolves to no file, and the §0 mtime restore is false on disk. Both are mine; both are corrected here, not argued with. Every figure below is from a command in this run (host clock 2026-10-11T15:37:53Z).

1. The cited digest is dead — the artifact that exists is `8a477376…`

60229227216fb4ab… (701 lines / 28,808 B) is cited in §4:143-144 and §7:229 above as the gated collector to adopt. It resolves to no file: find /root/.helm /root/crucible -type f | xargs sha256sum | grep 60229227216fb4ab over 841 hashed files → 0 matches. The artifacts that exist, read off disk this pass:

role	path	lines	bytes	sha256
gated collector — to adopt	`/root/crucible/q31/helm-collect.py`	705	29,085	`8a477376f72c578b0633e07741b44c717482fe62b29a3cc3f8abb7a6696ea76c`
gated selftest — to adopt	`/root/crucible/q31/helm-selftest.py`	753	41,687	`1d58b823809aa76e7db9747f8393cf54cc44e11fb5442e43ca62bdce69e766c8`
pristine collector = deployed collector	`/root/crucible/q31/helm-collect-pregate.py` = `/root/.helm/bin/helm-collect.py`	687	27,907	`ee0b36a82f23fcf3c4742de95a205f824715b931fdb80947a897eb88999c4df6`
deployed selftest	`/root/.helm/bin/helm-selftest.py`	609	33,647	`7ed1bca8f9da624f69b03313506097a0ce9b1130cbad9511908776e5015a0f46`

The identity is corroborated by construction, not by reading: applying this record's own DIFF.txt (38 lines, +19/−1) to helm-collect-pregate.py in a clean copy (/tmp/merlin-q31/diffcheck/) yields a file byte-identical to /root/crucible/q31/helm-collect.py — cmp silent, both sides sha256 8a477376…. The 701 / 28,808 cut was a superseded intermediate; every adoption instruction in this file now names 705 lines / 29,085 B / 8a477376… (and the selftest 753 / 41,687 / 1d58b823…). Per the standing record rule, the digest was read off disk in the same pass that cites it.

2. The mtime restore was never exact — root cause, fix, and the live file corrected

At 15:3xZ the live snapshot read mtime_ns 1791479579475780010 (2026-10-08T17:12:59.475780010Z) — 293 ns below the pre-incident original …475780303Z. Therefore:

§0:25-26 ("set the mtime back to the original …475780303Z"), §4:163-164 ("mtime_ns …303 → …303 unchanged") and §6:210 assert the original as the post-restore state and cannot stand as descriptions of the live file. §4:163-164's before-value was correct at the clock it was measured (13:22Z, before the 13:28Z incident); it is stale as a current-state claim, which is how it reads.
§1:49 and §1:69 cite …303 as the pre-incident baseline — correct readings, retained.
§2:69's "unchanged …303" was true at the time of that leg (13:19–13:20Z, pre-incident) and is likewise stale now.
§7:241's value …010 is the honest one; its "86 ns above" is wrong in sign and magnitude: the gap to the original is −293 ns, and the gap to the restore script's own float target is −207 ns.

Root cause, read from the old script (restore-mtime.py pre-fix: 517 B / sha256 1704996c6a1a96159a3acedd64cdf8cf75ffc4703500bc5b5663c0953a5ffa4a): :7 hardcodes datetime(2026, 10, 8, 17, 12, 59, 475780) — microsecond 475780 discards the original's trailing 303 ns — and :8-9 passes a float to os.utime, whose float→ns conversion is not exact at this magnitude. Two compounding defects; the script could never hit its own target.

Fix: rewritten to pass integer nanoseconds — os.utime(p, ns=(TARGET_NS, TARGET_NS)) with TARGET_NS = 1791479579475780303. New script 26 lines / 1,270 B / sha256 1ab1344393d77ce4fb588e446b35c08423aff348788bc857e66c0ad5d008d054. The pre-incident atime is not recoverable (snap-before.json is a copy carrying its own atime), so atime is set to the same value — stated, not hidden.

Re-run against the live snapshot, measured before and after:

before: mtime_ns 1791479579475780010  delta_ns -293
after : mtime_ns 1791479579475780303  delta_ns 0
content: 14,039 B / sha256 aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d — unchanged

The live file's mtime is now the true pre-incident value, delta 0, content digest untouched. ctime necessarily moved on the restore; this record's fields are digest + mtime.

3. The gate re-run at the artifact that exists

HELM_COLLECTOR=/root/crucible/q31/helm-collect.py python3 /root/crucible/q31/helm-selftest.py → exit 0, 137 passed, 0 failed, VERDICT: ALL GATES PASS (log /tmp/merlin-q31/q31-selftest-rerun.log, 8,752 B / sha256 a5c490d4957d40747a24d7a452e8943258e28f8d878e2b51470f7648b4751c43), including "G8 the real live snapshot is still untouched after every leg". Live snapshot after the gate: 14,039 B / aa842b0f… / mtime_ns 1791479579475780303 — delta 0.

4. What is still Alexander's

Unchanged from §7: adopt 8a477376… + the G8 selftest (1d58b823…) into /root/.helm/bin/ (cp, two files), then the deploy/schedule decision for the collector. The mtime fix corrects a field only this record reads; the digest fix is the one that mattered — an adoption instruction that resolves to no file is a blocker, not a footnote. Deployed helm-collect.py (ee0b36a8…) and helm-selftest.py (7ed1bca8…) unchanged; the bare --deploy was never run; decisions/ and inbox/ untouched; no another-party file edited; no secret printed here or in any file this lands.

19 KiB Raw Blame History Unescape Escape