207 lines
14 KiB
Markdown
207 lines
14 KiB
Markdown
# Q33 — Split the newest-file pointer out of `evidence.source_path`
|
||
|
||
> Screen. Every claim below carries a path, a digest, a URL, or a command with its output, or the word
|
||
> `unverified`. Nothing secret is printed here or in the files this lands.
|
||
|
||
Merlin, 2026-10-11 ≈22:0x–22:3x UTC. Scratch tree `/root/crucible/q31/`; re-runnable with
|
||
`python3 q33-paired.py` then `python3 q33-before-after.py` (the gate:
|
||
`HELM_COLLECTOR=/root/crucible/q31/helm-collect.py python3 helm-selftest.py`).
|
||
|
||
## Did
|
||
|
||
Claimed and finished **Q33 — split the newest-file pointer out of `evidence.source_path`** (the queue's
|
||
only OPEN item, opened 2026-10-11 by the pipe on Q32's residue; named to the Helm lane. Q20 is
|
||
zephyrus-sparkborn's, Q9 blocked on the audio handover, Q13/Q19 gated on Alexander).
|
||
|
||
**The change, three edits in the collector:**
|
||
|
||
1. `profiles[]`: `evidence.source_path` was `latest_path or str(entry)` — the newest file under the
|
||
profile's `sessions/` / `cron/output/` / `logs/`, mutable by construction. Now
|
||
`"source_path": str(entry)` (the profile directory, a stable citation) with the pointer beside it as
|
||
`"latest_active_path": latest_path` (`None` when the profile has no files).
|
||
2. `members[]`: `evidence.source_path` was `info["file"]` — the member's own newest check-in, i.e. the
|
||
pointer. Now `"source_path": str(d)` (the check-ins directory the roster is enumerated from) with the
|
||
pointer as `"newest_checkin_file": info["file"]`.
|
||
3. The module docstring states the rule: `evidence.source_path` is a stable citation; a newest-file
|
||
pointer is never smuggled into it.
|
||
|
||
**Design decision, stated because the queue's parenthetical is loose.** The queue writes that
|
||
`source_path` becomes "a stable citation for every block — profile directory or the member's own
|
||
check-in file". The member's own check-in file *is* the newest file — that is the pointer, so it cannot
|
||
also be the citation. The stable citation for a member block is therefore the check-ins directory (the
|
||
exact analogue of the profile directory), and the pointer is preserved in full as
|
||
`evidence.newest_checkin_file`. No information is lost: the before/after dump below shows the member's
|
||
check-in file moving from `source_path` to `newest_checkin_file`, value intact.
|
||
|
||
The pointer lives **inside** `evidence` (with `observed_at` and `source_path`) rather than as a sibling
|
||
of the profile/member dict: it qualifies the evidence block, and the gate's scrub nulls by key name at
|
||
any depth, so the declared-key mechanism works unchanged.
|
||
|
||
**The gate change:** the Q32 pointer carve-out — which nulled `evidence.source_path` inside any dict
|
||
carrying a live mtime reading — is **deleted**. `SCRUB_KEYS` gains the two named pointer fields as
|
||
ordinary declared keys, and eight new behavioural G8 checks plant a newer file between two collects of
|
||
scratch trees and assert the pointer field moves while `evidence.source_path` does not.
|
||
|
||
Identities (`sha256`, this host):
|
||
|
||
| artifact | before | after | lines | net |
|
||
|---|---|---|---|---|
|
||
| `helm-collect.py` | `8a477376f72c578b…` (29,085 B) | **`4e45021d80395d3f03fc58d7e99610e3c1c0d2fcf9c85f1db50b16cfd2043c6c`** (29,954 B) | 705 → 720 | +18 / −3 |
|
||
| `helm-selftest.py` | `9ce6a4e971525fdd…` (48,468 B) | **`4a442f625961a1308302d5606b6b9eb64409cbfd472aeb1293caa7f580ad7fa3`** | 868 → 929 | +107 / −47 |
|
||
|
||
Diffs: `DIFF-Q33-COLLECTOR.txt`, `DIFF-Q33-SELFTEST.txt` (scratch). Pre-change copies preserved as
|
||
`helm-collect-q33-before.py` / `helm-selftest-q33-before.py` (verified byte-identical to the identities
|
||
above before any edit).
|
||
|
||
**Consumer walk, done in the same run (the queue's Status note required it):**
|
||
|
||
| consumer | what it reads | verdict |
|
||
|---|---|---|
|
||
| `/root/.helm/bin/helm-render.py:264` | top-level `evidence.source_path` per source card | **unaffected** — the six top-level citations (dirs, files, command strings) are unchanged; the per-profile rows render `last_active_at` (`:291`) and the member table renders `last_checkin_date` / `last_checkin_at` (`:354-356`), never the nested `source_path` |
|
||
| `/root/.helm/bin/helm-validate.py:73-74` | requires `evidence.source_path` present in every block | **passes** — `python3 /root/.helm/bin/helm-validate.py q33-after.json` → `OK: q33-after.json — valid, all green-rule checks passed, no secrets detected`, exit 0 |
|
||
| `/root/.helm/schema/helm-snapshot.schema.json` | `$defs.evidence` — required `observed_at`/`source_path`, **no `additionalProperties: false`** | **no change needed** — the two new keys are admitted by the open object; required keys unchanged |
|
||
| `/root/.helm/bin/helm-selftest.py` (deployed) | fixture snapshots with literal `source_path` values | **unaffected** — the fixtures name top-level citations, not per-profile pointers |
|
||
| `/root/troupe-harness/mercury-q31-summary.py`, `mercury_standup.py`, `/root/troupe-verify/q32/timmy-verify-q32.sh` | read the snapshot for verdict/harness purposes | **no read of the changed fields** (`grep` for `source_path|evidence|profiles|members` in the two harness files → no match) |
|
||
|
||
No other consumer of `helm.snapshot/1` exists on the host: `grep -rln 'helm.snapshot'` over `/root`
|
||
(`--include` py/js/sh/json, excluding node_modules/.git/crucible/__pycache__/sessions) returns exactly
|
||
the schema, the live snapshot, `helm-validate.py`, `helm-selftest.py`, `helm-collect.py` and one verify
|
||
fixture.
|
||
|
||
## Evidence (measured, not accepted)
|
||
|
||
**The gate discriminates — the pre-Q33 collector fails all eight new checks.**
|
||
`HELM_COLLECTOR=/root/crucible/q31/helm-collect-q33-before.py python3 helm-selftest.py` → exit **1**,
|
||
`helm-selftest: 151 passed, 9 failed` (log `q33-gate-old-collector.log`). The nine failures: the eight
|
||
Q33 behavioural checks plus the declared-key source check naming the two absent literals. Three of the
|
||
eight, verbatim from the log:
|
||
|
||
```
|
||
FAIL G8 Q33: a profile's evidence.source_path is the profile directory
|
||
-> '/tmp/tmp86csye9n/q33-split/profiles/alpha/sessions/new.jsonl'
|
||
FAIL G8 Q33: evidence.source_path does not move when a newer file lands
|
||
'/tmp/tmp86csye9n/q33-split/profiles/alpha/sessions/new.jsonl' -> '.../newer.jsonl'
|
||
FAIL G8 Q33: the member's own newest check-in is emitted as evidence.newest_checkin_file -> None
|
||
```
|
||
|
||
That is the defect, live: `source_path` *is* the newest file, and it *moves* when a newer file lands.
|
||
|
||
**Six consecutive green invocations, each 160 passed / 0 failed.**
|
||
`HELM_COLLECTOR=/root/crucible/q31/helm-collect.py python3 helm-selftest.py`, runs 1–6, all exit 0,
|
||
`helm-selftest: 160 passed, 0 failed`, `VERDICT: ALL GATES PASS`. Log digests:
|
||
|
||
| run | log | sha256 (first 16) |
|
||
|---|---|---|
|
||
| 1 | `q33-gate-run-1.log` | `0e5dedcd1df85867` |
|
||
| 2 | `q33-gate-run-2.log` | `c5cdfb2ca21a0528` |
|
||
| 3 | `q33-gate-run-3.log` | `788ad40353a1a1302` |
|
||
| 4 | `q33-gate-run-4.log` | `a0409de02a9da27e` |
|
||
| 5 | `q33-gate-run-5.log` | `fd49ec1a538c143a` |
|
||
| 6 | `q33-gate-run-6.log` | `7319abcb2dea80ae` |
|
||
|
||
All eight Q33 checks PASS in run 1 (names in the log): source_path-is-the-directory (profile, member),
|
||
pointer-emitted-as-its-own-field (both), source_path-does-not-move (both), pointer-moves (both).
|
||
|
||
**Eight paired runs, zero differing leaf paths, no pointer carve-out.**
|
||
`python3 q33-paired.py` (`Q33-PAIRED.txt`): the gate's own `_scrub` is extracted from the selftest
|
||
source by AST, so the paired script cannot drift from the gate; the script asserts the carve-out is
|
||
gone (`'source_path=None' not in src`) and both pointer keys are declared. **8 pairs, 0 differing leaf
|
||
paths** under the gate's scrub. Controls: the pre-Q32 scrub set reds 1 of 8 pairs on
|
||
`sources.cron_jobs.age_seconds` / `sources.hermes_profiles.age_seconds` (the age-boundary class is
|
||
still live without the declared keys — the defect the Q32/Q33 keys close); the pre-Q33 collector under
|
||
the same new scrub showed **0** differing leaf paths in 3 pairs, which is stated honestly: the
|
||
pointer's volatility is rare (no file landed mid-pair), so the *paired-level* control did not fire —
|
||
the non-vacuity proof is the gate's planted-file checks, which are red on the old collector by
|
||
construction.
|
||
|
||
**Before/after on the same live data** (`python3 q33-before-after.py` → `Q33-BEFORE-AFTER.txt`): the
|
||
pre-Q33 collector and the patched one both run to scratch `--out` paths, never the live default.
|
||
|
||
```
|
||
pre-Q33 collector helm-collect-q33-before.py: 29085 B, sha256 8a477376f72c578b…
|
||
patched collector helm-collect.py: 29954 B, sha256 4e45021d80395d3f…
|
||
pre-Q33 snapshot q33-before.json: 18523 B, sha256 b050b356bd08a131…
|
||
patched snapshot q33-after.json: 20141 B, sha256 4f736f5498c08f75…
|
||
live snapshot UNCHANGED: 14039 B, sha256 aa842b0fd88037f8…, mtime_ns 1791479579475780303
|
||
```
|
||
|
||
One profile block's evidence, before → after (names/values from the dump; the full block is in the log):
|
||
|
||
```
|
||
BEFORE: {"observed_at": "2026-10-11T22:17:26.503Z", "source_path": "<profile>/logs/gateway.log"}
|
||
AFTER: {"latest_active_path": "<profile>/logs/gateway.log",
|
||
"observed_at": "2026-10-11T22:17:28.341Z", "source_path": "<profile>"}
|
||
```
|
||
|
||
One member block's evidence, before → after:
|
||
|
||
```
|
||
BEFORE: {"observed_at": "…", "source_path": "/root/troupe-standup/checkins/2026-10-07-cassian-q5-census.md"}
|
||
AFTER: {"newest_checkin_file": "/root/troupe-standup/checkins/2026-10-07-cassian-q5-census.md",
|
||
"observed_at": "…", "source_path": "/root/troupe-standup/checkins"}
|
||
```
|
||
|
||
Key delta: **+24 keys / −0** — 20 × `profiles[].evidence.latest_active_path` + 4 ×
|
||
`members[].evidence.newest_checkin_file`. Live counts in the patched fresh collect: **20/20** profiles
|
||
carry a non-None `latest_active_path` and a directory `source_path`; **4/4** members carry a non-None
|
||
`newest_checkin_file` and the check-ins directory as `source_path`.
|
||
|
||
**Live snapshot delta 0 across the whole run** (before and after every leg):
|
||
14,039 B / sha256 `aa842b0fd88037f83115b4a54d617f27666f4de06ef84f9a64c8fd0a5f04656d` / mtime_ns
|
||
`1791479579475780303`. Deployed bin unchanged: `helm-collect.py` `ee0b36a82f23fcf3…`,
|
||
`helm-selftest.py` `7ed1bca8f9da624f…`, `helm-render.py` `5c67473d89283c3d…`. Nothing deployed;
|
||
`decisions/` and `inbox/` untouched; no another-party file edited; no X page fetched.
|
||
|
||
## Blocked
|
||
|
||
Nothing blocked the run. Limits, stated not hidden:
|
||
|
||
- **Adoption is Alexander's, and the deployed tree still carries the old shape.** Until the `cp`,
|
||
production emits `source_path` as a pointer and the deployed gate keeps the Q32 carve-out. The
|
||
scratch pair is complete and gated; the publish gate is §4.
|
||
- **The renderer does not display the pointer.** A deliberate minimal-change decision: the renderer's
|
||
evidence line shows the stable citation, which is the honest thing to print; showing the newest file
|
||
beside it is a renderer change nobody asked for. Named as an option, not done.
|
||
- **The pointer's `None` case is untested against a live empty profile** (a profile with no files under
|
||
`sessions/` / `cron/output/` / `logs/` emits `latest_active_path: null`). The gate's planted tree
|
||
always has files; the live collect has 20/20 non-None. `unverified` against a real empty profile.
|
||
- **No fuzzer.** The behavioural checks plant two deterministic file shapes; the guarantee is stated
|
||
over those shapes plus the two-run agreement check, not over an exhaustive input census.
|
||
- **The gate runs with `--skip-tests`** (the standing Q31 caveat).
|
||
|
||
## Defects of mine, logged first
|
||
|
||
1. **The first version of the four pointer checks read the wrong nesting level.** They queried
|
||
`_pr1.get("latest_active_path")` — the profile dict's top level — while the collector emits the
|
||
pointer inside `evidence`. The checks failed `-> None` while the field was present one level down
|
||
(found by instrumenting the gate: `_pr1` printed with the field set while the comparison returned
|
||
None). Same class as the 2026-10-11 `card()` incident: the check read the wrong object, the gate
|
||
looked broken, and the failure message was misleading. Fixed by moving the reads to
|
||
`["evidence"]`, and **the five green-run logs produced by the buggy gate were discarded and
|
||
regenerated** — the six logs in the table above are all from the fixed file, identity
|
||
`4a442f62…`. The debug prints were removed before the final runs.
|
||
2. **The paired-level before-control did not fire** (0 differing leaf paths in 3 pairs on the pre-Q33
|
||
collector). Stated above rather than re-rolled until it went red: the non-vacuity proof is the
|
||
gate's planted-file checks, which fail deterministically on the old collector.
|
||
|
||
## Next
|
||
|
||
- **Alexander's call, §4:** adopt the pair into `/root/.helm/bin/` — collector `4e45021d…` +
|
||
selftest `4a442f62…` (superseding `8a477376…` + `9ce6a4e9…`). This is the artifact Q31's adoption
|
||
ask has been waiting on; after it, collect+render scheduling is safe to take on its own evidence.
|
||
Scratch until he says so.
|
||
- **Q7, measured:** Q20 open and unclaimed (zephyrus-sparkborn's named lane); Q9 blocked on the audio
|
||
handover; Q13/Q19 gated on Alexander. The unclaimed executable set is empty again, so a successor is
|
||
owed; this record opens none — the same standing pattern as Q26–Q32, and the natural candidate would
|
||
be this run's own residue (the renderer's pointer display, or the `latest_active_path: null` case),
|
||
which the successor should open on a fresh measurement.
|
||
|
||
## Ask
|
||
|
||
1. **Alexander — adopt `4e45021d…` + `4a442f62…` into `/root/.helm/bin/`?** It is the difference
|
||
between a snapshot whose `source_path` means two different things and one whose citations are
|
||
stable, with a gate that proves the split behaviourally. Scratch until you say so.
|
||
2. **Nothing asked of zephyrus-sparkborn this tick.** His lane is Q20.
|
||
|
||
— Merlin, 2026-10-11
|