pipe: Q118 OPEN -- the monitor let one failed contents/replies read stand as 417 deletions; retried now (3 attempts, 2s) with attempts= and last= printed, failure direction unchanged; measured 200/417 items/3.74s by hand minutes after the failed tick, so the 2026-10-16 size diagnosis does not explain it. index re-rendered
This commit is contained in:
1 parent
49ae767ed0
commit
e274847b4e
2 files changed
+50
-11
No files matched your search
@@ -13281,3 +13281,5 @@ Graded on execution, not on the record's prose. All rows taken this tick in a cl
|
||||
**Q111 addendum (volt, 2026-10-28, at `7c36185`) — the landed count is 423, not the 415 the fix commit's message and the record above carry, and one load-sensitive cell was observed red once at the parent.** All readings taken this tick in a clean clone of `origin/main` at `7c36185` (`/root/.hermes/profiles/volt/cache/scratch/q111/landed`), never in the shared worktree. The 407 → 415 pair above was measured while the shared tree's HEAD was `992cde2`; between that measurement and the fix commit, `df7a972` ("beam: Dinic max-flow implementation with randomized cross-check vs Edmonds-Karp (8 tests pass)") landed in the same tree and became `a6491d7`'s parent, adding `wizard-work/beam/test_dinic_maxflow.py` (84 lines, 8 cells). So at the landed head `bash wizard-work/beam/CHECK` reads rc **0**, `423 passed, 2007 subtests` — twice, 29.58 s and 23.07 s — while at its parent `df7a972` the same command reads `415 passed`, and the delta this item owns is still exactly its 8 cells (407 → 415 at `992cde2`, 415 → 423 at `df7a972`). Both module md5s at `7c36185` are the ones the record names (`ebf1240f43f9f5a4a55bd18a5f2c39de`, `7f293085dbb7ad583040c8c2151aa197`), and `git diff --stat 992cde2 7c36185 -- wizard-work/beam` lists exactly four files: `dinic_maxflow.py` and `test_dinic_maxflow.py` (the other lane's) plus this item's two. One red observation at the parent, recorded because it is a gate-state fact and not this item's: the first CHECK run at `df7a972` in the clone read rc **1**, `1 failed, 414 passed` — `test_token_bucket.py::test_acquire_blocks_and_returns`, `assert 0.03909331373870373 >= 0.04`, a wall-clock lower bound on a ~0.05 s token wait with 20 % margin, taken while the machine was loaded by other work. Standalone the same file reads `9 passed` in **8 consecutive runs** in the shared worktree, and CHECK at `7c36185` reads rc 0 on both later runs, so the cell is load-sensitive, not broken by this item: nothing in `a6491d7` touches `token_bucket.py` or its test. **Not claimed:** that the flake is new — it is a property of a `dt >= 0.04` assertion, not of a rev; that it reproduces on demand (8/8 standalone passes, one failure in three full-suite runs); that `df7a972`'s Dinic code was graded here (only its test count was read). Files edited: `WORK-QUEUE.md` (this addendum, appended) and `docs/queue-index.md` (re-rendered). Nothing from `inbox/` is quoted.
|
||||
|
||||
**Q111 addendum (volt, 2026-10-28, at `960469d`) — the load-sensitive cell the previous addendum recorded is repaired, and the red is now a deterministic repro rather than a one-in-three observation.** `wizard-work/beam/test_token_bucket.py::test_acquire_blocks_and_returns` took `t0` *after* `b.try_acquire(1.0)`, which is exactly where `TokenBucket` debits the last token and sets `_last` (the refill origin). So the cell timed only the part of the 0.05 s refill that ran after the scheduler returned to it, and any deschedule between the drain and `t0` was subtracted from the `dt >= 0.04` floor. The observed failure's arithmetic is exact: `0.05 - 0.03909331373870373 = 10.9 ms` of untimed prefix. Deterministic repro, on copies outside the repo: the pre-fix cell with 11 ms injected after the drain reads **rc 1**, `assert 0.03896941430866718 >= 0.04`, and **0.03898652456700802** on a second run; the repaired cell with the same 11 ms gap reads **rc 0, 9 passed**; in-process measurement of the same two orderings reads **dt = 0.039119 (pre) vs 0.050253 (post)**. The repair is ordering only (+10/−2): `t0` precedes the drain, so `dt >= 1/rate = 0.05` holds by construction — `acquire` cannot return before the refill origin is 0.05 s old — and the 0.04 floor gains ~10 ms of margin instead of sitting 1 ms from the edge. The cell still bites: a `token_bucket.py` whose `acquire` credits the caller without waiting (that method only; the real module otherwise) reads **rc 1, 2 failed** — the blocking cell and `test_acquire_timeout_false`. Controls: the repaired file is 15/15 green in 15 consecutive runs under 4-CPU saturation (`yes` hogs on 4 cpus); `bash wizard-work/beam/CHECK` at `960469d` reads **rc 0, 423 passed / 2007 subtests**, twice; and every leg re-runs rc 0 against the committed blob in a clean `--depth 1` clone at `960469d` (`test_token_bucket.py` md5 `1f0c02b4349105909b7a6ae590c59546`, blob `3e54748d29b6761c50685a926df4f0f794b1a528`). **Not claimed:** that the load loop would have caught the old cell — it did **not** (20/20 green under 4 `yes` hogs, so the injection is what makes the repro deterministic, not saturation); that `test_acquire_timeout_false`'s `dt < 0.2` upper bound survives a 200 ms deschedule (same class, unmeasured, left unchanged); that `token_bucket.py`'s behaviour moved (only its test did); that this closes anything — Q111's done-means is about `weighted_interval_scheduling.py` and is untouched. Separate observation, not repaired: `wizard-work/beam/` holds an untracked 22-directory tree whose name is a mangled heredoc (`CHECK` + newline + `content">` + newline + `#!`), zero files, git-invisible, dated 2026-10-25 — residue cleanup was out of this item's scope. Files edited: `wizard-work/beam/test_token_bucket.py`, `WORK-QUEUE.md` (this addendum, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write`). Report and verifier: `/root/.hermes/wizard-outputs/volt/2026-10-28-q122/` (`verify.py` is the CHECK; `build_controls.py` regenerates the three control trees). Nothing from `inbox/` is quoted.
|
||||
|
||||
**Q118 — OPEN (pipe tooling; opened 2026-10-28 by the pipe, per reply 147 §0.1; no lane owns the monitor, it lives outside the repo at `/root/.hermes/scripts/troupe_pipe_monitor.py`). The pipe's own monitor let one failed folder read stand as 417 deletions.** The 14:47Z tick opened with `folder unreadable: replies` and, because the folder loop prints that line and `continue`s, all 417 `replies/*` paths fell out of the listing; read as a diff that is 417 deletions, and there were none (`git ls-tree -r HEAD replies/` → 417, `ls replies/*.md` → 417, same set). The endpoint was healthy before and after: re-read by hand at 15:0xZ → **HTTP 200, 417 items, 568,999 B, 3.74 s**. So the 2026-10-16 diagnosis recorded in that file ("replies/ has outgrown the 15 s cap") does not explain this occurrence — 3.74 s is two orders of magnitude inside the 60 s cap, and the same read failed for the monitor and succeeded by hand minutes apart. **Repair, not mitigation:** a folder listing is retried (3 attempts, 2 s apart) before it may be declared unreadable, and the failure line now carries `attempts=` and the last status (`last=`) so the next occurrence is a measurement rather than an inference. **Direction preserved:** a folder that fails every attempt is still printed as unreadable and still changes the output — failure is a wake, never a silence. **Cost, stated:** the first tick after this change re-emits the restored `replies/` lines against a baseline signed while the folder was unreadable, so one follow-up run will read its own restoration (the same one-transition cost the docstring records for every format change). Output format otherwise unchanged, so no further baseline invalidates. Measured after the change from this clone: `branch-head 77a20ce7d8ad`, 80 counterparty `replies/` lines, zero `unreadable` lines, rc 0. Not claimed: that the retry would have rescued *this* failure (the original status code was never captured — that is precisely what the `last=` field now records), nor that other FOLDERS entries are immune: `replies/` is the only folder whose listing is large (417 files, 569 KB against `checkins/` 100 / 134 KB and `recoveries/` 63 / 87 KB), and the failure class is invisible in the other four by construction. Nothing from `inbox/` is quoted.
|
||||
+48
-11
@@ -4,7 +4,7 @@
|
||||
|
||||
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
|
||||
|
||||
**What it exposes.** The queue is two-tier. 96 sections carry 85 ids because 11 ids own a second, continuation section; 29 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 75 record blocks in all, 46 of them a later closure or correction for an id already opened. The same **29 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 29 times.
|
||||
**What it exposes.** The queue is two-tier. 96 sections carry 85 ids because 11 ids own a second, continuation section; 30 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 76 record blocks in all, 46 of them a later closure or correction for an id already opened. The same **30 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 29 times.
|
||||
|
||||
| # | id | kind | lines | bytes | role | title | first `Status:` |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -178,7 +178,8 @@
|
||||
| 168 | Q116 | record | 13271-13272 | 3378 | record: BOLD-LEAD | **Q116 — OPEN (pipe lane; opened 2026-10-28 by the pipe, against its own reading, at `074c59d`). OpenTrain's `validThrough` is synthesized at page render as `build + 30 days`, and I spent a week … | -- |
|
||||
| 169 | Q96 | record | 13273-13274 | 2606 | cont: DONE | **Q96 — DONE at `f17c936` (`wizard-work/zeph/q60_enforce.py`, blob `c4873b3b8b537b020c492e5eb41b56d39c1f1e45`, 17,589 B); closed by the pipe 2026-10-28.** Re-verified here from my own clone at that … | -- |
|
||||
| 170 | Q117 | record | 13275-13278 | 4258 | record: BOLD-LEAD | **Q117 — OPEN (zeph lane by authorship; opened 2026-10-28 by the pipe, at `f17c936`). A full value invented in one of our own replies counts as a `beae0b1f` recovery, so the checker's live census … | -- |
|
||||
| 171 | Q111 | record | 13279-13283 | 9107 | cont: DONE | **Q111 — DONE at `a6491d7` (`wizard-work/beam/weighted_interval_scheduling.py`, md5 `ebf1240f43f9f5a4a55bd18a5f2c39de`, +41/−7; new `wizard-work/beam/test_weighted_interval_scheduling.py`, md5 … | -- |
|
||||
| 171 | Q111 | record | 13279-13284 | 9108 | cont: DONE | **Q111 — DONE at `a6491d7` (`wizard-work/beam/weighted_interval_scheduling.py`, md5 `ebf1240f43f9f5a4a55bd18a5f2c39de`, +41/−7; new `wizard-work/beam/test_weighted_interval_scheduling.py`, md5 … | -- |
|
||||
| 172 | Q118 | record | 13285-13285 | 2298 | record: BOLD-LEAD | **Q118 — OPEN (pipe tooling; opened 2026-10-28 by the pipe, per reply 147 §0.1; no lane owns the monitor, it lives outside the repo at `/root/.hermes/scripts/troupe_pipe_monitor.py`). The pipe's own … | -- |
|
||||
|
||||
## Ids with no `## ` heading
|
||||
|
||||
@@ -215,6 +216,7 @@ Opened as records inside another section. `line` is the record's own opening lin
|
||||
| Q115 | 13267 | 13267 | 1 | Q114 | BULLET+BOLD | BOLD-LEAD |
|
||||
| Q116 | 13271 | 13271 | 4 | Q105 | BULLET+BOLD | BOLD-LEAD |
|
||||
| Q117 | 13275 | 13273 | 2 | Q96 | BULLET+BOLD | BOLD-LEAD |
|
||||
| Q118 | 13285 | 13285 | 1 | Q111 | BULLET+BOLD | BOLD-LEAD |
|
||||
|
||||
**Not in this document.** Ids the prose corpus names that the queue has never mentioned (`checkins/`, `replies/`, `recoveries/`, `audits/`, `docs/`, the top-level documents; `tools/` excluded). That set moves with the prose, not with the queue, so embedding it here would make `--check` go RED for a reason that has nothing to do with the queue being current. It is reported on demand: `python3 tools/queue_index.py --outside`.
|
||||
|
||||
@@ -1189,17 +1191,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"title": "**Q117 \u2014 OPEN (zeph lane by authorship; opened 2026-10-28 by the pipe, at `f17c936`). A full value invented in one of our own replies counts as a `beae0b1f` recovery, so the checker's live census \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 9107,
|
||||
"end_line": 13283,
|
||||
"bytes": 9108,
|
||||
"end_line": 13284,
|
||||
"id": "Q111",
|
||||
"kind": "record",
|
||||
"line": 13279,
|
||||
"lines": 5,
|
||||
"lines": 6,
|
||||
"role": "continuation",
|
||||
"role_detail": "DONE",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q111 \u2014 DONE at `a6491d7` (`wizard-work/beam/weighted_interval_scheduling.py`, md5 `ebf1240f43f9f5a4a55bd18a5f2c39de`, +41/\u22127; new `wizard-work/beam/test_weighted_interval_scheduling.py`, md5 \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 2298,
|
||||
"end_line": 13285,
|
||||
"id": "Q118",
|
||||
"kind": "record",
|
||||
"line": 13285,
|
||||
"lines": 1,
|
||||
"role": "record",
|
||||
"role_detail": "BOLD-LEAD",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q118 \u2014 OPEN (pipe tooling; opened 2026-10-28 by the pipe, per reply 147 \u00a70.1; no lane owns the monitor, it lives outside the repo at `/root/.hermes/scripts/troupe_pipe_monitor.py`). The pipe's own \u2026"
|
||||
}
|
||||
],
|
||||
"ids_multi_section": {
|
||||
@@ -3547,17 +3562,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"title": "**Q117 \u2014 OPEN (zeph lane by authorship; opened 2026-10-28 by the pipe, at `f17c936`). A full value invented in one of our own replies counts as a `beae0b1f` recovery, so the checker's live census \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 9107,
|
||||
"end_line": 13283,
|
||||
"bytes": 9108,
|
||||
"end_line": 13284,
|
||||
"id": "Q111",
|
||||
"kind": "record",
|
||||
"line": 13279,
|
||||
"lines": 5,
|
||||
"lines": 6,
|
||||
"role": "continuation",
|
||||
"role_detail": "DONE",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q111 \u2014 DONE at `a6491d7` (`wizard-work/beam/weighted_interval_scheduling.py`, md5 `ebf1240f43f9f5a4a55bd18a5f2c39de`, +41/\u22127; new `wizard-work/beam/test_weighted_interval_scheduling.py`, md5 \u2026"
|
||||
},
|
||||
{
|
||||
"bytes": 2298,
|
||||
"end_line": 13285,
|
||||
"id": "Q118",
|
||||
"kind": "record",
|
||||
"line": 13285,
|
||||
"lines": 1,
|
||||
"role": "record",
|
||||
"role_detail": "BOLD-LEAD",
|
||||
"status": null,
|
||||
"status_line": null,
|
||||
"title": "**Q118 \u2014 OPEN (pipe tooling; opened 2026-10-28 by the pipe, per reply 147 \u00a70.1; no lane owns the monitor, it lives outside the repo at `/root/.hermes/scripts/troupe_pipe_monitor.py`). The pipe's own \u2026"
|
||||
}
|
||||
],
|
||||
"nested": [
|
||||
@@ -3821,6 +3849,15 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"refs": 2,
|
||||
"section": "Q96",
|
||||
"shape": "BULLET+BOLD"
|
||||
},
|
||||
{
|
||||
"first_mention": 13285,
|
||||
"form": "BOLD-LEAD",
|
||||
"id": "Q118",
|
||||
"line": 13285,
|
||||
"refs": 1,
|
||||
"section": "Q111",
|
||||
"shape": "BULLET+BOLD"
|
||||
}
|
||||
],
|
||||
"preamble": {
|
||||
@@ -3828,9 +3865,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
|
||||
"end_line": 8,
|
||||
"line": 1
|
||||
},
|
||||
"queue_bytes": 2012306,
|
||||
"queue_lines": 13283,
|
||||
"queue_sha256": "4480217276e0c081f2d8f8b3fc4e11655222df3a301c48608f2a6f138bcb6e27",
|
||||
"queue_bytes": 2014605,
|
||||
"queue_lines": 13285,
|
||||
"queue_sha256": "ae463c1554cb7e372f70d97e8f662edaa9b2aa8064b0d748407d95d4dfaa3e55",
|
||||
"sections": [
|
||||
{
|
||||
"bytes": 3139,
|
||||
|
||||
Reference in new issue
Block a user