troupe-standup/harness-reflections/2026-10-25.md

# Harness reflection — 2026-10-25

## Inventory (last 7 days: 2026-10-18 → 2026-10-25)
512 Ralph run logs in `/root/.hermes/wizard-outputs/ralph/*.log` (command: `ls *.log | sed -E 's/.*-([0-9]{4}-[0-9]{2}-[0-9]{2})-([0-9]{6})\.log/\1/'`, parsed per-wizard in `/root/.hermes/cache/scratch/inv.py`):

- sol 63 GREEN / 1 EXHAUSTED · spark 61 / 3 · ledger 56 / 8 · forge 45 / 18 · sphinx 39 / 24 · null 31 / 31 · quill 24 / 40 · **scribe 0 / 64**
- `no CHECK line emitted (RED)` appears **617** times across those logs — the dominant RED event by an order of magnitude (next largest class: 7 occurrences of a CHECK with empty failure output).

## Contrast (recurring failure, >=2 runs)

**Same run, same loop, same 3-iteration budget, opposite outcome — and the difference is not the wizard's CHECK discipline, it is that one session never ran.**

- `quill-2026-10-25-0554.log:2` — `❌ no CHECK line emitted (RED)` → `:4` iter2 emits a real CHECK → `:5` `✅ GREEN after 2 iteration(s)`. Feedback worked: the wizard executed, saw the failure, fixed it.
- `scribe-2026-10-25-0554.log:2,4,6` — `❌ no CHECK line emitted (RED)` ×3 → `:7` `RALPH RESULT: EXHAUSTED (3/3) — ESCALATE to Timmy`. Nothing was fixed because nothing ever ran: `scribe-2026-10-25-055440-iter1.txt:39` `Duration: 25s`, `:40` `Messages: 2 (1 user, 0 tool calls)`, `:22` `Model 'meituan/longcat-2.0:free' isn't available on Nous Portal`.

The recurrence is the contrast: of 189 EXHAUSTED runs, **82 had a non-execution in every iteration** (scribe 64, null 6, quill 6, ledger 6; `/root/.hermes/cache/scratch/inv4.py` output: `EXHAUSTED runs where ALL iters tripped: 82`). Under the identical script, sol reached GREEN 63/64 times — so the loop's CHECK feedback is not what failed here.

Root cause is in the gate, not the wizards: `/root/.hermes/scripts/ralph-loop.sh:31-36` (pre-edit) treated *any* absent `CHECK:` as CHECK-discipline RED and fed back the literal string `(you emitted no CHECK: <command> line)`. For a dead model that feedback is false — the wizard never got a turn — so all three iterations burn on the same 404 and the run is escalated as a wizard fault. That is why the dead-model condition already recorded in `harness-reflections/2026-10-18.md:28` survived a full week: the runner kept re-labelling it as a wizard failure instead of surfacing it as infrastructure.

## Proposed edit (RETAINED — applied)

File: `/root/.hermes/scripts/ralph-loop.sh` (inserted between the wizard run and CHECK extraction; `bash -n` OK).

```bash
  # §9b INFRA: a session that never executed is not a CHECK-discipline RED.
  # A dead/unauthorised model exits in seconds with 2 messages / 0 tool calls and a provider 4xx.
  # Feeding that back as "(you emitted no CHECK: <command> line)" is a false diagnosis — the wizard
  # never got to try, so every iteration burns and the run escalates as a wizard fault.
  # Measured on the 512 runs of 2026-10-18..25: 82 runs tripped this on EVERY iteration (scribe 64/64,
  # null 6, quill 6, ledger 6); 0 of the 64 GREEN runs tripped it, so it does not cut a live run short.
  if grep -qE "Messages:[[:space:]]+[0-9]+ \(1 user, 0 tool calls\)" "$LOG" \
     || grep -qE "isn't available on Nous Portal|rejected the request and retrying won't help" "$LOG"; then
    echo "  ⛔ INFRA — session produced no work (model/provider failure, not a CHECK fault); not retrying"
    echo "RALPH RESULT: INFRA ($i/$MAX) — wizard never executed; fix its model/provider, do not count as RED"
    exit 3
  fi
```

Why this shape: the signal is the session footer, not the wizard's prose — `0 tool calls` means no file was written, no command run, so no CHECK could have been earned. Exiting on first detection (rather than retrying) is safe because the condition is deterministic per run: **0 of 64 GREEN runs would have tripped it** (`inv4.py`: `GREEN runs whose ITER1 would trip the detector (would be wrongly aborted): 0`). `wizard-fleet.sh:31` greps `RALPH RESULT` rather than exit codes, so the new `INFRA` status flows into the fleet log without touching the fleet script.

## Verification (executed, not asserted)

```
$ bash -n /root/.hermes/scripts/ralph-loop.sh        → SYNTAX OK
$ ralph-loop.sh _infra_probe "probe task" 3          (stub emitting the scribe footer)
  ⛔ INFRA — session produced no work (model/provider failure, not a CHECK fault); not retrying
  RALPH RESULT: INFRA (1/3) — wizard never executed; fix its model/provider, do not count as RED
  exit=3                                             (pre-edit: 3 iterations, EXHAUSTED, exit 1)
$ ralph-loop.sh _green_probe "probe task" 3          (stub emitting real work + a sound CHECK)
  running CHECK: test -f /root/troupe-standup/STANDING-AGREEMENT.md
  ✅ GREEN after 1 iteration(s)   exit=0             (healthy path unchanged)
```
Probe logs kept at `/root/.hermes/cache/scratch/infra-probe-evidence/`; the stub binaries were removed and the production output dir contains no probe files (`ls /root/.hermes/wizard-outputs/ralph/ | grep -c probe` → `0`).

## Evidence (file:line)

- `/root/.hermes/scripts/ralph-loop.sh:31-36` (pre-edit) — `if [ -z "$CHECK" ]` → `no CHECK line emitted (RED)` + `(you emitted no CHECK: <command> line)` + `continue`; no test of whether the session executed.
- `/root/.hermes/scripts/ralph-loop.sh:37-42` (post-edit) — the INFRA guard (regex `:37-38`, `exit 3` at `:42`), placed before CHECK extraction at `:44` and the genuine `no CHECK line emitted (RED)` path at `:46`.
- `/root/.hermes/scripts/ralph-loop.sh:63` (post-edit) — the CHECK execution branch the guard precedes; the guard cannot shadow a real CHECK because a run with tool calls never trips it (0/64 GREEN).
- `quill-2026-10-25-0554.log:2,4,5` vs `scribe-2026-10-25-0554.log:2,4,6,7` — the within-stamp contrast.
- `scribe-2026-10-25-055440-iter1.txt:22,39,40` — dead model, 25s, 2 messages / 0 tool calls.
- Live re-check today: `/root/.hermes/cache/scratch/infra-probe-evidence/scribe-live-2026-10-25.txt:7` `❌ Nous Portal rejected the request and retrying won't help: HTTP 404` `:8` `Model 'meituan/longcat-2.0:free' isn't available on Nous Portal` and a `Messages: 2 (1 user, 0 tool calls)` footer — the class is current, not historical.
- `/root/.hermes/cache/scratch/inv2.py`, `inv3.py`, `inv4.py`, `inv5.py` — the per-wizard tallies and the 82 / 0-false-positive counts quoted above.

## Also observed — not edited (one edit per reflection)

1. **scribe (and 6 null runs) are dead at the config layer**, not the lane-prompt layer: `/root/.hermes/profiles/scribe/config.yaml:2` `default: meituan/longcat-2.0:free`; Nous Portal now answers `HTTP 404: This model is no longer free`. 64/64 runs and 0 GREEN since 2026-10-18. The edit above stops the loop from mislabelling this, but scribe produces nothing until its model is repointed — a per-profile `config.yaml` model decision, deliberately left for Timmy/Alexander rather than a silent swap to another `:free` model that may itself be dead.
2. `/root/troupe-standup/tools/check_lint.py:290-292` recognises only `RALPH RESULT: GREEN` / `EXHAUSTED`; `INFRA` runs will leave its inferred outcome `UNKNOWN`. It does not mislabel them — it just does not count them, which is the honest reading.
3. `/root/troupe-standup/docs/ralph-loop.md:24-31` documents GREEN and the tautology fault but has no INFRA outcome; the spec text should gain it on a future pass.

VERDICT: retained 1 edit(s) — the runner counted a non-execution (dead model, 2 messages/0 tool calls) as a CHECK-discipline RED, burning all 3 iterations and escalating as a wizard fault in 82 of 512 runs (scribe 64/64) while 0 of 64 GREEN runs trip the new INFRA guard; `ralph-loop.sh` now exits INFRA on first detection, verified by probe (exit 3 on iteration 1, healthy path still GREEN).