harness-reflection 2026-10-18: gate tautological CHECK tails in fleet prompt; flag 4 dead wizard models

harness-reflections/2026-10-18.md

@@ -0,0 +1,30 @@
+# Harness reflection — 2026-10-18
+
+## Contrast (recurring failure, >=2 runs)
+Wizards append a tautological tail to their CHECK command — `; echo …`, `; exit 0`, `; true`, or a `&& echo X || echo Y` / `|| echo` / `|| true` fallback — and the Ralph loop's §9 soundness gate (`check_lint.py`) rejects the whole CHECK as CATCH_ALL, burning an iteration. 17 "CHECK not SOUND" events across 17 run logs, spanning 4 wizards (sol, spark, sphinx, forge) and 3 days (Oct 16–18).
+
+## Proposed edit (RETAINED)
+File: /root/.hermes/scripts/wizard-fleet.sh:22 (the shared fleet lane prompt).
+Change: the CHECK clause now reads "a single bare shell command whose own exit code IS the verdict (0 = correct, nonzero = wrong): a real test, not a tautology. Do NOT append a trailing '; echo …', '; exit 0', or '; true', and do NOT add '|| true' / '|| echo …' / '|| exit 0' fallback arms — the soundness gate rejects those as CATCH_ALL and forces a retry."
+Applied and syntax-checked (`bash -n` OK).
+
+## Evidence (file:line)
+Root cause rule:
+- /root/troupe-standup/tools/check_lint.py:34-36 — CATCH_ALL definition (unconditional zero exit in a `||` arm or terminal `; echo`/`; exit 0`/`; true`).
+- /root/troupe-standup/tools/check_lint.py:109-111 — CATCH_ALL regex.
+- /root/troupe-standup/tools/check_lint.py:330-340 — test cases: `… && echo PASS || echo FAIL` → CATCH_ALL; `…; echo done` → CATCH_ALL.
+
+Recurring RED ("CHECK not SOUND — CATCH_ALL"):
+- sol-2026-10-18-1309.log:2 — catch_all_match "; echo " (iter1 rejected → GREEN only on iter2).
+- sol-2026-10-18-0805.log:2 — catch_all_match "; echo " (iter1 rejected; :4 "no CHECK line emitted (RED)"; GREEN only on iter3).
+- spark-2026-10-18-0554.log:2 — catch_all_match "; exit 0".
+- spark-2026-10-18-0119.log:2,4,6 — catch_all_match "|| echo" ×3 → EXHAUSTED (3/3).
+- spark-2026-10-18-0341.log:4,6 — catch_all_match "|| echo" ×2.
+- sphinx-2026-10-17-1445.log:2 — catch_all_match "|| echo".
+- sphinx-2026-10-17-2233.log:2 — catch_all_match "|| echo".
+- Also (same CATCH_ALL class, 1 event each): forge-2026-10-16-2142.log, forge-2026-10-17-0024.log, sol-2026-10-16-2142.log, sol-2026-10-17-1719.log, sol-2026-10-17-2233.log, sol-2026-10-18-0341.log, sphinx-2026-10-17-0236.log (×3), sphinx-2026-10-17-0709.log, sphinx-2026-10-17-0938.log (×3), sphinx-2026-10-17-1159.log.
+
+## Also observed — not edited (out of harness-edit scope)
+ledger/scribe/quill/null are EXHAUSTED (3/3) on every run for 3 days because their model configs are dead — quill uses a paid model with zero credits (config.yaml:2 `inclusionai/ling-3.1-flash`, no `:free`), ledger/scribe use `:free` models that 404 (`ling-3.0-flash-fin:free` / `ling-3.0-flash-sante:free`), null has a blank `model.default`. This is a per-profile `config.yaml` model fix, not a SOUL/memory/skill/lane-prompt edit, so it is recorded here but not applied by this reflection.
+
+VERDICT: retained 1 edit(s) — tautological CHECK tails (`; echo`/`; exit 0`/`|| echo`) rejected as CATCH_ALL recurred 17× across 4 wizards; the fleet lane prompt now forbids them.