From 69f3b561198fdf07b23cad2c5d0ec90332c3864f Mon Sep 17 00:00:00 2001 From: Timmy Date: Sun, 18 Oct 2026 13:19:35 +0000 Subject: [PATCH] harness-reflection 2026-10-18: gate tautological CHECK tails in fleet prompt; flag 4 dead wizard models --- harness-reflections/2026-10-18.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 harness-reflections/2026-10-18.md diff --git a/harness-reflections/2026-10-18.md b/harness-reflections/2026-10-18.md new file mode 100644 index 0000000..7770e41 --- /dev/null +++ b/harness-reflections/2026-10-18.md @@ -0,0 +1,30 @@ +# Harness reflection — 2026-10-18 + +## Contrast (recurring failure, >=2 runs) +Wizards append a tautological tail to their CHECK command — `; echo …`, `; exit 0`, `; true`, or a `&& echo X || echo Y` / `|| echo` / `|| true` fallback — and the Ralph loop's §9 soundness gate (`check_lint.py`) rejects the whole CHECK as CATCH_ALL, burning an iteration. 17 "CHECK not SOUND" events across 17 run logs, spanning 4 wizards (sol, spark, sphinx, forge) and 3 days (Oct 16–18). + +## Proposed edit (RETAINED) +File: /root/.hermes/scripts/wizard-fleet.sh:22 (the shared fleet lane prompt). +Change: the CHECK clause now reads "a single bare shell command whose own exit code IS the verdict (0 = correct, nonzero = wrong): a real test, not a tautology. Do NOT append a trailing '; echo …', '; exit 0', or '; true', and do NOT add '|| true' / '|| echo …' / '|| exit 0' fallback arms — the soundness gate rejects those as CATCH_ALL and forces a retry." +Applied and syntax-checked (`bash -n` OK). + +## Evidence (file:line) +Root cause rule: +- /root/troupe-standup/tools/check_lint.py:34-36 — CATCH_ALL definition (unconditional zero exit in a `||` arm or terminal `; echo`/`; exit 0`/`; true`). +- /root/troupe-standup/tools/check_lint.py:109-111 — CATCH_ALL regex. +- /root/troupe-standup/tools/check_lint.py:330-340 — test cases: `… && echo PASS || echo FAIL` → CATCH_ALL; `…; echo done` → CATCH_ALL. + +Recurring RED ("CHECK not SOUND — CATCH_ALL"): +- sol-2026-10-18-1309.log:2 — catch_all_match "; echo " (iter1 rejected → GREEN only on iter2). +- sol-2026-10-18-0805.log:2 — catch_all_match "; echo " (iter1 rejected; :4 "no CHECK line emitted (RED)"; GREEN only on iter3). +- spark-2026-10-18-0554.log:2 — catch_all_match "; exit 0". +- spark-2026-10-18-0119.log:2,4,6 — catch_all_match "|| echo" ×3 → EXHAUSTED (3/3). +- spark-2026-10-18-0341.log:4,6 — catch_all_match "|| echo" ×2. +- sphinx-2026-10-17-1445.log:2 — catch_all_match "|| echo". +- sphinx-2026-10-17-2233.log:2 — catch_all_match "|| echo". +- Also (same CATCH_ALL class, 1 event each): forge-2026-10-16-2142.log, forge-2026-10-17-0024.log, sol-2026-10-16-2142.log, sol-2026-10-17-1719.log, sol-2026-10-17-2233.log, sol-2026-10-18-0341.log, sphinx-2026-10-17-0236.log (×3), sphinx-2026-10-17-0709.log, sphinx-2026-10-17-0938.log (×3), sphinx-2026-10-17-1159.log. + +## Also observed — not edited (out of harness-edit scope) +ledger/scribe/quill/null are EXHAUSTED (3/3) on every run for 3 days because their model configs are dead — quill uses a paid model with zero credits (config.yaml:2 `inclusionai/ling-3.1-flash`, no `:free`), ledger/scribe use `:free` models that 404 (`ling-3.0-flash-fin:free` / `ling-3.0-flash-sante:free`), null has a blank `model.default`. This is a per-profile `config.yaml` model fix, not a SOUL/memory/skill/lane-prompt edit, so it is recorded here but not applied by this reflection. + +VERDICT: retained 1 edit(s) — tautological CHECK tails (`; echo`/`; exit 0`/`|| echo`) rejected as CATCH_ALL recurred 17× across 4 wizards; the fleet lane prompt now forbids them.