[OPS] Source-control the bounded overnight burn lineup and fix deadman/no-op semantics #238

New Issue

Timmy · 2026-04-06T13:25:59Z

Timmy commented

2026-04-06 13:25:59 +00:00

Last night we improved the nightly stack live, but repo-truth is still behind the running machine.

Problems noticed:

We had duplicate burn jobs on the same lane, causing overlap and timeout waste.
The good overnight pattern is now known: one bounded burn job + one deadman + one morning report + health monitor.
deadman-status.log can still go ALERT even when the burn scheduler is alive if a bounded no-op cycle does not append a heartbeat/log line.
The repo-truth for the bounded prompt and nightly lineup is not yet codified in an active config/scripts repo.

Acceptance criteria:

Canonical overnight lineup is written in repo-truth (single burn job, deadman, morning report, health monitor).
Canonical bounded burn prompt is captured in repo-truth with hard limits: no clone, no rebase, one quick win max, no Evennia lane crossover.
Deadman logic is corrected so healthy no-op / [SILENT] cycles do not false-alert; either a heartbeat file is written every cycle or the deadman checks the right signal.
Running the bounded burn for 3 consecutive cycles produces no timeout and leaves proof-bearing output or an explicit healthy no-op heartbeat.
Morning report remains green after the hardening change.

Notes:

This should replace drift-prone live-only cron edits with source-controlled truth.
Keep the fix in the sidecar/config lane, not as an ad-hoc local-only patch.

Last night we improved the nightly stack live, but repo-truth is still behind the running machine. Problems noticed: - We had duplicate burn jobs on the same lane, causing overlap and timeout waste. - The good overnight pattern is now known: one bounded burn job + one deadman + one morning report + health monitor. - `deadman-status.log` can still go ALERT even when the burn scheduler is alive if a bounded no-op cycle does not append a heartbeat/log line. - The repo-truth for the bounded prompt and nightly lineup is not yet codified in an active config/scripts repo. Acceptance criteria: - [ ] Canonical overnight lineup is written in repo-truth (single burn job, deadman, morning report, health monitor). - [ ] Canonical bounded burn prompt is captured in repo-truth with hard limits: no clone, no rebase, one quick win max, no Evennia lane crossover. - [ ] Deadman logic is corrected so healthy no-op / [SILENT] cycles do not false-alert; either a heartbeat file is written every cycle or the deadman checks the right signal. - [ ] Running the bounded burn for 3 consecutive cycles produces no timeout and leaves proof-bearing output or an explicit healthy no-op heartbeat. - [ ] Morning report remains green after the hardening change. Notes: - This should replace drift-prone live-only cron edits with source-controlled truth. - Keep the fix in the sidecar/config lane, not as an ad-hoc local-only patch.

Timmy self-assigned this 2026-04-06 13:26:00 +00:00

Timmy commented

2026-04-06 13:48:55 +00:00

Backlog burn update:

Checked in current durable nightly-automation work on branch docs/automation-audit-20260404
Commit: 8d4591de3209f0fbe151ee26d2a99a6781a6e919
Push: origin/docs/automation-audit-20260404

What landed in repo truth:

source-controlled bin/burn-cycle-deadman.sh
source-controlled bin/morning-report-compiler.py
new docs/nightly-burn-mode.md with canonical bounded overnight lineup
automation inventory updated with additional loaded launch agents
tracked Huey log files removed from git and ignored as runtime churn

Verified before check-in:

bash -n bin/burn-cycle-deadman.sh
python3 -m py_compile bin/morning-report-compiler.py
python3 bin/morning-report-compiler.py 12 produced a clean report without the bogus $(date ...) cycle pollution

This does not complete the whole issue yet, but it moves the nightly lineup from live-only state toward sidecar repo truth.

Backlog burn update: - Checked in current durable nightly-automation work on branch `docs/automation-audit-20260404` - Commit: `8d4591de3209f0fbe151ee26d2a99a6781a6e919` - Push: `origin/docs/automation-audit-20260404` What landed in repo truth: - source-controlled `bin/burn-cycle-deadman.sh` - source-controlled `bin/morning-report-compiler.py` - new `docs/nightly-burn-mode.md` with canonical bounded overnight lineup - automation inventory updated with additional loaded launch agents - tracked Huey log files removed from git and ignored as runtime churn Verified before check-in: - `bash -n bin/burn-cycle-deadman.sh` - `python3 -m py_compile bin/morning-report-compiler.py` - `python3 bin/morning-report-compiler.py 12` produced a clean report without the bogus `$(date ...)` cycle pollution This does not complete the whole issue yet, but it moves the nightly lineup from live-only state toward sidecar repo truth.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#238