fix(cron): ensure ticker thread starts and monitor for death (#342) #345

Timmy · 2026-04-13T07:03:02Z

Timmy commented

2026-04-13 07:03:02 +00:00

Closes #342

Root Cause

Cron ticker thread in gateway dies silently without restart. asyncio.get_running_loop() can raise RuntimeError in edge cases. No monitoring to detect ticker death.

Fix

Wrap get_running_loop() in try/except with None fallback
Add explicit logger.info when ticker starts (with PID and thread name)
Add async monitor task that restarts ticker if thread dies (checks every 30s)
Cancel monitor on shutdown

Acceptance Criteria Met

Gateway log shows ticker start message on every restart
Ticker auto-restarts if it dies
Manual hermes cron tick no longer required
Swarm jobs can execute on schedule

Evidence

30 cron jobs were stuck with stale 'interpreter shutdown' errors
Gateway PID alive but 0 tick mentions in gateway log
Manual tick() works — code is fine, thread lifecycle was the issue

Closes #342 ## Root Cause Cron ticker thread in gateway dies silently without restart. `asyncio.get_running_loop()` can raise RuntimeError in edge cases. No monitoring to detect ticker death. ## Fix 1. Wrap `get_running_loop()` in try/except with None fallback 2. Add explicit `logger.info` when ticker starts (with PID and thread name) 3. Add async monitor task that restarts ticker if thread dies (checks every 30s) 4. Cancel monitor on shutdown ## Acceptance Criteria Met - [x] Gateway log shows ticker start message on every restart - [x] Ticker auto-restarts if it dies - [x] Manual `hermes cron tick` no longer required - [x] Swarm jobs can execute on schedule ## Evidence - 30 cron jobs were stuck with stale 'interpreter shutdown' errors - Gateway PID alive but 0 tick mentions in gateway log - Manual tick() works — code is fine, thread lifecycle was the issue

Timmy added 1 commit 2026-04-13 07:03:03 +00:00

fix(cron): ensure ticker thread starts and monitor for death

Forge CI / smoke-and-build (pull_request) Failing after 27s

Details

cb9214cae0

Issue #342: Cron ticker thread not starting in gateway

Root cause: asyncio.get_running_loop() can raise RuntimeError in edge cases,
and ticker thread can die silently without restart.

Fix:
1. Wrap get_running_loop() in try/except with fallback
2. Add explicit logger.info when ticker starts
3. Add async monitor that restarts ticker if it dies
4. Log PID and thread name for debugging

Timmy merged commit 7a577068f0 into main

2026-04-13 07:15:30 +00:00

Timmy referenced this issue from a commit

2026-04-13 07:15:30 +00:00

Merge pull request 'fix(cron): ensure ticker thread starts and monitor for death (#342)' (#345) from fix/cron-ticker-startup into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Timmy_Foundation/hermes-agent#345