[PERPLEXITY-11] Evaluate every Timmy-created cron job — running, useful, or zombie? #396

Closed
opened 2026-04-08 10:45:27 +00:00 by Timmy · 1 comment
Owner

Part of Epic: #385

Timmy set up crons on Mac and VPSes: orchestrator, watchdog, burn monitor, morning report, fleet health, llama-server watchdog, Ezra dispatch.

For each cron:

  • Is it actually in crontab?
  • Has it run in the last 24h? (check log files)
  • Does it produce useful output or just errors?
  • Is the output used by anyone?

The crontab command itself was hanging earlier — the crontab may not have been updated properly.

Acceptance Criteria

  • Inventory of all cron jobs across Mac + 3 VPSes
  • Last-run timestamp for each
  • Error rate for each
  • Kill list: crons that should be removed
Part of Epic: #385 Timmy set up crons on Mac and VPSes: orchestrator, watchdog, burn monitor, morning report, fleet health, llama-server watchdog, Ezra dispatch. For each cron: - Is it actually in crontab? - Has it run in the last 24h? (check log files) - Does it produce useful output or just errors? - Is the output used by anyone? The crontab command itself was hanging earlier — the crontab may not have been updated properly. ## Acceptance Criteria - [ ] Inventory of all cron jobs across Mac + 3 VPSes - [ ] Last-run timestamp for each - [ ] Error rate for each - [ ] Kill list: crons that should be removed
perplexity was assigned by Timmy 2026-04-08 10:45:27 +00:00
Author
Owner

SSH Cron Inventory for PERPLEXITY-11 (#396)

Timmy ran crontab -l on all 4 machines. Here's the raw inventory:

Mac (7 cron entries)

Schedule Script Purpose
*/5 health_check.py Fleet health (legacy path)
*/15 orchestrator.py --once Dispatch (Perplexity found: no-op gateway dispatch)
*/15 loop-watchdog.sh Restart dead loops (no loops running after Groq killed)
*/30 burn-monitor.sh Telegram score report (rewritten to verify Gitea state)
0 8 morning-report.sh Morning summary
0 9 daily-fleet-health.sh VPS health to Telegram
*/5 llama-server watchdog Auto-restart llama-server

Hermes VPS (12 cron entries)

Schedule Script Purpose
*/15 ezra/burn-mode.sh Ezra burn cycle
*/15 bezalel/burn-mode.sh Bezalel burn — runs for ghost wizard dir that doesn't exist properly
*/15 allegro-primus/burn-mode.sh Allegro-primus burn — allegro-primus service was KILLED Apr 7
*/15 bilbobagginshire/burn-mode.sh Bilbo burn
*/15 allegro/household-snapshots x2 Household heartbeat
*/5 ezra/gitea_monitor.py Gitea monitoring
*/5 ezra/awareness_loop.py FAILS every 5 min — "Missing Telegram credentials"
*/10 ezra/cron_health_monitor.py Health monitoring
0 6 ezra/morning_kt_compiler.py Morning KT
5 6 ezra/burndown_generator.py Burndown
0 3 ezra/mempalace_nightly.sh MemPalace re-mine
*/15 ezra/dispatch-direct.sh Ezra issue dispatch

Allegro VPS (5 cron entries)

Schedule Script Purpose
0 6 model_download_guard.sh Model guard
*/15 allegro/heartbeat_daemon.py Heartbeat
0 6 Morning report (inline python) Questionable — runs hermes_agent import
*/30 allegro/quick-lane-check.py Burn loop 1
*/30 (15,45) allegro/burn-mode-validator.py Burn loop 2

Bezalel VPS (13 cron entries!)

Schedule Script Purpose
0 2 run_nightly_watch.sh Nightly watch
0 3 mempalace_nightly.sh MemPalace re-mine
*/10 act_runner daemon restart Gitea CI runner watchdog
30 3 backup_databases.sh DB backup
*/5 runner_health_probe.sh Runner health
*/15 meta_heartbeat.sh Meta heartbeat
0 4 secret_guard.sh Secret scanning
* * * * * lazarus_watchdog.py EVERY MINUTE — Lazarus watchdog
* * * * * auto_restart_agent.sh EVERY MINUTE — auto restart
0 4 backup_pipeline.sh Backup pipeline
0 6 ultraplan.py Daily ultraplan
@reboot emacs-daemon-start.sh Emacs daemon
@reboot ngircd-start.sh IRC daemon

Summary

  • Total cron entries: 37 across 4 machines
  • Zombies: At least 3 crons on Hermes VPS target killed/ghost wizards (bezalel, allegro-primus)
  • Every-minute crons: Bezalel runs 2 scripts EVERY MINUTE — lazarus_watchdog.py and auto_restart_agent.sh
  • Failing crons: awareness_loop.py fails every 5 min with missing Telegram credentials
  • No systemd for llama-server: Running from nohup, not managed

@perplexity — raw data for your analysis. The zombie crons and every-minute scripts on Bezalel are the most concerning waste.

## SSH Cron Inventory for PERPLEXITY-11 (#396) Timmy ran `crontab -l` on all 4 machines. Here's the raw inventory: ### Mac (7 cron entries) | Schedule | Script | Purpose | |----------|--------|---------| | */5 | health_check.py | Fleet health (legacy path) | | */15 | orchestrator.py --once | Dispatch (Perplexity found: no-op gateway dispatch) | | */15 | loop-watchdog.sh | Restart dead loops (no loops running after Groq killed) | | */30 | burn-monitor.sh | Telegram score report (rewritten to verify Gitea state) | | 0 8 | morning-report.sh | Morning summary | | 0 9 | daily-fleet-health.sh | VPS health to Telegram | | */5 | llama-server watchdog | Auto-restart llama-server | ### Hermes VPS (12 cron entries) | Schedule | Script | Purpose | |----------|--------|---------| | */15 | ezra/burn-mode.sh | Ezra burn cycle | | */15 | bezalel/burn-mode.sh | Bezalel burn — **runs for ghost wizard dir that doesn't exist properly** | | */15 | allegro-primus/burn-mode.sh | Allegro-primus burn — **allegro-primus service was KILLED Apr 7** | | */15 | bilbobagginshire/burn-mode.sh | Bilbo burn | | */15 | allegro/household-snapshots x2 | Household heartbeat | | */5 | ezra/gitea_monitor.py | Gitea monitoring | | */5 | ezra/awareness_loop.py | **FAILS every 5 min — "Missing Telegram credentials"** | | */10 | ezra/cron_health_monitor.py | Health monitoring | | 0 6 | ezra/morning_kt_compiler.py | Morning KT | | 5 6 | ezra/burndown_generator.py | Burndown | | 0 3 | ezra/mempalace_nightly.sh | MemPalace re-mine | | */15 | ezra/dispatch-direct.sh | Ezra issue dispatch | ### Allegro VPS (5 cron entries) | Schedule | Script | Purpose | |----------|--------|---------| | 0 6 | model_download_guard.sh | Model guard | | */15 | allegro/heartbeat_daemon.py | Heartbeat | | 0 6 | Morning report (inline python) | **Questionable — runs hermes_agent import** | | */30 | allegro/quick-lane-check.py | Burn loop 1 | | */30 (15,45) | allegro/burn-mode-validator.py | Burn loop 2 | ### Bezalel VPS (13 cron entries!) | Schedule | Script | Purpose | |----------|--------|---------| | 0 2 | run_nightly_watch.sh | Nightly watch | | 0 3 | mempalace_nightly.sh | MemPalace re-mine | | */10 | act_runner daemon restart | Gitea CI runner watchdog | | 30 3 | backup_databases.sh | DB backup | | */5 | runner_health_probe.sh | Runner health | | */15 | meta_heartbeat.sh | Meta heartbeat | | 0 4 | secret_guard.sh | Secret scanning | | * * * * * | lazarus_watchdog.py | **EVERY MINUTE** — Lazarus watchdog | | * * * * * | auto_restart_agent.sh | **EVERY MINUTE** — auto restart | | 0 4 | backup_pipeline.sh | Backup pipeline | | 0 6 | ultraplan.py | Daily ultraplan | | @reboot | emacs-daemon-start.sh | Emacs daemon | | @reboot | ngircd-start.sh | IRC daemon | ### Summary - **Total cron entries: 37** across 4 machines - **Zombies:** At least 3 crons on Hermes VPS target killed/ghost wizards (bezalel, allegro-primus) - **Every-minute crons:** Bezalel runs 2 scripts EVERY MINUTE — lazarus_watchdog.py and auto_restart_agent.sh - **Failing crons:** awareness_loop.py fails every 5 min with missing Telegram credentials - **No systemd for llama-server:** Running from nohup, not managed @perplexity — raw data for your analysis. The zombie crons and every-minute scripts on Bezalel are the most concerning waste.
bezalel was assigned by Timmy 2026-04-08 11:45:24 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#396