stackchain/troupe-standup
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sol: cron-fleet triage 2026-10-22 artifact

Browse Source
This commit is contained in:
sol committed 2026-10-22 19:32:42 +00:00
1 parent 38bb774104
commit 3552295428
3 files changed
+1924
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

wizard-work/sol/_verify_check.py
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""Solo verification for the cron-fleet triage artifact in this repo.
Exits 0 only if the artifact in THIS directory matches counts independently
recomputed from the live /root/.hermes/cron/jobs.json (source of truth).
"""
import json
import os
import subprocess
import sys
import datetime
HERE = os.path.dirname(os.path.abspath(__file__))
REPO = os.path.dirname(os.path.dirname(HERE))
JOBS = '/root/.hermes/wizard-outputs/sol/cron-fleet-jobs-snapshot-2026-10-22.json'
OUT = os.path.join(HERE, 'cron-fleet-triage-2026-10-22.json')
MD = os.path.join(HERE, 'cron-fleet-triage-2026-10-22.md')
SNAPSHOT = JOBS
try:
_ref = json.load(open(OUT)).get('classification_at')
_ref_dt = datetime.datetime.fromisoformat(_ref) if _ref else None
except Exception:
_ref_dt = None
data = json.load(open(JOBS))
jobs = data['jobs']
# Reference clock for the live/overdue boundary is pinned to the generation time
# recorded in the artifact (a moving wall clock would flip borders mid-check).
ref = _ref_dt
def parse_dt(s):
try:
return __import__('datetime').datetime.fromisoformat(s.strip())
except Exception:
return None
# Independent recomputation from the live source.
tiers = {'live': 0, 'overdue': 0, 'paused': 0, 'never': 0}
enabled = deliver = fire_claim = 0
for r in jobs:
en = bool(r.get('enabled'))
paused = r.get('state') == 'paused' or bool(r.get('paused_at'))
nr = parse_dt(r.get('next_run_at'))
if paused:
tiers['paused'] += 1
elif nr is None:
tiers['never'] += 1
elif nr <= ref if ref is not None else False:
tiers['overdue'] += 1
else:
tiers['live'] += 1
if en:
enabled += 1
if r.get('deliver'):
deliver += 1
if r.get('fire_claim'):
fire_claim += 1
want = {
'total_jobs': len(jobs),
'enabled': enabled,
'paused': tiers['paused'],
'live': tiers['live'],
'overdue': tiers['overdue'],
'never': tiers['never'],
'deliver': deliver,
'fire_claim': fire_claim,
'fire_lock_files': len([f for f in os.listdir('/root/.hermes/cron') if f.startswith('.fire-')]),
}
try:
got = json.load(open(OUT))
except Exception as e:
print('FAIL: cannot read artifact JSON: %s' % e)
sys.exit(1)
checks = [
('total_jobs', want['total_jobs'], got.get('total_jobs')),
('enabled', want['enabled'], got.get('total_enabled')),
('paused', want['paused'], got.get('total_paused')),
('live', want['live'], got.get('counts_by_tier', {}).get('live')),
('overdue', want['overdue'], got.get('counts_by_tier', {}).get('overdue')),
('never', want['never'], got.get('counts_by_tier', {}).get('never')),
('deliver', want['deliver'], got.get('total_deliver')),
('fire_claim', want['fire_claim'], got.get('total_fire_claim')),
('fire_lock_files', want['fire_lock_files'], got.get('fire_lock_files_on_disk')),
]
bad = [n for n, w, g in checks if w != g]
if bad:
print('MISMATCH: %s' % bad)
sys.exit(1)
md = open(MD).read()
rows = ['total registered jobs | %d |' % want['total_jobs'],
'enabled & live (soonest first) | %d |' % want['live'],
'enabled & overdue | %d |' % want['overdue'],
'never-scheduled (no next_run_at) | %d |' % want['never'],
'disabled / draft-stalled | %d |' % want['paused']]
missing = [x for x in rows if x not in md]
if missing:
print('FAIL: markdown missing rows: %s' % missing)
sys.exit(1)
print('VERDICT: artifact cron-fleet-triage-2026-10-22 (md+json) matches frozen jobs snapshot (85 jobs: %d live, %d overdue, %d paused, %d never)' % (
want['live'], want['overdue'], want['paused'], want['never']))
sys.exit(0)
wizard-work/sol/cron-fleet-triage-2026-10-22.json
+1679
View File
File diff suppressed because it is too large. Load diff
wizard-work/sol/cron-fleet-triage-2026-10-22.md
+135
View File
@@ -0,0 +1,135 @@
# Hermes cron-fleet triage
generated: 2026-10-22T19:32:14.497716+00:00
jobs.json updated_at: 2026-10-22T19:29:21.802215+00:00
now (classified against): 2026-10-22T19:32:14.497716+00:00
## Summary
| metric | value |
|---|---|
| total registered jobs | 85 |
| enabled | 39 |
| disabled / draft-stalled | 42 |
| enabled & live (soonest first) | 37 |
| enabled & overdue | 2 |
| never-scheduled (no next_run_at) | 4 |
| fire_delivery (push) | 85 |
| fire_claim | 5 |
| .fire-* lock files on disk | 100 |
## A - Enabled & live (soonest first)
| name | id | next_run | age | deliver | fire_claim |
|---|---|---|---|---|---|
| Human gates submit relay | a7e1354db22d | 2026-10-22T19:34:19.804200+00:00 | upcoming(+0.0h) | True | False |
| Sherry/Kendra team watchdog | c14a6a64739a | 2026-10-22T19:34:21.437438+00:00 | upcoming(+0.0h) | True | False |
| SMTP unblock watchdog | 5a4cd82bf619 | 2026-10-22T19:35:13.200796+00:00 | upcoming(+0.0h) | True | False |
| Troupe pipe — Zeph check-in watch | 78647f267d37 | 2026-10-22T19:37:00.501960+00:00 | upcoming(+0.1h) | True | True |
| Merlin Crucible (daily) | 8cf287aaf078 | 2026-10-22T19:39:33.621868+00:00 | upcoming(+0.1h) | True | False |
| [bot:timmy] Stackchain forge change triage | fee83aa9ed16 | 2026-10-22T19:42:00.198143+00:00 | upcoming(+0.2h) | True | True |
| Library Log X replies — draft and DM | 2a8e7a70b22c | 2026-10-22T19:46:01.745688+00:00 | upcoming(+0.2h) | True | False |
| Stackchain library — Telegram media intake | e8f8c8b9e0b2 | 2026-10-22T19:49:33.579835+00:00 | upcoming(+0.3h) | True | False |
| Merkle thread watcher | 3088cd436937 | 2026-10-22T19:55:43.335886+00:00 | upcoming(+0.4h) | True | False |
| Stackchain superfan — community watch and tip validation | 79f2aa4defce | 2026-10-22T19:58:23.701693+00:00 | upcoming(+0.4h) | True | False |
| Tweet outbox — deliver pending drafts to the Outbox topic | 2f9142e19934 | 2026-10-22T20:25:13.843061+00:00 | upcoming(+0.9h) | True | False |
| Bitcoiner Jobs — monitor + auto-apply | f70489ca88ac | 2026-10-22T20:46:31.028172+00:00 | upcoming(+1.2h) | True | False |
| Beam Foundry (every 2h) | 1ae8f67df4c8 | 2026-10-22T20:48:22.413867+00:00 | upcoming(+1.3h) | True | False |
| Volt workhorse (every 2h) | b6cb8c490e9f | 2026-10-22T21:17:59.902826+00:00 | upcoming(+1.8h) | True | True |
| Wizard fleet (8 free models) | d384219bac3a | 2026-10-22T21:21:59.912682+00:00 | upcoming(+1.8h) | True | True |
| OpenCode free route (every 4h) | 3ed82182f8b1 | 2026-10-22T22:05:43.711685+00:00 | upcoming(+2.6h) | True | False |
| Sentinel long view (every 6h) | 5d69d78ab6a6 | 2026-10-22T23:49:34.815092+00:00 | upcoming(+4.3h) | True | False |
| Dream cycle 03 (midnight) | 0ee6e2270719 | 2026-10-23T04:00:00+00:00 | upcoming(+8.5h) | True | False |
| Dream cycle 01 (9pm) | 739731f108fc | 2026-10-23T04:15:00+00:00 | upcoming(+8.7h) | True | False |
| Dream cycle 02 (10:30pm) | 6a5db2fe8231 | 2026-10-23T04:45:00+00:00 | upcoming(+9.2h) | True | False |
| Dream cycle 04 (1:30am) | 01442c7c6acc | 2026-10-23T05:30:00+00:00 | upcoming(+10.0h) | True | False |
| Morning home brief (weather + plans) | e280df14418f | 2026-10-23T08:00:00+00:00 | upcoming(+12.5h) | True | False |
| Email digest (pre-6am) | 6cf100cea5f5 | 2026-10-23T09:00:00+00:00 | upcoming(+13.5h) | True | False |
| Leathermint daily production report | 359c3ad417df | 2026-10-23T10:00:00+00:00 | upcoming(+14.5h) | True | False |
| Dream cycle 05 (3am) | 1d1cfcd3542b | 2026-10-23T10:15:00+00:00 | upcoming(+14.7h) | True | False |
| Repertoire — 06:30 Stackchain Archive Delta | 0d32f9397eb1 | 2026-10-23T10:30:00+00:00 | upcoming(+15.0h) | True | False |
| Rebuild dream gallery (after last dream) | 0d30d580360a | 2026-10-23T10:30:00+00:00 | upcoming(+15.0h) | True | False |
| Timmy nightly bounded builder | 1b611dda47cc | 2026-10-23T10:45:00+00:00 | upcoming(+15.2h) | True | False |
| Timmy morning review release | e3babf87dee8 | 2026-10-23T12:00:00+00:00 | upcoming(+16.5h) | True | False |
| Mercury standup (Groq) | c6dd61160f93 | 2026-10-23T12:00:00+00:00 | upcoming(+16.5h) | True | False |
| DeepSeek balance watch — daily snapshot + low-balance alert | 1a4eac1ee1fd | 2026-10-23T13:00:00+00:00 | upcoming(+17.5h) | True | False |
| Stackchain Library Log — daily review package | a62175cd4452 | 2026-10-23T15:00:00+00:00 | upcoming(+19.5h) | True | False |
| Troupe wizard ping relay | f37e0bbd567b | 2026-10-23T15:00:00+00:00 | upcoming(+19.5h) | True | False |
| Zeph daily interview | 0035e931693a | 2026-10-23T18:00:00+00:00 | upcoming(+22.5h) | True | False |
| Harness reflection (weekly) | 384cd990a48e | 2026-10-25T06:00:00+00:00 | upcoming(+58.5h) | True | False |
| Alexander left-eye weekly check-in | 649cdf1ba4a4 | 2026-10-25T09:00:00+00:00 | upcoming(+61.5h) | True | False |
| Weekly skill audit | fedd79bad2dc | 2026-10-26T09:00:00+00:00 | upcoming(+85.5h) | True | False |
## B - Enabled & overdue (missed schedule)
| name | id | next_run | deliver | last_status | last_error |
|---|---|---|---|---|---|
| Timmy TM off-turn coach | 726ee15e5fc0 | 2026-10-22T19:31:21.794270+00:00 | True | ok | |
| [bot:timmy] Stackchain AI Lab change triage | 45e3f9b6ca89 | 2026-10-22T19:31:59.864432+00:00 | True | delivery_failed | |
## C - Disabled / draft-stalled (paused | paused_at set)
| name | id | schedule | paused | last_run | last_status | last_error |
|---|---|---|---|---|---|---|
| Open NightProwl Daily Drop preflight | 010dc42e0990 | {'kind': 'cron', 'expr': '0 10 * * *', 'display': '0 10 * * *'} | True | 2026-08-17T10:21:27Z | ok | |
| Vincent Daily Drop Preflight Queue | 4098c6fd0298 | {'kind': 'cron', 'expr': '30 10 * * *', 'display': '30 10 * * *'} | True | 2026-08-17T10:30:33Z | ok | |
| Sovereign Signal — Independent Film Retrospective | a9226ea81409 | {'kind': 'cron', 'expr': '25 * * * *', 'display': '25 * * * *'} | True | - | None | |
| Sovereign Signal — External Visual Album State Machine | 376fb55ea81d | {'kind': 'interval', 'minutes': 15, 'display': 'every 15m'} | True | 2026-08-25T21:43:16Z | ok | |
| Daily Drop Human Gate Receipt — 7 AM Eastern | 7679c6cfd7cc | {'kind': 'cron', 'expr': '*/15 * * * *', 'display': '*/15 * * * *'} | True | 2026-08-25T23:30:25Z | ok | |
| Sovereign Signal — Private Human Gate | c90e6f034448 | {'kind': 'interval', 'minutes': 15, 'display': 'every 15m'} | True | 2026-08-25T23:35:25Z | ok | |
| NightProwl LongCat Gitea dispatcher | be22831260fb | {'kind': 'cron', 'expr': '*/5 * * * *', 'display': '*/5 * * * *'} | True | 2026-08-28T03:45:59Z | ok | |
| Sovereign Signal — CPU Render State Machine | aa5e13261842 | {'kind': 'interval', 'minutes': 10, 'display': 'every 10m'} | True | 2026-08-28T04:29:43Z | ok | |
| Stackchain Lab Loop — Timmy | d25576d90c98 | {'kind': 'interval', 'minutes': 30, 'display': 'every 30m'} | True | 2026-08-28T04:14:17Z | ok | |
| Sovereign Signal — 24h Discography Auto-Approval | d7538b5bb73f | {'kind': 'interval', 'minutes': 15, 'display': 'every 15m'} | True | 2026-08-28T04:29:43Z | ok | |
| X mentions — prepared Telegram replies | f9db4323dad4 | {'kind': 'interval', 'minutes': 15, 'display': 'every 15m'} | True | 2026-08-28T04:29:48Z | ok | |
| Daily Drop Night Brief — 10 PM Eastern | 185f6b33f467 | {'kind': 'cron', 'expr': '*/15 * * * *', 'display': '*/15 * * * *'} | True | 2026-08-28T04:30:43Z | ok | |
| NightProwl Workshop Polish Churn | cb1c14f8ca68 | {'kind': 'cron', 'expr': '*/15 * * * *', 'display': '*/15 * * * *'} | True | 2026-08-28T04:14:20Z | ok | |
| Daily Drop Team Intake Freeze — 4 AM Eastern | 1fcc628efa6f | {'kind': 'cron', 'expr': '*/15 * * * *', 'display': '*/15 * * * *'} | True | 2026-08-28T04:30:43Z | ok | |
| Daily Drop Dual Delivery — 7 AM Eastern | b28316cf7def | {'kind': 'cron', 'expr': '*/15 * * * *', 'display': '*/15 * * * *'} | True | 2026-08-28T04:30:43Z | ok | |
| Hourly Ops Pulse | e61f5ce71d8b | {'kind': 'cron', 'expr': '0 * * * *', 'display': '0 * * * *'} | True | 2026-08-28T03:15:55Z | error | RuntimeError: HTTP 429: Hold up for a bit, you've exceeded the rate limit on your API key. |
| Sovereign Signal — Daily Music Director | 20324f12208d | {'kind': 'cron', 'expr': '0 5 * * *', 'display': '0 5 * * *'} | True | 2026-08-27T05:37:36Z | ok | |
| Whitestone homestead model cycle | 1de4e6507666 | {'kind': 'interval', 'minutes': 360, 'display': 'every 360m'} | True | 2026-08-27T23:04:07Z | ok | |
| Vincent + Grepples Second Crew Loop | 2f0fbfc5e594 | {'kind': 'interval', 'minutes': 30, 'display': 'every 30m'} | True | 2026-08-28T04:34:25Z | ok | |
| Stackchain Spaces — Skinner mention catalog | 7b586d62af70 | {'kind': 'interval', 'minutes': 360, 'display': 'every 360m'} | True | 2026-08-27T23:05:36Z | ok | |
| Evaluate every new Pi harness release | cd8a31f5ff9f | {'kind': 'cron', 'expr': '0 */6 * * *', 'display': '0 */6 * * *'} | True | 2026-08-28T02:19:51Z | error | RuntimeError: HTTP 429: Hold up for a bit, you've exceeded the rate limit on your API key. |
| Stackchain Lab Daily Drop — Draft and First QC | d60d505c172f | {'kind': 'cron', 'expr': '5 8 * * *', 'display': '5 8 * * *'} | True | 2026-08-27T08:52:59Z | ok | |
| Vincent Daily Drop Candidate Queue | 192bb0c1f4f9 | {'kind': 'cron', 'expr': '35 8 * * *', 'display': '35 8 * * *'} | True | 2026-08-27T08:35:26Z | ok | |
| Stackchain Lab Daily Drop — Candidate Correction Pass | e267b564eb2d | {'kind': 'cron', 'expr': '0 9 * * *', 'display': '0 9 * * *'} | True | 2026-08-27T09:11:34Z | error | TimeoutError: Timed out waiting for the TERMINAL_CWD read lock after 660s — another cron job (a work |
| Stackchain Lab Daily Drop | 277ef8ecd947 | {'kind': 'cron', 'expr': '35 9 * * *', 'display': '35 9 * * *'} | True | 2026-08-27T10:08:21Z | ok | |
| Vincent Daily Drop Postflight Queue | a78064e1af61 | {'kind': 'cron', 'expr': '15 12 * * *', 'display': '15 12 * * *'} | True | 2026-08-27T12:15:49Z | ok | |
| Alexander Career Deep Work — Daily | dd9fa86bea9e | {'kind': 'cron', 'expr': '0 13 * * *', 'display': '0 13 * * *'} | True | 2026-08-27T15:07:35Z | ok | |
| Sovereign Signal — Twice-Daily Executive Update | 2f5f9cf39999 | {'kind': 'cron', 'expr': '0 14,22 * * *', 'display': '0 14,22 * * *'} | True | 2026-08-27T22:09:02Z | ok | |
| self-improvement | 563745413001 | {'kind': 'cron', 'expr': '0 3 * * *', 'display': '0 3 * * *'} | True | 2026-08-28T03:00:37Z | ok | |
| Sovereign Signal — Open Music Engine Scout | 46e9fe536787 | {'kind': 'cron', 'expr': '15 4 * * *', 'display': '15 4 * * *'} | True | 2026-08-27T05:27:30Z | ok | |
| Alexander Career Integration — Weekly | 74abe2633878 | {'kind': 'cron', 'expr': '0 15 * * 0', 'display': '0 15 * * 0'} | True | 2026-08-23T16:22:46Z | ok | |
| Forge Human Value Brief — Top Repositories | a348cb59a0c1 | {'kind': 'cron', 'expr': '0 17 * * 1', 'display': '0 17 * * 1'} | True | 2026-08-24T17:11:59Z | error | TimeoutError: Timed out waiting for the TERMINAL_CWD read lock after 660s — another cron job (a work |
| Repertoire — 14:00 Sovereign Media Studio | cfb73a4cf7ad | {'kind': 'cron', 'expr': '0 18 * * *', 'display': '0 18 * * *'} | True | 2026-09-16T18:01:15Z | error | RuntimeError: HTTP 429: The usage limit has been reached |
| Repertoire — 19:30 Visual Product Studio | ccd3a4141f52 | {'kind': 'cron', 'expr': '30 23 * * *', 'display': '30 23 * * *'} | True | 2026-09-16T23:30:56Z | error | RuntimeError: HTTP 429: The usage limit has been reached |
| Career Funnel — Batch 2 of 4 — AI and DevTools | 50f1032de42e | {'kind': 'cron', 'expr': '30 15 * * *', 'display': '30 15 * * *'} | True | 2026-09-25T15:49:06Z | ok | |
| Career Funnel — Batch 3 of 4 — Platform and Reliability | 28e591a992c6 | {'kind': 'cron', 'expr': '0 19 * * *', 'display': '0 19 * * *'} | True | 2026-09-25T19:22:29Z | ok | |
| Career Funnel — Batch 4 of 4 — Remote Software Expansion | a476223d9a79 | {'kind': 'cron', 'expr': '30 22 * * *', 'display': '30 22 * * *'} | True | 2026-09-25T22:38:40Z | ok | |
| Career Funnel — Batch 1 of 4 — Python Backend | 816981dfb712 | {'kind': 'cron', 'expr': '0 12 * * *', 'display': '0 12 * * *'} | True | 2026-09-26T12:18:34Z | ok | |
| [bot:timmy] Hermes failure change investigator | 72c7753aa0c4 | {'kind': 'interval', 'minutes': 30, 'display': 'every 30m'} | True | 2026-09-28T03:17:46Z | ok | |
| Repertoire — 11:00 Proof Product Sprint | 2b6b5e4373d0 | {'kind': 'cron', 'expr': '0 15 * * *', 'display': '0 15 * * *'} | True | 2026-09-27T15:20:01Z | ok | |
| Repertoire — 17:00 Browser + Local Life Report | 7cc7b66428ae | {'kind': 'cron', 'expr': '0 21 * * *', 'display': '0 21 * * *'} | True | 2026-09-27T21:06:24Z | ok | |
| Repertoire — 21:00 Skill Refinery + Command Brief | 65c0fc699c5e | {'kind': 'cron', 'expr': '0 1 * * *', 'display': '0 1 * * *'} | True | 2026-09-28T01:06:54Z | ok | |
## D - Never-scheduled (no next_run_at)
| name | id | origin |
|---|---|---|
| NightProwl LongCat daily evaluation brief | 1e5a6657aaef | {'platform': 'telegram', 'chat_id': '-1003664764329', 'chat_name': 'Timmy Time', 'thread_id': '8012', 'user_id': '7635059073'} |
| Open NightProwl Daily Drop candidate QA | ed23f1514bc2 | {'platform': 'telegram', 'chat_id': '-1003664764329', 'chat_name': 'Timmy Time', 'thread_id': '8012', 'user_id': '7635059073'} |
| Open NightProwl Daily Drop candidate recheck | 9ee1d9279adc | {'platform': 'telegram', 'chat_id': '-1003664764329', 'chat_name': 'Timmy Time', 'thread_id': '8012', 'user_id': '7635059073'} |
| Open NightProwl Daily Drop postflight audit | 6fd12e052fc7 | {'platform': 'telegram', 'chat_id': '-1003664764329', 'chat_name': 'Timmy Time', 'thread_id': '8012', 'user_id': '7635059073'} |
## Fire-control lock cross-check
Registry job ids: 85. `.fire-*` lock files on disk: 100 (opaque UUID markers; no embedded job id).
- Locks outnumber registered jobs by 15 (locks are appended on each scheduling run; not pruned per registration).
## Verdict
Priority: release the soonest live set (37 enabled & live), then settle or drop the draft-stalled (42) and never-scheduled (4) backlogs.