timmy-config/scripts/nightly-pipeline-scheduler.md

Nightly Pipeline Scheduler

Auto-starts batch pipelines when inference is available.

What It Does

1. Checks inference provider health (OpenRouter, Ollama, RunPod)
2. Checks if it's off-peak hours (configurable, default: after 6PM)
3. Checks interactive session load (don't fight with live users)
4. Checks daily token budget (configurable limit)
5. Starts the highest-priority incomplete pipeline

Pipeline Priority Order

Priority	Pipeline	Deps	Max Tokens
1	playground-factory	none	100,000
2	training-factory	none	150,000
3	knowledge-mine	training-factory running	80,000
4	adversary	knowledge-mine running	50,000
5	codebase-genome	none	120,000

Usage

# Normal run (used by cron)
./scripts/nightly-pipeline-scheduler.sh

# Dry run (show what would start)
./scripts/nightly-pipeline-scheduler.sh --dry-run

# Status report
./scripts/nightly-pipeline-scheduler.sh --status

# Force start during peak hours
./scripts/nightly-pipeline-scheduler.sh --force

Configuration

Set via environment variables:

• PIPELINE_TOKEN_LIMIT: Daily token budget (default: 500,000)
• PIPELINE_PEAK_START: Peak hours start (default: 9)
• PIPELINE_PEAK_END: Peak hours end (default: 18)
• HERMES_HOME: Hermes home directory (default: ~/.hermes)

Cron

Runs every 30 minutes. Off-peak only (unless --force). See cron/pipeline-scheduler.yml.