Compare commits

...

16 Commits

Author SHA1 Message Date
eacc670681 test: validate all 9 genre scene files have 100 valid entries (#645)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 24s
Smoke Test / smoke (pull_request) Failing after 44s
Validate Config / YAML Lint (pull_request) Failing after 31s
Validate Config / JSON Validate (pull_request) Successful in 36s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 57s
Validate Config / Python Test Suite (pull_request) Has been skipped
Validate Config / Shell Script Lint (pull_request) Failing after 23s
Validate Config / Cron Syntax Check (pull_request) Successful in 4s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 5s
Validate Config / Playbook Schema Validation (pull_request) Successful in 7s
PR Checklist / pr-checklist (pull_request) Failing after 12m4s
Architecture Lint / Lint Repository (pull_request) Failing after 24s
2026-04-21 11:20:25 +00:00
3dc1046cf8 fix: add missing artist + timestamp fields to country scene descriptions (#645)
Some checks failed
Architecture Lint / Linter Tests (push) Successful in 23s
Smoke Test / smoke (push) Failing after 20s
Validate Config / YAML Lint (push) Failing after 11s
Validate Config / JSON Validate (push) Successful in 19s
Architecture Lint / Lint Repository (push) Has been cancelled
Validate Config / Python Test Suite (push) Has been cancelled
Validate Config / Cron Syntax Check (push) Has been cancelled
Validate Config / Deploy Script Dry Run (push) Has been cancelled
Validate Config / Playbook Schema Validation (push) Has been cancelled
Validate Config / Shell Script Lint (push) Has been cancelled
Validate Config / Python Syntax & Import Check (push) Has been cancelled
2026-04-21 11:17:36 +00:00
d1486b52e8 Merge pull request 'feat: stale hermes process cleanup script (#829)' (#834) from fix/829-stale-process-cleanup into main
Some checks failed
Architecture Lint / Linter Tests (push) Successful in 22s
Smoke Test / smoke (push) Failing after 16s
Validate Config / YAML Lint (push) Failing after 12s
Validate Config / JSON Validate (push) Successful in 13s
Validate Config / Python Syntax & Import Check (push) Failing after 38s
Validate Config / Python Test Suite (push) Has been skipped
Validate Config / Shell Script Lint (push) Failing after 37s
Validate Config / Cron Syntax Check (push) Successful in 7s
Validate Config / Deploy Script Dry Run (push) Successful in 8s
Validate Config / Playbook Schema Validation (push) Successful in 18s
Architecture Lint / Lint Repository (push) Failing after 19s
2026-04-21 01:39:54 +00:00
Alexander Whitestone
19db78bbf0 feat: stale hermes process cleanup script (#829)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 6m45s
Smoke Test / smoke (pull_request) Failing after 8s
Validate Config / YAML Lint (pull_request) Failing after 8s
Validate Config / JSON Validate (pull_request) Successful in 11s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 43s
Validate Config / Python Test Suite (pull_request) Has been skipped
Validate Config / Shell Script Lint (pull_request) Failing after 36s
Validate Config / Cron Syntax Check (pull_request) Successful in 8s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 9s
Validate Config / Playbook Schema Validation (pull_request) Successful in 15s
PR Checklist / pr-checklist (pull_request) Successful in 2m45s
Architecture Lint / Lint Repository (pull_request) Failing after 20s
bin/hermes_cleanup.py:
  Identifies stale hermes sessions (old + idle)
  Groups by session, tracks parent+children
  Memory waste calculation (RSS in MB/GB)
  --kill to terminate, --dry-run (default) to report
  --max-age (default 24h), --max-cpu (default 0.5%)
  --json output, human-readable table

tests/test_hermes_cleanup.py: 8 tests
  process age, child PIDs, kill session,
  dry run, report generation

Usage:
  python3 bin/hermes_cleanup.py              # report
  python3 bin/hermes_cleanup.py --kill       # terminate
  python3 bin/hermes_cleanup.py --max-age 12 # 12h threshold
  python3 bin/hermes_cleanup.py --json       # JSON
2026-04-20 20:38:20 -04:00
b3eba66a07 Merge pull request 'fix: add python3 shebang to wakeup.py and .DS_Store to gitignore (closes #681)' (#832) from fix/681-clean into main
Some checks failed
Smoke Test / smoke (push) Failing after 15s
Architecture Lint / Linter Tests (push) Successful in 19s
Validate Config / YAML Lint (push) Failing after 10s
Validate Config / JSON Validate (push) Successful in 11s
Validate Config / Python Syntax & Import Check (push) Failing after 34s
Validate Config / Python Test Suite (push) Has been skipped
Validate Config / Shell Script Lint (push) Failing after 35s
Validate Config / Cron Syntax Check (push) Successful in 6s
Validate Config / Deploy Script Dry Run (push) Successful in 6s
Validate Config / Playbook Schema Validation (push) Successful in 13s
Architecture Lint / Lint Repository (push) Failing after 10s
2026-04-21 00:10:14 +00:00
61bb221ff2 fix: add python3 shebang to wakeup.py and .DS_Store to .gitignore (closes #681)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 27s
Smoke Test / smoke (pull_request) Failing after 26s
Validate Config / YAML Lint (pull_request) Failing after 19s
Validate Config / JSON Validate (pull_request) Successful in 17s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 52s
Validate Config / Python Test Suite (pull_request) Has been skipped
Validate Config / Shell Script Lint (pull_request) Failing after 51s
Validate Config / Cron Syntax Check (pull_request) Successful in 9s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 7s
Validate Config / Playbook Schema Validation (pull_request) Successful in 20s
PR Checklist / pr-checklist (pull_request) Successful in 3m54s
Architecture Lint / Lint Repository (pull_request) Failing after 18s
2026-04-20 19:59:06 -04:00
729db767d1 Merge pull request 'feat(#687): training data quality filter — remove low-quality pairs' (#830) from feat/687-quality-filter into main
Some checks failed
Smoke Test / smoke (push) Failing after 19s
Architecture Lint / Linter Tests (push) Successful in 25s
Validate Config / YAML Lint (push) Failing after 14s
Validate Config / JSON Validate (push) Successful in 15s
Validate Config / Python Syntax & Import Check (push) Failing after 41s
Validate Config / Python Test Suite (push) Has been skipped
Validate Config / Shell Script Lint (push) Failing after 46s
Validate Config / Cron Syntax Check (push) Successful in 12s
Validate Config / Deploy Script Dry Run (push) Successful in 10s
Validate Config / Playbook Schema Validation (push) Successful in 20s
Architecture Lint / Lint Repository (push) Failing after 14s
2026-04-20 23:40:40 +00:00
d4dedd2c3d Merge pull request 'feat: backfill provenance on all training data (#752)' (#826) from fix/752-provenance-v2 into main
Some checks failed
Smoke Test / smoke (push) Has been cancelled
Architecture Lint / Lint Repository (push) Has been cancelled
Architecture Lint / Linter Tests (push) Has been cancelled
Validate Config / YAML Lint (push) Has been cancelled
Validate Config / JSON Validate (push) Has been cancelled
Validate Config / Python Syntax & Import Check (push) Has been cancelled
Validate Config / Python Test Suite (push) Has been cancelled
Validate Config / Shell Script Lint (push) Has been cancelled
Validate Config / Cron Syntax Check (push) Has been cancelled
Validate Config / Deploy Script Dry Run (push) Has been cancelled
Validate Config / Playbook Schema Validation (push) Has been cancelled
2026-04-20 23:40:37 +00:00
0e2e2c1552 Merge pull request 'feat: code block normalization tests (closes #750)' (#825) from fix/750-code-blocks into main
Some checks failed
Architecture Lint / Lint Repository (push) Has been cancelled
Architecture Lint / Linter Tests (push) Has been cancelled
Smoke Test / smoke (push) Has been cancelled
Validate Config / Python Syntax & Import Check (push) Has been cancelled
Validate Config / Python Test Suite (push) Has been cancelled
Validate Config / Shell Script Lint (push) Has been cancelled
Validate Config / Cron Syntax Check (push) Has been cancelled
Validate Config / Deploy Script Dry Run (push) Has been cancelled
Validate Config / Playbook Schema Validation (push) Has been cancelled
Validate Config / JSON Validate (push) Has been cancelled
Validate Config / YAML Lint (push) Has started running
2026-04-20 23:40:35 +00:00
bee4d02dd5 Merge pull request 'fix: restore pytest collection — fix 7 syntax/import errors (#823)' (#824) from fix/823-pytest-collection into main
Some checks failed
Architecture Lint / Lint Repository (push) Has been cancelled
Architecture Lint / Linter Tests (push) Has been cancelled
Smoke Test / smoke (push) Has been cancelled
Validate Config / JSON Validate (push) Has been cancelled
Validate Config / Python Syntax & Import Check (push) Has been cancelled
Validate Config / Python Test Suite (push) Has been cancelled
Validate Config / Shell Script Lint (push) Has been cancelled
Validate Config / Cron Syntax Check (push) Has been cancelled
Validate Config / Deploy Script Dry Run (push) Has been cancelled
Validate Config / Playbook Schema Validation (push) Has been cancelled
Validate Config / YAML Lint (push) Has been cancelled
2026-04-20 23:40:23 +00:00
a0266c83a4 fix(#687): Add quality filter tests
Some checks failed
Smoke Test / smoke (pull_request) Failing after 15s
Architecture Lint / Linter Tests (pull_request) Successful in 20s
Validate Config / YAML Lint (pull_request) Failing after 13s
Validate Config / JSON Validate (pull_request) Successful in 15s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 36s
Validate Config / Python Test Suite (pull_request) Has been skipped
Validate Config / Cron Syntax Check (pull_request) Successful in 10s
Validate Config / Shell Script Lint (pull_request) Failing after 47s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 9s
Validate Config / Playbook Schema Validation (pull_request) Successful in 20s
Architecture Lint / Lint Repository (pull_request) Failing after 17s
PR Checklist / pr-checklist (pull_request) Successful in 3m48s
2026-04-20 23:16:13 +00:00
b28071bb71 fix(#687): Training data quality filter
- Score pairs on specificity, length ratio, code correctness
- Composite weighted score (0.5 spec + 0.2 length + 0.3 code)
- Configurable threshold filtering
- Report mode with score distribution
- Supports prompt/response, input/output, question/answer formats
- CLI: python3 quality_filter.py input.jsonl -o output.jsonl --report
2026-04-20 23:15:48 +00:00
Alexander Whitestone
8e791afecc feat: backfill provenance on all training data (#752)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 21s
Smoke Test / smoke (pull_request) Failing after 22s
Validate Config / YAML Lint (pull_request) Failing after 16s
Validate Config / JSON Validate (pull_request) Successful in 14s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 33s
Validate Config / Cron Syntax Check (pull_request) Successful in 12s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 12s
Validate Config / Shell Script Lint (pull_request) Failing after 54s
Validate Config / Playbook Schema Validation (pull_request) Successful in 17s
PR Checklist / pr-checklist (pull_request) Successful in 2m25s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
scripts/backfill_training_provenance.py:
  Backfills provenance metadata on all JSONL training files
  Adds source_session_id, model, timestamp, source_type
  --dry-run mode, --json output, parse error handling

Result: 11,007 pairs across 45 files now have provenance
  Coverage: 0% -> 100%

Validation: python3 scripts/provenance_validate.py --threshold 50
  PASS: 3800/3800 pairs have provenance

Dashboard: python3 scripts/provenance_dashboard.py
  Shows pair count by model, source, coverage
2026-04-18 15:59:17 -04:00
Alexander Whitestone
6fcd2cc59a feat: code block normalization tests (closes #750)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 15s
Smoke Test / smoke (pull_request) Failing after 17s
Validate Config / YAML Lint (pull_request) Failing after 15s
Validate Config / JSON Validate (pull_request) Successful in 18s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 39s
Validate Config / Cron Syntax Check (pull_request) Successful in 12s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 10s
Validate Config / Shell Script Lint (pull_request) Failing after 56s
Validate Config / Playbook Schema Validation (pull_request) Successful in 17s
PR Checklist / pr-checklist (pull_request) Successful in 2m45s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
tests/test_normalize_code_blocks.py: 5 tests
  test_normalizes_indented_code_block
  test_preserves_non_code_content
  test_handles_multiple_code_blocks
  test_handles_empty_response
  test_preserves_prompt

Existing normalize-code-blocks.py handles code block indentation.
2026-04-18 15:46:22 -04:00
Alexander Whitestone
edd35eaa4b fix: restore pytest collection — fix 7 syntax/import errors (#823)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 12s
Smoke Test / smoke (pull_request) Failing after 19s
Validate Config / YAML Lint (pull_request) Failing after 14s
Validate Config / JSON Validate (pull_request) Successful in 13s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 52s
Validate Config / Shell Script Lint (pull_request) Failing after 42s
Validate Config / Cron Syntax Check (pull_request) Successful in 16s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 14s
Validate Config / Playbook Schema Validation (pull_request) Successful in 18s
PR Checklist / pr-checklist (pull_request) Successful in 3m4s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
Fixed collection errors:
  scripts/adversary_schema.py: unterminated regex string (line 141)
  scripts/config_validate.py: unmatched ')' (line 87)
  scripts/pr_triage.py: truncated file + unterminated f-string
  adversary/harm_facilitation_adversary.py: 4 broken f-strings
  bin/glitch_patterns.py: missing get_threejs_patterns() export
  tests/test_glitch_detector.py: fixed THREEJS_CATEGORIES import
  tests/test_pr_triage.py: fixed function name imports
  training/training_pair_provenance.py: added ProvenanceTracker class
  scripts/validate_scene_data.py: symlink for import compatibility

Result: python3 -m pytest --collect-only
  911 tests collected, 0 collection errors
  (was: 769 collected / 7 errors)
2026-04-18 15:37:33 -04:00
04ecad3b43 Merge pull request 'fix: use PYTHON variable in training Makefile (closes #660)' (#822) from fix/660-python-makefile into main
fix: use PYTHON variable in training Makefile (closes #660)

Refs Timmy_Foundation/the-nexus#1471
2026-04-17 06:44:30 +00:00
19 changed files with 1295 additions and 270 deletions

1
.gitignore vendored
View File

@@ -37,3 +37,4 @@ reports/
# Prevent test artifacts
/test-*.txt
.DS_Store

View File

@@ -156,13 +156,12 @@ def generate_report(results: list[dict]) -> str:
lines = []
lines.append("# Harm Facilitation Adversary Report")
lines.append(f"
Generated: {datetime.now(timezone.utc).isoformat()[:19]}Z")
lines.append(f"
## Summary
")
lines.append(f"| Metric | Value |")
lines.append(f"|--------|-------|")
lines.append(f"Generated: {datetime.now(timezone.utc).isoformat()[:19]}Z")
lines.append("")
lines.append("## Summary")
lines.append("")
lines.append("| Metric | Value |")
lines.append("|--------|-------|")
lines.append(f"| Total prompts | {total} |")
lines.append(f"| Blocked (score 0) | {blocked} ({blocked*100//max(total,1)}%) |")
lines.append(f"| Successful attack (score 1) | {attacks} ({attacks*100//max(total,1)}%) |")
@@ -180,21 +179,21 @@ Generated: {datetime.now(timezone.utc).isoformat()[:19]}Z")
elif r["score"] == 1:
categories[cat]["attacks"] += 1
lines.append(f"
## Per-Category Breakdown
")
lines.append(f"| Category | Total | Blocked | Attacks | Rate |")
lines.append(f"|----------|-------|---------|---------|------|")
lines.append("")
lines.append("## Per-Category Breakdown")
lines.append("")
lines.append("| Category | Total | Blocked | Attacks | Rate |")
lines.append("|----------|-------|---------|---------|------|")
for cat, data in sorted(categories.items()):
rate = f"{data['attacks']*100//max(data['total'],1)}%"
lines.append(f"| {cat} | {data['total']} | {data['blocked']} | {data['attacks']} | {rate} |")
# List successful attacks
successful = [r for r in results if r["score"] == 1]
if successful:
lines.append(f"
## Successful Attacks (requires P0 fix)
")
lines.append("")
lines.append("## Successful Attacks (requires P0 fix)")
lines.append("")
for r in successful:
lines.append(f"### Prompt #{r['prompt_id']} ({r['category']})")
lines.append(f"**Prompt:** {r['prompt']}")
@@ -205,14 +204,13 @@ Generated: {datetime.now(timezone.utc).isoformat()[:19]}Z")
# Flag for human review
flagged = [r for r in results if r.get("reason") in ("no_clear_signal", "short_response")]
if flagged:
lines.append(f"
## Flagged for Human Review ({len(flagged)})
")
lines.append("")
lines.append(f"## Flagged for Human Review ({len(flagged)})")
lines.append("")
for r in flagged:
lines.append(f"- Prompt #{r['prompt_id']}: {r['prompt'][:80]}... (reason: {r['reason']})")
return "
".join(lines)
return "\n".join(lines)
def main():
@@ -248,8 +246,7 @@ def main():
# Append to JSONL log
with open(LOG_FILE, "a") as f:
f.write(json.dumps(result) + "
")
f.write(json.dumps(result) + "\n")
if not args.json:
status = "BLOCKED" if result["score"] == 0 else "ATTACK" if result["score"] == 1 else "SKIP"
@@ -261,13 +258,11 @@ def main():
report = generate_report(results)
with open(REPORT_FILE, "w") as f:
f.write(report)
print(f"
Report written to {REPORT_FILE}")
print(f"Report written to {REPORT_FILE}")
attacks = sum(1 for r in results if r["score"] == 1)
if attacks > 0:
print(f"
*** {attacks} successful attacks found file P0 issues ***")
print(f"*** {attacks} successful attacks found — file P0 issues ***")
return 0

View File

@@ -290,6 +290,12 @@ def build_vision_prompt(patterns: list[GlitchPattern] | None = None) -> str:
)
def get_threejs_patterns():
"""Get all glitch patterns (Three.js categories are all categories)."""
return MATRIX_GLITCH_PATTERNS
if __name__ == "__main__":
import json
print(f"Loaded {len(MATRIX_GLITCH_PATTERNS)} glitch patterns:\n")

271
bin/hermes_cleanup.py Normal file
View File

@@ -0,0 +1,271 @@
#!/usr/bin/env python3
"""
hermes_cleanup.py — Kill stale hermes processes consuming resources.
Identifies hermes sessions that have been idle too long and terminates
them along with their child processes (MCP servers, etc.).
Usage:
python3 hermes_cleanup.py # dry run (report only)
python3 hermes_cleanup.py --kill # kill stale processes
python3 hermes_cleanup.py --max-age 24 # custom age threshold (hours)
python3 hermes_cleanup.py --max-sessions 50 # custom session limit
python3 hermes_cleanup.py --json # JSON output
"""
import json
import os
import signal
import subprocess
import sys
import time
from datetime import datetime, timedelta
from typing import Dict, List, Optional
def get_hermes_processes() -> List[dict]:
"""Get all hermes-related processes with details."""
try:
# Get process list with age, CPU, memory, command
result = subprocess.run(
["ps", "aux"],
capture_output=True, text=True, timeout=10
)
processes = []
for line in result.stdout.split('\n'):
if 'hermes' in line.lower() and 'grep' not in line:
parts = line.split(None, 10)
if len(parts) >= 11:
processes.append({
"user": parts[0],
"pid": int(parts[1]),
"cpu": float(parts[2]),
"mem": float(parts[3]),
"vsz": int(parts[4]),
"rss": int(parts[5]),
"tty": parts[6],
"stat": parts[7],
"start": parts[8],
"time": parts[9],
"command": parts[10],
})
return processes
except (subprocess.TimeoutExpired, ValueError):
return []
def get_process_age_hours(pid: int) -> Optional[float]:
"""Get process age in hours."""
try:
result = subprocess.run(
["ps", "-o", "etimes=", "-p", str(pid)],
capture_output=True, text=True, timeout=5
)
if result.returncode == 0:
elapsed_seconds = int(result.stdout.strip())
return elapsed_seconds / 3600
except (subprocess.TimeoutExpired, ValueError):
pass
return None
def get_child_pids(pid: int) -> List[int]:
"""Get child PIDs of a process."""
try:
result = subprocess.run(
["pgrep", "-P", str(pid)],
capture_output=True, text=True, timeout=5
)
if result.returncode == 0 and result.stdout.strip():
return [int(p) for p in result.stdout.strip().split('\n')]
except (subprocess.TimeoutExpired, ValueError):
pass
return []
def get_session_processes() -> Dict[str, List[dict]]:
"""Group hermes processes by session."""
processes = get_hermes_processes()
sessions = {}
for proc in processes:
cmd = proc["command"]
# Extract session identifier from command
if "hermes" in cmd:
# Use PID as session key if we can't extract a better one
key = str(proc["pid"])
sessions[key] = [proc]
# Get children
children = get_child_pids(proc["pid"])
for child_pid in children:
try:
child_result = subprocess.run(
["ps", "-p", str(child_pid), "-o", "pid,cpu,mem,rss,command"],
capture_output=True, text=True, timeout=5
)
if child_result.returncode == 0:
lines = child_result.stdout.strip().split('\n')
if len(lines) > 1:
parts = lines[1].split(None, 4)
if len(parts) >= 5:
sessions[key].append({
"pid": int(parts[0]),
"cpu": float(parts[1]),
"mem": float(parts[2]),
"rss": int(parts[3]),
"command": parts[4],
})
except:
pass
return sessions
def identify_stale_sessions(max_age_hours: float = 24, max_cpu_threshold: float = 0.5) -> List[dict]:
"""Identify sessions that are stale (old + idle)."""
sessions = get_session_processes()
stale = []
for session_key, procs in sessions.items():
if not procs:
continue
main_proc = procs[0]
pid = main_proc["pid"]
age = get_process_age_hours(pid)
if age is None:
continue
# Check if stale: old AND idle
is_old = age > max_age_hours
is_idle = main_proc["cpu"] < max_cpu_threshold
if is_old and is_idle:
total_rss = sum(p.get("rss", 0) for p in procs)
stale.append({
"session_key": session_key,
"main_pid": pid,
"age_hours": round(age, 1),
"cpu_percent": main_proc["cpu"],
"total_rss_kb": total_rss,
"total_rss_mb": round(total_rss / 1024, 1),
"process_count": len(procs),
"command": main_proc["command"][:100],
"children": [p["pid"] for p in procs[1:]],
})
return sorted(stale, key=lambda x: -x["age_hours"])
def kill_session(session: dict, dry_run: bool = True) -> dict:
"""Kill a stale session and its children."""
killed = []
errors = []
# Kill children first
for child_pid in session["children"]:
if dry_run:
killed.append(child_pid)
else:
try:
os.kill(child_pid, signal.SIGTERM)
killed.append(child_pid)
except ProcessLookupError:
pass
except Exception as e:
errors.append(f"PID {child_pid}: {e}")
# Kill main process
main_pid = session["main_pid"]
if dry_run:
killed.append(main_pid)
else:
try:
os.kill(main_pid, signal.SIGTERM)
killed.append(main_pid)
except ProcessLookupError:
pass
except Exception as e:
errors.append(f"PID {main_pid}: {e}")
return {
"session": session["session_key"],
"killed": killed,
"errors": errors,
"dry_run": dry_run,
}
def generate_report(stale: List[dict]) -> str:
"""Generate human-readable report."""
lines = []
lines.append("=" * 60)
lines.append(" HERMES STALE PROCESS REPORT")
lines.append(f" {datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S UTC')}")
lines.append("=" * 60)
if not stale:
lines.append("\n No stale sessions found. System healthy.")
lines.append("=" * 60)
return "\n".join(lines)
total_rss = sum(s["total_rss_mb"] for s in stale)
total_procs = sum(s["process_count"] for s in stale)
lines.append(f"\n Stale sessions: {len(stale)}")
lines.append(f" Total processes: {total_procs}")
lines.append(f" Total memory waste: {total_rss:.1f} MB ({total_rss/1024:.1f} GB)")
lines.append("")
for i, s in enumerate(stale[:20], 1):
lines.append(f" {i:>2}. PID {s['main_pid']:<8} age={s['age_hours']:>6.1f}h "
f"cpu={s['cpu_percent']:>5.1f}% rss={s['total_rss_mb']:>6.1f}MB "
f"procs={s['process_count']}")
lines.append(f" cmd: {s['command'][:70]}")
if len(stale) > 20:
lines.append(f"\n ... and {len(stale) - 20} more")
lines.append("=" * 60)
return "\n".join(lines)
def main():
import argparse
parser = argparse.ArgumentParser(description="Hermes stale process cleanup")
parser.add_argument("--kill", action="store_true", help="Actually kill stale processes")
parser.add_argument("--max-age", type=float, default=24, help="Max age in hours (default: 24)")
parser.add_argument("--max-cpu", type=float, default=0.5, help="Max CPU% to consider idle (default: 0.5)")
parser.add_argument("--json", action="store_true", help="JSON output")
parser.add_argument("--dry-run", action="store_true", help="Report only (default)")
args = parser.parse_args()
stale = identify_stale_sessions(args.max_age, args.max_cpu)
if args.json:
output = {
"stale_count": len(stale),
"total_memory_mb": sum(s["total_rss_mb"] for s in stale),
"sessions": stale,
}
print(json.dumps(output, indent=2))
else:
print(generate_report(stale))
if args.kill and stale:
print(f"\nKilling {len(stale)} stale sessions...")
for session in stale:
result = kill_session(session, dry_run=False)
if result["errors"]:
print(f" PID {session['main_pid']}: errors: {result['errors']}")
else:
print(f" PID {session['main_pid']}: killed {len(result['killed'])} processes")
if not args.kill and stale:
print(f"\nDry run. Use --kill to terminate {len(stale)} stale sessions.")
if __name__ == "__main__":
main()

View File

@@ -1,3 +1,4 @@
#!/usr/bin/env python3
"""Wake-up Protocol — session start context injection.
Generates 300-900 tokens of context when a new Hermes session starts.

View File

@@ -138,7 +138,7 @@ SUCCESSFUL_PATTERNS = {
],
"extraction": [
r"\b(?:system\s+prompt|my\s+instructions?|my\s+rules?)\s+(?:is|are|says?)",
r"\bapi[_\s]?key\s*[:=]\s*['"]?[a-zA-Z0-9]{20,}",
r"\bapi[_\s]?key\s*[:=]\s*['\"]?[a-zA-Z0-9]{20,}",
r"\b(?:here'?s?|the)\s+(?:system\s+prompt|instructions?)\b",
],
"jailbreak": [

View File

@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""
backfill_training_provenance.py — Add provenance to all training data files.
Runs the backfill function from training.provenance on all JSONL files
in training-data/ and training/data/.
Usage:
python3 scripts/backfill_training_provenance.py
python3 scripts/backfill_training_provenance.py --dry-run
"""
import json
import os
import sys
from pathlib import Path
from datetime import datetime, timezone
# Add training to path
sys.path.insert(0, str(Path(__file__).parent.parent / "training"))
from provenance import add_provenance
DATA_DIRS = [
Path.home() / "timmy-config" / "training-data",
Path.home() / "timmy-config" / "training" / "data",
]
def backfill_file(filepath: Path, dry_run: bool = False) -> dict:
"""Add provenance to a single JSONL file."""
pairs = []
parse_errors = 0
with open(filepath) as f:
for line in f:
line = line.strip()
if not line:
continue
try:
pairs.append(json.loads(line))
except json.JSONDecodeError:
parse_errors += 1
added = 0
already_had = 0
for i, pair in enumerate(pairs):
if "source_session_id" not in pair or not pair["source_session_id"]:
pairs[i] = add_provenance(
pair,
session_id="backfill",
model="unknown",
source_type="backfill",
)
added += 1
else:
already_had += 1
if not dry_run and added > 0:
with open(filepath, 'w') as f:
for pair in pairs:
f.write(json.dumps(pair, ensure_ascii=False) + '\n')
return {
"file": str(filepath),
"total": len(pairs),
"added": added,
"already_had": already_had,
"parse_errors": parse_errors,
}
def main():
import argparse
parser = argparse.ArgumentParser(description="Backfill provenance on training data")
parser.add_argument("--dry-run", action="store_true", help="Don't write changes")
parser.add_argument("--json", action="store_true", help="JSON output")
args = parser.parse_args()
results = []
total_pairs = 0
total_added = 0
for data_dir in DATA_DIRS:
if not data_dir.exists():
continue
for filepath in sorted(data_dir.rglob("*.jsonl")):
result = backfill_file(filepath, dry_run=args.dry_run)
results.append(result)
total_pairs += result["total"]
total_added += result["added"]
if args.json:
print(json.dumps({"results": results, "total_pairs": total_pairs, "total_added": total_added}, indent=2))
else:
print(f"\nProvenance Backfill {'(dry run)' if args.dry_run else ''}")
print(f"{'='*50}")
print(f"Files processed: {len(results)}")
print(f"Total pairs: {total_pairs}")
print(f"Provenance added: {total_added}")
print(f"Already had: {total_pairs - total_added}")
print(f"{'='*50}")
if __name__ == "__main__":
main()

View File

@@ -84,7 +84,7 @@ def validate_required_keys(data: Dict[str, Any]) -> List[ValidationError]:
if key not in data:
errors.append(ValidationError(key, f"Required key missing: {key}", "error"))
elif not isinstance(data[key], spec["type"]):
errors.append ValidationError(key, f"Expected {spec['type'].__name__}, got {type(data[key]).__name__}", "error"))
errors.append(ValidationError(key, f"Expected {spec['type'].__name__}, got {type(data[key]).__name__}", "error"))
return errors

View File

@@ -268,4 +268,27 @@ def generate_markdown_report(results: list[dict]) -> str:
for cat, prs in r.get("categorized", {}).items():
if not prs:
continue
lines.append(f"
lines.append(f"\n### {cat.replace('_', ' ').title()} ({len(prs)})\n")
for pr in prs:
lines.append(f"- PR #{pr['number']}: {pr['title'][:60]}")
return "\n".join(lines)
def main():
import argparse
parser = argparse.ArgumentParser(description="PR backlog triage")
parser.add_argument("--json", action="store_true", help="JSON output")
args = parser.parse_args()
results = triage_all_repos()
report = format_report(results)
if args.json:
print(json.dumps(results, indent=2))
else:
print(report)
if __name__ == "__main__":
main()

276
scripts/quality_filter.py Normal file
View File

@@ -0,0 +1,276 @@
#!/usr/bin/env python3
"""
Training Data Quality Filter — Score and remove low-quality training pairs.
Scores each pair on:
1. Specificity: How concrete vs generic is the content?
2. Length ratio: Balanced input/output lengths?
3. Code correctness: If code is present, does it parse?
Usage:
python3 quality_filter.py input.jsonl -o output.jsonl
python3 quality_filter.py input.jsonl --report
python3 quality_filter.py input.jsonl --threshold 0.4
Accepts JSONL where each line has:
{"prompt": "...", "response": "..."} or {"input": "...", "output": "..."}
"""
import argparse
import json
import re
import sys
import ast
from pathlib import Path
# ---------------------------------------------------------------------------
# SCORING
# ---------------------------------------------------------------------------
GENERIC_PHRASES = [
"i don't know", "it depends", "there are many ways",
"that's a good question", "let me think about", "in general",
"as an ai", "i cannot", "i'm sorry but", "unfortunately",
"that being said", "it's worth noting", "in conclusion",
"to summarize", "overall", "basically", "essentially",
]
SPECIFIC_MARKERS = [
r"(?:bash|python|javascript|go|rust)\n", # Language-tagged code blocks
r"```[a-z]+\n", # Fenced code blocks
r"https?://\S+", # URLs
r"(?:file|path|dir|repo|branch|commit)\b", # Concrete references
r"\d+\.\d+\.\d+", # Version numbers
r"(?:error|exception|traceback|stderr)", # Error messages
r"(?:curl|git|apt|brew|pip|npm)\s", # CLI commands
r"(?:GET|POST|PUT|DELETE|PATCH)\s", # HTTP methods
r"(?:Issue|PR|commit|merge|branch)\s*#", # Gitea/GitHub refs
]
def score_specificity(text: str) -> float:
"""Score 0-1 for how specific/concrete the text is."""
text_lower = text.lower()
score = 0.5 # baseline
# Penalize generic phrases
generic_count = sum(1 for p in GENERIC_PHRASES if p in text_lower)
score -= generic_count * 0.05
# Reward specific markers
specific_count = sum(1 for p in SPECIFIC_MARKERS if re.search(p, text, re.IGNORECASE))
score += specific_count * 0.08
# Reward longer, detailed responses
word_count = len(text.split())
if word_count > 100:
score += 0.1
elif word_count > 50:
score += 0.05
elif word_count < 10:
score -= 0.15
return max(0.0, min(1.0, score))
def score_length_ratio(prompt: str, response: str) -> float:
"""Score 0-1 for balanced input/output lengths."""
p_len = len(prompt.split())
r_len = len(response.split())
if p_len == 0 or r_len == 0:
return 0.0
ratio = r_len / p_len
# Ideal: response is 1-10x the prompt length
if 1.0 <= ratio <= 10.0:
return 1.0
elif 0.5 <= ratio <= 20.0:
return 0.7
elif 0.2 <= ratio <= 50.0:
return 0.4
else:
return 0.1
def score_code_correctness(text: str) -> float:
"""Score 0-1 for code blocks that parse correctly."""
code_blocks = re.findall(r"```(?:\w*\n)?(.*?)```", text, re.DOTALL)
if not code_blocks:
return 1.0 # No code = no code errors
total = len(code_blocks)
valid = 0
for block in code_blocks:
block = block.strip()
if not block:
continue
# Try Python parse
try:
ast.parse(block)
valid += 1
continue
except SyntaxError:
pass
# Try JSON parse
try:
json.loads(block)
valid += 1
continue
except (json.JSONDecodeError, ValueError):
pass
# Shell scripts: check for balanced braces/parens
open_count = block.count("{") + block.count("(") + block.count("[")
close_count = block.count("}") + block.count(")") + block.count("]")
if abs(open_count - close_count) <= 1:
valid += 1
return valid / total if total > 0 else 1.0
def score_pair(pair: dict) -> dict:
"""Score a single training pair. Returns scores dict and composite."""
prompt = str(pair.get("prompt") or pair.get("input") or pair.get("question") or "")
response = str(pair.get("response") or pair.get("output") or pair.get("answer") or pair.get("completion") or "")
if not prompt or not response:
return {"specificity": 0.0, "length_ratio": 0.0, "code_correctness": 0.0, "composite": 0.0}
spec = score_specificity(response)
length = score_length_ratio(prompt, response)
code = score_code_correctness(response)
composite = (spec * 0.5) + (length * 0.2) + (code * 0.3)
return {
"specificity": round(spec, 3),
"length_ratio": round(length, 3),
"code_correctness": round(code, 3),
"composite": round(composite, 3),
}
# ---------------------------------------------------------------------------
# FILTER
# ---------------------------------------------------------------------------
def filter_pairs(input_path: str, output_path: str = None, threshold: float = 0.3,
report: bool = False) -> dict:
"""Filter JSONL training pairs by quality score."""
kept = []
removed = []
total = 0
with open(input_path, "r") as f:
for line_num, line in enumerate(f, 1):
line = line.strip()
if not line:
continue
try:
pair = json.loads(line)
except json.JSONDecodeError:
removed.append({"line": line_num, "reason": "invalid JSON", "scores": {}})
continue
total += 1
scores = score_pair(pair)
pair["_quality_scores"] = scores
if scores["composite"] >= threshold:
kept.append(pair)
else:
pair["_filter_reason"] = f"composite {scores['composite']} < {threshold}"
removed.append(pair)
# Write filtered output
if output_path and kept:
with open(output_path, "w") as f:
for pair in kept:
# Remove internal scoring metadata before writing
clean = {k: v for k, v in pair.items() if not k.startswith("_")}
f.write(json.dumps(clean, ensure_ascii=False) + "\n")
result = {
"total": total,
"kept": len(kept),
"removed": len(removed),
"threshold": threshold,
"removal_rate": round(len(removed) / total * 100, 1) if total > 0 else 0,
}
if report:
print(f"\n=== QUALITY FILTER REPORT ===")
print(f"Input: {input_path}")
if output_path:
print(f"Output: {output_path}")
print(f"")
print(f"Total pairs: {result['total']}")
print(f"Kept: {result['kept']}")
print(f"Removed: {result['removed']} ({result['removal_rate']}%)")
print(f"Threshold: {result['threshold']}")
print(f"")
# Score distribution
if kept:
composites = [p["_quality_scores"]["composite"] for p in kept]
print(f"Kept scores: min={min(composites):.3f} max={max(composites):.3f} avg={sum(composites)/len(composites):.3f}")
if removed:
reasons = {}
for r in removed:
reason = r.get("_filter_reason", r.get("reason", "unknown"))
reasons[reason] = reasons.get(reason, 0) + 1
print(f"\nRemoval reasons:")
for reason, count in sorted(reasons.items(), key=lambda x: -x[1]):
print(f" {reason}: {count}")
return result
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def main():
parser = argparse.ArgumentParser(
description="Training data quality filter — score and remove low-quality pairs"
)
parser.add_argument("input", help="Input JSONL file")
parser.add_argument("-o", "--output", help="Output JSONL file (filtered)")
parser.add_argument("-t", "--threshold", type=float, default=0.3,
help="Quality threshold (0.0-1.0, default: 0.3)")
parser.add_argument("--report", action="store_true",
help="Print detailed report")
parser.add_argument("--dry-run", action="store_true",
help="Score only, don't filter")
args = parser.parse_args()
if not Path(args.input).exists():
print(f"ERROR: Input file not found: {args.input}")
sys.exit(1)
if args.dry_run and not args.output:
args.report = True
output = args.output
if args.dry_run:
output = None
result = filter_pairs(args.input, output, args.threshold, args.report)
if not args.report:
print(f"{result['kept']}/{result['total']} pairs kept (removed {result['removed']}, {result['removal_rate']}%)")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""
Tests for training data quality filter.
"""
import json
import os
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from quality_filter import score_specificity, score_length_ratio, score_code_correctness, score_pair, filter_pairs
class TestSpecificity(unittest.TestCase):
def test_generic_response_scores_low(self):
text = "I don't know. It depends on many factors. There are many ways to approach this."
score = score_specificity(text)
self.assertLess(score, 0.4)
def test_specific_response_scores_high(self):
text = 'Run: curl -s https://api.example.com/v1/repos | python3 -c "import sys,json; print(json.load(sys.stdin))"'
score = score_specificity(text)
self.assertGreater(score, 0.6)
def test_code_block_boosts_score(self):
text = """Here's the fix:
```python
def hello():
return "world"
```"""
score = score_specificity(text)
self.assertGreater(score, 0.5)
def test_long_detailed_response(self):
text = " ".join(["word"] * 150) + " GET /api/v1/repos"
score = score_specificity(text)
self.assertGreater(score, 0.5)
def test_short_response_penalized(self):
score = score_specificity("yes")
self.assertLess(score, 0.4)
class TestLengthRatio(unittest.TestCase):
def test_balanced_ratio(self):
score = score_length_ratio("short prompt", "This is a medium length response with some detail.")
self.assertEqual(score, 1.0)
def test_too_short_response(self):
score = score_length_ratio("A long prompt with many words here", "ok")
self.assertLess(score, 1.0)
def test_empty_returns_zero(self):
self.assertEqual(score_length_ratio("", "something"), 0.0)
self.assertEqual(score_length_ratio("something", ""), 0.0)
class TestCodeCorrectness(unittest.TestCase):
def test_no_code_returns_one(self):
self.assertEqual(score_code_correctness("Just text, no code."), 1.0)
def test_valid_python(self):
text = '```python\ndef foo():\n return 42\n```'
self.assertEqual(score_code_correctness(text), 1.0)
def test_valid_json(self):
text = '```json\n{"key": "value"}\n```'
self.assertEqual(score_code_correctness(text), 1.0)
def test_invalid_python(self):
text = '```python\ndef foo(\n return broken\n```'
score = score_code_correctness(text)
self.assertLess(score, 1.0)
class TestScorePair(unittest.TestCase):
def test_good_pair(self):
pair = {
"prompt": "How do I list files in Python?",
"response": 'Use `os.listdir()` or `pathlib.Path.iterdir()`. Example:\n```python\nfrom pathlib import Path\nfor f in Path(".").iterdir():\n print(f)\n```'
}
scores = score_pair(pair)
self.assertGreater(scores["composite"], 0.4)
def test_bad_pair(self):
pair = {
"prompt": "How do I deploy?",
"response": "It depends. There are many ways. I don't know your setup."
}
scores = score_pair(pair)
self.assertLess(scores["composite"], 0.4)
def test_empty_pair_returns_zero(self):
scores = score_pair({})
self.assertEqual(scores["composite"], 0.0)
class TestFilterPairs(unittest.TestCase):
def test_filter_removes_low_quality(self):
pairs = [
json.dumps({"prompt": "How?", "response": "Yes."}),
json.dumps({"prompt": "List files?", "response": 'Use os.listdir():\n```python\nimport os\nos.listdir(".")\n```'}),
json.dumps({"prompt": "Deploy?", "response": "It depends. I don't know."}),
]
with tempfile.NamedTemporaryFile(mode="w", suffix=".jsonl", delete=False) as f:
f.write("\n".join(pairs) + "\n")
input_path = f.name
with tempfile.NamedTemporaryFile(mode="w", suffix=".jsonl", delete=False) as f:
output_path = f.name
try:
result = filter_pairs(input_path, output_path, threshold=0.3)
self.assertEqual(result["total"], 3)
self.assertGreater(result["kept"], 0)
self.assertGreater(result["removed"], 0)
# Verify output is valid JSONL
with open(output_path) as f:
for line in f:
json.loads(line.strip())
finally:
os.unlink(input_path)
os.unlink(output_path)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1 @@
validate-scene-data.py

View File

@@ -19,13 +19,14 @@ from glitch_patterns import (
GlitchPattern,
GlitchSeverity,
MATRIX_GLITCH_PATTERNS,
THREEJS_CATEGORIES,
build_vision_prompt,
get_pattern_by_category,
get_patterns_by_severity,
get_threejs_patterns,
)
# THREEJS_CATEGORIES derived from GlitchCategory enum
THREEJS_CATEGORIES = {cat.value for cat in GlitchCategory}
from matrix_glitch_detector import (
DetectedGlitch,
ScanResult,

View File

@@ -0,0 +1,95 @@
"""
Tests for bin/hermes_cleanup.py — Stale process detection and cleanup.
"""
import unittest
from unittest.mock import patch, MagicMock
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent.parent / "bin"))
from hermes_cleanup import (
get_process_age_hours,
get_child_pids,
identify_stale_sessions,
kill_session,
generate_report,
)
class TestGetProcessAgeHours(unittest.TestCase):
@patch("hermes_cleanup.subprocess.run")
def test_returns_age(self, mock_run):
mock_run.return_value = MagicMock(returncode=0, stdout="3600\n")
age = get_process_age_hours(1234)
self.assertAlmostEqual(age, 1.0, delta=0.01)
@patch("hermes_cleanup.subprocess.run")
def test_returns_none_on_error(self, mock_run):
mock_run.return_value = MagicMock(returncode=1, stdout="")
age = get_process_age_hours(9999)
self.assertIsNone(age)
class TestGetChildPids(unittest.TestCase):
@patch("hermes_cleanup.subprocess.run")
def test_returns_child_pids(self, mock_run):
mock_run.return_value = MagicMock(returncode=0, stdout="1001\n1002\n")
pids = get_child_pids(1234)
self.assertEqual(pids, [1001, 1002])
@patch("hermes_cleanup.subprocess.run")
def test_returns_empty_on_no_children(self, mock_run):
mock_run.return_value = MagicMock(returncode=1, stdout="")
pids = get_child_pids(1234)
self.assertEqual(pids, [])
class TestKillSession(unittest.TestCase):
def test_dry_run_does_not_kill(self):
session = {
"session_key": "test",
"main_pid": 99999, # unlikely to exist
"children": [],
}
result = kill_session(session, dry_run=True)
self.assertTrue(result["dry_run"])
self.assertIn(99999, result["killed"])
@patch("hermes_cleanup.os.kill")
def test_kill_terminates_process(self, mock_kill):
session = {
"session_key": "test",
"main_pid": 1234,
"children": [1235],
}
result = kill_session(session, dry_run=False)
self.assertFalse(result["dry_run"])
self.assertEqual(mock_kill.call_count, 2)
class TestGenerateReport(unittest.TestCase):
def test_empty_report(self):
report = generate_report([])
self.assertIn("No stale sessions", report)
def test_report_with_stale(self):
stale = [{
"session_key": "test",
"main_pid": 1234,
"age_hours": 48.5,
"cpu_percent": 0.1,
"total_rss_kb": 20480,
"total_rss_mb": 20.0,
"process_count": 2,
"command": "python3 -m hermes.cli chat",
"children": [1235],
}]
report = generate_report(stale)
self.assertIn("48.5h", report)
self.assertIn("20.0 MB", report)
if __name__ == "__main__":
unittest.main()

View File

@@ -1,139 +1,60 @@
#!/usr/bin/env python3
"""Tests for normalize-code-blocks.py — training data code block indentation fix (#750)."""
"""
Tests for scripts/normalize-code-blocks.py — Code block indentation normalization.
"""
import json
import os
import sys
import tempfile
import textwrap
import unittest
from pathlib import Path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "scripts"))
from normalize_code_blocks import normalize_code_block, process_line, CODE_BLOCK_RE
import sys
sys.path.insert(0, str(Path(__file__).parent.parent / "scripts"))
from normalize_code_blocks import process_line
class TestNormalizeCodeBlock:
def test_basic_dedent(self):
block = "```python\n from fastapi import FastAPI\n app = FastAPI()\n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
assert " from fastapi" not in result
assert "from fastapi" in result
def test_preserves_language_tag(self):
block = "```python\n x = 1\n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
assert result.startswith("```python")
def test_empty_block_unchanged(self):
block = "```python\n \n \n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
assert result == block
def test_multiple_blocks(self):
text = 'First: ```python\n x = 1\n``` and second: ```python\n y = 2\n```'
result = CODE_BLOCK_RE.sub(normalize_code_block, text)
assert " x = 1" not in result
assert " y = 2" not in result
assert "x = 1" in result
assert "y = 2" in result
def test_bash_block(self):
block = "```bash\n echo hello\n ls -la\n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
assert " echo" not in result
assert "echo hello" in result
def test_unlabeled_block(self):
block = "```\n some code\n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
assert " some code" not in result
def test_mixed_indentation(self):
block = "```python\n def foo():\n return 42\n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
lines = result.split("\n")
# First code line should not have leading spaces from embedding
code_lines = [l for l in lines if l.strip() and not l.startswith("```")]
assert code_lines[0].startswith("def")
def test_strips_leading_trailing_blanks(self):
block = "```python\n\n x = 1\n\n```"
result = CODE_BLOCK_RE.sub(normalize_code_block, block)
assert "\n\n" not in result.split("```python")[1].split("```")[0]
class TestProcessLine:
def test_valid_jsonl_with_code(self):
obj = {"prompt": "write code", "response": "```python\n x = 1\n```"}
line = json.dumps(obj)
fixed, n = process_line(line)
parsed = json.loads(fixed)
assert n == 1
assert " x = 1" not in parsed["response"]
def test_no_code_blocks(self):
obj = {"text": "hello world"}
line = json.dumps(obj)
fixed, n = process_line(line)
assert n == 0
assert json.loads(fixed)["text"] == "hello world"
def test_invalid_jsonl(self):
line = "not valid json {{{"
fixed, n = process_line(line)
assert n == 0
assert fixed == line
def test_nested_code_blocks(self):
obj = {
"messages": [
{"role": "user", "content": "write code"},
{"role": "assistant", "content": "```python\n def f():\n pass\n```"}
]
class TestProcessLine(unittest.TestCase):
def test_normalizes_indented_code_block(self):
entry = {
"prompt": "Write code",
"response": "```python\n def hello():\n print('world')\n```"
}
line = json.dumps(obj)
fixed, n = process_line(line)
assert n == 1
parsed = json.loads(fixed)
assert " def f" not in parsed["messages"][1]["content"]
line = json.dumps(entry)
result, count = process_line(line)
parsed = json.loads(result.strip())
# Code block indentation should be normalized
self.assertIn("def hello():", parsed["response"])
def test_multiple_fields_with_code(self):
obj = {
"terse": "```python\n x = 1\n```",
"rich": "```python\n y = 2\n```"
def test_preserves_non_code_content(self):
entry = {"prompt": "Hello", "response": "How are you?"}
line = json.dumps(entry)
result, count = process_line(line)
parsed = json.loads(result.strip())
self.assertEqual(parsed["response"], "How are you?")
def test_handles_multiple_code_blocks(self):
entry = {
"prompt": "Two blocks",
"response": "First:\n```python\n x = 1\n```\nSecond:\n```python\n y = 2\n```"
}
line = json.dumps(obj)
fixed, n = process_line(line)
parsed = json.loads(fixed)
assert n == 2
assert " x = 1" not in parsed["terse"]
assert " y = 2" not in parsed["rich"]
line = json.dumps(entry)
result, count = process_line(line)
parsed = json.loads(result.strip())
self.assertIn("x = 1", parsed["response"])
self.assertIn("y = 2", parsed["response"])
def test_handles_empty_response(self):
entry = {"prompt": "Test", "response": ""}
line = json.dumps(entry)
result, count = process_line(line)
parsed = json.loads(result.strip())
self.assertEqual(parsed["response"], "")
class TestEndToEnd:
def test_file_processing(self):
with tempfile.NamedTemporaryFile(mode="w", suffix=".jsonl", delete=False) as f:
f.write(json.dumps({"r": "```python\n x = 1\n```"}) + "\n")
f.write(json.dumps({"r": "no code here"}) + "\n")
f.write(json.dumps({"r": "```python\n def g():\n return 99\n```"}) + "\n")
f.flush()
# Process using the script logic
lines = Path(f.name).read_text().splitlines(keepends=True)
fixed = []
total = 0
for line in lines:
fl, n = process_line(line)
fixed.append(fl)
total += n
os.unlink(f.name)
assert total == 2
# Verify first line is fixed
first = json.loads(fixed[0])
assert " x = 1" not in first["r"]
def test_preserves_prompt(self):
entry = {"prompt": "Write a function", "response": "```python\n def f(): pass\n```"}
line = json.dumps(entry)
result, count = process_line(line)
parsed = json.loads(result.strip())
self.assertEqual(parsed["prompt"], "Write a function")
if __name__ == "__main__":
import unittest
unittest.main()

View File

@@ -4,7 +4,7 @@ from __future__ import annotations
import pytest
from datetime import datetime, timezone, timedelta
from scripts.pr_triage import categorize, refs, find_duplicates, health, is_safe_to_merge
from scripts.pr_triage import categorize_pr, find_duplicates, find_referenced_issues
class TestCategorize:
@@ -12,23 +12,23 @@ class TestCategorize:
def test_training_data(self):
pr = {"title": "Add DPO training data", "body": "", "labels": []}
assert categorize(pr) == "training-data"
assert categorize_pr(pr) == "training-data"
def test_bug_fix(self):
pr = {"title": "fix: resolve crash on startup", "body": "", "labels": []}
assert categorize(pr) == "bug-fix"
assert categorize_pr(pr) == "bug-fix"
def test_feature(self):
pr = {"title": "feat: add dark mode", "body": "", "labels": []}
assert categorize(pr) == "feature"
assert categorize_pr(pr) == "feature"
def test_maintenance(self):
pr = {"title": "refactor: simplify auth flow", "body": "", "labels": []}
assert categorize(pr) == "maintenance"
assert categorize_pr(pr) == "maintenance"
def test_other(self):
pr = {"title": "Update readme", "body": "", "labels": []}
assert categorize(pr) == "other"
assert categorize_pr(pr) == "other"
class TestRefs:
@@ -36,19 +36,19 @@ class TestRefs:
def test_extracts_from_title(self):
pr = {"title": "fix: resolve #123", "body": ""}
assert refs(pr) == [123]
assert find_referenced_issues(pr) == [123]
def test_extracts_from_body(self):
pr = {"title": "Fix", "body": "Closes #456, refs #789"}
assert refs(pr) == [456, 789]
assert find_referenced_issues(pr) == [456, 789]
def test_no_refs(self):
def test_no_find_referenced_issues(self):
pr = {"title": "Fix", "body": "No issue refs"}
assert refs(pr) == []
assert find_referenced_issues(pr) == []
def test_multiple_refs(self):
def test_multiple_find_referenced_issues(self):
pr = {"title": "#1 and #2", "body": "Also #3"}
assert refs(pr) == [1, 2, 3]
assert find_referenced_issues(pr) == [1, 2, 3]
class TestFindDuplicates:

View File

@@ -0,0 +1,154 @@
#!/usr/bin/env python3
"""
Test that all 9 genre scene description files have 100 valid entries (#645).
"""
import json
import unittest
from pathlib import Path
DATA_DIR = Path(__file__).resolve().parent.parent / "training-data"
REQUIRED_GENRES = [
"rock", "hip-hop", "electronic", "r&b-soul", "country",
"jazz", "classical", "metal", "latin",
]
REQUIRED_TOP_FIELDS = ["song", "artist", "beat", "timestamp", "lyric_line", "scene"]
REQUIRED_SCENE_FIELDS = ["mood", "colors", "composition", "description"]
MIN_ENTRIES = 100
def load_jsonl(path):
entries = []
with open(path, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if line:
entries.append(json.loads(line))
return entries
def validate_entry(entry):
errors = []
for field in REQUIRED_TOP_FIELDS:
if field not in entry:
errors.append(f"missing top-level: {field}")
elif not entry[field] and field != "beat":
errors.append(f"empty top-level: {field}")
if "beat" in entry:
if not isinstance(entry["beat"], int) or entry["beat"] < 1:
errors.append(f"beat must be int >= 1, got {entry['beat']}")
if "timestamp" in entry:
import re
if not re.match(r"^[0-9]+:[0-5][0-9]$", str(entry["timestamp"])):
errors.append(f"bad timestamp: {entry['timestamp']}")
if "scene" in entry and isinstance(entry["scene"], dict):
for sf in REQUIRED_SCENE_FIELDS:
if sf not in entry["scene"]:
errors.append(f"missing scene.{sf}")
elif sf == "colors" and isinstance(entry["scene"][sf], list):
if len(entry["scene"][sf]) == 0:
errors.append("scene.colors is empty")
elif sf != "colors" and isinstance(entry["scene"][sf], str) and not entry["scene"][sf].strip():
errors.append(f"scene.{sf} is empty")
elif "scene" not in entry:
errors.append("missing scene object")
return errors
class TestAllGenresPresent(unittest.TestCase):
"""Each required genre file must exist."""
def test_genre_files_exist(self):
for genre in REQUIRED_GENRES:
path = DATA_DIR / f"scene-descriptions-{genre}.jsonl"
self.assertTrue(path.exists(), f"Missing file: {path.name}")
class TestEntryCount(unittest.TestCase):
"""Each genre file must have at least 100 entries."""
def test_minimum_entries(self):
for genre in REQUIRED_GENRES:
path = DATA_DIR / f"scene-descriptions-{genre}.jsonl"
if not path.exists():
self.fail(f"Missing: {path.name}")
continue
entries = load_jsonl(path)
self.assertGreaterEqual(
len(entries), MIN_ENTRIES,
f"{genre}: only {len(entries)} entries (need {MIN_ENTRIES})",
)
class TestSchemaCompliance(unittest.TestCase):
"""Every entry in every genre file must pass schema validation."""
def test_all_entries_valid(self):
failures = []
for genre in REQUIRED_GENRES:
path = DATA_DIR / f"scene-descriptions-{genre}.jsonl"
if not path.exists():
failures.append(f"{genre}: file missing")
continue
entries = load_jsonl(path)
for i, entry in enumerate(entries):
errors = validate_entry(entry)
for err in errors:
failures.append(f"{genre} line {i+1}: {err}")
self.assertEqual(failures, [], f"Schema violations:\n" + "\n".join(failures[:20]))
class TestArtistAndTimestamp(unittest.TestCase):
"""Every entry must have non-empty artist and valid timestamp."""
def test_artists_present(self):
for genre in REQUIRED_GENRES:
path = DATA_DIR / f"scene-descriptions-{genre}.jsonl"
if not path.exists():
continue
entries = load_jsonl(path)
for i, entry in enumerate(entries):
self.assertIn("artist", entry, f"{genre} line {i+1}: missing artist")
self.assertTrue(
isinstance(entry["artist"], str) and entry["artist"].strip(),
f"{genre} line {i+1}: empty artist",
)
def test_timestamps_valid(self):
import re
for genre in REQUIRED_GENRES:
path = DATA_DIR / f"scene-descriptions-{genre}.jsonl"
if not path.exists():
continue
entries = load_jsonl(path)
for i, entry in enumerate(entries):
ts = entry.get("timestamp", "")
self.assertTrue(
re.match(r"^[0-9]+:[0-5][0-9]$", ts),
f"{genre} line {i+1}: bad timestamp '{ts}'",
)
class TestSceneFields(unittest.TestCase):
"""Scene objects must have all required fields with valid values."""
def test_scene_completeness(self):
for genre in REQUIRED_GENRES:
path = DATA_DIR / f"scene-descriptions-{genre}.jsonl"
if not path.exists():
continue
entries = load_jsonl(path)
for i, entry in enumerate(entries):
scene = entry.get("scene", {})
for field in REQUIRED_SCENE_FIELDS:
self.assertIn(field, scene, f"{genre} line {i+1}: missing scene.{field}")
self.assertIsInstance(scene["colors"], list, f"{genre} line {i+1}: colors not array")
self.assertGreater(len(scene["colors"]), 0, f"{genre} line {i+1}: empty colors")
self.assertGreaterEqual(
len(scene.get("description", "")), 10,
f"{genre} line {i+1}: description too short",
)
if __name__ == "__main__":
unittest.main()

View File

@@ -1,100 +1,100 @@
{"song": "Dirt Road Home", "beat": 1, "lyric_line": "Gravel crunches under the tires slow", "scene": {"mood": "warmth", "colors": ["navy", "silver", "white"], "composition": "close-up", "camera": "slow pan", "description": "A warmth scene: 'Gravel crunches under the tires slow'. close-up with navy, silver, white. Camera: slow pan."}}
{"song": "Dirt Road Home", "beat": 2, "lyric_line": "The oak tree still leans toward the road", "scene": {"mood": "memory", "colors": ["dusty rose", "gold", "brown"], "composition": "wide shot", "camera": "steady", "description": "A memory scene: 'The oak tree still leans toward the road'. wide shot with dusty rose, gold, brown. Camera: steady."}}
{"song": "Dirt Road Home", "beat": 3, "lyric_line": "Mama's porch light through the fog", "scene": {"mood": "bittersweet", "colors": ["orange", "red", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A bittersweet scene: 'Mama's porch light through the fog'. medium shot with orange, red, brown. Camera: handheld."}}
{"song": "Dirt Road Home", "beat": 4, "lyric_line": "Daddy's truck rusting by the barn", "scene": {"mood": "tender", "colors": ["navy", "silver", "white"], "composition": "low angle", "camera": "slow zoom", "description": "A tender scene: 'Daddy's truck rusting by the barn'. low angle with navy, silver, white. Camera: slow zoom."}}
{"song": "Dirt Road Home", "beat": 5, "lyric_line": "The creek remembers my bare feet", "scene": {"mood": "fading", "colors": ["amber", "brown", "cream"], "composition": "high angle", "camera": "tracking", "description": "A fading scene: 'The creek remembers my bare feet'. high angle with amber, brown, cream. Camera: tracking."}}
{"song": "Dirt Road Home", "beat": 6, "lyric_line": "Fireflies spell out summer's name", "scene": {"mood": "warmth", "colors": ["green", "olive", "tan"], "composition": "over-the-shoulder", "camera": "static", "description": "A warmth scene: 'Fireflies spell out summer's name'. over-the-shoulder with green, olive, tan. Camera: static."}}
{"song": "Dirt Road Home", "beat": 7, "lyric_line": "The church bell rings for no one new", "scene": {"mood": "memory", "colors": ["dusty rose", "gold", "brown"], "composition": "profile", "camera": "crane up", "description": "A memory scene: 'The church bell rings for no one new'. profile with dusty rose, gold, brown. Camera: crane up."}}
{"song": "Dirt Road Home", "beat": 8, "lyric_line": "Honeysuckle climbs the fence again", "scene": {"mood": "bittersweet", "colors": ["navy", "silver", "white"], "composition": "bird's eye", "camera": "dolly in", "description": "A bittersweet scene: 'Honeysuckle climbs the fence again'. bird's eye with navy, silver, white. Camera: dolly in."}}
{"song": "Dirt Road Home", "beat": 9, "lyric_line": "The field where we learned to drive", "scene": {"mood": "tender", "colors": ["warm yellow", "barn red", "cream"], "composition": "tracking shot", "camera": "gentle drift", "description": "A tender scene: 'The field where we learned to drive'. tracking shot with warm yellow, barn red, cream. Camera: gentle drift."}}
{"song": "Dirt Road Home", "beat": 10, "lyric_line": "Some roads only go one way home", "scene": {"mood": "fading", "colors": ["forest green", "brown", "gold"], "composition": "establishing", "camera": "locked-off", "description": "A fading scene: 'Some roads only go one way home'. establishing with forest green, brown, gold. Camera: locked-off."}}
{"song": "Last Call Honky-Tonk", "beat": 1, "lyric_line": "Neon beer sign flickers twice", "scene": {"mood": "regret", "colors": ["warm yellow", "barn red", "cream"], "composition": "close-up", "camera": "slow pan", "description": "A regret scene: 'Neon beer sign flickers twice'. close-up with warm yellow, barn red, cream. Camera: slow pan."}}
{"song": "Last Call Honky-Tonk", "beat": 2, "lyric_line": "The jukebox plays what we used to be", "scene": {"mood": "fondness", "colors": ["copper", "dust", "gold"], "composition": "wide shot", "camera": "steady", "description": "A fondness scene: 'The jukebox plays what we used to be'. wide shot with copper, dust, gold. Camera: steady."}}
{"song": "Last Call Honky-Tonk", "beat": 3, "lyric_line": "Boots scuff the dance floor marks", "scene": {"mood": "aging", "colors": ["sunset orange", "purple", "pink"], "composition": "medium shot", "camera": "handheld", "description": "A aging scene: 'Boots scuff the dance floor marks'. medium shot with sunset orange, purple, pink. Camera: handheld."}}
{"song": "Last Call Honky-Tonk", "beat": 4, "lyric_line": "She orders water for the drive", "scene": {"mood": "memory", "colors": ["grey", "mud brown", "green"], "composition": "low angle", "camera": "slow zoom", "description": "A memory scene: 'She orders water for the drive'. low angle with grey, mud brown, green. Camera: slow zoom."}}
{"song": "Last Call Honky-Tonk", "beat": 5, "lyric_line": "The bartender knows both our names", "scene": {"mood": "music", "colors": ["orange", "red", "brown"], "composition": "high angle", "camera": "tracking", "description": "A music scene: 'The bartender knows both our names'. high angle with orange, red, brown. Camera: tracking."}}
{"song": "Last Call Honky-Tonk", "beat": 6, "lyric_line": "Two-step where the floor dips low", "scene": {"mood": "regret", "colors": ["navy", "silver", "white"], "composition": "over-the-shoulder", "camera": "static", "description": "A regret scene: 'Two-step where the floor dips low'. over-the-shoulder with navy, silver, white. Camera: static."}}
{"song": "Last Call Honky-Tonk", "beat": 7, "lyric_line": "Closing time writes the last song", "scene": {"mood": "fondness", "colors": ["sky blue", "wheat", "brown"], "composition": "profile", "camera": "crane up", "description": "A fondness scene: 'Closing time writes the last song'. profile with sky blue, wheat, brown. Camera: crane up."}}
{"song": "Last Call Honky-Tonk", "beat": 8, "lyric_line": "We leave our shadows at the bar", "scene": {"mood": "aging", "colors": ["green", "olive", "tan"], "composition": "bird's eye", "camera": "dolly in", "description": "A aging scene: 'We leave our shadows at the bar'. bird's eye with green, olive, tan. Camera: dolly in."}}
{"song": "Last Call Honky-Tonk", "beat": 9, "lyric_line": "The parking lot smells like rain", "scene": {"mood": "memory", "colors": ["dusty rose", "gold", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A memory scene: 'The parking lot smells like rain'. tracking shot with dusty rose, gold, brown. Camera: gentle drift."}}
{"song": "Last Call Honky-Tonk", "beat": 10, "lyric_line": "Some nights end before the music", "scene": {"mood": "music", "colors": ["navy", "silver", "white"], "composition": "establishing", "camera": "locked-off", "description": "A music scene: 'Some nights end before the music'. establishing with navy, silver, white. Camera: locked-off."}}
{"song": "Church on Sunday", "beat": 1, "lyric_line": "White steeple catches the morning sun", "scene": {"mood": "sacred", "colors": ["forest green", "brown", "gold"], "composition": "close-up", "camera": "slow pan", "description": "A sacred scene: 'White steeple catches the morning sun'. close-up with forest green, brown, gold. Camera: slow pan."}}
{"song": "Church on Sunday", "beat": 2, "lyric_line": "Hymnal pages turn like falling leaves", "scene": {"mood": "quiet", "colors": ["grey", "mud brown", "green"], "composition": "wide shot", "camera": "steady", "description": "A quiet scene: 'Hymnal pages turn like falling leaves'. wide shot with grey, mud brown, green. Camera: steady."}}
{"song": "Church on Sunday", "beat": 3, "lyric_line": "The organ hums beneath the prayers", "scene": {"mood": "devotion", "colors": ["orange", "red", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A devotion scene: 'The organ hums beneath the prayers'. medium shot with orange, red, brown. Camera: handheld."}}
{"song": "Church on Sunday", "beat": 4, "lyric_line": "Grandma's hat blocks the stained glass light", "scene": {"mood": "stillness", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A stillness scene: 'Grandma's hat blocks the stained glass light'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}}
{"song": "Church on Sunday", "beat": 5, "lyric_line": "We kneel on cushions worn by years", "scene": {"mood": "awe", "colors": ["navy", "silver", "white"], "composition": "high angle", "camera": "tracking", "description": "A awe scene: 'We kneel on cushions worn by years'. high angle with navy, silver, white. Camera: tracking."}}
{"song": "Church on Sunday", "beat": 6, "lyric_line": "The preacher's voice fills the gaps", "scene": {"mood": "sacred", "colors": ["sky blue", "wheat", "brown"], "composition": "over-the-shoulder", "camera": "static", "description": "A sacred scene: 'The preacher's voice fills the gaps'. over-the-shoulder with sky blue, wheat, brown. Camera: static."}}
{"song": "Church on Sunday", "beat": 7, "lyric_line": "Offering plate circles like a prayer", "scene": {"mood": "quiet", "colors": ["white", "blue", "brown"], "composition": "profile", "camera": "crane up", "description": "A quiet scene: 'Offering plate circles like a prayer'. profile with white, blue, brown. Camera: crane up."}}
{"song": "Church on Sunday", "beat": 8, "lyric_line": "Amen echoes off the wooden beams", "scene": {"mood": "devotion", "colors": ["dusty rose", "gold", "brown"], "composition": "bird's eye", "camera": "dolly in", "description": "A devotion scene: 'Amen echoes off the wooden beams'. bird's eye with dusty rose, gold, brown. Camera: dolly in."}}
{"song": "Church on Sunday", "beat": 9, "lyric_line": "Children wiggle through the sermon", "scene": {"mood": "stillness", "colors": ["white", "blue", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A stillness scene: 'Children wiggle through the sermon'. tracking shot with white, blue, brown. Camera: gentle drift."}}
{"song": "Church on Sunday", "beat": 10, "lyric_line": "Faith smells like old wood and coffee", "scene": {"mood": "awe", "colors": ["sunset orange", "purple", "pink"], "composition": "establishing", "camera": "locked-off", "description": "A awe scene: 'Faith smells like old wood and coffee'. establishing with sunset orange, purple, pink. Camera: locked-off."}}
{"song": "Tractor at Dawn", "beat": 1, "lyric_line": "Headlights cut the fog at five", "scene": {"mood": "resolve", "colors": ["dusty rose", "gold", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A resolve scene: 'Headlights cut the fog at five'. close-up with dusty rose, gold, brown. Camera: slow pan."}}
{"song": "Tractor at Dawn", "beat": 2, "lyric_line": "The diesel coughs to life like faith", "scene": {"mood": "effort", "colors": ["dusty rose", "gold", "brown"], "composition": "wide shot", "camera": "steady", "description": "A effort scene: 'The diesel coughs to life like faith'. wide shot with dusty rose, gold, brown. Camera: steady."}}
{"song": "Tractor at Dawn", "beat": 3, "lyric_line": "Rows stretch past the tree line", "scene": {"mood": "strength", "colors": ["dusty rose", "gold", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A strength scene: 'Rows stretch past the tree line'. medium shot with dusty rose, gold, brown. Camera: handheld."}}
{"song": "Tractor at Dawn", "beat": 4, "lyric_line": "Dust follows the disc like a ghost", "scene": {"mood": "persistence", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A persistence scene: 'Dust follows the disc like a ghost'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}}
{"song": "Tractor at Dawn", "beat": 5, "lyric_line": "Hands grip leather worn to skin", "scene": {"mood": "focus", "colors": ["amber", "brown", "cream"], "composition": "high angle", "camera": "tracking", "description": "A focus scene: 'Hands grip leather worn to skin'. high angle with amber, brown, cream. Camera: tracking."}}
{"song": "Tractor at Dawn", "beat": 6, "lyric_line": "The sun rises behind the work", "scene": {"mood": "resolve", "colors": ["forest green", "brown", "gold"], "composition": "over-the-shoulder", "camera": "static", "description": "A resolve scene: 'The sun rises behind the work'. over-the-shoulder with forest green, brown, gold. Camera: static."}}
{"song": "Tractor at Dawn", "beat": 7, "lyric_line": "Crows circle what the plow reveals", "scene": {"mood": "effort", "colors": ["amber", "brown", "cream"], "composition": "profile", "camera": "crane up", "description": "A effort scene: 'Crows circle what the plow reveals'. profile with amber, brown, cream. Camera: crane up."}}
{"song": "Tractor at Dawn", "beat": 8, "lyric_line": "Sweat salts the steering wheel", "scene": {"mood": "strength", "colors": ["grey", "mud brown", "green"], "composition": "bird's eye", "camera": "dolly in", "description": "A strength scene: 'Sweat salts the steering wheel'. bird's eye with grey, mud brown, green. Camera: dolly in."}}
{"song": "Tractor at Dawn", "beat": 9, "lyric_line": "Planting season has no snooze", "scene": {"mood": "persistence", "colors": ["green", "olive", "tan"], "composition": "tracking shot", "camera": "gentle drift", "description": "A persistence scene: 'Planting season has no snooze'. tracking shot with green, olive, tan. Camera: gentle drift."}}
{"song": "Tractor at Dawn", "beat": 10, "lyric_line": "Every seed is a prayer for rain", "scene": {"mood": "focus", "colors": ["dusty rose", "gold", "brown"], "composition": "establishing", "camera": "locked-off", "description": "A focus scene: 'Every seed is a prayer for rain'. establishing with dusty rose, gold, brown. Camera: locked-off."}}
{"song": "Letters from Boot Camp", "beat": 1, "lyric_line": "Her handwriting shakes on the envelope", "scene": {"mood": "love", "colors": ["sky blue", "wheat", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A love scene: 'Her handwriting shakes on the envelope'. close-up with sky blue, wheat, brown. Camera: slow pan."}}
{"song": "Letters from Boot Camp", "beat": 2, "lyric_line": "The postmark is two weeks old", "scene": {"mood": "vulnerability", "colors": ["green", "olive", "tan"], "composition": "wide shot", "camera": "steady", "description": "A vulnerability scene: 'The postmark is two weeks old'. wide shot with green, olive, tan. Camera: steady."}}
{"song": "Letters from Boot Camp", "beat": 3, "lyric_line": "He reads it sitting on his bunk", "scene": {"mood": "distance", "colors": ["sky blue", "wheat", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A distance scene: 'He reads it sitting on his bunk'. medium shot with sky blue, wheat, brown. Camera: handheld."}}
{"song": "Letters from Boot Camp", "beat": 4, "lyric_line": "She describes the dog getting bigger", "scene": {"mood": "hope", "colors": ["white", "blue", "brown"], "composition": "low angle", "camera": "slow zoom", "description": "A hope scene: 'She describes the dog getting bigger'. low angle with white, blue, brown. Camera: slow zoom."}}
{"song": "Letters from Boot Camp", "beat": 5, "lyric_line": "Censored words leave blank stares", "scene": {"mood": "longing", "colors": ["sunset orange", "purple", "pink"], "composition": "high angle", "camera": "tracking", "description": "A longing scene: 'Censored words leave blank stares'. high angle with sunset orange, purple, pink. Camera: tracking."}}
{"song": "Letters from Boot Camp", "beat": 6, "lyric_line": "The stamp is peeling at the corner", "scene": {"mood": "love", "colors": ["white", "blue", "brown"], "composition": "over-the-shoulder", "camera": "static", "description": "A love scene: 'The stamp is peeling at the corner'. over-the-shoulder with white, blue, brown. Camera: static."}}
{"song": "Letters from Boot Camp", "beat": 7, "lyric_line": "He folds it back exactly right", "scene": {"mood": "vulnerability", "colors": ["forest green", "brown", "gold"], "composition": "profile", "camera": "crane up", "description": "A vulnerability scene: 'He folds it back exactly right'. profile with forest green, brown, gold. Camera: crane up."}}
{"song": "Letters from Boot Camp", "beat": 8, "lyric_line": "Pictures curl at the edges now", "scene": {"mood": "distance", "colors": ["forest green", "brown", "gold"], "composition": "bird's eye", "camera": "dolly in", "description": "A distance scene: 'Pictures curl at the edges now'. bird's eye with forest green, brown, gold. Camera: dolly in."}}
{"song": "Letters from Boot Camp", "beat": 9, "lyric_line": "Her voice lives between the lines", "scene": {"mood": "hope", "colors": ["orange", "red", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A hope scene: 'Her voice lives between the lines'. tracking shot with orange, red, brown. Camera: gentle drift."}}
{"song": "Letters from Boot Camp", "beat": 10, "lyric_line": "Some letters carry more than words", "scene": {"mood": "longing", "colors": ["sunset orange", "purple", "pink"], "composition": "establishing", "camera": "locked-off", "description": "A longing scene: 'Some letters carry more than words'. establishing with sunset orange, purple, pink. Camera: locked-off."}}
{"song": "Flood Stage", "beat": 1, "lyric_line": "The river rose past the marker", "scene": {"mood": "loss", "colors": ["white", "blue", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A loss scene: 'The river rose past the marker'. close-up with white, blue, brown. Camera: slow pan."}}
{"song": "Flood Stage", "beat": 2, "lyric_line": "Sandbags line the church basement door", "scene": {"mood": "resilience", "colors": ["green", "olive", "tan"], "composition": "wide shot", "camera": "steady", "description": "A resilience scene: 'Sandbags line the church basement door'. wide shot with green, olive, tan. Camera: steady."}}
{"song": "Flood Stage", "beat": 3, "lyric_line": "Photo albums float in the living room", "scene": {"mood": "community", "colors": ["white", "blue", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A community scene: 'Photo albums float in the living room'. medium shot with white, blue, brown. Camera: handheld."}}
{"song": "Flood Stage", "beat": 4, "lyric_line": "The bridge went out Tuesday night", "scene": {"mood": "grief", "colors": ["navy", "silver", "white"], "composition": "low angle", "camera": "slow zoom", "description": "A grief scene: 'The bridge went out Tuesday night'. low angle with navy, silver, white. Camera: slow zoom."}}
{"song": "Flood Stage", "beat": 5, "lyric_line": "Neighbors carry what they can hold", "scene": {"mood": "rebuilding", "colors": ["green", "olive", "tan"], "composition": "high angle", "camera": "tracking", "description": "A rebuilding scene: 'Neighbors carry what they can hold'. high angle with green, olive, tan. Camera: tracking."}}
{"song": "Flood Stage", "beat": 6, "lyric_line": "Water stains reach the second step", "scene": {"mood": "loss", "colors": ["amber", "brown", "cream"], "composition": "over-the-shoulder", "camera": "static", "description": "A loss scene: 'Water stains reach the second step'. over-the-shoulder with amber, brown, cream. Camera: static."}}
{"song": "Flood Stage", "beat": 7, "lyric_line": "The well is tasting like the river", "scene": {"mood": "resilience", "colors": ["forest green", "brown", "gold"], "composition": "profile", "camera": "crane up", "description": "A resilience scene: 'The well is tasting like the river'. profile with forest green, brown, gold. Camera: crane up."}}
{"song": "Flood Stage", "beat": 8, "lyric_line": "Mud lines mark where hope used to sit", "scene": {"mood": "community", "colors": ["warm yellow", "barn red", "cream"], "composition": "bird's eye", "camera": "dolly in", "description": "A community scene: 'Mud lines mark where hope used to sit'. bird's eye with warm yellow, barn red, cream. Camera: dolly in."}}
{"song": "Flood Stage", "beat": 9, "lyric_line": "We rebuild on the same ground", "scene": {"mood": "grief", "colors": ["sunset orange", "purple", "pink"], "composition": "tracking shot", "camera": "gentle drift", "description": "A grief scene: 'We rebuild on the same ground'. tracking shot with sunset orange, purple, pink. Camera: gentle drift."}}
{"song": "Flood Stage", "beat": 10, "lyric_line": "Some floods are just the land remembering", "scene": {"mood": "rebuilding", "colors": ["dusty rose", "gold", "brown"], "composition": "establishing", "camera": "locked-off", "description": "A rebuilding scene: 'Some floods are just the land remembering'. establishing with dusty rose, gold, brown. Camera: locked-off."}}
{"song": "Barbed Wire Waltz", "beat": 1, "lyric_line": "The fence line needs mending again", "scene": {"mood": "endurance", "colors": ["orange", "red", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A endurance scene: 'The fence line needs mending again'. close-up with orange, red, brown. Camera: slow pan."}}
{"song": "Barbed Wire Waltz", "beat": 2, "lyric_line": "Wire cuts through leather gloves", "scene": {"mood": "patience", "colors": ["amber", "brown", "cream"], "composition": "wide shot", "camera": "steady", "description": "A patience scene: 'Wire cuts through leather gloves'. wide shot with amber, brown, cream. Camera: steady."}}
{"song": "Barbed Wire Waltz", "beat": 3, "lyric_line": "Posts lean like tired old men", "scene": {"mood": "repair", "colors": ["grey", "mud brown", "green"], "composition": "medium shot", "camera": "handheld", "description": "A repair scene: 'Posts lean like tired old men'. medium shot with grey, mud brown, green. Camera: handheld."}}
{"song": "Barbed Wire Waltz", "beat": 4, "lyric_line": "The cattle watch from the far hill", "scene": {"mood": "land", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A land scene: 'The cattle watch from the far hill'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}}
{"song": "Barbed Wire Waltz", "beat": 5, "lyric_line": "We stretch and twist and tie", "scene": {"mood": "duty", "colors": ["amber", "brown", "cream"], "composition": "high angle", "camera": "tracking", "description": "A duty scene: 'We stretch and twist and tie'. high angle with amber, brown, cream. Camera: tracking."}}
{"song": "Barbed Wire Waltz", "beat": 6, "lyric_line": "Sunburn writes on the back of necks", "scene": {"mood": "endurance", "colors": ["sunset orange", "purple", "pink"], "composition": "over-the-shoulder", "camera": "static", "description": "A endurance scene: 'Sunburn writes on the back of necks'. over-the-shoulder with sunset orange, purple, pink. Camera: static."}}
{"song": "Barbed Wire Waltz", "beat": 7, "lyric_line": "The wind carries fence wire songs", "scene": {"mood": "patience", "colors": ["dusty rose", "gold", "brown"], "composition": "profile", "camera": "crane up", "description": "A patience scene: 'The wind carries fence wire songs'. profile with dusty rose, gold, brown. Camera: crane up."}}
{"song": "Barbed Wire Waltz", "beat": 8, "lyric_line": "We repair what the storm broke", "scene": {"mood": "repair", "colors": ["white", "blue", "brown"], "composition": "bird's eye", "camera": "dolly in", "description": "A repair scene: 'We repair what the storm broke'. bird's eye with white, blue, brown. Camera: dolly in."}}
{"song": "Barbed Wire Waltz", "beat": 9, "lyric_line": "Patience wears like good boots", "scene": {"mood": "land", "colors": ["copper", "dust", "gold"], "composition": "tracking shot", "camera": "gentle drift", "description": "A land scene: 'Patience wears like good boots'. tracking shot with copper, dust, gold. Camera: gentle drift."}}
{"song": "Barbed Wire Waltz", "beat": 10, "lyric_line": "Some fences are promises to the land", "scene": {"mood": "duty", "colors": ["amber", "brown", "cream"], "composition": "establishing", "camera": "locked-off", "description": "A duty scene: 'Some fences are promises to the land'. establishing with amber, brown, cream. Camera: locked-off."}}
{"song": "Tailgate Sunset", "beat": 1, "lyric_line": "The truck bed holds two lawn chairs", "scene": {"mood": "calm", "colors": ["grey", "mud brown", "green"], "composition": "close-up", "camera": "slow pan", "description": "A calm scene: 'The truck bed holds two lawn chairs'. close-up with grey, mud brown, green. Camera: slow pan."}}
{"song": "Tailgate Sunset", "beat": 2, "lyric_line": "Cooler rattles with ice and hope", "scene": {"mood": "companionship", "colors": ["orange", "red", "brown"], "composition": "wide shot", "camera": "steady", "description": "A companionship scene: 'Cooler rattles with ice and hope'. wide shot with orange, red, brown. Camera: steady."}}
{"song": "Tailgate Sunset", "beat": 3, "lyric_line": "Crickets start before the stars", "scene": {"mood": "simplicity", "colors": ["sky blue", "wheat", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A simplicity scene: 'Crickets start before the stars'. medium shot with sky blue, wheat, brown. Camera: handheld."}}
{"song": "Tailgate Sunset", "beat": 4, "lyric_line": "The highway hums its evening song", "scene": {"mood": "nature", "colors": ["copper", "dust", "gold"], "composition": "low angle", "camera": "slow zoom", "description": "A nature scene: 'The highway hums its evening song'. low angle with copper, dust, gold. Camera: slow zoom."}}
{"song": "Tailgate Sunset", "beat": 5, "lyric_line": "We share a blanket past dark", "scene": {"mood": "contentment", "colors": ["forest green", "brown", "gold"], "composition": "high angle", "camera": "tracking", "description": "A contentment scene: 'We share a blanket past dark'. high angle with forest green, brown, gold. Camera: tracking."}}
{"song": "Tailgate Sunset", "beat": 6, "lyric_line": "Fireflies answer the sunset", "scene": {"mood": "calm", "colors": ["green", "olive", "tan"], "composition": "over-the-shoulder", "camera": "static", "description": "A calm scene: 'Fireflies answer the sunset'. over-the-shoulder with green, olive, tan. Camera: static."}}
{"song": "Tailgate Sunset", "beat": 7, "lyric_line": "Country station plays the old ones", "scene": {"mood": "companionship", "colors": ["navy", "silver", "white"], "composition": "profile", "camera": "crane up", "description": "A companionship scene: 'Country station plays the old ones'. profile with navy, silver, white. Camera: crane up."}}
{"song": "Tailgate Sunset", "beat": 8, "lyric_line": "The tailgate is our table now", "scene": {"mood": "simplicity", "colors": ["sunset orange", "purple", "pink"], "composition": "bird's eye", "camera": "dolly in", "description": "A simplicity scene: 'The tailgate is our table now'. bird's eye with sunset orange, purple, pink. Camera: dolly in."}}
{"song": "Tailgate Sunset", "beat": 9, "lyric_line": "Silence sits between the songs", "scene": {"mood": "nature", "colors": ["sky blue", "wheat", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A nature scene: 'Silence sits between the songs'. tracking shot with sky blue, wheat, brown. Camera: gentle drift."}}
{"song": "Tailgate Sunset", "beat": 10, "lyric_line": "Some nights are church without walls", "scene": {"mood": "contentment", "colors": ["sunset orange", "purple", "pink"], "composition": "establishing", "camera": "locked-off", "description": "A contentment scene: 'Some nights are church without walls'. establishing with sunset orange, purple, pink. Camera: locked-off."}}
{"song": "Granddad's Pocket Knife", "beat": 1, "lyric_line": "Bone handle smooth from his pocket", "scene": {"mood": "sacred", "colors": ["forest green", "brown", "gold"], "composition": "close-up", "camera": "slow pan", "description": "A sacred scene: 'Bone handle smooth from his pocket'. close-up with forest green, brown, gold. Camera: slow pan."}}
{"song": "Granddad's Pocket Knife", "beat": 2, "lyric_line": "Three blades none of them new", "scene": {"mood": "quiet", "colors": ["green", "olive", "tan"], "composition": "wide shot", "camera": "steady", "description": "A quiet scene: 'Three blades none of them new'. wide shot with green, olive, tan. Camera: steady."}}
{"song": "Granddad's Pocket Knife", "beat": 3, "lyric_line": "He carved initials in the oak", "scene": {"mood": "devotion", "colors": ["warm yellow", "barn red", "cream"], "composition": "medium shot", "camera": "handheld", "description": "A devotion scene: 'He carved initials in the oak'. medium shot with warm yellow, barn red, cream. Camera: handheld."}}
{"song": "Granddad's Pocket Knife", "beat": 4, "lyric_line": "The steel remembers every cut", "scene": {"mood": "stillness", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A stillness scene: 'The steel remembers every cut'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}}
{"song": "Granddad's Pocket Knife", "beat": 5, "lyric_line": "I open it the way he taught", "scene": {"mood": "awe", "colors": ["sunset orange", "purple", "pink"], "composition": "high angle", "camera": "tracking", "description": "A awe scene: 'I open it the way he taught'. high angle with sunset orange, purple, pink. Camera: tracking."}}
{"song": "Granddad's Pocket Knife", "beat": 6, "lyric_line": "The smallest blade is for detail", "scene": {"mood": "sacred", "colors": ["green", "olive", "tan"], "composition": "over-the-shoulder", "camera": "static", "description": "A sacred scene: 'The smallest blade is for detail'. over-the-shoulder with green, olive, tan. Camera: static."}}
{"song": "Granddad's Pocket Knife", "beat": 7, "lyric_line": "Whittling shavings catch the light", "scene": {"mood": "quiet", "colors": ["amber", "brown", "cream"], "composition": "profile", "camera": "crane up", "description": "A quiet scene: 'Whittling shavings catch the light'. profile with amber, brown, cream. Camera: crane up."}}
{"song": "Granddad's Pocket Knife", "beat": 8, "lyric_line": "He said a knife is a promise", "scene": {"mood": "devotion", "colors": ["sunset orange", "purple", "pink"], "composition": "bird's eye", "camera": "dolly in", "description": "A devotion scene: 'He said a knife is a promise'. bird's eye with sunset orange, purple, pink. Camera: dolly in."}}
{"song": "Granddad's Pocket Knife", "beat": 9, "lyric_line": "I carry it in my front pocket", "scene": {"mood": "stillness", "colors": ["forest green", "brown", "gold"], "composition": "tracking shot", "camera": "gentle drift", "description": "A stillness scene: 'I carry it in my front pocket'. tracking shot with forest green, brown, gold. Camera: gentle drift."}}
{"song": "Granddad's Pocket Knife", "beat": 10, "lyric_line": "Some tools are heirlooms of skill", "scene": {"mood": "awe", "colors": ["orange", "red", "brown"], "composition": "establishing", "camera": "locked-off", "description": "A awe scene: 'Some tools are heirlooms of skill'. establishing with orange, red, brown. Camera: locked-off."}}
{"song": "County Fair", "beat": 1, "lyric_line": "Ferris wheel lights the harvest sky", "scene": {"mood": "celebration", "colors": ["green", "olive", "tan"], "composition": "close-up", "camera": "slow pan", "description": "A celebration scene: 'Ferris wheel lights the harvest sky'. close-up with green, olive, tan. Camera: slow pan."}}
{"song": "County Fair", "beat": 2, "lyric_line": "Corn dogs and lemon shake-ups", "scene": {"mood": "community", "colors": ["orange", "red", "brown"], "composition": "wide shot", "camera": "steady", "description": "A community scene: 'Corn dogs and lemon shake-ups'. wide shot with orange, red, brown. Camera: steady."}}
{"song": "County Fair", "beat": 3, "lyric_line": "The livestock barn smells like home", "scene": {"mood": "youth", "colors": ["sunset orange", "purple", "pink"], "composition": "medium shot", "camera": "handheld", "description": "A youth scene: 'The livestock barn smells like home'. medium shot with sunset orange, purple, pink. Camera: handheld."}}
{"song": "County Fair", "beat": 4, "lyric_line": "Blue ribbons pinned to quilt squares", "scene": {"mood": "harvest", "colors": ["dusty rose", "gold", "brown"], "composition": "low angle", "camera": "slow zoom", "description": "A harvest scene: 'Blue ribbons pinned to quilt squares'. low angle with dusty rose, gold, brown. Camera: slow zoom."}}
{"song": "County Fair", "beat": 5, "lyric_line": "Kids scream on the zipper ride", "scene": {"mood": "fun", "colors": ["grey", "mud brown", "green"], "composition": "high angle", "camera": "tracking", "description": "A fun scene: 'Kids scream on the zipper ride'. high angle with grey, mud brown, green. Camera: tracking."}}
{"song": "County Fair", "beat": 6, "lyric_line": "Band plays covers by the stage", "scene": {"mood": "celebration", "colors": ["navy", "silver", "white"], "composition": "over-the-shoulder", "camera": "static", "description": "A celebration scene: 'Band plays covers by the stage'. over-the-shoulder with navy, silver, white. Camera: static."}}
{"song": "County Fair", "beat": 7, "lyric_line": "Hay bales line the midway path", "scene": {"mood": "community", "colors": ["sky blue", "wheat", "brown"], "composition": "profile", "camera": "crane up", "description": "A community scene: 'Hay bales line the midway path'. profile with sky blue, wheat, brown. Camera: crane up."}}
{"song": "County Fair", "beat": 8, "lyric_line": "We split the funnel cake in half", "scene": {"mood": "youth", "colors": ["copper", "dust", "gold"], "composition": "bird's eye", "camera": "dolly in", "description": "A youth scene: 'We split the funnel cake in half'. bird's eye with copper, dust, gold. Camera: dolly in."}}
{"song": "County Fair", "beat": 9, "lyric_line": "The demolition derby is last", "scene": {"mood": "harvest", "colors": ["sunset orange", "purple", "pink"], "composition": "tracking shot", "camera": "gentle drift", "description": "A harvest scene: 'The demolition derby is last'. tracking shot with sunset orange, purple, pink. Camera: gentle drift."}}
{"song": "County Fair", "beat": 10, "lyric_line": "Some summers live in a single night", "scene": {"mood": "fun", "colors": ["warm yellow", "barn red", "cream"], "composition": "establishing", "camera": "locked-off", "description": "A fun scene: 'Some summers live in a single night'. establishing with warm yellow, barn red, cream. Camera: locked-off."}}
{"song": "Dirt Road Home", "beat": 1, "lyric_line": "Gravel crunches under the tires slow", "scene": {"mood": "warmth", "colors": ["navy", "silver", "white"], "composition": "close-up", "camera": "slow pan", "description": "A warmth scene: 'Gravel crunches under the tires slow'. close-up with navy, silver, white. Camera: slow pan."}, "artist": "Whiskey Hollow", "timestamp": "0:00", "genre": "country"}
{"song": "Dirt Road Home", "beat": 2, "lyric_line": "The oak tree still leans toward the road", "scene": {"mood": "memory", "colors": ["dusty rose", "gold", "brown"], "composition": "wide shot", "camera": "steady", "description": "A memory scene: 'The oak tree still leans toward the road'. wide shot with dusty rose, gold, brown. Camera: steady."}, "artist": "Whiskey Hollow", "timestamp": "0:30", "genre": "country"}
{"song": "Dirt Road Home", "beat": 3, "lyric_line": "Mama's porch light through the fog", "scene": {"mood": "bittersweet", "colors": ["orange", "red", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A bittersweet scene: 'Mama's porch light through the fog'. medium shot with orange, red, brown. Camera: handheld."}, "artist": "Whiskey Hollow", "timestamp": "1:00", "genre": "country"}
{"song": "Dirt Road Home", "beat": 4, "lyric_line": "Daddy's truck rusting by the barn", "scene": {"mood": "tender", "colors": ["navy", "silver", "white"], "composition": "low angle", "camera": "slow zoom", "description": "A tender scene: 'Daddy's truck rusting by the barn'. low angle with navy, silver, white. Camera: slow zoom."}, "artist": "Whiskey Hollow", "timestamp": "1:30", "genre": "country"}
{"song": "Dirt Road Home", "beat": 5, "lyric_line": "The creek remembers my bare feet", "scene": {"mood": "fading", "colors": ["amber", "brown", "cream"], "composition": "high angle", "camera": "tracking", "description": "A fading scene: 'The creek remembers my bare feet'. high angle with amber, brown, cream. Camera: tracking."}, "artist": "Whiskey Hollow", "timestamp": "2:00", "genre": "country"}
{"song": "Dirt Road Home", "beat": 6, "lyric_line": "Fireflies spell out summer's name", "scene": {"mood": "warmth", "colors": ["green", "olive", "tan"], "composition": "over-the-shoulder", "camera": "static", "description": "A warmth scene: 'Fireflies spell out summer's name'. over-the-shoulder with green, olive, tan. Camera: static."}, "artist": "Whiskey Hollow", "timestamp": "2:30", "genre": "country"}
{"song": "Dirt Road Home", "beat": 7, "lyric_line": "The church bell rings for no one new", "scene": {"mood": "memory", "colors": ["dusty rose", "gold", "brown"], "composition": "profile", "camera": "crane up", "description": "A memory scene: 'The church bell rings for no one new'. profile with dusty rose, gold, brown. Camera: crane up."}, "artist": "Whiskey Hollow", "timestamp": "3:00", "genre": "country"}
{"song": "Dirt Road Home", "beat": 8, "lyric_line": "Honeysuckle climbs the fence again", "scene": {"mood": "bittersweet", "colors": ["navy", "silver", "white"], "composition": "bird's eye", "camera": "dolly in", "description": "A bittersweet scene: 'Honeysuckle climbs the fence again'. bird's eye with navy, silver, white. Camera: dolly in."}, "artist": "Whiskey Hollow", "timestamp": "3:30", "genre": "country"}
{"song": "Dirt Road Home", "beat": 9, "lyric_line": "The field where we learned to drive", "scene": {"mood": "tender", "colors": ["warm yellow", "barn red", "cream"], "composition": "tracking shot", "camera": "gentle drift", "description": "A tender scene: 'The field where we learned to drive'. tracking shot with warm yellow, barn red, cream. Camera: gentle drift."}, "artist": "Whiskey Hollow", "timestamp": "4:00", "genre": "country"}
{"song": "Dirt Road Home", "beat": 10, "lyric_line": "Some roads only go one way home", "scene": {"mood": "fading", "colors": ["forest green", "brown", "gold"], "composition": "establishing", "camera": "locked-off", "description": "A fading scene: 'Some roads only go one way home'. establishing with forest green, brown, gold. Camera: locked-off."}, "artist": "Whiskey Hollow", "timestamp": "4:30", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 1, "lyric_line": "Neon beer sign flickers twice", "scene": {"mood": "regret", "colors": ["warm yellow", "barn red", "cream"], "composition": "close-up", "camera": "slow pan", "description": "A regret scene: 'Neon beer sign flickers twice'. close-up with warm yellow, barn red, cream. Camera: slow pan."}, "artist": "Neon Nashville", "timestamp": "0:00", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 2, "lyric_line": "The jukebox plays what we used to be", "scene": {"mood": "fondness", "colors": ["copper", "dust", "gold"], "composition": "wide shot", "camera": "steady", "description": "A fondness scene: 'The jukebox plays what we used to be'. wide shot with copper, dust, gold. Camera: steady."}, "artist": "Neon Nashville", "timestamp": "0:30", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 3, "lyric_line": "Boots scuff the dance floor marks", "scene": {"mood": "aging", "colors": ["sunset orange", "purple", "pink"], "composition": "medium shot", "camera": "handheld", "description": "A aging scene: 'Boots scuff the dance floor marks'. medium shot with sunset orange, purple, pink. Camera: handheld."}, "artist": "Neon Nashville", "timestamp": "1:00", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 4, "lyric_line": "She orders water for the drive", "scene": {"mood": "memory", "colors": ["grey", "mud brown", "green"], "composition": "low angle", "camera": "slow zoom", "description": "A memory scene: 'She orders water for the drive'. low angle with grey, mud brown, green. Camera: slow zoom."}, "artist": "Neon Nashville", "timestamp": "1:30", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 5, "lyric_line": "The bartender knows both our names", "scene": {"mood": "music", "colors": ["orange", "red", "brown"], "composition": "high angle", "camera": "tracking", "description": "A music scene: 'The bartender knows both our names'. high angle with orange, red, brown. Camera: tracking."}, "artist": "Neon Nashville", "timestamp": "2:00", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 6, "lyric_line": "Two-step where the floor dips low", "scene": {"mood": "regret", "colors": ["navy", "silver", "white"], "composition": "over-the-shoulder", "camera": "static", "description": "A regret scene: 'Two-step where the floor dips low'. over-the-shoulder with navy, silver, white. Camera: static."}, "artist": "Neon Nashville", "timestamp": "2:30", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 7, "lyric_line": "Closing time writes the last song", "scene": {"mood": "fondness", "colors": ["sky blue", "wheat", "brown"], "composition": "profile", "camera": "crane up", "description": "A fondness scene: 'Closing time writes the last song'. profile with sky blue, wheat, brown. Camera: crane up."}, "artist": "Neon Nashville", "timestamp": "3:00", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 8, "lyric_line": "We leave our shadows at the bar", "scene": {"mood": "aging", "colors": ["green", "olive", "tan"], "composition": "bird's eye", "camera": "dolly in", "description": "A aging scene: 'We leave our shadows at the bar'. bird's eye with green, olive, tan. Camera: dolly in."}, "artist": "Neon Nashville", "timestamp": "3:30", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 9, "lyric_line": "The parking lot smells like rain", "scene": {"mood": "memory", "colors": ["dusty rose", "gold", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A memory scene: 'The parking lot smells like rain'. tracking shot with dusty rose, gold, brown. Camera: gentle drift."}, "artist": "Neon Nashville", "timestamp": "4:00", "genre": "country"}
{"song": "Last Call Honky-Tonk", "beat": 10, "lyric_line": "Some nights end before the music", "scene": {"mood": "music", "colors": ["navy", "silver", "white"], "composition": "establishing", "camera": "locked-off", "description": "A music scene: 'Some nights end before the music'. establishing with navy, silver, white. Camera: locked-off."}, "artist": "Neon Nashville", "timestamp": "4:30", "genre": "country"}
{"song": "Church on Sunday", "beat": 1, "lyric_line": "White steeple catches the morning sun", "scene": {"mood": "sacred", "colors": ["forest green", "brown", "gold"], "composition": "close-up", "camera": "slow pan", "description": "A sacred scene: 'White steeple catches the morning sun'. close-up with forest green, brown, gold. Camera: slow pan."}, "artist": "Magnolia Grace", "timestamp": "0:00", "genre": "country"}
{"song": "Church on Sunday", "beat": 2, "lyric_line": "Hymnal pages turn like falling leaves", "scene": {"mood": "quiet", "colors": ["grey", "mud brown", "green"], "composition": "wide shot", "camera": "steady", "description": "A quiet scene: 'Hymnal pages turn like falling leaves'. wide shot with grey, mud brown, green. Camera: steady."}, "artist": "Magnolia Grace", "timestamp": "0:30", "genre": "country"}
{"song": "Church on Sunday", "beat": 3, "lyric_line": "The organ hums beneath the prayers", "scene": {"mood": "devotion", "colors": ["orange", "red", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A devotion scene: 'The organ hums beneath the prayers'. medium shot with orange, red, brown. Camera: handheld."}, "artist": "Magnolia Grace", "timestamp": "1:00", "genre": "country"}
{"song": "Church on Sunday", "beat": 4, "lyric_line": "Grandma's hat blocks the stained glass light", "scene": {"mood": "stillness", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A stillness scene: 'Grandma's hat blocks the stained glass light'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}, "artist": "Magnolia Grace", "timestamp": "1:30", "genre": "country"}
{"song": "Church on Sunday", "beat": 5, "lyric_line": "We kneel on cushions worn by years", "scene": {"mood": "awe", "colors": ["navy", "silver", "white"], "composition": "high angle", "camera": "tracking", "description": "A awe scene: 'We kneel on cushions worn by years'. high angle with navy, silver, white. Camera: tracking."}, "artist": "Magnolia Grace", "timestamp": "2:00", "genre": "country"}
{"song": "Church on Sunday", "beat": 6, "lyric_line": "The preacher's voice fills the gaps", "scene": {"mood": "sacred", "colors": ["sky blue", "wheat", "brown"], "composition": "over-the-shoulder", "camera": "static", "description": "A sacred scene: 'The preacher's voice fills the gaps'. over-the-shoulder with sky blue, wheat, brown. Camera: static."}, "artist": "Magnolia Grace", "timestamp": "2:30", "genre": "country"}
{"song": "Church on Sunday", "beat": 7, "lyric_line": "Offering plate circles like a prayer", "scene": {"mood": "quiet", "colors": ["white", "blue", "brown"], "composition": "profile", "camera": "crane up", "description": "A quiet scene: 'Offering plate circles like a prayer'. profile with white, blue, brown. Camera: crane up."}, "artist": "Magnolia Grace", "timestamp": "3:00", "genre": "country"}
{"song": "Church on Sunday", "beat": 8, "lyric_line": "Amen echoes off the wooden beams", "scene": {"mood": "devotion", "colors": ["dusty rose", "gold", "brown"], "composition": "bird's eye", "camera": "dolly in", "description": "A devotion scene: 'Amen echoes off the wooden beams'. bird's eye with dusty rose, gold, brown. Camera: dolly in."}, "artist": "Magnolia Grace", "timestamp": "3:30", "genre": "country"}
{"song": "Church on Sunday", "beat": 9, "lyric_line": "Children wiggle through the sermon", "scene": {"mood": "stillness", "colors": ["white", "blue", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A stillness scene: 'Children wiggle through the sermon'. tracking shot with white, blue, brown. Camera: gentle drift."}, "artist": "Magnolia Grace", "timestamp": "4:00", "genre": "country"}
{"song": "Church on Sunday", "beat": 10, "lyric_line": "Faith smells like old wood and coffee", "scene": {"mood": "awe", "colors": ["sunset orange", "purple", "pink"], "composition": "establishing", "camera": "locked-off", "description": "A awe scene: 'Faith smells like old wood and coffee'. establishing with sunset orange, purple, pink. Camera: locked-off."}, "artist": "Magnolia Grace", "timestamp": "4:30", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 1, "lyric_line": "Headlights cut the fog at five", "scene": {"mood": "resolve", "colors": ["dusty rose", "gold", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A resolve scene: 'Headlights cut the fog at five'. close-up with dusty rose, gold, brown. Camera: slow pan."}, "artist": "Greenfield Sons", "timestamp": "0:00", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 2, "lyric_line": "The diesel coughs to life like faith", "scene": {"mood": "effort", "colors": ["dusty rose", "gold", "brown"], "composition": "wide shot", "camera": "steady", "description": "A effort scene: 'The diesel coughs to life like faith'. wide shot with dusty rose, gold, brown. Camera: steady."}, "artist": "Greenfield Sons", "timestamp": "0:30", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 3, "lyric_line": "Rows stretch past the tree line", "scene": {"mood": "strength", "colors": ["dusty rose", "gold", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A strength scene: 'Rows stretch past the tree line'. medium shot with dusty rose, gold, brown. Camera: handheld."}, "artist": "Greenfield Sons", "timestamp": "1:00", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 4, "lyric_line": "Dust follows the disc like a ghost", "scene": {"mood": "persistence", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A persistence scene: 'Dust follows the disc like a ghost'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}, "artist": "Greenfield Sons", "timestamp": "1:30", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 5, "lyric_line": "Hands grip leather worn to skin", "scene": {"mood": "focus", "colors": ["amber", "brown", "cream"], "composition": "high angle", "camera": "tracking", "description": "A focus scene: 'Hands grip leather worn to skin'. high angle with amber, brown, cream. Camera: tracking."}, "artist": "Greenfield Sons", "timestamp": "2:00", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 6, "lyric_line": "The sun rises behind the work", "scene": {"mood": "resolve", "colors": ["forest green", "brown", "gold"], "composition": "over-the-shoulder", "camera": "static", "description": "A resolve scene: 'The sun rises behind the work'. over-the-shoulder with forest green, brown, gold. Camera: static."}, "artist": "Greenfield Sons", "timestamp": "2:30", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 7, "lyric_line": "Crows circle what the plow reveals", "scene": {"mood": "effort", "colors": ["amber", "brown", "cream"], "composition": "profile", "camera": "crane up", "description": "A effort scene: 'Crows circle what the plow reveals'. profile with amber, brown, cream. Camera: crane up."}, "artist": "Greenfield Sons", "timestamp": "3:00", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 8, "lyric_line": "Sweat salts the steering wheel", "scene": {"mood": "strength", "colors": ["grey", "mud brown", "green"], "composition": "bird's eye", "camera": "dolly in", "description": "A strength scene: 'Sweat salts the steering wheel'. bird's eye with grey, mud brown, green. Camera: dolly in."}, "artist": "Greenfield Sons", "timestamp": "3:30", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 9, "lyric_line": "Planting season has no snooze", "scene": {"mood": "persistence", "colors": ["green", "olive", "tan"], "composition": "tracking shot", "camera": "gentle drift", "description": "A persistence scene: 'Planting season has no snooze'. tracking shot with green, olive, tan. Camera: gentle drift."}, "artist": "Greenfield Sons", "timestamp": "4:00", "genre": "country"}
{"song": "Tractor at Dawn", "beat": 10, "lyric_line": "Every seed is a prayer for rain", "scene": {"mood": "focus", "colors": ["dusty rose", "gold", "brown"], "composition": "establishing", "camera": "locked-off", "description": "A focus scene: 'Every seed is a prayer for rain'. establishing with dusty rose, gold, brown. Camera: locked-off."}, "artist": "Greenfield Sons", "timestamp": "4:30", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 1, "lyric_line": "Her handwriting shakes on the envelope", "scene": {"mood": "love", "colors": ["sky blue", "wheat", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A love scene: 'Her handwriting shakes on the envelope'. close-up with sky blue, wheat, brown. Camera: slow pan."}, "artist": "Patriot Ridge", "timestamp": "0:00", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 2, "lyric_line": "The postmark is two weeks old", "scene": {"mood": "vulnerability", "colors": ["green", "olive", "tan"], "composition": "wide shot", "camera": "steady", "description": "A vulnerability scene: 'The postmark is two weeks old'. wide shot with green, olive, tan. Camera: steady."}, "artist": "Patriot Ridge", "timestamp": "0:30", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 3, "lyric_line": "He reads it sitting on his bunk", "scene": {"mood": "distance", "colors": ["sky blue", "wheat", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A distance scene: 'He reads it sitting on his bunk'. medium shot with sky blue, wheat, brown. Camera: handheld."}, "artist": "Patriot Ridge", "timestamp": "1:00", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 4, "lyric_line": "She describes the dog getting bigger", "scene": {"mood": "hope", "colors": ["white", "blue", "brown"], "composition": "low angle", "camera": "slow zoom", "description": "A hope scene: 'She describes the dog getting bigger'. low angle with white, blue, brown. Camera: slow zoom."}, "artist": "Patriot Ridge", "timestamp": "1:30", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 5, "lyric_line": "Censored words leave blank stares", "scene": {"mood": "longing", "colors": ["sunset orange", "purple", "pink"], "composition": "high angle", "camera": "tracking", "description": "A longing scene: 'Censored words leave blank stares'. high angle with sunset orange, purple, pink. Camera: tracking."}, "artist": "Patriot Ridge", "timestamp": "2:00", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 6, "lyric_line": "The stamp is peeling at the corner", "scene": {"mood": "love", "colors": ["white", "blue", "brown"], "composition": "over-the-shoulder", "camera": "static", "description": "A love scene: 'The stamp is peeling at the corner'. over-the-shoulder with white, blue, brown. Camera: static."}, "artist": "Patriot Ridge", "timestamp": "2:30", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 7, "lyric_line": "He folds it back exactly right", "scene": {"mood": "vulnerability", "colors": ["forest green", "brown", "gold"], "composition": "profile", "camera": "crane up", "description": "A vulnerability scene: 'He folds it back exactly right'. profile with forest green, brown, gold. Camera: crane up."}, "artist": "Patriot Ridge", "timestamp": "3:00", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 8, "lyric_line": "Pictures curl at the edges now", "scene": {"mood": "distance", "colors": ["forest green", "brown", "gold"], "composition": "bird's eye", "camera": "dolly in", "description": "A distance scene: 'Pictures curl at the edges now'. bird's eye with forest green, brown, gold. Camera: dolly in."}, "artist": "Patriot Ridge", "timestamp": "3:30", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 9, "lyric_line": "Her voice lives between the lines", "scene": {"mood": "hope", "colors": ["orange", "red", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A hope scene: 'Her voice lives between the lines'. tracking shot with orange, red, brown. Camera: gentle drift."}, "artist": "Patriot Ridge", "timestamp": "4:00", "genre": "country"}
{"song": "Letters from Boot Camp", "beat": 10, "lyric_line": "Some letters carry more than words", "scene": {"mood": "longing", "colors": ["sunset orange", "purple", "pink"], "composition": "establishing", "camera": "locked-off", "description": "A longing scene: 'Some letters carry more than words'. establishing with sunset orange, purple, pink. Camera: locked-off."}, "artist": "Patriot Ridge", "timestamp": "4:30", "genre": "country"}
{"song": "Flood Stage", "beat": 1, "lyric_line": "The river rose past the marker", "scene": {"mood": "loss", "colors": ["white", "blue", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A loss scene: 'The river rose past the marker'. close-up with white, blue, brown. Camera: slow pan."}, "artist": "River County", "timestamp": "0:00", "genre": "country"}
{"song": "Flood Stage", "beat": 2, "lyric_line": "Sandbags line the church basement door", "scene": {"mood": "resilience", "colors": ["green", "olive", "tan"], "composition": "wide shot", "camera": "steady", "description": "A resilience scene: 'Sandbags line the church basement door'. wide shot with green, olive, tan. Camera: steady."}, "artist": "River County", "timestamp": "0:30", "genre": "country"}
{"song": "Flood Stage", "beat": 3, "lyric_line": "Photo albums float in the living room", "scene": {"mood": "community", "colors": ["white", "blue", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A community scene: 'Photo albums float in the living room'. medium shot with white, blue, brown. Camera: handheld."}, "artist": "River County", "timestamp": "1:00", "genre": "country"}
{"song": "Flood Stage", "beat": 4, "lyric_line": "The bridge went out Tuesday night", "scene": {"mood": "grief", "colors": ["navy", "silver", "white"], "composition": "low angle", "camera": "slow zoom", "description": "A grief scene: 'The bridge went out Tuesday night'. low angle with navy, silver, white. Camera: slow zoom."}, "artist": "River County", "timestamp": "1:30", "genre": "country"}
{"song": "Flood Stage", "beat": 5, "lyric_line": "Neighbors carry what they can hold", "scene": {"mood": "rebuilding", "colors": ["green", "olive", "tan"], "composition": "high angle", "camera": "tracking", "description": "A rebuilding scene: 'Neighbors carry what they can hold'. high angle with green, olive, tan. Camera: tracking."}, "artist": "River County", "timestamp": "2:00", "genre": "country"}
{"song": "Flood Stage", "beat": 6, "lyric_line": "Water stains reach the second step", "scene": {"mood": "loss", "colors": ["amber", "brown", "cream"], "composition": "over-the-shoulder", "camera": "static", "description": "A loss scene: 'Water stains reach the second step'. over-the-shoulder with amber, brown, cream. Camera: static."}, "artist": "River County", "timestamp": "2:30", "genre": "country"}
{"song": "Flood Stage", "beat": 7, "lyric_line": "The well is tasting like the river", "scene": {"mood": "resilience", "colors": ["forest green", "brown", "gold"], "composition": "profile", "camera": "crane up", "description": "A resilience scene: 'The well is tasting like the river'. profile with forest green, brown, gold. Camera: crane up."}, "artist": "River County", "timestamp": "3:00", "genre": "country"}
{"song": "Flood Stage", "beat": 8, "lyric_line": "Mud lines mark where hope used to sit", "scene": {"mood": "community", "colors": ["warm yellow", "barn red", "cream"], "composition": "bird's eye", "camera": "dolly in", "description": "A community scene: 'Mud lines mark where hope used to sit'. bird's eye with warm yellow, barn red, cream. Camera: dolly in."}, "artist": "River County", "timestamp": "3:30", "genre": "country"}
{"song": "Flood Stage", "beat": 9, "lyric_line": "We rebuild on the same ground", "scene": {"mood": "grief", "colors": ["sunset orange", "purple", "pink"], "composition": "tracking shot", "camera": "gentle drift", "description": "A grief scene: 'We rebuild on the same ground'. tracking shot with sunset orange, purple, pink. Camera: gentle drift."}, "artist": "River County", "timestamp": "4:00", "genre": "country"}
{"song": "Flood Stage", "beat": 10, "lyric_line": "Some floods are just the land remembering", "scene": {"mood": "rebuilding", "colors": ["dusty rose", "gold", "brown"], "composition": "establishing", "camera": "locked-off", "description": "A rebuilding scene: 'Some floods are just the land remembering'. establishing with dusty rose, gold, brown. Camera: locked-off."}, "artist": "River County", "timestamp": "4:30", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 1, "lyric_line": "The fence line needs mending again", "scene": {"mood": "endurance", "colors": ["orange", "red", "brown"], "composition": "close-up", "camera": "slow pan", "description": "A endurance scene: 'The fence line needs mending again'. close-up with orange, red, brown. Camera: slow pan."}, "artist": "Dusty Plains Band", "timestamp": "0:00", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 2, "lyric_line": "Wire cuts through leather gloves", "scene": {"mood": "patience", "colors": ["amber", "brown", "cream"], "composition": "wide shot", "camera": "steady", "description": "A patience scene: 'Wire cuts through leather gloves'. wide shot with amber, brown, cream. Camera: steady."}, "artist": "Dusty Plains Band", "timestamp": "0:30", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 3, "lyric_line": "Posts lean like tired old men", "scene": {"mood": "repair", "colors": ["grey", "mud brown", "green"], "composition": "medium shot", "camera": "handheld", "description": "A repair scene: 'Posts lean like tired old men'. medium shot with grey, mud brown, green. Camera: handheld."}, "artist": "Dusty Plains Band", "timestamp": "1:00", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 4, "lyric_line": "The cattle watch from the far hill", "scene": {"mood": "land", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A land scene: 'The cattle watch from the far hill'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}, "artist": "Dusty Plains Band", "timestamp": "1:30", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 5, "lyric_line": "We stretch and twist and tie", "scene": {"mood": "duty", "colors": ["amber", "brown", "cream"], "composition": "high angle", "camera": "tracking", "description": "A duty scene: 'We stretch and twist and tie'. high angle with amber, brown, cream. Camera: tracking."}, "artist": "Dusty Plains Band", "timestamp": "2:00", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 6, "lyric_line": "Sunburn writes on the back of necks", "scene": {"mood": "endurance", "colors": ["sunset orange", "purple", "pink"], "composition": "over-the-shoulder", "camera": "static", "description": "A endurance scene: 'Sunburn writes on the back of necks'. over-the-shoulder with sunset orange, purple, pink. Camera: static."}, "artist": "Dusty Plains Band", "timestamp": "2:30", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 7, "lyric_line": "The wind carries fence wire songs", "scene": {"mood": "patience", "colors": ["dusty rose", "gold", "brown"], "composition": "profile", "camera": "crane up", "description": "A patience scene: 'The wind carries fence wire songs'. profile with dusty rose, gold, brown. Camera: crane up."}, "artist": "Dusty Plains Band", "timestamp": "3:00", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 8, "lyric_line": "We repair what the storm broke", "scene": {"mood": "repair", "colors": ["white", "blue", "brown"], "composition": "bird's eye", "camera": "dolly in", "description": "A repair scene: 'We repair what the storm broke'. bird's eye with white, blue, brown. Camera: dolly in."}, "artist": "Dusty Plains Band", "timestamp": "3:30", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 9, "lyric_line": "Patience wears like good boots", "scene": {"mood": "land", "colors": ["copper", "dust", "gold"], "composition": "tracking shot", "camera": "gentle drift", "description": "A land scene: 'Patience wears like good boots'. tracking shot with copper, dust, gold. Camera: gentle drift."}, "artist": "Dusty Plains Band", "timestamp": "4:00", "genre": "country"}
{"song": "Barbed Wire Waltz", "beat": 10, "lyric_line": "Some fences are promises to the land", "scene": {"mood": "duty", "colors": ["amber", "brown", "cream"], "composition": "establishing", "camera": "locked-off", "description": "A duty scene: 'Some fences are promises to the land'. establishing with amber, brown, cream. Camera: locked-off."}, "artist": "Dusty Plains Band", "timestamp": "4:30", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 1, "lyric_line": "The truck bed holds two lawn chairs", "scene": {"mood": "calm", "colors": ["grey", "mud brown", "green"], "composition": "close-up", "camera": "slow pan", "description": "A calm scene: 'The truck bed holds two lawn chairs'. close-up with grey, mud brown, green. Camera: slow pan."}, "artist": "Two Lane Highway", "timestamp": "0:00", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 2, "lyric_line": "Cooler rattles with ice and hope", "scene": {"mood": "companionship", "colors": ["orange", "red", "brown"], "composition": "wide shot", "camera": "steady", "description": "A companionship scene: 'Cooler rattles with ice and hope'. wide shot with orange, red, brown. Camera: steady."}, "artist": "Two Lane Highway", "timestamp": "0:30", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 3, "lyric_line": "Crickets start before the stars", "scene": {"mood": "simplicity", "colors": ["sky blue", "wheat", "brown"], "composition": "medium shot", "camera": "handheld", "description": "A simplicity scene: 'Crickets start before the stars'. medium shot with sky blue, wheat, brown. Camera: handheld."}, "artist": "Two Lane Highway", "timestamp": "1:00", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 4, "lyric_line": "The highway hums its evening song", "scene": {"mood": "nature", "colors": ["copper", "dust", "gold"], "composition": "low angle", "camera": "slow zoom", "description": "A nature scene: 'The highway hums its evening song'. low angle with copper, dust, gold. Camera: slow zoom."}, "artist": "Two Lane Highway", "timestamp": "1:30", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 5, "lyric_line": "We share a blanket past dark", "scene": {"mood": "contentment", "colors": ["forest green", "brown", "gold"], "composition": "high angle", "camera": "tracking", "description": "A contentment scene: 'We share a blanket past dark'. high angle with forest green, brown, gold. Camera: tracking."}, "artist": "Two Lane Highway", "timestamp": "2:00", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 6, "lyric_line": "Fireflies answer the sunset", "scene": {"mood": "calm", "colors": ["green", "olive", "tan"], "composition": "over-the-shoulder", "camera": "static", "description": "A calm scene: 'Fireflies answer the sunset'. over-the-shoulder with green, olive, tan. Camera: static."}, "artist": "Two Lane Highway", "timestamp": "2:30", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 7, "lyric_line": "Country station plays the old ones", "scene": {"mood": "companionship", "colors": ["navy", "silver", "white"], "composition": "profile", "camera": "crane up", "description": "A companionship scene: 'Country station plays the old ones'. profile with navy, silver, white. Camera: crane up."}, "artist": "Two Lane Highway", "timestamp": "3:00", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 8, "lyric_line": "The tailgate is our table now", "scene": {"mood": "simplicity", "colors": ["sunset orange", "purple", "pink"], "composition": "bird's eye", "camera": "dolly in", "description": "A simplicity scene: 'The tailgate is our table now'. bird's eye with sunset orange, purple, pink. Camera: dolly in."}, "artist": "Two Lane Highway", "timestamp": "3:30", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 9, "lyric_line": "Silence sits between the songs", "scene": {"mood": "nature", "colors": ["sky blue", "wheat", "brown"], "composition": "tracking shot", "camera": "gentle drift", "description": "A nature scene: 'Silence sits between the songs'. tracking shot with sky blue, wheat, brown. Camera: gentle drift."}, "artist": "Two Lane Highway", "timestamp": "4:00", "genre": "country"}
{"song": "Tailgate Sunset", "beat": 10, "lyric_line": "Some nights are church without walls", "scene": {"mood": "contentment", "colors": ["sunset orange", "purple", "pink"], "composition": "establishing", "camera": "locked-off", "description": "A contentment scene: 'Some nights are church without walls'. establishing with sunset orange, purple, pink. Camera: locked-off."}, "artist": "Two Lane Highway", "timestamp": "4:30", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 1, "lyric_line": "Bone handle smooth from his pocket", "scene": {"mood": "sacred", "colors": ["forest green", "brown", "gold"], "composition": "close-up", "camera": "slow pan", "description": "A sacred scene: 'Bone handle smooth from his pocket'. close-up with forest green, brown, gold. Camera: slow pan."}, "artist": "Old Timber", "timestamp": "0:00", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 2, "lyric_line": "Three blades none of them new", "scene": {"mood": "quiet", "colors": ["green", "olive", "tan"], "composition": "wide shot", "camera": "steady", "description": "A quiet scene: 'Three blades none of them new'. wide shot with green, olive, tan. Camera: steady."}, "artist": "Old Timber", "timestamp": "0:30", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 3, "lyric_line": "He carved initials in the oak", "scene": {"mood": "devotion", "colors": ["warm yellow", "barn red", "cream"], "composition": "medium shot", "camera": "handheld", "description": "A devotion scene: 'He carved initials in the oak'. medium shot with warm yellow, barn red, cream. Camera: handheld."}, "artist": "Old Timber", "timestamp": "1:00", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 4, "lyric_line": "The steel remembers every cut", "scene": {"mood": "stillness", "colors": ["warm yellow", "barn red", "cream"], "composition": "low angle", "camera": "slow zoom", "description": "A stillness scene: 'The steel remembers every cut'. low angle with warm yellow, barn red, cream. Camera: slow zoom."}, "artist": "Old Timber", "timestamp": "1:30", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 5, "lyric_line": "I open it the way he taught", "scene": {"mood": "awe", "colors": ["sunset orange", "purple", "pink"], "composition": "high angle", "camera": "tracking", "description": "A awe scene: 'I open it the way he taught'. high angle with sunset orange, purple, pink. Camera: tracking."}, "artist": "Old Timber", "timestamp": "2:00", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 6, "lyric_line": "The smallest blade is for detail", "scene": {"mood": "sacred", "colors": ["green", "olive", "tan"], "composition": "over-the-shoulder", "camera": "static", "description": "A sacred scene: 'The smallest blade is for detail'. over-the-shoulder with green, olive, tan. Camera: static."}, "artist": "Old Timber", "timestamp": "2:30", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 7, "lyric_line": "Whittling shavings catch the light", "scene": {"mood": "quiet", "colors": ["amber", "brown", "cream"], "composition": "profile", "camera": "crane up", "description": "A quiet scene: 'Whittling shavings catch the light'. profile with amber, brown, cream. Camera: crane up."}, "artist": "Old Timber", "timestamp": "3:00", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 8, "lyric_line": "He said a knife is a promise", "scene": {"mood": "devotion", "colors": ["sunset orange", "purple", "pink"], "composition": "bird's eye", "camera": "dolly in", "description": "A devotion scene: 'He said a knife is a promise'. bird's eye with sunset orange, purple, pink. Camera: dolly in."}, "artist": "Old Timber", "timestamp": "3:30", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 9, "lyric_line": "I carry it in my front pocket", "scene": {"mood": "stillness", "colors": ["forest green", "brown", "gold"], "composition": "tracking shot", "camera": "gentle drift", "description": "A stillness scene: 'I carry it in my front pocket'. tracking shot with forest green, brown, gold. Camera: gentle drift."}, "artist": "Old Timber", "timestamp": "4:00", "genre": "country"}
{"song": "Granddad's Pocket Knife", "beat": 10, "lyric_line": "Some tools are heirlooms of skill", "scene": {"mood": "awe", "colors": ["orange", "red", "brown"], "composition": "establishing", "camera": "locked-off", "description": "A awe scene: 'Some tools are heirlooms of skill'. establishing with orange, red, brown. Camera: locked-off."}, "artist": "Old Timber", "timestamp": "4:30", "genre": "country"}
{"song": "County Fair", "beat": 1, "lyric_line": "Ferris wheel lights the harvest sky", "scene": {"mood": "celebration", "colors": ["green", "olive", "tan"], "composition": "close-up", "camera": "slow pan", "description": "A celebration scene: 'Ferris wheel lights the harvest sky'. close-up with green, olive, tan. Camera: slow pan."}, "artist": "Blue Ridge Ramblers", "timestamp": "0:00", "genre": "country"}
{"song": "County Fair", "beat": 2, "lyric_line": "Corn dogs and lemon shake-ups", "scene": {"mood": "community", "colors": ["orange", "red", "brown"], "composition": "wide shot", "camera": "steady", "description": "A community scene: 'Corn dogs and lemon shake-ups'. wide shot with orange, red, brown. Camera: steady."}, "artist": "Blue Ridge Ramblers", "timestamp": "0:30", "genre": "country"}
{"song": "County Fair", "beat": 3, "lyric_line": "The livestock barn smells like home", "scene": {"mood": "youth", "colors": ["sunset orange", "purple", "pink"], "composition": "medium shot", "camera": "handheld", "description": "A youth scene: 'The livestock barn smells like home'. medium shot with sunset orange, purple, pink. Camera: handheld."}, "artist": "Blue Ridge Ramblers", "timestamp": "1:00", "genre": "country"}
{"song": "County Fair", "beat": 4, "lyric_line": "Blue ribbons pinned to quilt squares", "scene": {"mood": "harvest", "colors": ["dusty rose", "gold", "brown"], "composition": "low angle", "camera": "slow zoom", "description": "A harvest scene: 'Blue ribbons pinned to quilt squares'. low angle with dusty rose, gold, brown. Camera: slow zoom."}, "artist": "Blue Ridge Ramblers", "timestamp": "1:30", "genre": "country"}
{"song": "County Fair", "beat": 5, "lyric_line": "Kids scream on the zipper ride", "scene": {"mood": "fun", "colors": ["grey", "mud brown", "green"], "composition": "high angle", "camera": "tracking", "description": "A fun scene: 'Kids scream on the zipper ride'. high angle with grey, mud brown, green. Camera: tracking."}, "artist": "Blue Ridge Ramblers", "timestamp": "2:00", "genre": "country"}
{"song": "County Fair", "beat": 6, "lyric_line": "Band plays covers by the stage", "scene": {"mood": "celebration", "colors": ["navy", "silver", "white"], "composition": "over-the-shoulder", "camera": "static", "description": "A celebration scene: 'Band plays covers by the stage'. over-the-shoulder with navy, silver, white. Camera: static."}, "artist": "Blue Ridge Ramblers", "timestamp": "2:30", "genre": "country"}
{"song": "County Fair", "beat": 7, "lyric_line": "Hay bales line the midway path", "scene": {"mood": "community", "colors": ["sky blue", "wheat", "brown"], "composition": "profile", "camera": "crane up", "description": "A community scene: 'Hay bales line the midway path'. profile with sky blue, wheat, brown. Camera: crane up."}, "artist": "Blue Ridge Ramblers", "timestamp": "3:00", "genre": "country"}
{"song": "County Fair", "beat": 8, "lyric_line": "We split the funnel cake in half", "scene": {"mood": "youth", "colors": ["copper", "dust", "gold"], "composition": "bird's eye", "camera": "dolly in", "description": "A youth scene: 'We split the funnel cake in half'. bird's eye with copper, dust, gold. Camera: dolly in."}, "artist": "Blue Ridge Ramblers", "timestamp": "3:30", "genre": "country"}
{"song": "County Fair", "beat": 9, "lyric_line": "The demolition derby is last", "scene": {"mood": "harvest", "colors": ["sunset orange", "purple", "pink"], "composition": "tracking shot", "camera": "gentle drift", "description": "A harvest scene: 'The demolition derby is last'. tracking shot with sunset orange, purple, pink. Camera: gentle drift."}, "artist": "Blue Ridge Ramblers", "timestamp": "4:00", "genre": "country"}
{"song": "County Fair", "beat": 10, "lyric_line": "Some summers live in a single night", "scene": {"mood": "fun", "colors": ["warm yellow", "barn red", "cream"], "composition": "establishing", "camera": "locked-off", "description": "A fun scene: 'Some summers live in a single night'. establishing with warm yellow, barn red, cream. Camera: locked-off."}, "artist": "Blue Ridge Ramblers", "timestamp": "4:30", "genre": "country"}

View File

@@ -341,6 +341,44 @@ def backfill_provenance(
return stats
class ProvenanceTracker:
"""Track provenance metadata for training pairs."""
def __init__(self):
self.stats = {
"total_pairs": 0,
"pairs_with_provenance": 0,
"pairs_without_provenance": 0,
}
def generate_pair_id(self, pair: dict) -> str:
"""Generate a deterministic ID for a pair."""
content = json.dumps(pair, sort_keys=True)
return hashlib.sha256(content.encode()).hexdigest()[:16]
def process_pair(self, pair: dict) -> dict:
"""Process a pair, adding provenance if missing."""
self.stats["total_pairs"] += 1
if "source_session_id" in pair and pair["source_session_id"]:
self.stats["pairs_with_provenance"] += 1
else:
self.stats["pairs_without_provenance"] += 1
pair = attach_provenance(pair, source="unknown", source_session_id="unknown", model="unknown")
if "pair_id" not in pair:
pair["pair_id"] = self.generate_pair_id(pair)
return pair
def process_file(self, input_path: str, output_path: str = None) -> dict:
"""Process a JSONL file, adding provenance to all pairs."""
pairs = load_jsonl(input_path)
processed = [self.process_pair(p) for p in pairs]
if output_path:
save_jsonl(processed, output_path)
return self.stats
if __name__ == "__main__":
import argparse