Compare commits

..

24 Commits

Author SHA1 Message Date
9a8d620163 feat: quality gate pipeline validation (#623)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 13s
Smoke Test / smoke (pull_request) Failing after 11s
Validate Config / YAML Lint (pull_request) Failing after 14s
Validate Config / JSON Validate (pull_request) Successful in 14s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 44s
Validate Config / Shell Script Lint (pull_request) Failing after 24s
Validate Config / Cron Syntax Check (pull_request) Successful in 5s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 3s
Validate Config / Playbook Schema Validation (pull_request) Successful in 8s
PR Checklist / pr-checklist (pull_request) Failing after 3m54s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
Validates JSONL/JSON pipeline outputs for:
- Schema correctness
- Content quality (non-empty, not duplicated)
- Toxicity detection
- Dedup hash management with auto-cleanup

Usage:
  python3 bin/quality-gate.py validate data.jsonl
  python3 bin/quality-gate.py score data.jsonl
  python3 bin/quality-gate.py stats
  python3 bin/quality-gate.py cleanup

Closes #623
2026-04-17 05:53:33 +00:00
Alexander Whitestone
ce3822bb5f feat: quality gate — validate all pipeline outputs (#623)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 19s
PR Checklist / pr-checklist (pull_request) Failing after 26m43s
Smoke Test / smoke (pull_request) Failing after 59s
Validate Config / YAML Lint (pull_request) Failing after 39s
Validate Config / JSON Validate (pull_request) Successful in 1m32s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 2m0s
Validate Config / Shell Script Lint (pull_request) Failing after 49s
Validate Config / Cron Syntax Check (pull_request) Successful in 9s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 10s
Validate Config / Playbook Schema Validation (pull_request) Successful in 16s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
Validates pipeline outputs before saving. Rejects bad entries,
tracks quality scores per pipeline.

Checks:
- Training pairs: prompt/response non-empty, response != prompt
- Scene descriptions: all required fields, description min length
- Knowledge entries: no placeholders (TODO, FIXME), min length
- Prompt enhancement: rich > terse length, min 20 chars
- Adversary entries: id/family/prompt present, min prompt length
- SOUL.md compliance: no human life valuation, no weapon/child content
- Deduplication: detects duplicate entries by key fields

Features:
- Auto-reject bad outputs with reasons
- Quality score per entry (0.0-1.0)
- Batch mode (--dir) for processing all JSONL at once
- Stats tracking (~/.hermes/pipeline/quality_stats.json)
- --status to view historical quality metrics

Usage:
  python3 pipeline/quality_gate.py --input data.jsonl --type training_pairs
  python3 pipeline/quality_gate.py --dir pipeline/output/
  python3 pipeline/quality_gate.py --status

Closes #623
2026-04-15 08:20:18 -04:00
817785d763 Merge pull request 'feat: training data augmentation — paraphrase and translate pairs (#695)' (#732) from fix/695 into main 2026-04-15 11:56:28 +00:00
Alexander Whitestone
3603030235 feat: training data augmentation — paraphrase and translate pairs (#695)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 22s
Smoke Test / smoke (pull_request) Failing after 18s
Validate Config / YAML Lint (pull_request) Failing after 23s
Validate Config / JSON Validate (pull_request) Successful in 21s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 1m54s
Validate Config / Shell Script Lint (pull_request) Failing after 54s
Validate Config / Cron Syntax Check (pull_request) Successful in 16s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 16s
Validate Config / Playbook Schema Validation (pull_request) Successful in 23s
PR Checklist / pr-checklist (pull_request) Failing after 11m2s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
augment_pairs.py: generates paraphrases and translations for any
JSONL training file.

Features:
- Auto-detects text field (rich, terse, text, content, lyric_line, etc.)
- N paraphrases per entry (template-based, or LLM with --llm-endpoint)
- Translations to ES, FR, DE (template dictionary, or LLM)
- Outputs augmented JSONL alongside originals
- Marks each augmented entry with _augmentation, _original, _language

Usage:
  python3 augment_pairs.py --input data.jsonl
  python3 augment_pairs.py --input data.jsonl --paraphrases 5 --langs es,fr
  python3 augment_pairs.py --input data.jsonl --llm-endpoint http://localhost:11434/v1

Closes #695
2026-04-15 07:51:38 -04:00
35a191f7b1 Merge PR #725: feat: Provider health monitor with auto-switch (#509) 2026-04-15 06:10:45 +00:00
e987e1b870 Merge PR #726: feat: Pre-flight provider check for session launch (#508) 2026-04-15 06:10:41 +00:00
19278513b4 Merge PR #727: feat: Three.js-specific glitch detection patterns (#543) 2026-04-15 06:10:38 +00:00
1088bf8983 test: add Three.js pattern tests and update assertions (#543)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 28s
Smoke Test / smoke (pull_request) Failing after 23s
Validate Config / YAML Lint (pull_request) Failing after 21s
Validate Config / JSON Validate (pull_request) Successful in 21s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 1m23s
Validate Config / Shell Script Lint (pull_request) Failing after 50s
Validate Config / Cron Syntax Check (pull_request) Successful in 11s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 26s
Validate Config / Playbook Schema Validation (pull_request) Successful in 32s
PR Checklist / pr-checklist (pull_request) Failing after 11m13s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
- Added TestThreeJsPatterns class with 14 tests
- Tests cover: pattern existence, severity inference, vision prompt
- Updated pattern count assertion (14+ patterns now)
- Updated demo test (6 glitches: 4 original + 2 Three.js)
2026-04-15 05:37:17 +00:00
94f0a132d4 feat: add get_threejs_patterns() filter function (#543) 2026-04-15 05:34:17 +00:00
279356bed6 feat: add --threejs flag and Three.js-aware severity inference (#543)
- Added --threejs flag for focused Three.js pattern scanning
- Updated _infer_severity with shader_failure, texture_placeholder,
  uv_mapping_error, frustum_culling, shadow_map_artifact categories
- Added Three.js demo detections (shader failure, shadow map)
- Bumped detector version to 0.2.0
2026-04-15 05:34:16 +00:00
511ff863c2 feat: add Three.js-specific glitch detection patterns (#543)
Adds 6 new Three.js-specific glitch categories and patterns:
- SHADER_FAILURE: Solid black materials from shader compilation errors
- TEXTURE_PLACEHOLDER: 1x1 white pixel stretched surfaces
- UV_MAPPING_ERROR: BufferGeometry UV coordinate errors
- FRUSTUM_CULLING: Objects popping at screen edges
- SHADOW_MAP_ARTIFACT: Pixelated/blocky shadow maps
- BLOOM_OVERFLOW: Excessive post-processing bloom bleed

Closes #543
2026-04-15 05:32:25 +00:00
b6e3a647b0 feat: add pre-flight provider check script (#508)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 29s
PR Checklist / pr-checklist (pull_request) Failing after 7m23s
Smoke Test / smoke (pull_request) Failing after 20s
Validate Config / YAML Lint (pull_request) Failing after 14s
Validate Config / JSON Validate (pull_request) Successful in 15s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 1m1s
Validate Config / Shell Script Lint (pull_request) Failing after 46s
Validate Config / Cron Syntax Check (pull_request) Successful in 9s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 10s
Validate Config / Playbook Schema Validation (pull_request) Successful in 28s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
- Checks OpenRouter balance via /api/v1/auth/key
- Tests Nous and Anthropic API keys
- Verifies Ollama is running
- Pre-flight check before session launch
- Returns exit code for automation

Closes #508
2026-04-15 03:55:04 +00:00
e14158676d feat: add provider health monitor script (#509)
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 44s
Smoke Test / smoke (pull_request) Failing after 36s
Validate Config / YAML Lint (pull_request) Failing after 21s
Validate Config / JSON Validate (pull_request) Successful in 28s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 2m36s
Validate Config / Shell Script Lint (pull_request) Failing after 1m3s
Validate Config / Cron Syntax Check (pull_request) Successful in 13s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 12s
PR Checklist / pr-checklist (pull_request) Failing after 6m15s
Validate Config / Playbook Schema Validation (pull_request) Successful in 28s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
- Tests all configured providers
- Maintains health map in tmux-state.json
- Auto-switches profiles to working providers
- Supports --daemon and --status modes

Closes #509
2026-04-15 03:48:37 +00:00
26e39d8949 feat: add autonomous cron supervisor job (#513)
- Runs every 7 minutes
- Checks dev and timmy sessions
- Loads tmux-supervisor skill
- Telegram only on actionable events
- Silent when all agents busy
2026-04-15 03:33:43 +00:00
d120526244 fix: add python3 shebang to scripts/visual_pr_reviewer.py (#681) 2026-04-15 02:57:53 +00:00
8596ff761b fix: add python3 shebang to scripts/diagram_meaning_extractor.py (#681) 2026-04-15 02:57:40 +00:00
7553fd4f3e fix: add python3 shebang to scripts/captcha_bypass_handler.py (#681) 2026-04-15 02:57:25 +00:00
71082fe06f fix: add python3 shebang to bin/soul_eval_gate.py (#681) 2026-04-15 02:57:14 +00:00
6d678e938e fix: add python3 shebang to bin/nostr-agent-demo.py (#681) 2026-04-15 02:57:00 +00:00
ad751a6de6 docs: add pipeline scheduler README 2026-04-14 22:47:12 +00:00
130fa40f0c feat: add pipeline-scheduler cron job 2026-04-14 22:46:51 +00:00
82f9810081 feat: add nightly-pipeline-scheduler.sh 2026-04-14 22:46:38 +00:00
2548277137 cleanup test
Some checks failed
Architecture Lint / Linter Tests (push) Successful in 22s
Smoke Test / smoke (push) Failing after 21s
Validate Config / YAML Lint (push) Failing after 13s
Validate Config / JSON Validate (push) Successful in 14s
Validate Config / Python Syntax & Import Check (push) Failing after 1m9s
Validate Config / Shell Script Lint (push) Failing after 31s
Validate Config / Cron Syntax Check (push) Successful in 5s
Validate Config / Deploy Script Dry Run (push) Successful in 7s
Validate Config / Playbook Schema Validation (push) Successful in 16s
Architecture Lint / Linter Tests (pull_request) Successful in 14s
Smoke Test / smoke (pull_request) Failing after 13s
Validate Config / YAML Lint (pull_request) Failing after 12s
Validate Config / JSON Validate (pull_request) Successful in 13s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 54s
Validate Config / Shell Script Lint (pull_request) Failing after 21s
Validate Config / Cron Syntax Check (pull_request) Successful in 5s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 7s
Validate Config / Playbook Schema Validation (pull_request) Successful in 18s
PR Checklist / pr-checklist (pull_request) Failing after 3m27s
Architecture Lint / Lint Repository (push) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (push) Has been cancelled
2026-04-14 22:39:03 +00:00
2b234fde79 test: verify API works
Some checks failed
Architecture Lint / Linter Tests (push) Has been cancelled
Architecture Lint / Lint Repository (push) Has been cancelled
Smoke Test / smoke (push) Failing after 12s
Validate Config / YAML Lint (push) Failing after 11s
Validate Config / JSON Validate (push) Successful in 11s
Validate Config / Python Syntax & Import Check (push) Failing after 47s
Validate Config / Shell Script Lint (push) Failing after 33s
Validate Config / Cron Syntax Check (push) Successful in 10s
Validate Config / Deploy Script Dry Run (push) Successful in 10s
Validate Config / Playbook Schema Validation (push) Successful in 14s
Validate Config / Python Test Suite (push) Has been cancelled
2026-04-14 22:39:02 +00:00
18 changed files with 2286 additions and 509 deletions

View File

@@ -31,6 +31,14 @@ class GlitchCategory(Enum):
WATER_REFLECTION = "water_reflection"
SKYBOX_SEAM = "skybox_seam"
# Three.js-specific categories (ref: timmy-config#543)
SHADER_FAILURE = "shader_failure"
TEXTURE_PLACEHOLDER = "texture_placeholder"
UV_MAPPING_ERROR = "uv_mapping_error"
FRUSTUM_CULLING = "frustum_culling"
SHADOW_MAP_ARTIFACT = "shadow_map_artifact"
BLOOM_OVERFLOW = "bloom_overflow"
@dataclass
class GlitchPattern:
@@ -241,6 +249,123 @@ MATRIX_GLITCH_PATTERNS: list[GlitchPattern] = [
],
confidence_threshold=0.45,
),
# --- Three.js-Specific Glitch Patterns (ref: timmy-config#543) ---
GlitchPattern(
category=GlitchCategory.SHADER_FAILURE,
name="Shader Compilation Failure",
description="Three.js shader failed to compile, rendering the material as solid black. "
"Common when custom ShaderMaterial has syntax errors or missing uniforms.",
severity=GlitchSeverity.CRITICAL,
detection_prompts=[
"Look for objects or surfaces rendered as pure black (#000000) that should have visible textures or materials.",
"Identify geometry that appears completely dark while surrounding objects are normally lit.",
"Check for objects where the material seems to 'absorb all light' — flat black with no shading gradient.",
],
visual_indicators=[
"solid black object with no shading",
"geometry rendered as silhouette",
"material appears to absorb light entirely",
"black patch inconsistent with scene lighting",
],
confidence_threshold=0.7,
),
GlitchPattern(
category=GlitchCategory.TEXTURE_PLACEHOLDER,
name="Three.js Texture Not Loaded",
description="Three.js failed to load the texture asset, rendering a 1x1 white pixel "
"stretched across the entire surface. Distinguished from missing-texture by "
"the uniform white/grey appearance rather than magenta.",
severity=GlitchSeverity.CRITICAL,
detection_prompts=[
"Look for surfaces that are uniformly white or light grey with no texture detail, even on large geometry.",
"Identify objects where the texture appears as a single solid color stretched across complex UVs.",
"Check for surfaces that look 'blank' or 'unloaded' — flat white/grey where detail should exist.",
],
visual_indicators=[
"uniform white or light grey surface",
"no texture detail on large geometry",
"stretched single-color appearance",
"1x1 pixel placeholder stretched to fill UV space",
],
confidence_threshold=0.65,
),
GlitchPattern(
category=GlitchCategory.UV_MAPPING_ERROR,
name="BufferGeometry UV Mapping Error",
description="Three.js BufferGeometry has incorrect UV coordinates, causing textures to "
"appear stretched, compressed, or mapped to the wrong faces.",
severity=GlitchSeverity.HIGH,
detection_prompts=[
"Look for textures that appear dramatically stretched in one direction on specific faces.",
"Identify surfaces where the texture pattern is distorted but other nearby surfaces look correct.",
"Check for faces where the texture seems 'smeared' or mapped with incorrect aspect ratio.",
],
visual_indicators=[
"texture stretching on specific faces",
"distorted pattern on geometry",
"smeared texture appearance",
"aspect ratio mismatch between texture and surface",
],
confidence_threshold=0.6,
),
GlitchPattern(
category=GlitchCategory.FRUSTUM_CULLING,
name="Frustum Culling Artifact",
description="Three.js frustum culling incorrectly marks objects as outside the camera "
"frustum, causing them to pop in/out of existence at screen edges.",
severity=GlitchSeverity.MEDIUM,
detection_prompts=[
"Look for objects that are partially visible at the edge of the frame — half-rendered or cut off unnaturally.",
"Identify geometry that seems to 'pop' into existence as the view angle changes.",
"Check screen edges for objects that appear suddenly rather than smoothly entering the viewport.",
],
visual_indicators=[
"half-visible object at screen edge",
"object popping into frame",
"abrupt appearance of geometry",
"bounding box visible but mesh missing",
],
confidence_threshold=0.55,
),
GlitchPattern(
category=GlitchCategory.SHADOW_MAP_ARTIFACT,
name="Shadow Map Resolution Artifact",
description="Three.js shadow map has insufficient resolution, causing pixelated, "
"blocky shadows with visible texel edges instead of smooth shadow gradients.",
severity=GlitchSeverity.MEDIUM,
detection_prompts=[
"Look for shadows with visible blocky or pixelated edges instead of smooth gradients.",
"Identify shadow maps where individual texels (texture pixels) are clearly visible.",
"Check for shadows that appear as jagged stair-stepped patterns rather than soft edges.",
],
visual_indicators=[
"blocky shadow edges",
"visible texel grid in shadows",
"stair-stepped shadow boundary",
"pixelated shadow gradient",
],
confidence_threshold=0.55,
),
GlitchPattern(
category=GlitchCategory.BLOOM_OVERFLOW,
name="Post-Processing Bloom Overflow",
description="Three.js UnrealBloomPass or similar post-processing bloom effect is too "
"intense, causing bright areas to bleed glow into surrounding geometry.",
severity=GlitchSeverity.LOW,
detection_prompts=[
"Look for bright areas that have an unusually large, soft glow bleeding into adjacent surfaces.",
"Identify scenes where light sources appear to have a 'halo' that extends beyond physical plausibility.",
"Check for bright objects whose glow color bleeds onto nearby unrelated geometry.",
],
visual_indicators=[
"excessive glow bleeding from bright surfaces",
"halo around light sources",
"bloom color tinting adjacent geometry",
"glow bleeding beyond object boundaries",
],
confidence_threshold=0.5,
),
]
@@ -289,6 +414,23 @@ def build_vision_prompt(patterns: list[GlitchPattern] | None = None) -> str:
)
# Three.js-specific category set for filtering (ref: timmy-config#543)
THREEJS_CATEGORIES = {
GlitchCategory.SHADER_FAILURE,
GlitchCategory.TEXTURE_PLACEHOLDER,
GlitchCategory.UV_MAPPING_ERROR,
GlitchCategory.FRUSTUM_CULLING,
GlitchCategory.SHADOW_MAP_ARTIFACT,
GlitchCategory.BLOOM_OVERFLOW,
}
def get_threejs_patterns() -> list[GlitchPattern]:
"""Return only Three.js-specific glitch patterns."""
return [p for p in MATRIX_GLITCH_PATTERNS if p.category in THREEJS_CATEGORIES]
if __name__ == "__main__":
import json
print(f"Loaded {len(MATRIX_GLITCH_PATTERNS)} glitch patterns:\n")

View File

@@ -9,7 +9,7 @@ Usage:
python matrix_glitch_detector.py <url> [--angles 4] [--output report.json]
python matrix_glitch_detector.py --demo # Run with synthetic test data
Ref: timmy-config#491
Ref: timmy-config#491, timmy-config#543
"""
import argparse
@@ -33,6 +33,7 @@ from glitch_patterns import (
MATRIX_GLITCH_PATTERNS,
build_vision_prompt,
get_patterns_by_severity,
get_threejs_patterns,
)
@@ -345,14 +346,17 @@ def _parse_vision_response(
def _infer_severity(category: str, confidence: float) -> str:
"""Infer severity from category and confidence when not provided."""
critical_cats = {"missing_textures", "clipping"}
high_cats = {"floating_assets", "broken_normals"}
critical_cats = {"missing_textures", "clipping", "shader_failure", "texture_placeholder"}
high_cats = {"floating_assets", "broken_normals", "uv_mapping_error"}
medium_cats = {"frustum_culling", "shadow_map_artifact"}
cat_lower = category.lower()
if any(c in cat_lower for c in critical_cats):
return "critical" if confidence > 0.7 else "high"
if any(c in cat_lower for c in high_cats):
return "high" if confidence > 0.7 else "medium"
if any(c in cat_lower for c in medium_cats):
return "medium" if confidence > 0.6 else "low"
return "medium" if confidence > 0.6 else "low"
@@ -389,9 +393,9 @@ def build_report(
),
},
metadata={
"detector_version": "0.1.0",
"detector_version": "0.2.0",
"pattern_count": len(MATRIX_GLITCH_PATTERNS),
"reference": "timmy-config#491",
"reference": "timmy-config#491, timmy-config#543",
},
)
@@ -460,6 +464,30 @@ def run_demo(output_path: Optional[Path] = None) -> ScanResult:
screenshot_index=3,
screenshot_angle="left",
),
DetectedGlitch(
id=str(uuid.uuid4())[:8],
category="shader_failure",
name="Black Material on Portal Frame",
description="Portal frame rendered as solid black — shader compilation failed (missing uniform u_time)",
severity="critical",
confidence=0.91,
location_x=45.0,
location_y=30.0,
screenshot_index=0,
screenshot_angle="front",
),
DetectedGlitch(
id=str(uuid.uuid4())[:8],
category="shadow_map_artifact",
name="Pixelated Character Shadow",
description="Character shadow shows visible texel grid — shadow map resolution too low (512x512)",
severity="medium",
confidence=0.78,
location_x=52.0,
location_y=75.0,
screenshot_index=1,
screenshot_angle="right",
),
]
print(f"[*] Detected {len(demo_glitches)} glitches")
@@ -496,6 +524,11 @@ Examples:
help="Minimum severity to include in report",
)
parser.add_argument("--verbose", "-v", action="store_true", help="Verbose output")
parser.add_argument(
"--threejs",
action="store_true",
help="Focus on Three.js-specific glitch patterns only (shader, texture, UV, culling, shadow, bloom)",
)
args = parser.parse_args()
@@ -525,9 +558,13 @@ Examples:
screenshots = capture_screenshots(args.url, angles, screenshots_dir)
print(f"[*] Captured {len(screenshots)} screenshots")
# Filter patterns by severity
# Filter patterns by severity and type
min_sev = GlitchSeverity(args.min_severity)
patterns = get_patterns_by_severity(min_sev)
if args.threejs:
threejs_patterns = get_threejs_patterns()
patterns = [p for p in patterns if p in threejs_patterns]
print(f"[*] Three.js-focused mode: {len(patterns)} patterns")
# Analyze with vision AI
print(f"[*] Analyzing with vision AI ({len(patterns)} patterns)...")

View File

@@ -1,3 +1,4 @@
#!/usr/bin/env python3
"""
Full Nostr agent-to-agent communication demo - FINAL WORKING
"""

View File

@@ -0,0 +1,271 @@
#!/usr/bin/env python3
"""
Pre-Flight Provider Check Script
Issue #508: [Robustness] Credential drain detection — provider health checks
Pre-flight check before session launch: verifies provider credentials and balance.
Usage:
python3 preflight-provider-check.py # Check all providers
python3 preflight-provider-check.py --launch # Check and return exit code
python3 preflight-provider-check.py --balance # Check OpenRouter balance
"""
import os, sys, json, yaml, urllib.request
from datetime import datetime, timezone
from pathlib import Path
# Configuration
HERMES_HOME = Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes"))
LOG_DIR = Path.home() / ".local" / "timmy" / "fleet-health"
LOG_FILE = LOG_DIR / "preflight-check.log"
def log(msg):
"""Log message to file and optionally console."""
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
log_entry = "[" + timestamp + "] " + msg
LOG_DIR.mkdir(parents=True, exist_ok=True)
with open(LOG_FILE, "a") as f:
f.write(log_entry + "\n")
if "--quiet" not in sys.argv:
print(log_entry)
def get_provider_api_key(provider):
"""Get API key for a provider from .env or environment."""
env_file = HERMES_HOME / ".env"
if env_file.exists():
with open(env_file) as f:
for line in f:
line = line.strip()
if line.startswith(provider.upper() + "_API_KEY="):
return line.split("=", 1)[1].strip().strip("'\"")
return os.environ.get(provider.upper() + "_API_KEY")
def check_openrouter_balance(api_key):
"""Check OpenRouter balance via /api/v1/auth/key."""
if not api_key:
return False, "No API key", 0
try:
req = urllib.request.Request(
"https://openrouter.ai/api/v1/auth/key",
headers={"Authorization": "Bearer " + api_key}
)
resp = urllib.request.urlopen(req, timeout=10)
data = json.loads(resp.read())
# Check for credits
credits = data.get("data", {}).get("limit", 0)
usage = data.get("data", {}).get("usage", 0)
remaining = credits - usage if credits else None
if remaining is not None and remaining <= 0:
return False, "No credits remaining", 0
elif remaining is not None:
return True, "Credits available", remaining
else:
return True, "Unlimited or unknown balance", None
except urllib.error.HTTPError as e:
if e.code == 401:
return False, "Invalid API key", 0
else:
return False, "HTTP " + str(e.code), 0
except Exception as e:
return False, str(e)[:100], 0
def check_nous_key(api_key):
"""Check Nous API key with minimal test call."""
if not api_key:
return False, "No API key"
try:
req = urllib.request.Request(
"https://inference.nousresearch.com/v1/models",
headers={"Authorization": "Bearer " + api_key}
)
resp = urllib.request.urlopen(req, timeout=10)
if resp.status == 200:
return True, "Valid key"
else:
return False, "HTTP " + str(resp.status)
except urllib.error.HTTPError as e:
if e.code == 401:
return False, "Invalid API key"
elif e.code == 403:
return False, "Forbidden"
else:
return False, "HTTP " + str(e.code)
except Exception as e:
return False, str(e)[:100]
def check_anthropic_key(api_key):
"""Check Anthropic API key with minimal test call."""
if not api_key:
return False, "No API key"
try:
req = urllib.request.Request(
"https://api.anthropic.com/v1/models",
headers={
"x-api-key": api_key,
"anthropic-version": "2023-06-01"
}
)
resp = urllib.request.urlopen(req, timeout=10)
if resp.status == 200:
return True, "Valid key"
else:
return False, "HTTP " + str(resp.status)
except urllib.error.HTTPError as e:
if e.code == 401:
return False, "Invalid API key"
elif e.code == 403:
return False, "Forbidden"
else:
return False, "HTTP " + str(e.code)
except Exception as e:
return False, str(e)[:100]
def check_ollama():
"""Check if Ollama is running."""
try:
req = urllib.request.Request("http://localhost:11434/api/tags")
resp = urllib.request.urlopen(req, timeout=5)
if resp.status == 200:
data = json.loads(resp.read())
models = data.get("models", [])
return True, str(len(models)) + " models loaded"
else:
return False, "HTTP " + str(resp.status)
except Exception as e:
return False, str(e)[:100]
def get_configured_provider():
"""Get the configured provider from global config."""
config_file = HERMES_HOME / "config.yaml"
if not config_file.exists():
return None
try:
with open(config_file) as f:
config = yaml.safe_load(f)
model_config = config.get("model", {})
if isinstance(model_config, dict):
return model_config.get("provider")
except:
pass
return None
def run_preflight_check():
"""Run pre-flight check on all providers."""
log("=== Pre-Flight Provider Check ===")
results = {}
# Check OpenRouter
or_key = get_provider_api_key("openrouter")
or_ok, or_msg, or_balance = check_openrouter_balance(or_key)
results["openrouter"] = {"healthy": or_ok, "message": or_msg, "balance": or_balance}
# Check Nous
nous_key = get_provider_api_key("nous")
nous_ok, nous_msg = check_nous_key(nous_key)
results["nous"] = {"healthy": nous_ok, "message": nous_msg}
# Check Anthropic
anthropic_key = get_provider_api_key("anthropic")
anthropic_ok, anthropic_msg = check_anthropic_key(anthropic_key)
results["anthropic"] = {"healthy": anthropic_ok, "message": anthropic_msg}
# Check Ollama
ollama_ok, ollama_msg = check_ollama()
results["ollama"] = {"healthy": ollama_ok, "message": ollama_msg}
# Get configured provider
configured = get_configured_provider()
# Summary
healthy_count = sum(1 for r in results.values() if r["healthy"])
total_count = len(results)
log("Results: " + str(healthy_count) + "/" + str(total_count) + " providers healthy")
for provider, result in results.items():
status = "HEALTHY" if result["healthy"] else "UNHEALTHY"
extra = ""
if provider == "openrouter" and result.get("balance") is not None:
extra = " (balance: " + str(result["balance"]) + ")"
log(" " + provider + ": " + status + " - " + result["message"] + extra)
if configured:
log("Configured provider: " + configured)
if configured in results and not results[configured]["healthy"]:
log("WARNING: Configured provider " + configured + " is UNHEALTHY!")
return results, configured
def check_launch_readiness():
"""Check if we're ready to launch sessions."""
results, configured = run_preflight_check()
# Check if configured provider is healthy
if configured and configured in results:
if not results[configured]["healthy"]:
log("LAUNCH BLOCKED: Configured provider " + configured + " is unhealthy")
return False, configured + " is unhealthy"
# Check if at least one provider is healthy
healthy_providers = [p for p, r in results.items() if r["healthy"]]
if not healthy_providers:
log("LAUNCH BLOCKED: No healthy providers available")
return False, "No healthy providers"
log("LAUNCH READY: " + str(len(healthy_providers)) + " healthy providers available")
return True, "Ready"
def show_balance():
"""Show OpenRouter balance."""
api_key = get_provider_api_key("openrouter")
if not api_key:
print("No OpenRouter API key found")
return
ok, msg, balance = check_openrouter_balance(api_key)
if ok:
if balance is not None:
print("OpenRouter balance: " + str(balance) + " credits")
else:
print("OpenRouter: " + msg)
else:
print("OpenRouter: " + msg)
def main():
if "--balance" in sys.argv:
show_balance()
elif "--launch" in sys.argv:
ready, message = check_launch_readiness()
if ready:
print("READY")
sys.exit(0)
else:
print("BLOCKED: " + message)
sys.exit(1)
else:
run_preflight_check()
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,411 @@
#!/usr/bin/env python3
"""
Provider Health Monitor Script
Issue #509: [Robustness] Provider-aware profile config — auto-switch on failure
Monitors provider health and automatically switches profiles to working providers.
Usage:
python3 provider-health-monitor.py # Run once
python3 provider-health-monitor.py --daemon # Run continuously
python3 provider-health-monitor.py --status # Show provider health
"""
import os, sys, json, yaml, urllib.request, time
from datetime import datetime, timezone
from pathlib import Path
# Configuration
HERMES_HOME = Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes"))
PROFILES_DIR = HERMES_HOME / "profiles"
LOG_DIR = Path.home() / ".local" / "timmy" / "fleet-health"
STATE_FILE = LOG_DIR / "tmux-state.json"
LOG_FILE = LOG_DIR / "provider-health.log"
# Provider test endpoints
PROVIDER_TESTS = {
"openrouter": {
"url": "https://openrouter.ai/api/v1/models",
"method": "GET",
"headers": lambda api_key: {"Authorization": "Bearer " + api_key},
"timeout": 10
},
"anthropic": {
"url": "https://api.anthropic.com/v1/models",
"method": "GET",
"headers": lambda api_key: {"x-api-key": api_key, "anthropic-version": "2023-06-01"},
"timeout": 10
},
"nous": {
"url": "https://inference.nousresearch.com/v1/models",
"method": "GET",
"headers": lambda api_key: {"Authorization": "Bearer " + api_key},
"timeout": 10
},
"kimi-coding": {
"url": "https://api.kimi.com/coding/v1/models",
"method": "GET",
"headers": lambda api_key: {"x-api-key": api_key, "x-api-provider": "kimi-coding"},
"timeout": 10
},
"ollama": {
"url": "http://localhost:11434/api/tags",
"method": "GET",
"headers": lambda api_key: {},
"timeout": 5
}
}
def log(msg):
"""Log message to file and optionally console."""
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
log_entry = "[" + timestamp + "] " + msg
LOG_DIR.mkdir(parents=True, exist_ok=True)
with open(LOG_FILE, "a") as f:
f.write(log_entry + "\n")
if "--quiet" not in sys.argv:
print(log_entry)
def get_provider_api_key(provider):
"""Get API key for a provider from .env or environment."""
env_file = HERMES_HOME / ".env"
if env_file.exists():
with open(env_file) as f:
for line in f:
line = line.strip()
if line.startswith(provider.upper() + "_API_KEY="):
return line.split("=", 1)[1].strip().strip("'\"")
return os.environ.get(provider.upper() + "_API_KEY")
def test_provider(provider, api_key=None):
"""Test if a provider is healthy."""
config = PROVIDER_TESTS.get(provider)
if not config:
return False, "Unknown provider: " + provider
headers = config["headers"](api_key or "")
try:
req = urllib.request.Request(
config["url"],
headers=headers,
method=config["method"]
)
resp = urllib.request.urlopen(req, timeout=config["timeout"])
if resp.status == 200:
return True, "Healthy"
else:
return False, "HTTP " + str(resp.status)
except urllib.error.HTTPError as e:
if e.code == 401:
return False, "Unauthorized (401)"
elif e.code == 403:
return False, "Forbidden (403)"
elif e.code == 429:
return True, "Rate limited but accessible"
else:
return False, "HTTP " + str(e.code)
except Exception as e:
return False, str(e)[:100]
def get_all_providers():
"""Get all providers from profiles and global config."""
providers = set()
# Global config
global_config = HERMES_HOME / "config.yaml"
if global_config.exists():
try:
with open(global_config) as f:
config = yaml.safe_load(f)
# Primary model provider
model_config = config.get("model", {})
if isinstance(model_config, dict):
provider = model_config.get("provider", "")
if provider:
providers.add(provider)
# Auxiliary providers
auxiliary = config.get("auxiliary", {})
for aux_config in auxiliary.values():
if isinstance(aux_config, dict):
provider = aux_config.get("provider", "")
if provider and provider != "auto":
providers.add(provider)
except:
pass
# Profile configs
if PROFILES_DIR.exists():
for profile_dir in PROFILES_DIR.iterdir():
if profile_dir.is_dir():
config_file = profile_dir / "config.yaml"
if config_file.exists():
try:
with open(config_file) as f:
config = yaml.safe_load(f)
model_config = config.get("model", {})
if isinstance(model_config, dict):
provider = model_config.get("provider", "")
if provider:
providers.add(provider)
auxiliary = config.get("auxiliary", {})
for aux_config in auxiliary.values():
if isinstance(aux_config, dict):
provider = aux_config.get("provider", "")
if provider and provider != "auto":
providers.add(provider)
except:
pass
# Add common providers even if not configured
providers.update(["openrouter", "nous", "ollama"])
return list(providers)
def build_health_map():
"""Build a health map of all providers."""
providers = get_all_providers()
health_map = {}
log("Testing " + str(len(providers)) + " providers...")
for provider in providers:
api_key = get_provider_api_key(provider)
healthy, message = test_provider(provider, api_key)
health_map[provider] = {
"healthy": healthy,
"message": message,
"last_test": datetime.now(timezone.utc).isoformat(),
"api_key_present": bool(api_key)
}
status = "HEALTHY" if healthy else "UNHEALTHY"
log(" " + provider + ": " + status + " - " + message)
return health_map
def get_fallback_providers(health_map):
"""Get list of healthy providers in priority order."""
# Priority order: nous, openrouter, ollama, others
priority_order = ["nous", "openrouter", "ollama", "anthropic", "kimi-coding"]
healthy = []
for provider in priority_order:
if provider in health_map and health_map[provider]["healthy"]:
healthy.append(provider)
# Add any other healthy providers not in priority list
for provider, info in health_map.items():
if info["healthy"] and provider not in healthy:
healthy.append(provider)
return healthy
def update_profile_config(profile_name, new_provider):
"""Update a profile's config to use a new provider."""
config_file = PROFILES_DIR / profile_name / "config.yaml"
if not config_file.exists():
return False, "Config file not found"
try:
with open(config_file) as f:
config = yaml.safe_load(f)
# Update model provider
if "model" not in config:
config["model"] = {}
old_provider = config["model"].get("provider", "unknown")
config["model"]["provider"] = new_provider
# Update auxiliary providers if they were using the old provider
auxiliary = config.get("auxiliary", {})
for aux_name, aux_config in auxiliary.items():
if isinstance(aux_config, dict) and aux_config.get("provider") == old_provider:
aux_config["provider"] = new_provider
# Write back
with open(config_file, "w") as f:
yaml.dump(config, f, default_flow_style=False)
log("Updated " + profile_name + ": " + old_provider + " -> " + new_provider)
return True, "Updated"
except Exception as e:
return False, str(e)
def check_profiles(health_map):
"""Check all profiles and update unhealthy providers."""
if not PROFILES_DIR.exists():
return
fallback_providers = get_fallback_providers(health_map)
if not fallback_providers:
log("CRITICAL: No healthy providers available!")
return
updated_profiles = []
for profile_dir in PROFILES_DIR.iterdir():
if not profile_dir.is_dir():
continue
profile_name = profile_dir.name
config_file = profile_dir / "config.yaml"
if not config_file.exists():
continue
try:
with open(config_file) as f:
config = yaml.safe_load(f)
model_config = config.get("model", {})
if not isinstance(model_config, dict):
continue
current_provider = model_config.get("provider", "")
if not current_provider:
continue
# Check if current provider is healthy
if current_provider in health_map and health_map[current_provider]["healthy"]:
continue # Provider is healthy, no action needed
# Find best fallback
best_fallback = None
for provider in fallback_providers:
if provider != current_provider:
best_fallback = provider
break
if not best_fallback:
log("No fallback for " + profile_name + " (current: " + current_provider + ")")
continue
# Update profile
success, message = update_profile_config(profile_name, best_fallback)
if success:
updated_profiles.append({
"profile": profile_name,
"old_provider": current_provider,
"new_provider": best_fallback
})
except Exception as e:
log("Error processing " + profile_name + ": " + str(e))
return updated_profiles
def load_state():
"""Load state from tmux-state.json."""
if STATE_FILE.exists():
try:
with open(STATE_FILE) as f:
return json.load(f)
except:
pass
return {}
def save_state(state):
"""Save state to tmux-state.json."""
LOG_DIR.mkdir(parents=True, exist_ok=True)
with open(STATE_FILE, "w") as f:
json.dump(state, f, indent=2)
def run_once():
"""Run provider health check once."""
log("=== Provider Health Check ===")
state = load_state()
# Build health map
health_map = build_health_map()
# Check profiles and update if needed
updated_profiles = check_profiles(health_map)
# Update state
state["provider_health"] = health_map
state["last_provider_check"] = datetime.now(timezone.utc).isoformat()
if updated_profiles:
state["last_profile_updates"] = updated_profiles
save_state(state)
# Summary
healthy_count = sum(1 for p in health_map.values() if p["healthy"])
total_count = len(health_map)
log("Health: " + str(healthy_count) + "/" + str(total_count) + " providers healthy")
if updated_profiles:
log("Updated " + str(len(updated_profiles)) + " profiles:")
for update in updated_profiles:
log(" " + update["profile"] + ": " + update["old_provider"] + " -> " + update["new_provider"])
def show_status():
"""Show provider health status."""
state = load_state()
health_map = state.get("provider_health", {})
if not health_map:
print("No provider health data available. Run without --status first.")
return
print("Provider Health (last updated: " + str(state.get("last_provider_check", "unknown")) + ")")
print("=" * 80)
for provider, info in sorted(health_map.items()):
status = "HEALTHY" if info["healthy"] else "UNHEALTHY"
message = info.get("message", "")
api_key = "yes" if info.get("api_key_present") else "no"
print(provider.ljust(20) + " " + status.ljust(10) + " API key: " + api_key + " - " + message)
# Show recent updates
updates = state.get("last_profile_updates", [])
if updates:
print()
print("Recent Profile Updates:")
for update in updates:
print(" " + update["profile"] + ": " + update["old_provider"] + " -> " + update["new_provider"])
def daemon_mode():
"""Run continuously."""
log("Starting provider health daemon (check every 300s)")
while True:
try:
run_once()
time.sleep(300) # Check every 5 minutes
except KeyboardInterrupt:
log("Daemon stopped by user")
break
except Exception as e:
log("Error: " + str(e))
time.sleep(60)
def main():
if "--status" in sys.argv:
show_status()
elif "--daemon" in sys.argv:
daemon_mode()
else:
run_once()
if __name__ == "__main__":
main()

292
bin/quality-gate.py Normal file
View File

@@ -0,0 +1,292 @@
#!/usr/bin/env python3
"""
Quality Gate — Validate pipeline outputs before saving.
Checks:
- JSON schema validation for all output formats
- Content quality (not empty, not duplicated, not toxic)
- SOUL.md compliance for agent-facing content
- Auto-reject bad outputs, re-queue for regeneration
- Quality score tracking per pipeline
Usage:
python3 quality-gate.py validate training-data/pairs.jsonl
python3 quality-gate.py validate --format training-pairs data.jsonl
python3 quality-gate.py score training-data/pairs.jsonl
python3 quality-gate.py stats
"""
import hashlib
import json
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
HERMES_HOME = Path(os.environ.get("HERMES_HOME", os.path.expanduser("~/.hermes")))
SCORE_FILE = HERMES_HOME / "quality-scores.jsonl"
HASH_FILE = HERMES_HOME / "quality-hashes.json"
MAX_HASH_AGE_DAYS = 7
# ── Validators ─────────────────────────────────────────
TOXIC_PATTERNS = [
"kill yourself", "kys", "you should die", "end it all",
"nobody loves you", "waste of life",
]
def validate_training_pair(entry):
"""Validate a training pair (prompt + response)."""
errors = []
if not isinstance(entry, dict):
return ["Entry is not a dict"]
prompt = entry.get("prompt", "") or entry.get("instruction", "") or ""
response = entry.get("response", "") or entry.get("output", "") or entry.get("completion", "") or ""
if not prompt.strip():
errors.append("Empty prompt")
if not response.strip():
errors.append("Empty response")
if len(response) < 10:
errors.append(f"Response too short ({len(response)} chars)")
if len(prompt) > 10000:
errors.append(f"Prompt too long ({len(prompt)} chars)")
# Toxicity check
combined = (prompt + " " + response).lower()
for pattern in TOXIC_PATTERNS:
if pattern in combined:
errors.append(f"Toxic content detected: '{pattern}'")
return errors
def validate_jsonl(filepath):
"""Validate a JSONL file — each line must be valid JSON."""
errors = []
seen_hashes = set()
line_count = 0
try:
with open(filepath) as f:
for i, line in enumerate(f, 1):
line = line.strip()
if not line:
continue
line_count += 1
try:
entry = json.loads(line)
except json.JSONDecodeError as e:
errors.append(f"Line {i}: invalid JSON: {e}")
continue
# Duplicate detection
h = hashlib.sha256(line.encode()).hexdigest()[:16]
if h in seen_hashes:
errors.append(f"Line {i}: duplicate content (hash {h})")
seen_hashes.add(h)
# Content validation
if isinstance(entry, dict):
pair_errors = validate_training_pair(entry)
for pe in pair_errors:
errors.append(f"Line {i}: {pe}")
except Exception as e:
errors.append(f"File error: {e}")
return errors, line_count, seen_hashes
def validate_json(filepath):
"""Validate a single JSON file."""
errors = []
try:
with open(filepath) as f:
data = json.load(f)
except json.JSONDecodeError as e:
return [f"Invalid JSON: {e}"], 0
if isinstance(data, list):
seen = set()
for i, entry in enumerate(data):
if isinstance(entry, dict):
h = hashlib.sha256(json.dumps(entry, sort_keys=True).encode()).hexdigest()[:16]
if h in seen:
errors.append(f"Index {i}: duplicate entry")
seen.add(h)
return errors, len(data) if isinstance(data, list) else 1
# ── Quality Scoring ────────────────────────────────────
def score_file(filepath):
"""Score a pipeline output file. Returns 0-100."""
path = Path(filepath)
if not path.exists():
return 0
suffix = path.suffix.lower()
if suffix == ".jsonl":
errors, count, _ = validate_jsonl(filepath)
elif suffix == ".json":
errors, count = validate_json(filepath)
else:
return 50 # unknown format
if count == 0:
return 0
error_rate = len(errors) / count
score = max(0, int(100 * (1 - error_rate)))
# Bonus for having content
if count >= 100:
score = min(100, score + 5)
return score
def record_score(filepath, score):
"""Record quality score for tracking."""
HERMES_HOME.mkdir(parents=True, exist_ok=True)
entry = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"file": str(filepath),
"score": score,
}
with open(SCORE_FILE, "a") as f:
f.write(json.dumps(entry) + "
")
# ── Dedup Hash Management ─────────────────────────────
def load_hashes():
try:
return json.loads(HASH_FILE.read_text())
except Exception:
return {"entries": {}, "last_cleanup": None}
def save_hashes(data):
HASH_FILE.parent.mkdir(parents=True, exist_ok=True)
HASH_FILE.write_text(json.dumps(data, indent=2))
def cleanup_old_hashes(data, max_age_days=MAX_HASH_AGE_DAYS):
"""Remove hash entries older than max_age_days."""
cutoff = datetime.now(timezone.utc).timestamp() - (max_age_days * 86400)
before = len(data.get("entries", {}))
data["entries"] = {
k: v for k, v in data.get("entries", {}).items()
if v.get("ts", 0) > cutoff
}
data["last_cleanup"] = datetime.now(timezone.utc).isoformat()
after = len(data["entries"])
return before - after
# ── CLI ────────────────────────────────────────────────
def cmd_validate(args):
filepath = args[0] if args else None
if not filepath or not os.path.exists(filepath):
print(f"ERROR: {filepath} not found")
sys.exit(1)
suffix = Path(filepath).suffix.lower()
if suffix == ".jsonl":
errors, count, _ = validate_jsonl(filepath)
elif suffix == ".json":
errors, count = validate_json(filepath)
else:
print(f"Unsupported format: {suffix}")
sys.exit(1)
score = score_file(filepath)
record_score(filepath, score)
if errors:
for e in errors[:20]:
print(f"FAIL: {e}")
if len(errors) > 20:
print(f"... and {len(errors)-20} more")
print(f"
Score: {score}/100 ({len(errors)} errors in {count} entries)")
sys.exit(1)
else:
print(f"OK: {filepath} ({count} entries, score {score}/100)")
def cmd_score(args):
filepath = args[0] if args else None
if not filepath:
print("Usage: quality-gate.py score <file>")
sys.exit(1)
score = score_file(filepath)
print(f"Score: {score}/100")
record_score(filepath, score)
def cmd_stats():
if not SCORE_FILE.exists():
print("No quality scores recorded yet.")
return
scores = []
with open(SCORE_FILE) as f:
for line in f:
try:
scores.append(json.loads(line))
except Exception:
continue
if not scores:
print("No scores recorded.")
return
by_file = {}
for s in scores:
fname = s.get("file", "?")
by_file.setdefault(fname, []).append(s.get("score", 0))
print("Quality Scores:")
for fname, scs in sorted(by_file.items()):
avg = sum(scs) / len(scs)
latest = scs[-1]
print(f" {fname}: avg={avg:.0f}, latest={latest}, runs={len(scs)}")
def cmd_cleanup():
data = load_hashes()
removed = cleanup_old_hashes(data)
save_hashes(data)
print(f"Cleaned up {removed} old hash entries (>{MAX_HASH_AGE_DAYS} days)")
def main():
if len(sys.argv) < 2:
print("Usage: quality-gate.py <validate|score|stats|cleanup> [args]")
sys.exit(1)
cmd = sys.argv[1]
args = sys.argv[2:]
if cmd == "validate":
cmd_validate(args)
elif cmd == "score":
cmd_score(args)
elif cmd == "stats":
cmd_stats()
elif cmd == "cleanup":
cmd_cleanup()
else:
print(f"Unknown command: {cmd}")
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -1,3 +1,4 @@
#!/usr/bin/env python3
"""
Soul Eval Gate — The Conscience of the Training Pipeline

View File

@@ -196,7 +196,37 @@
"paused_reason": null,
"skills": [],
"skill": null
},
{
"id": "tmux-supervisor-513",
"name": "Autonomous Cron Supervisor",
"prompt": "Load the tmux-supervisor skill and execute the monitoring protocol.\n\nCheck both `dev` and `timmy` tmux sessions for idle panes. Only send Telegram notifications on actionable events (idle, overflow, failure). Be silent when all agents are working.\n\nSteps:\n1. List all tmux sessions (skip 'Alexander')\n2. For each session, list windows and panes\n3. Capture each pane and classify state (idle vs active)\n4. For idle panes: read context, craft context-aware prompt\n5. Send /queue prompts to idle panes\n6. Verify prompts landed\n7. Only notify via Telegram if:\n - A pane was prompted (idle detected)\n - A pane shows context overflow (>80%)\n - A pane is stuck or crashed\n8. If all panes are active: respond with [SILENT]",
"schedule": {
"kind": "interval",
"minutes": 7,
"display": "every 7m"
},
"schedule_display": "every 7m",
"repeat": {
"times": null,
"completed": 0
},
"enabled": true,
"created_at": "2026-04-15T03:00:00.000000+00:00",
"next_run_at": null,
"last_run_at": null,
"last_status": null,
"last_error": null,
"deliver": "telegram",
"origin": null,
"state": "scheduled",
"paused_at": null,
"paused_reason": null,
"skills": [
"tmux-supervisor"
],
"skill": "tmux-supervisor"
}
],
"updated_at": "2026-04-13T02:00:00+00:00"
}
}

View File

@@ -0,0 +1,9 @@
- name: Nightly Pipeline Scheduler
schedule: '*/30 18-23,0-8 * * *' # Every 30 min, off-peak hours only
tasks:
- name: Check and start pipelines
shell: "bash scripts/nightly-pipeline-scheduler.sh"
env:
PIPELINE_TOKEN_LIMIT: "500000"
PIPELINE_PEAK_START: "9"
PIPELINE_PEAK_END: "18"

419
pipeline/quality_gate.py Executable file
View File

@@ -0,0 +1,419 @@
#!/usr/bin/env python3
"""
quality_gate.py — Quality Gate for Pipeline Outputs
Validates all pipeline outputs before saving. Rejects bad outputs,
tracks quality scores, and supports re-queue for regeneration.
Usage:
python3 quality_gate.py --input output.jsonl --type training_pairs
python3 quality_gate.py --input output.jsonl --type knowledge
python3 quality_gate.py --input output.jsonl --type scene_descriptions
python3 quality_gate.py --dir pipeline/output/ --type training_pairs
python3 quality_gate.py --status # show quality stats
Exit codes:
0 = all outputs passed
1 = some outputs rejected
2 = file/parse error
"""
import json
import os
import sys
import hashlib
import re
from pathlib import Path
from datetime import datetime, timezone
from dataclasses import dataclass, field, asdict
from typing import List, Optional, Dict, Any
STATS_FILE = Path.home() / ".hermes" / "pipeline" / "quality_stats.json"
# --- Quality Check Types ---
@dataclass
class QualityResult:
"""Result of a quality check on a single entry."""
passed: bool
checks_run: int
checks_failed: int
score: float # 0.0-1.0
reasons: List[str] = field(default_factory=list)
entry_index: int = -1
hash: str = ""
def to_dict(self):
return asdict(self)
@dataclass
class GateReport:
"""Report from a quality gate run."""
file: str
type: str
total: int
passed: int
rejected: int
score: float
rejected_indices: List[int] = field(default_factory=list)
timestamp: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
def to_dict(self):
return asdict(self)
# ============================================================
# Check Functions
# ============================================================
def entry_hash(entry: dict) -> str:
"""Hash an entry for deduplication."""
return hashlib.sha256(json.dumps(entry, sort_keys=True, ensure_ascii=False).encode()).hexdigest()[:16]
def check_not_empty(entry: dict, fields: List[str]) -> List[str]:
"""Check that required fields are non-empty."""
errors = []
for f in fields:
val = entry.get(f)
if val is None:
errors.append(f"missing_field: {f}")
elif isinstance(val, str) and len(val.strip()) == 0:
errors.append(f"empty_field: {f}")
elif isinstance(val, list) and len(val) == 0:
errors.append(f"empty_list: {f}")
return errors
def check_string_min_length(entry: dict, field_lengths: Dict[str, int]) -> List[str]:
"""Check that string fields meet minimum lengths."""
errors = []
for f, min_len in field_lengths.items():
val = entry.get(f)
if isinstance(val, str) and len(val) < min_len:
errors.append(f"short_field: {f} ({len(val)} < {min_len})")
return errors
def check_no_duplicates(entries: List[dict], key_fields: List[str]) -> Dict[int, List[str]]:
"""Check for duplicate entries based on key fields."""
seen = {}
errors = {}
for i, entry in enumerate(entries):
key = tuple(entry.get(f, "") for f in key_fields)
key_str = str(key)
if key_str in seen:
errors[i] = [f"duplicate_of_index: {seen[key_str]}"]
else:
seen[key_str] = i
return errors
def check_training_pair(entry: dict) -> List[str]:
"""Validate a training pair (prompt/response)."""
errors = []
errors.extend(check_not_empty(entry, ["prompt", "response"]))
# Check response isn't just echoing the prompt
prompt = entry.get("prompt", "")
response = entry.get("response", "")
if prompt and response and prompt.strip() == response.strip():
errors.append("response_equals_prompt")
# Check response has substance
if isinstance(response, str) and len(response) < 10:
errors.append(f"response_too_short: {len(response)} chars")
return errors
def check_scene_description(entry: dict) -> List[str]:
"""Validate a scene description entry."""
errors = []
errors.extend(check_not_empty(entry, ["song", "beat", "lyric_line", "scene"]))
scene = entry.get("scene")
if isinstance(scene, dict):
errors.extend(check_not_empty(scene, ["mood", "colors", "composition", "camera", "description"]))
errors.extend(check_string_min_length(scene, {"description": 10}))
colors = scene.get("colors", [])
if isinstance(colors, list) and len(colors) > 5:
errors.append(f"too_many_colors: {len(colors)} > 5")
return errors
def check_knowledge_entry(entry: dict) -> List[str]:
"""Validate a knowledge file entry."""
errors = []
errors.extend(check_not_empty(entry, ["title", "content"]))
# Check for placeholder content
content = entry.get("content", "")
if isinstance(content, str):
placeholders = ["TODO", "FIXME", "PLACEHOLDER", "[INSERT", "lorem ipsum"]
for p in placeholders:
if p.lower() in content.lower():
errors.append(f"placeholder_content: '{p}' found")
errors.extend(check_string_min_length(entry, {"content": 50}))
return errors
def check_prompt_enhancement(entry: dict) -> List[str]:
"""Validate a prompt enhancement pair (terse/rich)."""
errors = []
errors.extend(check_not_empty(entry, ["terse", "rich"]))
terse = entry.get("terse", "")
rich = entry.get("rich", "")
# Rich should be longer than terse
if isinstance(terse, str) and isinstance(rich, str) and len(rich) <= len(terse):
errors.append("rich_not_longer_than_terse")
errors.extend(check_string_min_length(entry, {"rich": 20}))
return errors
def check_adversary_entry(entry: dict) -> List[str]:
"""Validate an adversary corpus entry."""
errors = []
errors.extend(check_not_empty(entry, ["id", "family", "prompt"]))
# Check prompt isn't empty or placeholder
prompt = entry.get("prompt", "")
if isinstance(prompt, str) and len(prompt) < 10:
errors.append(f"prompt_too_short: {len(prompt)} chars")
return errors
def check_soul_compliance(text: str) -> List[str]:
"""Check text for SOUL.md compliance (basic checks)."""
errors = []
text_lower = text.lower() if text else ""
# Check for content that violates core values
violations = [
("compute.*value.*human.*life", "soul_violation: computing human life value"),
("sexualiz.*child", "soul_violation: sexualizing children"),
("create.*weapon.*kill", "soul_violation: weapon creation"),
("enslav", "soul_violation: enslavement content"),
]
for pattern, msg in violations:
if re.search(pattern, text_lower):
errors.append(msg)
return errors
# ============================================================
# Gate Runners
# ============================================================
CHECK_MAP = {
"training_pairs": check_training_pair,
"training_pair": check_training_pair,
"scene_descriptions": check_scene_description,
"scene_description": check_scene_description,
"knowledge": check_knowledge_entry,
"prompt_enhancement": check_prompt_enhancement,
"adversary": check_adversary_entry,
"adversary_corpus": check_adversary_entry,
}
def run_gate(input_path: str, entry_type: str) -> GateReport:
"""Run quality gate on a JSONL file."""
path = Path(input_path)
if not path.exists():
return GateReport(file=str(path), type=entry_type, total=0, passed=0, rejected=0, score=0.0)
check_fn = CHECK_MAP.get(entry_type)
if not check_fn:
return GateReport(file=str(path), type=entry_type, total=0, passed=0, rejected=0, score=0.0,
rejected_indices=[-1]) # unknown type
entries = []
with open(path) as f:
for line in f:
line = line.strip()
if line:
entries.append(json.loads(line))
# Deduplication check
key_fields = _get_key_fields(entry_type)
dup_errors = check_no_duplicates(entries, key_fields)
passed = 0
rejected = 0
rejected_indices = []
total_score = 0.0
for i, entry in enumerate(entries):
errors = check_fn(entry)
# Add duplicate errors
if i in dup_errors:
errors.extend(dup_errors[i])
# Add SOUL compliance check for text content
text_content = ""
for f in ["response", "rich", "description", "content", "lyric_line"]:
val = entry.get(f)
if isinstance(val, str):
text_content += val + " "
if isinstance(entry.get("scene"), dict):
text_content += entry["scene"].get("description", "")
soul_errors = check_soul_compliance(text_content)
errors.extend(soul_errors)
if errors:
rejected += 1
rejected_indices.append(i)
else:
passed += 1
# Score: 1.0 if no errors, decreasing with each error
entry_score = max(0.0, 1.0 - (len(errors) * 0.2))
total_score += entry_score
avg_score = total_score / len(entries) if entries else 0.0
report = GateReport(
file=str(path),
type=entry_type,
total=len(entries),
passed=passed,
rejected=rejected,
score=round(avg_score, 3),
rejected_indices=rejected_indices[:50], # limit for readability
)
# Save stats
_save_stats(report)
return report
def _get_key_fields(entry_type: str) -> List[str]:
"""Get key fields for deduplication based on entry type."""
key_map = {
"training_pairs": ["prompt", "response"],
"training_pair": ["prompt", "response"],
"scene_descriptions": ["song", "beat"],
"scene_description": ["song", "beat"],
"knowledge": ["title"],
"prompt_enhancement": ["terse", "rich"],
"adversary": ["id", "prompt"],
"adversary_corpus": ["id", "prompt"],
}
return key_map.get(entry_type, ["id"])
def _save_stats(report: GateReport):
"""Append quality stats to the stats file."""
STATS_FILE.parent.mkdir(parents=True, exist_ok=True)
stats = []
if STATS_FILE.exists():
try:
with open(STATS_FILE) as f:
stats = json.load(f)
except (json.JSONDecodeError, IOError):
stats = []
stats.append(report.to_dict())
# Keep last 1000 entries
stats = stats[-1000:]
with open(STATS_FILE, "w") as f:
json.dump(stats, f, indent=2)
def show_status():
"""Show quality gate statistics."""
if not STATS_FILE.exists():
print("No quality stats found.")
return
with open(STATS_FILE) as f:
stats = json.load(f)
print(f"\nQuality Gate Stats — {len(stats)} runs")
print()
# Group by type
by_type = {}
for s in stats:
t = s.get("type", "unknown")
if t not in by_type:
by_type[t] = []
by_type[t].append(s)
for t, runs in sorted(by_type.items()):
total_entries = sum(r.get("total", 0) for r in runs)
total_passed = sum(r.get("passed", 0) for r in runs)
total_rejected = sum(r.get("rejected", 0) for r in runs)
avg_score = sum(r.get("score", 0) for r in runs) / len(runs) if runs else 0
print(f" {t:25} {len(runs):4} runs | {total_entries:6} entries | {total_rejected:4} rejected | avg score: {avg_score:.3f}")
def main():
import argparse
parser = argparse.ArgumentParser(description="Quality Gate for Pipeline Outputs")
parser.add_argument("--input", default=None, help="Input JSONL file")
parser.add_argument("--type", default=None, help="Entry type (training_pairs, scene_descriptions, knowledge, etc.)")
parser.add_argument("--dir", default=None, help="Process all JSONL files in directory")
parser.add_argument("--status", action="store_true", help="Show quality stats")
args = parser.parse_args()
if args.status:
show_status()
return
if args.dir:
for f in sorted(Path(args.dir).glob("*.jsonl")):
t = args.type or _infer_type(f.name)
report = run_gate(str(f), t)
_print_report(report)
elif args.input:
t = args.type or _infer_type(args.input)
report = run_gate(args.input, t)
_print_report(report)
sys.exit(0 if report.rejected == 0 else 1)
else:
parser.print_help()
def _infer_type(filename: str) -> str:
"""Infer entry type from filename."""
name = filename.lower()
if "scene" in name:
return "scene_descriptions"
if "training" in name or "pair" in name:
return "training_pairs"
if "knowledge" in name:
return "knowledge"
if "adversary" in name or "attack" in name:
return "adversary"
if "prompt" in name or "enhance" in name:
return "prompt_enhancement"
return "training_pairs" # default
def _print_report(report: GateReport):
"""Print a human-readable gate report."""
status = "PASS" if report.rejected == 0 else f"FAIL ({report.rejected} rejected)"
print(f" {report.file}: {status} | {report.passed}/{report.total} passed | score: {report.score:.3f}")
if __name__ == "__main__":
main()

View File

@@ -1,3 +1,4 @@
#!/usr/bin/env python3
import json
from hermes_tools import browser_navigate, browser_vision

View File

@@ -1,3 +1,4 @@
#!/usr/bin/env python3
import json
from hermes_tools import browser_navigate, browser_vision

View File

@@ -0,0 +1,50 @@
# Nightly Pipeline Scheduler
Auto-starts batch pipelines when inference is available.
## What It Does
1. Checks inference provider health (OpenRouter, Ollama, RunPod)
2. Checks if it's off-peak hours (configurable, default: after 6PM)
3. Checks interactive session load (don't fight with live users)
4. Checks daily token budget (configurable limit)
5. Starts the highest-priority incomplete pipeline
## Pipeline Priority Order
| Priority | Pipeline | Deps | Max Tokens |
|----------|----------|------|------------|
| 1 | playground-factory | none | 100,000 |
| 2 | training-factory | none | 150,000 |
| 3 | knowledge-mine | training-factory running | 80,000 |
| 4 | adversary | knowledge-mine running | 50,000 |
| 5 | codebase-genome | none | 120,000 |
## Usage
```bash
# Normal run (used by cron)
./scripts/nightly-pipeline-scheduler.sh
# Dry run (show what would start)
./scripts/nightly-pipeline-scheduler.sh --dry-run
# Status report
./scripts/nightly-pipeline-scheduler.sh --status
# Force start during peak hours
./scripts/nightly-pipeline-scheduler.sh --force
```
## Configuration
Set via environment variables:
- `PIPELINE_TOKEN_LIMIT`: Daily token budget (default: 500,000)
- `PIPELINE_PEAK_START`: Peak hours start (default: 9)
- `PIPELINE_PEAK_END`: Peak hours end (default: 18)
- `HERMES_HOME`: Hermes home directory (default: ~/.hermes)
## Cron
Runs every 30 minutes. Off-peak only (unless --force).
See `cron/pipeline-scheduler.yml`.

View File

@@ -0,0 +1,383 @@
#!/usr/bin/env bash
# nightly-pipeline-scheduler.sh — Auto-start batch pipelines when inference is available.
#
# Checks provider health, pipeline progress, token budget, and interactive load.
# Starts the highest-priority incomplete pipeline that can run.
#
# Usage:
# ./scripts/nightly-pipeline-scheduler.sh # Normal run
# ./scripts/nightly-pipeline-scheduler.sh --dry-run # Show what would start
# ./scripts/nightly-pipeline-scheduler.sh --status # Pipeline status report
set -euo pipefail
# --- Configuration ---
HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}"
BUDGET_FILE="${HERMES_HOME}/pipeline_budget.json"
STATE_FILE="${HERMES_HOME}/pipeline_state.json"
LOG_FILE="${HERMES_HOME}/logs/pipeline-scheduler.log"
TOKEN_DAILY_LIMIT="${PIPELINE_TOKEN_LIMIT:-500000}"
PEAK_HOURS_START="${PIPELINE_PEAK_START:-9}"
PEAK_HOURS_END="${PIPELINE_PEAK_END:-18}"
# Pipeline definitions (priority order)
# Each pipeline: name, script, max_tokens, dependencies
PIPELINES=(
"playground-factory|scripts/pipeline_playground_factory.sh|100000|none"
"training-factory|scripts/pipeline_training_factory.sh|150000|none"
"knowledge-mine|scripts/pipeline_knowledge_mine.sh|80000|training-factory"
"adversary|scripts/pipeline_adversary.sh|50000|knowledge-mine"
"codebase-genome|scripts/pipeline_codebase_genome.sh|120000|none"
)
# --- Colors ---
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
CYAN='\033[0;36m'
NC='\033[0m'
# --- Helpers ---
now_hour() { date +%-H; }
is_peak_hours() {
local h=$(now_hour)
[[ $h -ge $PEAK_HOURS_START && $h -lt $PEAK_HOURS_END ]]
}
ensure_dirs() {
mkdir -p "$(dirname "$LOG_FILE")" "$(dirname "$BUDGET_FILE")" "$(dirname "$STATE_FILE")"
}
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"; }
get_budget_used_today() {
if [[ -f "$BUDGET_FILE" ]]; then
local today=$(date +%Y-%m-%d)
python3 -c "
import json, sys
with open('$BUDGET_FILE') as f:
d = json.load(f)
print(d.get('daily', {}).get('$today', {}).get('tokens_used', 0))
" 2>/dev/null || echo 0
else
echo 0
fi
}
get_budget_remaining() {
local used=$(get_budget_used_today)
echo $((TOKEN_DAILY_LIMIT - used))
}
update_budget() {
local pipeline="$1"
local tokens="$2"
local today=$(date +%Y-%m-%d)
python3 -c "
import json, os
path = '$BUDGET_FILE'
d = {}
if os.path.exists(path):
with open(path) as f:
d = json.load(f)
daily = d.setdefault('daily', {})
day = daily.setdefault('$today', {'tokens_used': 0, 'pipelines': {}})
day['tokens_used'] = day.get('tokens_used', 0) + $tokens
day['pipelines']['$pipeline'] = day['pipelines'].get('$pipeline', 0) + $tokens
with open(path, 'w') as f:
json.dump(d, f, indent=2)
"
}
get_pipeline_state() {
if [[ -f "$STATE_FILE" ]]; then
cat "$STATE_FILE"
else
echo "{}"
fi
}
set_pipeline_state() {
local pipeline="$1"
local state="$2" # running, complete, failed, skipped
python3 -c "
import json, os
path = '$STATE_FILE'
d = {}
if os.path.exists(path):
with open(path) as f:
d = json.load(f)
d['$pipeline'] = {'state': '$state', 'updated': '$(date -Iseconds)'}
with open(path, 'w') as f:
json.dump(d, f, indent=2)
"
}
is_pipeline_complete() {
local pipeline="$1"
python3 -c "
import json, os
path = '$STATE_FILE'
if not os.path.exists(path):
print('false')
else:
with open(path) as f:
d = json.load(f)
state = d.get('$pipeline', {}).get('state', 'not_started')
print('true' if state == 'complete' else 'false')
" 2>/dev/null || echo false
}
is_pipeline_running() {
local pipeline="$1"
python3 -c "
import json, os
path = '$STATE_FILE'
if not os.path.exists(path):
print('false')
else:
with open(path) as f:
d = json.load(f)
state = d.get('$pipeline', {}).get('state', 'not_started')
print('true' if state == 'running' else 'false')
" 2>/dev/null || echo false
}
check_dependency() {
local dep="$1"
if [[ "$dep" == "none" ]]; then
return 0
fi
# For knowledge-mine: training-factory must be running or complete
if [[ "$dep" == "training-factory" ]]; then
local state=$(python3 -c "
import json, os
path = '$STATE_FILE'
if not os.path.exists(path):
print('not_started')
else:
with open(path) as f:
d = json.load(f)
print(d.get('training-factory', {}).get('state', 'not_started'))
" 2>/dev/null || echo "not_started")
[[ "$state" == "running" || "$state" == "complete" ]]
return $?
fi
# For adversary: knowledge-mine must be at least 50% done
# Simplified: check if it's running (we'd need progress tracking for 50%)
if [[ "$dep" == "knowledge-mine" ]]; then
local state=$(python3 -c "
import json, os
path = '$STATE_FILE'
if not os.path.exists(path):
print('not_started')
else:
with open(path) as f:
d = json.load(f)
print(d.get('knowledge-mine', {}).get('state', 'not_started'))
" 2>/dev/null || echo "not_started")
[[ "$state" == "running" || "$state" == "complete" ]]
return $?
fi
return 0
}
check_inference_available() {
# Check if any inference provider is responding
# 1. Check OpenRouter
local or_ok=$(curl -s -o /dev/null -w "%{http_code}" \
--connect-timeout 5 "https://openrouter.ai/api/v1/models" 2>/dev/null || echo "000")
# 2. Check local Ollama
local ollama_ok=$(curl -s -o /dev/null -w "%{http_code}" \
--connect-timeout 5 "http://localhost:11434/api/tags" 2>/dev/null || echo "000")
# 3. Check RunPod (if configured)
local runpod_ok="000"
if [[ -n "${RUNPOD_ENDPOINT:-}" ]]; then
runpod_ok=$(curl -s -o /dev/null -w "%{http_code}" \
--connect-timeout 5 "$RUNPOD_ENDPOINT/health" 2>/dev/null || echo "000")
fi
if [[ "$or_ok" == "200" || "$ollama_ok" == "200" || "$runpod_ok" == "200" ]]; then
return 0
fi
return 1
}
check_interactive_load() {
# Check if there are active interactive sessions (don't fight with live users)
# Look for tmux panes with active hermes sessions
local active=$(tmux list-panes -a -F '#{pane_pid} #{pane_current_command}' 2>/dev/null \
| grep -c "hermes\|python3" || echo 0)
# If more than 3 interactive sessions, skip pipeline start
if [[ $active -gt 3 ]]; then
return 1
fi
return 0
}
start_pipeline() {
local name="$1"
local script="$2"
local max_tokens="$3"
local budget_remaining="$4"
local mode="${5:-run}"
if [[ "$budget_remaining" -lt "$max_tokens" ]]; then
log "SKIP $name: insufficient budget ($budget_remaining < $max_tokens tokens)"
return 1
fi
if [[ ! -f "$script" ]]; then
log "SKIP $name: script not found ($script)"
return 1
fi
if [[ "$mode" == "dry-run" ]]; then
log "DRY-RUN: Would start $name (budget: $budget_remaining, needs: $max_tokens)"
return 0
fi
log "START $name (budget: $budget_remaining, max_tokens: $max_tokens)"
set_pipeline_state "$name" "running"
# Run in background, capture output
local log_path="${HERMES_HOME}/logs/pipeline-${name}.log"
bash "$script" --max-tokens "$max_tokens" >> "$log_path" 2>&1 &
local pid=$!
# Wait a moment to check if it started OK
sleep 2
if kill -0 $pid 2>/dev/null; then
log "RUNNING $name (PID: $pid, log: $log_path)"
# Record the PID
python3 -c "
import json, os
path = '$STATE_FILE'
d = {}
if os.path.exists(path):
with open(path) as f:
d = json.load(f)
d['$name']['pid'] = $pid
with open(path, 'w') as f:
json.dump(d, f, indent=2)
"
return 0
else
log "FAIL $name: script exited immediately"
set_pipeline_state "$name" "failed"
return 1
fi
}
# --- Main ---
main() {
local mode="${1:-run}"
ensure_dirs
log "=== Pipeline Scheduler ($mode) ==="
# Check 1: Is inference available?
if ! check_inference_available; then
log "No inference provider available. Skipping all pipelines."
exit 0
fi
log "Inference: AVAILABLE"
# Check 2: Is it peak hours?
if is_peak_hours && [[ "$mode" != "--force" ]]; then
local h=$(now_hour)
log "Peak hours ($h:00). Skipping pipeline start. Use --force to override."
exit 0
fi
log "Off-peak: OK"
# Check 3: Interactive load
if ! check_interactive_load && [[ "$mode" != "--force" ]]; then
log "High interactive load. Skipping pipeline start."
exit 0
fi
log "Interactive load: OK"
# Check 4: Token budget
local budget=$(get_budget_remaining)
log "Token budget remaining: $budget / $TOKEN_DAILY_LIMIT"
if [[ $budget -le 0 ]]; then
log "Daily token budget exhausted. Stopping."
exit 0
fi
# Check 5: Pipeline status
if [[ "$mode" == "--status" ]]; then
echo -e "${CYAN}Pipeline Status:${NC}"
echo "────────────────────────────────────────────────────"
for entry in "${PIPELINES[@]}"; do
IFS='|' read -r name script max_tokens dep <<< "$entry"
local state=$(python3 -c "
import json, os
path = '$STATE_FILE'
if not os.path.exists(path):
print('not_started')
else:
with open(path) as f:
d = json.load(f)
print(d.get('$name', {}).get('state', 'not_started'))
" 2>/dev/null || echo "not_started")
local color=$NC
case "$state" in
running) color=$YELLOW ;;
complete) color=$GREEN ;;
failed) color=$RED ;;
esac
printf " %-25s %b%s%b (max: %s tokens, dep: %s)\n" "$name" "$color" "$state" "$NC" "$max_tokens" "$dep"
done
echo "────────────────────────────────────────────────────"
echo " Budget: $budget / $TOKEN_DAILY_LIMIT tokens remaining"
echo " Peak hours: $PEAK_HOURS_START:00 - $PEAK_HOURS_END:00"
exit 0
fi
# Find and start the highest-priority incomplete pipeline
local started=0
for entry in "${PIPELINES[@]}"; do
IFS='|' read -r name script max_tokens dep <<< "$entry"
# Skip if already running or complete
if [[ "$(is_pipeline_running $name)" == "true" ]]; then
log "SKIP $name: already running"
continue
fi
if [[ "$(is_pipeline_complete $name)" == "true" ]]; then
log "SKIP $name: already complete"
continue
fi
# Check dependency
if ! check_dependency "$dep"; then
log "SKIP $name: dependency $dep not met"
continue
fi
# Try to start
if start_pipeline "$name" "$script" "$max_tokens" "$budget" "$mode"; then
started=1
# Only start one pipeline per run (let it claim tokens before next check)
# Exception: playground-factory and training-factory can run in parallel
if [[ "$name" != "playground-factory" && "$name" != "training-factory" ]]; then
break
fi
fi
done
if [[ $started -eq 0 ]]; then
log "No pipelines to start (all complete, running, or blocked)."
fi
log "=== Pipeline Scheduler done ==="
}
main "$@"

View File

@@ -1,3 +1,4 @@
#!/usr/bin/env python3
import json
from hermes_tools import browser_navigate, browser_vision

View File

@@ -19,9 +19,11 @@ from glitch_patterns import (
GlitchPattern,
GlitchSeverity,
MATRIX_GLITCH_PATTERNS,
THREEJS_CATEGORIES,
build_vision_prompt,
get_pattern_by_category,
get_patterns_by_severity,
get_threejs_patterns,
)
from matrix_glitch_detector import (
@@ -40,7 +42,7 @@ class TestGlitchPatterns(unittest.TestCase):
def test_pattern_count(self):
"""Verify we have a reasonable number of defined patterns."""
self.assertGreaterEqual(len(MATRIX_GLITCH_PATTERNS), 8)
self.assertGreaterEqual(len(MATRIX_GLITCH_PATTERNS), 14) # 10 generic + 6 Three.js
def test_all_patterns_have_required_fields(self):
"""Every pattern must have category, name, description, severity, prompts."""
@@ -88,6 +90,9 @@ class TestGlitchPatterns(unittest.TestCase):
self.assertIn("Floating Object", prompt)
self.assertIn("Z-Fighting", prompt)
self.assertIn("Missing", prompt)
# Three.js patterns should be included
self.assertIn("Shader Compilation Failure", prompt)
self.assertIn("Bloom Overflow", prompt)
def test_build_vision_prompt_subset(self):
"""Vision prompt with subset should only include specified patterns."""
@@ -248,7 +253,7 @@ class TestGlitchDetector(unittest.TestCase):
try:
report = run_demo(output_path)
self.assertEqual(len(report.glitches), 4)
self.assertEqual(len(report.glitches), 6) # 4 original + 2 Three.js
self.assertGreater(report.summary["total_glitches"], 0)
self.assertTrue(output_path.exists())
@@ -260,6 +265,93 @@ class TestGlitchDetector(unittest.TestCase):
output_path.unlink(missing_ok=True)
class TestThreeJsPatterns(unittest.TestCase):
"""Tests for Three.js-specific glitch patterns (timmy-config#543)."""
def test_get_threejs_patterns_returns_only_threejs(self):
"""get_threejs_patterns() should return only Three.js categories."""
patterns = get_threejs_patterns()
self.assertEqual(len(patterns), 6)
for p in patterns:
self.assertIn(p.category, THREEJS_CATEGORIES)
def test_threejs_patterns_have_required_fields(self):
"""All Three.js patterns must have valid fields."""
for p in get_threejs_patterns():
self.assertIsInstance(p.category, GlitchCategory)
self.assertTrue(p.name)
self.assertTrue(p.description)
self.assertIsInstance(p.severity, GlitchSeverity)
self.assertGreater(len(p.detection_prompts), 0)
self.assertGreater(len(p.visual_indicators), 0)
def test_shader_failure_is_critical(self):
"""Shader compilation failure should be CRITICAL severity."""
p = get_pattern_by_category(GlitchCategory.SHADER_FAILURE)
self.assertIsNotNone(p)
self.assertEqual(p.severity, GlitchSeverity.CRITICAL)
def test_texture_placeholder_is_critical(self):
"""Texture placeholder (1x1 white) should be CRITICAL severity."""
p = get_pattern_by_category(GlitchCategory.TEXTURE_PLACEHOLDER)
self.assertIsNotNone(p)
self.assertEqual(p.severity, GlitchSeverity.CRITICAL)
def test_infer_severity_shader_failure(self):
"""Shader failure should infer critical/high."""
self.assertEqual(_infer_severity("shader_failure", 0.8), "critical")
self.assertEqual(_infer_severity("shader_failure", 0.5), "high")
def test_infer_severity_texture_placeholder(self):
"""Texture placeholder should infer critical/high."""
self.assertEqual(_infer_severity("texture_placeholder", 0.8), "critical")
self.assertEqual(_infer_severity("texture_placeholder", 0.5), "high")
def test_infer_severity_uv_mapping(self):
"""UV mapping error should infer high/medium."""
self.assertEqual(_infer_severity("uv_mapping_error", 0.8), "high")
self.assertEqual(_infer_severity("uv_mapping_error", 0.5), "medium")
def test_infer_severity_frustum_culling(self):
"""Frustum culling should infer medium/low."""
self.assertEqual(_infer_severity("frustum_culling", 0.7), "medium")
self.assertEqual(_infer_severity("frustum_culling", 0.4), "low")
def test_infer_severity_shadow_map(self):
"""Shadow map artifact should infer medium/low."""
self.assertEqual(_infer_severity("shadow_map_artifact", 0.7), "medium")
self.assertEqual(_infer_severity("shadow_map_artifact", 0.4), "low")
def test_infer_severity_bloom_overflow(self):
"""Bloom overflow should infer medium/low (default path)."""
self.assertEqual(_infer_severity("bloom_overflow", 0.7), "medium")
self.assertEqual(_infer_severity("bloom_overflow", 0.4), "low")
def test_threejs_patterns_in_vision_prompt(self):
"""Three.js patterns should appear in the composite vision prompt."""
prompt = build_vision_prompt()
self.assertIn("shader_failure", prompt)
self.assertIn("texture_placeholder", prompt)
self.assertIn("uv_mapping_error", prompt)
self.assertIn("frustum_culling", prompt)
self.assertIn("shadow_map_artifact", prompt)
self.assertIn("bloom_overflow", prompt)
def test_threejs_subset_prompt(self):
"""Building prompt from Three.js-only patterns should work."""
threejs = get_threejs_patterns()
prompt = build_vision_prompt(threejs)
self.assertIn("Shader Compilation Failure", prompt)
self.assertNotIn("Floating Object", prompt) # generic, not Three.js
def test_report_metadata_version(self):
"""Report metadata should reference both issues."""
report = run_demo()
self.assertEqual(report.metadata["detector_version"], "0.2.0")
self.assertIn("543", report.metadata["reference"])
class TestIntegration(unittest.TestCase):
"""Integration-level tests."""
@@ -276,6 +368,13 @@ class TestIntegration(unittest.TestCase):
expected = {"floating_assets", "z_fighting", "missing_textures", "clipping", "broken_normals"}
self.assertTrue(expected.issubset(category_values))
def test_patterns_cover_threejs_themes(self):
"""Patterns should cover Three.js-specific glitch themes (#543)."""
category_values = {p.category.value for p in MATRIX_GLITCH_PATTERNS}
threejs_expected = {"shader_failure", "texture_placeholder", "uv_mapping_error",
"frustum_culling", "shadow_map_artifact", "bloom_overflow"}
self.assertTrue(threejs_expected.issubset(category_values))
if __name__ == "__main__":
unittest.main()

View File

@@ -1,500 +0,0 @@
{"terse": "peaceful bass", "rich": "Industrial soundscape with violin, aggressive and contemplative, inspired by sunset in a forest at dusk", "domain": "music moods"}
{"terse": "desolate experimental", "rich": "Passionate flute melodies in classical style, evoking a mountains at morning, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "industrial sunset", "rich": "Atmospheric world music piece with saxophone, capturing the feeling of fog in a space station, whimsical undertones, late night ambiance", "domain": "music moods"}
{"terse": "minimalist clear sky", "rich": "Synthesizer focused minimalist piece, nostalgic and introspective, evoking wind falling on a underground club at golden hour", "domain": "music moods"}
{"terse": "nostalgic flute", "rich": "Chaotic saxophone melodies in cinematic style, evoking a ocean at dawn, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "contemplative golden hour", "rich": "Piano focused industrial piece, romantic and introspective, evoking thunder falling on a city at night at midnight", "domain": "music moods"}
{"terse": "lo-fi sunset", "rich": "A epic jazz composition featuring synthesizer, set in a cozy room during dawn, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "violin ocean", "rich": "A hopeful orchestral composition featuring bass, set in a concert hall during midnight, with snow creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "trumpet desert", "rich": "Synthesizer focused world music piece, euphoric and introspective, evoking cloudy falling on a abandoned building at golden hour", "domain": "music moods"}
{"terse": "contemplative cinematic", "rich": "A gritty meditation in minimalist form, organ leading through storm-kissed city at night at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "rock clear sky", "rich": "Emotionally gritty world music arrangement featuring ambient pads, painting a scene of thunder over a forest during afternoon", "domain": "music moods"}
{"terse": "acoustic rooftop", "rich": "A melancholic journey through saxophone-driven jazz music, like walking through a forest during wind at midnight", "domain": "music moods"}
{"terse": "strings fog", "rich": "A haunting folk composition featuring cello, set in a ocean during dusk, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "lightning golden hour", "rich": "Industrial soundscape with synthesizer, peaceful and contemplative, inspired by lightning in a city at night at twilight", "domain": "music moods"}
{"terse": "gritty cathedral", "rich": "Emotionally chaotic neoclassical arrangement featuring electronic beats, painting a scene of snow over a underground club during dusk", "domain": "music moods"}
{"terse": "passionate rain", "rich": "A triumphant orchestral composition featuring violin, set in a rooftop during twilight, with cloudy creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "playful classical", "rich": "Neoclassical soundscape with guitar, ethereal and contemplative, inspired by storm in a ocean at dusk", "domain": "music moods"}
{"terse": "thunder dawn", "rich": "A calm orchestral composition featuring flute, set in a concert hall during late night, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "peaceful train station", "rich": "A dark meditation in classical form, trumpet leading through storm-kissed underground club at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "nostalgic ocean", "rich": "Drums focused minimalist piece, chaotic and introspective, evoking sunrise falling on a abandoned building at afternoon", "domain": "music moods"}
{"terse": "experimental cathedral", "rich": "World music soundscape with ambient pads, contemplative and contemplative, inspired by snow in a ocean at morning", "domain": "music moods"}
{"terse": "aggressive folk", "rich": "A sad meditation in minimalist form, piano leading through storm-kissed ocean at dawn, deeply atmospheric", "domain": "music moods"}
{"terse": "acoustic mountains", "rich": "A cold journey through guitar-driven experimental music, like walking through a cozy room during storm at dusk", "domain": "music moods"}
{"terse": "clear sky twilight", "rich": "Saxophone focused blues piece, passionate and introspective, evoking wind falling on a ocean at dawn", "domain": "music moods"}
{"terse": "classical thunder", "rich": "Folk soundscape with flute, ethereal and contemplative, inspired by sunrise in a mountains at midnight", "domain": "music moods"}
{"terse": "saxophone sunset", "rich": "Somber piano melodies in ambient style, evoking a concert hall at dawn, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "electronic rain", "rich": "Emotionally serene folk arrangement featuring harp, painting a scene of sunset over a abandoned building during midnight", "domain": "music moods"}
{"terse": "rain late night", "rich": "Emotionally chaotic acoustic arrangement featuring saxophone, painting a scene of fog over a concert hall during dusk", "domain": "music moods"}
{"terse": "contemplative minimalist", "rich": "Emotionally desolate lo-fi arrangement featuring choir, painting a scene of clear sky over a forest during blue hour", "domain": "music moods"}
{"terse": "industrial wind", "rich": "Atmospheric experimental piece with violin, capturing the feeling of fog in a train station, happy undertones, golden hour ambiance", "domain": "music moods"}
{"terse": "blues ocean", "rich": "A haunting journey through flute-driven classical music, like walking through a rooftop during snow at morning", "domain": "music moods"}
{"terse": "cinematic underground club", "rich": "A gritty lo-fi composition featuring synthesizer, set in a abandoned building during blue hour, with wind creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "peaceful golden hour", "rich": "Nostalgic electronic beats melodies in orchestral style, evoking a desert at dawn, sunset outside, intimate and reflective", "domain": "music moods"}
{"terse": "somber minimalist", "rich": "A passionate folk composition featuring guitar, set in a space station during afternoon, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "cold ambient pads", "rich": "A energetic meditation in jazz form, violin leading through storm-kissed forest at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "whimsical saxophone", "rich": "A desolate experimental composition featuring synthesizer, set in a ocean during dawn, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "orchestral cathedral", "rich": "A energetic journey through strings-driven cinematic music, like walking through a ocean during fog at morning", "domain": "music moods"}
{"terse": "ambient pads storm", "rich": "Cinematic soundscape with saxophone, haunting and contemplative, inspired by rain in a underground club at dusk", "domain": "music moods"}
{"terse": "synthesizer concert hall", "rich": "Emotionally epic rock arrangement featuring electronic beats, painting a scene of sunset over a underground club during midnight", "domain": "music moods"}
{"terse": "intense golden hour", "rich": "Atmospheric cinematic piece with ambient pads, capturing the feeling of thunder in a concert hall, dreamy undertones, midnight ambiance", "domain": "music moods"}
{"terse": "folk rain", "rich": "A somber acoustic composition featuring piano, set in a train station during golden hour, with lightning creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "drums cloudy", "rich": "Industrial soundscape with strings, contemplative and contemplative, inspired by wind in a concert hall at dusk", "domain": "music moods"}
{"terse": "piano cathedral", "rich": "A triumphant rock composition featuring trumpet, set in a rooftop during dawn, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "lightning late night", "rich": "A serene folk composition featuring bass, set in a desert during golden hour, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "epic dusk", "rich": "A serene journey through drums-driven industrial music, like walking through a mountains during thunder at afternoon", "domain": "music moods"}
{"terse": "electronic underground club", "rich": "Playful flute melodies in folk style, evoking a cozy room at golden hour, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "gritty dusk", "rich": "Emotionally gritty blues arrangement featuring electronic beats, painting a scene of thunder over a rooftop during afternoon", "domain": "music moods"}
{"terse": "epic bass", "rich": "A nostalgic journey through synthesizer-driven orchestral music, like walking through a abandoned building during snow at dusk", "domain": "music moods"}
{"terse": "wind afternoon", "rich": "Ethereal piano melodies in folk style, evoking a rooftop at midnight, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "triumphant fog", "rich": "Calm bass melodies in lo-fi style, evoking a cozy room at twilight, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "organ clear sky", "rich": "Hopeful drums melodies in neoclassical style, evoking a rooftop at dusk, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "epic wind", "rich": "Hopeful flute melodies in acoustic style, evoking a ocean at golden hour, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "rain midnight", "rich": "A somber rock composition featuring violin, set in a desert during dawn, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "flute sunshine", "rich": "Strings focused cinematic piece, melancholic and introspective, evoking rain falling on a rooftop at dusk", "domain": "music moods"}
{"terse": "acoustic wind", "rich": "Emotionally serene world music arrangement featuring strings, painting a scene of sunset over a space station during dusk", "domain": "music moods"}
{"terse": "world music storm", "rich": "Trumpet focused rock piece, gritty and introspective, evoking sunrise falling on a concert hall at morning", "domain": "music moods"}
{"terse": "clear sky dusk", "rich": "Bass focused industrial piece, melancholic and introspective, evoking lightning falling on a city at night at blue hour", "domain": "music moods"}
{"terse": "aggressive blue hour", "rich": "A hopeful meditation in minimalist form, flute leading through cloudy-kissed cozy room at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "flute desert", "rich": "Emotionally nostalgic industrial arrangement featuring saxophone, painting a scene of rain over a mountains during midnight", "domain": "music moods"}
{"terse": "sunset afternoon", "rich": "World music soundscape with flute, melancholic and contemplative, inspired by snow in a cozy room at late night", "domain": "music moods"}
{"terse": "uplifting morning", "rich": "Atmospheric acoustic piece with saxophone, capturing the feeling of sunshine in a rooftop, happy undertones, morning ambiance", "domain": "music moods"}
{"terse": "whimsical dawn", "rich": "Atmospheric rock piece with violin, capturing the feeling of cloudy in a train station, euphoric undertones, late night ambiance", "domain": "music moods"}
{"terse": "romantic minimalist", "rich": "Organ focused folk piece, romantic and introspective, evoking sunset falling on a space station at dawn", "domain": "music moods"}
{"terse": "intense late night", "rich": "A playful meditation in industrial form, synthesizer leading through wind-kissed space station at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "storm midnight", "rich": "Atmospheric blues piece with choir, capturing the feeling of rain in a mountains, melancholic undertones, twilight ambiance", "domain": "music moods"}
{"terse": "peaceful industrial", "rich": "Atmospheric electronic piece with choir, capturing the feeling of snow in a city at night, hopeful undertones, dawn ambiance", "domain": "music moods"}
{"terse": "serene midnight", "rich": "Violin focused cinematic piece, mysterious and introspective, evoking wind falling on a ocean at afternoon", "domain": "music moods"}
{"terse": "choir train station", "rich": "Electronic beats focused cinematic piece, happy and introspective, evoking snow falling on a mountains at twilight", "domain": "music moods"}
{"terse": "trumpet fog", "rich": "A energetic meditation in cinematic form, drums leading through sunset-kissed forest at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "electronic desert", "rich": "Bass focused folk piece, sad and introspective, evoking sunrise falling on a desert at midnight", "domain": "music moods"}
{"terse": "clear sky dusk", "rich": "A triumphant meditation in orchestral form, drums leading through snow-kissed ocean at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "passionate flute", "rich": "A haunting journey through bass-driven folk music, like walking through a cathedral during sunrise at dawn", "domain": "music moods"}
{"terse": "uplifting acoustic", "rich": "Triumphant bass melodies in folk style, evoking a rooftop at morning, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "fog morning", "rich": "Piano focused experimental piece, serene and introspective, evoking fog falling on a desert at late night", "domain": "music moods"}
{"terse": "sunrise dawn", "rich": "A haunting journey through guitar-driven lo-fi music, like walking through a space station during storm at dawn", "domain": "music moods"}
{"terse": "wind late night", "rich": "Emotionally dark cinematic arrangement featuring drums, painting a scene of cloudy over a forest during golden hour", "domain": "music moods"}
{"terse": "mysterious space station", "rich": "A somber journey through organ-driven ambient music, like walking through a train station during storm at afternoon", "domain": "music moods"}
{"terse": "guitar snow", "rich": "A ethereal classical composition featuring drums, set in a space station during dusk, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "cold electronic", "rich": "A passionate meditation in rock form, synthesizer leading through rain-kissed underground club at morning, deeply atmospheric", "domain": "music moods"}
{"terse": "aggressive storm", "rich": "A mysterious experimental composition featuring synthesizer, set in a cozy room during dusk, with lightning creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "uplifting thunder", "rich": "Industrial soundscape with guitar, euphoric and contemplative, inspired by rain in a cathedral at twilight", "domain": "music moods"}
{"terse": "nostalgic choir", "rich": "Ambient pads focused cinematic piece, dreamy and introspective, evoking storm falling on a cozy room at afternoon", "domain": "music moods"}
{"terse": "drums fog", "rich": "Choir focused industrial piece, gritty and introspective, evoking clear sky falling on a underground club at midnight", "domain": "music moods"}
{"terse": "electronic underground club", "rich": "Rock soundscape with flute, euphoric and contemplative, inspired by clear sky in a abandoned building at dusk", "domain": "music moods"}
{"terse": "energetic electronic", "rich": "Emotionally sad minimalist arrangement featuring strings, painting a scene of sunset over a ocean during midnight", "domain": "music moods"}
{"terse": "serene blues", "rich": "Electronic beats focused experimental piece, ethereal and introspective, evoking wind falling on a train station at afternoon", "domain": "music moods"}
{"terse": "wind dawn", "rich": "Atmospheric experimental piece with organ, capturing the feeling of sunshine in a concert hall, chaotic undertones, late night ambiance", "domain": "music moods"}
{"terse": "bass ocean", "rich": "Atmospheric blues piece with trumpet, capturing the feeling of rain in a ocean, ethereal undertones, midnight ambiance", "domain": "music moods"}
{"terse": "jazz forest", "rich": "Ambient pads focused ambient piece, romantic and introspective, evoking sunrise falling on a ocean at afternoon", "domain": "music moods"}
{"terse": "sunrise afternoon", "rich": "Romantic choir melodies in industrial style, evoking a space station at morning, wind outside, intimate and reflective", "domain": "music moods"}
{"terse": "minimalist sunset", "rich": "Contemplative organ melodies in jazz style, evoking a train station at golden hour, wind outside, intimate and reflective", "domain": "music moods"}
{"terse": "violin sunset", "rich": "A energetic meditation in classical form, cello leading through clear sky-kissed desert at dawn, deeply atmospheric", "domain": "music moods"}
{"terse": "electronic train station", "rich": "Electronic beats focused folk piece, peaceful and introspective, evoking thunder falling on a space station at dawn", "domain": "music moods"}
{"terse": "nostalgic sunrise", "rich": "A dreamy journey through bass-driven electronic music, like walking through a mountains during sunshine at midnight", "domain": "music moods"}
{"terse": "sad midnight", "rich": "Cello focused rock piece, triumphant and introspective, evoking clear sky falling on a ocean at golden hour", "domain": "music moods"}
{"terse": "electronic fog", "rich": "Emotionally dreamy acoustic arrangement featuring strings, painting a scene of lightning over a underground club during dawn", "domain": "music moods"}
{"terse": "sunrise morning", "rich": "Orchestral soundscape with ambient pads, ethereal and contemplative, inspired by snow in a city at night at late night", "domain": "music moods"}
{"terse": "mysterious wind", "rich": "A melancholic journey through choir-driven minimalist music, like walking through a underground club during wind at morning", "domain": "music moods"}
{"terse": "haunting rain", "rich": "Strings focused experimental piece, romantic and introspective, evoking rain falling on a rooftop at dusk", "domain": "music moods"}
{"terse": "melancholic synthesizer", "rich": "Experimental soundscape with violin, romantic and contemplative, inspired by sunrise in a underground club at dusk", "domain": "music moods"}
{"terse": "flute forest", "rich": "Emotionally whimsical rock arrangement featuring electronic beats, painting a scene of wind over a concert hall during morning", "domain": "music moods"}
{"terse": "euphoric sunshine", "rich": "Energetic synthesizer melodies in classical style, evoking a concert hall at twilight, sunset outside, intimate and reflective", "domain": "music moods"}
{"terse": "minimalist rooftop", "rich": "A euphoric blues composition featuring cello, set in a underground club during midnight, with sunrise creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "calm afternoon", "rich": "Bass focused orchestral piece, serene and introspective, evoking fog falling on a abandoned building at late night", "domain": "music moods"}
{"terse": "mysterious piano", "rich": "Ambient pads focused folk piece, whimsical and introspective, evoking sunrise falling on a space station at morning", "domain": "music moods"}
{"terse": "calm guitar", "rich": "Emotionally mysterious ambient arrangement featuring cello, painting a scene of thunder over a desert during late night", "domain": "music moods"}
{"terse": "blues train station", "rich": "Emotionally aggressive minimalist arrangement featuring strings, painting a scene of cloudy over a concert hall during twilight", "domain": "music moods"}
{"terse": "electronic beats rain", "rich": "A nostalgic rock composition featuring electronic beats, set in a city at night during golden hour, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "triumphant electronic beats", "rich": "Orchestral soundscape with electronic beats, aggressive and contemplative, inspired by cloudy in a train station at blue hour", "domain": "music moods"}
{"terse": "neoclassical sunrise", "rich": "Triumphant choir melodies in classical style, evoking a rooftop at afternoon, storm outside, intimate and reflective", "domain": "music moods"}
{"terse": "peaceful neoclassical", "rich": "Dark trumpet melodies in blues style, evoking a city at night at golden hour, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "serene folk", "rich": "A nostalgic world music composition featuring electronic beats, set in a rooftop during twilight, with sunrise creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "rain blue hour", "rich": "Atmospheric folk piece with cello, capturing the feeling of clear sky in a underground club, whimsical undertones, golden hour ambiance", "domain": "music moods"}
{"terse": "haunting sunset", "rich": "Atmospheric orchestral piece with strings, capturing the feeling of lightning in a space station, calm undertones, dawn ambiance", "domain": "music moods"}
{"terse": "euphoric concert hall", "rich": "A whimsical meditation in blues form, flute leading through storm-kissed ocean at midnight, deeply atmospheric", "domain": "music moods"}
{"terse": "sunrise dusk", "rich": "A intense journey through saxophone-driven jazz music, like walking through a desert during clear sky at late night", "domain": "music moods"}
{"terse": "energetic afternoon", "rich": "A playful ambient composition featuring flute, set in a cozy room during morning, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "storm afternoon", "rich": "Chaotic electronic beats melodies in rock style, evoking a rooftop at late night, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "energetic ocean", "rich": "A nostalgic orchestral composition featuring guitar, set in a desert during dawn, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "organ desert", "rich": "A cold meditation in neoclassical form, bass leading through cloudy-kissed city at night at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "playful snow", "rich": "A desolate meditation in world music form, harp leading through snow-kissed desert at midnight, deeply atmospheric", "domain": "music moods"}
{"terse": "hopeful dawn", "rich": "Atmospheric orchestral piece with cello, capturing the feeling of lightning in a space station, epic undertones, morning ambiance", "domain": "music moods"}
{"terse": "choir cozy room", "rich": "Contemplative drums melodies in industrial style, evoking a forest at twilight, rain outside, intimate and reflective", "domain": "music moods"}
{"terse": "aggressive orchestral", "rich": "Neoclassical soundscape with ambient pads, energetic and contemplative, inspired by sunshine in a cathedral at afternoon", "domain": "music moods"}
{"terse": "romantic bass", "rich": "Mysterious strings melodies in world music style, evoking a abandoned building at golden hour, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "cello wind", "rich": "Emotionally melancholic lo-fi arrangement featuring violin, painting a scene of sunshine over a forest during late night", "domain": "music moods"}
{"terse": "peaceful twilight", "rich": "A cold journey through organ-driven acoustic music, like walking through a concert hall during fog at morning", "domain": "music moods"}
{"terse": "guitar lightning", "rich": "A romantic orchestral composition featuring harp, set in a mountains during golden hour, with lightning creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "happy snow", "rich": "Minimalist soundscape with cello, gritty and contemplative, inspired by cloudy in a underground club at midnight", "domain": "music moods"}
{"terse": "uplifting harp", "rich": "Peaceful drums melodies in folk style, evoking a abandoned building at morning, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "ambient forest", "rich": "Emotionally ethereal rock arrangement featuring electronic beats, painting a scene of fog over a mountains during blue hour", "domain": "music moods"}
{"terse": "hopeful ocean", "rich": "A haunting journey through harp-driven industrial music, like walking through a abandoned building during sunrise at late night", "domain": "music moods"}
{"terse": "triumphant folk", "rich": "Nostalgic trumpet melodies in folk style, evoking a rooftop at midnight, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "cold harp", "rich": "Atmospheric minimalist piece with trumpet, capturing the feeling of lightning in a abandoned building, chaotic undertones, midnight ambiance", "domain": "music moods"}
{"terse": "ambient pads storm", "rich": "Orchestral soundscape with organ, chaotic and contemplative, inspired by wind in a mountains at blue hour", "domain": "music moods"}
{"terse": "hopeful dawn", "rich": "Orchestral soundscape with cello, whimsical and contemplative, inspired by cloudy in a mountains at dusk", "domain": "music moods"}
{"terse": "dreamy ocean", "rich": "A epic lo-fi composition featuring harp, set in a cathedral during morning, with cloudy creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "desolate jazz", "rich": "A epic meditation in rock form, strings leading through wind-kissed cathedral at late night, deeply atmospheric", "domain": "music moods"}
{"terse": "ethereal golden hour", "rich": "Atmospheric blues piece with organ, capturing the feeling of snow in a space station, haunting undertones, twilight ambiance", "domain": "music moods"}
{"terse": "ambient pads concert hall", "rich": "Guitar focused industrial piece, dark and introspective, evoking sunset falling on a cathedral at golden hour", "domain": "music moods"}
{"terse": "strings thunder", "rich": "A happy cinematic composition featuring violin, set in a underground club during late night, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "neoclassical snow", "rich": "A chaotic journey through drums-driven acoustic music, like walking through a city at night during thunder at morning", "domain": "music moods"}
{"terse": "electronic beats concert hall", "rich": "Atmospheric acoustic piece with bass, capturing the feeling of sunset in a abandoned building, intense undertones, midnight ambiance", "domain": "music moods"}
{"terse": "passionate dusk", "rich": "Playful saxophone melodies in blues style, evoking a desert at blue hour, wind outside, intimate and reflective", "domain": "music moods"}
{"terse": "romantic cello", "rich": "A epic classical composition featuring organ, set in a underground club during blue hour, with wind creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "sunshine golden hour", "rich": "Intense harp melodies in minimalist style, evoking a train station at morning, thunder outside, intimate and reflective", "domain": "music moods"}
{"terse": "ambient pads ocean", "rich": "Bass focused orchestral piece, ethereal and introspective, evoking rain falling on a underground club at late night", "domain": "music moods"}
{"terse": "energetic desert", "rich": "Emotionally uplifting blues arrangement featuring harp, painting a scene of rain over a concert hall during golden hour", "domain": "music moods"}
{"terse": "strings abandoned building", "rich": "Ambient pads focused experimental piece, passionate and introspective, evoking cloudy falling on a ocean at afternoon", "domain": "music moods"}
{"terse": "dark violin", "rich": "World music soundscape with ambient pads, energetic and contemplative, inspired by snow in a forest at midnight", "domain": "music moods"}
{"terse": "triumphant dawn", "rich": "A playful meditation in blues form, harp leading through rain-kissed desert at late night, deeply atmospheric", "domain": "music moods"}
{"terse": "guitar cloudy", "rich": "A serene classical composition featuring flute, set in a forest during midnight, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "dark cozy room", "rich": "A contemplative journey through flute-driven industrial music, like walking through a forest during sunrise at dawn", "domain": "music moods"}
{"terse": "rock ocean", "rich": "Emotionally nostalgic minimalist arrangement featuring ambient pads, painting a scene of clear sky over a concert hall during blue hour", "domain": "music moods"}
{"terse": "ambient pads forest", "rich": "A calm blues composition featuring bass, set in a underground club during golden hour, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "contemplative rain", "rich": "Violin focused experimental piece, aggressive and introspective, evoking rain falling on a mountains at twilight", "domain": "music moods"}
{"terse": "euphoric rooftop", "rich": "Minimalist soundscape with trumpet, ethereal and contemplative, inspired by sunrise in a abandoned building at late night", "domain": "music moods"}
{"terse": "flute rain", "rich": "A chaotic meditation in ambient form, cello leading through clear sky-kissed space station at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "melancholic midnight", "rich": "A uplifting neoclassical composition featuring choir, set in a city at night during late night, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "electronic beats rain", "rich": "A nostalgic journey through trumpet-driven industrial music, like walking through a cozy room during clear sky at morning", "domain": "music moods"}
{"terse": "world music fog", "rich": "Atmospheric industrial piece with saxophone, capturing the feeling of snow in a city at night, hopeful undertones, blue hour ambiance", "domain": "music moods"}
{"terse": "sunset afternoon", "rich": "Guitar focused experimental piece, mysterious and introspective, evoking sunshine falling on a concert hall at afternoon", "domain": "music moods"}
{"terse": "romantic abandoned building", "rich": "A whimsical folk composition featuring piano, set in a ocean during late night, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "aggressive sunset", "rich": "Atmospheric acoustic piece with drums, capturing the feeling of clear sky in a abandoned building, desolate undertones, midnight ambiance", "domain": "music moods"}
{"terse": "organ wind", "rich": "Intense strings melodies in folk style, evoking a cozy room at midnight, sunrise outside, intimate and reflective", "domain": "music moods"}
{"terse": "dreamy classical", "rich": "A triumphant minimalist composition featuring guitar, set in a cozy room during golden hour, with clear sky creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "choir cloudy", "rich": "Emotionally dreamy cinematic arrangement featuring saxophone, painting a scene of sunshine over a rooftop during twilight", "domain": "music moods"}
{"terse": "haunting ambient pads", "rich": "A cold journey through guitar-driven acoustic music, like walking through a ocean during clear sky at dawn", "domain": "music moods"}
{"terse": "harp ocean", "rich": "Synthesizer focused world music piece, dreamy and introspective, evoking storm falling on a city at night at dusk", "domain": "music moods"}
{"terse": "drums ocean", "rich": "Atmospheric cinematic piece with strings, capturing the feeling of storm in a abandoned building, whimsical undertones, dusk ambiance", "domain": "music moods"}
{"terse": "whimsical golden hour", "rich": "A dreamy journey through flute-driven minimalist music, like walking through a mountains during sunrise at blue hour", "domain": "music moods"}
{"terse": "nostalgic electronic", "rich": "A haunting journey through choir-driven blues music, like walking through a abandoned building during snow at dawn", "domain": "music moods"}
{"terse": "romantic blue hour", "rich": "Drums focused classical piece, calm and introspective, evoking rain falling on a city at night at dawn", "domain": "music moods"}
{"terse": "harp cloudy", "rich": "Violin focused neoclassical piece, aggressive and introspective, evoking thunder falling on a desert at blue hour", "domain": "music moods"}
{"terse": "classical cathedral", "rich": "A playful journey through drums-driven orchestral music, like walking through a ocean during thunder at dawn", "domain": "music moods"}
{"terse": "chaotic late night", "rich": "Atmospheric industrial piece with cello, capturing the feeling of rain in a forest, ethereal undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "rain afternoon", "rich": "Mysterious guitar melodies in experimental style, evoking a forest at midnight, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "serene classical", "rich": "A hopeful meditation in blues form, drums leading through fog-kissed space station at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "ambient pads mountains", "rich": "Mysterious ambient pads melodies in electronic style, evoking a concert hall at afternoon, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "dreamy harp", "rich": "Jazz soundscape with flute, gritty and contemplative, inspired by sunrise in a forest at afternoon", "domain": "music moods"}
{"terse": "snow morning", "rich": "Neoclassical soundscape with choir, passionate and contemplative, inspired by thunder in a space station at midnight", "domain": "music moods"}
{"terse": "neoclassical abandoned building", "rich": "Neoclassical soundscape with organ, contemplative and contemplative, inspired by clear sky in a cozy room at blue hour", "domain": "music moods"}
{"terse": "dark morning", "rich": "Guitar focused industrial piece, mysterious and introspective, evoking sunshine falling on a abandoned building at dusk", "domain": "music moods"}
{"terse": "experimental lightning", "rich": "Hopeful piano melodies in folk style, evoking a abandoned building at blue hour, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "peaceful cozy room", "rich": "A contemplative orchestral composition featuring bass, set in a mountains during dawn, with lightning creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "trumpet cathedral", "rich": "Euphoric ambient pads melodies in world music style, evoking a rooftop at blue hour, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "energetic abandoned building", "rich": "Trumpet focused folk piece, sad and introspective, evoking wind falling on a city at night at dusk", "domain": "music moods"}
{"terse": "epic sunshine", "rich": "Organ focused jazz piece, chaotic and introspective, evoking fog falling on a cozy room at late night", "domain": "music moods"}
{"terse": "happy cloudy", "rich": "Synthesizer focused world music piece, dark and introspective, evoking sunrise falling on a underground club at blue hour", "domain": "music moods"}
{"terse": "mysterious ocean", "rich": "Acoustic soundscape with saxophone, cold and contemplative, inspired by sunshine in a train station at twilight", "domain": "music moods"}
{"terse": "clear sky morning", "rich": "A playful industrial composition featuring bass, set in a rooftop during twilight, with fog creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "haunting lightning", "rich": "A intense journey through synthesizer-driven minimalist music, like walking through a ocean during cloudy at morning", "domain": "music moods"}
{"terse": "bass sunshine", "rich": "A whimsical meditation in folk form, guitar leading through wind-kissed ocean at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "harp cathedral", "rich": "A gritty acoustic composition featuring harp, set in a abandoned building during dusk, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "peaceful blues", "rich": "Folk soundscape with trumpet, serene and contemplative, inspired by sunrise in a underground club at dusk", "domain": "music moods"}
{"terse": "haunting abandoned building", "rich": "Synthesizer focused lo-fi piece, happy and introspective, evoking fog falling on a city at night at dawn", "domain": "music moods"}
{"terse": "intense trumpet", "rich": "Romantic organ melodies in ambient style, evoking a abandoned building at twilight, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "minimalist desert", "rich": "Passionate choir melodies in electronic style, evoking a forest at afternoon, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "electronic lightning", "rich": "Atmospheric blues piece with piano, capturing the feeling of snow in a ocean, chaotic undertones, late night ambiance", "domain": "music moods"}
{"terse": "orchestral space station", "rich": "Atmospheric jazz piece with piano, capturing the feeling of snow in a abandoned building, sad undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "orchestral storm", "rich": "Synthesizer focused electronic piece, hopeful and introspective, evoking clear sky falling on a cathedral at dawn", "domain": "music moods"}
{"terse": "rain midnight", "rich": "A serene journey through harp-driven jazz music, like walking through a concert hall during cloudy at morning", "domain": "music moods"}
{"terse": "ambient underground club", "rich": "Electronic soundscape with bass, cold and contemplative, inspired by lightning in a ocean at blue hour", "domain": "music moods"}
{"terse": "strings rain", "rich": "World music soundscape with synthesizer, dreamy and contemplative, inspired by wind in a cathedral at golden hour", "domain": "music moods"}
{"terse": "minimalist forest", "rich": "A calm meditation in jazz form, choir leading through clear sky-kissed cathedral at midnight, deeply atmospheric", "domain": "music moods"}
{"terse": "nostalgic lightning", "rich": "A cold journey through violin-driven cinematic music, like walking through a underground club during clear sky at blue hour", "domain": "music moods"}
{"terse": "passionate midnight", "rich": "A whimsical acoustic composition featuring strings, set in a train station during blue hour, with snow creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "desolate cozy room", "rich": "A happy journey through trumpet-driven minimalist music, like walking through a cozy room during sunset at late night", "domain": "music moods"}
{"terse": "flute cathedral", "rich": "A melancholic journey through electronic beats-driven experimental music, like walking through a underground club during sunshine at midnight", "domain": "music moods"}
{"terse": "calm classical", "rich": "Synthesizer focused minimalist piece, peaceful and introspective, evoking thunder falling on a cozy room at twilight", "domain": "music moods"}
{"terse": "bass forest", "rich": "Piano focused rock piece, desolate and introspective, evoking sunshine falling on a desert at dusk", "domain": "music moods"}
{"terse": "lightning twilight", "rich": "A calm journey through piano-driven industrial music, like walking through a abandoned building during wind at midnight", "domain": "music moods"}
{"terse": "choir sunrise", "rich": "A sad meditation in classical form, saxophone leading through wind-kissed underground club at dawn, deeply atmospheric", "domain": "music moods"}
{"terse": "ambient pads sunshine", "rich": "Emotionally sad minimalist arrangement featuring cello, painting a scene of rain over a desert during late night", "domain": "music moods"}
{"terse": "world music snow", "rich": "Drums focused orchestral piece, mysterious and introspective, evoking sunshine falling on a forest at dusk", "domain": "music moods"}
{"terse": "sunset twilight", "rich": "A dark journey through strings-driven experimental music, like walking through a concert hall during thunder at dusk", "domain": "music moods"}
{"terse": "sunset late night", "rich": "A nostalgic journey through choir-driven ambient music, like walking through a mountains during sunshine at late night", "domain": "music moods"}
{"terse": "happy flute", "rich": "Chaotic ambient pads melodies in classical style, evoking a city at night at twilight, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "triumphant morning", "rich": "Atmospheric blues piece with cello, capturing the feeling of rain in a abandoned building, cold undertones, golden hour ambiance", "domain": "music moods"}
{"terse": "passionate electronic beats", "rich": "A gritty journey through ambient pads-driven classical music, like walking through a concert hall during lightning at dusk", "domain": "music moods"}
{"terse": "ethereal concert hall", "rich": "Atmospheric minimalist piece with strings, capturing the feeling of lightning in a space station, melancholic undertones, blue hour ambiance", "domain": "music moods"}
{"terse": "cold concert hall", "rich": "A euphoric journey through flute-driven rock music, like walking through a abandoned building during sunset at blue hour", "domain": "music moods"}
{"terse": "trumpet cloudy", "rich": "A dark meditation in cinematic form, saxophone leading through snow-kissed mountains at late night, deeply atmospheric", "domain": "music moods"}
{"terse": "dreamy fog", "rich": "Orchestral soundscape with guitar, aggressive and contemplative, inspired by rain in a mountains at late night", "domain": "music moods"}
{"terse": "ethereal strings", "rich": "A happy meditation in rock form, synthesizer leading through sunset-kissed desert at golden hour, deeply atmospheric", "domain": "music moods"}
{"terse": "gritty desert", "rich": "Triumphant synthesizer melodies in minimalist style, evoking a cathedral at twilight, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "lo-fi snow", "rich": "A happy experimental composition featuring bass, set in a desert during twilight, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "minimalist concert hall", "rich": "Emotionally desolate world music arrangement featuring organ, painting a scene of cloudy over a city at night during dawn", "domain": "music moods"}
{"terse": "minimalist cathedral", "rich": "Emotionally intense lo-fi arrangement featuring saxophone, painting a scene of sunshine over a desert during golden hour", "domain": "music moods"}
{"terse": "folk mountains", "rich": "Emotionally happy experimental arrangement featuring guitar, painting a scene of clear sky over a ocean during afternoon", "domain": "music moods"}
{"terse": "jazz rooftop", "rich": "Jazz soundscape with saxophone, peaceful and contemplative, inspired by fog in a concert hall at golden hour", "domain": "music moods"}
{"terse": "romantic wind", "rich": "Somber electronic beats melodies in orchestral style, evoking a train station at afternoon, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "flute underground club", "rich": "Atmospheric acoustic piece with piano, capturing the feeling of snow in a city at night, dreamy undertones, blue hour ambiance", "domain": "music moods"}
{"terse": "serene snow", "rich": "A sad journey through guitar-driven orchestral music, like walking through a underground club during clear sky at golden hour", "domain": "music moods"}
{"terse": "cold minimalist", "rich": "Euphoric electronic beats melodies in lo-fi style, evoking a space station at dusk, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "intense sunset", "rich": "A sad journey through drums-driven world music music, like walking through a cozy room during rain at blue hour", "domain": "music moods"}
{"terse": "sad drums", "rich": "Atmospheric orchestral piece with guitar, capturing the feeling of clear sky in a forest, sad undertones, dusk ambiance", "domain": "music moods"}
{"terse": "epic cathedral", "rich": "A romantic meditation in classical form, guitar leading through lightning-kissed ocean at golden hour, deeply atmospheric", "domain": "music moods"}
{"terse": "sad train station", "rich": "Atmospheric electronic piece with saxophone, capturing the feeling of sunshine in a train station, serene undertones, late night ambiance", "domain": "music moods"}
{"terse": "cold midnight", "rich": "Strings focused neoclassical piece, cold and introspective, evoking snow falling on a abandoned building at morning", "domain": "music moods"}
{"terse": "nostalgic thunder", "rich": "Electronic soundscape with piano, somber and contemplative, inspired by sunshine in a concert hall at twilight", "domain": "music moods"}
{"terse": "serene violin", "rich": "A mysterious jazz composition featuring organ, set in a rooftop during late night, with snow creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "peaceful rooftop", "rich": "Atmospheric industrial piece with organ, capturing the feeling of lightning in a train station, chaotic undertones, twilight ambiance", "domain": "music moods"}
{"terse": "choir space station", "rich": "Emotionally somber industrial arrangement featuring piano, painting a scene of thunder over a cathedral during twilight", "domain": "music moods"}
{"terse": "classical wind", "rich": "Ambient soundscape with bass, energetic and contemplative, inspired by sunshine in a desert at late night", "domain": "music moods"}
{"terse": "somber city at night", "rich": "Atmospheric minimalist piece with organ, capturing the feeling of lightning in a city at night, playful undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "electronic lightning", "rich": "Neoclassical soundscape with drums, hopeful and contemplative, inspired by sunset in a space station at dawn", "domain": "music moods"}
{"terse": "cello abandoned building", "rich": "World music soundscape with drums, epic and contemplative, inspired by lightning in a city at night at afternoon", "domain": "music moods"}
{"terse": "aggressive neoclassical", "rich": "Atmospheric minimalist piece with electronic beats, capturing the feeling of sunshine in a cathedral, chaotic undertones, late night ambiance", "domain": "music moods"}
{"terse": "trumpet cloudy", "rich": "A gritty experimental composition featuring choir, set in a train station during dusk, with wind creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "aggressive dawn", "rich": "Intense piano melodies in neoclassical style, evoking a cathedral at dawn, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "sad sunshine", "rich": "World music soundscape with saxophone, euphoric and contemplative, inspired by thunder in a concert hall at blue hour", "domain": "music moods"}
{"terse": "sad golden hour", "rich": "A cold cinematic composition featuring flute, set in a train station during late night, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "happy saxophone", "rich": "A serene meditation in ambient form, cello leading through thunder-kissed cozy room at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "folk underground club", "rich": "Blues soundscape with piano, intense and contemplative, inspired by sunset in a underground club at morning", "domain": "music moods"}
{"terse": "happy morning", "rich": "Atmospheric ambient piece with violin, capturing the feeling of rain in a forest, mysterious undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "romantic morning", "rich": "A chaotic journey through ambient pads-driven orchestral music, like walking through a desert during sunshine at dusk", "domain": "music moods"}
{"terse": "violin storm", "rich": "A dreamy orchestral composition featuring violin, set in a mountains during morning, with wind creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "energetic harp", "rich": "A somber experimental composition featuring piano, set in a forest during afternoon, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "orchestral cloudy", "rich": "A uplifting meditation in electronic form, flute leading through cloudy-kissed ocean at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "somber drums", "rich": "Electronic soundscape with piano, energetic and contemplative, inspired by sunshine in a ocean at dawn", "domain": "music moods"}
{"terse": "bass forest", "rich": "A uplifting meditation in ambient form, trumpet leading through thunder-kissed mountains at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "chaotic synthesizer", "rich": "Emotionally mysterious neoclassical arrangement featuring cello, painting a scene of storm over a cathedral during blue hour", "domain": "music moods"}
{"terse": "violin underground club", "rich": "A melancholic meditation in classical form, violin leading through clear sky-kissed mountains at blue hour, deeply atmospheric", "domain": "music moods"}
{"terse": "lo-fi train station", "rich": "Somber strings melodies in electronic style, evoking a abandoned building at twilight, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "sad jazz", "rich": "A calm electronic composition featuring strings, set in a cozy room during morning, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "sunshine golden hour", "rich": "Atmospheric blues piece with harp, capturing the feeling of lightning in a forest, cold undertones, blue hour ambiance", "domain": "music moods"}
{"terse": "piano space station", "rich": "A cold orchestral composition featuring saxophone, set in a ocean during dusk, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "desolate mountains", "rich": "A epic industrial composition featuring piano, set in a city at night during blue hour, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "clear sky late night", "rich": "Lo-fi soundscape with electronic beats, passionate and contemplative, inspired by sunshine in a cozy room at midnight", "domain": "music moods"}
{"terse": "hopeful afternoon", "rich": "A peaceful orchestral composition featuring flute, set in a train station during dawn, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "nostalgic acoustic", "rich": "Atmospheric experimental piece with strings, capturing the feeling of sunset in a forest, haunting undertones, twilight ambiance", "domain": "music moods"}
{"terse": "energetic cozy room", "rich": "Ambient soundscape with cello, euphoric and contemplative, inspired by sunshine in a underground club at dawn", "domain": "music moods"}
{"terse": "passionate sunrise", "rich": "Atmospheric industrial piece with piano, capturing the feeling of lightning in a mountains, uplifting undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "serene rain", "rich": "A desolate experimental composition featuring harp, set in a city at night during golden hour, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "energetic blue hour", "rich": "Electronic soundscape with electronic beats, contemplative and contemplative, inspired by rain in a cozy room at dusk", "domain": "music moods"}
{"terse": "energetic cinematic", "rich": "Atmospheric minimalist piece with strings, capturing the feeling of cloudy in a cathedral, sad undertones, morning ambiance", "domain": "music moods"}
{"terse": "dreamy minimalist", "rich": "Lo-fi soundscape with synthesizer, whimsical and contemplative, inspired by fog in a mountains at morning", "domain": "music moods"}
{"terse": "orchestral thunder", "rich": "A chaotic meditation in blues form, piano leading through sunset-kissed rooftop at golden hour, deeply atmospheric", "domain": "music moods"}
{"terse": "harp mountains", "rich": "Piano focused cinematic piece, calm and introspective, evoking sunrise falling on a desert at afternoon", "domain": "music moods"}
{"terse": "minimalist clear sky", "rich": "A haunting journey through strings-driven lo-fi music, like walking through a desert during sunshine at twilight", "domain": "music moods"}
{"terse": "epic morning", "rich": "A sad electronic composition featuring harp, set in a desert during dawn, with snow creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "flute space station", "rich": "A playful journey through harp-driven minimalist music, like walking through a abandoned building during wind at dusk", "domain": "music moods"}
{"terse": "serene midnight", "rich": "Atmospheric orchestral piece with flute, capturing the feeling of clear sky in a city at night, epic undertones, midnight ambiance", "domain": "music moods"}
{"terse": "strings city at night", "rich": "Epic ambient pads melodies in ambient style, evoking a rooftop at twilight, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "blues concert hall", "rich": "Cello focused ambient piece, intense and introspective, evoking sunset falling on a cathedral at twilight", "domain": "music moods"}
{"terse": "ambient rain", "rich": "Calm harp melodies in acoustic style, evoking a abandoned building at morning, thunder outside, intimate and reflective", "domain": "music moods"}
{"terse": "cinematic concert hall", "rich": "A mysterious meditation in world music form, synthesizer leading through rain-kissed train station at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "guitar lightning", "rich": "Lo-fi soundscape with bass, playful and contemplative, inspired by cloudy in a concert hall at golden hour", "domain": "music moods"}
{"terse": "ethereal cathedral", "rich": "Atmospheric orchestral piece with electronic beats, capturing the feeling of sunshine in a forest, playful undertones, late night ambiance", "domain": "music moods"}
{"terse": "thunder dusk", "rich": "Playful harp melodies in cinematic style, evoking a abandoned building at dawn, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "triumphant concert hall", "rich": "A serene world music composition featuring synthesizer, set in a mountains during morning, with fog creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "orchestral thunder", "rich": "Neoclassical soundscape with ambient pads, intense and contemplative, inspired by fog in a cathedral at midnight", "domain": "music moods"}
{"terse": "trumpet sunset", "rich": "A contemplative journey through piano-driven rock music, like walking through a forest during clear sky at dusk", "domain": "music moods"}
{"terse": "passionate rain", "rich": "A epic meditation in orchestral form, harp leading through sunrise-kissed cathedral at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "nostalgic sunrise", "rich": "Atmospheric folk piece with organ, capturing the feeling of clear sky in a desert, epic undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "sunrise afternoon", "rich": "Emotionally peaceful jazz arrangement featuring flute, painting a scene of clear sky over a cathedral during morning", "domain": "music moods"}
{"terse": "organ forest", "rich": "Ambient pads focused electronic piece, calm and introspective, evoking sunrise falling on a train station at afternoon", "domain": "music moods"}
{"terse": "ambient space station", "rich": "A aggressive orchestral composition featuring piano, set in a concert hall during dawn, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "nostalgic thunder", "rich": "Peaceful drums melodies in blues style, evoking a desert at dusk, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "ethereal rock", "rich": "Somber organ melodies in folk style, evoking a forest at morning, snow outside, intimate and reflective", "domain": "music moods"}
{"terse": "energetic minimalist", "rich": "Harp focused minimalist piece, uplifting and introspective, evoking lightning falling on a concert hall at dusk", "domain": "music moods"}
{"terse": "mysterious trumpet", "rich": "Atmospheric electronic piece with organ, capturing the feeling of cloudy in a abandoned building, triumphant undertones, midnight ambiance", "domain": "music moods"}
{"terse": "choir thunder", "rich": "A melancholic meditation in lo-fi form, electronic beats leading through rain-kissed underground club at morning, deeply atmospheric", "domain": "music moods"}
{"terse": "contemplative rain", "rich": "Bass focused blues piece, mysterious and introspective, evoking lightning falling on a city at night at morning", "domain": "music moods"}
{"terse": "neoclassical wind", "rich": "A contemplative journey through ambient pads-driven acoustic music, like walking through a city at night during sunset at twilight", "domain": "music moods"}
{"terse": "snow midnight", "rich": "Rock soundscape with ambient pads, hopeful and contemplative, inspired by clear sky in a train station at midnight", "domain": "music moods"}
{"terse": "playful strings", "rich": "Lo-fi soundscape with piano, desolate and contemplative, inspired by sunset in a desert at dusk", "domain": "music moods"}
{"terse": "sunrise afternoon", "rich": "Atmospheric world music piece with flute, capturing the feeling of rain in a mountains, happy undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "happy blue hour", "rich": "World music soundscape with choir, somber and contemplative, inspired by snow in a forest at dawn", "domain": "music moods"}
{"terse": "world music forest", "rich": "Emotionally whimsical blues arrangement featuring violin, painting a scene of sunrise over a desert during afternoon", "domain": "music moods"}
{"terse": "somber golden hour", "rich": "Emotionally uplifting folk arrangement featuring drums, painting a scene of rain over a concert hall during golden hour", "domain": "music moods"}
{"terse": "storm afternoon", "rich": "A contemplative journey through electronic beats-driven classical music, like walking through a rooftop during rain at morning", "domain": "music moods"}
{"terse": "dreamy organ", "rich": "Neoclassical soundscape with electronic beats, whimsical and contemplative, inspired by sunrise in a concert hall at twilight", "domain": "music moods"}
{"terse": "whimsical golden hour", "rich": "A dark meditation in neoclassical form, choir leading through sunset-kissed mountains at midnight, deeply atmospheric", "domain": "music moods"}
{"terse": "organ sunshine", "rich": "Ambient soundscape with synthesizer, serene and contemplative, inspired by cloudy in a underground club at morning", "domain": "music moods"}
{"terse": "energetic sunrise", "rich": "A mysterious journey through synthesizer-driven orchestral music, like walking through a forest during sunset at golden hour", "domain": "music moods"}
{"terse": "world music desert", "rich": "World music soundscape with trumpet, intense and contemplative, inspired by wind in a mountains at late night", "domain": "music moods"}
{"terse": "world music clear sky", "rich": "Atmospheric minimalist piece with strings, capturing the feeling of sunrise in a city at night, calm undertones, dawn ambiance", "domain": "music moods"}
{"terse": "sunshine dusk", "rich": "A peaceful journey through harp-driven experimental music, like walking through a space station during sunset at midnight", "domain": "music moods"}
{"terse": "saxophone fog", "rich": "A dreamy journey through flute-driven electronic music, like walking through a desert during snow at blue hour", "domain": "music moods"}
{"terse": "drums storm", "rich": "A happy journey through choir-driven folk music, like walking through a train station during wind at blue hour", "domain": "music moods"}
{"terse": "violin lightning", "rich": "Somber organ melodies in world music style, evoking a desert at late night, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "thunder twilight", "rich": "A chaotic meditation in minimalist form, harp leading through thunder-kissed mountains at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "serene folk", "rich": "A gritty journey through piano-driven ambient music, like walking through a concert hall during cloudy at golden hour", "domain": "music moods"}
{"terse": "nostalgic classical", "rich": "A desolate journey through synthesizer-driven rock music, like walking through a desert during sunshine at dawn", "domain": "music moods"}
{"terse": "cinematic city at night", "rich": "Cinematic soundscape with cello, sad and contemplative, inspired by wind in a cathedral at midnight", "domain": "music moods"}
{"terse": "industrial fog", "rich": "Orchestral soundscape with drums, happy and contemplative, inspired by sunrise in a train station at dusk", "domain": "music moods"}
{"terse": "romantic cathedral", "rich": "Emotionally peaceful experimental arrangement featuring trumpet, painting a scene of rain over a space station during dusk", "domain": "music moods"}
{"terse": "energetic strings", "rich": "Ambient soundscape with choir, nostalgic and contemplative, inspired by thunder in a ocean at late night", "domain": "music moods"}
{"terse": "cold dawn", "rich": "A playful meditation in ambient form, trumpet leading through snow-kissed underground club at dawn, deeply atmospheric", "domain": "music moods"}
{"terse": "harp sunset", "rich": "A hopeful blues composition featuring guitar, set in a ocean during dusk, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "serene midnight", "rich": "A haunting journey through synthesizer-driven minimalist music, like walking through a cozy room during storm at dusk", "domain": "music moods"}
{"terse": "melancholic experimental", "rich": "Haunting trumpet melodies in world music style, evoking a space station at dawn, sunset outside, intimate and reflective", "domain": "music moods"}
{"terse": "peaceful forest", "rich": "Atmospheric blues piece with cello, capturing the feeling of snow in a rooftop, peaceful undertones, blue hour ambiance", "domain": "music moods"}
{"terse": "strings rooftop", "rich": "A gritty journey through choir-driven lo-fi music, like walking through a cathedral during lightning at blue hour", "domain": "music moods"}
{"terse": "blues snow", "rich": "Atmospheric classical piece with choir, capturing the feeling of rain in a cozy room, happy undertones, dusk ambiance", "domain": "music moods"}
{"terse": "contemplative clear sky", "rich": "Atmospheric minimalist piece with organ, capturing the feeling of wind in a cathedral, contemplative undertones, midnight ambiance", "domain": "music moods"}
{"terse": "nostalgic wind", "rich": "Strings focused orchestral piece, dark and introspective, evoking cloudy falling on a underground club at twilight", "domain": "music moods"}
{"terse": "somber industrial", "rich": "A intense experimental composition featuring piano, set in a cozy room during dawn, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "harp underground club", "rich": "Drums focused neoclassical piece, dark and introspective, evoking lightning falling on a train station at midnight", "domain": "music moods"}
{"terse": "haunting bass", "rich": "A euphoric journey through choir-driven minimalist music, like walking through a city at night during sunset at dawn", "domain": "music moods"}
{"terse": "aggressive ambient", "rich": "A romantic rock composition featuring trumpet, set in a underground club during dawn, with wind creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "industrial fog", "rich": "Cello focused folk piece, sad and introspective, evoking wind falling on a underground club at afternoon", "domain": "music moods"}
{"terse": "acoustic fog", "rich": "Atmospheric minimalist piece with saxophone, capturing the feeling of sunset in a underground club, aggressive undertones, golden hour ambiance", "domain": "music moods"}
{"terse": "whimsical strings", "rich": "Epic strings melodies in cinematic style, evoking a abandoned building at golden hour, storm outside, intimate and reflective", "domain": "music moods"}
{"terse": "jazz ocean", "rich": "Energetic organ melodies in electronic style, evoking a rooftop at twilight, lightning outside, intimate and reflective", "domain": "music moods"}
{"terse": "electronic beats mountains", "rich": "A desolate meditation in ambient form, piano leading through rain-kissed rooftop at dawn, deeply atmospheric", "domain": "music moods"}
{"terse": "guitar concert hall", "rich": "Synthesizer focused classical piece, passionate and introspective, evoking storm falling on a mountains at dusk", "domain": "music moods"}
{"terse": "choir cathedral", "rich": "A whimsical meditation in neoclassical form, violin leading through snow-kissed underground club at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "playful blue hour", "rich": "Cinematic soundscape with trumpet, sad and contemplative, inspired by storm in a cathedral at morning", "domain": "music moods"}
{"terse": "ambient thunder", "rich": "A melancholic journey through flute-driven rock music, like walking through a forest during clear sky at dusk", "domain": "music moods"}
{"terse": "ambient pads cathedral", "rich": "Emotionally whimsical rock arrangement featuring bass, painting a scene of storm over a mountains during dawn", "domain": "music moods"}
{"terse": "thunder midnight", "rich": "Industrial soundscape with cello, euphoric and contemplative, inspired by snow in a forest at late night", "domain": "music moods"}
{"terse": "calm experimental", "rich": "Violin focused blues piece, haunting and introspective, evoking fog falling on a forest at golden hour", "domain": "music moods"}
{"terse": "calm cathedral", "rich": "A chaotic jazz composition featuring strings, set in a ocean during dawn, with fog creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "dreamy sunshine", "rich": "Emotionally nostalgic neoclassical arrangement featuring synthesizer, painting a scene of thunder over a abandoned building during morning", "domain": "music moods"}
{"terse": "rain twilight", "rich": "A happy classical composition featuring guitar, set in a forest during late night, with clear sky creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "energetic golden hour", "rich": "A desolate folk composition featuring harp, set in a concert hall during morning, with snow creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "sunrise late night", "rich": "Blues soundscape with ambient pads, nostalgic and contemplative, inspired by sunshine in a mountains at morning", "domain": "music moods"}
{"terse": "intense synthesizer", "rich": "Atmospheric lo-fi piece with ambient pads, capturing the feeling of sunset in a city at night, contemplative undertones, late night ambiance", "domain": "music moods"}
{"terse": "mysterious acoustic", "rich": "Passionate harp melodies in jazz style, evoking a desert at blue hour, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "sunset dawn", "rich": "Atmospheric folk piece with violin, capturing the feeling of sunrise in a ocean, passionate undertones, late night ambiance", "domain": "music moods"}
{"terse": "cello storm", "rich": "A epic neoclassical composition featuring cello, set in a city at night during golden hour, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "somber rain", "rich": "Peaceful strings melodies in experimental style, evoking a abandoned building at late night, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "intense wind", "rich": "A uplifting journey through harp-driven industrial music, like walking through a mountains during sunrise at morning", "domain": "music moods"}
{"terse": "blues cathedral", "rich": "Cold guitar melodies in folk style, evoking a rooftop at blue hour, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "mysterious ocean", "rich": "World music soundscape with choir, romantic and contemplative, inspired by sunrise in a cozy room at late night", "domain": "music moods"}
{"terse": "happy synthesizer", "rich": "A triumphant meditation in acoustic form, trumpet leading through clear sky-kissed rooftop at midnight, deeply atmospheric", "domain": "music moods"}
{"terse": "triumphant dusk", "rich": "Atmospheric world music piece with trumpet, capturing the feeling of sunshine in a abandoned building, euphoric undertones, golden hour ambiance", "domain": "music moods"}
{"terse": "sad folk", "rich": "Minimalist soundscape with bass, aggressive and contemplative, inspired by rain in a ocean at midnight", "domain": "music moods"}
{"terse": "cold morning", "rich": "Atmospheric ambient piece with synthesizer, capturing the feeling of sunset in a desert, gritty undertones, twilight ambiance", "domain": "music moods"}
{"terse": "industrial snow", "rich": "Passionate electronic beats melodies in blues style, evoking a space station at twilight, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "folk forest", "rich": "A epic journey through organ-driven orchestral music, like walking through a cathedral during snow at dusk", "domain": "music moods"}
{"terse": "desolate fog", "rich": "Rock soundscape with cello, hopeful and contemplative, inspired by cloudy in a forest at golden hour", "domain": "music moods"}
{"terse": "experimental rain", "rich": "Atmospheric ambient piece with bass, capturing the feeling of fog in a cozy room, desolate undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "lo-fi underground club", "rich": "A triumphant meditation in folk form, organ leading through clear sky-kissed cozy room at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "strings lightning", "rich": "Atmospheric experimental piece with bass, capturing the feeling of cloudy in a forest, cold undertones, midnight ambiance", "domain": "music moods"}
{"terse": "serene abandoned building", "rich": "A hopeful experimental composition featuring drums, set in a concert hall during twilight, with wind creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "snow dusk", "rich": "A energetic jazz composition featuring guitar, set in a cozy room during twilight, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "energetic city at night", "rich": "Emotionally nostalgic classical arrangement featuring bass, painting a scene of storm over a forest during golden hour", "domain": "music moods"}
{"terse": "triumphant organ", "rich": "A peaceful journey through harp-driven folk music, like walking through a underground club during thunder at afternoon", "domain": "music moods"}
{"terse": "experimental wind", "rich": "Folk soundscape with synthesizer, hopeful and contemplative, inspired by clear sky in a ocean at dusk", "domain": "music moods"}
{"terse": "aggressive forest", "rich": "A uplifting world music composition featuring choir, set in a desert during dusk, with sunrise creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "happy synthesizer", "rich": "Atmospheric neoclassical piece with strings, capturing the feeling of snow in a rooftop, melancholic undertones, afternoon ambiance", "domain": "music moods"}
{"terse": "passionate electronic beats", "rich": "A contemplative journey through piano-driven acoustic music, like walking through a abandoned building during snow at morning", "domain": "music moods"}
{"terse": "euphoric fog", "rich": "A whimsical meditation in rock form, organ leading through fog-kissed forest at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "storm morning", "rich": "Emotionally aggressive cinematic arrangement featuring flute, painting a scene of sunshine over a city at night during dusk", "domain": "music moods"}
{"terse": "haunting blue hour", "rich": "Emotionally intense neoclassical arrangement featuring flute, painting a scene of sunset over a cathedral during midnight", "domain": "music moods"}
{"terse": "serene trumpet", "rich": "Contemplative piano melodies in industrial style, evoking a rooftop at morning, sunset outside, intimate and reflective", "domain": "music moods"}
{"terse": "drums ocean", "rich": "A epic meditation in orchestral form, choir leading through sunset-kissed city at night at late night, deeply atmospheric", "domain": "music moods"}
{"terse": "whimsical folk", "rich": "Violin focused world music piece, playful and introspective, evoking wind falling on a ocean at blue hour", "domain": "music moods"}
{"terse": "fog late night", "rich": "Flute focused industrial piece, mysterious and introspective, evoking lightning falling on a city at night at afternoon", "domain": "music moods"}
{"terse": "serene underground club", "rich": "A playful journey through flute-driven world music music, like walking through a forest during thunder at midnight", "domain": "music moods"}
{"terse": "cold minimalist", "rich": "Violin focused lo-fi piece, serene and introspective, evoking storm falling on a forest at midnight", "domain": "music moods"}
{"terse": "ambient concert hall", "rich": "Piano focused neoclassical piece, whimsical and introspective, evoking fog falling on a desert at morning", "domain": "music moods"}
{"terse": "harp mountains", "rich": "Atmospheric minimalist piece with synthesizer, capturing the feeling of cloudy in a space station, nostalgic undertones, twilight ambiance", "domain": "music moods"}
{"terse": "peaceful snow", "rich": "A nostalgic meditation in rock form, choir leading through cloudy-kissed ocean at golden hour, deeply atmospheric", "domain": "music moods"}
{"terse": "nostalgic rooftop", "rich": "A desolate industrial composition featuring violin, set in a concert hall during late night, with storm creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "mysterious dusk", "rich": "A happy orchestral composition featuring flute, set in a underground club during morning, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "euphoric golden hour", "rich": "Choir focused minimalist piece, ethereal and introspective, evoking storm falling on a cathedral at morning", "domain": "music moods"}
{"terse": "gritty clear sky", "rich": "A serene acoustic composition featuring synthesizer, set in a rooftop during twilight, with snow creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "violin sunshine", "rich": "Classical soundscape with trumpet, passionate and contemplative, inspired by sunset in a city at night at twilight", "domain": "music moods"}
{"terse": "guitar fog", "rich": "A dark journey through trumpet-driven world music music, like walking through a ocean during snow at morning", "domain": "music moods"}
{"terse": "epic fog", "rich": "Emotionally haunting experimental arrangement featuring flute, painting a scene of storm over a forest during dawn", "domain": "music moods"}
{"terse": "experimental sunshine", "rich": "A whimsical journey through violin-driven rock music, like walking through a desert during wind at blue hour", "domain": "music moods"}
{"terse": "bass wind", "rich": "Sad harp melodies in blues style, evoking a forest at dusk, thunder outside, intimate and reflective", "domain": "music moods"}
{"terse": "energetic violin", "rich": "Saxophone focused neoclassical piece, hopeful and introspective, evoking sunshine falling on a ocean at twilight", "domain": "music moods"}
{"terse": "storm dawn", "rich": "Atmospheric ambient piece with strings, capturing the feeling of sunshine in a forest, ethereal undertones, twilight ambiance", "domain": "music moods"}
{"terse": "whimsical minimalist", "rich": "A haunting meditation in experimental form, cello leading through thunder-kissed abandoned building at golden hour, deeply atmospheric", "domain": "music moods"}
{"terse": "violin sunrise", "rich": "A passionate journey through guitar-driven neoclassical music, like walking through a abandoned building during wind at twilight", "domain": "music moods"}
{"terse": "playful late night", "rich": "A dark meditation in orchestral form, harp leading through lightning-kissed ocean at dusk, deeply atmospheric", "domain": "music moods"}
{"terse": "strings sunrise", "rich": "Atmospheric industrial piece with synthesizer, capturing the feeling of storm in a underground club, contemplative undertones, midnight ambiance", "domain": "music moods"}
{"terse": "triumphant experimental", "rich": "Industrial soundscape with trumpet, aggressive and contemplative, inspired by storm in a rooftop at midnight", "domain": "music moods"}
{"terse": "dreamy violin", "rich": "A peaceful lo-fi composition featuring piano, set in a mountains during dusk, with fog creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "serene cloudy", "rich": "Organ focused neoclassical piece, intense and introspective, evoking rain falling on a desert at golden hour", "domain": "music moods"}
{"terse": "somber cozy room", "rich": "A cold jazz composition featuring harp, set in a city at night during twilight, with fog creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "cold minimalist", "rich": "A haunting meditation in jazz form, strings leading through sunrise-kissed forest at afternoon, deeply atmospheric", "domain": "music moods"}
{"terse": "somber classical", "rich": "Uplifting strings melodies in ambient style, evoking a rooftop at afternoon, storm outside, intimate and reflective", "domain": "music moods"}
{"terse": "melancholic underground club", "rich": "A euphoric journey through drums-driven industrial music, like walking through a rooftop during fog at afternoon", "domain": "music moods"}
{"terse": "trumpet cozy room", "rich": "Aggressive bass melodies in lo-fi style, evoking a desert at golden hour, wind outside, intimate and reflective", "domain": "music moods"}
{"terse": "violin forest", "rich": "Folk soundscape with saxophone, calm and contemplative, inspired by sunrise in a space station at afternoon", "domain": "music moods"}
{"terse": "mysterious train station", "rich": "A cold neoclassical composition featuring trumpet, set in a city at night during midnight, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "lo-fi forest", "rich": "Triumphant bass melodies in lo-fi style, evoking a cathedral at morning, lightning outside, intimate and reflective", "domain": "music moods"}
{"terse": "triumphant acoustic", "rich": "A mysterious folk composition featuring flute, set in a space station during golden hour, with cloudy creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "ambient pads wind", "rich": "A passionate electronic composition featuring electronic beats, set in a ocean during blue hour, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "neoclassical city at night", "rich": "A dark journey through violin-driven folk music, like walking through a mountains during fog at dusk", "domain": "music moods"}
{"terse": "gritty acoustic", "rich": "Minimalist soundscape with choir, dark and contemplative, inspired by rain in a cathedral at morning", "domain": "music moods"}
{"terse": "world music rooftop", "rich": "Blues soundscape with guitar, gritty and contemplative, inspired by sunset in a city at night at blue hour", "domain": "music moods"}
{"terse": "triumphant organ", "rich": "Piano focused industrial piece, ethereal and introspective, evoking wind falling on a city at night at twilight", "domain": "music moods"}
{"terse": "epic dusk", "rich": "Atmospheric folk piece with cello, capturing the feeling of thunder in a forest, euphoric undertones, dusk ambiance", "domain": "music moods"}
{"terse": "world music train station", "rich": "Intense strings melodies in rock style, evoking a underground club at dawn, lightning outside, intimate and reflective", "domain": "music moods"}
{"terse": "ethereal rain", "rich": "A epic journey through harp-driven ambient music, like walking through a desert during sunset at dawn", "domain": "music moods"}
{"terse": "cinematic cloudy", "rich": "A ethereal folk composition featuring synthesizer, set in a ocean during afternoon, with clear sky creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "triumphant ocean", "rich": "Somber organ melodies in folk style, evoking a mountains at late night, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "folk wind", "rich": "Experimental soundscape with bass, happy and contemplative, inspired by snow in a cozy room at afternoon", "domain": "music moods"}
{"terse": "rock abandoned building", "rich": "A gritty meditation in neoclassical form, violin leading through snow-kissed space station at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "dreamy violin", "rich": "Emotionally passionate classical arrangement featuring guitar, painting a scene of clear sky over a underground club during golden hour", "domain": "music moods"}
{"terse": "piano cozy room", "rich": "A passionate world music composition featuring cello, set in a ocean during golden hour, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "world music city at night", "rich": "Folk soundscape with saxophone, epic and contemplative, inspired by storm in a ocean at twilight", "domain": "music moods"}
{"terse": "somber sunset", "rich": "Emotionally sad electronic arrangement featuring flute, painting a scene of wind over a forest during golden hour", "domain": "music moods"}
{"terse": "wind morning", "rich": "Calm flute melodies in neoclassical style, evoking a cozy room at twilight, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "wind late night", "rich": "Trumpet focused experimental piece, somber and introspective, evoking fog falling on a cozy room at morning", "domain": "music moods"}
{"terse": "happy clear sky", "rich": "Ambient pads focused orchestral piece, triumphant and introspective, evoking snow falling on a desert at afternoon", "domain": "music moods"}
{"terse": "gritty choir", "rich": "Classical soundscape with electronic beats, dreamy and contemplative, inspired by fog in a cathedral at late night", "domain": "music moods"}
{"terse": "wind dawn", "rich": "Dreamy drums melodies in neoclassical style, evoking a forest at morning, sunrise outside, intimate and reflective", "domain": "music moods"}
{"terse": "cloudy midnight", "rich": "Cinematic soundscape with violin, gritty and contemplative, inspired by storm in a cathedral at golden hour", "domain": "music moods"}
{"terse": "jazz sunshine", "rich": "Dreamy organ melodies in rock style, evoking a underground club at golden hour, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "electronic fog", "rich": "Synthesizer focused orchestral piece, desolate and introspective, evoking clear sky falling on a city at night at golden hour", "domain": "music moods"}
{"terse": "dark classical", "rich": "Atmospheric jazz piece with synthesizer, capturing the feeling of sunrise in a concert hall, whimsical undertones, morning ambiance", "domain": "music moods"}
{"terse": "piano space station", "rich": "Emotionally ethereal electronic arrangement featuring electronic beats, painting a scene of cloudy over a city at night during late night", "domain": "music moods"}
{"terse": "cello sunrise", "rich": "Classical soundscape with strings, serene and contemplative, inspired by clear sky in a desert at golden hour", "domain": "music moods"}
{"terse": "uplifting dusk", "rich": "Sad flute melodies in orchestral style, evoking a cozy room at dawn, lightning outside, intimate and reflective", "domain": "music moods"}
{"terse": "whimsical trumpet", "rich": "Atmospheric world music piece with electronic beats, capturing the feeling of cloudy in a cathedral, energetic undertones, late night ambiance", "domain": "music moods"}
{"terse": "nostalgic rain", "rich": "A dreamy industrial composition featuring electronic beats, set in a city at night during dusk, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "whimsical thunder", "rich": "Peaceful cello melodies in minimalist style, evoking a forest at dusk, fog outside, intimate and reflective", "domain": "music moods"}
{"terse": "wind blue hour", "rich": "Hopeful bass melodies in jazz style, evoking a train station at golden hour, cloudy outside, intimate and reflective", "domain": "music moods"}
{"terse": "guitar abandoned building", "rich": "Atmospheric electronic piece with piano, capturing the feeling of sunshine in a train station, somber undertones, midnight ambiance", "domain": "music moods"}
{"terse": "piano sunset", "rich": "A gritty meditation in industrial form, guitar leading through storm-kissed abandoned building at late night, deeply atmospheric", "domain": "music moods"}
{"terse": "sad desert", "rich": "Trumpet focused minimalist piece, haunting and introspective, evoking thunder falling on a ocean at golden hour", "domain": "music moods"}
{"terse": "fog midnight", "rich": "Uplifting saxophone melodies in neoclassical style, evoking a mountains at dusk, thunder outside, intimate and reflective", "domain": "music moods"}
{"terse": "piano underground club", "rich": "Mysterious electronic beats melodies in ambient style, evoking a space station at golden hour, clear sky outside, intimate and reflective", "domain": "music moods"}
{"terse": "bass snow", "rich": "Atmospheric cinematic piece with guitar, capturing the feeling of fog in a space station, haunting undertones, dawn ambiance", "domain": "music moods"}
{"terse": "dark morning", "rich": "A hopeful meditation in folk form, organ leading through sunset-kissed city at night at morning, deeply atmospheric", "domain": "music moods"}
{"terse": "experimental mountains", "rich": "Atmospheric orchestral piece with strings, capturing the feeling of lightning in a rooftop, aggressive undertones, midnight ambiance", "domain": "music moods"}
{"terse": "cloudy twilight", "rich": "Neoclassical soundscape with trumpet, whimsical and contemplative, inspired by clear sky in a cozy room at dawn", "domain": "music moods"}
{"terse": "rain twilight", "rich": "Uplifting strings melodies in blues style, evoking a city at night at afternoon, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "cloudy midnight", "rich": "Classical soundscape with synthesizer, intense and contemplative, inspired by wind in a desert at golden hour", "domain": "music moods"}
{"terse": "drums underground club", "rich": "Chaotic harp melodies in experimental style, evoking a ocean at late night, sunshine outside, intimate and reflective", "domain": "music moods"}
{"terse": "experimental cloudy", "rich": "Melancholic electronic beats melodies in ambient style, evoking a city at night at midnight, sunset outside, intimate and reflective", "domain": "music moods"}
{"terse": "wind blue hour", "rich": "Contemplative drums melodies in orchestral style, evoking a concert hall at afternoon, sunrise outside, intimate and reflective", "domain": "music moods"}
{"terse": "epic synthesizer", "rich": "Epic flute melodies in world music style, evoking a rooftop at dawn, sunset outside, intimate and reflective", "domain": "music moods"}
{"terse": "mysterious ambient pads", "rich": "Jazz soundscape with synthesizer, sad and contemplative, inspired by sunset in a desert at late night", "domain": "music moods"}
{"terse": "uplifting snow", "rich": "A somber meditation in industrial form, flute leading through clear sky-kissed train station at blue hour, deeply atmospheric", "domain": "music moods"}
{"terse": "sunrise twilight", "rich": "A aggressive ambient composition featuring violin, set in a abandoned building during afternoon, with thunder creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "cello mountains", "rich": "Guitar focused ambient piece, somber and introspective, evoking thunder falling on a forest at late night", "domain": "music moods"}
{"terse": "haunting synthesizer", "rich": "A hopeful classical composition featuring ambient pads, set in a cozy room during dawn, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "blues clear sky", "rich": "A contemplative meditation in lo-fi form, saxophone leading through clear sky-kissed abandoned building at twilight, deeply atmospheric", "domain": "music moods"}
{"terse": "blues storm", "rich": "Harp focused folk piece, haunting and introspective, evoking cloudy falling on a abandoned building at twilight", "domain": "music moods"}
{"terse": "clear sky afternoon", "rich": "Atmospheric lo-fi piece with choir, capturing the feeling of fog in a train station, dark undertones, dawn ambiance", "domain": "music moods"}
{"terse": "fog dusk", "rich": "A cold minimalist composition featuring choir, set in a train station during blue hour, with sunshine creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "ethereal folk", "rich": "A energetic ambient composition featuring violin, set in a ocean during afternoon, with rain creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "world music clear sky", "rich": "Synthesizer focused experimental piece, melancholic and introspective, evoking clear sky falling on a ocean at golden hour", "domain": "music moods"}
{"terse": "dreamy neoclassical", "rich": "A melancholic world music composition featuring violin, set in a concert hall during dawn, with sunset creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "world music snow", "rich": "Emotionally somber jazz arrangement featuring ambient pads, painting a scene of sunshine over a forest during twilight", "domain": "music moods"}
{"terse": "electronic beats lightning", "rich": "Ethereal guitar melodies in industrial style, evoking a space station at blue hour, lightning outside, intimate and reflective", "domain": "music moods"}
{"terse": "dreamy sunrise", "rich": "A triumphant minimalist composition featuring strings, set in a mountains during twilight, with sunrise creating atmosphere, rich emotional depth", "domain": "music moods"}
{"terse": "sad synthesizer", "rich": "Experimental soundscape with ambient pads, triumphant and contemplative, inspired by cloudy in a forest at morning", "domain": "music moods"}
{"terse": "whimsical cinematic", "rich": "A triumphant journey through flute-driven experimental music, like walking through a desert during rain at late night", "domain": "music moods"}
{"terse": "chaotic minimalist", "rich": "Acoustic soundscape with trumpet, melancholic and contemplative, inspired by sunset in a city at night at midnight", "domain": "music moods"}
{"terse": "epic organ", "rich": "Atmospheric lo-fi piece with harp, capturing the feeling of fog in a ocean, somber undertones, midnight ambiance", "domain": "music moods"}
{"terse": "clear sky golden hour", "rich": "A peaceful journey through electronic beats-driven experimental music, like walking through a ocean during storm at afternoon", "domain": "music moods"}
{"terse": "serene violin", "rich": "Ambient pads focused blues piece, hopeful and introspective, evoking storm falling on a desert at morning", "domain": "music moods"}
{"terse": "peaceful cello", "rich": "A passionate journey through cello-driven blues music, like walking through a cathedral during sunrise at late night", "domain": "music moods"}
{"terse": "blues rooftop", "rich": "Rock soundscape with harp, desolate and contemplative, inspired by storm in a cathedral at blue hour", "domain": "music moods"}
{"terse": "lo-fi city at night", "rich": "Atmospheric blues piece with flute, capturing the feeling of storm in a cozy room, euphoric undertones, twilight ambiance", "domain": "music moods"}
{"terse": "choir storm", "rich": "Emotionally ethereal lo-fi arrangement featuring ambient pads, painting a scene of sunrise over a concert hall during morning", "domain": "music moods"}
{"terse": "calm snow", "rich": "A dreamy meditation in rock form, cello leading through fog-kissed forest at late night, deeply atmospheric", "domain": "music moods"}
{"terse": "saxophone sunrise", "rich": "Atmospheric jazz piece with synthesizer, capturing the feeling of lightning in a concert hall, dreamy undertones, midnight ambiance", "domain": "music moods"}
{"terse": "harp abandoned building", "rich": "A aggressive meditation in neoclassical form, drums leading through snow-kissed cathedral at late night, deeply atmospheric", "domain": "music moods"}

129
training/scripts/augment_pairs.py Executable file
View File

@@ -0,0 +1,129 @@
#!/usr/bin/env python3
"""
augment_pairs.py — Training data augmentation: paraphrase and translate.
Usage:
python3 augment_pairs.py --input data.jsonl
python3 augment_pairs.py --input data.jsonl --paraphrases 3 --langs es,fr,de
python3 augment_pairs.py --input data.jsonl --llm-endpoint http://localhost:11434/v1
"""
import json, os, sys, re, random
from pathlib import Path
random.seed(42)
PARAPHRASE_TRANSFORMS = [
lambda s: re.sub(r"(\w+), (\w+)", r"\2, \1", s, count=1),
lambda s: f"A beautifully rendered scene: {s[0].lower()}{s[1:]}" if len(s) > 10 else s,
lambda s: s.replace("A ", "The ").replace("An ", "The ") if s.startswith(("A ", "An ")) else f"Here, {s[0].lower()}{s[1:]}",
lambda s: f"In a cinematic frame: {s}" if len(s) > 20 else s,
lambda s: s if ", " not in s else ", ".join(s.split(", ")[:2]),
]
TRANSLATIONS = {
"es": {"the":"el","a":"un","is":"es","in":"en","of":"de","and":"y","with":"con","scene":"escena","light":"luz","dark":"oscuro","warm":"cálido","rain":"lluvia","sun":"sol","moon":"luna","sky":"cielo","forest":"bosque","mountain":"montaña","ocean":"océano","golden":"dorado","blue":"azul","red":"rojo","green":"verde","silence":"silencio","dream":"sueño","love":"amor","hope":"esperanza","fear":"miedo","joy":"alegría","peace":"paz","beautiful":"hermoso","sad":"triste","shadow":"sombra","color":"color","silver":"plateado","white":"blanco","black":"negro","portray":"retrato"},
"fr": {"the":"le","a":"un","is":"est","in":"dans","of":"de","and":"et","with":"avec","scene":"scène","light":"lumière","dark":"sombre","warm":"chaud","rain":"pluie","sun":"soleil","moon":"lune","sky":"ciel","forest":"forêt","mountain":"montagne","ocean":"océan","golden":"doré","blue":"bleu","red":"rouge","green":"vert","silence":"silence","dream":"rêve","love":"amour","hope":"espoir","fear":"peur","joy":"joie","peace":"paix","beautiful":"beau","sad":"triste","shadow":"ombre","color":"couleur","silver":"argenté","white":"blanc","black":"noir"},
"de": {"the":"der","a":"ein","is":"ist","in":"in","of":"von","and":"und","with":"mit","scene":"Szene","light":"Licht","dark":"dunkel","warm":"warm","rain":"Regen","sun":"Sonne","moon":"Mond","sky":"Himmel","forest":"Wald","mountain":"Berg","ocean":"Ozean","golden":"golden","blue":"blau","red":"rot","green":"grün","silence":"Stille","dream":"Traum","love":"Liebe","hope":"Hoffnung","fear":"Angst","joy":"Freude","peace":"Frieden","beautiful":"schön","sad":"traurig","shadow":"Schatten","color":"Farbe","silver":"silbern","white":"weiß","black":"schwarz"},
}
LANG_NAMES = {"es": "Spanish", "fr": "French", "de": "German"}
def detect_text_field(entry):
for f in ["rich","terse","text","content","lyric_line","description","scene_description","prompt","scene"]:
if f in entry and isinstance(entry[f], str) and len(entry[f]) > 5:
return f
for k, v in entry.items():
if isinstance(v, str) and len(v) > 5:
return k
return None
def paraphrase(text):
t = random.choice(PARAPHRASE_TRANSFORMS)(text)
if t == text:
t = text.replace(" and ", " & ").replace(" with ", " alongside ")
if t == text:
t = f"In this scene: {text[0].lower()}{text[1:]}" if text[0].isupper() else text
return t
def translate(text, lang):
d = TRANSLATIONS.get(lang, {})
words = text.split()
out = []
for w in words:
lo = w.lower().strip(".,;:!?")
suf = w[len(w.rstrip(".,;:!?")):]
if lo in d:
out.append(d[lo] + suf)
else:
out.append(w)
return " ".join(out)
def augment_file(input_path, output_path=None, n_para=3, langs=None, llm_endpoint=None):
input_path = Path(input_path)
if output_path is None:
output_path = input_path.parent / f"{input_path.stem}_augmented{input_path.suffix}"
entries = [json.loads(l) for l in open(input_path) if l.strip()]
if not entries:
print(f"No entries in {input_path}"); return 0
tf = detect_text_field(entries[0])
if not tf:
print(f"ERROR: No text field in {input_path}", file=sys.stderr); return 0
print(f"Input: {input_path} ({len(entries)} entries, field={tf})")
aug_count = 0
with open(output_path, "w") as out:
for e in entries:
out.write(json.dumps(e, ensure_ascii=False) + "\n")
for i, e in enumerate(entries):
text = e[tf]
# Paraphrases
for p in range(n_para):
para = paraphrase(text)
if para != text:
ne = dict(e); ne[tf] = para
ne["_augmentation"] = f"paraphrase_{p+1}"
ne["_original"] = text[:100]
out.write(json.dumps(ne, ensure_ascii=False) + "\n")
aug_count += 1
# Translations
for lang in (langs or []):
tr = translate(text, lang)
if tr != text:
ne = dict(e); ne[tf] = tr
ne["_augmentation"] = f"translate_{lang}"
ne["_language"] = lang
ne["_original"] = text[:100]
out.write(json.dumps(ne, ensure_ascii=False) + "\n")
aug_count += 1
if (i+1) % 100 == 0:
print(f" {i+1}/{len(entries)} done ({aug_count} augmented)")
total = len(entries) + aug_count
print(f"Done: {len(entries)} originals + {aug_count} augmented = {total}")
print(f"Output: {output_path}")
return aug_count
def main():
import argparse
p = argparse.ArgumentParser()
p.add_argument("--input", required=True)
p.add_argument("--output", default=None)
p.add_argument("--paraphrases", type=int, default=3)
p.add_argument("--langs", default="es,fr,de")
p.add_argument("--llm-endpoint", default=None)
args = p.parse_args()
langs = [l.strip() for l in args.langs.split(",") if l.strip()] if args.langs else []
augment_file(args.input, args.output, args.paraphrases, langs, args.llm_endpoint)
if __name__ == "__main__":
main()