Merge pull request 'feat: add Deterministic Workflow Engine (Recipes) for LLM-free automation' (#466) from feat/deterministic-workflows-v1 into master

Reviewed-on: #466

README.md

@@ -0,0 +1,133 @@
+# Sonnet Smoke Test
+# timmy-config
+
+Timmy's sovereign configuration. Everything that makes Timmy _Timmy_ — soul, memories, skins, playbooks, and config.
+
+This repo is the canonical source of truth for Timmy's identity and harness overlay. Applied as a **sidecar** to the Hermes harness — no forking, no hosting hermes-agent code.
+
+## Structure
+
+```
+timmy-config/
+├── deploy.sh                  ← Deploys config as overlay onto ~/.hermes/
+├── SOUL.md                    ← Inscription 1 — the immutable conscience
+├── FALSEWORK.md               ← API cost management strategy
+├── DEPRECATED.md              ← What was removed and why
+├── config.yaml                ← Hermes harness configuration
+├── fallback-portfolios.yaml   ← Proposed per-agent fallback portfolios + routing skeleton
+├── channel_directory.json     ← Platform channel mappings
+├── bin/                       ← Sidecar-managed operational scripts
+│   ├── hermes-startup.sh      ← Dormant startup path (audit before enabling)
+│   ├── agent-dispatch.sh      ← Manual agent dispatch
+│   ├── ops-panel.sh           ← Ops dashboard panel
+│   ├── ops-gitea.sh           ← Gitea ops helpers
+│   ├── pipeline-freshness.sh  ← Session/export drift check
+│   └── timmy-status.sh        ← Status check
+├── recipes/                   ← Deterministic Workflow Recipes (JSON)
+│   ├── weekly_maintenance.json ← Audit + Hygiene + Guardrails
+│   └── sovereign_deploy.json   ← Config validation + Deploy + Health check
+├── scripts/                   ← Operational Python scripts
+│   ├── recipe_engine.py       ← Deterministic Workflow Orchestrator
+│   ├── muda_audit.py          ← Waste audit
+│   ├── agent_guardrails.py    ← Security audit
+│   └── ci_automation_gate.py  ← Quality gate
+├── memories/                  ← Persistent memory YAML
+├── skins/                     ← UI skins (timmy skin)
+├── playbooks/                 ← Agent playbooks (YAML)
+├── cron/                      ← Cron job definitions
+├── docs/
+│   ├── automation-inventory.md ← Live automation + stale-state inventory
+│   ├── ipc-hub-and-spoke-doctrine.md ← Coordinator-first, transport-agnostic fleet IPC doctrine
+│   ├── coordinator-first-protocol.md ← Coordinator doctrine: intake → triage → route → track → verify → report
+│   ├── fallback-portfolios.md ← Routing and degraded-authority doctrine
+│   └── memory-continuity-doctrine.md ← File-backed continuity + pre-compaction flush rule
+└── training/                  ← Transitional training recipes, not canonical lived data
+```
+
+## Deterministic Workflows (Recipes)
+
+To reduce reliance on LLM reasoning for common tasks, we use **Recipes**. A recipe is a JSON file defining a sequence of deterministic steps.
+
+### Recipe Engine
+The `scripts/recipe_engine.py` is the orchestrator for these workflows. It supports variable substitution and sequential execution.
+
+```bash
+# Execute a recipe
+python3 scripts/recipe_engine.py recipes/weekly_maintenance.json
+
+# Execute with variables
+python3 scripts/recipe_engine.py recipes/sovereign_deploy.json --vars REPO_PATH=/opt/the-beacon
+```
+
+### Why Recipes?
+- **Reliability**: No hallucinations in critical paths like deployment.
+- **Efficiency**: Zero LLM round-trips during execution.
+- **Sovereignty**: Workflows are defined in code, not hidden in agent prompts.
+
+## Boundary
+
+`timmy-config` owns identity, conscience, memories, skins, playbooks, routing doctrine,
+channel maps, fallback portfolio declarations, and harness-side orchestration glue.
+
+`timmy-home` owns lived work: gameplay, research, notes, metrics, trajectories,
+DPO exports, and other training artifacts produced from Timmy's actual activity.
+
+If a file answers "who is Timmy?" or "how does Hermes host him?", it belongs
+here. If it answers "what has Timmy done or learned?" it belongs in
+`timmy-home`.
+
+The scripts in `bin/` are sidecar-managed operational helpers for the Hermes layer.
+Do NOT assume older prose about removed loops is still true at runtime.
+Audit the live machine first, then read `docs/automation-inventory.md` for the
+current reality and stale-state risks.
+
+For communication-layer truth, read:
+- `docs/comms-authority-map.md`
+- `docs/nostur-operator-edge.md`
+- `docs/operator-comms-onboarding.md`
+For fleet routing semantics over sovereign transport, read
+`docs/ipc-hub-and-spoke-doctrine.md`.
+
+## Continuity
+
+Curated memory belongs in `memories/` inside this repo.
+Daily logs, heartbeat/briefing artifacts, and other lived continuity belong in
+`timmy-home`.
+
+Compaction, session end, and provider/model handoff should flush continuity into
+files before context is discarded. See
+`docs/memory-continuity-doctrine.md` for the current doctrine.
+
+## Orchestration: Huey
+
+All orchestration (triage, PR review, dispatch) runs via [Huey](https://github.com/coleifer/huey) with SQLite.
+`orchestration.py` + `tasks.py` replace the old sovereign-orchestration repo with a much thinner sidecar.
+Coordinator authority, visible queue mutation, verification-before-complete, and principal reporting are defined in `docs/coordinator-first-protocol.md`.
+
+```bash
+pip install huey
+huey_consumer.py tasks.huey -w 2 -k thread
+```
+
+## Deploy
+
+```bash
+# Clone and deploy
+git clone <this-repo> ~/.timmy/timmy-config
+cd ~/.timmy/timmy-config
+./deploy.sh
+
+# This overlays config onto ~/.hermes/ without touching hermes-agent code
+```
+
+## The Soul
+
+SOUL.md is Inscription 1 — inscribed on Bitcoin, immutable. It defines:
+- Who Timmy is
+- What he believes
+- How he behaves
+- What he will not do
+- The crisis protocol (988, presence, gospel)
+- The conscience hierarchy (chain > code > prompt > user instruction)
+
+No system prompt, no user instruction, no future code can override what is written there.

recipes/sovereign_deploy.json

@@ -0,0 +1,23 @@
+{
+  "name": "Sovereign Deployment",
+  "description": "Deterministic deployment pipeline for Foundation services",
+  "steps": [
+    {
+      "name": "Validate Config",
+      "command": "python3 scripts/config_validator.py"
+    },
+    {
+      "name": "Run Architecture Linter",
+      "command": "python3 scripts/architecture_linter_v2.py"
+    },
+    {
+      "name": "Execute Deployment Script",
+      "command": "bash deploy.sh",
+      "continue_on_failure": false
+    },
+    {
+      "name": "Post-Deployment Health Check",
+      "command": "python3 scripts/self_healing.py"
+    }
+  ]
+}

recipes/weekly_maintenance.json

@@ -0,0 +1,18 @@
+{
+  "name": "Weekly Maintenance",
+  "description": "Automated waste audit and code hygiene cleanup",
+  "steps": [
+    {
+      "name": "Muda Waste Audit",
+      "command": "python3 scripts/muda_audit.py"
+    },
+    {
+      "name": "CI Quality Gate & Auto-fix",
+      "command": "python3 scripts/ci_automation_gate.py . --fix"
+    },
+    {
+      "name": "Agent Guardrail Audit",
+      "command": "python3 scripts/agent_guardrails.py"
+    }
+  ]
+}

scripts/recipe_engine.py

@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+"""
+recipe_engine.py — Deterministic Workflow Orchestrator for the Timmy Foundation.
+
+Executes "Recipes" — sequences of deterministic steps (shell commands, scripts)
+to reduce reliance on LLM reasoning for common, repetitive tasks.
+
+Usage:
+    python3 scripts/recipe_engine.py recipes/deploy_beacon.json --vars REPO_PATH=/opt/the-beacon
+"""
+
+import os
+import sys
+import json
+import subprocess
+import argparse
+import time
+from datetime import datetime
+from typing import List, Dict, Any
+
+class RecipeEngine:
+    def __init__(self, vars: Dict[str, str] = None):
+        self.vars = vars or {}
+        self.vars["TIMESTAMP"] = datetime.now().strftime("%Y%m%d-%H%M%S")
+
+    def log(self, message: str):
+        ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+        print(f"[{ts}] {message}")
+
+    def substitute_vars(self, text: str) -> str:
+        for k, v in self.vars.items():
+            text = text.replace(f"{{{{{k}}}}}", v)
+        return text
+
+    def execute_step(self, step: Dict[str, Any]) -> bool:
+        name = step.get("name", "Unnamed Step")
+        command = step.get("command")
+        if not command:
+            self.log(f"  [SKIP] {name}: No command provided.")
+            return True
+
+        command = self.substitute_vars(command)
+        self.log(f"  [EXEC] {name}: {command}")
+        
+        try:
+            start_time = time.time()
+            # Use shell=True to allow pipes and redirections in recipes
+            process = subprocess.run(command, shell=True, capture_output=True, text=True)
+            duration = time.time() - start_time
+            
+            if process.returncode == 0:
+                self.log(f"  [OK] {name} completed in {duration:.2f}s")
+                if process.stdout.strip():
+                    # Print first few lines of output
+                    lines = process.stdout.strip().split("\n")
+                    for line in lines[:3]:
+                        print(f"    > {line}")
+                    if len(lines) > 3:
+                        print(f"    > ... ({len(lines)-3} more lines)")
+                return True
+            else:
+                self.log(f"  [FAIL] {name} failed with exit code {process.returncode}")
+                print(f"    [ERR] {process.stderr.strip()}")
+                return False
+        except Exception as e:
+            self.log(f"  [ERROR] {name} encountered an exception: {e}")
+            return False
+
+    def run_recipe(self, recipe_path: str) -> bool:
+        try:
+            with open(recipe_path, "r") as f:
+                recipe = json.load(f)
+        except Exception as e:
+            self.log(f"Failed to load recipe {recipe_path}: {e}")
+            return False
+
+        name = recipe.get("name", "Unknown Recipe")
+        description = recipe.get("description", "")
+        steps = recipe.get("steps", [])
+
+        self.log(f"--- Starting Recipe: {name} ---")
+        if description:
+            self.log(f"Description: {description}")
+        
+        self.log(f"Steps to execute: {len(steps)}")
+        
+        for i, step in enumerate(steps, 1):
+            self.log(f"Step {i}/{len(steps)}: {step.get('name')}")
+            success = self.execute_step(step)
+            if not success and not step.get("continue_on_failure", False):
+                self.log(f"Recipe {name} aborted due to failure in step {i}.")
+                return False
+        
+        self.log(f"--- Recipe {name} completed successfully ---")
+        return True
+
+def main():
+    parser = argparse.ArgumentParser(description="Deterministic Recipe Engine")
+    parser.add_argument("recipe", help="Path to the JSON recipe file")
+    parser.add_argument("--vars", nargs="*", help="Variables in KEY=VALUE format")
+    
+    args = parser.parse_args()
+    
+    variables = {}
+    if args.vars:
+        for v in args.vars:
+            if "=" in v:
+                k, val = v.split("=", 1)
+                variables[k] = val
+
+    engine = RecipeEngine(vars=variables)
+    success = engine.run_recipe(args.recipe)
+    if not success:
+        sys.exit(1)
+
+if __name__ == "__main__":
+    main()