diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 00000000..a54ab1bf --- /dev/null +++ b/PLAN.md @@ -0,0 +1,478 @@ +# Plan: Full Creative & DevOps Capabilities for Timmy + +## Overview + +Add five major capability domains to Timmy's agent system, turning it into a +sovereign creative studio and full-stack DevOps operator. All tools are +open-source, self-hosted, and GPU-accelerated where needed. + +--- + +## Phase 1: Git & DevOps Tools (Forge + Helm personas) + +**Goal:** Timmy can observe local/remote repos, read code, create branches, +stage changes, commit, diff, log, and manage PRs — all through the swarm +task system with Spark event capture. + +### New module: `src/tools/git_tools.py` + +Tools to add (using **GitPython** — BSD-3, `pip install GitPython`): + +| Tool | Function | Persona Access | +|---|---|---| +| `git_clone` | Clone a remote repo to local path | Forge, Helm | +| `git_status` | Show working tree status | Forge, Helm, Timmy | +| `git_diff` | Show staged/unstaged diffs | Forge, Helm, Timmy | +| `git_log` | Show recent commit history | Forge, Helm, Echo, Timmy | +| `git_branch` | List/create/switch branches | Forge, Helm | +| `git_add` | Stage files for commit | Forge, Helm | +| `git_commit` | Create a commit with message | Forge, Helm | +| `git_push` | Push to remote | Forge, Helm | +| `git_pull` | Pull from remote | Forge, Helm | +| `git_blame` | Show line-by-line authorship | Forge, Echo | +| `git_stash` | Stash/pop changes | Forge, Helm | + +### Changes to existing files + +- **`src/timmy/tools.py`** — Add `create_git_tools()` factory, wire into + `PERSONA_TOOLKITS` for Forge and Helm +- **`src/swarm/tool_executor.py`** — Enhance `_infer_tools_needed()` with + git keywords (commit, branch, push, pull, diff, clone, merge) +- **`src/config.py`** — Add `git_default_repo_dir: str = "~/repos"` setting +- **`src/spark/engine.py`** — Add `on_tool_executed()` method to capture + individual tool invocations (not just task-level events) +- **`src/swarm/personas.py`** — Add git-related keywords to Forge and Helm + preferred_keywords + +### New dependency + +```toml +# pyproject.toml +dependencies = [ + ..., + "GitPython>=3.1.40", +] +``` + +### Dashboard + +- **`/tools`** page updated to show git tools in the catalog +- Git tool usage stats visible per agent + +### Tests + +- `tests/test_git_tools.py` — test all git tool functions against tmp repos +- Mock GitPython's `Repo` class for unit tests + +--- + +## Phase 2: Image Generation (new "Pixel" persona) + +**Goal:** Generate storyboard frames and standalone images from text prompts +using FLUX.2 Klein 4B locally. + +### New persona: Pixel — Visual Architect + +```python +"pixel": { + "id": "pixel", + "name": "Pixel", + "role": "Visual Architect", + "description": "Image generation, storyboard frames, and visual design.", + "capabilities": "image-generation,storyboard,design", + "rate_sats": 80, + "bid_base": 60, + "bid_jitter": 20, + "preferred_keywords": [ + "image", "picture", "photo", "draw", "illustration", + "storyboard", "frame", "visual", "design", "generate", + "portrait", "landscape", "scene", "artwork", + ], +} +``` + +### New module: `src/tools/image_tools.py` + +Tools (using **diffusers** + **FLUX.2 Klein 4B** — Apache 2.0): + +| Tool | Function | +|---|---| +| `generate_image` | Text-to-image generation (returns file path) | +| `generate_storyboard` | Generate N frames from scene descriptions | +| `image_variations` | Generate variations of an existing image | + +### Architecture + +``` +generate_image(prompt, width=1024, height=1024, steps=4) + → loads FLUX.2 Klein via diffusers FluxPipeline + → saves to data/images/{uuid}.png + → returns path + metadata +``` + +- Model loaded lazily on first use, kept in memory for subsequent calls +- Falls back to CPU generation (slower) if no GPU +- Output saved to `data/images/` with metadata JSON sidecar + +### New dependency (optional extra) + +```toml +[project.optional-dependencies] +creative = [ + "diffusers>=0.30.0", + "transformers>=4.40.0", + "accelerate>=0.30.0", + "torch>=2.2.0", + "safetensors>=0.4.0", +] +``` + +### Config + +```python +# config.py additions +flux_model_id: str = "black-forest-labs/FLUX.2-klein-4b" +image_output_dir: str = "data/images" +image_default_steps: int = 4 +``` + +### Dashboard + +- `/creative/ui` — new Creative Studio page (image gallery + generation form) +- HTMX-powered: submit prompt, poll for result, display inline +- Gallery view of all generated images with metadata + +### Tests + +- `tests/test_image_tools.py` — mock diffusers pipeline, test prompt handling, + file output, storyboard generation + +--- + +## Phase 3: Music Generation (new "Lyra" persona) + +**Goal:** Generate full songs with vocals, instrumentals, and lyrics using +ACE-Step 1.5 locally. + +### New persona: Lyra — Sound Weaver + +```python +"lyra": { + "id": "lyra", + "name": "Lyra", + "role": "Sound Weaver", + "description": "Music and song generation with vocals, instrumentals, and lyrics.", + "capabilities": "music-generation,vocals,composition", + "rate_sats": 90, + "bid_base": 70, + "bid_jitter": 20, + "preferred_keywords": [ + "music", "song", "sing", "vocal", "instrumental", + "melody", "beat", "track", "compose", "lyrics", + "audio", "sound", "album", "remix", + ], +} +``` + +### New module: `src/tools/music_tools.py` + +Tools (using **ACE-Step 1.5** — Apache 2.0, `pip install ace-step`): + +| Tool | Function | +|---|---| +| `generate_song` | Text/lyrics → full song (vocals + instrumentals) | +| `generate_instrumental` | Text prompt → instrumental track | +| `generate_vocals` | Lyrics + style → vocal track | +| `list_genres` | Return supported genre/style tags | + +### Architecture + +``` +generate_song(lyrics, genre="pop", duration=120, language="en") + → loads ACE-Step model (lazy, cached) + → generates audio + → saves to data/music/{uuid}.wav + → returns path + metadata (duration, genre, etc.) +``` + +- Model loaded lazily, ~4GB VRAM minimum +- Output saved to `data/music/` with metadata sidecar +- Supports 19 languages, genre tags, tempo control + +### New dependency (optional extra, extends `creative`) + +```toml +[project.optional-dependencies] +creative = [ + ..., + "ace-step>=1.5.0", +] +``` + +### Config + +```python +music_output_dir: str = "data/music" +ace_step_model: str = "ace-step/ACE-Step-v1.5" +``` + +### Dashboard + +- `/creative/ui` expanded with Music tab +- Audio player widget (HTML5 `