diff --git a/docs/GOOGLE_AI_ULTRA_INTEGRATION.md b/docs/GOOGLE_AI_ULTRA_INTEGRATION.md new file mode 100644 index 0000000..b053fbc --- /dev/null +++ b/docs/GOOGLE_AI_ULTRA_INTEGRATION.md @@ -0,0 +1,127 @@ +# Google AI Ultra Integration Plan + +> Master tracking document for integrating all Google AI Ultra products into +> Project Timmy (Sovereign AI Agent) and The Nexus (3D World). + +**Epic**: #739 +**Milestone**: M5: Google AI Ultra Integration +**Label**: `google-ai-ultra` + +--- + +## Product Inventory + +| # | Product | Capability | API | Priority | Status | +|---|---------|-----------|-----|----------|--------| +| 1 | Gemini 3.1 Pro | Primary reasoning engine | ✅ | P0 | 🔲 Not started | +| 2 | Deep Research | Autonomous research reports | ✅ | P1 | 🔲 Not started | +| 3 | Veo 3.1 | Text/image → video | ✅ | P2 | 🔲 Not started | +| 4 | Nano Banana Pro | Image generation | ✅ | P1 | 🔲 Not started | +| 5 | Lyria 3 | Music/audio generation | ✅ | P2 | 🔲 Not started | +| 6 | NotebookLM | Doc synthesis + Audio Overviews | ❌ | P1 | 🔲 Not started | +| 7 | AI Studio | API portal + Vibe Code | N/A | P0 | 🔲 Not started | +| 8 | Project Genie | Interactive 3D world gen | ❌ | P1 | 🔲 Not started | +| 9 | Live API | Real-time voice streaming | ✅ | P2 | 🔲 Not started | +| 10 | Computer Use | Browser automation | ✅ | P2 | 🔲 Not started | + +--- + +## Phase 1: Identity & Branding (Week 1) + +| Issue | Title | Status | +|-------|-------|--------| +| #740 | Generate Timmy avatar set with Nano Banana Pro | 🔲 | +| #741 | Upload SOUL.md to NotebookLM → Audio Overview | 🔲 | +| #742 | Generate Timmy audio signature with Lyria 3 | 🔲 | +| #680 | Project Genie + Nano Banana concept pack | 🔲 | + +## Phase 2: Research & Planning (Week 1-2) + +| Issue | Title | Status | +|-------|-------|--------| +| #743 | Deep Research: Three.js multiplayer 3D world architecture | 🔲 | +| #744 | Deep Research: Sovereign AI agent frameworks | 🔲 | +| #745 | Deep Research: WebGL/WebGPU rendering comparison | 🔲 | +| #746 | NotebookLM synthesis: cross-reference all research | 🔲 | + +## Phase 3: Prototype & Build (Week 2-4) + +| Issue | Title | Status | +|-------|-------|--------| +| #747 | Provision Gemini API key + Hermes config | 🔲 | +| #748 | Integrate Gemini 3.1 Pro as reasoning backbone | 🔲 | +| #749 | AI Studio Vibe Code UI prototypes | 🔲 | +| #750 | Project Genie explorable world prototypes | 🔲 | +| #681 | Veo/Flow flythrough prototypes | 🔲 | + +## Phase 4: Media & Content (Ongoing) + +| Issue | Title | Status | +|-------|-------|--------| +| #682 | Lyria soundtrack palette for Nexus zones | 🔲 | +| #751 | Lyria RealTime dynamic reactive music | 🔲 | +| #752 | NotebookLM Audio Overviews for all docs | 🔲 | +| #753 | Nano Banana concept art batch pipeline | 🔲 | + +## Phase 5: Advanced Integration (Month 2+) + +| Issue | Title | Status | +|-------|-------|--------| +| #754 | Gemini Live API for voice conversations | 🔲 | +| #755 | Computer Use API for browser automation | 🔲 | +| #756 | Gemini RAG via File Search for Timmy memory | 🔲 | +| #757 | Gemini Native Audio + TTS for Timmy's voice | 🔲 | +| #758 | Programmatic image generation pipeline | 🔲 | +| #759 | Programmatic video generation pipeline | 🔲 | +| #760 | Deep Research Agent API integration | 🔲 | +| #761 | OpenAI-compatible endpoint config | 🔲 | +| #762 | Context caching + batch API for cost optimization | 🔲 | + +--- + +## API Quick Reference + +```python +# pip install google-genai +from google import genai +client = genai.Client() # reads GOOGLE_API_KEY env var + +# Text generation (Gemini 3.1 Pro) +response = client.models.generate_content( + model="gemini-3.1-pro-preview", + contents="..." +) +``` + +| API | Documentation | +|-----|--------------| +| Image Gen (Nano Banana) | ai.google.dev/gemini-api/docs/image-generation | +| Video Gen (Veo) | ai.google.dev/gemini-api/docs/video | +| Music Gen (Lyria) | ai.google.dev/gemini-api/docs/music-generation | +| TTS | ai.google.dev/gemini-api/docs/speech-generation | +| Deep Research | ai.google.dev/gemini-api/docs/deep-research | + +## Key URLs + +| Tool | URL | +|------|-----| +| Gemini App | gemini.google.com | +| AI Studio | aistudio.google.com | +| NotebookLM | notebooklm.google.com | +| Project Genie | labs.google/projectgenie | +| Flow (video) | labs.google/flow | +| Stitch (UI) | labs.google/stitch | + +## Hidden Features to Exploit + +1. **AI Studio Free Tier** — generous API access even without subscription +2. **OpenAI-Compatible API** — drop-in replacement for existing OpenAI tooling +3. **Context Caching** — cache SOUL.md to cut cost/latency on repeated calls +4. **Batch API** — bulk operations at discounted rates +5. **File Search Tool** — RAG without custom vector store +6. **Computer Use API** — programmatic browser control for agent automation +7. **Interactions API** — managed multi-turn conversational state + +--- + +*Generated: 2026-03-29. Epic #739, Milestone M5.*