docs: project README and MVP specification
This commit is contained in:
53
README.md
53
README.md
@@ -1,3 +1,52 @@
|
|||||||
# timmy-time-app
|
# Timmy Time
|
||||||
|
|
||||||
Timmy Time — sovereign AI command center for iPad
|
Sovereign AI command center for iPad. Chat with Timmy, share images, voice,
|
||||||
|
links, and documents. All processing happens on your Mac — the iPad is the
|
||||||
|
eyes, ears, and hands; the Mac is the brain.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
iPad (native SwiftUI app) Mac (headless server)
|
||||||
|
┌─────────────────────────┐ ┌──────────────────────────┐
|
||||||
|
│ Chat UI │ │ FastAPI backend │
|
||||||
|
│ Camera / Photos │──────→ │ POST /api/v1/chat │
|
||||||
|
│ Microphone │ Tail- │ POST /api/v1/upload │
|
||||||
|
│ File picker │ scale │ WS /api/v1/chat/ws │
|
||||||
|
│ URL sharing │ │ │
|
||||||
|
│ │ ←───── │ Media pipeline: │
|
||||||
|
│ Streaming responses │ │ Images → Vision model │
|
||||||
|
│ Voice playback │ │ Audio → Whisper STT │
|
||||||
|
│ Rich media display │ │ URLs → Web extract │
|
||||||
|
│ │ │ Docs → OCR/parse │
|
||||||
|
│ │ │ │
|
||||||
|
│ │ │ Timmy agent (Agno) │
|
||||||
|
│ │ │ Ollama (qwen3:30b) │
|
||||||
|
└─────────────────────────┘ └──────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- iPad Pro with iPadOS 26.1+
|
||||||
|
- Mac with macOS 14+ running the Timmy dashboard server
|
||||||
|
- Tailscale connecting both devices
|
||||||
|
- Xcode 26+ on Mac for building
|
||||||
|
|
||||||
|
## MVP Features
|
||||||
|
|
||||||
|
- [x] Project structure
|
||||||
|
- [ ] Chat screen with streaming responses
|
||||||
|
- [ ] Image attachment (camera + photo library)
|
||||||
|
- [ ] Voice input (mic → Whisper on Mac)
|
||||||
|
- [ ] URL link sharing and summarization
|
||||||
|
- [ ] Persistent chat history
|
||||||
|
- [ ] Server-side API endpoints for the app
|
||||||
|
|
||||||
|
## Future
|
||||||
|
|
||||||
|
- Apple Pencil drawing/annotation
|
||||||
|
- On-device models (Core ML)
|
||||||
|
- LiDAR / ARKit spatial awareness
|
||||||
|
- Siri Shortcuts integration
|
||||||
|
- Push notifications
|
||||||
|
- Video processing
|
||||||
|
|||||||
143
SPEC.md
Normal file
143
SPEC.md
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
# Timmy Time App — MVP Specification
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Native iPad app that serves as the primary interface to a sovereign AI agent
|
||||||
|
(Timmy) running on a Mac server. All heavy computation stays on the Mac.
|
||||||
|
The iPad handles UI, sensor capture, and local preprocessing.
|
||||||
|
|
||||||
|
## Target Platform
|
||||||
|
|
||||||
|
- iPadOS 26.1+
|
||||||
|
- iPad Pro 13" (primary target)
|
||||||
|
- Swift 6 / SwiftUI
|
||||||
|
- Minimum deployment target: iPadOS 26.0
|
||||||
|
|
||||||
|
## Server Requirements
|
||||||
|
|
||||||
|
The Mac runs the existing Timmy dashboard (FastAPI) with new API endpoints
|
||||||
|
added specifically for the app. Communication over Tailscale (private network).
|
||||||
|
|
||||||
|
### API Endpoints (to build on Mac side)
|
||||||
|
|
||||||
|
#### Chat
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/v1/chat
|
||||||
|
Body: { "message": "string", "session_id": "string", "attachments": [...] }
|
||||||
|
Response: streaming text/event-stream (SSE)
|
||||||
|
|
||||||
|
GET /api/v1/chat/history?session_id=X&limit=50
|
||||||
|
Response: { "messages": [...] }
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Upload / Media
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/v1/upload
|
||||||
|
Body: multipart/form-data with file
|
||||||
|
Response: { "id": "string", "type": "image|audio|document|url",
|
||||||
|
"summary": "string", "metadata": {...} }
|
||||||
|
```
|
||||||
|
|
||||||
|
The upload endpoint auto-detects media type and routes to the right processor:
|
||||||
|
- Images (jpg, png, heic) → vision model analysis
|
||||||
|
- Audio (m4a, mp3, wav, caf) → Whisper transcription
|
||||||
|
- Documents (pdf, txt, md) → text extraction
|
||||||
|
- URLs (detected from text) → web page extraction
|
||||||
|
|
||||||
|
#### Status
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /api/v1/status
|
||||||
|
Response: { "timmy": "online|offline", "model": "qwen3:30b",
|
||||||
|
"ollama": "running|stopped", "uptime": "..." }
|
||||||
|
```
|
||||||
|
|
||||||
|
## App Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
TimmyTime/
|
||||||
|
├── TimmyTimeApp.swift # App entry point
|
||||||
|
├── Models/
|
||||||
|
│ ├── Message.swift # Chat message model
|
||||||
|
│ ├── Attachment.swift # Media attachment model
|
||||||
|
│ └── ServerConfig.swift # Server URL, auth config
|
||||||
|
├── Views/
|
||||||
|
│ ├── ChatView.swift # Main chat interface
|
||||||
|
│ ├── MessageBubble.swift # Individual message rendering
|
||||||
|
│ ├── AttachmentPicker.swift # Photo/file/camera picker
|
||||||
|
│ ├── VoiceButton.swift # Hold-to-talk microphone
|
||||||
|
│ ├── SettingsView.swift # Server URL config
|
||||||
|
│ └── StatusBar.swift # Connection/model status
|
||||||
|
├── Services/
|
||||||
|
│ ├── ChatService.swift # HTTP + SSE streaming client
|
||||||
|
│ ├── UploadService.swift # Multipart file upload
|
||||||
|
│ ├── AudioRecorder.swift # AVFoundation mic recording
|
||||||
|
│ └── PersistenceService.swift # Local chat history (SwiftData)
|
||||||
|
├── Assets.xcassets/ # App icons, colors
|
||||||
|
└── Info.plist
|
||||||
|
```
|
||||||
|
|
||||||
|
## Screen Layout (iPad Landscape)
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────────────────────────────────────────┐
|
||||||
|
│ ◉ Timmy Time qwen3:30b ● Online ⚙️ │
|
||||||
|
├──────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────────────────────┐ │
|
||||||
|
│ │ Timmy: │ │
|
||||||
|
│ │ Here's what I found... │ │
|
||||||
|
│ └─────────────────────────────┘ │
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────────────────────┐ │
|
||||||
|
│ │ You: │ │
|
||||||
|
│ │ [📷 photo.jpg] │ │
|
||||||
|
│ │ What's in this image? │ │
|
||||||
|
│ └─────────────────────────────┘ │
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────────────────────┐ │
|
||||||
|
│ │ Timmy: │ │
|
||||||
|
│ │ I see a circuit board... │ │
|
||||||
|
│ │ ▋ (streaming) │ │
|
||||||
|
│ └─────────────────────────────┘ │
|
||||||
|
│ │
|
||||||
|
├──────────────────────────────────────────────────────────┤
|
||||||
|
│ 📎 📷 🎤 │ Type a message... ➤ │
|
||||||
|
└──────────────────────────────────────────────────────────┘
|
||||||
|
|
||||||
|
📎 = file picker 📷 = camera 🎤 = hold to talk ➤ = send
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
|
||||||
|
1. User types/speaks/attaches media
|
||||||
|
2. App sends to Mac: POST /api/v1/chat (text) or /api/v1/upload (media)
|
||||||
|
3. Upload endpoint processes media, returns summary + ID
|
||||||
|
4. Chat endpoint receives message + attachment IDs, streams response
|
||||||
|
5. App renders streaming response in real time
|
||||||
|
6. Chat saved to local SwiftData store for offline viewing
|
||||||
|
|
||||||
|
## Design Principles
|
||||||
|
|
||||||
|
- **Touch-first.** Everything reachable by thumb. No tiny tap targets.
|
||||||
|
- **Sovereign.** No cloud dependencies. All traffic stays on Tailscale.
|
||||||
|
- **Media-rich.** Images, audio, links displayed inline, not as file names.
|
||||||
|
- **Fast.** Streaming responses start appearing immediately.
|
||||||
|
- **Simple.** One screen. Chat is the interface. Everything else is secondary.
|
||||||
|
|
||||||
|
## Color Palette
|
||||||
|
|
||||||
|
- Dark mode primary (easier on eyes, looks good on OLED iPad Pro)
|
||||||
|
- Accent color: match Timmy's personality — warm gold or sovereign blue
|
||||||
|
- Message bubbles: subtle differentiation between user and Timmy
|
||||||
|
|
||||||
|
## Phase 2 Features (post-MVP)
|
||||||
|
|
||||||
|
- Apple Pencil: draw on images, handwriting input
|
||||||
|
- Core ML: on-device Whisper for instant transcription
|
||||||
|
- Split View: chat + status side by side
|
||||||
|
- Drag and drop from other apps
|
||||||
|
- Share Sheet extension ("Send to Timmy")
|
||||||
|
- Push notifications for long-running task completion
|
||||||
Reference in New Issue
Block a user