feat: overnight R&D automation — Deep Dive + tightening + DPO export (#503)

docs/overnight-rd.md

@@ -0,0 +1,68 @@
+# Overnight R&D Automation
+
+**Schedule**: Nightly at 10 PM EDT (02:00 UTC)
+**Duration**: ~2-4 hours (self-limiting, finishes before 6 AM morning report)
+**Cost**: $0 — all local Ollama inference
+
+## Phases
+
+### Phase 1: Deep Dive Intelligence
+Runs the `intelligence/deepdive/pipeline.py` from the-nexus:
+- Aggregates arXiv CS.AI, CS.CL, CS.LG RSS feeds (last 24h)
+- Fetches OpenAI, Anthropic, DeepMind blog updates
+- Filters for relevance using sentence-transformers embeddings
+- Synthesizes a briefing using local Gemma 4 12B
+- Saves briefing to `~/briefings/`
+
+### Phase 2: Tightening Loop
+Exercises Timmy's local tool-use capability:
+- 10 tasks × 3 cycles = 30 task attempts per night
+- File reading, writing, searching against real workspace files
+- Each result logged as JSONL for training data analysis
+- Tests sovereignty compliance (SOUL.md alignment, banned provider detection)
+
+### Phase 3: DPO Export
+Sweeps overnight Hermes sessions for training pair extraction:
+- Converts good conversation pairs into DPO training format
+- Saves to `~/.timmy/training-data/dpo-pairs/`
+
+### Phase 4: Morning Prep
+Compiles overnight findings into `~/.timmy/overnight-rd/latest_summary.md`
+for consumption by the 6 AM `good_morning_report` task.
+
+## Approved Providers
+
+| Slot | Provider | Model |
+|------|----------|-------|
+| Synthesis | Ollama | gemma4:12b |
+| Tool tasks | Ollama | hermes4:14b |
+| Fallback | Ollama | gemma4:12b |
+
+Anthropic is permanently banned (BANNED_PROVIDERS.yml, 2026-04-09).
+
+## Outputs
+
+| Path | Content |
+|------|---------|
+| `~/.timmy/overnight-rd/{run_id}/rd_log.jsonl` | Full task log |
+| `~/.timmy/overnight-rd/{run_id}/rd_summary.md` | Run summary |
+| `~/.timmy/overnight-rd/latest_summary.md` | Latest summary (for morning report) |
+| `~/briefings/briefing_*.json` | Deep Dive briefings |
+
+## Monitoring
+
+Check the Huey consumer log:
+```bash
+tail -f ~/.timmy/timmy-config/logs/huey.log | grep overnight
+```
+
+Check the latest run summary:
+```bash
+cat ~/.timmy/overnight-rd/latest_summary.md
+```
+
+## Dependencies
+
+- Deep Dive pipeline installed: `cd the-nexus/intelligence/deepdive && make install`
+- Ollama running with gemma4:12b and hermes4:14b models
+- Huey consumer running: `huey_consumer.py tasks.huey -w 2 -k thread`