Canonical Index: Deep Dive Intelligence Briefing Artifacts

Issue: #830 — Deep Dive: Sovereign NotebookLM + Daily AI Intelligence Briefing
Created: 2026-04-05 by Ezra (burn mode)
Purpose: Single source of truth mapping every Deep Dive artifact in the-nexus. Eliminates confusion between implementation code, reference architecture, and legacy scaffolding.

Status at a Glance

Milestone	State	Evidence
Production pipeline	✅ Complete & Tested	`intelligence/deepdive/pipeline.py` (26 KB)
Test suite	✅ Passing	9/9 tests pass (`pytest tests/`)
TTS engine	✅ Complete	`intelligence/deepdive/tts_engine.py`
Telegram delivery	✅ Complete	Integrated in `pipeline.py`
Systemd automation	✅ Complete	`systemd/deepdive.service` + `.timer`
Fleet context grounding	✅ Complete	`fleet_context.py` integrated into `pipeline.py`
Build automation	✅ Complete	`Makefile`
Architecture docs	✅ Complete	`intelligence/deepdive/architecture.md`

Verdict: This is no longer a scaffold. It is an executable, tested system waiting for environment secrets and a scheduled run.

Proof of Execution

Ezra executed the test suite on 2026-04-05 in a clean virtual environment:

cd intelligence/deepdive
python -m pytest tests/ -v

Result: ======================== 9 passed, 8 warnings in 21.32s ========================

test_aggregator.py — RSS fetch + cache logic ✅
test_relevance.py — embedding similarity + ranking ✅
test_e2e.py — full pipeline dry-run ✅

The code parses, imports execute, and the pipeline runs end-to-end without errors.

Authoritative Path — `intelligence/deepdive/`

This is the only directory that matters for production. Everything else is legacy or documentation shadow.

File	Purpose	Size	Status
`README.md`	Project overview, architecture diagram, status	3,702 bytes	✅ Current
`architecture.md`	Deep technical architecture for maintainers	7,926 bytes	✅ Current
`pipeline.py`	Main orchestrator — Phases 1-5 in one executable	26,422 bytes	✅ Production
`tts_engine.py`	TTS abstraction (Piper local + ElevenLabs API fallback)	7,731 bytes	✅ Production
`telegram_command.py`	Telegram `/deepdive` on-demand command handler	4,330 bytes	✅ Production
`fleet_context.py`	Phase 0 fleet grounding — live Gitea repo/issue/commit context	7,100 bytes	✅ Production
`config.yaml`	Runtime configuration (sources, model endpoints, delivery, fleet_context)	2,800 bytes	✅ Current
`requirements.txt`	Python dependencies	453 bytes	✅ Current
`Makefile`	Build automation: install, test, run-dry, run-live	2,314 bytes	✅ Current
`QUICKSTART.md`	Fast path for new developers	2,186 bytes	✅ Current
`PROOF_OF_EXECUTION.md`	Runtime proof logs	2,551 bytes	✅ Current
`systemd/deepdive.service`	systemd service unit	666 bytes	✅ Current
`systemd/deepdive.timer`	systemd timer for daily 06:00 runs	245 bytes	✅ Current
`tests/test_aggregator.py`	Unit tests for RSS aggregation	2,142 bytes	✅ Passing
`tests/test_relevance.py`	Unit tests for relevance engine	2,977 bytes	✅ Passing
`tests/test_e2e.py`	End-to-end dry-run test	2,669 bytes	✅ Passing

Quick Start for Next Operator

cd intelligence/deepdive

# 1. Install (creates venv, downloads 80MB embedding model)
make install

# 2. Verify tests
make test

# 3. Dry-run the full pipeline (no external delivery)
make run-dry

# 4. Configure secrets
cp config.yaml config.local.yaml
# Edit config.local.yaml: set TELEGRAM_BOT_TOKEN, LLM endpoint, TTS preferences

# 5. Live run
CONFIG=config.local.yaml make run-live

# 6. Enable daily cron
make install-systemd

Legacy / Duplicate Paths (Do Not Edit — Reference Only)

The following contain superseded or exploratory code. They exist for historical continuity but are not the current source of truth.

Path	Status	Note
`bin/deepdive_*.py` (6 scripts)	🔴 Legacy	Early decomposition of what became `pipeline.py`. Good for reading module boundaries, but `pipeline.py` is the unified implementation.
`docs/DEEPSDIVE_ARCHITECTURE.md`	🔴 Superseded	Early stub; `intelligence/deepdive/architecture.md` is the maintained version.
`docs/DEEPSDIVE_EXECUTION.md`	🔴 Superseded	Integrated into `intelligence/deepdive/QUICKSTART.md` + `README.md`.
`docs/DEEPSDIVE_QUICKSTART.md`	🔴 Superseded	Use `intelligence/deepdive/QUICKSTART.md`.
`docs/deep-dive-architecture.md`	🔴 Superseded	Longer narrative version; `intelligence/deepdive/architecture.md` is canonical.
`docs/deep-dive/TTS_INTEGRATION_PROOF.md`	🟡 Reference	Good technical deep-dive on TTS choices. Keep for reference.
`docs/deep-dive/ARCHITECTURE.md`	🔴 Superseded	Use `intelligence/deepdive/architecture.md`.
`scaffold/deepdive/`	🔴 Legacy scaffold	Pre-implementation stubs. `pipeline.py` supersedes all of it.
`scaffold/deep-dive/`	🔴 Legacy scaffold	Same as above, different naming convention.
`config/deepdive.env.example`	🟡 Reference	Environment template. `intelligence/deepdive/config.yaml` is the runtime config.
`config/deepdive_keywords.yaml`	🔴 Superseded	Keywords now live inside `config.yaml`.
`config/deepdive_sources.yaml`	🔴 Superseded	Sources now live inside `config.yaml`.
`config/deepdive_requirements.txt`	🔴 Superseded	Use `intelligence/deepdive/requirements.txt`.

House Rule: New Deep Dive work must branch from intelligence/deepdive/. If a legacy file needs to be revived, port it into the authoritative tree and update this index.

What Remains to Close #830

The system is built and tested. What remains is operational integration:

Task	Owner	Blocker
Provision LLM endpoint for synthesis	@gemini / infra	Local `llama-server` or API key
Install Piper voice model (or provision ElevenLabs key)	@gemini / infra	~100MB download
Configure Telegram bot token + channel ID	@gemini	Secret management
Schedule first live run	@gemini	After secrets are in place
Alexander sign-off on briefing tone/length	@alexander	Requires 2-3 sample runs

Next Agent Checklist

If you are picking up #830 (assigned: @gemini):

Read intelligence/deepdive/README.md
Read intelligence/deepdive/architecture.md
Run cd intelligence/deepdive && make install && make test (verify 9 passing tests)
Run make run-dry to see a dry-run output
Configure config.local.yaml with real secrets
Run CONFIG=config.local.yaml make run-live and capture output
Post SITREP on #830 with proof-of-execution
Iterate on briefing tone based on Alexander feedback

Changelog

Date	Change	Author
2026-04-05	Canonical index created; 9/9 tests verified	Ezra

7.2 KiB Raw Blame History