fix: docs: verify epic slice for #582 on main (closes #789) (closes #795)

docs/issue-582-verification.md

@@ -1,57 +1,61 @@
-# Issue #582 Verification — Parent-Epic Orchestration Slice
+# Issue #582 Verification — Parent-Epic Slice on Main
 
-**Date:** 2026-04-14
-**Status:** ✅ Verified on `main`
-**Refs:** #582, #789, #795
+Refs #582
+Closes #789
 
-## Summary
+## Purpose
 
-The parent-epic orchestration slice for the Know Thy Father multimodal pipeline is **already implemented on `main`**. This document records the verification evidence.
+This document provides a durable, in-repo evidence trail confirming that the
+**repo-side parent-epic orchestration slice** for #582 is already implemented
+on `main` and fully tested.
 
-## What Exists
+## What is implemented
 
-### Core Orchestrator
-- **File:** `scripts/know_thy_father/epic_pipeline.py`
-- **Function:** Builds and executes a 5-phase pipeline plan; reports status snapshots
-- **Phases:**
-  1. Media Indexing (`scripts/know_thy_father/index_media.py`)
-  2. Multimodal Analysis (`scripts/twitter_archive/analyze_media.py`)
-  3. Holographic Synthesis (`scripts/know_thy_father/synthesize_kernels.py`)
-  4. Cross-Reference Audit (`scripts/know_thy_father/crossref_audit.py`)
-  5. Processing Log (`twitter-archive/know-thy-father/tracker.py`)
+The epic's operational decomposition lives in:
 
-### Supporting Scripts (All Present)
-| Script | Purpose |
-|--------|---------|
-| `scripts/know_thy_father/index_media.py` | Phase 1 — scan tweets, emit media manifest |
-| `scripts/twitter_archive/analyze_media.py` | Phase 2 — batch multimodal analysis |
-| `scripts/know_thy_father/synthesize_kernels.py` | Phase 3 — kernels → Father's Ledger |
-| `scripts/know_thy_father/crossref_audit.py` | Phase 4 — cross-reference against SOUL.md |
-| `twitter-archive/know-thy-father/tracker.py` | Phase 5 — processing log / status report |
+| Artifact | Path |
+|----------|------|
+| Runner script | `scripts/know_thy_father/epic_pipeline.py` |
+| Pipeline doc | `docs/KNOW_THY_FATHER_MULTIMODAL_PIPELINE.md` |
+| Pipeline tests | `tests/test_know_thy_father_pipeline.py` |
+| Index tests | `tests/test_know_thy_father_index.py` |
+| Synthesis tests | `tests/test_know_thy_father_synthesis.py` |
+| Crossref tests | `tests/test_know_thy_father_crossref.py` |
+| KTF tracker tests | `tests/twitter_archive/test_ktf_tracker.py` |
+| Analyze media tests | `tests/twitter_archive/test_analyze_media.py` |
 
-### Test Coverage
-- `tests/test_know_thy_father_pipeline.py` — orchestrator structure
-- `tests/test_know_thy_father_index.py` — Phase 1 logic
-- `tests/test_know_thy_father_synthesis.py` — Phase 3 logic
-- `tests/test_know_thy_father_crossref.py` — Phase 4 logic
-- `tests/twitter_archive/test_ktf_tracker.py` — Phase 5 tracker
-- `tests/twitter_archive/test_analyze_media.py` — Phase 2 analysis
+Together these cover all five phases:
 
-## Why `Refs #582`, Not `Closes #582`
+1. **Media Indexing** — `scripts/know_thy_father/index_media.py`
+2. **Multimodal Analysis** — `scripts/twitter_archive/analyze_media.py --batch 10`
+3. **Holographic Synthesis** — `scripts/know_thy_father/synthesize_kernels.py`
+4. **Cross-Reference Audit** — `scripts/know_thy_father/crossref_audit.py`
+5. **Processing Log** — `twitter-archive/know-thy-father/tracker.py report`
 
-The repo-side operational slice is complete, but the parent epic (#582) also encompasses:
-- Full archive consumption (not yet finished)
-- Downstream memory integration (pending)
+## Why Refs #582, not Closes
 
-Therefore #582 remains open; this evidence trail closes the verification sub-issues (#789, #795).
+The **repo-side operational slice** is complete and tested. However, the parent
+epic (#582) itself remains open because:
 
-## Verification Commands
+- Full Twitter archive consumption (batch processing at scale) is not yet complete.
+- Downstream memory integration with the broader Timmy knowledge graph is pending.
+
+Closing this verification document honestly acknowledges: the *orchestration
+wiring* is done; the *data throughput* is not.
+
+## Historical trail
+
+- Parent epic: #582
+- Prior closed parent-epic PR: #789 (closed as superseded by this verification)
+- This PR/commit: provides the verification evidence trail
+
+## Verification commands
 
 ```bash
-# Orchestrator slice tests (10 tests)
+# 10 tests specific to this verification
 python3 -m pytest tests/test_issue_582_verification.py -q
 
-# Full Know Thy Father suite (71 tests)
+# 71 tests across the full KTF pipeline
 python3 -m pytest \
   tests/test_know_thy_father_pipeline.py \
   tests/test_know_thy_father_index.py \

tests/test_issue_582_verification.py

@@ -1,94 +1,131 @@
-"""Verification tests for issue #582 — parent-epic orchestration slice on main.
-
-These 10 tests confirm that the Know Thy Father epic pipeline orchestrator
-and all five phase scripts are present and structurally correct on main.
-
-Refs #582 | Closes #789 | Closes #795
 """
+Verification tests proving the #582 parent-epic orchestration slice exists on main.
 
-import importlib.util
+These 10 tests form the durable evidence trail for issue #789 / #795.
+"""
 from pathlib import Path
-
-import pytest
+import importlib.util
+import unittest
 
 
 ROOT = Path(__file__).resolve().parent.parent
-EPIC_PIPELINE = ROOT / "scripts" / "know_thy_father" / "epic_pipeline.py"
+PIPELINE_SCRIPT = ROOT / "scripts" / "know_thy_father" / "epic_pipeline.py"
+PIPELINE_DOC = ROOT / "docs" / "KNOW_THY_FATHER_MULTIMODAL_PIPELINE.md"
+VERIFICATION_DOC = ROOT / "docs" / "issue-582-verification.md"
+
+REQUIRED_KTF_SCRIPTS = [
+    "scripts/know_thy_father/index_media.py",
+    "scripts/twitter_archive/analyze_media.py",
+    "scripts/know_thy_father/synthesize_kernels.py",
+    "scripts/know_thy_father/crossref_audit.py",
+]
+
+REQUIRED_KTF_TESTS = [
+    "tests/test_know_thy_father_pipeline.py",
+    "tests/test_know_thy_father_index.py",
+    "tests/test_know_thy_father_synthesis.py",
+    "tests/test_know_thy_father_crossref.py",
+    "tests/twitter_archive/test_ktf_tracker.py",
+    "tests/twitter_archive/test_analyze_media.py",
+]
 
 
-def _load(name: str, path: Path):
+def load_module(path: Path, name: str):
     spec = importlib.util.spec_from_file_location(name, path)
     assert spec and spec.loader, f"cannot load {path}"
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    return mod
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
 
 
-class TestEpicPipelineExists:
-    def test_epic_pipeline_script_exists(self):
-        """epic_pipeline.py must be committed on main."""
-        assert EPIC_PIPELINE.exists(), "scripts/know_thy_father/epic_pipeline.py missing"
+class TestIssue582Verification(unittest.TestCase):
+    """10 tests confirming #582 epic slice is on main."""
 
+    # --- scripts exist ---
 
-class TestEpicPipelineAPI:
-    @pytest.fixture(autouse=True)
-    def _mod(self):
-        self.mod = _load("epic_pipeline", EPIC_PIPELINE)
+    def test_01_epic_pipeline_runner_exists(self):
+        """The epic orchestration runner script is committed."""
+        self.assertTrue(PIPELINE_SCRIPT.exists(), "epic_pipeline.py missing")
 
-    def test_has_build_pipeline_plan(self):
-        assert hasattr(self.mod, "build_pipeline_plan")
+    def test_02_all_ktf_phase_scripts_exist(self):
+        """Each KTF phase script referenced by the runner is present."""
+        for rel in REQUIRED_KTF_SCRIPTS:
+            path = ROOT / rel
+            self.assertTrue(path.exists(), f"{rel} missing")
 
-    def test_has_build_status_snapshot(self):
-        assert hasattr(self.mod, "build_status_snapshot")
+    # --- docs exist ---
 
+    def test_03_pipeline_doc_exists(self):
+        """The Know Thy Father multimodal pipeline doc is committed."""
+        self.assertTrue(PIPELINE_DOC.exists(), "pipeline doc missing")
 
-class TestPipelinePlanStructure:
-    @pytest.fixture(autouse=True)
-    def _mod(self):
-        self.mod = _load("epic_pipeline", EPIC_PIPELINE)
+    def test_04_verification_doc_exists(self):
+        """This verification document itself is committed."""
+        self.assertTrue(VERIFICATION_DOC.exists(), "verification doc missing")
 
-    def test_plan_has_five_phases(self):
-        plan = self.mod.build_pipeline_plan(batch_size=10)
-        assert len(plan) == 5
+    def test_05_verification_doc_refs_582(self):
+        """Verification doc references parent epic #582."""
+        text = VERIFICATION_DOC.read_text(encoding="utf-8")
+        self.assertIn("#582", text)
+        self.assertIn("#789", text)
 
-    def test_phase_ids_in_order(self):
-        plan = self.mod.build_pipeline_plan(batch_size=10)
-        ids = [step["id"] for step in plan]
-        assert ids == [
+    # --- runner functionality ---
+
+    def test_06_runner_builds_five_phase_plan(self):
+        """build_pipeline_plan returns exactly five phases in order."""
+        mod = load_module(PIPELINE_SCRIPT, "ktf_epic_pipeline")
+        plan = mod.build_pipeline_plan(batch_size=10)
+        phase_ids = [step["id"] for step in plan]
+        self.assertEqual(phase_ids, [
             "phase1_media_indexing",
             "phase2_multimodal_analysis",
             "phase3_holographic_synthesis",
             "phase4_cross_reference_audit",
             "phase5_processing_log",
-        ]
+        ])
+
+    def test_07_runner_status_snapshot_has_all_phases(self):
+        """build_status_snapshot reports all five phases."""
+        mod = load_module(PIPELINE_SCRIPT, "ktf_epic_pipeline")
+        status = mod.build_status_snapshot(ROOT)
+        for phase_id in [
+            "phase1_media_indexing",
+            "phase2_multimodal_analysis",
+            "phase3_holographic_synthesis",
+            "phase4_cross_reference_audit",
+            "phase5_processing_log",
+        ]:
+            self.assertIn(phase_id, status, f"{phase_id} missing from status")
+
+    def test_08_status_scripts_all_exist_on_disk(self):
+        """Every script reported by status snapshot actually exists."""
+        mod = load_module(PIPELINE_SCRIPT, "ktf_epic_pipeline")
+        status = mod.build_status_snapshot(ROOT)
+        for phase_id, info in status.items():
+            self.assertTrue(
+                info.get("script_exists"),
+                f"{phase_id} script {info.get('script')} not found on disk",
+            )
+
+    # --- test files exist ---
+
+    def test_09_all_ktf_test_files_exist(self):
+        """All six KTF test files are committed."""
+        for rel in REQUIRED_KTF_TESTS:
+            path = ROOT / rel
+            self.assertTrue(path.exists(), f"{rel} missing")
+
+    # --- pipeline doc content ---
+
+    def test_10_pipeline_doc_has_all_five_phases(self):
+        """Pipeline doc names all five phases."""
+        text = PIPELINE_DOC.read_text(encoding="utf-8")
+        self.assertIn("Media Indexing", text)
+        self.assertIn("Multimodal Analysis", text)
+        self.assertIn("Holographic Synthesis", text)
+        self.assertIn("Cross-Reference Audit", text)
+        self.assertIn("Processing Log", text)
 
 
-class TestPhaseScriptsExist:
-    def test_all_four_phase_scripts_exist(self):
-        expected = [
-            "scripts/know_thy_father/index_media.py",
-            "scripts/twitter_archive/analyze_media.py",
-            "scripts/know_thy_father/synthesize_kernels.py",
-            "scripts/know_thy_father/crossref_audit.py",
-        ]
-        for rel in expected:
-            assert (ROOT / rel).exists(), f"{rel} missing"
-
-    def test_tracker_script_exists(self):
-        candidates = [
-            ROOT / "twitter-archive" / "know-thy-father" / "tracker.py",
-            ROOT / "scripts" / "know_thy_father" / "tracker.py",
-        ]
-        assert any(c.exists() for c in candidates), "tracker.py not found"
-
-
-class TestPipelineDocs:
-    def test_pipeline_design_doc_exists(self):
-        assert (ROOT / "docs" / "KNOW_THY_FATHER_MULTIMODAL_PIPELINE.md").exists()
-
-    def test_verification_doc_exists(self):
-        assert (ROOT / "docs" / "issue-582-verification.md").exists()
-
-    def test_verification_doc_mentions_epic(self):
-        text = (ROOT / "docs" / "issue-582-verification.md").read_text()
-        assert "#582" in text
+if __name__ == "__main__":
+    unittest.main()