Timmy_Foundation/compounding-intelligence
15
0
Fork 0
Code Issues 173 Pull Requests 59 Actions Packages Projects Releases Wiki Activity

feat: Session transcript harvester — extract Q&A, decisions, patterns #195

New Issue
Open
opened 2026-04-15 15:18:01 +00:00 by Rockachopa · 0 comments
Rockachopa commented 2026-04-15 15:18:01 +00:00
Owner
Copy Link

Epic: #136 (Knowledge Pipeline v2)

Task

Build a harvester that processes hermes session JSONL files and extracts durable knowledge.

Extraction Categories

  1. Q&A pairs: user question → agent answer (for training data)
  2. Decisions: "we decided to X because Y" (for decision log)
  3. Patterns: "the fix for Z is to do W" (for skill generation)
  4. Preferences: "Alexander prefers/always/never" (for user profile)
  5. Errors & fixes: "error X was fixed by Y" (for debugging knowledge)

Implementation

def harvest_session(session_path: str) -> List[KnowledgeEntry]:
    messages = parse_jsonl(session_path)
    entries = []
    for i, msg in enumerate(messages):
        if is_qa_pair(messages, i):
            entries.append(extract_qa(messages, i))
        if is_decision(messages, i):
            entries.append(extract_decision(messages, i))
        # ... etc
    return entries

Deliverables

  • Harvester script: compounding-intelligence/harvester.py
  • Run against 50 most recent sessions
  • Output: knowledge entries in structured JSON
  • Report: counts per category, sample entries

Labels: harvester, extraction, priority:critical

## Epic: #136 (Knowledge Pipeline v2) ### Task Build a harvester that processes hermes session JSONL files and extracts durable knowledge. ### Extraction Categories 1. **Q&A pairs**: user question → agent answer (for training data) 2. **Decisions**: "we decided to X because Y" (for decision log) 3. **Patterns**: "the fix for Z is to do W" (for skill generation) 4. **Preferences**: "Alexander prefers/always/never" (for user profile) 5. **Errors & fixes**: "error X was fixed by Y" (for debugging knowledge) ### Implementation ```python def harvest_session(session_path: str) -> List[KnowledgeEntry]: messages = parse_jsonl(session_path) entries = [] for i, msg in enumerate(messages): if is_qa_pair(messages, i): entries.append(extract_qa(messages, i)) if is_decision(messages, i): entries.append(extract_decision(messages, i)) # ... etc return entries ``` ### Deliverables - [ ] Harvester script: `compounding-intelligence/harvester.py` - [ ] Run against 50 most recent sessions - [ ] Output: knowledge entries in structured JSON - [ ] Report: counts per category, sample entries ### Labels: harvester, extraction, priority:critical
codex-agent was assigned by Rockachopa 2026-04-17 01:34:51 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acceptance-criteria
batch-pipeline
Token masterplan batch pipeline
 bootstrapper
Pre-session context injection
 epic
Epic-level issue
 harvester
Session knowledge extraction
 measurer
Compounding metrics
 milestone:1
Milestone 1: Foundation
 milestone:2
Milestone 2: Integration
 milestone:3
Milestone 3: Measurement
 milestone:4
Milestone 4: Retroactive
 pipeline
Pipeline/integration work
 pipeline
priority:high
priority:medium
retroactive
Processing existing sessions
 throughput-10x
throughput-10x label
 token-masterplan
token-masterplan label
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Rockachopa Timmy allegro antigravity bezalel claude codex-agent ezra gemini google grok hermes kimi manus perplexity
No Assignees codex-agent
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/compounding-intelligence#195
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?