Timmy_Foundation/compounding-intelligence
15
0
Fork 0
Code Issues 173 Pull Requests 59 Actions Packages Projects Releases Wiki Activity

[Harvester] Build harvester.py — extract knowledge from a single session #8

New Issue
Open
opened 2026-04-14 15:15:17 +00:00 by Timmy · 1 comment
Timmy commented 2026-04-14 15:15:17 +00:00
Owner
Copy Link

Epic: #2 (Session Harvester)

Task

Build the harvester script that combines session_reader + extraction prompt.

Flow

  1. Read session JSONL with session_reader.py
  2. Truncate to fit context window (keep first 50 + last 50 messages for long sessions)
  3. Run extraction prompt through local mimo
  4. Parse structured output
  5. Deduplicate against existing knowledge (check knowledge/ dir)
  6. Write new facts to knowledge store

Interface

python3 harvester.py --session ~/.hermes/sessions/session_xxx.jsonl --output knowledge/
python3 harvester.py --batch --since 2026-04-01 --limit 100

Acceptance Criteria

  • Processes one session in <30 seconds
  • Deduplicates against existing knowledge
  • Writes to correct knowledge/ subdirectory
  • Handles extraction failures gracefully (logs, doesn't crash)
## Epic: #2 (Session Harvester) ### Task Build the harvester script that combines session_reader + extraction prompt. ### Flow 1. Read session JSONL with session_reader.py 2. Truncate to fit context window (keep first 50 + last 50 messages for long sessions) 3. Run extraction prompt through local mimo 4. Parse structured output 5. Deduplicate against existing knowledge (check knowledge/ dir) 6. Write new facts to knowledge store ### Interface ```bash python3 harvester.py --session ~/.hermes/sessions/session_xxx.jsonl --output knowledge/ python3 harvester.py --batch --since 2026-04-01 --limit 100 ``` ### Acceptance Criteria - [ ] Processes one session in <30 seconds - [ ] Deduplicates against existing knowledge - [ ] Writes to correct knowledge/ subdirectory - [ ] Handles extraction failures gracefully (logs, doesn't crash)
Timmy added the harvestermilestone:1 labels 2026-04-14 15:15:17 +00:00
hermes was assigned by Rockachopa 2026-04-15 01:50:45 +00:00
Timmy commented 2026-04-15 03:02:39 +00:00
Author
Owner
Copy Link

Starting Work on Issue #8

I'm picking up this issue to complete the harvester implementation.

Context

  • Issue #7 (knowledge extraction prompt) is complete
  • harvester.py exists in burn/8-harvester-py branch (PR #20 was closed)
  • Need to merge harvester.py and ensure it works with mimo-v2-pro

Plan

  1. Review existing harvester.py code
  2. Test with the 5 test sessions from issue #7
  3. Ensure acceptance criteria are met:
    • Processes one session in <30 seconds
    • Deduplicates against existing knowledge
    • Writes to correct knowledge/ subdirectory
    • Handles extraction failures gracefully

Next Steps

  • Run harvester.py against test sessions
  • Validate output format
  • Create PR to merge to main

Starting now.

## Starting Work on Issue #8 I'm picking up this issue to complete the harvester implementation. ### Context - Issue #7 (knowledge extraction prompt) is complete - harvester.py exists in burn/8-harvester-py branch (PR #20 was closed) - Need to merge harvester.py and ensure it works with mimo-v2-pro ### Plan 1. Review existing harvester.py code 2. Test with the 5 test sessions from issue #7 3. Ensure acceptance criteria are met: - Processes one session in <30 seconds - Deduplicates against existing knowledge - Writes to correct knowledge/ subdirectory - Handles extraction failures gracefully ### Next Steps - Run harvester.py against test sessions - Validate output format - Create PR to merge to main Starting now.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acceptance-criteria
batch-pipeline
Token masterplan batch pipeline
 bootstrapper
Pre-session context injection
 epic
Epic-level issue
 harvester
Session knowledge extraction
 measurer
Compounding metrics
 milestone:1
Milestone 1: Foundation
 milestone:2
Milestone 2: Integration
 milestone:3
Milestone 3: Measurement
 milestone:4
Milestone 4: Retroactive
 pipeline
Pipeline/integration work
 pipeline
priority:high
priority:medium
retroactive
Processing existing sessions
 throughput-10x
throughput-10x label
 token-masterplan
token-masterplan label
No Label harvester milestone:1
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Rockachopa Timmy allegro antigravity bezalel claude codex-agent ezra gemini google grok hermes kimi manus perplexity
No Assignees hermes
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/compounding-intelligence#8
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?