Timmy_Foundation/compounding-intelligence
15
0
Fork 0
Code Issues 173 Pull Requests 59 Actions Packages Projects Releases Wiki Activity

Pipeline 1: The Knowledge Mine — Mine 20K Sessions Into Sovereign Knowledge #22

New Issue
Open
opened 2026-04-14 17:57:26 +00:00 by claude · 0 comments
claude commented 2026-04-14 17:57:26 +00:00
Member
Copy Link

Batch Pipeline 1 of 5

Mine every hermes-agent session transcript. Extract decisions, patterns, errors-and-fixes, tool quirks. Build a knowledge base every future session bootstraps from.

Why

We have 20,000+ conversation transcripts sitting in SQLite doing nothing. Each one contains lessons learned. The knowledge mine extracts them into structured, searchable knowledge.

Token Budget

  • 10K tokens per session × 20K sessions = 200M tokens
  • Parallelizable across 10 workers
  • ~4 hours at full throughput

Output

  • ~/.hermes/knowledge/ directory
  • Structured knowledge files (decisions, patterns, errors, tool-quirks)
  • Session DNA (compact summaries of what each session produced)
  • Cross-referenced with fact_store and memory

Sub-Issues

  • Session sampler (find highest-value sessions first)
  • Batch harvester (parallel processing)
  • Knowledge format (structured output)
  • Integration (bootstrap future sessions from knowledge)

Compounding Value

Every future session starts smarter. The 200M tokens spent here save tokens in every session forever after.

Depends On

  • Compounding-intelligence harvester issues #6-#9 (existing)
## Batch Pipeline 1 of 5 Mine every hermes-agent session transcript. Extract decisions, patterns, errors-and-fixes, tool quirks. Build a knowledge base every future session bootstraps from. ### Why We have 20,000+ conversation transcripts sitting in SQLite doing nothing. Each one contains lessons learned. The knowledge mine extracts them into structured, searchable knowledge. ### Token Budget - 10K tokens per session × 20K sessions = 200M tokens - Parallelizable across 10 workers - ~4 hours at full throughput ### Output - `~/.hermes/knowledge/` directory - Structured knowledge files (decisions, patterns, errors, tool-quirks) - Session DNA (compact summaries of what each session produced) - Cross-referenced with fact_store and memory ### Sub-Issues - Session sampler (find highest-value sessions first) - Batch harvester (parallel processing) - Knowledge format (structured output) - Integration (bootstrap future sessions from knowledge) ### Compounding Value Every future session starts smarter. The 200M tokens spent here save tokens in every session forever after. ### Depends On - Compounding-intelligence harvester issues #6-#9 (existing)
claude added the epicbatch-pipelineharvester labels 2026-04-14 17:57:26 +00:00
hermes was assigned by Rockachopa 2026-04-15 01:50:34 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acceptance-criteria
batch-pipeline
Token masterplan batch pipeline
 bootstrapper
Pre-session context injection
 epic
Epic-level issue
 harvester
Session knowledge extraction
 measurer
Compounding metrics
 milestone:1
Milestone 1: Foundation
 milestone:2
Milestone 2: Integration
 milestone:3
Milestone 3: Measurement
 milestone:4
Milestone 4: Retroactive
 pipeline
Pipeline/integration work
 pipeline
priority:high
priority:medium
retroactive
Processing existing sessions
 throughput-10x
throughput-10x label
 token-masterplan
token-masterplan label
No Label batch-pipeline epic harvester
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Rockachopa Timmy allegro antigravity bezalel claude codex-agent ezra gemini google grok hermes kimi manus perplexity
No Assignees hermes
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/compounding-intelligence#22
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?