From c203010e3a756deee8ace11f8c5b7564e9b63214 Mon Sep 17 00:00:00 2001 From: Alexander Whitestone Date: Tue, 21 Apr 2026 04:43:54 +0000 Subject: [PATCH] fix(#676): update GENOME.md for compounding-intelligence Previous version was outdated (said scripts were 'not implemented'). Updated to reflect actual state: 18 scripts, 14 test files, populated knowledge store, active development. --- GENOME.md | 374 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 193 insertions(+), 181 deletions(-) diff --git a/GENOME.md b/GENOME.md index 8863c65..362b0eb 100644 --- a/GENOME.md +++ b/GENOME.md @@ -1,16 +1,16 @@ # GENOME.md — compounding-intelligence -*Auto-generated codebase genome. Addresses timmy-home#676.* +**Generated:** 2026-04-17 +**Repo:** Timmy_Foundation/compounding-intelligence +**Description:** Turn 1B+ daily agent tokens into durable, compounding fleet intelligence. --- ## Project Overview -**What:** A system that turns 1B+ daily agent tokens into durable, compounding fleet intelligence. +Every agent session starts at zero. The same HTTP 405 gets rediscovered as a branch protection issue. The same token path gets searched from scratch. Intelligence evaporates when the session ends. -**Why:** Every agent session starts at zero. The same mistakes get made repeatedly — the same HTTP 405 is rediscovered as a branch protection issue, the same token path is searched for from scratch. Intelligence evaporates when the session ends. - -**How:** Three pipelines form a compounding loop: +Compounding-intelligence solves this with three pipelines forming a loop: ``` SESSION ENDS → HARVESTER → KNOWLEDGE STORE → BOOTSTRAPPER → NEW SESSION STARTS SMARTER @@ -18,222 +18,234 @@ SESSION ENDS → HARVESTER → KNOWLEDGE STORE → BOOTSTRAPPER → NEW SESSION MEASURER → Prove it's working ``` -**Status:** Early stage. Template and test scaffolding exist. Core pipeline scripts (harvester.py, bootstrapper.py, measurer.py, session_reader.py) are planned but not yet implemented. The knowledge extraction prompt is complete and validated. - ---- +**Status:** Active development. Core pipelines implemented. 20+ scripts, 14 test files, knowledge store populated with real data. ## Architecture ```mermaid graph TD - A[Session Transcript
.jsonl] --> B[Harvester] - B --> C{Extract Knowledge} - C --> D[knowledge/index.json] - C --> E[knowledge/global/*.md] - C --> F[knowledge/repos/{repo}.md] - C --> G[knowledge/agents/{agent}.md] - D --> H[Bootstrapper] - H --> I[Bootstrap Context
2k token injection] - I --> J[New Session
starts smarter] - J --> A - D --> K[Measurer] - K --> L[metrics/dashboard.md] - K --> M[Velocity / Hit Rate
Error Reduction] + TRANS[Session Transcripts
~/.hermes/sessions/*.jsonl] --> READER[session_reader.py] + READER --> HARVESTER[harvester.py] + HARVESTER -->|LLM extraction| PROMPT[harvest-prompt.md] + HARVESTER --> DEDUP[deduplicate()] + DEDUP --> INDEX[knowledge/index.json] + DEDUP --> GLOBAL[knowledge/global/*.yaml] + DEDUP --> REPO[knowledge/repos/*.yaml] + + INDEX --> BOOTSTRAPPER[bootstrapper.py] + BOOTSTRAPPER -->|filter + rank + truncate| CONTEXT[Bootstrap Context
2k token injection] + CONTEXT --> SESSION[New Session starts smarter] + + INDEX --> VALIDATOR[validate_knowledge.py] + INDEX --> STALENESS[knowledge_staleness_check.py] + INDEX --> GAPS[knowledge_gap_identifier.py] + + TRANS --> SAMPLER[sampler.py] + SAMPLER -->|score + rank| BEST[High-value sessions] + BEST --> HARVESTER + + TRANS --> METADATA[session_metadata.py] + METADATA --> SUMMARY[SessionSummary objects] + + KNOWLEDGE --> DIFF[diff_analyzer.py] + DIFF --> PROPOSALS[improvement_proposals.py] + PROPOSALS --> PRIORITIES[priority_rebalancer.py] ``` -### Pipeline 1: Harvester +## Entry Points -**Status:** Prompt designed. Script not implemented. +### Core Pipelines -Reads finished session transcripts (JSONL). Uses `templates/harvest-prompt.md` to extract durable knowledge into five categories: +| Script | Purpose | Key Functions | +|--------|---------|---------------| +| `harvester.py` | Extract knowledge from session transcripts | `harvest_session()`, `call_llm()`, `deduplicate()`, `validate_fact()` | +| `bootstrapper.py` | Build pre-session context from knowledge store | `build_bootstrap_context()`, `filter_facts()`, `sort_facts()`, `truncate_to_tokens()` | +| `session_reader.py` | Parse JSONL session transcripts | `read_session()`, `extract_conversation()`, `messages_to_text()` | +| `sampler.py` | Score and rank sessions for harvesting value | `scan_session_fast()`, `score_session()` | +| `session_metadata.py` | Extract structured metadata from sessions | `extract_session_metadata()`, `SessionSummary` | -| Category | Description | Example | -|----------|-------------|---------| -| `fact` | Concrete, verifiable information | "Repository X has 5 files" | -| `pitfall` | Errors encountered, wrong assumptions | "Token is at ~/.config/gitea/token, not env var" | -| `pattern` | Successful action sequences | "Deploy: test → build → push → webhook" | -| `tool-quirk` | Environment-specific behaviors | "URL format requires trailing slash" | -| `question` | Identified but unanswered | "Need optimal batch size for harvesting" | +### Analysis & Quality -Output schema per knowledge item: -```json -{ - "fact": "One sentence description", - "category": "fact|pitfall|pattern|tool-quirk|question", - "repo": "repo-name or 'global'", - "confidence": 0.0-1.0 -} -``` +| Script | Purpose | +|--------|---------| +| `validate_knowledge.py` | Validate knowledge index schema compliance | +| `knowledge_staleness_check.py` | Detect stale knowledge (source changed since extraction) | +| `knowledge_gap_identifier.py` | Find untested functions, undocumented APIs, missing tests | +| `diff_analyzer.py` | Analyze code diffs for improvement signals | +| `improvement_proposals.py` | Generate ranked improvement proposals | +| `priority_rebalancer.py` | Rebalance priorities across proposals | +| `automation_opportunity_finder.py` | Find manual steps that can be automated | +| `dead_code_detector.py` | Detect unused code | +| `dependency_graph.py` | Map dependency relationships | +| `perf_bottleneck_finder.py` | Find performance bottlenecks | +| `refactoring_opportunity_finder.py` | Identify refactoring targets | +| `gitea_issue_parser.py` | Parse Gitea issues for knowledge extraction | -### Pipeline 2: Bootstrapper +### Automation -**Status:** Not implemented. +| Script | Purpose | +|--------|---------| +| `session_pair_harvester.py` | Extract training pairs from sessions | -Queries knowledge store before session start. Assembles a compact 2k-token context from relevant facts. Injects into session startup so the agent begins with full situational awareness. - -### Pipeline 3: Measurer - -**Status:** Not implemented. - -Tracks compounding metrics: knowledge velocity (facts/day), error reduction (%), hit rate (knowledge used / knowledge available), task completion improvement. - ---- - -## Directory Structure +## Data Flow ``` -compounding-intelligence/ -├── README.md # Project overview and architecture -├── GENOME.md # This file (codebase genome) -├── knowledge/ # [PLANNED] Knowledge store -│ ├── index.json # Machine-readable fact index -│ ├── global/ # Cross-repo knowledge -│ ├── repos/{repo}.md # Per-repo knowledge -│ └── agents/{agent}.md # Agent-type notes -├── scripts/ -│ ├── test_harvest_prompt.py # Basic prompt validation (2.5KB) -│ └── test_harvest_prompt_comprehensive.py # Full prompt structure test (6.8KB) -├── templates/ -│ └── harvest-prompt.md # Knowledge extraction prompt (3.5KB) -├── test_sessions/ -│ ├── session_success.jsonl # Happy path test data -│ ├── session_failure.jsonl # Failure path test data -│ ├── session_partial.jsonl # Incomplete session test data -│ ├── session_patterns.jsonl # Pattern extraction test data -│ └── session_questions.jsonl # Question identification test data -└── metrics/ # [PLANNED] Compounding metrics - └── dashboard.md +1. Session ends → .jsonl written to ~/.hermes/sessions/ +2. sampler.py scores sessions by age, recency, repo coverage +3. harvester.py reads top sessions, calls LLM with harvest-prompt.md +4. LLM extracts facts/pitfalls/patterns/quirks/questions +5. deduplicate() checks against existing index via fact_fingerprint() +6. validate_fact() checks schema compliance +7. write_knowledge() appends to knowledge/index.json + per-repo YAML +8. On next session start, bootstrapper.py: + a. Loads knowledge/index.json + b. Filters by session's repo and agent type + c. Sorts by confidence (high first), then recency + d. Truncates to 2k token budget + e. Injects as pre-context +9. Agent starts with full situational awareness instead of zero ``` ---- - -## Entry Points and Data Flow - -### Entry Point 1: Knowledge Extraction (Harvester) - -``` -Input: Session transcript (JSONL) - ↓ - templates/harvest-prompt.md (LLM prompt) - ↓ - Knowledge items (JSON array) - ↓ -Output: knowledge/index.json + per-repo/per-agent markdown files -``` - -### Entry Point 2: Session Bootstrap (Bootstrapper) - -``` -Input: Session context (repo, agent type, task type) - ↓ - knowledge/index.json (query relevant facts) - ↓ - 2k-token bootstrap context - ↓ -Output: Injected into session startup -``` - -### Entry Point 3: Measurement (Measurer) - -``` -Input: knowledge/index.json + session history - ↓ - Velocity, hit rate, error reduction calculations - ↓ -Output: metrics/dashboard.md -``` - ---- - ## Key Abstractions -### Knowledge Item -The atomic unit. One sentence, one category, one confidence score. Designed to be small enough that 1000 items fit in a 2k-token bootstrap context. +### Knowledge Item (fact/pitfall/pattern/quirk/question) +```json +{ + "fact": "Gitea token is at ~/.config/gitea/token", + "category": "tool-quirk", + "repo": "global", + "confidence": 0.9, + "evidence": "Found during clone attempt", + "source_session": "2026-04-13_abc123", + "extracted_at": "2026-04-13T20:00:00Z" +} +``` -### Knowledge Store -A directory structure that mirrors the fleet's mental model: -- `global/` — knowledge that applies everywhere (tool quirks, environment facts) -- `repos/` — knowledge specific to each repo -- `agents/` — knowledge specific to each agent type +### SessionSummary (session_metadata.py) +Extracted metadata per session: duration, token count, tools used, repos touched, error count, outcome. -### Confidence Score -0.0–1.0 scale. Defines how certain the harvester is about each extracted fact: -- 0.9–1.0: Explicitly stated with verification -- 0.7–0.8: Clearly implied by multiple data points -- 0.5–0.6: Suggested but not fully verified -- 0.3–0.4: Inferred from limited data -- 0.1–0.2: Speculative or uncertain +### Gap / GapReport (knowledge_gap_identifier.py) +Structured gap analysis: untested functions, undocumented APIs, missing tests. Severity: critical/high/medium/low. -### Bootstrap Context -The 2k-token injection that a new session receives. Assembled from the most relevant knowledge items for the current task, filtered by confidence > 0.7, deduplicated, and compressed. +### Knowledge Index (knowledge/index.json) +Machine-readable fact store. 12KB, populated with real data. Categories: fact, pitfall, pattern, tool-quirk, question. ---- +## Knowledge Store + +``` +knowledge/ +├── index.json # Master fact store (12KB, populated) +├── SCHEMA.md # Schema documentation +├── global/ +│ ├── pitfalls.yaml # Cross-repo pitfalls (2KB) +│ └── tool-quirks.yaml # Tool-specific quirks (2KB) +├── repos/ +│ ├── hermes-agent.yaml # hermes-agent knowledge (2KB) +│ └── the-nexus.yaml # the-nexus knowledge (2KB) +└── agents/ # Per-agent knowledge (empty) +``` ## API Surface -### Internal (scripts not yet implemented) +### LLM API (consumed) +| Provider | Endpoint | Usage | +|----------|----------|-------| +| Nous Research | `https://inference-api.nousresearch.com/v1` | Knowledge extraction | +| Ollama | `http://localhost:11434/v1` | Local fallback | -| Script | Input | Output | Status | -|--------|-------|--------|--------| -| `harvester.py` | Session JSONL path | Knowledge items JSON | PLANNED | -| `bootstrapper.py` | Repo + agent type | 2k-token context string | PLANNED | -| `measurer.py` | Knowledge store path | Metrics JSON | PLANNED | -| `session_reader.py` | Session JSONL path | Parsed transcript | PLANNED | - -### Prompt (templates/harvest-prompt.md) - -The extraction prompt is the core "API." It takes a session transcript and returns structured JSON. It defines: -- Five extraction categories -- Output format (JSON array of knowledge items) -- Confidence scoring rubric -- Constraints (no hallucination, specificity, relevance, brevity) -- Example input/output pair - ---- +### File API (consumed/produced) +| Path | Format | Direction | +|------|--------|-----------| +| `~/.hermes/sessions/*.jsonl` | JSONL | Input (session transcripts) | +| `knowledge/index.json` | JSON | Output (master fact store) | +| `knowledge/global/*.yaml` | YAML | Output (cross-repo knowledge) | +| `knowledge/repos/*.yaml` | YAML | Output (per-repo knowledge) | +| `templates/harvest-prompt.md` | Markdown | Config (extraction prompt) | ## Test Coverage -### What Exists +**14 test files** covering core pipelines: -| File | Tests | Coverage | -|------|-------|----------| -| `scripts/test_harvest_prompt.py` | 2 tests | Prompt file existence, sample transcript | -| `scripts/test_harvest_prompt_comprehensive.py` | 5 tests | Prompt structure, categories, fields, confidence scoring, size limits | -| `test_sessions/*.jsonl` | 5 sessions | Success, failure, partial, patterns, questions | +| Test File | Covers | +|-----------|--------| +| `test_harvest_prompt.py` | Prompt validation, hallucination detection | +| `test_harvest_prompt_comprehensive.py` | Extended prompt testing | +| `test_harvester_pipeline.py` | Harvester extraction + dedup | +| `test_bootstrapper.py` | Context building, filtering, truncation | +| `test_session_pair_harvester.py` | Training pair extraction | +| `test_improvement_proposals.py` | Proposal generation | +| `test_priority_rebalancer.py` | Priority scoring | +| `test_knowledge_staleness.py` | Staleness detection | +| `test_automation_opportunity_finder.py` | Automation detection | +| `test_diff_analyzer.py` | Diff analysis | +| `test_gitea_issue_parser.py` | Issue parsing | +| `test_refactoring_opportunity_finder.py` | Refactoring signals | +| `test_knowledge_gap_identifier.py` | Gap analysis | +| `test_perf_bottleneck_finder.py` | Perf bottleneck detection | -### What's Missing +### Coverage Gaps -1. **Harvester integration test** — Does the prompt actually extract correct knowledge from real transcripts? -2. **Bootstrapper test** — Does it assemble relevant context correctly? -3. **Knowledge store test** — Does the index.json maintain consistency? -4. **Confidence calibration test** — Do high-confidence facts actually prove true in later sessions? -5. **Deduplication test** — Are duplicate facts across sessions handled? -6. **Staleness test** — How does the system handle outdated knowledge? - ---- +1. **session_reader.py** — No dedicated test file (tested indirectly) +2. **sampler.py** — No test file (scoring logic untested) +3. **session_metadata.py** — No test file +4. **validate_knowledge.py** — No test file +5. **knowledge_staleness_check.py** — Tested but limited ## Security Considerations -1. **No secrets in knowledge store** — The harvester must filter out API keys, tokens, and credentials from extracted facts. The prompt constraints mention this but there is no automated guard. +### API Key Handling +- `harvester.py` reads API key from `~/.hermes/auth.json` or env vars +- Key passed to LLM API in request headers only +- No key logging -2. **Knowledge poisoning** — A malicious or corrupted session could inject false facts. Confidence scoring partially mitigates this, but there is no verification step. +### Knowledge Integrity +- `validate_fact()` checks schema before writing +- `deduplicate()` prevents duplicate entries via fingerprint +- `knowledge_staleness_check.py` detects when source code changed but knowledge didn't +- Confidence scores prevent low-quality knowledge from polluting the store -3. **Access control** — The knowledge store has no access control. Any process that can read the directory can read all facts. In a multi-tenant setup, this is a concern. +### File Safety +- Knowledge writes are append-only (never deletes) +- Bootstrap context is truncated to budget (no prompt injection via knowledge) +- Session reader handles malformed JSONL gracefully -4. **Transcript privacy** — Session transcripts may contain user data. The harvester must not extract personally identifiable information into the knowledge store. +## File Index + +``` +scripts/ + harvester.py (473 lines) — Core knowledge extraction + bootstrapper.py (302 lines) — Pre-session context builder + session_reader.py (137 lines) — JSONL session parser + sampler.py (363 lines) — Session scoring + ranking + session_metadata.py (271 lines) — Session metadata extraction + validate_knowledge.py (44 lines) — Index validation + knowledge_staleness_check.py (125 lines) — Staleness detection + knowledge_gap_identifier.py (291 lines) — Gap analysis engine + diff_analyzer.py (203 lines) — Diff analysis + improvement_proposals.py (518 lines) — Proposal generation + priority_rebalancer.py (745 lines) — Priority scoring + automation_opportunity_finder.py (600 lines) — Automation detection + dead_code_detector.py (270 lines) — Dead code detection + dependency_graph.py (220 lines) — Dependency mapping + perf_bottleneck_finder.py (635 lines) — Perf analysis + refactoring_opportunity_finder.py (46 lines) — Refactoring signals + gitea_issue_parser.py (140 lines) — Gitea issue parsing + session_pair_harvester.py (224 lines) — Training pair extraction +knowledge/ + index.json (12KB) — Master fact store + SCHEMA.md (3KB) — Schema docs + global/pitfalls.yaml (2KB) — Cross-repo pitfalls + global/tool-quirks.yaml (2KB) — Tool quirks + repos/hermes-agent.yaml (2KB) — Repo-specific knowledge + repos/the-nexus.yaml (2KB) — Repo-specific knowledge +templates/ + harvest-prompt.md (4KB) — Extraction prompt +test_sessions/ (5 files) — Sample transcripts +tests/ + scripts/test_* (14 files)— Test suite +``` + +**Total:** ~6,500 lines of code across 18 scripts + 14 test files. --- -## The 100x Path (from README) - -``` -Month 1: 15,000 facts, sessions 20% faster -Month 2: 45,000 facts, sessions 40% faster, first-try success up 30% -Month 3: 90,000 facts, fleet measurably smarter per token -``` - -Each new session is better than the last. The intelligence compounds. - ---- - -*Generated by codebase-genome pipeline. Ref: timmy-home#676.* +*Generated by Codebase Genome pipeline — Issue #676*