Alexander Payne c0dc4052a3 feat: add memory bakeoff runner for baseline vs MemPalace vs Hindsight evaluation ...

Implements issue #230 by creating:
- prompts/matrix.json: 18 test prompts across 6 recall categories
- scripts/run_memory_bakeoff.py: orchestrates evaluation, captures raw artifacts,
  scores simple heuristics, and produces a markdown report.

Backends:
- Baseline: knowledge/index.json bootstrap (keyword-match retrieval)
- MemPalace: via nexus.mempalace.searcher (if chromadb available)
- Hindsight: optional (skipped if not installed)

Accepts CLI options for matrix, category, limit, model, and dry-run.
Captures context and answers for downstream manual review.

Closes #230

2026-04-29 18:00:00 -04:00

..

.gitkeep

Initial structure: knowledge store, scripts, metrics, templates

2026-04-14 11:17:01 -04:00

automation_opportunity_finder.py

fix: add directory exclusions for scan performance (#170)

2026-04-15 15:06:09 +00:00

bootstrapper.py

feat: build bootstrapper.py - pre-session context assembler

2026-04-14 14:05:30 -04:00

dead_code_detector.py

feat: dead code detector for Python codebases (#94)

2026-04-15 03:46:43 +00:00

dedup.py

feat: knowledge deduplication — content hash + token similarity (#196)

2026-04-21 07:58:09 -04:00

dependency_graph.py

fix: escape quotes in DOT renderer (#212)

2026-04-21 11:20:25 +00:00

diff_analyzer.py

feat: Add diff analyzer (closes #176)

2026-04-15 03:56:27 +00:00

freshness.py

feat: Add freshness.py (#200)

2026-04-21 11:57:53 +00:00

gitea_issue_parser.py

feat: Add Gitea issue body parser (closes #177)

2026-04-15 03:49:00 +00:00

harvester.py

feat: add harvester.py — session knowledge extractor (#8)

2026-04-14 14:03:30 -04:00

improvement_proposals.py

feat: add improvement proposal generator (#168)

2026-04-15 14:47:26 +00:00

knowledge_gap_identifier.py

feat: knowledge gap identifier — Pipeline 10.7 (#172)

2026-04-15 14:42:28 +00:00

knowledge_staleness_check.py

feat: Add knowledge staleness detector (closes #179)

2026-04-15 04:00:12 +00:00

perf_bottleneck_finder.py

fix: syntax errors in perf_bottleneck_finder.py #211

2026-04-21 11:21:58 +00:00

pr_complexity_scorer.py

feat: add PR complexity scorer — estimate review effort\n\nImplements issue #135: a script that analyzes open PRs and computes\na complexity score (1-10) based on files changed, lines added/removed,\ndependency changes, and test coverage delta. Also estimates review time.\n\nThe scorer can be run with --dry-run to preview or --apply to post\nscore comments directly on PRs.\n\nOutput: metrics/pr_complexity.json with full analysis.\n\nCloses #135

2026-04-26 09:34:57 -04:00

priority_rebalancer.py

feat: Priority Rebalancer — re-score issues from pipeline data (#174)

2026-04-15 10:52:51 -04:00

refactoring_opportunity_finder.py

fix: implement refactoring_opportunity_finder API (#210)

2026-04-21 07:29:44 -04:00

run_memory_bakeoff.py

feat: add memory bakeoff runner for baseline vs MemPalace vs Hindsight evaluation

2026-04-29 18:00:00 -04:00

sampler.py

feat: add sampler.py — session value scorer (#17)

2026-04-15 03:02:12 +00:00

session_metadata.py

feat: add session_metadata.py — structured session metadata extractor (#6)

2026-04-14 19:06:16 +00:00

session_pair_harvester.py

feat: session transcript → training pair harvester (#91)

2026-04-15 03:39:08 +00:00

session_reader.py

feat: add harvester.py — session knowledge extractor (#8)

2026-04-14 14:03:30 -04:00

test_automation_opportunity_finder.py

test: automation opportunity finder tests (#170)

2026-04-15 14:53:43 +00:00

test_bootstrapper.py

feat: build bootstrapper.py - pre-session context assembler

2026-04-14 14:05:30 -04:00

test_diff_analyzer.py

feat: Add diff analyzer tests (closes #176)

2026-04-15 03:57:21 +00:00

test_gitea_issue_parser.py

feat: Add parser tests (closes #177)

2026-04-15 03:50:04 +00:00

test_harvest_prompt_comprehensive.py

fix(#201): rewrite comprehensive tests with proper pytest-compatible functions

2026-04-17 05:17:40 +00:00

test_harvest_prompt.py

feat: build bootstrapper.py - pre-session context assembler

2026-04-14 14:05:30 -04:00

test_harvester_pipeline.py

feat: add harvester.py — session knowledge extractor (#8)

2026-04-14 14:03:30 -04:00

test_improvement_proposals.py

test: add improvement proposal generator tests (#168)

2026-04-15 14:47:30 +00:00

test_knowledge_staleness.py

feat: Add staleness detector tests (closes #179)

2026-04-15 04:00:46 +00:00

test_pr_complexity_scorer.py

feat: add PR complexity scorer — estimate review effort\n\nImplements issue #135: a script that analyzes open PRs and computes\na complexity score (1-10) based on files changed, lines added/removed,\ndependency changes, and test coverage delta. Also estimates review time.\n\nThe scorer can be run with --dry-run to preview or --apply to post\nscore comments directly on PRs.\n\nOutput: metrics/pr_complexity.json with full analysis.\n\nCloses #135

2026-04-26 09:34:57 -04:00

test_priority_rebalancer.py

feat: Priority Rebalancer — re-score issues from pipeline data (#174)

2026-04-15 10:52:51 -04:00

test_refactoring_opportunity_finder.py

feat: Add refactoring opportunity finder (#169)

2026-04-15 10:54:58 -04:00

test_session_pair_harvester.py

test: add tests for session pair harvester (#91)

2026-04-15 03:39:09 +00:00

validate_knowledge.py

feat(#10): knowledge file format schema + example knowledge files

2026-04-14 14:21:21 -04:00