feat: Sovereign Memory Store — zero-API durable memory (SQLite + FTS5 + HRR) #380

perplexity · 2026-04-07T22:42:04Z

perplexity commented

2026-04-07 22:42:04 +00:00

Summary

The missing backbone of the zero-API memory story. Replaces the third-party mempalace CLI (and its ONNX dependency) with a self-contained store built on SQLite, FTS5, and Holographic Reduced Representations.

Zero external dependencies. Zero API calls. Zero cloud.

Closes the open gaps in epic #367:

Addresses #370 (MP-3) — sovereign store provides the durable backend that scratchpad.pys promote_to_palace()` can target instead of shelling out to the mempalace CLI
Implements #371 (MP-4) — quality-gated promotion with 4 heuristic gates

What's in the PR

`sovereign_store.py` (474 lines)

Self-contained memory store at ~/.hermes/palace/sovereign.db:

Feature	How
Keyword search	SQLite FTS5 with porter stemmer + unicode61 tokenizer
Semantic search	HRR phase vectors (SHA-256 deterministic, circular mean bundling)
Ranking fusion	Reciprocal Rank Fusion (RRF) merges keyword + semantic rankings
Trust lifecycle	boost/decay API — facts gain trust when retrieved, lose it when contradicted
Room organization	Maps to existing PalaceRoom model
numpy optional	Falls back to pure Python math if numpy is missing

The HRR approach (ported from hermes-agent/plugins/memory/holographic/) generates semantic vectors without any embedding model or API call — just SHA-256 hashing and trigonometry. Not as precise as transformer embeddings, but sufficient for memory retrieval and completely sovereign.

`promotion.py` (188 lines)

Quality-gated scratchpad → palace promotion (MP-4, #371):

Gate	What it checks
Length	Min 5 words, max 500
Structure	Rejects fragments, pure code, non-alphabetic content
Duplicate	FTS5 search + Jaccard word overlap > 0.8
Staleness	Notes older than 7 days need manual review

Also provides: evaluate_for_promotion() dry-run, promote_session_batch() for end-of-session cleanup, force=True override, and full audit logging in promotion_log table.

`tests/test_sovereign_store.py` (255 lines, 21 tests)

4 tests for HRR vector encoding/similarity
9 tests for store operations (CRUD, search, trust, rooms, stats)
8 tests for promotion gates (accept, reject-short, reject-duplicate, reject-stale, force, batch, audit log)

All 21 pass. All run against in-memory SQLite — no network, no fixtures.

How this fits the architecture

Existing:                          New:

scratchpad.py                      sovereign_store.py
  └── promote_to_palace()            └── SovereignStore
        └── shells out to               ├── store()
            `mempalace` CLI             ├── search() [FTS5 + HRR + RRF]
            (ONNX dependency)           ├── boost_trust() / decay_trust()
                                        └── log_promotion()
                                   
                                   promotion.py
                                     └── promote() [4 quality gates]
                                     └── evaluate_for_promotion() [dry-run]

Next step: wire scratchpad.py:promote_to_palace() to call promotion.promote() instead of shelling out to the CLI. That's a ~10 line change.

Dependencies eliminated

Before	After
`mempalace` pip package	SQLite (stdlib)
ONNX Runtime	SHA-256 + trig (stdlib)
numpy (required)	numpy (optional, graceful degradation)
`mempalace` CLI binary path hardcoded	No external binaries

Refs: #367 #370 #371

## Summary The missing backbone of the zero-API memory story. Replaces the third-party `mempalace` CLI (and its ONNX dependency) with a self-contained store built on SQLite, FTS5, and Holographic Reduced Representations. **Zero external dependencies. Zero API calls. Zero cloud.** Closes the open gaps in epic #367: - Addresses #370 (MP-3) — sovereign store provides the durable backend that scratchpad.py`s `promote_to_palace()` can target instead of shelling out to the mempalace CLI - Implements #371 (MP-4) — quality-gated promotion with 4 heuristic gates --- ## What's in the PR ### `sovereign_store.py` (474 lines) Self-contained memory store at `~/.hermes/palace/sovereign.db`: | Feature | How | |---------|-----| | Keyword search | SQLite FTS5 with porter stemmer + unicode61 tokenizer | | Semantic search | HRR phase vectors (SHA-256 deterministic, circular mean bundling) | | Ranking fusion | Reciprocal Rank Fusion (RRF) merges keyword + semantic rankings | | Trust lifecycle | boost/decay API — facts gain trust when retrieved, lose it when contradicted | | Room organization | Maps to existing PalaceRoom model | | numpy optional | Falls back to pure Python math if numpy is missing | The HRR approach (ported from `hermes-agent/plugins/memory/holographic/`) generates semantic vectors without any embedding model or API call — just SHA-256 hashing and trigonometry. Not as precise as transformer embeddings, but sufficient for memory retrieval and **completely sovereign**. ### `promotion.py` (188 lines) Quality-gated scratchpad → palace promotion (MP-4, #371): | Gate | What it checks | |------|----------------| | Length | Min 5 words, max 500 | | Structure | Rejects fragments, pure code, non-alphabetic content | | Duplicate | FTS5 search + Jaccard word overlap > 0.8 | | Staleness | Notes older than 7 days need manual review | Also provides: `evaluate_for_promotion()` dry-run, `promote_session_batch()` for end-of-session cleanup, `force=True` override, and full audit logging in `promotion_log` table. ### `tests/test_sovereign_store.py` (255 lines, 21 tests) - 4 tests for HRR vector encoding/similarity - 9 tests for store operations (CRUD, search, trust, rooms, stats) - 8 tests for promotion gates (accept, reject-short, reject-duplicate, reject-stale, force, batch, audit log) All 21 pass. All run against in-memory SQLite — no network, no fixtures. --- ## How this fits the architecture ``` Existing: New: scratchpad.py sovereign_store.py └── promote_to_palace() └── SovereignStore └── shells out to ├── store() `mempalace` CLI ├── search() [FTS5 + HRR + RRF] (ONNX dependency) ├── boost_trust() / decay_trust() └── log_promotion() promotion.py └── promote() [4 quality gates] └── evaluate_for_promotion() [dry-run] ``` Next step: wire `scratchpad.py:promote_to_palace()` to call `promotion.promote()` instead of shelling out to the CLI. That's a ~10 line change. --- ## Dependencies eliminated | Before | After | |--------|-------| | `mempalace` pip package | SQLite (stdlib) | | ONNX Runtime | SHA-256 + trig (stdlib) | | numpy (required) | numpy (optional, graceful degradation) | | `mempalace` CLI binary path hardcoded | No external binaries | Refs: #367 #370 #371

perplexity self-assigned this 2026-04-07 22:42:04 +00:00

perplexity added 1 commit 2026-04-07 22:42:05 +00:00

feat: sovereign memory store — zero-API durable memory (SQLite + FTS5 + HRR) 593621c5e0

Implements the missing pieces of the MemPalace epic (#367):

- sovereign_store.py: Self-contained memory store replacing the third-party
  mempalace CLI and its ONNX dependency. Uses:
  * SQLite + FTS5 for keyword search (porter stemmer, unicode61)
  * HRR phase vectors (SHA-256 deterministic, numpy optional) for semantic similarity
  * Reciprocal Rank Fusion to merge keyword and semantic rankings
  * Trust scoring with boost/decay lifecycle
  * Room-based organization matching the existing PalaceRoom model

- promotion.py (MP-4, #371): Quality-gated scratchpad-to-palace promotion.
  Four heuristic gates, no LLM call:
  1. Length gate (min 5 words, max 500)
  2. Structure gate (rejects fragments and pure code)
  3. Duplicate gate (FTS5 + Jaccard overlap detection)
  4. Staleness gate (7-day threshold for old notes)
  Includes force override, batch promotion, and audit logging.

- 21 unit tests covering HRR vectors, store operations, search,
  trust lifecycle, and all promotion gates.

Zero external dependencies. Zero API calls. Zero cloud.

Refs: #367 #370 #371