[Sovereignty P0] PerceptionCache — Template Matching for VLM Replacement #955

New Issue

perplexity · 2026-03-22T18:23:02Z

perplexity commented

2026-03-22 18:23:02 +00:00

Parent

Part of #953 (The Sovereignty Loop) — P0 priority

Governing Principle

"See Once, Template Forever" — Eliminates 90% of VLM cost within hours.

Goal

Build the perception cache that intercepts VLM calls. First encounter triggers full VLM analysis; every subsequent encounter uses OpenCV template matching at 3ms instead of 3-6 seconds.

Implementation

Create src/timmy/sovereignty/perception_cache.py:
- PerceptionCache class with:
  - match(screenshot) -> CacheResult (confidence + cached state)
  - add(templates) — store new OpenCV templates from crystallization
  - persist() — write to disk
- Confidence threshold: 0.85 (configurable)
Create crystallize_perception(screenshot, vlm_response) -> list[Template]:
- Extracts reusable patterns from VLM output
- Generates OpenCV templates: element type, bounding box, color ranges
- Writes to data/templates.json
Integration pattern (from the governing doc):

async def sovereign_perceive(screenshot, cache, vlm):
    cached = cache.match(screenshot)
    if cached.confidence > 0.85:
        metrics.record("perception_cache_hit")
        return cached.state
    metrics.record("perception_vlm_call")
    raw = await vlm.analyze(screenshot)
    state = parse_game_state(raw)
    new_templates = crystallize(screenshot, state)
    cache.add(new_templates)
    cache.persist()
    metrics.record("skills_crystallized", len(new_templates))
    return state

Metrics Targets

90% perception sovereignty within first hour of play
99% by hour four

Effort Estimate

2 days

Cross-references

#953 (Sovereignty Loop — Section III.1)
#871 (WorldInterface — cache should be per-game via portal adapter)
#819 (Perception Script — this replaces raw VLM dependence)

## Parent Part of #953 (The Sovereignty Loop) — P0 priority ## Governing Principle > "See Once, Template Forever" — Eliminates 90% of VLM cost within hours. ## Goal Build the perception cache that intercepts VLM calls. First encounter triggers full VLM analysis; every subsequent encounter uses OpenCV template matching at 3ms instead of 3-6 seconds. ## Implementation 1. Create `src/timmy/sovereignty/perception_cache.py`: - `PerceptionCache` class with: - `match(screenshot) -> CacheResult` (confidence + cached state) - `add(templates)` — store new OpenCV templates from crystallization - `persist()` — write to disk - Confidence threshold: 0.85 (configurable) 2. Create `crystallize_perception(screenshot, vlm_response) -> list[Template]`: - Extracts reusable patterns from VLM output - Generates OpenCV templates: element type, bounding box, color ranges - Writes to `data/templates.json` 3. Integration pattern (from the governing doc): ```python async def sovereign_perceive(screenshot, cache, vlm): cached = cache.match(screenshot) if cached.confidence > 0.85: metrics.record("perception_cache_hit") return cached.state metrics.record("perception_vlm_call") raw = await vlm.analyze(screenshot) state = parse_game_state(raw) new_templates = crystallize(screenshot, state) cache.add(new_templates) cache.persist() metrics.record("skills_crystallized", len(new_templates)) return state ``` ## Metrics Targets - 90% perception sovereignty within first hour of play - 99% by hour four ## Effort Estimate 2 days ## Cross-references - #953 (Sovereignty Loop — Section III.1) - #871 (WorldInterface — cache should be per-game via portal adapter) - #819 (Perception Script — this replaces raw VLM dependence)

perplexity referenced this issue

2026-03-22 18:23:04 +00:00

[GOVERNING] The Sovereignty Loop — Falsework-Native Architecture for Autonomous Game Agents #953

perplexity referenced this issue

2026-03-22 18:45:49 +00:00

Implement three-tier metabolic LLM router (Qwen3-3B / Llama-8B / Qwen3-32B) #966

perplexity referenced this issue

2026-03-22 18:46:45 +00:00

[Study] Solving the Perception Bottleneck — API-First Architecture on Apple Silicon #963

perplexity referenced this issue

2026-03-22 19:08:53 +00:00

[P1] Build semantic index for research outputs (nomic-embed-text + SQLite) #976

gemini was assigned by Rockachopa

2026-03-22 23:31:25 +00:00

claude added the harness heartbeat p0-critical labels 2026-03-23 13:52:52 +00:00

Timmy added the kimi-ready label 2026-03-23 23:19:27 +00:00

Timmy commented

2026-03-23 23:19:29 +00:00

🤖 Vassal dispatch → routed to Kimi