[Memory P5] Memory performance budget enforcement #256

Closed
opened 2026-04-09 00:42:33 +00:00 by Rockachopa · 0 comments
Owner

Context

Memory retrieval runs before every turn. If it's slow, every interaction feels slow. Need strict performance budgets.

Task

  1. Define budgets:
    • Total prefetch latency: <200ms
    • Tier 1 (semantic): <50ms
    • Tier 2 (operational): <10ms (file read)
    • Tier 3 (episodic probe): <10ms (FTS5 count)
    • Total injected tokens: <2000
  2. Measure actual latency with all tiers active
  3. Optimize: parallel prefetch across tiers
  4. Token budget allocation: semantic 1000, operational 500, episodic 500
  5. Enforce budgets with timeouts and token truncation

Acceptance Criteria

  • All tiers complete prefetch in <200ms combined
  • Injected context never exceeds 2000 tokens
  • Performance metrics logged per turn
  • Graceful degradation if a tier is slow (skip it, don't block)

Part of: [EPIC] Unified Memory Architecture

## Context Memory retrieval runs before every turn. If it's slow, every interaction feels slow. Need strict performance budgets. ## Task 1. Define budgets: - Total prefetch latency: <200ms - Tier 1 (semantic): <50ms - Tier 2 (operational): <10ms (file read) - Tier 3 (episodic probe): <10ms (FTS5 count) - Total injected tokens: <2000 2. Measure actual latency with all tiers active 3. Optimize: parallel prefetch across tiers 4. Token budget allocation: semantic 1000, operational 500, episodic 500 5. Enforce budgets with timeouts and token truncation ## Acceptance Criteria - All tiers complete prefetch in <200ms combined - Injected context never exceeds 2000 tokens - Performance metrics logged per turn - Graceful degradation if a tier is slow (skip it, don't block) Part of: [EPIC] Unified Memory Architecture
Rockachopa added this to the Unified Memory Architecture milestone 2026-04-09 00:42:33 +00:00
Rockachopa added the phase:5-integrationepic:memory labels 2026-04-09 00:42:34 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/hermes-agent#256