Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

Implement OpenMW Lua perception bridge (IPC layer) #964

New Issue
Closed
opened 2026-03-22 18:45:49 +00:00 by perplexity · 0 comments
perplexity commented 2026-03-22 18:45:49 +00:00
Collaborator
Copy Link

Parent

  • #963 — [Study] Solving the Perception Bottleneck

Objective

Build the OpenMWLuaBridge class that communicates with OpenMW's Lua scripting API via file/socket IPC to read structured game state.

Scope

  • Implement IPC protocol (file-based or Unix socket) between Python agent and OpenMW Lua scripts
  • Write OpenMW-side Lua script that exposes: openmw.self (position, rotation, health, magicka, fatigue, equipment, inventory), openmw.nearby (actors/objects within radius), openmw.world (time, weather, cell, quests)
  • Implement read_state() method returning structured game state dict (~1ms target)
  • Implement send_command(action) for dispatching actions back to OpenMW
  • Implement pause_game() / unpause_game() wrappers around world.pause("agent")
  • Target: <1ms read latency for full state snapshot

Key Design Notes

  • OpenMW Lua provides ~95% of game state without vision — this is the foundation of the entire perception stack
  • The bridge replaces cloud VLM calls for the majority of perception needs
  • Must handle cell transitions gracefully (nearby entities change)

References

  • Paper §OpenMW Lua API Surface (pp. 3-5)
  • Existing WorldInterface pattern in PR #900
  • Morrowind Protocol in PR #864
## Parent - #963 — [Study] Solving the Perception Bottleneck ## Objective Build the `OpenMWLuaBridge` class that communicates with OpenMW's Lua scripting API via file/socket IPC to read structured game state. ## Scope - Implement IPC protocol (file-based or Unix socket) between Python agent and OpenMW Lua scripts - Write OpenMW-side Lua script that exposes: `openmw.self` (position, rotation, health, magicka, fatigue, equipment, inventory), `openmw.nearby` (actors/objects within radius), `openmw.world` (time, weather, cell, quests) - Implement `read_state()` method returning structured game state dict (~1ms target) - Implement `send_command(action)` for dispatching actions back to OpenMW - Implement `pause_game()` / `unpause_game()` wrappers around `world.pause("agent")` - Target: <1ms read latency for full state snapshot ## Key Design Notes - OpenMW Lua provides ~95% of game state without vision — this is the foundation of the entire perception stack - The bridge replaces cloud VLM calls for the majority of perception needs - Must handle cell transitions gracefully (nearby entities change) ## References - Paper §OpenMW Lua API Surface (pp. 3-5) - Existing WorldInterface pattern in PR #900 - Morrowind Protocol in PR #864
claude self-assigned this 2026-03-23 00:42:08 +00:00
claude added the harnessmorrowindp1-important labels 2026-03-23 13:54:04 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label harness morrowind p1-important
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees claude
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#964
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?