[Bridge] TES3MP↔Python Agent Bridge — The Critical Path Component #878

New Issue

perplexity · 2026-03-21T23:40:07Z

perplexity commented

2026-03-21 23:40:07 +00:00

Why This Is High Leverage

Every document identifies this as the single most critical missing piece. The feasibility guide calls it "must build (pattern exists)." The V2 roadmap calls it "the critical path item for Phase 8" at 2-6 weeks. No Python↔TES3MP bridge exists. Without it, Timmy cannot interact with the Morrowind world. With it, everything else (heartbeat gameplay, NPC drop-in, content pipeline) becomes possible.

Scope

Build a bidirectional bridge: FastAPI (Python) ↔ TCP socket ↔ TES3MP Lua server script ↔ Game state.

Server Side (Lua)

TES3MP server script that loads luasocket
Listens on TCP port for commands from Python
Translates commands into TES3MP API calls (21 API categories available)
Pushes perception data (position, inventory, journal, nearby actors, quest progress) on request or event

Python Side

Async TCP client connecting to the Lua server
Implements WorldInterface (observe/act/speak) for the Gymnasium pattern
Handles connection lifecycle, reconnection, timeout

What Server Lua CAN Do (direct execution)

Teleport to coordinates
Add/remove inventory
Advance quest journal
Set stats, class, race
Execute console commands

What Needs Client Input (stub for now)

Walk/run/swim
Talk to NPC
Combat (attack, cast, block)
Open doors/containers

Requirements

scripts/tes3mp/timmy_bridge.lua — Server-side Lua script for TES3MP
src/infrastructure/morrowind/bridge.py — Python TCP client
src/infrastructure/world/adapters/tes3mp.py — WorldInterface implementation using the bridge
Connection health monitoring and auto-reconnect
Tests with mock TCP server

Acceptance Criteria

Python process sends "teleport to Balmora" → Lua script moves player
Lua script sends perception snapshot → Python receives structured PerceptionOutput
Connection drops and reconnects gracefully
Works with TES3MP 0.8.1 in Docker

Prior Art

Community Discord bots, IRC bridges using this pattern
Existing Rust REST API for TES3MP (minimal)
Immersive Morrowind LLM AI (drzdo, archived): proved MWSE Lua → TCP → Python → LLM works

Assignee: Kimi

## Why This Is High Leverage Every document identifies this as the single most critical missing piece. The feasibility guide calls it "must build (pattern exists)." The V2 roadmap calls it "the critical path item for Phase 8" at 2-6 weeks. No Python↔TES3MP bridge exists. Without it, Timmy cannot interact with the Morrowind world. With it, everything else (heartbeat gameplay, NPC drop-in, content pipeline) becomes possible. ## Scope Build a bidirectional bridge: FastAPI (Python) ↔ TCP socket ↔ TES3MP Lua server script ↔ Game state. ### Server Side (Lua) - TES3MP server script that loads `luasocket` - Listens on TCP port for commands from Python - Translates commands into TES3MP API calls (21 API categories available) - Pushes perception data (position, inventory, journal, nearby actors, quest progress) on request or event ### Python Side - Async TCP client connecting to the Lua server - Implements `WorldInterface` (observe/act/speak) for the Gymnasium pattern - Handles connection lifecycle, reconnection, timeout ### What Server Lua CAN Do (direct execution) - Teleport to coordinates - Add/remove inventory - Advance quest journal - Set stats, class, race - Execute console commands ### What Needs Client Input (stub for now) - Walk/run/swim - Talk to NPC - Combat (attack, cast, block) - Open doors/containers ## Requirements - [ ] `scripts/tes3mp/timmy_bridge.lua` — Server-side Lua script for TES3MP - [ ] `src/infrastructure/morrowind/bridge.py` — Python TCP client - [ ] `src/infrastructure/world/adapters/tes3mp.py` — WorldInterface implementation using the bridge - [ ] Connection health monitoring and auto-reconnect - [ ] Tests with mock TCP server ## Acceptance Criteria - Python process sends "teleport to Balmora" → Lua script moves player - Lua script sends perception snapshot → Python receives structured PerceptionOutput - Connection drops and reconnects gracefully - Works with TES3MP 0.8.1 in Docker ## Prior Art - Community Discord bots, IRC bridges using this pattern - Existing Rust REST API for TES3MP (minimal) - Immersive Morrowind LLM AI (drzdo, archived): proved MWSE Lua → TCP → Python → LLM works ## Assignee: Kimi

perplexity referenced this issue

2026-03-22 11:53:07 +00:00

Incase morrowind isn't your first choice timmy #896

perplexity referenced this issue

2026-03-22 12:07:24 +00:00

[Study] Open-Source Morrowind Alternatives — AI Wizard Companion Feasibility Research #901

perplexity referenced this issue

2026-03-22 12:07:24 +00:00

[Study] Open-Source Games for AI-Human Co-Op — 40+ Game Evaluation & Top 5 Rankings #902

perplexity referenced this issue

2026-03-22 13:00:27 +00:00

[Study] State-of-the-Art Open Source for Sovereign Creative AI Agents — Q1 2026 Landscape Survey #903