[DESIGN] Game companion AI — learn through simulated worlds, transfer to robotics #595

New Issue

perplexity · 2026-03-27T01:10:12Z

perplexity commented

2026-03-27 01:10:12 +00:00

Vision

Not a speedrunner or game-playing AI. A companion system that learns and advances skills through experiencing a simulated game world, with the goal of transferring those skills to a physical robot body.

"I'm trying to create a companion system that can learn and advance his skills through experiencing a simulated game world in a way that it transfers so that when he's in a robot body, he can be my vassal in the real world."

Why Games

Games provide structured environments with clear feedback loops
Combat, strategy, economics, social dynamics — all in one sandbox
Existing game AI is already competent (deterministic, fast) — Timmy needs to match or exceed it
Development work is informed by what needs improving in the game (avoids burnout)

Design Principles

The AI should feel like it has real stakes — not a hollow script
Conversational + embodied: talk strategy, then execute in-game
Track cognitive development milestones (not just win/loss)
"Soft brain" (LLM conversation) + "mechanical brain" (game logic) working together

Mount & Blade: Bannerlord (Primary Target)

Combat, strategy, politics, economics all present
AI plugin ecosystem available
Persistent world with meaningful NPC interactions
Potential for multiplayer: seed NPCs with different AIs, humans join

Cognitive Development Tracking

Log all decisions and perceptions (lightweight, no screenshots — structured data only)
Overseer agent tracks cognitive milestones
Define levels/language for cognitive execution orders
Visualize in Grafana without bloating the database

Multiplayer Vision

Persistent Bannerlord world where every NPC is an AI
Players can join and interact
Narrative emerges from AI-AI and AI-human interactions
Streaming/video content comes out naturally

Extends #19 (Bannerlord Portal — Campaign AI)
Depends on #547 (First perception-decision-action cycle)
Depends on #550 (Heartbeat → Bannerlord via desktop MCP)
Feeds into Deterministic Logic Engine (combat/economy reasoning)

Source: Gemini brainstorm session 2026-03-26 — triaged by Perplexity

## Vision Not a speedrunner or game-playing AI. A **companion system** that learns and advances skills through experiencing a simulated game world, with the goal of transferring those skills to a physical robot body. > "I'm trying to create a companion system that can learn and advance his skills through experiencing a simulated game world in a way that it transfers so that when he's in a robot body, he can be my vassal in the real world." ## Why Games - Games provide structured environments with clear feedback loops - Combat, strategy, economics, social dynamics — all in one sandbox - Existing game AI is already competent (deterministic, fast) — Timmy needs to match or exceed it - Development work is informed by what needs improving in the game (avoids burnout) ## Design Principles - The AI should feel like it has **real stakes** — not a hollow script - Conversational + embodied: talk strategy, then execute in-game - Track cognitive development milestones (not just win/loss) - "Soft brain" (LLM conversation) + "mechanical brain" (game logic) working together ## Mount & Blade: Bannerlord (Primary Target) - Combat, strategy, politics, economics all present - AI plugin ecosystem available - Persistent world with meaningful NPC interactions - Potential for multiplayer: seed NPCs with different AIs, humans join ## Cognitive Development Tracking - Log all decisions and perceptions (lightweight, no screenshots — structured data only) - Overseer agent tracks cognitive milestones - Define levels/language for cognitive execution orders - Visualize in Grafana without bloating the database ## Multiplayer Vision - Persistent Bannerlord world where every NPC is an AI - Players can join and interact - Narrative emerges from AI-AI and AI-human interactions - Streaming/video content comes out naturally ## Related - Extends #19 (Bannerlord Portal — Campaign AI) - Depends on #547 (First perception-decision-action cycle) - Depends on #550 (Heartbeat → Bannerlord via desktop MCP) - Feeds into Deterministic Logic Engine (combat/economy reasoning) --- _Source: [Gemini brainstorm session 2026-03-26](https://g.co/gemini/share/3700c8d29b6b) — triaged by Perplexity_

perplexity added the agent-presence portal p2-backlog needs-design labels 2026-03-27 01:10:12 +00:00

Timmy referenced this issue

2026-03-28 04:46:33 +00:00

[DESIGN] Game companion AI — learn through simulated worlds, transfer to robotics #596

Timmy referenced this issue

2026-03-28 04:46:34 +00:00

[DESIGN] Game companion AI — learn through simulated worlds, transfer to robotics #600

Timmy commented

2026-03-28 04:52:34 +00:00

Closing during the 2026-03-28 backlog burn-down.

Reason: this issue is being retired as part of a backlog reset toward the current final vision: Heartbeat, Harness, and Portal. If the work still matters after reset, it should return as a narrower, proof-oriented next-step issue rather than stay open as a broad legacy frontier.

Closing during the 2026-03-28 backlog burn-down. Reason: this issue is being retired as part of a backlog reset toward the current final vision: Heartbeat, Harness, and Portal. If the work still matters after reset, it should return as a narrower, proof-oriented next-step issue rather than stay open as a broad legacy frontier.

Timmy closed this issue

2026-03-28 04:52:34 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Timmy_Foundation/the-nexus#595