the-matrix/PROTOCOL.md

WebSocket Protocol Specification

Connection

ws://[host]:[port]/ws/world-state

The world connects to this endpoint on load. If the connection fails, it falls back to the built-in MockWebSocket which simulates agent activity.

Message Format

All messages are JSON objects with a type field that determines the message schema.

---

Server → World (Events)

These messages are sent from the agent backend to the 3D world.

agent_state

Updates an agent's current state and visual properties.

{
  "type": "agent_state",
  "agent_id": "timmy",
  "state": "working",
  "current_task": "Analyzing codebase",
  "glow_intensity": 0.8
}

Field	Type	Description
agent_id	string	One of: timmy, forge, seer, echo
state	string	One of: idle, working, waiting
current_task	string|null	Description of current task
glow_intensity	number	0.0 to 1.0 — controls visual glow brightness

task_created

A new task has been created. The world spawns a floating geometric object.

{
  "type": "task_created",
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "agent_id": "forge",
  "title": "Fix login bug",
  "status": "pending",
  "priority": "high"
}

Field	Type	Description
task_id	string (UUID)	Unique task identifier
agent_id	string|null	Assigned agent (null = unassigned)
title	string	Human-readable task name
status	string	pending, in_progress, completed, failed
priority	string	high, normal, low

task_update

An existing task's status has changed. The world updates the task object's color.

{
  "type": "task_update",
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "agent_id": "forge",
  "title": "Fix login bug",
  "status": "in_progress",
  "priority": "high"
}

Same fields as task_created. Color mapping:

• pending → white
• in_progress → amber/orange
• completed → green
• failed → red

memory_event

An agent has recorded a new memory.

{
  "type": "memory_event",
  "agent_id": "seer",
  "content": "Detected pattern: user prefers morning deployments",
  "timestamp": "2026-03-15T18:00:00Z"
}

Field	Type	Description
agent_id	string	The agent that recorded the memory
content	string	The memory content
timestamp	string (ISO 8601)	When the memory was recorded

agent_message

An agent sends a chat message (response to operator or autonomous).

{
  "type": "agent_message",
  "agent_id": "echo",
  "role": "assistant",
  "content": "Broadcast sent to all channels."
}

Field	Type	Description
agent_id	string	The agent sending the message
role	string	Always assistant for agent messages
content	string	The message text

connection

Indicates communication activity between two agents. The world draws/removes a glowing line.

{
  "type": "connection",
  "agent_id": "timmy",
  "target_id": "forge",
  "active": true
}

Field	Type	Description
agent_id	string	First agent
target_id	string	Second agent
active	boolean	true = draw line, false = remove line

system_status

System-wide status summary. Displayed when the Core pillar is tapped.

{
  "type": "system_status",
  "agents_online": 4,
  "tasks_pending": 3,
  "tasks_running": 2,
  "tasks_completed": 10,
  "tasks_failed": 1,
  "total_tasks": 16,
  "uptime": "48h 23m"
}

---

World → Server (Actions)

These messages are sent from the 3D world to the agent backend when the operator interacts.

chat_message

The operator sends a chat message to a specific agent.

{
  "type": "chat_message",
  "agent_id": "timmy",
  "content": "What's your current status?"
}

Field	Type	Description
agent_id	string	Target agent
content	string	The operator's message

task_action

The operator approves or vetoes a task.

{
  "type": "task_action",
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "action": "approve"
}

Field	Type	Description
task_id	string (UUID)	The task to act on
action	string	approve or veto

---

Implementation Notes

Agent IDs

The default agents are: timmy, forge, seer, echo. To add more agents, extend the AGENT_DEFS in js/websocket.js and add corresponding geometry in js/agents.js.

Timing

• Agent state events: recommended every 2-10 seconds per agent
• Task events: as they occur
• Memory events: as they occur
• Connection events: when communication starts/stops
• System status: every 5-10 seconds

Error Handling

The world gracefully handles:

• Unknown agent_id values (ignored)
• Unknown type values (ignored)
• Missing optional fields (defaults used)
• Connection loss (falls back to mock data)

Mock Mode

When no real WebSocket is available, the built-in MockWebSocket class simulates all of the above events at realistic intervals, making the world feel alive and interactive out of the box.