the-matrix/PROTOCOL.md

# WebSocket Protocol Specification

## Connection

```
ws://[host]:[port]/ws/world-state
```

The world connects to this endpoint on load. If the connection fails, it falls back to the built-in MockWebSocket which simulates agent activity.

## Message Format

All messages are JSON objects with a `type` field that determines the message schema.

---

## Server → World (Events)

These messages are sent from the agent backend to the 3D world.

### `agent_state`

Updates an agent's current state and visual properties.

```json
{
  "type": "agent_state",
  "agent_id": "timmy",
  "state": "working",
  "current_task": "Analyzing codebase",
  "glow_intensity": 0.8
}
```

| Field | Type | Description |
|-------|------|-------------|
| `agent_id` | string | One of: `timmy`, `forge`, `seer`, `echo` |
| `state` | string | One of: `idle`, `working`, `waiting` |
| `current_task` | string\|null | Description of current task |
| `glow_intensity` | number | 0.0 to 1.0 — controls visual glow brightness |

### `task_created`

A new task has been created. The world spawns a floating geometric object.

```json
{
  "type": "task_created",
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "agent_id": "forge",
  "title": "Fix login bug",
  "status": "pending",
  "priority": "high"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `task_id` | string (UUID) | Unique task identifier |
| `agent_id` | string\|null | Assigned agent (null = unassigned) |
| `title` | string | Human-readable task name |
| `status` | string | `pending`, `in_progress`, `completed`, `failed` |
| `priority` | string | `high`, `normal`, `low` |

### `task_update`

An existing task's status has changed. The world updates the task object's color.

```json
{
  "type": "task_update",
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "agent_id": "forge",
  "title": "Fix login bug",
  "status": "in_progress",
  "priority": "high"
}
```

Same fields as `task_created`. Color mapping:
- `pending` → white
- `in_progress` → amber/orange
- `completed` → green
- `failed` → red

### `memory_event`

An agent has recorded a new memory.

```json
{
  "type": "memory_event",
  "agent_id": "seer",
  "content": "Detected pattern: user prefers morning deployments",
  "timestamp": "2026-03-15T18:00:00Z"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `agent_id` | string | The agent that recorded the memory |
| `content` | string | The memory content |
| `timestamp` | string (ISO 8601) | When the memory was recorded |

### `agent_message`

An agent sends a chat message (response to operator or autonomous).

```json
{
  "type": "agent_message",
  "agent_id": "echo",
  "role": "assistant",
  "content": "Broadcast sent to all channels."
}
```

| Field | Type | Description |
|-------|------|-------------|
| `agent_id` | string | The agent sending the message |
| `role` | string | Always `assistant` for agent messages |
| `content` | string | The message text |

### `connection`

Indicates communication activity between two agents. The world draws/removes a glowing line.

```json
{
  "type": "connection",
  "agent_id": "timmy",
  "target_id": "forge",
  "active": true
}
```

| Field | Type | Description |
|-------|------|-------------|
| `agent_id` | string | First agent |
| `target_id` | string | Second agent |
| `active` | boolean | `true` = draw line, `false` = remove line |

### `system_status`

System-wide status summary. Displayed when the Core pillar is tapped.

```json
{
  "type": "system_status",
  "agents_online": 4,
  "tasks_pending": 3,
  "tasks_running": 2,
  "tasks_completed": 10,
  "tasks_failed": 1,
  "total_tasks": 16,
  "uptime": "48h 23m"
}
```

---

## World → Server (Actions)

These messages are sent from the 3D world to the agent backend when the operator interacts.

### `chat_message`

The operator sends a chat message to a specific agent.

```json
{
  "type": "chat_message",
  "agent_id": "timmy",
  "content": "What's your current status?"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `agent_id` | string | Target agent |
| `content` | string | The operator's message |

### `task_action`

The operator approves or vetoes a task.

```json
{
  "type": "task_action",
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "action": "approve"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `task_id` | string (UUID) | The task to act on |
| `action` | string | `approve` or `veto` |

---

## Implementation Notes

### Agent IDs

The default agents are: `timmy`, `forge`, `seer`, `echo`. To add more agents, extend the `AGENT_DEFS` in `js/websocket.js` and add corresponding geometry in `js/agents.js`.

### Timing

- Agent state events: recommended every 2-10 seconds per agent
- Task events: as they occur
- Memory events: as they occur
- Connection events: when communication starts/stops
- System status: every 5-10 seconds

### Error Handling

The world gracefully handles:
- Unknown `agent_id` values (ignored)
- Unknown `type` values (ignored)
- Missing optional fields (defaults used)
- Connection loss (falls back to mock data)

### Mock Mode

When no real WebSocket is available, the built-in `MockWebSocket` class simulates all of the above events at realistic intervals, making the world feel alive and interactive out of the box.