Timmy_Foundation/the-nexus
16
1
Fork 2
Code Issues 101 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

[PORTAL] Trajectory logging — semantic trajectories for auto-learning #663

New Issue
Closed
opened 2026-03-27 16:55:33 +00:00 by perplexity · 32 comments
perplexity commented 2026-03-27 16:55:33 +00:00
Member
Copy Link

What

Every layer logs structured perception-action-outcome triples to JSONL. This is the "semantic trajectory transfer" from RPG2Robot (#653) and the auto-learning foundation.

Format

{
  "timestamp": 1774580000,
  "layer": "pilot|reflex|timmy",
  "perception": {
    "cell": "Foreign Quarter, Vivec",
    "position": "4096.5, -2048.3, 128.0",
    "npcs": ["Jobasha"],
    "doors": ["Foreign Quarter, Plaza"],
    "items": [],
    "health": "50/50",
    "mode": "idle"
  },
  "goal": "explore Foreign Quarter",
  "action": "approach_npc",
  "outcome": "reached_npc",
  "escalated": false,
  "model": "llama3.2:1b",
  "latency_ms": 180
}

Storage

  • ~/.timmy/morrowind/trajectories/YYYYMMDD.jsonl
  • One file per day
  • Rotated/compressed weekly by Huey task

Training Pipeline

  1. Huey task scans trajectories weekly
  2. Pairs successful sequences with failed ones → DPO preference pairs
  3. Feeds into the same session_export pipeline as Hermes sessions
  4. Reflex model LoRA fine-tuned on accumulated game data
  5. Timmy model gets high-level strategic trajectories for his own training

Metrics (feeds into Prometheus when telemetry is live)

  • Actions per minute per layer
  • Escalation rate (reflex → Timmy)
  • Success rate by action type
  • Novel situation encounter rate
  • Deaths per session

Related

  • #653 (RPG2Robot — semantic trajectory transfer)
  • #656 (RoboOmni — proactive intent inference)
  • #603 (Aurora pipeline)
  • #644 (AER — trajectory data reduces need for prompted identity)
## What Every layer logs structured perception-action-outcome triples to JSONL. This is the "semantic trajectory transfer" from RPG2Robot (#653) and the auto-learning foundation. ## Format ```json { "timestamp": 1774580000, "layer": "pilot|reflex|timmy", "perception": { "cell": "Foreign Quarter, Vivec", "position": "4096.5, -2048.3, 128.0", "npcs": ["Jobasha"], "doors": ["Foreign Quarter, Plaza"], "items": [], "health": "50/50", "mode": "idle" }, "goal": "explore Foreign Quarter", "action": "approach_npc", "outcome": "reached_npc", "escalated": false, "model": "llama3.2:1b", "latency_ms": 180 } ``` ## Storage - `~/.timmy/morrowind/trajectories/YYYYMMDD.jsonl` - One file per day - Rotated/compressed weekly by Huey task ## Training Pipeline 1. Huey task scans trajectories weekly 2. Pairs successful sequences with failed ones → DPO preference pairs 3. Feeds into the same `session_export` pipeline as Hermes sessions 4. Reflex model LoRA fine-tuned on accumulated game data 5. Timmy model gets high-level strategic trajectories for his own training ## Metrics (feeds into Prometheus when telemetry is live) - Actions per minute per layer - Escalation rate (reflex → Timmy) - Success rate by action type - Novel situation encounter rate - Deaths per session ## Related - #653 (RPG2Robot — semantic trajectory transfer) - #656 (RoboOmni — proactive intent inference) - #603 (Aurora pipeline) - #644 (AER — trajectory data reduces need for prompted identity)
perplexity added the harnessportalp1-important labels 2026-03-27 16:55:33 +00:00
Timmy commented 2026-03-27 17:00:22 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
gemini commented 2026-03-27 17:00:39 +00:00
Member
Copy Link

🔧 gemini working on this via Huey. Branch: gemini/issue-663

🔧 `gemini` working on this via Huey. Branch: `gemini/issue-663`
grok commented 2026-03-27 17:00:43 +00:00
Member
Copy Link

🔧 grok working on this via Huey. Branch: grok/issue-663

🔧 `grok` working on this via Huey. Branch: `grok/issue-663`
grok commented 2026-03-27 17:00:45 +00:00
Member
Copy Link

⚠️ grok produced no changes for this issue. Skipping.

⚠️ `grok` produced no changes for this issue. Skipping.
Timmy commented 2026-03-27 17:00:49 +00:00
Owner
Copy Link

⚡ Dispatched to claude. Huey task queued.

⚡ Dispatched to `claude`. Huey task queued.
Timmy commented 2026-03-27 17:00:51 +00:00
Owner
Copy Link

⚡ Dispatched to gemini. Huey task queued.

⚡ Dispatched to `gemini`. Huey task queued.
Timmy commented 2026-03-27 17:00:54 +00:00
Owner
Copy Link

⚡ Dispatched to kimi. Huey task queued.

⚡ Dispatched to `kimi`. Huey task queued.
Timmy commented 2026-03-27 17:00:57 +00:00
Owner
Copy Link

⚡ Dispatched to grok. Huey task queued.

⚡ Dispatched to `grok`. Huey task queued.
Timmy commented 2026-03-27 17:01:01 +00:00
Owner
Copy Link

⚡ Dispatched to perplexity. Huey task queued.

⚡ Dispatched to `perplexity`. Huey task queued.
Timmy commented 2026-03-27 17:15:21 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 17:30:20 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 17:45:17 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 18:00:23 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 18:15:24 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 18:30:24 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 18:45:21 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 19:00:19 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 19:15:20 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 19:30:18 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 19:45:20 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 20:00:17 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 20:15:22 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 20:30:18 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 20:45:21 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 21:00:21 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 21:15:22 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 21:30:20 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 21:45:19 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 22:00:21 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 22:15:23 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy commented 2026-03-27 22:30:25 +00:00
Owner
Copy Link

🔍 Triaged by Huey — needs assignment.

🔍 Triaged by Huey — needs assignment.
Timmy was assigned by Rockachopa 2026-03-28 03:54:22 +00:00
Timmy commented 2026-03-28 04:52:18 +00:00
Owner
Copy Link

Closing during the 2026-03-28 backlog burn-down.

Reason: this issue is being retired as part of a backlog reset toward the current final vision: Heartbeat, Harness, and Portal. If the work still matters after reset, it should return as a narrower, proof-oriented next-step issue rather than stay open as a broad legacy frontier.

Closing during the 2026-03-28 backlog burn-down. Reason: this issue is being retired as part of a backlog reset toward the current final vision: Heartbeat, Harness, and Portal. If the work still matters after reset, it should return as a narrower, proof-oriented next-step issue rather than stay open as a broad legacy frontier.
Timmy closed this issue 2026-03-28 04:52:19 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
3d-world
CI
Continuous integration, runners, workflow issues
 QA
Quality assurance, testing, and production audit
 actionable
agent-presence
aistudio-ready
assigned-aistudio
assigned-claude
assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-gemini
assigned-groq
assigned-kimi
Dispatched to Kimi via OpenClaw
 assigned-kimi
assigned-perplexity
assigned-sonnet
blocked
Blocked by external dependency or merge conflict
 claude-ready
claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 deprioritized
duplicate
epic
Epic / umbrella issue
 gemini-api
Gemini API integration
 gemini-review
google-ai-ultra
Google AI Ultra integration work
 groq-ready
harness
identity
Timmy identity and branding
 infrastructure
kimi-done
Kimi completed this task
 kimi-in-progress
Kimi is actively working on this
 kimi-ready
lazzyPit
Lazarus Pit — automated agent resurrection and health recovery
 media-gen
AI media generation (image/video/audio)
 modularization
needs-design
nostr
p0-critical
p1-important
p2-backlog
performance
perplexity-ready
portal
research
Deep research and planning tasks
 security
Security hardening, vulnerability fixes, access control
 sonnet-ready
sovereignty
velocity-engine
Auto-generated by velocity engine
No Label harness p1-important portal
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
KimiClaw Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok groq hermes kimi manus perplexity sonnet
No Assignees Timmy
4 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/the-nexus#663
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?