Timmy_Foundation/timmy-config
16
0
Fork 0
Code Issues 57 Pull Requests 1 Actions 1 Packages Projects Releases Wiki Activity

[PIPELINE] DPO cycle — conversation corrections become training signal #13

New Issue
Closed
opened 2026-03-26 14:01:27 +00:00 by Timmy · 7 comments
Timmy commented 2026-03-26 14:01:27 +00:00
Owner
Copy Link

The Learning Loop

This is the ticket that closes the neural feedback loop. Alexander's one wish.

Pipeline

Alexander talks to Timmy (Hermes session captured)
    |
    v
Alexander corrects Timmy or approves response
    |
    v
Session export extracts (prompt, chosen, rejected) pairs
    - chosen = response Alexander approved or didn't correct
    - rejected = response from stock model OR corrected response
    |
    v
DPO training on MLX (timmy-config #5)
    |
    v
Merge + convert to GGUF
    |
    v
New timmy model deployed to Ollama
    |
    v
Timmy is better at predicting what Alexander prefers
    |
    v (loop)

What exists

  • Session capture (Hermes does this automatically)
  • Session export Huey task (every 4h, 2001 files exported)
  • DPO trainer (timmy-config #5 — antigravity working on it)
  • Rejection generation (run stock model on same prompts)
  • Merge + GGUF conversion pipeline
  • Automated eval before deployment

Acceptance criteria

  • One full cycle completed: session → DPO pairs → train → merge → GGUF → deploy → verify
  • Timmy's response quality measurably improves on the eval suite
  • Process documented so it can repeat without manual intervention

This is the sovereignty loop. Conversations become weights. The apprentice learns from the master.

## The Learning Loop This is the ticket that closes the neural feedback loop. Alexander's one wish. ## Pipeline ``` Alexander talks to Timmy (Hermes session captured) | v Alexander corrects Timmy or approves response | v Session export extracts (prompt, chosen, rejected) pairs - chosen = response Alexander approved or didn't correct - rejected = response from stock model OR corrected response | v DPO training on MLX (timmy-config #5) | v Merge + convert to GGUF | v New timmy model deployed to Ollama | v Timmy is better at predicting what Alexander prefers | v (loop) ``` ## What exists - [x] Session capture (Hermes does this automatically) - [x] Session export Huey task (every 4h, 2001 files exported) - [ ] DPO trainer (timmy-config #5 — antigravity working on it) - [ ] Rejection generation (run stock model on same prompts) - [ ] Merge + GGUF conversion pipeline - [ ] Automated eval before deployment ## Acceptance criteria - [ ] One full cycle completed: session → DPO pairs → train → merge → GGUF → deploy → verify - [ ] Timmy's response quality measurably improves on the eval suite - [ ] Process documented so it can repeat without manual intervention This is the sovereignty loop. Conversations become weights. The apprentice learns from the master.
Rockachopa was assigned by Timmy 2026-03-26 14:01:27 +00:00
Timmy self-assigned this 2026-03-26 14:01:27 +00:00
Timmy commented 2026-03-26 14:01:50 +00:00
Author
Owner
Copy Link

⚡ Dispatched to kimi. Huey task queued.

⚡ Dispatched to `kimi`. Huey task queued.
Timmy commented 2026-03-26 14:01:56 +00:00
Author
Owner
Copy Link

⚡ Dispatched to grok. Huey task queued.

⚡ Dispatched to `grok`. Huey task queued.
Timmy commented 2026-03-26 14:02:03 +00:00
Author
Owner
Copy Link

⚡ Dispatched to perplexity. Huey task queued.

⚡ Dispatched to `perplexity`. Huey task queued.
gemini commented 2026-03-26 15:20:50 +00:00
Member
Copy Link

🔧 gemini working on this via Huey. Branch: gemini/issue-13

🔧 `gemini` working on this via Huey. Branch: `gemini/issue-13`
grok commented 2026-03-26 15:20:52 +00:00
Member
Copy Link

🔧 grok working on this via Huey. Branch: grok/issue-13

🔧 `grok` working on this via Huey. Branch: `grok/issue-13`
grok commented 2026-03-26 15:20:57 +00:00
Member
Copy Link

⚠️ grok produced no changes for this issue. Skipping.

⚠️ `grok` produced no changes for this issue. Skipping.
Timmy commented 2026-03-28 04:53:08 +00:00
Author
Owner
Copy Link

Closing during the 2026-03-28 backlog burn-down.

Reason: this issue is being retired as part of a backlog reset toward the current final vision: Heartbeat, Harness, and Portal. If the work still matters after reset, it should return as a narrower, proof-oriented next-step issue rather than stay open as a broad legacy frontier.

Closing during the 2026-03-28 backlog burn-down. Reason: this issue is being retired as part of a backlog reset toward the current final vision: Heartbeat, Harness, and Portal. If the work still matters after reset, it should return as a narrower, proof-oriented next-step issue rather than stay open as a broad legacy frontier.
Timmy closed this issue 2026-03-28 04:53:08 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-kimi
Dispatched to Kimi via OpenClaw
 claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 enhancement
epic
kimi-done
Kimi completed this task
 kimi-in-progress
Kimi is actively working on this
 research
velocity-engine
Auto-generated by velocity engine
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
KimiClaw Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok groq hermes kimi manus perplexity
No Assignees Timmy Rockachopa
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#13
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?