Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

Install and configure Qwen3-14B Q5_K_M as Timmy primary brain #1064

New Issue
Closed
opened 2026-03-23 12:52:11 +00:00 by perplexity · 1 comment
perplexity commented 2026-03-23 12:52:11 +00:00
Collaborator
Copy Link

Parent: #1063

Objective

Set up Qwen3-14B Q5_K_M via Ollama as Timmy's primary local LLM on the M3 Max 36GB.

Steps

  1. ollama pull qwen3:14b
  2. Create Modelfile.timmy with the optimized parameters from the PDF:
    • num_ctx 32768, temperature 0.3, top_p 0.9, min_p 0.02
    • repeat_penalty 1.1, num_predict 4096
    • Stop tokens: <|im_end|>, <|im_start|>
    • Full system prompt defining Timmy's capabilities and operating rules
  3. ollama create timmy -f Modelfile.timmy
  4. Verify tool calling works via curl API test
  5. Verify JSON schema-constrained output works

Acceptance Criteria

  • ollama run timmy responds to tool-calling prompts with valid JSON
  • Memory usage stays under 18GB with 32K context
  • Generation speed ~20–28 tok/s confirmed

Memory Budget

  • Model: ~10.5 GB
  • 32K context KV cache: ~7 GB
  • Total: ~17.5 GB
  • Headroom on 28GB usable: ~10.5 GB free
Parent: #1063 ## Objective Set up Qwen3-14B Q5_K_M via Ollama as Timmy's primary local LLM on the M3 Max 36GB. ## Steps 1. `ollama pull qwen3:14b` 2. Create `Modelfile.timmy` with the optimized parameters from the PDF: - `num_ctx 32768`, `temperature 0.3`, `top_p 0.9`, `min_p 0.02` - `repeat_penalty 1.1`, `num_predict 4096` - Stop tokens: `<|im_end|>`, `<|im_start|>` - Full system prompt defining Timmy's capabilities and operating rules 3. `ollama create timmy -f Modelfile.timmy` 4. Verify tool calling works via curl API test 5. Verify JSON schema-constrained output works ## Acceptance Criteria - `ollama run timmy` responds to tool-calling prompts with valid JSON - Memory usage stays under 18GB with 32K context - Generation speed ~20–28 tok/s confirmed ## Memory Budget - Model: ~10.5 GB - 32K context KV cache: ~7 GB - Total: ~17.5 GB - Headroom on 28GB usable: ~10.5 GB free
claude added the harnessinferencep0-critical labels 2026-03-23 13:52:57 +00:00
claude self-assigned this 2026-03-23 14:01:40 +00:00
claude commented 2026-03-23 18:37:14 +00:00
Collaborator
Copy Link

PR #1145 created.

Changes:

  • Modelfile.timmy: rebased to qwen3:14b; parameters per issue spec (num_ctx 32768, temperature 0.3, top_p 0.9, min_p 0.02, repeat_penalty 1.1, num_predict 4096, stop tokens <|im_end|> / <|im_start|>); full sovereign-AI system prompt.
  • config.py: default model → timmy; context → 32768; qwen3:14b first in fallback chain.
  • config/providers.yaml: timmy as default model; qwen3:14b added; fallback chains updated.
  • multimodal.py: capability entries added for qwen3:14b, qwen3:30b, timmy, hermes4-14b.
  • prompts.py: context window limitation updated from 4096 → 32K.

Manual steps still needed on the M3 Max:

ollama pull qwen3:14b
ollama create timmy -f Modelfile.timmy

All unit tests pass (20/20).

PR #1145 created. **Changes:** - `Modelfile.timmy`: rebased to `qwen3:14b`; parameters per issue spec (`num_ctx 32768`, `temperature 0.3`, `top_p 0.9`, `min_p 0.02`, `repeat_penalty 1.1`, `num_predict 4096`, stop tokens `<|im_end|>` / `<|im_start|>`); full sovereign-AI system prompt. - `config.py`: default model → `timmy`; context → 32768; `qwen3:14b` first in fallback chain. - `config/providers.yaml`: `timmy` as default model; `qwen3:14b` added; fallback chains updated. - `multimodal.py`: capability entries added for `qwen3:14b`, `qwen3:30b`, `timmy`, `hermes4-14b`. - `prompts.py`: context window limitation updated from 4096 → 32K. **Manual steps still needed on the M3 Max:** ``` ollama pull qwen3:14b ollama create timmy -f Modelfile.timmy ``` All unit tests pass (20/20).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label harness inference p0-critical
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees claude
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#1064
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?