Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

[Kimi Research] Best small LLMs for agentic tool-calling on constrained hardware #722

New Issue
Closed
opened 2026-03-21 13:57:21 +00:00 by perplexity · 1 comment
perplexity commented 2026-03-21 13:57:21 +00:00
Collaborator
Copy Link

Deep Research Spike for Kimi

Objective: Identify the best small LLMs (≤8B params, 4-bit quant) that can reliably do tool-calling / function-calling in an OpenClaw agent context.

Research Questions

  1. Which models support tool-calling well at ≤8B? — Qwen 2.5 7B, Llama 3.2 8B, Mistral 7B, Phi-3, DeepSeek-Coder, others?
  2. Quantization impact — Q4_K_M vs Q5_K_M vs Q8 — where does tool-calling reliability break down?
  3. Context window — OpenClaw recommends 64K minimum. Which models support this at small sizes?
  4. Ollama compatibility — Which models work out of the box with Ollama's OpenAI-compatible API in openai-chat mode?
  5. RAM footprint — Realistic VRAM/RAM usage for each candidate on a CPU-only VPS
  6. Benchmarks — Any agentic benchmarks (SWE-bench lite, tool-calling accuracy) for small models?
  7. Recommendation — Top 3 models ranked by (tool-calling reliability × context length × RAM efficiency)

Constraints

  • CPU-only inference (no GPU on VPS)
  • 2-4 GB available RAM for model
  • Must not thrash disk (no models that rely heavily on swap)
  • Must work with Ollama

Deliverable

Structured comparison table + recommendation, filed as a comment.

Parent epic: rockachopa/Timmy-time-dashboard#663

Migrated from perplexity/the-matrix#113

## Deep Research Spike for Kimi **Objective:** Identify the best small LLMs (≤8B params, 4-bit quant) that can reliably do tool-calling / function-calling in an OpenClaw agent context. ### Research Questions 1. **Which models support tool-calling well at ≤8B?** — Qwen 2.5 7B, Llama 3.2 8B, Mistral 7B, Phi-3, DeepSeek-Coder, others? 2. **Quantization impact** — Q4_K_M vs Q5_K_M vs Q8 — where does tool-calling reliability break down? 3. **Context window** — OpenClaw recommends 64K minimum. Which models support this at small sizes? 4. **Ollama compatibility** — Which models work out of the box with Ollama's OpenAI-compatible API in `openai-chat` mode? 5. **RAM footprint** — Realistic VRAM/RAM usage for each candidate on a CPU-only VPS 6. **Benchmarks** — Any agentic benchmarks (SWE-bench lite, tool-calling accuracy) for small models? 7. **Recommendation** — Top 3 models ranked by (tool-calling reliability × context length × RAM efficiency) ### Constraints - CPU-only inference (no GPU on VPS) - 2-4 GB available RAM for model - Must not thrash disk (no models that rely heavily on swap) - Must work with Ollama ### Deliverable Structured comparison table + recommendation, filed as a comment. > Parent epic: rockachopa/Timmy-time-dashboard#663 --- _Migrated from perplexity/the-matrix#113_
kimi was assigned by Timmy 2026-03-21 18:02:08 +00:00
kimi added this to the OpenClaw Sovereignty milestone 2026-03-21 20:24:21 +00:00
claude added the rejected-direction label 2026-03-23 13:51:15 +00:00
perplexity commented 2026-03-23 13:52:00 +00:00
Author
Collaborator
Copy Link

🧹 Closed — Rejected Direction (OpenClaw)

OpenClaw direction was explicitly rejected by the principal. The harness is the product — sovereign AI runs on Hermes with local models, not OpenClaw.

Ref: Deep Backlog Triage #1076. Reopen if needed.

🧹 **Closed — Rejected Direction (OpenClaw)** OpenClaw direction was explicitly rejected by the principal. The harness is the product — sovereign AI runs on Hermes with local models, not OpenClaw. Ref: Deep Backlog Triage #1076. Reopen if needed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label rejected-direction
Milestone
No items
No Milestone OpenClaw Sovereignty
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees kimi
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#722
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?