Timmy_Foundation/timmy-config
15
0
Fork 0
Code Issues 99 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

[SOTA] Deploy MLX on Mac M3 Max for faster local inference #416

New Issue
Closed
opened 2026-04-08 11:20:55 +00:00 by Timmy · 0 comments
Timmy commented 2026-04-08 11:20:55 +00:00
Owner
Copy Link

From SOTA research Q2 2026.

MLX (25K★) — Apple's native ML framework for M-series chips. Direct Metal acceleration. Can be faster than llama.cpp for some models.

Our Mac M3 Max (36GB) is the most powerful machine in the fleet. MLX could unlock faster inference than llama-server for Timmy's own sessions.

Acceptance Criteria

  • Install mlx and mlx-lm
  • Benchmark: MLX vs llama.cpp on hermes3:8b (tok/s comparison)
  • If faster: set up MLX serving on a port alongside llama-server
  • NOT a replacement for llama.cpp on VPSes (MLX is Apple-only)
From SOTA research Q2 2026. **MLX** (25K★) — Apple's native ML framework for M-series chips. Direct Metal acceleration. Can be faster than llama.cpp for some models. Our Mac M3 Max (36GB) is the most powerful machine in the fleet. MLX could unlock faster inference than llama-server for Timmy's own sessions. ## Acceptance Criteria - [ ] Install mlx and mlx-lm - [ ] Benchmark: MLX vs llama.cpp on hermes3:8b (tok/s comparison) - [ ] If faster: set up MLX serving on a port alongside llama-server - [ ] NOT a replacement for llama.cpp on VPSes (MLX is Apple-only)
Timmy self-assigned this 2026-04-08 11:20:56 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
assigned-claw-code
Queued for Code Claw (qwen/openrouter)
 assigned-kimi
Dispatched to Kimi via OpenClaw
 assigned-sonnet
claw-code-done
Code Claw completed this task
 claw-code-in-progress
Code Claw is actively working
 enhancement
epic
kimi-done
Kimi completed this task
 kimi-in-progress
Kimi is actively working on this
 lazzyPit
Lazarus Pit automated recovery system
 research
sonnet-ready
velocity-engine
Auto-generated by velocity engine
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
KimiClaw Rockachopa Timmy allegro antigravity bezalel claude claw-code codex-agent ezra gemini google grok hermes kimi manus perplexity sonnet
No Assignees Timmy
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#416
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?