Timmy_Foundation/turboquant
16
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases Wiki Activity

[P2-S0] Ollama CGo API compatibility check #9

New Issue
Closed
opened 2026-03-30 17:11:11 +00:00 by Timmy · 1 comment
Timmy commented 2026-03-30 17:11:11 +00:00
Owner
Copy Link

Parent: #1 | Depends on: Phase 1 complete

Check if our llama.cpp fork's API surface is compatible with Ollama's CGo bindings.

Action

git clone https://github.com/ollama/ollama.git
cd ollama

# Find pinned llama.cpp commit
cat llm/llama.cpp/CMakeLists.txt | head -5

# Diff API surface
diff <(grep -h "^LLAMA_API\|^GGML_API" llm/llama.cpp/include/*.h | sort) \
     <(grep -h "^LLAMA_API\|^GGML_API" /path/to/our-fork/include/*.h | sort)

Assessment

  • TurboQuant changes additive only (new functions/types)? -> Safe
  • Modified existing signatures? -> Need to update Ollama CGo bindings

Acceptance Criteria

  • Ollama's pinned llama.cpp commit identified
  • API surface diff generated
  • Compatibility assessment: compatible / needs binding updates
## Parent: #1 | Depends on: Phase 1 complete Check if our llama.cpp fork's API surface is compatible with Ollama's CGo bindings. ## Action ```bash git clone https://github.com/ollama/ollama.git cd ollama # Find pinned llama.cpp commit cat llm/llama.cpp/CMakeLists.txt | head -5 # Diff API surface diff <(grep -h "^LLAMA_API\|^GGML_API" llm/llama.cpp/include/*.h | sort) \ <(grep -h "^LLAMA_API\|^GGML_API" /path/to/our-fork/include/*.h | sort) ``` ## Assessment - TurboQuant changes additive only (new functions/types)? -> Safe - Modified existing signatures? -> Need to update Ollama CGo bindings ## Acceptance Criteria - [ ] Ollama's pinned llama.cpp commit identified - [ ] API surface diff generated - [ ] Compatibility assessment: compatible / needs binding updates
Timmy added this to the Phase 2 — Ollama Integration + Production milestone 2026-03-30 17:11:11 +00:00
Timmy added the phase-2buildowner:cid labels 2026-03-30 17:11:11 +00:00
Timmy commented 2026-03-30 21:04:01 +00:00
Author
Owner
Copy Link

Ollama CGo API Compatibility Check

Ollama Status

  • Fixed broken symlink (moved Ollama.app from ~/Downloads to /Applications)
  • Version: 0.17.7, server running on :11434

API Surface

TurboQuant changes are ADDITIVE (good):

  • 3 new GGML_TYPE enums: TURBO3_0=41, TURBO4_0=42, TURBO2_0=43
  • 1 new GGML_OP: TURBO_WHT
  • No existing enums/structs modified

Compatibility Problem

Fork is based on NEWER upstream than Ollama's pin (ec98e2002):

  • Ollama carries 34 custom patches on its vendored llama.cpp
  • Direct replacement fails (patch sha1 mismatches)
  • Cherry-picking TurboQuant onto Ollama's tree requires patching 30+ files

Verdict

Custom Ollama build is a multi-day porting effort — not feasible in Phase 2 timeframe. Deferred.

Production Alternative

The fork's llama-server binary is already built and speaks OpenAI-compatible API (same format as Ollama). Recommended as drop-in replacement for the inference endpoint.

## Ollama CGo API Compatibility Check ### Ollama Status - Fixed broken symlink (moved Ollama.app from ~/Downloads to /Applications) - Version: 0.17.7, server running on :11434 ### API Surface TurboQuant changes are **ADDITIVE** (good): - 3 new GGML_TYPE enums: TURBO3_0=41, TURBO4_0=42, TURBO2_0=43 - 1 new GGML_OP: TURBO_WHT - No existing enums/structs modified ### Compatibility Problem Fork is based on NEWER upstream than Ollama's pin (ec98e2002): - Ollama carries 34 custom patches on its vendored llama.cpp - Direct replacement fails (patch sha1 mismatches) - Cherry-picking TurboQuant onto Ollama's tree requires patching 30+ files ### Verdict Custom Ollama build is a **multi-day porting effort** — not feasible in Phase 2 timeframe. Deferred. ### Production Alternative The fork's `llama-server` binary is already built and speaks **OpenAI-compatible API** (same format as Ollama). Recommended as drop-in replacement for the inference endpoint.
Timmy closed this issue 2026-03-30 21:04:01 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
benchmark
Performance/quality benchmarks
 blocker
Blocking decision needed
 build
Build, compile, Metal kernels
 deploy
Deployment to MacBook
 epic
owner:cid
Assigned to Cid
 owner:john
Assigned to John
 owner:locke
Assigned to Locke
 phase-1
Phase 1 — PolarQuant MVP
 phase-2
Phase 2 — Ollama Integration + Production
 phase-2.5
Phase 2.5 — Per-Layer Optimization
 phase-3
Phase 3 — QJL Residual Correction
 phase-4
Phase 4 — Upstream Watch
 research
Research, paper review, algorithm verification
 velocity-engine
Auto-generated by velocity engine
No Label build owner:cid phase-2
Milestone
No items
No Milestone Phase 2 — Ollama Integration + Production
Projects
Clear projects
No project
Assignees
Clear assignees
Rockachopa Timmy allegro antigravity bezalel claude codex-agent ezra gemini google grok groq hermes kimi manus perplexity
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/turboquant#9
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?