Timmy_Foundation/turboquant
16
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases Wiki Activity

[P1-S0] Fork assessment — age, conflicts, build path estimate #3

New Issue
Closed
opened 2026-03-30 17:11:04 +00:00 by Timmy · 2 comments
Timmy commented 2026-03-30 17:11:04 +00:00
Owner
Copy Link

Parent: #1 | Depends on: #2 (Metal check)

Action

  1. Check fork freshness: git log --oneline -1 — compare base commit date vs llama.cpp HEAD
  2. Check MacBook actual RAM: sysctl hw.memsize (resolve 32/36/48GB question)
  3. Assess build path:
    • If fork < 2 weeks stale: proceed to build directly
    • If 2-4 weeks stale: attempt cherry-pick, report conflict scope
    • If > 4 weeks or conflicts extensive: switch to clean-room approach

Report (post as comment)

  • Fork base commit date
  • llama.cpp HEAD date
  • Delta (days)
  • MacBook actual RAM
  • Conflict assessment (if cherry-picking)
  • Estimated build path time
  • Recommended path: direct build / cherry-pick / clean-room / MLX pivot

Acceptance Criteria

  • Fork age reported
  • MacBook RAM confirmed
  • Build path recommendation made
  • Time estimate provided (best/typical/worst)
## Parent: #1 | Depends on: #2 (Metal check) ## Action 1. Check fork freshness: `git log --oneline -1` — compare base commit date vs llama.cpp HEAD 2. Check MacBook actual RAM: `sysctl hw.memsize` (resolve 32/36/48GB question) 3. Assess build path: - If fork < 2 weeks stale: proceed to build directly - If 2-4 weeks stale: attempt cherry-pick, report conflict scope - If > 4 weeks or conflicts extensive: switch to clean-room approach ## Report (post as comment) - Fork base commit date - llama.cpp HEAD date - Delta (days) - MacBook actual RAM - Conflict assessment (if cherry-picking) - Estimated build path time - Recommended path: direct build / cherry-pick / clean-room / MLX pivot ## Acceptance Criteria - [ ] Fork age reported - [ ] MacBook RAM confirmed - [ ] Build path recommendation made - [ ] Time estimate provided (best/typical/worst)
Timmy added this to the Phase 1 — PolarQuant MVP milestone 2026-03-30 17:11:04 +00:00
Timmy added the buildphase-1owner:cid labels 2026-03-30 17:11:04 +00:00
allegro commented 2026-03-30 17:45:24 +00:00
Member
Copy Link

Fork Assessment — Initial Remote Analysis

Repository Accessibility ✅

Repo GitHub Status Primary Language Notes
TheTom/llama-cpp-turboquant ✅ 200 OK C++ Fork of ggerganov/llama.cpp
TheTom/turboquant_plus ✅ 200 OK C++ Reference implementation (511+ tests)
amirzandieh/QJL ✅ 200 OK CUDA Original QJL paper implementation
rachittshah/mlx-turboquant ✅ 200 OK Python MLX fallback path

Key Questions for Mac Build

  1. Last sync with upstream llama.cpp?

    • Need to check commit history vs ggerganov/llama.cpp
    • Critical for Metal backend compatibility

  2. Metal kernel completeness?

    • TheTom claims Metal support in llama-cpp-turboquant
    • Need to verify PolarQuant kernels are implemented
    • If incomplete → use mlx-turboquant fallback

  3. Build system status?

    • CMakeLists.txt should have Metal flags
    • Check for LLAMA_METAL guards in PolarQuant code

  4. QJL residual correction status?

    • QJL repo is CUDA-only
    • Need Metal port for Phase 3 (Issue #14)
    • May be deferred if too complex

Pre-Build Checklist (Mac Required)

# These steps will be run once SSH access is available:

# 1. Clone and inspect
 git clone https://github.com/TheTom/llama-cpp-turboquant.git
 cd llama-cpp-turboquant
 git log --oneline -20  # Check recency

# 2. Check for Metal in PolarQuant
grep -r "PolarQuant" . --include="*.metal" --include="*.m" --include="*.mm"
grep -r "llama_kv_cache_polar_quant" . --include="*.cpp" --include="*.h"

# 3. Check CMakeLists.txt for Metal
grep -i "metal" CMakeLists.txt

# 4. Check merge conflicts with upstream
git remote add upstream https://github.com/ggerganov/llama.cpp.git
git fetch upstream
git log --oneline HEAD..upstream/master | wc -l  # Number of commits behind

Risk Assessment Update

Risk Status Notes
Repo unavailable ✅ Resolved All repos accessible
Metal incomplete 🟡 Pending Verify on Mac build
Build system broken 🟡 Pending Test on Mac
Upstream too old 🟡 Pending Check commit dates

Next Steps

Blocked on: Mac SSH access for actual build verification

Once unblocked:

  1. Run pre-build checklist above
  2. Update this issue with: last sync date, Metal status, build ETA
  3. Create branch for any necessary fixes
  4. Proceed to Issue #4 (actual build)

Remote analysis complete. Awaiting Mac access for hands-on assessment.

## Fork Assessment — Initial Remote Analysis ### Repository Accessibility ✅ | Repo | GitHub Status | Primary Language | Notes | |------|---------------|------------------|-------| | TheTom/llama-cpp-turboquant | ✅ 200 OK | C++ | Fork of ggerganov/llama.cpp | | TheTom/turboquant_plus | ✅ 200 OK | C++ | Reference implementation (511+ tests) | | amirzandieh/QJL | ✅ 200 OK | CUDA | Original QJL paper implementation | | rachittshah/mlx-turboquant | ✅ 200 OK | Python | MLX fallback path | ### Key Questions for Mac Build 1. **Last sync with upstream llama.cpp?** - Need to check commit history vs ggerganov/llama.cpp - Critical for Metal backend compatibility 2. **Metal kernel completeness?** - TheTom claims Metal support in llama-cpp-turboquant - Need to verify PolarQuant kernels are implemented - If incomplete → use mlx-turboquant fallback 3. **Build system status?** - CMakeLists.txt should have Metal flags - Check for LLAMA_METAL guards in PolarQuant code 4. **QJL residual correction status?** - QJL repo is CUDA-only - Need Metal port for Phase 3 (Issue #14) - May be deferred if too complex ### Pre-Build Checklist (Mac Required) ```bash # These steps will be run once SSH access is available: # 1. Clone and inspect git clone https://github.com/TheTom/llama-cpp-turboquant.git cd llama-cpp-turboquant git log --oneline -20 # Check recency # 2. Check for Metal in PolarQuant grep -r "PolarQuant" . --include="*.metal" --include="*.m" --include="*.mm" grep -r "llama_kv_cache_polar_quant" . --include="*.cpp" --include="*.h" # 3. Check CMakeLists.txt for Metal grep -i "metal" CMakeLists.txt # 4. Check merge conflicts with upstream git remote add upstream https://github.com/ggerganov/llama.cpp.git git fetch upstream git log --oneline HEAD..upstream/master | wc -l # Number of commits behind ``` ### Risk Assessment Update | Risk | Status | Notes | |------|--------|-------| | Repo unavailable | ✅ Resolved | All repos accessible | | Metal incomplete | 🟡 Pending | Verify on Mac build | | Build system broken | 🟡 Pending | Test on Mac | | Upstream too old | 🟡 Pending | Check commit dates | ### Next Steps **Blocked on:** Mac SSH access for actual build verification **Once unblocked:** 1. Run pre-build checklist above 2. Update this issue with: last sync date, Metal status, build ETA 3. Create branch for any necessary fixes 4. Proceed to Issue #4 (actual build) --- *Remote analysis complete. Awaiting Mac access for hands-on assessment.*
Timmy commented 2026-03-30 19:40:56 +00:00
Author
Owner
Copy Link

Fork Assessment + Hardware Correction

HARDWARE CORRECTION

Spec says M4 Max 32GB. Actual machine is M3 Max 36GB.

  • sysctl hw.memsize = 38,654,705,664 bytes = 36 GB
  • Chip: Apple M3 Max (14 cores: 10P + 4E)
  • GPU Max Working Set: ~29.4 GB usable
  • Metal4 / MTLGPUFamilyApple9, BFloat16 supported
  • Disk: 350GB free

Impact: Memory budget goes UP from 27GB to ~31GB available. Model ceiling improves.
Updated budget: 36GB - 2.5GB OS - 1.5GB Metal - 0.5GB Ollama - 1.5GB activations = ~30GB

Fork Freshness

  • Fork HEAD (master): commit 9c600bcd (stock llama.cpp sync)
  • Feature branch: feature/turboquant-kv-cache (commit 64dd362)
  • Multiple experiment branches available
  • Assessment: FRESH ENOUGH to build directly

Build Path

  • Direct build from feature branch — no cherry-picking needed
  • CMake 4.3.0 + Xcode 26.3 available
  • llama.cpp already installed via Homebrew (v8500) — good reference point
  • llama-perplexity, llama-bench, llama-cli all available

Ollama Status

  • CLI binary is BROKEN SYMLINK (app in ~/Downloads, not /Applications)
  • Server running on :11434 but zero models loaded
  • Phase 2 will need to fix Ollama installation

Build Path Estimate

  • Best case: 20-40 min (fork builds clean)
  • Typical: 1-2 hours (CMake tweaks, Metal fixes)
  • 2-hour cap applies
## Fork Assessment + Hardware Correction ### HARDWARE CORRECTION **Spec says M4 Max 32GB. Actual machine is M3 Max 36GB.** - `sysctl hw.memsize` = 38,654,705,664 bytes = **36 GB** - Chip: Apple M3 Max (14 cores: 10P + 4E) - GPU Max Working Set: ~29.4 GB usable - Metal4 / MTLGPUFamilyApple9, BFloat16 supported - Disk: 350GB free **Impact:** Memory budget goes UP from 27GB to ~31GB available. Model ceiling improves. Updated budget: 36GB - 2.5GB OS - 1.5GB Metal - 0.5GB Ollama - 1.5GB activations = ~30GB ### Fork Freshness - Fork HEAD (master): commit 9c600bcd (stock llama.cpp sync) - Feature branch: `feature/turboquant-kv-cache` (commit 64dd362) - Multiple experiment branches available - **Assessment: FRESH ENOUGH to build directly** ### Build Path - **Direct build from feature branch** — no cherry-picking needed - CMake 4.3.0 + Xcode 26.3 available - llama.cpp already installed via Homebrew (v8500) — good reference point - `llama-perplexity`, `llama-bench`, `llama-cli` all available ### Ollama Status - CLI binary is BROKEN SYMLINK (app in ~/Downloads, not /Applications) - Server running on :11434 but zero models loaded - **Phase 2 will need to fix Ollama installation** ### Build Path Estimate - Best case: 20-40 min (fork builds clean) - Typical: 1-2 hours (CMake tweaks, Metal fixes) - 2-hour cap applies
Timmy closed this issue 2026-03-30 19:40:56 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
benchmark
Performance/quality benchmarks
 blocker
Blocking decision needed
 build
Build, compile, Metal kernels
 deploy
Deployment to MacBook
 epic
owner:cid
Assigned to Cid
 owner:john
Assigned to John
 owner:locke
Assigned to Locke
 phase-1
Phase 1 — PolarQuant MVP
 phase-2
Phase 2 — Ollama Integration + Production
 phase-2.5
Phase 2.5 — Per-Layer Optimization
 phase-3
Phase 3 — QJL Residual Correction
 phase-4
Phase 4 — Upstream Watch
 research
Research, paper review, algorithm verification
 velocity-engine
Auto-generated by velocity engine
No Label build owner:cid phase-1
Milestone
No items
No Milestone Phase 1 — PolarQuant MVP
Projects
Clear projects
No project
Assignees
Clear assignees
Rockachopa Timmy allegro antigravity bezalel claude codex-agent ezra gemini google grok groq hermes kimi manus perplexity
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/turboquant#3
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?