3172415da1
All checks were successful
Smoke Test / smoke (pull_request) Successful in 28s
- Add scripts/upstream_watch.py for monitoring upstream repositories - Add .github/workflows/upstream-watch.yml for weekly automated monitoring - Add docs/upstream-watch.md for documentation - Add scripts/run_upstream_watch.sh for easy execution - Add scripts/test_upstream_watch.py for testing Addresses issue #15: [P4] Upstream llama.cpp / Ollama TurboQuant watch Features: 1. Monitor llama.cpp, Ollama, and ggml repositories 2. Search for TurboQuant/PolarQuant/QJL keywords 3. Check issues, PRs, and release notes 4. Generate text and JSON reports 5. Weekly GitHub Action for continuous monitoring 6. Automated issue creation when findings detected Usage: - Run monitor: python3 scripts/upstream_watch.py --days 30 - JSON output: python3 scripts/upstream_watch.py --format json - Weekly monitoring: GitHub Action runs every Monday at 9:00 AM UTC When upstream lands: 1. Detection: Monitor will detect mentions 2. Evaluation: Compare upstream vs fork 3. Decision: Migrate if upstream is better Closes #15
TurboQuant
KV cache compression for local inference on M4 Max MacBook Pro.
What
TurboQuant (Google, ICLR 2026) is a three-stage KV cache compression method:
- PolarQuant — WHT rotation + polar coordinates + Lloyd-Max codebook (~4.2x compression)
- QJL — 1-bit quantized Johnson-Lindenstrauss residual correction
- TurboQuant — PolarQuant + QJL = ~3.5 bits/channel, zero accuracy loss
Why
Unlock 64K-128K context on qwen3.5:27b within 32GB unified memory. A 27B model at 128K context with TurboQuant beats a 72B at Q2 with 8K context.
Status
See issues for current progress.
Roles
- Strago: Build spec author
- Cid: Implementation, benchmarks, deployment
- Locke: Research support, upstream watch
- John: Quality review
- Frankie: Coordination
Source Repos
- TheTom/llama-cpp-turboquant — llama.cpp fork with Metal
- TheTom/turboquant_plus — Reference impl, 511+ tests
- amirzandieh/QJL — Author QJL code (CUDA)
- rachittshah/mlx-turboquant — MLX fallback
Docs
- BUILD-SPEC.md — Full build specification (Strago, v2.2)