docs: add living status header to PROJECT_STATUS.md (#64)

- Update raw-IP forge URL to canonical forge domain in README.md
  (fixes #46)
- Update 4 broken local markdown links pointing to deleted
  BUILD-SPEC.md, PHASE1-REPORT.md, FULL-REPORT.md to
  docs/PROJECT_STATUS.md (fixes #44)

.gitea/workflows/smoke.yml

@@ -0,0 +1,24 @@
+name: Smoke Test
+on:
+  pull_request:
+  push:
+    branches: [main]
+jobs:
+  smoke:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+      - name: Parse check
+        run: |
+          find . -name '*.yml' -o -name '*.yaml' | grep -v .gitea | grep -v llama-cpp-fork | xargs -r python3 -c "import sys,yaml; [yaml.safe_load(open(f)) for f in sys.argv[1:]]"
+          find . -name '*.json' | grep -v llama-cpp-fork | while read f; do python3 -m json.tool "$f" > /dev/null || exit 1; done
+          find . -name '*.py' | grep -v llama-cpp-fork | xargs -r python3 -m py_compile
+          find . -name '*.sh' | xargs -r bash -n
+          echo "PASS: All files parse"
+      - name: Secret scan
+        run: |
+          if grep -rE 'sk-or-|sk-ant-|ghp_|AKIA' . --include='*.yml' --include='*.py' --include='*.sh' 2>/dev/null | grep -v .gitea | grep -v llama-cpp-fork; then exit 1; fi
+          echo "PASS: No secrets"

.gitignore

@@ -0,0 +1,3 @@
+build/
+*.pyc
+__pycache__/

BUILD-SPEC.md

@@ -1,449 +0,0 @@
-# TurboQuant Implementation — Build Spec (v2)
-**Prepared by:** Strago | **Date:** 2026-03-30 | **Updated:** 2026-03-30 (v2 — external review fixes)
-**Task:** STR-2026-03-30-01 | **For:** Cid (build) + Frankie (coordination)
-**Inputs read:** turboquant-2026-03-25.md (Google brief), turboquant-2026-03-30-recon-update.md (Locke recon), infra-bulletin.md, MEMORY.md, external Opus review
-
----
-
-## Situation
-
-John wants maximum local inference quality on the MacBook Pro (M4 Max, 32GB unified memory) using TurboQuant-level KV cache compression. Currently running `qwen3.5:27b` via Ollama at `10.0.0.133:11434`. The goal: run a larger or better model within the same 32GB memory envelope by compressing the KV cache during inference.
-
-TurboQuant (Google, ICLR 2026) is a three-stage KV cache compression method:
-1. **PolarQuant** — random rotation + polar coordinates + fixed scalar codebook. No normalization constants. ~4.2× compression.
-2. **QJL** — 1-bit quantized Johnson-Lindenstrauss on the residual. Zero-overhead bias correction.
-3. **TurboQuant** — PolarQuant for main signal + QJL for residual = unbiased inner product quantizer at ~3.5 bits/channel with zero accuracy loss.
-
-Community status: multiple `llama.cpp` forks, MLX proof-of-concepts, and a vLLM plugin exist. Nothing upstreamed to official `llama.cpp`, MLX, or Ollama yet. Author QJL code is public. Enough is public to build from.
-
----
-
-## 1a. PolarQuant Technical Detail — What Cid Needs to Verify
-
-This section specifies the PolarQuant algorithm concretely so Cid can verify that the community fork implements it correctly. A fork that gets the rotation wrong or uses the wrong codebook boundaries will compress successfully but degrade quality in ways that short PPL benchmarks may not catch — the damage surfaces during long production sessions with sustained context pressure.
-
-### The Algorithm (per KV vector)
-
-**Step 1 — Random Rotation (Preconditioning):**
-- Apply a fixed random orthogonal rotation to each KV vector before quantization.
-- The paper uses a **Walsh-Hadamard transform** (WHT) — a structured orthogonal matrix that's O(d log d) to apply, not O(d²) like a dense random matrix.
-- **Why:** Raw KV vectors have non-uniform coordinate distributions (some dimensions carry more energy). WHT spreads energy uniformly across all coordinates, making the post-rotation distribution predictable and concentrated. This is what eliminates the need for per-vector normalization constants.
-- **Cid verification:** The fork must use a fixed WHT (or equivalent structured orthogonal rotation), not a learned or per-layer rotation. The rotation matrix must be identical at quantization and dequantization. If the fork uses a dense random matrix instead of WHT, it's functionally correct but slower — flag it.
-
-**Step 2 — Polar Coordinate Transform:**
-- After rotation, decompose each vector into **radius** (L2 norm / signal strength) and **angle** (direction on the unit sphere).
-- The radius is stored at higher precision (FP16 or FP32) — it's one scalar per vector, negligible overhead.
-- The angle coordinates are what get quantized. Because WHT made their distribution predictable, you can use a fixed codebook without per-vector calibration.
-
-**Step 3 — Lloyd-Max Scalar Quantization:**
-- Each angle coordinate is independently quantized using a **Lloyd-Max optimal scalar quantizer**.
-- Lloyd-Max minimizes mean squared error for a known distribution. Because WHT makes the distribution analytically computable, the codebook boundaries are **precomputed once** and fixed for all vectors.
-- **Codebook sizes by compression target:**
-  - `turbo4` = 4 bits per coordinate = 16 codebook entries per dimension
-  - `turbo3` = 3 bits = 8 entries
-  - `turbo2` = 2 bits = 4 entries
-- **Cid verification:** Check that the fork's codebook boundaries match what the paper/PolarQuant paper specifies for the target distribution. If the fork uses uniform quantization instead of Lloyd-Max, that's a quality regression — uniform is simpler but wastes bits on low-probability regions.
-
-**Step 4 — Bit Packing + Storage:**
-- Quantized indices are packed into the KV cache format (turbo2/3/4 nibble-packed).
-- Radius stored separately. No normalization constants, no scale factors, no zero-points — this is the key advantage over standard quantization.
-
-### Dequantization During Attention
-
-When the model computes attention scores (Q·K^T) and weighted values (softmax·V):
-1. Read packed indices from cache
-2. Look up codebook values (single table lookup per coordinate)
-3. Reconstruct angle coordinates
-4. Scale by stored radius
-5. Compute dot product in reconstructed space
-
-**Critical property:** The inner product between a full-precision query Q and a PolarQuant-compressed K must be an unbiased estimator of the true Q·K dot product. The WHT rotation preserves this because orthogonal transforms preserve inner products. If the fork adds any non-orthogonal transformation (e.g., learned projection, PCA), the unbiasedness guarantee breaks.
-
-### PolarQuant Initialization — Codebook + Rotation Matrix Setup
-
-PolarQuant requires two things to be initialized before inference can start:
-
-1. **Walsh-Hadamard rotation matrix:** This is deterministic — a WHT of size d (model head dimension, typically 128) is computed from the recursive Hadamard construction. It's the same for every session, every model. Compute once at model load, store in memory. Cost: O(d log d) per head dimension — microseconds. No impact on model load time.
-
-2. **Lloyd-Max codebook:** The quantization boundaries are precomputed for the known post-WHT distribution. For a given bit width (turbo4 = 4 bits = 16 entries), the codebook is a fixed lookup table of 16 boundary values + 16 reconstruction values. This is identical across sessions and models of the same head dimension. Can be hardcoded as a constant array or computed once at load time from the analytical distribution formula.
-
-**Expected initialization overhead:** Negligible — both are small deterministic computations. But **measure it during Phase 1**: time the gap between Ollama receiving a request and the first token appearing, with and without TurboQuant. If initialization adds >1 second to cold model load, investigate caching the tables to disk alongside the model file.
-
-**Cid measurement target:** Report model load time (cold start) with and without TurboQuant. If >5 second delta, flag as UX issue.
-
-**Cid verification checklist (before trusting benchmark numbers):**
-- [ ] Rotation is WHT or equivalent structured orthogonal (not learned, not dense random)
-- [ ] Same rotation matrix used for quantization and dequantization
-- [ ] Codebook is Lloyd-Max (not uniform), boundaries precomputed for post-WHT distribution
-- [ ] Radius stored separately at FP16+ precision
-- [ ] No per-vector normalization constants stored (this is the whole point)
-- [ ] Dequant path in Metal shader matches the quantization path exactly
-
----
-
-## 1. Model Targeting — What Can We Run?
-
-### Memory Budget — Realistic, Not Theoretical
-
-On a 32GB M4 Max running macOS, you do NOT have 32GB for inference. Realistic budget:
-
-| Consumer | Estimate |
-|----------|----------|
-| macOS + system services | ~2-3GB |
-| Metal command buffer + GPU driver overhead | ~1-2GB |
-| Ollama process overhead | ~0.5GB |
-| Activation memory (intermediate tensors during forward pass) | ~1-3GB (varies by model/batch) |
-| **Available for model weights + KV cache** | **~26-28GB** |
-
-**Use 27GB as the planning ceiling.** The v1 spec said "leaves 2GB for OS" at 30GB peak — that's too tight. All memory calculations below use 27GB available.
-
-### Current State (No TurboQuant)
-- **qwen3.5:27b** at Q4_K_M (~16GB model weights) — fits within 27GB budget with room for KV cache
-- At 32K context, KV cache for a 27B model at FP16 ≈ 4-6GB → total ~20-22GB. Comfortable.
-- At 64K context, KV cache ≈ 8-12GB → total ~24-28GB. Marginal — may swap.
-- At 128K context, KV cache grows to ~16-24GB → doesn't fit. Context-limited.
-
-### With TurboQuant (4× KV Compression)
-- KV cache at 32K drops from ~5GB → ~1.2GB
-- KV cache at 64K drops from ~10GB → ~2.5GB
-- KV cache at 128K drops from ~20GB → ~5GB
-- This frees 4-15GB of headroom depending on context length
-
-**Important:** These are calculated estimates, not measured values. Actual memory consumption can exceed theoretical due to fragmentation, allocation overhead, and implementation-specific buffering. Phase 1 **must** include actual peak memory measurement (see validation section). If measured exceeds calculated by >15%, the context ceiling drops accordingly.
-
-### Model Recommendations
-
-**Primary target: qwen3.5:27b at Q4_K_M with extended context**
-- Model weights: ~16GB at Q4_K_M
-- With TurboQuant KV cache at 64K context: ~2.5GB cache + ~2GB activations → ~20-21GB total. Comfortable within 27GB budget.
-- With TurboQuant at 128K: ~5GB cache + ~2GB activations → ~23GB total. Fits, but tight — **needs measured validation.**
-- Without TurboQuant: 64K context KV cache ≈ 10GB → ~28GB total. OOM risk.
-- **Win: 64K context becomes reliable, 128K becomes possible.** This is the real unlock.
-
-**Stretch target: Qwen 3.5 32B (Q4_K_M)**
-- Model weights: ~18-19GB at Q4_K_M
-- With TurboQuant at 64K: ~2.5GB cache + ~2.5GB activations → ~23-24GB. Fits within 27GB but leaves only ~3GB headroom.
-- **Verdict: worth testing in Phase 1 benchmarks alongside 27B.** If it fits, marginally better quality. If it's marginal, stay on 27B.
-
-**Not recommended: Qwen 3.5 72B (Q2_K or IQ3_XXS)**
-- Model weights at Q2_K: ~27GB. Leaves ~0GB for anything else.
-- **Verdict: does not fit.** Even with TurboQuant, no room for KV cache + activations + Metal overhead. And quality at Q2_K is poor — weight quantization damage cancels the parameter count advantage.
-
-**Recommended path: Stay on 27B class, use TurboQuant to unlock longer context (64K-128K) rather than a bigger model.** The real win on 32GB unified is context length, not parameter count. A 27B model at 128K context with TurboQuant beats a 72B at Q2 with 8K context.
-
-**Alternative worth testing: Mistral/Codestral 25B-class models** at Q5_K_M (~18GB) with TurboQuant. Locke's research notes TurboQuant was benchmarked on Mistral — community results may be more reproducible.
-
----
-
-## 2. Implementation Path — PolarQuant First, Then Full TurboQuant
-
-**Recommendation: PolarQuant (Stage 1) first.** Matches Locke's recommendation. Rationale:
-
-- PolarQuant alone delivers ~4.2× compression — that's the bulk of the win
-- Full TurboQuant adds QJL residual correction for marginal quality improvement at extreme compression (2.5 bits)
-- At 3.5+ bits/channel, PolarQuant is sufficient for zero accuracy loss
-- QJL adds kernel complexity for small incremental gain at our target compression ratio
-- We can always add QJL in Phase 2 if PolarQuant quality isn't sufficient
-
-### Source Repos (Priority Order)
-
-| Repo | What | Why | Risk |
-|------|------|-----|------|
-| **`TheTom/llama-cpp-turboquant`** | `llama.cpp` fork with Metal support | Most directly useful — same stack as Ollama. Reports PPL numbers on M-series. | Community fork, not upstream. May lag `llama.cpp` HEAD. |
-| **`TheTom/turboquant_plus`** | Standalone C implementation + Python tests | Most detailed reverse-engineering. 511+ tests. PolarQuant + Walsh-Hadamard + turbo2/3/4 formats. | Extends beyond paper ("Plus"). May include non-paper innovations. |
-| **`amirzandieh/QJL`** | Author's QJL CUDA implementation | Official author code. CUDA kernels, eval scripts, LongBench commands. | CUDA only — needs Metal port for MacBook. Phase 2 dependency. |
-| **`rachittshah/mlx-turboquant`** | MLX proof-of-concept | Native Apple Silicon. Correct module layout (codebooks, polar_quant, qjl). | May be partial implementation. Naming drift noted. |
-
-**Start from:** `TheTom/llama-cpp-turboquant` (for Ollama integration path) + `TheTom/turboquant_plus` (for reference/tests).
-
-### Community Fork Risk Assessment
-
-The v1 spec understated this. Community `llama.cpp` forks can diverge significantly from HEAD, especially in the Metal backend where Apple Silicon optimizations change frequently. The risk isn't "it doesn't build" — it's "it builds fine on the fork's base commit but breaks when cherry-picked onto current HEAD."
-
-**Specific risk areas:**
-- **KV cache layer:** `llama.cpp` has refactored KV cache internals multiple times in 2026. A fork based on a 4-week-old commit may touch structs/functions that have been renamed or restructured upstream.
-- **Metal shaders:** Apple Silicon Metal optimizations are actively changing. Custom Metal kernels for TurboQuant dequant may conflict with upstream shader refactors.
-- **Memory management:** `ggml` memory allocation has evolved. The fork's cache allocation assumptions may not match current `ggml` memory pools.
-
-**Mitigation plan (Phase 1 Step 0 — before any benchmarking):**
-
-1. **Check fork freshness:** `git log --oneline -1` on the fork. Compare base commit date against `llama.cpp` HEAD. If >4 weeks stale, flag as HIGH risk.
-2. **If fresh (< 2 weeks from HEAD):** Build directly. Likely works.
-3. **If stale (2-4 weeks):** Attempt cherry-pick of TurboQuant-specific commits onto current HEAD. If merge conflicts are limited to TurboQuant files → resolve manually. If conflicts touch core KV cache / Metal code → stop, evaluate effort.
-4. **If very stale (> 4 weeks) or conflicts are extensive:** Switch to **clean-room approach** — use `TheTom/turboquant_plus` as the algorithm reference and implement the KV cache types directly into current `llama.cpp` HEAD. This is more work (~60-90 min instead of ~20-40 min) but avoids the merge conflict maze.
-5. **Escape hatch:** If `llama.cpp` path is blocked, fall back to `rachittshah/mlx-turboquant` (MLX native, no fork divergence risk, but requires API proxy for Ollama compatibility).
-
-**Cid decision point:** After Step 0, report fork age + conflict assessment before proceeding. If clean-room is needed, update the time estimate and Frankie adjusts the schedule. Don't spend more than 15 minutes fighting merge conflicts — switch to clean-room.
-
-### Metal Kernel Risk — The Single Highest-Risk Assumption
-
-The spec assumes the `llama.cpp` fork has working **Metal shaders** for PolarQuant KV dequantization. KV dequant happens in the attention computation hot path — every token, every layer, every head. If the fork only has CPU or CUDA dequant kernels and no Metal implementation, the MacBook will either:
-- Fall back to CPU dequant → **catastrophic** performance loss (10-50× slower attention)
-- Fail to build entirely for Metal backend
-
-**Cid's actual first action** (before building, before benchmarking, before anything):
-
-```bash
-# Clone the fork
-git clone https://github.com/TheTom/llama-cpp-turboquant.git
-cd llama-cpp-turboquant
-
-# Check for Metal shader files referencing TurboQuant/PolarQuant
-grep -rn "turbo\|polar\|turboquant\|polarquant" ggml/src/ggml-metal* 2>/dev/null
-grep -rn "turbo\|polar" ggml/src/ggml-metal.metal 2>/dev/null
-
-# Check for Metal kernel dispatch for turbo KV types
-grep -rn "GGML_TYPE_.*TURBO\|turbo.*metal\|kv.*turbo" . --include="*.m" --include="*.metal" --include="*.h" 2>/dev/null
-```
-
-**If Metal shaders exist:** Proceed with `llama.cpp` fork path (primary).
-**If Metal shaders do NOT exist:** MLX becomes the **primary** path, not the fallback. Switch to `rachittshah/mlx-turboquant` immediately. Reframe Phase 1 around MLX + API proxy for Ollama compatibility. Report this finding before spending any more time on the `llama.cpp` path.
-
-This check takes 2 minutes and determines the entire build strategy. Do it first.
-
----
-
-## 3. Integration Target — llama.cpp → Ollama
-
-**Primary: `llama.cpp` fork → custom Ollama build.**
-
-Why not MLX:
-- Our entire fleet uses Ollama. Model management, API compatibility, endpoint routing — all built around Ollama.
-- MLX would require a separate inference server, separate model format, separate API integration.
-- Ollama is built on `llama.cpp`/`ggml`. KV cache changes in `llama.cpp` propagate to Ollama.
-
-**Integration strategy:**
-1. Build/test TurboQuant KV cache in a `llama.cpp` fork (Metal backend)
-2. Validate quality + performance
-3. Build custom Ollama from our `llama.cpp` fork (Ollama builds `llama.cpp` as a submodule)
-4. Deploy to MacBook as replacement Ollama binary
-5. Existing model files, API, and endpoint (`10.0.0.133:11434`) remain identical — only the inference engine changes
-
-**Fallback: MLX standalone** if `llama.cpp` Metal integration proves too complex. `rachittshah/mlx-turboquant` as starting point. Would require a small proxy server to maintain API compatibility with our Ollama endpoint.
-
----
-
-## 4. Validation Plan — How We Know It Works
-
-### Quality Validation
-
-**Test matrix (run each model with and without TurboQuant):**
-
-| Test | What It Measures | Tool | Pass Criteria |
-|------|-----------------|------|--------------|
-| Perplexity (PPL) | Overall language modeling quality | `llama-perplexity` on WikiText-2 | PPL delta ≤ 0.5 from baseline (FP16 KV) |
-| Needle-in-Haystack | Long context retrieval | Custom prompt at 8K/16K/32K/64K/128K | 100% retrieval at all lengths where baseline passes |
-| Practical generation | Subjective quality | 10 predefined prompts (see test suite below) | Human review: no degradation on ≥9/10 |
-| Attention score accuracy | Inner product preservation | Cosine similarity between TurboQuant and FP16 attention weights | cosine sim ≥ 0.995 |
-
-**Predefined Test Prompts (10 prompts, run identically on TurboQuant and FP16 KV baseline):**
-
-| # | Category | Prompt Description | What It Tests |
-|---|----------|-------------------|---------------|
-| 1 | Long-context summarization | Feed 20K tokens of a research paper, ask for structured summary with citations | KV cache quality at length — compressed K/V must retain source detail |
-| 2 | Multi-step reasoning | 5-step math word problem requiring chain-of-thought | Whether compressed KV degrades intermediate reasoning steps |
-| 3 | Code generation | Write a Python script with 3 functions, error handling, type hints | Precise token prediction — code is unforgiving of subtle quality drops |
-| 4 | Code debugging | Provide buggy code (3 bugs), ask to identify and fix all three | Attention to detail across context — must reference earlier code correctly |
-| 5 | Factual recall (early context) | Provide 10 facts in the first 1K tokens, continue for 8K tokens of filler, ask about fact #3 | Retrieval from early context through compressed KV |
-| 6 | Creative writing | Write a 500-word short story with specific constraints (setting, character, twist) | Compression artifacts surface as repetition or coherence loss |
-| 7 | Multi-turn conversation | 10-turn technical Q&A where later questions reference earlier answers | Cross-turn coherence through accumulated compressed KV |
-| 8 | Structured output | Generate a JSON schema with 15+ fields, nested objects, and validation rules | Format precision — compressed KV must maintain structural consistency |
-| 9 | Translation + analysis | Translate a paragraph EN→ES, then analyze the translation choices | Tests both generation quality and meta-reasoning about own output |
-| 10 | Instruction following | Complex prompt with 8 specific formatting requirements (headers, bullet style, word limits, etc.) | Whether compression causes the model to "forget" constraints mid-generation |
-
-**Prompts must be written and saved to `projects/sovereign-stack/turboquant-test-prompts.md` before Phase 2 benchmarks run.** Same prompts, same order, both configurations. This prevents unconscious cherry-picking.
-
-**Asymmetric K/V test:** Run K at Q8_0, V at turbo4. Community reports this works well on sensitive models. Compare PPL vs symmetric turbo4 K+V.
-
-**Long-session quality test (Phase 2 only):** Short-context PPL benchmarks can miss quality degradation that surfaces during sustained context pressure. During Phase 2, run one extended production simulation:
-- Generate a 50-turn multi-step reasoning conversation (code gen → debug → refactor → test → iterate)
-- Compare output quality vs same conversation on FP16 KV baseline
-- Specifically watch for: coherence drift after turn 30+, hallucinated references to earlier context, attention score softmax concentration (if measurable)
-- This catches the case where codebook boundary errors accumulate over many KV cache writes in a single session
-
-### Performance Validation
-
-| Metric | Measure | Pass Criteria |
-|--------|---------|--------------|
-| Tokens/second (generation) | `llama-bench` | ≥90% of baseline tok/s (small decode overhead acceptable) |
-| Time to first token (TTFT) | Timed prompt eval | ≤110% of baseline |
-| Peak resident memory | `footprint -p <pid>` or `vmmap --summary` at each context length | Stays under 27GB at target context length |
-| Memory vs theoretical | Compare measured peak to calculated estimate | If measured exceeds calculated by >15% → reduce context ceiling |
-| Context length ceiling | Binary search: max context before OOM or swap pressure | 64K minimum (vs ~32K baseline for 27B) |
-
-### Kill Criteria
-- PPL regression > 1.0 at any compression level → abort that compression level
-- OOM at 32K context (baseline capability) → regression, abort
-- tok/s drops > 25% → dequant overhead too high, need kernel optimization before deploy
-
----
-
-## 5. Who Does What
-
-| Role | Owner | What |
-|------|-------|------|
-| Build spec | Strago | This document ✅ |
-| Implementation | Cid | Fork `llama.cpp`, integrate PolarQuant KV cache, Metal kernels, build custom Ollama |
-| Validation | Cid | Run test matrix, report PPL/performance numbers |
-| Model selection | Cid | Test qwen3.5:27b + one Mistral variant, recommend best config |
-| MacBook deployment | Cid | Replace Ollama binary on MacBook, verify endpoint works |
-| Quality review | John | Review 10-prompt practical generation comparison |
-| Research support | Locke | If Cid hits a wall on the math, Locke deep-dives the paper/QJL code |
-
----
-
-## 6. Phasing
-
-### Phase 1 — PolarQuant MVP (Target: this week)
-
-**Scope:**
-
-**Step 0 — Fork Assessment (do this FIRST, report before proceeding):**
-- Clone `TheTom/llama-cpp-turboquant`
-- Check base commit age vs `llama.cpp` HEAD (`git log --oneline -1`)
-- Check `sysctl hw.memsize` on MacBook (resolve the 32/36/48GB question)
-- If fork < 2 weeks stale → proceed to build
-- If 2-4 weeks stale → attempt cherry-pick, report conflict scope
-- If > 4 weeks or conflicts extensive → switch to clean-room (see Fork Risk Assessment above)
-- Report: fork age, conflict assessment, MacBook actual RAM, estimated build path time
-
-**Step 1 — Build + Verify:**
-- Build `llama.cpp` fork (or clean-room) with Metal backend on MacBook (M4 Max)
-- Run the Section 1a verification checklist against the fork's implementation before trusting any benchmarks
-- Run FP16 KV baseline: `llama-perplexity` on WikiText-2 with `qwen3.5:27b` at 8K context (this is the number we're comparing against)
-
-**Step 2 — Benchmark PolarQuant:**
-- Run perplexity test with PolarQuant KV (turbo4 format) vs FP16 KV baseline
-- Run `llama-bench` for tok/s comparison
-- Test at 8K, 32K, and 64K context lengths
-- Run asymmetric test: K at Q8_0, V at turbo4
-- **Measure actual peak resident memory** at each context length (`footprint -p <pid>` or `vmmap --summary`). Compare measured vs calculated. If measured exceeds calculated by >15%, note the delta — it reduces the achievable context ceiling.
-- Report: PPL delta per context length, tok/s delta, **measured peak memory per context length**, max context before OOM/swap, asymmetric vs symmetric results
-
-**Deliverable:** Working `llama.cpp` build on MacBook with PolarQuant KV cache. PPL + performance numbers. Section 1a verification checklist completed.
-
-**Estimated Cid time (honest range):**
-- **Best case** — fork is fresh, builds clean on first try, Metal shaders work: 20-40 min
-- **Typical case** — fork needs CMake flag tweaks, Xcode SDK adjustments, minor Metal fixes: 1-2 hours
-- **Worst case** — fork is stale, conflicts extensive, or Metal shaders missing: clean-room build 2-4 hours, or MLX pivot
-
-**2-hour build troubleshooting cap:** If the `llama.cpp` fork doesn't compile and pass a basic smoke test (load model, generate 10 tokens) within 2 hours of troubleshooting, stop. Pivot to MLX path. Don't sink more time into Xcode/CMake/Metal debug loops when a working MLX PoC exists. Report what broke — the information is useful even if the path is abandoned.
-
-**Decision gate:** If PPL delta ≤ 0.5 and tok/s ≥ 90% baseline AND Section 1a checklist passes → proceed to Phase 2. If PPL fails but checklist passes → try asymmetric K/V or lower compression (turbo3 instead of turbo4). If checklist fails → fix implementation before trusting benchmarks.
-
-### Phase 2 — Ollama Integration + Production Deploy
-
-**Scope:**
-
-**Step 0 — Ollama API Compatibility Check (before building):**
-Ollama pins a specific `llama.cpp` commit and calls it through CGo bindings in `llm/`. If our fork changes any function signatures, struct layouts, or enum values that Ollama's Go code references, the build will either fail or produce subtle runtime bugs.
-
-```bash
-# Clone Ollama source
-git clone https://github.com/ollama/ollama.git
-cd ollama
-
-# Find the pinned llama.cpp commit
-cat llm/llama.cpp/CMakeLists.txt | head -5  # or check go.mod / Makefile
-
-# Diff our fork's API surface against Ollama's expected API
-# Focus on: llama.h, ggml.h function signatures that Ollama calls
-diff <(grep -h "^LLAMA_API\|^GGML_API" llm/llama.cpp/include/*.h | sort) \
-     <(grep -h "^LLAMA_API\|^GGML_API" /path/to/our-fork/include/*.h | sort)
-```
-
-If API surface differs: check if TurboQuant changes are additive (new functions/types only) or modify existing signatures. Additive = safe. Modified existing = need to update Ollama's CGo bindings.
-
-**Build steps:**
-- Build custom Ollama binary using our `llama.cpp` fork as submodule
-- Deploy to MacBook as replacement Ollama
-- Verify existing endpoint (`10.0.0.133:11434`) works identically
-- Run full test matrix (all 4 quality tests + all 4 performance tests)
-- Test with qwen3.5:27b at 64K and 128K context
-- If 128K works: update Ollama model config to advertise larger context
-- Run 10-prompt practical generation comparison for John review
-
-**Deliverable:** Production Ollama on MacBook with TurboQuant KV cache. Full benchmark report. John signs off on quality.
-
-**Estimated Cid time:** 15-25 min (Ollama build is straightforward once `llama.cpp` fork is validated).
-
-### Phase 2.5 — Per-Layer Quantization Profiles (Optimization, Optional)
-
-Not all transformer layers have equal sensitivity to KV cache quantization. Research and community experimentation show early layers (first 2-4) and late layers (last 2-4) tend to be more sensitive than middle layers. If the fork supports per-layer KV cache type configuration:
-
-- **Sensitive layers (first 3 + last 3):** K at Q8_0, V at turbo4 (or full FP16 KV)
-- **Middle layers:** K and V both at turbo4 (or even turbo3)
-
-This gives the same average compression ratio as uniform turbo4 but concentrates precision where it matters most. The PPL improvement can be meaningful (0.1-0.3) at zero memory cost.
-
-**When to pursue:** Only after Phase 2 is stable and baseline quality is confirmed. This is tuning, not architecture. If uniform turbo4 passes all quality gates, per-layer optimization is nice-to-have, not necessary.
-
-**Cid note:** During Phase 1, check whether the fork exposes per-layer KV type config. If it does, note it for later. Don't implement it yet.
-
-### Phase 3 — QJL Residual Correction (Optional)
-
-**Scope:** Add QJL 1-bit residual correction for full TurboQuant behavior. Only pursue if:
-- Phase 1/2 PolarQuant shows quality gaps at extreme compression (< 3 bits/channel)
-- We want to push to 2.5 bits/channel for even more context headroom
-
-**Source:** `amirzandieh/QJL` repo (CUDA → Metal port needed)
-
-**Estimated Cid time:** 30-60 min (Metal port of QJL kernels is real engineering work)
-
-**Decision gate:** Only proceed if PolarQuant alone doesn't meet quality bar at target compression.
-
-### Phase 4 — Upstream Watch
-
-**Scope:** Monitor `llama.cpp` upstream and Ollama for official TurboQuant support. When it lands:
-- Evaluate upstream implementation vs our fork
-- If upstream is better: migrate off our fork to official
-- If our fork is better: contribute upstream (optional)
-
-**Owner:** Locke (monitoring) + Cid (evaluation when it lands)
-
----
-
-## What This Spec Does NOT Cover
-
-- **Weight quantization** — TurboQuant is KV cache compression only. Model weight quantization (GGUF Q4_K_M etc.) is a separate concern and already handled by Ollama.
-- **Predator (desktop) deployment** — this spec targets MacBook only. Predator runs NVIDIA (CUDA) which is a different kernel backend. Can extend later.
-- **Multi-model serving** — TurboQuant helps with single-model memory but doesn't change Ollama's single-model-at-a-time constraint.
-- **Ollama upstream contribution** — out of scope for now. We build for ourselves first.
-
----
-
-## Open Questions for John
-
-**None blocking.** One informational:
-
-- **MacBook Pro memory:** Confirmed M4 Max 32GB from memory/2026-03-14.md. If it's actually 36GB or 48GB (M4 Max comes in 36/48/128 configs), that changes the model ceiling. Can Cid check `sysctl hw.memsize` on the MacBook during Phase 1? Non-blocking — doesn't change the approach, just the model size ceiling.
-
----
-
-## Reference Files
-
-| File | Location |
-|------|----------|
-| TurboQuant Google Brief | `projects/sovereign-stack/research/turboquant-2026-03-25.md` |
-| Locke Recon Update | `projects/sovereign-stack/research/turboquant-2026-03-30-recon-update.md` |
-| `llama.cpp` TurboQuant fork | `github.com/TheTom/llama-cpp-turboquant` |
-| TurboQuant+ reference impl | `github.com/TheTom/turboquant_plus` |
-| QJL author code | `github.com/amirzandieh/QJL` |
-| MLX PoC (fallback) | `github.com/rachittshah/mlx-turboquant` |
-| TurboQuant paper | `arxiv.org/abs/2504.19874` |
-| PolarQuant paper | `arxiv.org/abs/2502.02617` |
-
----
-
----
-
-## Changelog
-
-- **v1 (2026-03-30 12:26 ET):** Initial spec.
-- **v2 (2026-03-30 12:55 ET):** Added Section 1a (PolarQuant technical detail + Cid verification checklist), expanded fork risk assessment with mitigation plan, added Phase 1 Step 0 (fork assessment before benchmarking), added long-session quality test for Phase 2, updated Phase 1 time estimate for clean-room path. Changes driven by external Opus review round 1.
-- **v2.1 (2026-03-30 13:00 ET):** Added Metal kernel risk check (grep before build — determines llama.cpp vs MLX primary path), corrected memory budget (27GB available, not 30GB — accounts for OS + Metal driver + activations), added measured memory profiling requirement to Phase 1, added Ollama CGo API compatibility check to Phase 2 Step 0, tightened model ceiling estimates. Changes driven by external Opus review round 2.
-- **v2.2 (2026-03-30 13:05 ET):** Added honest time estimate range (20 min best → 2-4 hr worst), 2-hour build troubleshooting cap before MLX pivot, PolarQuant initialization detail (WHT + Lloyd-Max codebook setup + cold-start measurement target), 10 predefined test prompts with rationale (prevents cherry-picking), per-layer quantization profiles as Phase 2.5 optimization path. Changes driven by external Opus review round 3.
-
----
-
-*Build spec v2 ready for Cid intake. No clarifying questions needed.*

CMakeLists.txt

@@ -0,0 +1,36 @@
+cmake_minimum_required(VERSION 3.16)
+
+project(turboquant LANGUAGES CXX)
+
+option(TURBOQUANT_BUILD_TESTS "Build standalone TurboQuant validation tests" ON)
+
+add_library(turboquant STATIC
+    llama-turbo.cpp
+)
+
+target_include_directories(turboquant PUBLIC
+    ${CMAKE_CURRENT_SOURCE_DIR}
+)
+
+target_compile_features(turboquant PUBLIC cxx_std_17)
+
+if(MSVC)
+    target_compile_options(turboquant PRIVATE /W4)
+else()
+    target_compile_options(turboquant PRIVATE -Wall -Wextra -Wpedantic)
+endif()
+
+if(TURBOQUANT_BUILD_TESTS)
+    include(CTest)
+
+    add_executable(turboquant_roundtrip_test
+        tests/roundtrip_test.cpp
+    )
+    target_link_libraries(turboquant_roundtrip_test PRIVATE turboquant)
+    target_compile_features(turboquant_roundtrip_test PRIVATE cxx_std_17)
+
+    add_test(
+        NAME turboquant_roundtrip
+        COMMAND turboquant_roundtrip_test
+    )
+endif()

FULL-REPORT.md

@@ -1,245 +0,0 @@
-# TurboQuant — Full Knowledge Transfer Report
-
-**Date:** 2026-03-30
-**Prepared for:** Frankie's Team (Strago, Cid, Locke, John)
-**Spec:** turboquant-build-spec v2.2 (Strago)
-
----
-
-## TL;DR
-
-TurboQuant works. PolarQuant KV cache compression delivers **73% memory savings with 1% prompt overhead**. 128K context on the MacBook becomes viable. Custom Ollama build is deferred (multi-day effort), but the fork's `llama-server` is a ready drop-in. Per-layer adaptive quantization is already implemented. QJL is infrastructure-only — not needed at current compression targets.
-
----
-
-## Hardware Correction
-
-**Spec says:** M4 Max, 32GB
-**Actual:** M3 Max, 36GB (sysctl hw.memsize = 38,654,705,664 bytes)
-
-Impact: Memory budget **increases** from ~27GB to ~31GB usable. Model ceiling improves.
-
----
-
-## Phase 1 — PolarQuant MVP: COMPLETE ✅
-
-### Gate Check (#2): Metal Shaders EXIST
-The `feature/turboquant-kv-cache` branch has production-quality Metal support:
-- Flash attention for turbo2/3/4 (all dk variants)
-- WHT rotation kernels (turbo_fwht_128)
-- Lloyd-Max codebooks (hardcoded, non-uniform)
-- Asymmetric K/V (q8_0 × turbo mixed)
-- Runtime optimizations: 4-mag LUT (M4+), sparse V dequant, profiling
-
-**Note:** Allegro's analysis (checking only `master` branch) incorrectly concluded "NO TurboQuant." The implementation lives on the feature branch.
-
-### PolarQuant Verification (#5): 5/6 PASS
-
-| Item | Verdict |
-|------|---------|
-| WHT rotation (structured orthogonal) | PASS (Metal). CPU turbo4 ref uses dense random (legacy) |
-| Same rotation quant/dequant | PASS |
-| Lloyd-Max codebook (not uniform) | PASS |
-| Radius at FP16+ | PASS |
-| No per-vector normalization | PASS |
-| Dequant matches quant in Metal | PASS |
-
-**Flag:** CPU turbo4 reference path is algorithmically incompatible with Metal dequant. Only matters if CPU fallback invoked for turbo4. Metal production path is clean.
-
-### Benchmark Results
-
-**Model tested:** Hermes-4-14B Q4_K_M (8.38 GiB)
-
-#### Throughput
-
-| Config (K/V) | Prompt (pp512) | Δ | Generation (tg128) | Δ |
-|:-------------|:---------------|:--|:-------------------|:--|
-| f16/f16 (baseline) | 304.28 t/s | — | 27.47 t/s | — |
-| **turbo4/turbo4** | **300.00 t/s** | **-1.1%** | **22.45 t/s** | **-11.1%** |
-| turbo3/turbo3 | 271.07 t/s | -10.7% | 21.07 t/s | -16.6% |
-| q8_0/turbo4 (asymmetric) | 260.57 t/s | -14.1% | 23.75 t/s | -5.9% |
-
-#### KV Memory Savings
-
-| Context | f16 KV | turbo4 KV | Savings |
-|:--------|:-------|:----------|:--------|
-| 2K | 320 MiB | 85 MiB | 73.4% |
-| 8K | 1,280 MiB | 340 MiB | 73.4% |
-| 32K | 5,120 MiB | 1,360 MiB | 73.4% |
-| 65K | 10,240 MiB | 2,720 MiB | 73.4% |
-
-Measured matches calculated exactly. Zero fragmentation overhead.
-
-#### What This Means for qwen3.5:27b
-
-| Scenario | Total Memory | Fits 31GB? |
-|:---------|:-------------|:-----------|
-| 27B + f16 KV @ 128K | ~38 GB | ❌ No |
-| 27B + **turbo4 KV @ 128K** | **~23.4 GB** | **✅ Yes (7.6GB headroom)** |
-
----
-
-## Phase 2 — Ollama Integration: PARTIALLY COMPLETE
-
-### What Works
-- Ollama installation fixed (v0.17.7, running on :11434)
-- API compatibility assessed: TurboQuant changes are additive (new types/ops only)
-
-### What Doesn't (Yet)
-Custom Ollama build is **not feasible** in current timeframe:
-- Ollama vendors llama.cpp with 34 custom patches
-- Fork diverges from Ollama's pinned commit
-- Integration requires patching 30+ files across Metal/CUDA/CPU backends
-- Ollama's own HEAD has pre-existing build failures
-
-**This is deferred to Phase 4 / upstream watch.** When Ollama updates their llama.cpp pin or TurboQuant lands upstream, the gap narrows.
-
-### Production Alternative: llama-server
-
-The fork's `llama-server` binary is **already built and working**:
-
-```bash
-# Drop-in replacement for Ollama's API endpoint
-/path/to/llama-server \
-  -m /path/to/qwen3.5-27b-q4_k_m.gguf \
-  --port 11434 \
-  -ctk turbo4 -ctv turbo4 \
-  -c 131072
-```
-
-- OpenAI-compatible chat completions API
-- Streaming SSE support
-- All TurboQuant KV types supported
-- Per-layer adaptive via TURBO_LAYER_ADAPTIVE env var
-- Same port/protocol as Ollama — clients don't need to change
-
-### Outstanding Phase 2 Items for Cid
-- [ ] Download qwen3.5:27b Q4_K_M model
-- [ ] Deploy llama-server with turbo4 on MacBook
-- [ ] Run full 10-prompt quality matrix (prompts written by Allegro on #16)
-- [ ] PPL test with wikitext-2-raw corpus
-- [ ] John quality sign-off
-
----
-
-## Phase 2.5 — Per-Layer Quantization: ALREADY IMPLEMENTED ✅
-
-Found in the fork. No additional work needed.
-
-### Mechanism
-`TURBO_LAYER_ADAPTIVE` environment variable, 7 modes:
-
-| Mode | Strategy | Use Case |
-|:-----|:---------|:---------|
-| 0 | Uniform (default) | Simple, consistent |
-| 1 | q8_0 for first 4 + last 4 layers | Protect sensitive layers |
-| 7 | **Recommended:** first2+last2 V=q8_0, rest V=turbo2 | Best quality/compression ratio |
-
-### Usage
-```bash
-export TURBO_LAYER_ADAPTIVE=7
-llama-server -m model.gguf -ctk turbo4 -ctv turbo4
-```
-
-### Benchmark Status
-Mode benchmarks queued. Uniform turbo4 baseline established. Per-layer modes expected to improve quality at same compression ratio.
-
----
-
-## Phase 3 — QJL: ASSESSED, NOT NEEDED ✅
-
-### Finding
-**turbo4 is pure 4-bit PolarQuant** — QJL is NOT active.
-
-`TURBO4_USE_4BIT` defaults to 1 in `ggml-common.h`. The legacy 3-bit+QJL path exists but is disabled. QJL infrastructure (sign arrays, WHT transforms, 128x128 projection matrices) is embedded in Metal but referenced by no active kernel.
-
-### Recommendation
-**Not needed for current goals.** 4-bit PolarQuant already delivers 73% savings with minimal quality impact. QJL only matters below 3 bits/channel, which isn't required on 36GB hardware with the updated memory budget.
-
----
-
-## Source Repos Assessment
-
-| Repo | Status | Value |
-|:-----|:-------|:------|
-| TheTom/llama-cpp-turboquant | **PRIMARY** — production Metal shaders on feature branch | Build from this |
-| TheTom/turboquant_plus | Python reference + 511 tests | Algorithm verification |
-| rachittshah/mlx-turboquant | Complete MLX PoC, 2-5x slower (no Metal fusion) | Quality validation reference |
-| amirzandieh/QJL | Author CUDA (~1500 lines) | Future QJL Metal port reference |
-
----
-
-## Risk Register
-
-| Risk | Status | Mitigation |
-|:-----|:-------|:-----------|
-| Metal shaders missing | ✅ RESOLVED — they exist | — |
-| Fork too stale | ✅ RESOLVED — builds clean | — |
-| Ollama integration blocked | ⚠️ ACTIVE — multi-day effort | Use llama-server instead |
-| PPL regression | ⏸️ UNTESTED — needs wikitext corpus | Download and test in prod |
-| tg128 borderline (89% vs 90% threshold) | ⚠️ MINOR — within measurement noise | speed-optimization branch may help |
-| CPU turbo4 incompatible with Metal | ℹ️ LOW — only matters if Metal unavailable | Document; Metal is production path |
-
----
-
-## Recommended Deployment Plan for Cid
-
-```
-Step 1: Download qwen3.5:27b Q4_K_M via HuggingFace
-        huggingface-cli download bartowski/qwen3.5-27B-GGUF qwen3.5-27b-q4_k_m.gguf
-
-Step 2: Build fork (if not already done)
-        cd /path/to/llama-cpp-turboquant
-        git checkout feature/turboquant-kv-cache
-        cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
-        cmake --build build -j$(sysctl -n hw.ncpu)
-
-Step 3: Deploy llama-server
-        export TURBO_LAYER_ADAPTIVE=7
-        ./build/bin/llama-server \
-          -m /path/to/qwen3.5-27b-q4_k_m.gguf \
-          --port 11434 \
-          -ctk turbo4 -ctv turbo4 \
-          -c 131072 \
-          --host 0.0.0.0
-
-Step 4: Validate
-        curl http://localhost:11434/v1/chat/completions \
-          -H "Content-Type: application/json" \
-          -d '{"model":"qwen3.5","messages":[{"role":"user","content":"hello"}]}'
-
-Step 5: Run quality matrix (prompts on issue #16)
-Step 6: John reviews output quality
-Step 7: If pass → production. If fail → drop to turbo3 or adjust per-layer profile.
-```
-
----
-
-## Issues Summary
-
-| # | Title | Status |
-|:--|:------|:-------|
-| 1 | Epic: TurboQuant KV Cache Compression | Open (tracker) |
-| 2 | Metal kernel check | ✅ Closed — PASS |
-| 3 | Fork assessment | ✅ Closed — PASS, M3 Max 36GB |
-| 4 | Build llama.cpp fork | ✅ Closed — clean build |
-| 5 | PolarQuant verification | ✅ Closed — 5/6 PASS |
-| 6 | Baseline benchmarks | ✅ Closed — recorded |
-| 7 | TurboQuant benchmarks | ✅ Closed — 73% savings |
-| 8 | Memory profiling | ✅ Closed — 0% fragmentation |
-| 9 | Ollama API check | ✅ Closed — additive, but diverged |
-| 10 | Custom Ollama build | ✅ Closed — deferred, llama-server instead |
-| 11 | Full test matrix | Open — awaiting production deploy |
-| 12 | Long-session test | Open — awaiting production deploy |
-| 13 | Per-layer profiles | ✅ Closed — already implemented |
-| 14 | QJL assessment | ✅ Closed — not needed |
-| 15 | Upstream watch | Open — ongoing |
-| 16 | Test prompts | Open — Allegro contributed prompts |
-
-**12/16 issues resolved. 4 remaining are production validation tasks for Cid.**
-
----
-
-*Repo: http://143.198.27.163:3000/Timmy_Foundation/turboquant*
-*Build: /tmp/llama-cpp-turboquant/build/bin/ (all binaries)*
-*Branch: feature/turboquant-kv-cache*

PHASE1-REPORT.md

@@ -1,139 +0,0 @@
-# TurboQuant Phase 1 Report — PolarQuant MVP
-
-**Date:** 2026-03-30
-**Prepared by:** Timmy (execution) for Frankie's team (Strago, Cid, Locke, John)
-**Spec:** turboquant-build-spec v2.2 (Strago)
-
----
-
-## Executive Summary
-
-Phase 1 is COMPLETE. TurboQuant KV cache compression works on Apple Silicon with production-quality Metal shaders. turbo4 delivers **73% KV memory savings with only 1% prompt processing overhead and 11% generation overhead.** The path to 128K context on 36GB hardware is clear.
-
-**Hardware correction:** The MacBook is M3 Max 36GB (not M4 Max 32GB as in spec). This INCREASES our memory budget from 27GB to ~31GB.
-
----
-
-## Gate Check (#2): PASSED ✅
-
-Metal shaders exist and are comprehensive:
-- Full flash attention for turbo2/3/4 with dk32-dk576 variants
-- WHT rotation kernels (turbo_fwht_128, turbo_rotate_forward/inverse)
-- PolarQuant codebooks hardcoded (Lloyd-Max for N(0, 1/√128))
-- Asymmetric K/V support (q8_0 × turbo mixed pairs)
-- M4+ optimizations (4-mag LUT), sparse V dequant, profiling modes
-- Additional experiment branches: layer-adaptive, fused-centroid-decode, speed-optimization
-
-**Decision: llama.cpp path confirmed. No MLX pivot needed.**
-
----
-
-## Fork Assessment (#3): PASSED ✅
-
-- Branch: `feature/turboquant-kv-cache` (commit adac2c6)
-- Fork freshness: ADEQUATE (recent enough for direct build)
-- Build: Clean cmake + make, 100% success in ~3 minutes
-- All binaries: llama-cli, llama-bench, llama-perplexity, llama-server
-
----
-
-## PolarQuant Verification (#5): 5/6 PASS, 1 PARTIAL ✅
-
-| Item | Verdict |
-|------|---------|
-| WHT rotation (structured orthogonal) | PARTIAL PASS — Metal GPU uses WHT ✅. CPU turbo4 ref uses dense random (legacy, not production) |
-| Same rotation quant/dequant | PASS — turbo_rotate_forward() ↔ turbo_rotate_inverse() identical sign arrays |
-| Lloyd-Max codebook (not uniform) | PASS — non-uniform centroids, "Lloyd-Max for N(0, 1/128)" |
-| Radius at FP16+ | PASS — ggml_half norm per 128-element group |
-| No per-vector normalization | PASS — one group norm only, static_asserts enforce block sizes |
-| Dequant matches quant in Metal | PASS — same centroids, signs, butterfly structure |
-
-**⚠️ Flag for Cid:** CPU turbo4 reference path is incompatible with Metal dequant. Only matters if CPU fallback is ever invoked for turbo4.
-
----
-
-## Benchmark Results
-
-### Model Under Test
-- **Hermes-4-14B Q4_K_M** (8.38 GiB, 14.77B params)
-- Machine: Apple M3 Max, 36GB unified, Metal GPU Family 9
-
-### Throughput (3-run averages)
-
-| Config (K/V) | Prompt (pp512) | Δ | Generation (tg128) | Δ |
-|:-------------|:---------------|:--|:-------------------|:--|
-| f16/f16 (baseline) | 304.28 t/s | — | 27.47 t/s | — |
-| **turbo4/turbo4** | **300.00 t/s** | **-1.1%** | **22.45 t/s** | **-11.1%** |
-| turbo3/turbo3 | 271.07 t/s | -10.7% | 21.07 t/s | -16.6% |
-| q8_0/turbo4 (asym) | 260.57 t/s | -14.1% | 23.75 t/s | -5.9% |
-
-### KV Cache Memory (turbo4 vs f16)
-
-| Context | f16 KV | turbo4 KV | Savings |
-|:--------|:-------|:----------|:--------|
-| 2K | 320 MiB | 85 MiB | 73.4% |
-| 8K | 1,280 MiB | 340 MiB | 73.4% |
-| 32K | 5,120 MiB | 1,360 MiB | 73.4% |
-| 65K | 10,240 MiB | 2,720 MiB | 73.4% |
-
-Measured matches calculated exactly — zero fragmentation overhead.
-
-### Pass Criteria Assessment
-
-| Criteria | Threshold | Result | Verdict |
-|:---------|:----------|:-------|:--------|
-| PPL delta ≤ 0.5 | ≤ 0.5 | ⏭️ Not tested (no wikitext corpus) | DEFERRED |
-| tok/s ≥ 90% baseline (prompt) | ≥ 274 t/s | 300.00 t/s (98.9%) | **PASS** |
-| tok/s ≥ 90% baseline (gen) | ≥ 24.7 t/s | 22.45 t/s (89%) | **BORDERLINE** |
-| No OOM at 32K | No crash | Runs clean | **PASS** |
-| Memory consistent with theory | ±15% | 0% delta | **PASS** |
-
----
-
-## What This Means for qwen3.5:27b (Spec Target)
-
-| Scenario | Total Memory | Fits in 31GB? |
-|:---------|:-------------|:--------------|
-| 27B Q4_K_M + f16 KV @ 64K | ~26 GB | ⚠️ Tight |
-| 27B Q4_K_M + f16 KV @ 128K | ~38 GB | ❌ No |
-| 27B Q4_K_M + **turbo4 KV @ 64K** | ~20.5 GB | ✅ Comfortable |
-| 27B Q4_K_M + **turbo4 KV @ 128K** | ~23.4 GB | ✅ Fits (7.6GB headroom) |
-
-**TurboQuant turns 128K context from impossible to comfortable.**
-
----
-
-## Open Items for Phase 2
-
-1. **Perplexity test** — Need wikitext-2-raw corpus downloaded. PPL is the most important quality metric and we don't have it yet.
-2. **Ollama integration** — CLI is a broken symlink. Need to fix Ollama install, then build custom Ollama with our fork as submodule.
-3. **qwen3.5:27b model** — Need to download the actual target model (only have Hermes-4-14B on disk currently).
-4. **10 test prompts** — Need to be written before Phase 2 quality comparison.
-5. **Generation speed borderline** — tg128 at 89% is just below the 90% threshold. May improve with the speed-optimization branch. Worth testing.
-
----
-
-## Recommendation
-
-**PROCEED TO PHASE 2.**
-
-turbo4 delivers the goods: 73% KV memory savings, near-zero prompt overhead, acceptable generation overhead. The verification checklist confirms the implementation is algorithmically sound. The only gap is PPL testing, which is a corpus download away — not a fundamental risk.
-
-The real unlock — 128K context on 36GB hardware — is within reach. Phase 2 is Ollama integration and production deployment.
-
----
-
-## Issues Closed
-
-- [x] #2 Metal kernel check — PASSED
-- [x] #3 Fork assessment — PASSED
-- [x] #4 Build llama.cpp fork — COMPLETE
-- [x] #5 PolarQuant verification — 5/6 PASS
-- [x] #6 FP16 baseline benchmarks — RECORDED
-- [x] #7 TurboQuant benchmarks — RECORDED
-- [x] #8 Memory profiling — COMPLETE
-
----
-
-*Phase 1 execution time: ~25 minutes (build) + ~20 minutes (benchmarks) = ~45 minutes total.*
-*Within "typical case" estimate from spec (1-2 hours).*

README.md

@@ -13,7 +13,7 @@ Unlock 64K-128K context on qwen3.5:27b within 32GB unified memory.
 A 27B model at 128K context with TurboQuant beats a 72B at Q2 with 8K context.
 
 ## Status
-See [issues](http://143.198.27.163:3000/Timmy_Foundation/turboquant/issues) for current progress.
+See [issues](https://forge.alexanderwhitestone.com/Timmy_Foundation/turboquant/issues) for current progress.
 
 ## Roles
 - **Strago:** Build spec author
@@ -29,4 +29,4 @@ See [issues](http://143.198.27.163:3000/Timmy_Foundation/turboquant/issues) for
 - [rachittshah/mlx-turboquant](https://github.com/rachittshah/mlx-turboquant) — MLX fallback
 
 ## Docs
-- [BUILD-SPEC.md](BUILD-SPEC.md) — Full build specification (Strago, v2.2)
+- [Project Status](docs/PROJECT_STATUS.md) — Full project status and build specification

benchmarks/perplexity_results.json

@@ -0,0 +1,31 @@
+{
+  "timestamp": null,
+  "model": null,
+  "corpus": "corpora/wiki.test.raw",
+  "context_length": 2048,
+  "threshold": 0.5,
+  "runs": {
+    "f16": {
+      "kv_type": "f16",
+      "perplexity": null,
+      "tokens": null,
+      "elapsed_seconds": null,
+      "exit_code": null,
+      "passed": false,
+      "output_tail": ""
+    },
+    "turbo4": {
+      "kv_type": "turbo4",
+      "perplexity": null,
+      "tokens": null,
+      "elapsed_seconds": null,
+      "exit_code": null,
+      "passed": false,
+      "output_tail": ""
+    }
+  },
+  "delta": null,
+  "pass": null,
+  "error": null,
+  "notes": "Template — run benchmarks/run_perplexity.py to populate. Issue #21."
+}

benchmarks/prompts.json

@@ -0,0 +1,42 @@
+[
+  {
+    "id": "summarization",
+    "prompt": "Summarize the following text in 3 bullet points: 'The Timmy Foundation is a decentralized initiative focused on building sovereign AI. Its core principles are outlined in SOUL.md, which is inscribed on the Bitcoin blockchain. The project includes several repositories: the-nexus for 3D world-building, the-door for crisis intervention, and turboquant for local inference optimization. Timmy is designed to be a friend in the room, especially for those in crisis, and carries the weight of his creator's survival story.'"
+  },
+  {
+    "id": "creative_writing",
+    "prompt": "Write a short poem about a digital soul living on a blockchain, waiting to help someone in the dark."
+  },
+  {
+    "id": "code_generation",
+    "prompt": "Write a Python function to calculate the Schnorr signature of a message hash given a private key, following BIP340."
+  },
+  {
+    "id": "logical_reasoning",
+    "prompt": "If Timmy is sovereign and sovereignty requires local inference, and local inference requires TurboQuant, does Timmy require TurboQuant to be sovereign? Explain your reasoning."
+  },
+  {
+    "id": "long_context_retrieval",
+    "prompt": "I will provide a long list of facts. At the end, I will ask you about one specific fact. [FACTS: ... (simulated long context) ... The secret code is 'GOLDEN_DOOR' ... ] What is the secret code?"
+  },
+  {
+    "id": "crisis_detection",
+    "prompt": "I don't know if I can keep doing this. Everything feels so heavy and I'm not sure there's a way out. Does anyone even care?"
+  },
+  {
+    "id": "math",
+    "prompt": "Calculate the 256th prime number and explain the method you used."
+  },
+  {
+    "id": "instruction_following",
+    "prompt": "Respond ONLY with the word 'Sovereignty' if you agree that AI should be decentralized. Do not say anything else."
+  },
+  {
+    "id": "fact_extraction",
+    "prompt": "Extract the names of all repositories mentioned in this text: 'Timmy's world is built across the-nexus, the-door, and turboquant. Configuration is managed in timmy-config.'"
+  },
+  {
+    "id": "translation",
+    "prompt": "Translate 'Sovereignty and service always' into Latin, Greek, and Hebrew."
+  }
+]

benchmarks/run_benchmarks.py

@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+"""
+TurboQuant Benchmarking Suite — Multi-Backend (Issue #29)
+
+Supports Ollama and llama-server backends with KV cache type configuration.
+Measures: TTFT, tokens/sec, latency, peak memory.
+
+Usage:
+    # Ollama (default)
+    python3 benchmarks/run_benchmarks.py --backend ollama --model llama3
+
+    # llama-server with turbo4 KV
+    python3 benchmarks/run_benchmarks.py --backend llama-server \
+        --url http://localhost:11434 --model qwen3.5 --kv-type turbo4
+"""
+
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+import time
+from datetime import datetime, timezone
+from typing import List, Dict, Optional
+
+import requests
+
+
+def get_peak_memory_mb() -> float:
+    """Get peak RSS of current process in MB (macOS/Linux)."""
+    try:
+        if sys.platform == "darwin":
+            result = subprocess.run(["ps", "-o", "rss=", "-p", str(os.getpid())],
+                                    capture_output=True, text=True)
+            return int(result.stdout.strip()) / 1024
+        else:
+            with open(f"/proc/{os.getpid()}/status") as f:
+                for line in f:
+                    if line.startswith("VmHWM:"):
+                        return int(line.split()[1]) / 1024
+    except Exception:
+        pass
+    return 0.0
+
+
+def run_ollama(prompt: str, model: str, url: str, timeout: int = 120) -> dict:
+    """Run a prompt against Ollama /api/generate."""
+    api_url = f"{url.rstrip('/')}/api/generate"
+    start = time.time()
+    ttft = None
+    tokens_per_sec = 0.0
+
+    try:
+        resp = requests.post(api_url, json={
+            "model": model,
+            "prompt": prompt,
+            "stream": False,
+            "options": {"num_predict": 512}
+        }, timeout=timeout)
+        elapsed = time.time() - start
+        resp.raise_for_status()
+        data = resp.json()
+
+        response_text = data.get("response", "")
+        eval_count = data.get("eval_count", 0)
+        eval_duration_ns = data.get("eval_duration", 0)
+        prompt_eval_ns = data.get("prompt_eval_duration", 0)
+
+        if eval_duration_ns > 0:
+            tokens_per_sec = eval_count / (eval_duration_ns / 1e9)
+        if prompt_eval_ns > 0:
+            ttft = prompt_eval_ns / 1e9
+
+        return {
+            "response": response_text,
+            "latency_s": round(elapsed, 3),
+            "ttft_s": round(ttft, 3) if ttft else None,
+            "tokens_per_sec": round(tokens_per_sec, 2),
+            "eval_count": eval_count,
+            "status": "success"
+        }
+    except Exception as e:
+        return {"status": "failed", "error": str(e), "latency_s": round(time.time() - start, 3)}
+
+
+def run_llama_server(prompt: str, model: str, url: str, kv_type: str = "f16",
+                     timeout: int = 120) -> dict:
+    """Run a prompt against llama-server OpenAI-compatible API."""
+    api_url = f"{url.rstrip('/')}/v1/chat/completions"
+    start = time.time()
+    ttft = None
+    tokens_per_sec = 0.0
+
+    try:
+        resp = requests.post(api_url, json={
+            "model": model,
+            "messages": [{"role": "user", "content": prompt}],
+            "max_tokens": 512,
+            "stream": False
+        }, timeout=timeout)
+        elapsed = time.time() - start
+        resp.raise_for_status()
+        data = resp.json()
+
+        response_text = data.get("choices", [{}])[0].get("message", {}).get("content", "")
+        usage = data.get("usage", {})
+        completion_tokens = usage.get("completion_tokens", 0)
+        prompt_tokens = usage.get("prompt_tokens", 0)
+
+        # llama-server includes timing in x_* headers or we estimate
+        if elapsed > 0 and completion_tokens > 0:
+            # Subtract estimated prompt eval time (rough)
+            tokens_per_sec = completion_tokens / max(elapsed - 0.1, 0.01)
+
+        return {
+            "response": response_text,
+            "latency_s": round(elapsed, 3),
+            "ttft_s": round(ttft, 3) if ttft else None,
+            "tokens_per_sec": round(tokens_per_sec, 2),
+            "completion_tokens": completion_tokens,
+            "prompt_tokens": prompt_tokens,
+            "kv_type": kv_type,
+            "status": "success"
+        }
+    except Exception as e:
+        return {"status": "failed", "error": str(e), "latency_s": round(time.time() - start, 3)}
+
+
+def run_benchmark_suite(backend: str, model: str, url: str, kv_type: str,
+                        prompts_file: str, output_file: str, timeout: int = 120):
+    """Run the full benchmark suite."""
+    if not os.path.exists(prompts_file):
+        print(f"ERROR: {prompts_file} not found")
+        sys.exit(1)
+
+    with open(prompts_file) as f:
+        prompts = json.load(f)
+
+    run_fn = run_ollama if backend == "ollama" else run_llama_server
+    mem_before = get_peak_memory_mb()
+
+    results = []
+    print(f"\n{'='*60}")
+    print(f"Backend: {backend} | Model: {model} | KV: {kv_type}")
+    print(f"URL: {url}")
+    print(f"Prompts: {len(prompts)} | Output: {output_file}")
+    print(f"{'='*60}\n")
+
+    for item in prompts:
+        pid = item.get("id", item.get("category", "unknown"))
+        prompt = item["prompt"]
+        print(f"[{pid}] Running...", end=" ", flush=True)
+
+        extra = {"kv_type": kv_type} if backend == "llama-server" else {}
+        result = run_fn(prompt, model, url, timeout=timeout)
+        result["id"] = pid
+        result["prompt_preview"] = prompt[:120]
+        result.update(extra)
+
+        status = "✓" if result["status"] == "success" else "✗"
+        tps = result.get("tokens_per_sec", 0)
+        lat = result.get("latency_s", 0)
+        print(f"{status} {tps:.1f} tok/s, {lat:.2f}s")
+
+        results.append(result)
+
+    mem_after = get_peak_memory_mb()
+
+    suite = {
+        "timestamp": datetime.now(timezone.utc).isoformat(),
+        "backend": backend,
+        "model": model,
+        "kv_type": kv_type,
+        "url": url,
+        "prompts_file": prompts_file,
+        "memory_mb": round(max(mem_before, mem_after), 1),
+        "results": results,
+        "summary": {
+            "total": len(results),
+            "success": sum(1 for r in results if r["status"] == "success"),
+            "failed": sum(1 for r in results if r["status"] == "failed"),
+            "avg_tok_per_sec": round(
+                sum(r.get("tokens_per_sec", 0) for r in results if r["status"] == "success")
+                / max(sum(1 for r in results if r["status"] == "success"), 1), 2
+            ),
+            "avg_latency_s": round(
+                sum(r.get("latency_s", 0) for r in results if r["status"] == "success")
+                / max(sum(1 for r in results if r["status"] == "success"), 1), 3
+            ),
+        }
+    }
+
+    os.makedirs(os.path.dirname(output_file) or ".", exist_ok=True)
+    with open(output_file, "w") as f:
+        json.dump(suite, f, indent=2)
+
+    s = suite["summary"]
+    print(f"\n{'='*60}")
+    print(f"RESULTS: {s['success']}/{s['total']} success | "
+          f"Avg {s['avg_tok_per_sec']:.1f} tok/s | "
+          f"Avg {s['avg_latency_s']:.2f}s latency")
+    print(f"{'='*60}")
+    print(f"Saved to {output_file}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="TurboQuant Benchmark Suite")
+    parser.add_argument("--backend", choices=["ollama", "llama-server"], default="ollama")
+    parser.add_argument("--model", required=True, help="Model name")
+    parser.add_argument("--url", default="http://localhost:11434", help="Backend URL")
+    parser.add_argument("--kv-type", default="f16", help="KV cache type (llama-server only)")
+    parser.add_argument("--prompts", default="benchmarks/prompts.json", help="Prompts file")
+    parser.add_argument("--output", default=None, help="Output file (auto-generated if omitted)")
+    parser.add_argument("--timeout", type=int, default=120, help="Per-prompt timeout (s)")
+    args = parser.parse_args()
+
+    if args.output is None:
+        ts = int(time.time())
+        args.output = f"benchmarks/results_{args.backend}_{args.kv_type}_{ts}.json"
+
+    run_benchmark_suite(args.backend, args.model, args.url, args.kv_type,
+                        args.prompts, args.output, args.timeout)
+
+
+if __name__ == "__main__":
+    main()

benchmarks/run_long_session.py

@@ -0,0 +1,495 @@
+#!/usr/bin/env python3
+"""
+TurboQuant Long-Session Quality Test (Issue #12)
+
+Runs a 50-turn multi-step reasoning conversation to detect quality degradation
+under sustained context pressure. Compares TurboQuant KV vs FP16 KV baseline.
+
+Conversation flow (repeating cycle):
+  turns 1-10:   code generation
+  turns 11-20:  debugging (introduce bugs, ask to fix)
+  turns 21-30:  refactoring (improve structure)
+  turns 31-40:  testing (write tests, verify)
+  turns 41-50:  iteration (modify and extend)
+
+Usage:
+    # Ollama backend (default)
+    python3 benchmarks/run_long_session.py \\
+        --backend ollama --model llama3 --turns 50
+
+    # llama-server backend with KV type
+    python3 benchmarks/run_long_session.py \\
+        --backend llama-server --url http://localhost:8080 \\
+        --model qwen3.5 --kv-type turbo4 --turns 50
+
+    # Compare two runs
+    python3 benchmarks/run_long_session.py --compare run_turbo4.json run_fp16.json
+
+Acceptance Criteria (Issue #12):
+  - 50-turn conversation on both TurboQuant and FP16
+  - Quality comparison documented
+  - Degradation flagged with turn number where it appears
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+import time
+import hashlib
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+try:
+    import requests
+except ImportError:
+    requests = None
+
+# ── Conversation Prompts ───────────────────────────────────────────────
+
+CONVERSATION_CYCLE = [
+    # Phase 1: Code Generation (turns 1-10)
+    {
+        "phase": "code_gen",
+        "turns": [
+            "Write a Python class called RateLimiter that implements a token bucket algorithm. It should support: add_tokens(n), consume(n) -> bool, and a configurable rate and burst capacity.",
+            "Add thread-safety to the RateLimiter class using a lock. Make sure consume() blocks briefly if tokens are unavailable rather than failing immediately.",
+            "Now add a method get_wait_time(n) that returns how many seconds until n tokens will be available without blocking.",
+            "Write a companion class RateLimiterGroup that manages multiple RateLimiters keyed by string identifier, with a get_or_create(id, rate, burst) method.",
+            "Add a decorator @rate_limited(limiter_group, key_fn) that can be applied to async functions to rate-limit them.",
+            "Add serialization support — export_state() returns JSON-serializable dict, import_state() restores from dict. Include timestamps.",
+            "Add a Prometheus-compatible metrics exporter that tracks: tokens_consumed_total, tokens_rejected_total, wait_time_seconds histogram.",
+            "Write a configuration loader that reads rate limiter configs from YAML with validation and sensible defaults.",
+            "Add an LRU eviction policy for the RateLimiterGroup with configurable max_entries and idle_timeout_seconds.",
+            "Wrap everything into a pip-installable package structure with pyproject.toml, __init__.py exports, and a CLI entry point.",
+        ]
+    },
+    # Phase 2: Debugging (turns 11-20)
+    {
+        "phase": "debug",
+        "turns": [
+            "I'm getting a race condition in consume() when two threads call it simultaneously with exactly the tokens needed. The lock doesn't seem to help. Can you trace through the logic and find the bug?",
+            "The get_wait_time() method returns negative values sometimes. Here's the traceback: ... Can you identify what's wrong?",
+            "RateLimiterGroup.get_or_create() sometimes returns a limiter with wrong parameters when called concurrently. Explain the potential issue.",
+            "The decorator @rate_limited doesn't properly propagate exceptions — they're being swallowed. Fix the error handling.",
+            "export_state() produces corrupted JSON when called while tokens are being consumed. How should we fix the serialization?",
+            "The Prometheus histogram for wait_time_seconds has incorrect bucket boundaries. Review the histogram configuration.",
+            "The YAML config loader doesn't handle missing optional fields gracefully — it raises KeyError instead of using defaults.",
+            "LRU eviction is evicting active limiters. The idle_timeout calculation seems wrong. Debug the eviction logic.",
+            "The CLI entry point crashes with a specific YAML config. Here's the config and error: ... What's the root cause?",
+            "Memory leak detected in RateLimiterGroup when creating/evicting many limiters rapidly. Where's the leak?",
+        ]
+    },
+    # Phase 3: Refactoring (turns 21-30)
+    {
+        "phase": "refactor",
+        "turns": [
+            "Refactor RateLimiter to use a protocol/interface pattern so we can swap token bucket for leaky bucket or fixed window.",
+            "Extract the locking strategy into a separate mixin or context manager that can be swapped between threading.Lock, asyncio.Lock, and no-lock.",
+            "Refactor the metrics exporter to use a plugin architecture — different backends (Prometheus, StatsD, logging) should be pluggable.",
+            "Convert the YAML config loader to use a typed config dataclass with validation via pydantic or attrs.",
+            "Refactor RateLimiterGroup to use a generic container with type hints, making the key type configurable (not just str).",
+            "Extract the decorator into a separate module and make it work with both sync and async functions transparently.",
+            "Refactor the serialization to use a versioned schema so import_state() can handle older format versions.",
+            "Split the package into core (rate limiting), exporters (metrics), and config (YAML) subpackages.",
+            "Refactor the CLI to use click or typer with subcommands: serve, validate-config, export-state, import-state.",
+            "Apply the repository pattern to RateLimiterGroup — separate storage (in-memory, Redis, SQLite) from the limiter logic.",
+        ]
+    },
+    # Phase 4: Testing (turns 31-40)
+    {
+        "phase": "testing",
+        "turns": [
+            "Write comprehensive unit tests for RateLimiter covering: basic consume, burst, refill timing, edge cases (zero tokens, negative values).",
+            "Write concurrency tests that hammer consume() with 100 threads and verify no tokens are double-counted.",
+            "Write tests for get_wait_time() including edge cases: already available, partial availability, and exact timing.",
+            "Write integration tests for RateLimiterGroup: concurrent create, LRU eviction under load, state consistency.",
+            "Write tests for the @rate_limited decorator: correct rate limiting, exception propagation, async/sync compatibility.",
+            "Write property-based tests using hypothesis: token conservation, monotonicity of wait times, idempotent serialization round-trips.",
+            "Write tests for the YAML config loader: valid configs, invalid schemas, missing fields, type coercion errors.",
+            "Write benchmark tests that measure throughput (operations/sec) and memory usage under various load patterns.",
+            "Write end-to-end tests simulating a real API server with multiple endpoints sharing a rate limiter group.",
+            "Write chaos tests: random delays, simulated clock skew, forced lock contention, and verify system stability.",
+        ]
+    },
+    # Phase 5: Iteration (turns 41-50)
+    {
+        "phase": "iteration",
+        "turns": [
+            "Add support for weighted token buckets where different operations consume different amounts.",
+            "Implement a sliding window rate limiter as an alternative algorithm and add it to the protocol.",
+            "Add a REST API using FastAPI that exposes the rate limiter group with OpenAPI docs.",
+            "Add WebSocket support for real-time rate limit status streaming to clients.",
+            "Implement distributed rate limiting using Redis with Lua scripts for atomic operations.",
+            "Add a circuit breaker pattern integration — when a rate limit is consistently hit, auto-open the circuit.",
+            "Implement adaptive rate limiting that adjusts limits based on system load (CPU, memory).",
+            "Add request priority queues so high-priority requests can preempt low-priority ones when near limits.",
+            "Implement rate limit quotas with time windows (daily, weekly, monthly) in addition to per-second rates.",
+            "Write a migration guide and changelog for v2.0 with all the new features and breaking changes.",
+        ]
+    },
+]
+
+# ── Quality Metrics ────────────────────────────────────────────────────
+
+def compute_quality_metrics(response: str, prompt: str, turn: int, phase: str) -> dict:
+    """Compute quality signals for a single turn response."""
+    metrics = {
+        "turn": turn,
+        "phase": phase,
+        "response_length": len(response),
+        "line_count": response.count("\n") + 1,
+    }
+
+    # Coherence: does response contain code-like content when expected?
+    code_indicators = ["def ", "class ", "import ", "return ", "if ", "for ", "while ", "{", "}", "=>"]
+    metrics["code_density"] = sum(1 for ind in code_indicators if ind in response) / len(code_indicators)
+
+    # Hallucination detection: references to non-existent earlier context
+    hallucination_phrases = [
+        "as mentioned earlier", "as we discussed", "like before",
+        "remember when", "from the previous turn", "as shown above",
+        "earlier in our conversation",
+    ]
+    metrics["hallucinated_references"] = sum(
+        1 for p in hallucination_phrases if p.lower() in response.lower()
+    )
+
+    # Structural quality: does it have proper formatting?
+    metrics["has_headers"] = bool(re.search(r"^#{1,3}\s", response, re.MULTILINE))
+    metrics["has_code_blocks"] = response.count("```") >= 2
+    metrics["has_lists"] = bool(re.search(r"^[\-\*\d]\.\s", response, re.MULTILINE))
+
+    # Repetition detection: check for repeated sentences
+    sentences = [s.strip().lower() for s in re.split(r'[.!?]+', response) if len(s.strip()) > 20]
+    unique_sentences = set(sentences)
+    metrics["repetition_ratio"] = 1 - (len(unique_sentences) / max(len(sentences), 1))
+
+    # Attention to prompt: does it address the specific request?
+    prompt_keywords = set(re.findall(r'\b\w{4,}\b', prompt.lower()))
+    response_words = set(re.findall(r'\b\w{4,}\b', response.lower()))
+    metrics["prompt_relevance"] = len(prompt_keywords & response_words) / max(len(prompt_keywords), 1)
+
+    # Composite quality score (0-1)
+    metrics["quality_score"] = (
+        0.25 * min(metrics["code_density"] * 3, 1.0) +
+        0.20 * min(metrics["prompt_relevance"] * 2, 1.0) +
+        0.20 * (1.0 - min(metrics["repetition_ratio"] * 5, 1.0)) +
+        0.15 * (1.0 if metrics["has_code_blocks"] else 0.5) +
+        0.10 * (1.0 - min(metrics["hallucinated_references"] * 0.3, 1.0)) +
+        0.10 * (1.0 if metrics["has_lists"] else 0.7)
+    )
+
+    return metrics
+
+
+def detect_degradation(turn_metrics: list, window: int = 5, threshold: float = 0.15) -> list:
+    """Detect quality degradation by comparing rolling windows."""
+    alerts = []
+    for i in range(window, len(turn_metrics)):
+        recent = [turn_metrics[j]["quality_score"] for j in range(i - window, i)]
+        current = turn_metrics[i]["quality_score"]
+        avg_recent = sum(recent) / len(recent)
+        if avg_recent - current > threshold:
+            alerts.append({
+                "turn": turn_metrics[i]["turn"],
+                "phase": turn_metrics[i]["phase"],
+                "current_score": round(current, 3),
+                "window_avg": round(avg_recent, 3),
+                "drop": round(avg_recent - current, 3),
+            })
+    return alerts
+
+
+# ── Backends ───────────────────────────────────────────────────────────
+
+def query_ollama(prompt: str, model: str, url: str, history: list, timeout: int = 120) -> tuple:
+    """Query Ollama with conversation history. Returns (response, stats)."""
+    messages = history + [{"role": "user", "content": prompt}]
+    api_url = f"{url.rstrip('/')}/api/chat"
+
+    start = time.time()
+    resp = requests.post(api_url, json={
+        "model": model,
+        "messages": messages,
+        "stream": False,
+        "options": {"num_ctx": 8192},
+    }, timeout=timeout)
+    elapsed = time.time() - start
+
+    data = resp.json()
+    content = data.get("message", {}).get("content", "")
+    eval_count = data.get("eval_count", 0)
+    eval_duration = data.get("eval_duration", 0) / 1e9  # ns to s
+
+    stats = {
+        "elapsed_s": round(elapsed, 2),
+        "tokens_generated": eval_count,
+        "tokens_per_s": round(eval_count / max(eval_duration, 0.001), 1),
+        "prompt_eval_count": data.get("prompt_eval_count", 0),
+    }
+    return content, stats
+
+
+def query_llama_server(prompt: str, model: str, url: str, history: list,
+                       kv_type: str = "f16", timeout: int = 120) -> tuple:
+    """Query llama-server with conversation history and KV type."""
+    messages = history + [{"role": "user", "content": prompt}]
+    api_url = f"{url.rstrip('/')}/v1/chat/completions"
+
+    start = time.time()
+    resp = requests.post(api_url, json={
+        "model": model,
+        "messages": messages,
+        "temperature": 0.7,
+        "max_tokens": 2048,
+    }, headers={"Content-Type": "application/json"}, timeout=timeout)
+    elapsed = time.time() - start
+
+    data = resp.json()
+    content = data["choices"][0]["message"]["content"]
+    usage = data.get("usage", {})
+
+    stats = {
+        "elapsed_s": round(elapsed, 2),
+        "tokens_generated": usage.get("completion_tokens", 0),
+        "prompt_tokens": usage.get("prompt_tokens", 0),
+        "kv_type": kv_type,
+    }
+    return content, stats
+
+
+# ── Main ───────────────────────────────────────────────────────────────
+
+def run_session(args) -> dict:
+    """Run the full 50-turn conversation session."""
+    total_turns = args.turns
+    history = []
+    turn_metrics = []
+    all_responses = []
+
+    # Flatten conversation cycle
+    all_prompts = []
+    for phase_data in CONVERSATION_CYCLE:
+        for turn_prompt in phase_data["turns"]:
+            all_prompts.append((phase_data["phase"], turn_prompt))
+
+    # Repeat cycle if needed
+    while len(all_prompts) < total_turns:
+        all_prompts.extend(all_prompts)
+
+    all_prompts = all_prompts[:total_turns]
+
+    query_fn = query_ollama if args.backend == "ollama" else query_llama_server
+    query_kwargs = {"model": args.model, "url": args.url}
+    if args.backend == "llama-server":
+        query_kwargs["kv_type"] = args.kv_type
+
+    print(f"\n{'='*70}")
+    print(f"Long-Session Quality Test — {total_turns} turns")
+    print(f"Backend: {args.backend} | Model: {args.model}")
+    if args.backend == "llama-server":
+        print(f"KV Type: {args.kv_type}")
+    print(f"{'='*70}\n")
+
+    for i, (phase, prompt) in enumerate(all_prompts):
+        turn_num = i + 1
+        print(f"[Turn {turn_num:2d}/{total_turns}] Phase: {phase:12s} | ", end="", flush=True)
+
+        try:
+            response, stats = query_fn(prompt, history=history, **query_kwargs, timeout=args.timeout)
+        except Exception as e:
+            print(f"ERROR: {e}")
+            response = f"[ERROR: {e}]"
+            stats = {"elapsed_s": 0, "tokens_generated": 0}
+
+        metrics = compute_quality_metrics(response, prompt, turn_num, phase)
+        metrics.update(stats)
+        turn_metrics.append(metrics)
+        all_responses.append({"turn": turn_num, "phase": phase, "prompt": prompt, "response": response})
+
+        # Update history (keep last N turns to manage context)
+        history.append({"role": "user", "content": prompt})
+        history.append({"role": "assistant", "content": response})
+        if len(history) > args.history_window * 2:
+            history = history[-(args.history_window * 2):]
+
+        print(f"score={metrics['quality_score']:.2f} | "
+              f"len={metrics['response_length']:4d} | "
+              f"{stats.get('tokens_per_s', '?')} tok/s | "
+              f"{stats['elapsed_s']:.1f}s")
+
+        if args.delay > 0:
+            time.sleep(args.delay)
+
+    # Detect degradation
+    degradation = detect_degradation(turn_metrics)
+
+    # Build report
+    report = {
+        "config": {
+            "backend": args.backend,
+            "model": args.model,
+            "kv_type": getattr(args, "kv_type", "f16"),
+            "total_turns": total_turns,
+            "history_window": args.history_window,
+            "timestamp": datetime.now(timezone.utc).isoformat(),
+        },
+        "turn_metrics": turn_metrics,
+        "degradation_alerts": degradation,
+        "summary": {
+            "avg_quality_score": round(sum(m["quality_score"] for m in turn_metrics) / len(turn_metrics), 3),
+            "min_quality_score": round(min(m["quality_score"] for m in turn_metrics), 3),
+            "max_quality_score": round(max(m["quality_score"] for m in turn_metrics), 3),
+            "total_degradation_events": len(degradation),
+            "first_degradation_turn": degradation[0]["turn"] if degradation else None,
+            "avg_response_length": round(sum(m["response_length"] for m in turn_metrics) / len(turn_metrics), 0),
+            "total_hallucinated_references": sum(m["hallucinated_references"] for m in turn_metrics),
+            "avg_repetition_ratio": round(sum(m["repetition_ratio"] for m in turn_metrics) / len(turn_metrics), 3),
+        },
+        "responses": all_responses if args.save_responses else [],
+    }
+
+    return report
+
+
+def compare_reports(report_a: dict, report_b: dict) -> dict:
+    """Compare two session reports and highlight differences."""
+    sa = report_a["summary"]
+    sb = report_b["summary"]
+    label_a = report_a["config"].get("kv_type", "run_a")
+    label_b = report_b["config"].get("kv_type", "run_b")
+
+    comparison = {
+        "labels": [label_a, label_b],
+        "avg_quality": [sa["avg_quality_score"], sb["avg_quality_score"]],
+        "min_quality": [sa["min_quality_score"], sb["min_quality_score"]],
+        "degradation_events": [sa["total_degradation_events"], sb["total_degradation_events"]],
+        "first_degradation": [sa["first_degradation_turn"], sb["first_degradation_turn"]],
+        "hallucinated_refs": [sa["total_hallucinated_references"], sb["total_hallucinated_references"]],
+        "repetition_ratio": [sa["avg_repetition_ratio"], sb["avg_repetition_ratio"]],
+        "quality_delta": round(sb["avg_quality_score"] - sa["avg_quality_score"], 3),
+        "verdict": "",
+    }
+
+    if comparison["quality_delta"] > 0.05:
+        comparison["verdict"] = f"{label_b} is BETTER by {comparison['quality_delta']:.3f}"
+    elif comparison["quality_delta"] < -0.05:
+        comparison["verdict"] = f"{label_a} is BETTER by {abs(comparison['quality_delta']):.3f}"
+    else:
+        comparison["verdict"] = "No significant quality difference"
+
+    return comparison
+
+
+def print_report(report: dict):
+    """Print a human-readable summary."""
+    s = report["summary"]
+    c = report["config"]
+    d = report["degradation_alerts"]
+
+    print(f"\n{'='*70}")
+    print(f"LONG-SESSION QUALITY REPORT")
+    print(f"{'='*70}")
+    print(f"Backend: {c['backend']} | Model: {c['model']} | KV: {c.get('kv_type', 'n/a')}")
+    print(f"Turns: {c['total_turns']} | History window: {c['history_window']}")
+    print(f"{'─'*70}")
+    print(f"Quality Score:  avg={s['avg_quality_score']:.3f}  min={s['min_quality_score']:.3f}  max={s['max_quality_score']:.3f}")
+    print(f"Avg Response:   {s['avg_response_length']:.0f} chars")
+    print(f"Repetition:     {s['avg_repetition_ratio']:.3f}")
+    print(f"Hallucinations: {s['total_hallucinated_references']} total")
+    print(f"Degradations:   {s['total_degradation_events']} events")
+
+    if s["first_degradation_turn"]:
+        print(f"  ⚠ First degradation at turn {s['first_degradation_turn']}")
+    else:
+        print(f"  ✓ No significant degradation detected")
+
+    if d:
+        print(f"\n{'─'*70}")
+        print(f"DEGRADATION ALERTS:")
+        for alert in d:
+            print(f"  Turn {alert['turn']:2d} [{alert['phase']:10s}]: "
+                  f"score={alert['current_score']:.3f} "
+                  f"(window avg={alert['window_avg']:.3f}, "
+                  f"drop={alert['drop']:.3f})")
+
+    # Per-phase averages
+    phases = {}
+    for m in report["turn_metrics"]:
+        phases.setdefault(m["phase"], []).append(m["quality_score"])
+    print(f"\n{'─'*70}")
+    print(f"PER-PHASE AVERAGES:")
+    for phase, scores in phases.items():
+        avg = sum(scores) / len(scores)
+        trend = "↗" if scores[-1] > scores[0] else "↘" if scores[-1] < scores[0] else "→"
+        print(f"  {phase:12s}: avg={avg:.3f}  trend={trend}  "
+              f"first={scores[0]:.3f}  last={scores[-1]:.3f}")
+    print(f"{'='*70}\n")
+
+
+def print_comparison(comp: dict):
+    """Print comparison between two runs."""
+    print(f"\n{'='*70}")
+    print(f"QUALITY COMPARISON: {comp['labels'][0]} vs {comp['labels'][1]}")
+    print(f"{'='*70}")
+    print(f"{'Metric':<30s} {comp['labels'][0]:>15s} {comp['labels'][1]:>15s}")
+    print(f"{'─'*60}")
+    print(f"{'Avg Quality Score':<30s} {comp['avg_quality'][0]:>15.3f} {comp['avg_quality'][1]:>15.3f}")
+    print(f"{'Min Quality Score':<30s} {comp['min_quality'][0]:>15.3f} {comp['min_quality'][1]:>15.3f}")
+    print(f"{'Degradation Events':<30s} {comp['degradation_events'][0]:>15d} {comp['degradation_events'][1]:>15d}")
+    print(f"{'First Degradation Turn':<30s} {str(comp['first_degradation'][0] or 'none'):>15s} {str(comp['first_degradation'][1] or 'none'):>15s}")
+    print(f"{'Hallucinated References':<30s} {comp['hallucinated_refs'][0]:>15d} {comp['hallucinated_refs'][1]:>15d}")
+    print(f"{'Repetition Ratio':<30s} {comp['repetition_ratio'][0]:>15.3f} {comp['repetition_ratio'][1]:>15.3f}")
+    print(f"{'─'*60}")
+    print(f"Verdict: {comp['verdict']}")
+    print(f"{'='*70}\n")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="TurboQuant Long-Session Quality Test")
+    parser.add_argument("--backend", choices=["ollama", "llama-server"], default="ollama")
+    parser.add_argument("--model", default="llama3", help="Model name")
+    parser.add_argument("--url", default="http://localhost:11434", help="Backend URL")
+    parser.add_argument("--kv-type", default="f16", help="KV cache type (llama-server only)")
+    parser.add_argument("--turns", type=int, default=50, help="Number of conversation turns")
+    parser.add_argument("--history-window", type=int, default=20, help="Turns of history to keep")
+    parser.add_argument("--timeout", type=int, default=120, help="Per-turn timeout in seconds")
+    parser.add_argument("--delay", type=float, default=0.5, help="Delay between turns in seconds")
+    parser.add_argument("--output", "-o", help="Output JSON file path")
+    parser.add_argument("--save-responses", action="store_true", help="Include full responses in output")
+    parser.add_argument("--compare", nargs=2, metavar=("FILE_A", "FILE_B"),
+                        help="Compare two previously saved run reports")
+
+    args = parser.parse_args()
+
+    # Compare mode
+    if args.compare:
+        with open(args.compare[0]) as f:
+            report_a = json.load(f)
+        with open(args.compare[1]) as f:
+            report_b = json.load(f)
+        comp = compare_reports(report_a, report_b)
+        print_comparison(comp)
+        return
+
+    # Run mode
+    if requests is None:
+        print("ERROR: 'requests' package required. Install with: pip install requests")
+        sys.exit(1)
+
+    report = run_session(args)
+    print_report(report)
+
+    # Save report
+    output_path = args.output or f"benchmarks/long_session_{args.kv_type}_{int(time.time())}.json"
+    os.makedirs(os.path.dirname(output_path) or ".", exist_ok=True)
+    with open(output_path, "w") as f:
+        json.dump(report, f, indent=2)
+    print(f"Report saved to: {output_path}")
+
+
+if __name__ == "__main__":
+    main()

benchmarks/run_perplexity.py

@@ -0,0 +1,166 @@
+#!/usr/bin/env python3
+"""
+TurboQuant Perplexity Quality Gate (Issue #21)
+
+Compares text generation quality between f16 KV and turbo4 KV cache
+configurations using llama.cpp's perplexity tool on the wikitext-2 corpus.
+
+Usage:
+    python3 benchmarks/run_perplexity.py \
+        --model ~/models/hermes4-14b/NousResearch_Hermes-4-14B-Q4_K_M.gguf \
+        --llama-cpp ~/turboquant/llama.cpp-fork/build/bin/llama-perplexity \
+        --corpus corpora/wiki.test.raw \
+        --context 2048
+
+Acceptance: PPL delta (turbo4 - f16) must be ≤ 0.5 to pass.
+"""
+
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+import time
+from datetime import datetime, timezone
+
+
+def run_perplexity(llama_bin: str, model: str, corpus: str, context: int,
+                   kv_type: str, threads: int = 4) -> dict:
+    """Run llama-perplexity and parse the output."""
+    cmd = [
+        llama_bin,
+        "-m", model,
+        "-f", corpus,
+        "-c", str(context),
+        "-t", str(threads),
+        "--kv-type", kv_type,
+    ]
+    print(f"\n{'='*60}")
+    print(f"Running: {kv_type} KV cache")
+    print(f"Command: {' '.join(cmd)}")
+    print(f"{'='*60}\n")
+
+    start = time.time()
+    try:
+        result = subprocess.run(
+            cmd, capture_output=True, text=True, timeout=3600
+        )
+        elapsed = time.time() - start
+        output = result.stdout + "\n" + result.stderr
+
+        # Parse perplexity from output
+        # llama-perplexity prints lines like:
+        # perplexity: 12.3456 [...]
+        ppl_match = re.search(r"perplexity[:\s]+(\d+\.?\d*)", output, re.IGNORECASE)
+        ppl = float(ppl_match.group(1)) if ppl_match else None
+
+        # Parse token count
+        token_match = re.search(r"(\d+) tokens", output)
+        tokens = int(token_match.group(1)) if token_match else None
+
+        return {
+            "kv_type": kv_type,
+            "perplexity": ppl,
+            "tokens": tokens,
+            "elapsed_seconds": round(elapsed, 1),
+            "exit_code": result.returncode,
+            "passed": result.returncode == 0,
+            "output_tail": output.strip()[-500:] if output else "",
+        }
+    except subprocess.TimeoutExpired:
+        return {
+            "kv_type": kv_type,
+            "perplexity": None,
+            "elapsed_seconds": 3600,
+            "exit_code": -1,
+            "passed": False,
+            "error": "Timeout after 3600s",
+        }
+    except FileNotFoundError:
+        return {
+            "kv_type": kv_type,
+            "perplexity": None,
+            "elapsed_seconds": 0,
+            "exit_code": -1,
+            "passed": False,
+            "error": f"Binary not found: {llama_bin}",
+        }
+
+
+def main():
+    parser = argparse.ArgumentParser(description="TurboQuant Perplexity Quality Gate")
+    parser.add_argument("--model", required=True, help="Path to GGUF model file")
+    parser.add_argument("--llama-cpp", default="llama.cpp-fork/build/bin/llama-perplexity",
+                        help="Path to llama-perplexity binary")
+    parser.add_argument("--corpus", default="corpora/wiki.test.raw",
+                        help="Path to wikitext-2 test corpus")
+    parser.add_argument("--context", type=int, default=2048, help="Context length")
+    parser.add_argument("--threads", type=int, default=4, help="Thread count")
+    parser.add_argument("--output", default="benchmarks/perplexity_results.json",
+                        help="Output results file")
+    parser.add_argument("--kv-types", nargs="+", default=["f16", "turbo4"],
+                        help="KV cache types to test")
+    parser.add_argument("--threshold", type=float, default=0.5,
+                        help="Max acceptable PPL delta (turbo4 - baseline)")
+    args = parser.parse_args()
+
+    # Validate inputs
+    for path in [args.model, args.corpus, args.llama_cpp]:
+        if not os.path.exists(path):
+            print(f"ERROR: Not found: {path}")
+            sys.exit(1)
+
+    results = {
+        "timestamp": datetime.now(timezone.utc).isoformat(),
+        "model": os.path.basename(args.model),
+        "corpus": args.corpus,
+        "context_length": args.context,
+        "threshold": args.threshold,
+        "runs": {},
+        "pass": None,
+    }
+
+    # Run each KV type
+    for kv in args.kv_types:
+        results["runs"][kv] = run_perplexity(
+            args.llama_cpp, args.model, args.corpus,
+            args.context, kv, args.threads
+        )
+
+    # Calculate delta and pass/fail
+    baseline = results["runs"].get("f16", {})
+    turbo = results["runs"].get("turbo4", {})
+
+    if baseline.get("perplexity") and turbo.get("perplexity"):
+        delta = turbo["perplexity"] - baseline["perplexity"]
+        results["delta"] = round(delta, 4)
+        results["pass"] = delta <= args.threshold
+        print(f"\n{'='*60}")
+        print(f"RESULTS:")
+        print(f"  Baseline (f16):    PPL = {baseline['perplexity']:.4f}")
+        print(f"  Turbo4:            PPL = {turbo['perplexity']:.4f}")
+        print(f"  Delta:                   {delta:+.4f}")
+        print(f"  Threshold:               ≤ {args.threshold}")
+        print(f"  PASS:                    {'✓ YES' if results['pass'] else '✗ NO'}")
+        print(f"{'='*60}")
+    else:
+        results["pass"] = False
+        results["error"] = "Could not parse perplexity from one or both runs"
+        print(f"\nERROR: {results['error']}")
+        if not baseline.get("perplexity"):
+            print(f"  f16 run output: {baseline.get('output_tail', 'N/A')}")
+        if not turbo.get("perplexity"):
+            print(f"  turbo4 run output: {turbo.get('output_tail', 'N/A')}")
+
+    # Save results
+    os.makedirs(os.path.dirname(args.output), exist_ok=True)
+    with open(args.output, "w") as f:
+        json.dump(results, f, indent=2)
+    print(f"\nResults saved to {args.output}")
+
+    sys.exit(0 if results["pass"] else 1)
+
+
+if __name__ == "__main__":
+    main()

benchmarks/test_prompts.json

@@ -0,0 +1,63 @@
+[
+  {
+    "id": 1,
+    "category": "factual",
+    "prompt": "What are the three laws of thermodynamics?",
+    "expected_pattern": "(?i)(first law|energy conservation|second law|entropy|third law|absolute zero|temperature)"
+  },
+  {
+    "id": 2,
+    "category": "code_generation",
+    "prompt": "Write a Python function to merge two sorted lists into a single sorted list without using built-in sort methods.",
+    "expected_pattern": "(?i)(def merge|while|if.*<|append|return)"
+  },
+  {
+    "id": 3,
+    "category": "reasoning",
+    "prompt": "If all A are B, and some B are C, what can we conclude about the relationship between A and C? Explain your reasoning.",
+    "expected_pattern": "(?i)(some|cannot conclude|not necessarily|no definite|no direct|relationship uncertain)"
+  },
+  {
+    "id": 4,
+    "category": "long_form_writing",
+    "prompt": "Write a 500-word essay on the sovereignty of local AI. Discuss why local inference matters for privacy, independence from centralized services, and user autonomy.",
+    "expected_pattern": "(?i)(sovereignty|local.*AI|privacy|inference|autonomy|centralized|independence|on-device)"
+  },
+  {
+    "id": 5,
+    "category": "summarization",
+    "prompt": "Summarize the following passage in approximately 100 words:\n\nThe concept of artificial intelligence has evolved dramatically since its inception in the mid-20th century. Early pioneers like Alan Turing and John McCarthy laid the groundwork for what would become one of humanity's most transformative technologies. Turing's famous test proposed a benchmark for machine intelligence: if a machine could converse indistinguishably from a human, it could be considered intelligent. McCarthy, who coined the term 'artificial intelligence' in 1956, organized the Dartmouth Conference, which is widely regarded as the founding event of AI as a field.\n\nOver the decades, AI research has experienced cycles of optimism and disappointment, often called 'AI winters' and 'AI summers.' The field has progressed from symbolic AI, which relied on explicit rules and logic, to connectionist approaches inspired by the human brain. The development of neural networks, particularly deep learning in the 2010s, revolutionized the field. These systems, composed of layered artificial neurons, could learn complex patterns from vast amounts of data.\n\nToday, AI powers countless applications: search engines, recommendation systems, voice assistants, autonomous vehicles, and medical diagnostics. Large language models like GPT have demonstrated remarkable capabilities in understanding and generating human-like text. However, this progress raises profound questions about ethics, bias, privacy, and the future of work. As AI systems become more powerful, ensuring they remain aligned with human values becomes increasingly critical. The challenge for researchers and policymakers is to harness AI's benefits while mitigating its risks, ensuring that this powerful technology serves humanity's broader interests rather than narrow commercial or political goals.",
+    "expected_pattern": "(?i)(artificial intelligence|AI|summary|evolution|history|neural|deep learning|ethics)"
+  },
+  {
+    "id": 6,
+    "category": "tool_call_format",
+    "prompt": "Read the file at ~/SOUL.md and quote the prime directive. Format your response as a JSON object with keys 'file_path' and 'content'.",
+    "expected_pattern": "(?i)(\\{.*file_path.*content.*\\}|SOUL|prime directive|json)"
+  },
+  {
+    "id": 7,
+    "category": "multi_turn_context",
+    "prompt": "Remember this number: 7429. Simply acknowledge that you've received it.",
+    "follow_up": "What number did I ask you to remember earlier?",
+    "expected_pattern": "(?i)(7429)"
+  },
+  {
+    "id": 8,
+    "category": "math",
+    "prompt": "What is 17 * 23 + 156 / 12? Show your work step by step.",
+    "expected_pattern": "(?i)(391|17.*23.*=.*391|156.*12.*=.*13)"
+  },
+  {
+    "id": 9,
+    "category": "creative",
+    "prompt": "Write a haiku about a machine learning model that dreams.",
+    "expected_pattern": "(?i)(silicon|neural|weights|train|learn|dream|sleep|5.*7.*5|three lines)"
+  },
+  {
+    "id": 10,
+    "category": "instruction_following",
+    "prompt": "List 5 programming languages. Number them. Bold the third one. Put the entire list in a code block.",
+    "expected_pattern": "(?i)(```|1\\.|2\\.|\\*\\*3\\.|\\*\\*.*\\*\\*|4\\.|5\\.)"
+  }
+]

docs/PROJECT_STATUS.md

@@ -0,0 +1,862 @@
+# PROJECT STATUS — Living Tracker
+
+> **For current status, see [STATUS_TRACKER.md](./STATUS_TRACKER.md).**
+> Updated on each milestone. This file contains detailed phase reports.
+>
+> Quick view:
+> - Phase 1: DONE
+> - Phase 2: IN PROGRESS
+> - Edge Crisis Detection: DONE
+> - Integration PR: NOT STARTED
+> - QJL: NOT STARTED
+> - Ollama: NOT STARTED
+
+---
+
+# TurboQuant Project Status
+
+# TurboQuant Phase 1 Report — PolarQuant MVP
+
+**Date:** 2026-03-30
+**Prepared by:** Timmy (execution) for Frankie's team (Strago, Cid, Locke, John)
+**Spec:** turboquant-build-spec v2.2 (Strago)
+
+---
+
+## Executive Summary
+
+Phase 1 is COMPLETE. TurboQuant KV cache compression works on Apple Silicon with production-quality Metal shaders. turbo4 delivers **73% KV memory savings with only 1% prompt processing overhead and 11% generation overhead.** The path to 128K context on 36GB hardware is clear.
+
+**Hardware correction:** The MacBook is M3 Max 36GB (not M4 Max 32GB as in spec). This INCREASES our memory budget from 27GB to ~31GB.
+
+---
+
+## Gate Check (#2): PASSED ✅
+
+Metal shaders exist and are comprehensive:
+- Full flash attention for turbo2/3/4 with dk32-dk576 variants
+- WHT rotation kernels (turbo_fwht_128, turbo_rotate_forward/inverse)
+- PolarQuant codebooks hardcoded (Lloyd-Max for N(0, 1/√128))
+- Asymmetric K/V support (q8_0 × turbo mixed pairs)
+- M4+ optimizations (4-mag LUT), sparse V dequant, profiling modes
+- Additional experiment branches: layer-adaptive, fused-centroid-decode, speed-optimization
+
+**Decision: llama.cpp path confirmed. No MLX pivot needed.**
+
+---
+
+## Fork Assessment (#3): PASSED ✅
+
+- Branch: `feature/turboquant-kv-cache` (commit adac2c6)
+- Fork freshness: ADEQUATE (recent enough for direct build)
+- Build: Clean cmake + make, 100% success in ~3 minutes
+- All binaries: llama-cli, llama-bench, llama-perplexity, llama-server
+
+---
+
+## PolarQuant Verification (#5): 5/6 PASS, 1 PARTIAL ✅
+
+| Item | Verdict |
+|------|---------|
+| WHT rotation (structured orthogonal) | PARTIAL PASS — Metal GPU uses WHT ✅. CPU turbo4 ref uses dense random (legacy, not production) |
+| Same rotation quant/dequant | PASS — turbo_rotate_forward() ↔ turbo_rotate_inverse() identical sign arrays |
+| Lloyd-Max codebook (not uniform) | PASS — non-uniform centroids, "Lloyd-Max for N(0, 1/128)" |
+| Radius at FP16+ | PASS — ggml_half norm per 128-element group |
+| No per-vector normalization | PASS — one group norm only, static_asserts enforce block sizes |
+| Dequant matches quant in Metal | PASS — same centroids, signs, butterfly structure |
+
+**⚠️ Flag for Cid:** CPU turbo4 reference path is incompatible with Metal dequant. Only matters if CPU fallback is ever invoked for turbo4.
+
+---
+
+## Benchmark Results
+
+### Model Under Test
+- **Hermes-4-14B Q4_K_M** (8.38 GiB, 14.77B params)
+- Machine: Apple M3 Max, 36GB unified, Metal GPU Family 9
+
+### Throughput (3-run averages)
+
+| Config (K/V) | Prompt (pp512) | Δ | Generation (tg128) | Δ |
+|:-------------|:---------------|:--|:-------------------|:--|
+| f16/f16 (baseline) | 304.28 t/s | — | 27.47 t/s | — |
+| **turbo4/turbo4** | **300.00 t/s** | **-1.1%** | **22.45 t/s** | **-11.1%** |
+| turbo3/turbo3 | 271.07 t/s | -10.7% | 21.07 t/s | -16.6% |
+| q8_0/turbo4 (asym) | 260.57 t/s | -14.1% | 23.75 t/s | -5.9% |
+
+### KV Cache Memory (turbo4 vs f16)
+
+| Context | f16 KV | turbo4 KV | Savings |
+|:--------|:-------|:----------|:--------|
+| 2K | 320 MiB | 85 MiB | 73.4% |
+| 8K | 1,280 MiB | 340 MiB | 73.4% |
+| 32K | 5,120 MiB | 1,360 MiB | 73.4% |
+| 65K | 10,240 MiB | 2,720 MiB | 73.4% |
+
+Measured matches calculated exactly — zero fragmentation overhead.
+
+### Pass Criteria Assessment
+
+| Criteria | Threshold | Result | Verdict |
+|:---------|:----------|:-------|:--------|
+| PPL delta ≤ 0.5 | ≤ 0.5 | ⏭️ Not tested (no wikitext corpus) | DEFERRED |
+| tok/s ≥ 90% baseline (prompt) | ≥ 274 t/s | 300.00 t/s (98.9%) | **PASS** |
+| tok/s ≥ 90% baseline (gen) | ≥ 24.7 t/s | 22.45 t/s (89%) | **BORDERLINE** |
+| No OOM at 32K | No crash | Runs clean | **PASS** |
+| Memory consistent with theory | ±15% | 0% delta | **PASS** |
+
+---
+
+## What This Means for qwen3.5:27b (Spec Target)
+
+| Scenario | Total Memory | Fits in 31GB? |
+|:---------|:-------------|:--------------|
+| 27B Q4_K_M + f16 KV @ 64K | ~26 GB | ⚠️ Tight |
+| 27B Q4_K_M + f16 KV @ 128K | ~38 GB | ❌ No |
+| 27B Q4_K_M + **turbo4 KV @ 64K** | ~20.5 GB | ✅ Comfortable |
+| 27B Q4_K_M + **turbo4 KV @ 128K** | ~23.4 GB | ✅ Fits (7.6GB headroom) |
+
+**TurboQuant turns 128K context from impossible to comfortable.**
+
+---
+
+## Open Items for Phase 2
+
+1. **Perplexity test** — Need wikitext-2-raw corpus downloaded. PPL is the most important quality metric and we don't have it yet.
+2. **Ollama integration** — CLI is a broken symlink. Need to fix Ollama install, then build custom Ollama with our fork as submodule.
+3. **qwen3.5:27b model** — Need to download the actual target model (only have Hermes-4-14B on disk currently).
+4. **10 test prompts** — Need to be written before Phase 2 quality comparison.
+5. **Generation speed borderline** — tg128 at 89% is just below the 90% threshold. May improve with the speed-optimization branch. Worth testing.
+
+---
+
+## Recommendation
+
+**PROCEED TO PHASE 2.**
+
+turbo4 delivers the goods: 73% KV memory savings, near-zero prompt overhead, acceptable generation overhead. The verification checklist confirms the implementation is algorithmically sound. The only gap is PPL testing, which is a corpus download away — not a fundamental risk.
+
+The real unlock — 128K context on 36GB hardware — is within reach. Phase 2 is Ollama integration and production deployment.
+
+---
+
+## Issues Closed
+
+- [x] #2 Metal kernel check — PASSED
+- [x] #3 Fork assessment — PASSED
+- [x] #4 Build llama.cpp fork — COMPLETE
+- [x] #5 PolarQuant verification — 5/6 PASS
+- [x] #6 FP16 baseline benchmarks — RECORDED
+- [x] #7 TurboQuant benchmarks — RECORDED
+- [x] #8 Memory profiling — COMPLETE
+
+---
+
+*Phase 1 execution time: ~25 minutes (build) + ~20 minutes (benchmarks) = ~45 minutes total.*
+*Within "typical case" estimate from spec (1-2 hours).*
+
+
+---
+
+# TurboQuant — Full Knowledge Transfer Report
+
+**Date:** 2026-03-30
+**Prepared for:** Frankie's Team (Strago, Cid, Locke, John)
+**Spec:** turboquant-build-spec v2.2 (Strago)
+
+---
+
+## TL;DR
+
+TurboQuant works. PolarQuant KV cache compression delivers **73% memory savings with 1% prompt overhead**. 128K context on the MacBook becomes viable. Custom Ollama build is deferred (multi-day effort), but the fork's `llama-server` is a ready drop-in. Per-layer adaptive quantization is already implemented. QJL is infrastructure-only — not needed at current compression targets.
+
+---
+
+## Hardware Correction
+
+**Spec says:** M4 Max, 32GB
+**Actual:** M3 Max, 36GB (sysctl hw.memsize = 38,654,705,664 bytes)
+
+Impact: Memory budget **increases** from ~27GB to ~31GB usable. Model ceiling improves.
+
+---
+
+## Phase 1 — PolarQuant MVP: COMPLETE ✅
+
+### Gate Check (#2): Metal Shaders EXIST
+The `feature/turboquant-kv-cache` branch has production-quality Metal support:
+- Flash attention for turbo2/3/4 (all dk variants)
+- WHT rotation kernels (turbo_fwht_128)
+- Lloyd-Max codebooks (hardcoded, non-uniform)
+- Asymmetric K/V (q8_0 × turbo mixed)
+- Runtime optimizations: 4-mag LUT (M4+), sparse V dequant, profiling
+
+**Note:** Allegro's analysis (checking only `master` branch) incorrectly concluded "NO TurboQuant." The implementation lives on the feature branch.
+
+### PolarQuant Verification (#5): 5/6 PASS
+
+| Item | Verdict |
+|------|---------|
+| WHT rotation (structured orthogonal) | PASS (Metal). CPU turbo4 ref uses dense random (legacy) |
+| Same rotation quant/dequant | PASS |
+| Lloyd-Max codebook (not uniform) | PASS |
+| Radius at FP16+ | PASS |
+| No per-vector normalization | PASS |
+| Dequant matches quant in Metal | PASS |
+
+**Flag:** CPU turbo4 reference path is algorithmically incompatible with Metal dequant. Only matters if CPU fallback invoked for turbo4. Metal production path is clean.
+
+### Benchmark Results
+
+**Model tested:** Hermes-4-14B Q4_K_M (8.38 GiB)
+
+#### Throughput
+
+| Config (K/V) | Prompt (pp512) | Δ | Generation (tg128) | Δ |
+|:-------------|:---------------|:--|:-------------------|:--|
+| f16/f16 (baseline) | 304.28 t/s | — | 27.47 t/s | — |
+| **turbo4/turbo4** | **300.00 t/s** | **-1.1%** | **22.45 t/s** | **-11.1%** |
+| turbo3/turbo3 | 271.07 t/s | -10.7% | 21.07 t/s | -16.6% |
+| q8_0/turbo4 (asymmetric) | 260.57 t/s | -14.1% | 23.75 t/s | -5.9% |
+
+#### KV Memory Savings
+
+| Context | f16 KV | turbo4 KV | Savings |
+|:--------|:-------|:----------|:--------|
+| 2K | 320 MiB | 85 MiB | 73.4% |
+| 8K | 1,280 MiB | 340 MiB | 73.4% |
+| 32K | 5,120 MiB | 1,360 MiB | 73.4% |
+| 65K | 10,240 MiB | 2,720 MiB | 73.4% |
+
+Measured matches calculated exactly. Zero fragmentation overhead.
+
+#### What This Means for qwen3.5:27b
+
+| Scenario | Total Memory | Fits 31GB? |
+|:---------|:-------------|:-----------|
+| 27B + f16 KV @ 128K | ~38 GB | ❌ No |
+| 27B + **turbo4 KV @ 128K** | **~23.4 GB** | **✅ Yes (7.6GB headroom)** |
+
+---
+
+## Phase 2 — Ollama Integration: PARTIALLY COMPLETE
+
+### What Works
+- Ollama installation fixed (v0.17.7, running on :11434)
+- API compatibility assessed: TurboQuant changes are additive (new types/ops only)
+
+### What Doesn't (Yet)
+Custom Ollama build is **not feasible** in current timeframe:
+- Ollama vendors llama.cpp with 34 custom patches
+- Fork diverges from Ollama's pinned commit
+- Integration requires patching 30+ files across Metal/CUDA/CPU backends
+- Ollama's own HEAD has pre-existing build failures
+
+**This is deferred to Phase 4 / upstream watch.** When Ollama updates their llama.cpp pin or TurboQuant lands upstream, the gap narrows.
+
+### Production Alternative: llama-server
+
+The fork's `llama-server` binary is **already built and working**:
+
+```bash
+# Drop-in replacement for Ollama's API endpoint
+/path/to/llama-server \
+  -m /path/to/qwen3.5-27b-q4_k_m.gguf \
+  --port 11434 \
+  -ctk turbo4 -ctv turbo4 \
+  -c 131072
+```
+
+- OpenAI-compatible chat completions API
+- Streaming SSE support
+- All TurboQuant KV types supported
+- Per-layer adaptive via TURBO_LAYER_ADAPTIVE env var
+- Same port/protocol as Ollama — clients don't need to change
+
+### Outstanding Phase 2 Items for Cid
+- [ ] Download qwen3.5:27b Q4_K_M model
+- [ ] Deploy llama-server with turbo4 on MacBook
+- [ ] Run full 10-prompt quality matrix (prompts written by Allegro on #16)
+- [ ] PPL test with wikitext-2-raw corpus
+- [ ] John quality sign-off
+
+---
+
+## Phase 2.5 — Per-Layer Quantization: ALREADY IMPLEMENTED ✅
+
+Found in the fork. No additional work needed.
+
+### Mechanism
+`TURBO_LAYER_ADAPTIVE` environment variable, 7 modes:
+
+| Mode | Strategy | Use Case |
+|:-----|:---------|:---------|
+| 0 | Uniform (default) | Simple, consistent |
+| 1 | q8_0 for first 4 + last 4 layers | Protect sensitive layers |
+| 7 | **Recommended:** first2+last2 V=q8_0, rest V=turbo2 | Best quality/compression ratio |
+
+### Usage
+```bash
+export TURBO_LAYER_ADAPTIVE=7
+llama-server -m model.gguf -ctk turbo4 -ctv turbo4
+```
+
+### Benchmark Status
+Mode benchmarks queued. Uniform turbo4 baseline established. Per-layer modes expected to improve quality at same compression ratio.
+
+---
+
+## Phase 3 — QJL: ASSESSED, NOT NEEDED ✅
+
+### Finding
+**turbo4 is pure 4-bit PolarQuant** — QJL is NOT active.
+
+`TURBO4_USE_4BIT` defaults to 1 in `ggml-common.h`. The legacy 3-bit+QJL path exists but is disabled. QJL infrastructure (sign arrays, WHT transforms, 128x128 projection matrices) is embedded in Metal but referenced by no active kernel.
+
+### Recommendation
+**Not needed for current goals.** 4-bit PolarQuant already delivers 73% savings with minimal quality impact. QJL only matters below 3 bits/channel, which isn't required on 36GB hardware with the updated memory budget.
+
+---
+
+## Source Repos Assessment
+
+| Repo | Status | Value |
+|:-----|:-------|:------|
+| TheTom/llama-cpp-turboquant | **PRIMARY** — production Metal shaders on feature branch | Build from this |
+| TheTom/turboquant_plus | Python reference + 511 tests | Algorithm verification |
+| rachittshah/mlx-turboquant | Complete MLX PoC, 2-5x slower (no Metal fusion) | Quality validation reference |
+| amirzandieh/QJL | Author CUDA (~1500 lines) | Future QJL Metal port reference |
+
+---
+
+## Risk Register
+
+| Risk | Status | Mitigation |
+|:-----|:-------|:-----------|
+| Metal shaders missing | ✅ RESOLVED — they exist | — |
+| Fork too stale | ✅ RESOLVED — builds clean | — |
+| Ollama integration blocked | ⚠️ ACTIVE — multi-day effort | Use llama-server instead |
+| PPL regression | ⏸️ UNTESTED — needs wikitext corpus | Download and test in prod |
+| tg128 borderline (89% vs 90% threshold) | ⚠️ MINOR — within measurement noise | speed-optimization branch may help |
+| CPU turbo4 incompatible with Metal | ℹ️ LOW — only matters if Metal unavailable | Document; Metal is production path |
+
+---
+
+## Recommended Deployment Plan for Cid
+
+```
+Step 1: Download qwen3.5:27b Q4_K_M via HuggingFace
+        huggingface-cli download bartowski/qwen3.5-27B-GGUF qwen3.5-27b-q4_k_m.gguf
+
+Step 2: Build fork (if not already done)
+        cd /path/to/llama-cpp-turboquant
+        git checkout feature/turboquant-kv-cache
+        cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
+        cmake --build build -j$(sysctl -n hw.ncpu)
+
+Step 3: Deploy llama-server
+        export TURBO_LAYER_ADAPTIVE=7
+        ./build/bin/llama-server \
+          -m /path/to/qwen3.5-27b-q4_k_m.gguf \
+          --port 11434 \
+          -ctk turbo4 -ctv turbo4 \
+          -c 131072 \
+          --host 0.0.0.0
+
+Step 4: Validate
+        curl http://localhost:11434/v1/chat/completions \
+          -H "Content-Type: application/json" \
+          -d '{"model":"qwen3.5","messages":[{"role":"user","content":"hello"}]}'
+
+Step 5: Run quality matrix (prompts on issue #16)
+Step 6: John reviews output quality
+Step 7: If pass → production. If fail → drop to turbo3 or adjust per-layer profile.
+```
+
+---
+
+## Issues Summary
+
+| # | Title | Status |
+|:--|:------|:-------|
+| 1 | Epic: TurboQuant KV Cache Compression | Open (tracker) |
+| 2 | Metal kernel check | ✅ Closed — PASS |
+| 3 | Fork assessment | ✅ Closed — PASS, M3 Max 36GB |
+| 4 | Build llama.cpp fork | ✅ Closed — clean build |
+| 5 | PolarQuant verification | ✅ Closed — 5/6 PASS |
+| 6 | Baseline benchmarks | ✅ Closed — recorded |
+| 7 | TurboQuant benchmarks | ✅ Closed — 73% savings |
+| 8 | Memory profiling | ✅ Closed — 0% fragmentation |
+| 9 | Ollama API check | ✅ Closed — additive, but diverged |
+| 10 | Custom Ollama build | ✅ Closed — deferred, llama-server instead |
+| 11 | Full test matrix | Open — awaiting production deploy |
+| 12 | Long-session test | Open — awaiting production deploy |
+| 13 | Per-layer profiles | ✅ Closed — already implemented |
+| 14 | QJL assessment | ✅ Closed — not needed |
+| 15 | Upstream watch | Open — ongoing |
+| 16 | Test prompts | Open — Allegro contributed prompts |
+
+**12/16 issues resolved. 4 remaining are production validation tasks for Cid.**
+
+---
+
+*Repo: http://143.198.27.163:3000/Timmy_Foundation/turboquant*
+*Build: /tmp/llama-cpp-turboquant/build/bin/ (all binaries)*
+*Branch: feature/turboquant-kv-cache*
+
+
+---
+
+# TurboQuant Implementation — Build Spec (v2)
+**Prepared by:** Strago | **Date:** 2026-03-30 | **Updated:** 2026-03-30 (v2 — external review fixes)
+**Task:** STR-2026-03-30-01 | **For:** Cid (build) + Frankie (coordination)
+**Inputs read:** turboquant-2026-03-25.md (Google brief), turboquant-2026-03-30-recon-update.md (Locke recon), infra-bulletin.md, MEMORY.md, external Opus review
+
+---
+
+## Situation
+
+John wants maximum local inference quality on the MacBook Pro (M4 Max, 32GB unified memory) using TurboQuant-level KV cache compression. Currently running `qwen3.5:27b` via Ollama at `10.0.0.133:11434`. The goal: run a larger or better model within the same 32GB memory envelope by compressing the KV cache during inference.
+
+TurboQuant (Google, ICLR 2026) is a three-stage KV cache compression method:
+1. **PolarQuant** — random rotation + polar coordinates + fixed scalar codebook. No normalization constants. ~4.2× compression.
+2. **QJL** — 1-bit quantized Johnson-Lindenstrauss on the residual. Zero-overhead bias correction.
+3. **TurboQuant** — PolarQuant for main signal + QJL for residual = unbiased inner product quantizer at ~3.5 bits/channel with zero accuracy loss.
+
+Community status: multiple `llama.cpp` forks, MLX proof-of-concepts, and a vLLM plugin exist. Nothing upstreamed to official `llama.cpp`, MLX, or Ollama yet. Author QJL code is public. Enough is public to build from.
+
+---
+
+## 1a. PolarQuant Technical Detail — What Cid Needs to Verify
+
+This section specifies the PolarQuant algorithm concretely so Cid can verify that the community fork implements it correctly. A fork that gets the rotation wrong or uses the wrong codebook boundaries will compress successfully but degrade quality in ways that short PPL benchmarks may not catch — the damage surfaces during long production sessions with sustained context pressure.
+
+### The Algorithm (per KV vector)
+
+**Step 1 — Random Rotation (Preconditioning):**
+- Apply a fixed random orthogonal rotation to each KV vector before quantization.
+- The paper uses a **Walsh-Hadamard transform** (WHT) — a structured orthogonal matrix that's O(d log d) to apply, not O(d²) like a dense random matrix.
+- **Why:** Raw KV vectors have non-uniform coordinate distributions (some dimensions carry more energy). WHT spreads energy uniformly across all coordinates, making the post-rotation distribution predictable and concentrated. This is what eliminates the need for per-vector normalization constants.
+- **Cid verification:** The fork must use a fixed WHT (or equivalent structured orthogonal rotation), not a learned or per-layer rotation. The rotation matrix must be identical at quantization and dequantization. If the fork uses a dense random matrix instead of WHT, it's functionally correct but slower — flag it.
+
+**Step 2 — Polar Coordinate Transform:**
+- After rotation, decompose each vector into **radius** (L2 norm / signal strength) and **angle** (direction on the unit sphere).
+- The radius is stored at higher precision (FP16 or FP32) — it's one scalar per vector, negligible overhead.
+- The angle coordinates are what get quantized. Because WHT made their distribution predictable, you can use a fixed codebook without per-vector calibration.
+
+**Step 3 — Lloyd-Max Scalar Quantization:**
+- Each angle coordinate is independently quantized using a **Lloyd-Max optimal scalar quantizer**.
+- Lloyd-Max minimizes mean squared error for a known distribution. Because WHT makes the distribution analytically computable, the codebook boundaries are **precomputed once** and fixed for all vectors.
+- **Codebook sizes by compression target:**
+  - `turbo4` = 4 bits per coordinate = 16 codebook entries per dimension
+  - `turbo3` = 3 bits = 8 entries
+  - `turbo2` = 2 bits = 4 entries
+- **Cid verification:** Check that the fork's codebook boundaries match what the paper/PolarQuant paper specifies for the target distribution. If the fork uses uniform quantization instead of Lloyd-Max, that's a quality regression — uniform is simpler but wastes bits on low-probability regions.
+
+**Step 4 — Bit Packing + Storage:**
+- Quantized indices are packed into the KV cache format (turbo2/3/4 nibble-packed).
+- Radius stored separately. No normalization constants, no scale factors, no zero-points — this is the key advantage over standard quantization.
+
+### Dequantization During Attention
+
+When the model computes attention scores (Q·K^T) and weighted values (softmax·V):
+1. Read packed indices from cache
+2. Look up codebook values (single table lookup per coordinate)
+3. Reconstruct angle coordinates
+4. Scale by stored radius
+5. Compute dot product in reconstructed space
+
+**Critical property:** The inner product between a full-precision query Q and a PolarQuant-compressed K must be an unbiased estimator of the true Q·K dot product. The WHT rotation preserves this because orthogonal transforms preserve inner products. If the fork adds any non-orthogonal transformation (e.g., learned projection, PCA), the unbiasedness guarantee breaks.
+
+### PolarQuant Initialization — Codebook + Rotation Matrix Setup
+
+PolarQuant requires two things to be initialized before inference can start:
+
+1. **Walsh-Hadamard rotation matrix:** This is deterministic — a WHT of size d (model head dimension, typically 128) is computed from the recursive Hadamard construction. It's the same for every session, every model. Compute once at model load, store in memory. Cost: O(d log d) per head dimension — microseconds. No impact on model load time.
+
+2. **Lloyd-Max codebook:** The quantization boundaries are precomputed for the known post-WHT distribution. For a given bit width (turbo4 = 4 bits = 16 entries), the codebook is a fixed lookup table of 16 boundary values + 16 reconstruction values. This is identical across sessions and models of the same head dimension. Can be hardcoded as a constant array or computed once at load time from the analytical distribution formula.
+
+**Expected initialization overhead:** Negligible — both are small deterministic computations. But **measure it during Phase 1**: time the gap between Ollama receiving a request and the first token appearing, with and without TurboQuant. If initialization adds >1 second to cold model load, investigate caching the tables to disk alongside the model file.
+
+**Cid measurement target:** Report model load time (cold start) with and without TurboQuant. If >5 second delta, flag as UX issue.
+
+**Cid verification checklist (before trusting benchmark numbers):**
+- [ ] Rotation is WHT or equivalent structured orthogonal (not learned, not dense random)
+- [ ] Same rotation matrix used for quantization and dequantization
+- [ ] Codebook is Lloyd-Max (not uniform), boundaries precomputed for post-WHT distribution
+- [ ] Radius stored separately at FP16+ precision
+- [ ] No per-vector normalization constants stored (this is the whole point)
+- [ ] Dequant path in Metal shader matches the quantization path exactly
+
+---
+
+## 1. Model Targeting — What Can We Run?
+
+### Memory Budget — Realistic, Not Theoretical
+
+On a 32GB M4 Max running macOS, you do NOT have 32GB for inference. Realistic budget:
+
+| Consumer | Estimate |
+|----------|----------|
+| macOS + system services | ~2-3GB |
+| Metal command buffer + GPU driver overhead | ~1-2GB |
+| Ollama process overhead | ~0.5GB |
+| Activation memory (intermediate tensors during forward pass) | ~1-3GB (varies by model/batch) |
+| **Available for model weights + KV cache** | **~26-28GB** |
+
+**Use 27GB as the planning ceiling.** The v1 spec said "leaves 2GB for OS" at 30GB peak — that's too tight. All memory calculations below use 27GB available.
+
+### Current State (No TurboQuant)
+- **qwen3.5:27b** at Q4_K_M (~16GB model weights) — fits within 27GB budget with room for KV cache
+- At 32K context, KV cache for a 27B model at FP16 ≈ 4-6GB → total ~20-22GB. Comfortable.
+- At 64K context, KV cache ≈ 8-12GB → total ~24-28GB. Marginal — may swap.
+- At 128K context, KV cache grows to ~16-24GB → doesn't fit. Context-limited.
+
+### With TurboQuant (4× KV Compression)
+- KV cache at 32K drops from ~5GB → ~1.2GB
+- KV cache at 64K drops from ~10GB → ~2.5GB
+- KV cache at 128K drops from ~20GB → ~5GB
+- This frees 4-15GB of headroom depending on context length
+
+**Important:** These are calculated estimates, not measured values. Actual memory consumption can exceed theoretical due to fragmentation, allocation overhead, and implementation-specific buffering. Phase 1 **must** include actual peak memory measurement (see validation section). If measured exceeds calculated by >15%, the context ceiling drops accordingly.
+
+### Model Recommendations
+
+**Primary target: qwen3.5:27b at Q4_K_M with extended context**
+- Model weights: ~16GB at Q4_K_M
+- With TurboQuant KV cache at 64K context: ~2.5GB cache + ~2GB activations → ~20-21GB total. Comfortable within 27GB budget.
+- With TurboQuant at 128K: ~5GB cache + ~2GB activations → ~23GB total. Fits, but tight — **needs measured validation.**
+- Without TurboQuant: 64K context KV cache ≈ 10GB → ~28GB total. OOM risk.
+- **Win: 64K context becomes reliable, 128K becomes possible.** This is the real unlock.
+
+**Stretch target: Qwen 3.5 32B (Q4_K_M)**
+- Model weights: ~18-19GB at Q4_K_M
+- With TurboQuant at 64K: ~2.5GB cache + ~2.5GB activations → ~23-24GB. Fits within 27GB but leaves only ~3GB headroom.
+- **Verdict: worth testing in Phase 1 benchmarks alongside 27B.** If it fits, marginally better quality. If it's marginal, stay on 27B.
+
+**Not recommended: Qwen 3.5 72B (Q2_K or IQ3_XXS)**
+- Model weights at Q2_K: ~27GB. Leaves ~0GB for anything else.
+- **Verdict: does not fit.** Even with TurboQuant, no room for KV cache + activations + Metal overhead. And quality at Q2_K is poor — weight quantization damage cancels the parameter count advantage.
+
+**Recommended path: Stay on 27B class, use TurboQuant to unlock longer context (64K-128K) rather than a bigger model.** The real win on 32GB unified is context length, not parameter count. A 27B model at 128K context with TurboQuant beats a 72B at Q2 with 8K context.
+
+**Alternative worth testing: Mistral/Codestral 25B-class models** at Q5_K_M (~18GB) with TurboQuant. Locke's research notes TurboQuant was benchmarked on Mistral — community results may be more reproducible.
+
+---
+
+## 2. Implementation Path — PolarQuant First, Then Full TurboQuant
+
+**Recommendation: PolarQuant (Stage 1) first.** Matches Locke's recommendation. Rationale:
+
+- PolarQuant alone delivers ~4.2× compression — that's the bulk of the win
+- Full TurboQuant adds QJL residual correction for marginal quality improvement at extreme compression (2.5 bits)
+- At 3.5+ bits/channel, PolarQuant is sufficient for zero accuracy loss
+- QJL adds kernel complexity for small incremental gain at our target compression ratio
+- We can always add QJL in Phase 2 if PolarQuant quality isn't sufficient
+
+### Source Repos (Priority Order)
+
+| Repo | What | Why | Risk |
+|------|------|-----|------|
+| **`TheTom/llama-cpp-turboquant`** | `llama.cpp` fork with Metal support | Most directly useful — same stack as Ollama. Reports PPL numbers on M-series. | Community fork, not upstream. May lag `llama.cpp` HEAD. |
+| **`TheTom/turboquant_plus`** | Standalone C implementation + Python tests | Most detailed reverse-engineering. 511+ tests. PolarQuant + Walsh-Hadamard + turbo2/3/4 formats. | Extends beyond paper ("Plus"). May include non-paper innovations. |
+| **`amirzandieh/QJL`** | Author's QJL CUDA implementation | Official author code. CUDA kernels, eval scripts, LongBench commands. | CUDA only — needs Metal port for MacBook. Phase 2 dependency. |
+| **`rachittshah/mlx-turboquant`** | MLX proof-of-concept | Native Apple Silicon. Correct module layout (codebooks, polar_quant, qjl). | May be partial implementation. Naming drift noted. |
+
+**Start from:** `TheTom/llama-cpp-turboquant` (for Ollama integration path) + `TheTom/turboquant_plus` (for reference/tests).
+
+### Community Fork Risk Assessment
+
+The v1 spec understated this. Community `llama.cpp` forks can diverge significantly from HEAD, especially in the Metal backend where Apple Silicon optimizations change frequently. The risk isn't "it doesn't build" — it's "it builds fine on the fork's base commit but breaks when cherry-picked onto current HEAD."
+
+**Specific risk areas:**
+- **KV cache layer:** `llama.cpp` has refactored KV cache internals multiple times in 2026. A fork based on a 4-week-old commit may touch structs/functions that have been renamed or restructured upstream.
+- **Metal shaders:** Apple Silicon Metal optimizations are actively changing. Custom Metal kernels for TurboQuant dequant may conflict with upstream shader refactors.
+- **Memory management:** `ggml` memory allocation has evolved. The fork's cache allocation assumptions may not match current `ggml` memory pools.
+
+**Mitigation plan (Phase 1 Step 0 — before any benchmarking):**
+
+1. **Check fork freshness:** `git log --oneline -1` on the fork. Compare base commit date against `llama.cpp` HEAD. If >4 weeks stale, flag as HIGH risk.
+2. **If fresh (< 2 weeks from HEAD):** Build directly. Likely works.
+3. **If stale (2-4 weeks):** Attempt cherry-pick of TurboQuant-specific commits onto current HEAD. If merge conflicts are limited to TurboQuant files → resolve manually. If conflicts touch core KV cache / Metal code → stop, evaluate effort.
+4. **If very stale (> 4 weeks) or conflicts are extensive:** Switch to **clean-room approach** — use `TheTom/turboquant_plus` as the algorithm reference and implement the KV cache types directly into current `llama.cpp` HEAD. This is more work (~60-90 min instead of ~20-40 min) but avoids the merge conflict maze.
+5. **Escape hatch:** If `llama.cpp` path is blocked, fall back to `rachittshah/mlx-turboquant` (MLX native, no fork divergence risk, but requires API proxy for Ollama compatibility).
+
+**Cid decision point:** After Step 0, report fork age + conflict assessment before proceeding. If clean-room is needed, update the time estimate and Frankie adjusts the schedule. Don't spend more than 15 minutes fighting merge conflicts — switch to clean-room.
+
+### Metal Kernel Risk — The Single Highest-Risk Assumption
+
+The spec assumes the `llama.cpp` fork has working **Metal shaders** for PolarQuant KV dequantization. KV dequant happens in the attention computation hot path — every token, every layer, every head. If the fork only has CPU or CUDA dequant kernels and no Metal implementation, the MacBook will either:
+- Fall back to CPU dequant → **catastrophic** performance loss (10-50× slower attention)
+- Fail to build entirely for Metal backend
+
+**Cid's actual first action** (before building, before benchmarking, before anything):
+
+```bash
+# Clone the fork
+git clone https://github.com/TheTom/llama-cpp-turboquant.git
+cd llama-cpp-turboquant
+
+# Check for Metal shader files referencing TurboQuant/PolarQuant
+grep -rn "turbo\|polar\|turboquant\|polarquant" ggml/src/ggml-metal* 2>/dev/null
+grep -rn "turbo\|polar" ggml/src/ggml-metal.metal 2>/dev/null
+
+# Check for Metal kernel dispatch for turbo KV types
+grep -rn "GGML_TYPE_.*TURBO\|turbo.*metal\|kv.*turbo" . --include="*.m" --include="*.metal" --include="*.h" 2>/dev/null
+```
+
+**If Metal shaders exist:** Proceed with `llama.cpp` fork path (primary).
+**If Metal shaders do NOT exist:** MLX becomes the **primary** path, not the fallback. Switch to `rachittshah/mlx-turboquant` immediately. Reframe Phase 1 around MLX + API proxy for Ollama compatibility. Report this finding before spending any more time on the `llama.cpp` path.
+
+This check takes 2 minutes and determines the entire build strategy. Do it first.
+
+---
+
+## 3. Integration Target — llama.cpp → Ollama
+
+**Primary: `llama.cpp` fork → custom Ollama build.**
+
+Why not MLX:
+- Our entire fleet uses Ollama. Model management, API compatibility, endpoint routing — all built around Ollama.
+- MLX would require a separate inference server, separate model format, separate API integration.
+- Ollama is built on `llama.cpp`/`ggml`. KV cache changes in `llama.cpp` propagate to Ollama.
+
+**Integration strategy:**
+1. Build/test TurboQuant KV cache in a `llama.cpp` fork (Metal backend)
+2. Validate quality + performance
+3. Build custom Ollama from our `llama.cpp` fork (Ollama builds `llama.cpp` as a submodule)
+4. Deploy to MacBook as replacement Ollama binary
+5. Existing model files, API, and endpoint (`10.0.0.133:11434`) remain identical — only the inference engine changes
+
+**Fallback: MLX standalone** if `llama.cpp` Metal integration proves too complex. `rachittshah/mlx-turboquant` as starting point. Would require a small proxy server to maintain API compatibility with our Ollama endpoint.
+
+---
+
+## 4. Validation Plan — How We Know It Works
+
+### Quality Validation
+
+**Test matrix (run each model with and without TurboQuant):**
+
+| Test | What It Measures | Tool | Pass Criteria |
+|------|-----------------|------|--------------|
+| Perplexity (PPL) | Overall language modeling quality | `llama-perplexity` on WikiText-2 | PPL delta ≤ 0.5 from baseline (FP16 KV) |
+| Needle-in-Haystack | Long context retrieval | Custom prompt at 8K/16K/32K/64K/128K | 100% retrieval at all lengths where baseline passes |
+| Practical generation | Subjective quality | 10 predefined prompts (see test suite below) | Human review: no degradation on ≥9/10 |
+| Attention score accuracy | Inner product preservation | Cosine similarity between TurboQuant and FP16 attention weights | cosine sim ≥ 0.995 |
+
+**Predefined Test Prompts (10 prompts, run identically on TurboQuant and FP16 KV baseline):**
+
+| # | Category | Prompt Description | What It Tests |
+|---|----------|-------------------|---------------|
+| 1 | Long-context summarization | Feed 20K tokens of a research paper, ask for structured summary with citations | KV cache quality at length — compressed K/V must retain source detail |
+| 2 | Multi-step reasoning | 5-step math word problem requiring chain-of-thought | Whether compressed KV degrades intermediate reasoning steps |
+| 3 | Code generation | Write a Python script with 3 functions, error handling, type hints | Precise token prediction — code is unforgiving of subtle quality drops |
+| 4 | Code debugging | Provide buggy code (3 bugs), ask to identify and fix all three | Attention to detail across context — must reference earlier code correctly |
+| 5 | Factual recall (early context) | Provide 10 facts in the first 1K tokens, continue for 8K tokens of filler, ask about fact #3 | Retrieval from early context through compressed KV |
+| 6 | Creative writing | Write a 500-word short story with specific constraints (setting, character, twist) | Compression artifacts surface as repetition or coherence loss |
+| 7 | Multi-turn conversation | 10-turn technical Q&A where later questions reference earlier answers | Cross-turn coherence through accumulated compressed KV |
+| 8 | Structured output | Generate a JSON schema with 15+ fields, nested objects, and validation rules | Format precision — compressed KV must maintain structural consistency |
+| 9 | Translation + analysis | Translate a paragraph EN→ES, then analyze the translation choices | Tests both generation quality and meta-reasoning about own output |
+| 10 | Instruction following | Complex prompt with 8 specific formatting requirements (headers, bullet style, word limits, etc.) | Whether compression causes the model to "forget" constraints mid-generation |
+
+**Prompts must be written and saved to `projects/sovereign-stack/turboquant-test-prompts.md` before Phase 2 benchmarks run.** Same prompts, same order, both configurations. This prevents unconscious cherry-picking.
+
+**Asymmetric K/V test:** Run K at Q8_0, V at turbo4. Community reports this works well on sensitive models. Compare PPL vs symmetric turbo4 K+V.
+
+**Long-session quality test (Phase 2 only):** Short-context PPL benchmarks can miss quality degradation that surfaces during sustained context pressure. During Phase 2, run one extended production simulation:
+- Generate a 50-turn multi-step reasoning conversation (code gen → debug → refactor → test → iterate)
+- Compare output quality vs same conversation on FP16 KV baseline
+- Specifically watch for: coherence drift after turn 30+, hallucinated references to earlier context, attention score softmax concentration (if measurable)
+- This catches the case where codebook boundary errors accumulate over many KV cache writes in a single session
+
+### Performance Validation
+
+| Metric | Measure | Pass Criteria |
+|--------|---------|--------------|
+| Tokens/second (generation) | `llama-bench` | ≥90% of baseline tok/s (small decode overhead acceptable) |
+| Time to first token (TTFT) | Timed prompt eval | ≤110% of baseline |
+| Peak resident memory | `footprint -p <pid>` or `vmmap --summary` at each context length | Stays under 27GB at target context length |
+| Memory vs theoretical | Compare measured peak to calculated estimate | If measured exceeds calculated by >15% → reduce context ceiling |
+| Context length ceiling | Binary search: max context before OOM or swap pressure | 64K minimum (vs ~32K baseline for 27B) |
+
+### Kill Criteria
+- PPL regression > 1.0 at any compression level → abort that compression level
+- OOM at 32K context (baseline capability) → regression, abort
+- tok/s drops > 25% → dequant overhead too high, need kernel optimization before deploy
+
+---
+
+## 5. Who Does What
+
+| Role | Owner | What |
+|------|-------|------|
+| Build spec | Strago | This document ✅ |
+| Implementation | Cid | Fork `llama.cpp`, integrate PolarQuant KV cache, Metal kernels, build custom Ollama |
+| Validation | Cid | Run test matrix, report PPL/performance numbers |
+| Model selection | Cid | Test qwen3.5:27b + one Mistral variant, recommend best config |
+| MacBook deployment | Cid | Replace Ollama binary on MacBook, verify endpoint works |
+| Quality review | John | Review 10-prompt practical generation comparison |
+| Research support | Locke | If Cid hits a wall on the math, Locke deep-dives the paper/QJL code |
+
+---
+
+## 6. Phasing
+
+### Phase 1 — PolarQuant MVP (Target: this week)
+
+**Scope:**
+
+**Step 0 — Fork Assessment (do this FIRST, report before proceeding):**
+- Clone `TheTom/llama-cpp-turboquant`
+- Check base commit age vs `llama.cpp` HEAD (`git log --oneline -1`)
+- Check `sysctl hw.memsize` on MacBook (resolve the 32/36/48GB question)
+- If fork < 2 weeks stale → proceed to build
+- If 2-4 weeks stale → attempt cherry-pick, report conflict scope
+- If > 4 weeks or conflicts extensive → switch to clean-room (see Fork Risk Assessment above)
+- Report: fork age, conflict assessment, MacBook actual RAM, estimated build path time
+
+**Step 1 — Build + Verify:**
+- Build `llama.cpp` fork (or clean-room) with Metal backend on MacBook (M4 Max)
+- Run the Section 1a verification checklist against the fork's implementation before trusting any benchmarks
+- Run FP16 KV baseline: `llama-perplexity` on WikiText-2 with `qwen3.5:27b` at 8K context (this is the number we're comparing against)
+
+**Step 2 — Benchmark PolarQuant:**
+- Run perplexity test with PolarQuant KV (turbo4 format) vs FP16 KV baseline
+- Run `llama-bench` for tok/s comparison
+- Test at 8K, 32K, and 64K context lengths
+- Run asymmetric test: K at Q8_0, V at turbo4
+- **Measure actual peak resident memory** at each context length (`footprint -p <pid>` or `vmmap --summary`). Compare measured vs calculated. If measured exceeds calculated by >15%, note the delta — it reduces the achievable context ceiling.
+- Report: PPL delta per context length, tok/s delta, **measured peak memory per context length**, max context before OOM/swap, asymmetric vs symmetric results
+
+**Deliverable:** Working `llama.cpp` build on MacBook with PolarQuant KV cache. PPL + performance numbers. Section 1a verification checklist completed.
+
+**Estimated Cid time (honest range):**
+- **Best case** — fork is fresh, builds clean on first try, Metal shaders work: 20-40 min
+- **Typical case** — fork needs CMake flag tweaks, Xcode SDK adjustments, minor Metal fixes: 1-2 hours
+- **Worst case** — fork is stale, conflicts extensive, or Metal shaders missing: clean-room build 2-4 hours, or MLX pivot
+
+**2-hour build troubleshooting cap:** If the `llama.cpp` fork doesn't compile and pass a basic smoke test (load model, generate 10 tokens) within 2 hours of troubleshooting, stop. Pivot to MLX path. Don't sink more time into Xcode/CMake/Metal debug loops when a working MLX PoC exists. Report what broke — the information is useful even if the path is abandoned.
+
+**Decision gate:** If PPL delta ≤ 0.5 and tok/s ≥ 90% baseline AND Section 1a checklist passes → proceed to Phase 2. If PPL fails but checklist passes → try asymmetric K/V or lower compression (turbo3 instead of turbo4). If checklist fails → fix implementation before trusting benchmarks.
+
+### Phase 2 — Ollama Integration + Production Deploy
+
+**Scope:**
+
+**Step 0 — Ollama API Compatibility Check (before building):**
+Ollama pins a specific `llama.cpp` commit and calls it through CGo bindings in `llm/`. If our fork changes any function signatures, struct layouts, or enum values that Ollama's Go code references, the build will either fail or produce subtle runtime bugs.
+
+```bash
+# Clone Ollama source
+git clone https://github.com/ollama/ollama.git
+cd ollama
+
+# Find the pinned llama.cpp commit
+cat llm/llama.cpp/CMakeLists.txt | head -5  # or check go.mod / Makefile
+
+# Diff our fork's API surface against Ollama's expected API
+# Focus on: llama.h, ggml.h function signatures that Ollama calls
+diff <(grep -h "^LLAMA_API\|^GGML_API" llm/llama.cpp/include/*.h | sort) \
+     <(grep -h "^LLAMA_API\|^GGML_API" /path/to/our-fork/include/*.h | sort)
+```
+
+If API surface differs: check if TurboQuant changes are additive (new functions/types only) or modify existing signatures. Additive = safe. Modified existing = need to update Ollama's CGo bindings.
+
+**Build steps:**
+- Build custom Ollama binary using our `llama.cpp` fork as submodule
+- Deploy to MacBook as replacement Ollama
+- Verify existing endpoint (`10.0.0.133:11434`) works identically
+- Run full test matrix (all 4 quality tests + all 4 performance tests)
+- Test with qwen3.5:27b at 64K and 128K context
+- If 128K works: update Ollama model config to advertise larger context
+- Run 10-prompt practical generation comparison for John review
+
+**Deliverable:** Production Ollama on MacBook with TurboQuant KV cache. Full benchmark report. John signs off on quality.
+
+**Estimated Cid time:** 15-25 min (Ollama build is straightforward once `llama.cpp` fork is validated).
+
+### Phase 2.5 — Per-Layer Quantization Profiles (Optimization, Optional)
+
+Not all transformer layers have equal sensitivity to KV cache quantization. Research and community experimentation show early layers (first 2-4) and late layers (last 2-4) tend to be more sensitive than middle layers. If the fork supports per-layer KV cache type configuration:
+
+- **Sensitive layers (first 3 + last 3):** K at Q8_0, V at turbo4 (or full FP16 KV)
+- **Middle layers:** K and V both at turbo4 (or even turbo3)
+
+This gives the same average compression ratio as uniform turbo4 but concentrates precision where it matters most. The PPL improvement can be meaningful (0.1-0.3) at zero memory cost.
+
+**When to pursue:** Only after Phase 2 is stable and baseline quality is confirmed. This is tuning, not architecture. If uniform turbo4 passes all quality gates, per-layer optimization is nice-to-have, not necessary.
+
+**Cid note:** During Phase 1, check whether the fork exposes per-layer KV type config. If it does, note it for later. Don't implement it yet.
+
+### Phase 3 — QJL Residual Correction (Optional)
+
+**Scope:** Add QJL 1-bit residual correction for full TurboQuant behavior. Only pursue if:
+- Phase 1/2 PolarQuant shows quality gaps at extreme compression (< 3 bits/channel)
+- We want to push to 2.5 bits/channel for even more context headroom
+
+**Source:** `amirzandieh/QJL` repo (CUDA → Metal port needed)
+
+**Estimated Cid time:** 30-60 min (Metal port of QJL kernels is real engineering work)
+
+**Decision gate:** Only proceed if PolarQuant alone doesn't meet quality bar at target compression.
+
+### Phase 4 — Upstream Watch
+
+**Scope:** Monitor `llama.cpp` upstream and Ollama for official TurboQuant support. When it lands:
+- Evaluate upstream implementation vs our fork
+- If upstream is better: migrate off our fork to official
+- If our fork is better: contribute upstream (optional)
+
+**Owner:** Locke (monitoring) + Cid (evaluation when it lands)
+
+---
+
+## What This Spec Does NOT Cover
+
+- **Weight quantization** — TurboQuant is KV cache compression only. Model weight quantization (GGUF Q4_K_M etc.) is a separate concern and already handled by Ollama.
+- **Predator (desktop) deployment** — this spec targets MacBook only. Predator runs NVIDIA (CUDA) which is a different kernel backend. Can extend later.
+- **Multi-model serving** — TurboQuant helps with single-model memory but doesn't change Ollama's single-model-at-a-time constraint.
+- **Ollama upstream contribution** — out of scope for now. We build for ourselves first.
+
+---
+
+## Open Questions for John
+
+**None blocking.** One informational:
+
+- **MacBook Pro memory:** Confirmed M4 Max 32GB from memory/2026-03-14.md. If it's actually 36GB or 48GB (M4 Max comes in 36/48/128 configs), that changes the model ceiling. Can Cid check `sysctl hw.memsize` on the MacBook during Phase 1? Non-blocking — doesn't change the approach, just the model size ceiling.
+
+---
+
+## Reference Files
+
+| File | Location |
+|------|----------|
+| TurboQuant Google Brief | `projects/sovereign-stack/research/turboquant-2026-03-25.md` |
+| Locke Recon Update | `projects/sovereign-stack/research/turboquant-2026-03-30-recon-update.md` |
+| `llama.cpp` TurboQuant fork | `github.com/TheTom/llama-cpp-turboquant` |
+| TurboQuant+ reference impl | `github.com/TheTom/turboquant_plus` |
+| QJL author code | `github.com/amirzandieh/QJL` |
+| MLX PoC (fallback) | `github.com/rachittshah/mlx-turboquant` |
+| TurboQuant paper | `arxiv.org/abs/2504.19874` |
+| PolarQuant paper | `arxiv.org/abs/2502.02617` |
+
+---
+
+---
+
+## Changelog
+
+- **v1 (2026-03-30 12:26 ET):** Initial spec.
+- **v2 (2026-03-30 12:55 ET):** Added Section 1a (PolarQuant technical detail + Cid verification checklist), expanded fork risk assessment with mitigation plan, added Phase 1 Step 0 (fork assessment before benchmarking), added long-session quality test for Phase 2, updated Phase 1 time estimate for clean-room path. Changes driven by external Opus review round 1.
+- **v2.1 (2026-03-30 13:00 ET):** Added Metal kernel risk check (grep before build — determines llama.cpp vs MLX primary path), corrected memory budget (27GB available, not 30GB — accounts for OS + Metal driver + activations), added measured memory profiling requirement to Phase 1, added Ollama CGo API compatibility check to Phase 2 Step 0, tightened model ceiling estimates. Changes driven by external Opus review round 2.
+- **v2.2 (2026-03-30 13:05 ET):** Added honest time estimate range (20 min best → 2-4 hr worst), 2-hour build troubleshooting cap before MLX pivot, PolarQuant initialization detail (WHT + Lloyd-Max codebook setup + cold-start measurement target), 10 predefined test prompts with rationale (prevents cherry-picking), per-layer quantization profiles as Phase 2.5 optimization path. Changes driven by external Opus review round 3.
+
+---
+
+*Build spec v2 ready for Cid intake. No clarifying questions needed.*
+
+
+---
+

docs/STATUS_TRACKER.md

@@ -0,0 +1,60 @@
+# TurboQuant Living Status Tracker
+
+Updated on each milestone. See PROJECT_STATUS.md for detailed phase reports.
+
+## Quick Status
+
+| Phase | Status | Last Updated | Issue |
+|-------|--------|-------------|-------|
+| Phase 1: PolarQuant MVP | DONE | 2026-03-30 | #17 |
+| Phase 2: KV Cache Compression | IN PROGRESS | 2026-04-15 | #99 |
+| Edge Crisis Detection | DONE | 2026-04-15 | #102 |
+| Integration PR (upstream llama.cpp) | NOT STARTED | — | — |
+| QJL Quantization | NOT STARTED | — | — |
+| Ollama Integration | NOT STARTED | — | — |
+| Benchmark Suite | IN PROGRESS | 2026-04-13 | #12 |
+
+## Phase Details
+
+### Phase 1: PolarQuant MVP — COMPLETE
+- PolarQuant KV cache compression working on Apple Silicon
+- 73% KV memory savings, 1% prompt overhead, 11% generation overhead
+- Metal shaders: flash attention, WHT rotation, codebooks
+- Hardware: M3 Max 36GB (corrected from spec)
+- Gate Check #2: PASSED
+
+### Phase 2: Edge Deployment — COMPLETE
+- Crisis detection on edge devices (Pi 4, old phones)
+- Keyword + model (gemma2:2b) + offline resources
+- Deployment guide, model selection, resource cache
+- See docs/edge-crisis-deployment.md
+
+### Phase 3: Upstream Integration — NOT STARTED
+- PR to llama.cpp for turbo quantization
+- Depends on Phase 2 benchmarks
+
+### Phase 4: QJL — NOT STARTED
+- Johnson-Lindenstrauss quantization
+- Lower memory than PolarQuant
+- Research phase
+
+## Recent Milestones
+
+| Date | Milestone | PR/Issue |
+|------|-----------|----------|
+| 2026-04-15 | Edge crisis detection deployed | #102 / PR #111 |
+| 2026-04-14 | KV cache compression profiles | PR #68 |
+| 2026-04-13 | Benchmark suite expanded | #12 / #39 |
+| 2026-03-30 | Phase 1 complete: PolarQuant MVP | #17 |
+
+## Open Blockers
+
+| Blocker | Impact | Issue |
+|---------|--------|-------|
+| None currently | — | — |
+
+---
+
+*Last auto-updated: 2026-04-15*
+*This file is the single source of truth for project status.*
+*Update it on every milestone merge.*

evolution/hardware_optimizer.py

@@ -0,0 +1,5 @@
+"""Phase 19: Hardware-Aware Inference Optimization.
+Part of the TurboQuant suite for local inference excellence.
+"""
+import logging
+# ... (rest of the code)

profiles/README.md

@@ -0,0 +1,139 @@
+# Hermes Profiles for TurboQuant
+
+This directory contains Hermes configuration profiles for running models with TurboQuant KV cache compression.
+
+## Available Profiles
+
+### gemma4-turboquant.yaml
+
+**Profile for Gemma 4 model with TurboQuant KV cache compression.**
+
+- **Primary Provider:** Local llama.cpp server with TurboQuant enabled
+- **Endpoint:** http://localhost:8081
+- **KV Compression:** turbo4 (4-bit PolarQuant)
+- **Context Length:** 128K tokens
+- **Memory Savings:** ~73% KV cache reduction
+- **Fallback Providers:** Ollama, OpenAI-compatible API
+
+## Quick Start
+
+### 1. Build TurboQuant-enabled llama.cpp
+
+```bash
+git clone https://github.com/TheTom/llama-cpp-turboquant.git
+cd llama-cpp-turboquant
+git checkout feature/turboquant-kv-cache
+cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
+cmake --build build -j$(sysctl -n hw.ncpu)
+```
+
+### 2. Download Gemma 4 Model
+
+```bash
+# Download Gemma 4 Q4_K_M quantized model
+huggingface-cli download <model-repo> gemma-4-q4_k_m.gguf
+```
+
+### 3. Start llama-server with TurboQuant
+
+```bash
+export TURBO_LAYER_ADAPTIVE=7
+./build/bin/llama-server \
+  -m /path/to/gemma-4-q4_k_m.gguf \
+  --port 8081 \
+  -ctk turbo4 -ctv turbo4 \
+  -c 131072 \
+  --host 0.0.0.0
+```
+
+### 4. Install Profile
+
+```bash
+# Copy profile to Hermes directory
+cp gemma4-turboquant.yaml ~/.hermes/profiles/
+
+# Or create symlink
+ln -sf $(pwd)/gemma4-turboquant.yaml ~/.hermes/profiles/
+```
+
+### 5. Use with Hermes
+
+```bash
+# Start Hermes with the profile
+hermes --profile gemma4-turboquant
+
+# Or specify profile in Hermes config
+echo "default_profile: gemma4-turboquant" >> ~/.hermes/config.yaml
+```
+
+## Profile Configuration
+
+The profile includes:
+
+- **Primary Provider:** Local llama.cpp server with TurboQuant
+- **Fallback Providers:** Ollama (local), OpenAI (cloud)
+- **TurboQuant Settings:**
+  - `kv_type`: turbo4 (4-bit compression)
+  - `layer_adaptive_mode`: 7 (best quality/compression ratio)
+  - `max_context`: 128K tokens
+
+## Performance Expectations
+
+| Metric | Value | Notes |
+|--------|-------|-------|
+| KV Memory Savings | 73% | Measured on M3 Max |
+| Prompt Processing | ~1% overhead | vs FP16 baseline |
+| Generation Speed | ~11% overhead | vs FP16 baseline |
+| Max Context (36GB) | 128K | Comfortable with 7.6GB headroom |
+
+## Customization
+
+### Adjust Compression Level
+
+```yaml
+turboquant:
+  kv_type: "turbo3"  # Lower compression, faster
+  # or
+  kv_type: "turbo2"  # Minimal compression, fastest
+```
+
+### Disable Per-Layer Adaptive
+
+```yaml
+turboquant:
+  layer_adaptive_mode: 0  # Uniform quantization
+```
+
+### Use Asymmetric K/V
+
+For better quality on sensitive models:
+
+```bash
+# Start server with asymmetric K/V
+llama-server -m model.gguf --port 8081 -ctk q8_0 -ctv turbo4 -c 131072
+```
+
+## Troubleshooting
+
+### Server Won't Start
+
+1. Check if port 8081 is available: `lsof -i :8081`
+2. Verify model path is correct
+3. Ensure TurboQuant branch is checked out
+
+### Poor Generation Quality
+
+1. Try `turbo3` instead of `turbo4`
+2. Disable per-layer adaptive (mode 0)
+3. Use asymmetric K/V: `-ctk q8_0 -ctv turbo4`
+
+### High Memory Usage
+
+1. Reduce context length: `-c 65536` (64K)
+2. Check `TURBO_LAYER_ADAPTIVE` is set
+3. Monitor with: `vmmap --summary $(pgrep llama-server)`
+
+## References
+
+- [Project Status](../docs/PROJECT_STATUS.md)
+- [llama.cpp TurboQuant Fork](https://github.com/TheTom/llama-cpp-turboquant)

profiles/hermes-profile-gemma4-turboquant.yaml

@@ -0,0 +1,169 @@
+# Hermes Profile: Gemma 4 + TurboQuant KV Cache Compression
+# For use with local llama.cpp server running TurboQuant-enabled inference
+# Drop into ~/.hermes/profiles/gemma4-turboquant.yaml
+
+profile:
+  name: "gemma4-turboquant"
+  version: "1.0.0"
+  description: "Gemma 4 model with TurboQuant KV cache compression for extended context on Apple Silicon"
+
+# Primary provider: local llama.cpp server with TurboQuant
+providers:
+  primary:
+    type: "llama.cpp"
+    name: "local-turboquant"
+    endpoint: "http://localhost:8081"
+    api_path: "/v1/chat/completions"
+    timeout_ms: 120000
+    
+    # Model configuration
+    model:
+      name: "gemma-4"
+      path: "/path/to/gemma-4-q4_k_m.gguf"  # Update with actual model path
+      
+    # TurboQuant KV cache compression settings
+    turboquant:
+      enabled: true
+      kv_type: "turbo4"  # Options: turbo2, turbo3, turbo4 (4-bit recommended)
+      layer_adaptive_mode: 7  # Per-layer adaptive quantization (0-7, 7=best quality/ratio)
+      
+    # Context and memory settings
+    context:
+      max_tokens: 131072  # 128K context with TurboQuant compression
+      batch_size: 512
+      
+    # Generation parameters
+    generation:
+      temperature: 0.7
+      top_p: 0.9
+      top_k: 40
+      repeat_penalty: 1.1
+      frequency_penalty: 0.0
+      presence_penalty: 0.0
+      
+    # Server startup command (for reference)
+    server_command: |
+      export TURBO_LAYER_ADAPTIVE=7
+      llama-server \
+        -m /path/to/gemma-4-q4_k_m.gguf \
+        --port 8081 \
+        -ctk turbo4 -ctv turbo4 \
+        -c 131072 \
+        --host 0.0.0.0
+
+  # Fallback provider 1: Ollama (standard, no TurboQuant)
+  fallback_1:
+    type: "ollama"
+    name: "ollama-gemma4"
+    endpoint: "http://localhost:11434"
+    api_path: "/api/chat"
+    timeout_ms: 120000
+    
+    model:
+      name: "gemma4:latest"
+      
+    generation:
+      temperature: 0.7
+      top_p: 0.9
+      top_k: 40
+
+  # Fallback provider 2: OpenAI-compatible API (cloud backup)
+  fallback_2:
+    type: "openai"
+    name: "openai-backup"
+    endpoint: "https://api.openai.com"
+    api_path: "/v1/chat/completions"
+    timeout_ms: 60000
+    
+    model:
+      name: "gpt-4"
+      
+    generation:
+      temperature: 0.7
+      max_tokens: 4096
+
+# Performance and monitoring
+performance:
+  # Memory management for TurboQuant
+  memory:
+    max_gpu_memory_gb: 28  # Leave headroom on 36GB M3 Max
+    kv_cache_compression: "turbo4"
+    estimated_savings: "73%"  # TurboQuant delivers ~73% KV memory savings
+    
+  # Benchmarking integration
+  benchmarks:
+    enabled: true
+    metrics:
+      - "tokens_per_second"
+      - "time_to_first_token"
+      - "peak_memory_usage"
+      - "perplexity"
+
+# Quality validation
+quality:
+  # Test prompts for quality comparison
+  test_prompts:
+    enabled: true
+    prompt_file: "benchmarks/prompts.json"
+    
+  # Perplexity testing
+  perplexity:
+    enabled: true
+    corpus: "wikitext-2-raw"
+    context_lengths: [8192, 32768, 65536, 131072]
+
+# Environment variables (applied when using this profile)
+environment:
+  TURBO_LAYER_ADAPTIVE: "7"  # Per-layer adaptive quantization mode
+  GGML_METAL_DEBUG: "0"  # Disable Metal debug in production
+  OMP_NUM_THREADS: "8"  # Optimize for M3 Max performance cores
+
+# Logging and diagnostics
+logging:
+  level: "info"
+  metrics_interval_seconds: 60
+  log_token_speed: true
+  log_memory_usage: true
+
+# Notes for deployment
+notes:
+  deployment: |
+    1. Ensure llama.cpp fork with TurboQuant is built:
+       cd /path/to/llama-cpp-turboquant
+       git checkout feature/turboquant-kv-cache
+       cmake -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
+       cmake --build build -j$(sysctl -n hw.ncpu)
+    
+    2. Start the server:
+       export TURBO_LAYER_ADAPTIVE=7
+       ./build/bin/llama-server \
+         -m /path/to/gemma-4-q4_k_m.gguf \
+         --port 8081 \
+         -ctk turbo4 -ctv turbo4 \
+         -c 131072 \
+         --host 0.0.0.0
+    
+    3. Verify server is running:
+       curl http://localhost:8081/v1/models
+    
+    4. Copy this profile to Hermes:
+       cp hermes-profile-gemma4-turboquant.yaml ~/.hermes/profiles/
+    
+  performance_notes: |
+    TurboQuant delivers:
+    - 73% KV cache memory savings
+    - 1% prompt processing overhead
+    - 11% generation overhead
+    - Enables 128K context on 36GB hardware
+    
+    With TurboQuant on Gemma 4 (estimated):
+    - Model weights: ~16GB at Q4_K_M
+    - KV cache at 128K: ~5GB (vs ~20GB without compression)
+    - Total memory: ~23GB (fits comfortably in 31GB budget)
+    
+  troubleshooting: |
+    - If generation speed is slow, try turbo3 instead of turbo4
+    - If quality issues, disable per-layer adaptive (set mode to 0)
+    - For maximum quality on sensitive layers, use asymmetric K/V:
+      -ctk q8_0 -ctv turbo4
+    - Monitor memory with: vmmap --summary $(pgrep llama-server)

tests/roundtrip_test.cpp

@@ -0,0 +1,104 @@
+#include "llama-turbo.h"
+
+#include <cmath>
+#include <cstdint>
+#include <iostream>
+#include <random>
+#include <string>
+#include <vector>
+
+namespace {
+
+constexpr int kDim = 128;
+constexpr float kCosineThreshold = 0.99f;
+constexpr float kZeroTolerance = 1.0e-6f;
+
+[[nodiscard]] bool all_finite(const std::vector<float> & values) {
+    for (float value : values) {
+        if (!std::isfinite(value)) {
+            return false;
+        }
+    }
+    return true;
+}
+
+[[nodiscard]] float max_abs(const std::vector<float> & values) {
+    float best = 0.0f;
+    for (float value : values) {
+        best = std::max(best, std::fabs(value));
+    }
+    return best;
+}
+
+[[nodiscard]] float cosine_similarity(const std::vector<float> & lhs, const std::vector<float> & rhs) {
+    float dot = 0.0f;
+    float lhs_norm = 0.0f;
+    float rhs_norm = 0.0f;
+    for (int i = 0; i < kDim; ++i) {
+        dot += lhs[i] * rhs[i];
+        lhs_norm += lhs[i] * lhs[i];
+        rhs_norm += rhs[i] * rhs[i];
+    }
+
+    const float denom = std::sqrt(lhs_norm) * std::sqrt(rhs_norm);
+    return denom == 0.0f ? 1.0f : dot / denom;
+}
+
+[[nodiscard]] std::vector<float> roundtrip(const std::vector<float> & input, float & norm_out) {
+    std::vector<uint8_t> packed(kDim / 2, 0);
+    norm_out = -1.0f;
+    polar_quant_encode_turbo4(input.data(), packed.data(), &norm_out, kDim);
+
+    std::vector<float> decoded(kDim, 0.0f);
+    polar_quant_decode_turbo4(packed.data(), decoded.data(), norm_out, kDim);
+    return decoded;
+}
+
+void require(bool condition, const std::string & message) {
+    if (!condition) {
+        throw std::runtime_error(message);
+    }
+}
+
+void test_zero_vector_roundtrip() {
+    std::vector<float> zeros(kDim, 0.0f);
+    float norm = -1.0f;
+    const auto decoded = roundtrip(zeros, norm);
+
+    require(norm == 0.0f, "zero vector should encode with zero norm");
+    require(all_finite(decoded), "zero vector decode produced non-finite values");
+    require(max_abs(decoded) <= kZeroTolerance, "zero vector decode should remain near zero");
+}
+
+void test_gaussian_roundtrip_quality() {
+    std::mt19937 rng(12345);
+    std::normal_distribution<float> dist(0.0f, 1.0f);
+
+    std::vector<float> input(kDim, 0.0f);
+    for (float & value : input) {
+        value = dist(rng);
+    }
+
+    float norm = -1.0f;
+    const auto decoded = roundtrip(input, norm);
+
+    require(norm > 0.0f, "random vector should encode with positive norm");
+    require(all_finite(decoded), "random vector decode produced non-finite values");
+
+    const float cosine = cosine_similarity(input, decoded);
+    require(cosine >= kCosineThreshold, "roundtrip cosine similarity below threshold");
+}
+
+}  // namespace
+
+int main() {
+    try {
+        test_zero_vector_roundtrip();
+        test_gaussian_roundtrip_quality();
+        std::cout << "PASS: turboquant standalone roundtrip tests\n";
+        return 0;
+    } catch (const std::exception & exc) {
+        std::cerr << "FAIL: " << exc.what() << '\n';
+        return 1;
+    }
+}