Compare commits

...

6 Commits

Author SHA1 Message Date
Alexander Whitestone
baded69890 feat: weekly progress update system for TurboQuant (#76)
All checks were successful
Smoke Test / smoke (pull_request) Successful in 14s
- scripts/weekly_update.py: Auto-generates weekly update from git log,
  Gitea API (issues/PRs/blockers), and benchmark results. Supports
  --post to issue #76, --json for raw data, --since for date range.
- scripts/weekly_update.sh: Shell wrapper for convenience.
- docs/WEEKLY_TEMPLATE.md: Manual update template.
- docs/PROJECT_STATUS.md: Added Weekly Progress Updates section with
  process (weekly cadence, benchmark-as-happens, blocker escalation).
- tests/test_weekly_update.py: Validates script runs, JSON output,
  and handles edge cases.
2026-04-15 22:00:17 -04:00
3cd8750cbb Merge pull request 'feat: standalone build system and roundtrip tests - #17' (#51) from dispatch/17-1776180746 into main
All checks were successful
Smoke Test / smoke (pull_request) Successful in 15s
2026-04-15 11:57:58 +00:00
ef765bbd30 Merge pull request 'fix(docs): resolve broken markdown links and stale forge URL' (#52) from burn/fix-doc-links into main 2026-04-15 11:57:55 +00:00
Hermes Agent
5f0d00f127 fix(docs): resolve broken markdown links and stale forge URL
All checks were successful
Smoke Test / smoke (pull_request) Successful in 6s
- Update raw-IP forge URL to canonical forge domain in README.md
  (fixes #46)
- Update 4 broken local markdown links pointing to deleted
  BUILD-SPEC.md, PHASE1-REPORT.md, FULL-REPORT.md to
  docs/PROJECT_STATUS.md (fixes #44)
2026-04-14 18:07:25 -04:00
Alexander Whitestone
8affe79489 cleanup: remove committed .pyc and redundant Python test, add .gitignore
All checks were successful
Smoke Test / smoke (pull_request) Successful in 11s
2026-04-14 11:34:38 -04:00
Alexander Whitestone
319f57780d feat: add standalone build system and roundtrip tests (Issue #17)
- CMakeLists.txt: builds turboquant as static library
- TURBOQUANT_BUILD_TESTS option enables ctest roundtrip tests
- tests/roundtrip_test.cpp: validates zero-vector roundtrip and
  gaussian cosine similarity (>=0.99)
- Makefile wrapper for convenience (build/test/clean targets)
- Addresses contributor feedback on spec-to-code gap and CI from #17
2026-04-14 11:34:38 -04:00
10 changed files with 595 additions and 5 deletions

3
.gitignore vendored Normal file
View File

@@ -0,0 +1,3 @@
build/
*.pyc
__pycache__/

36
CMakeLists.txt Normal file
View File

@@ -0,0 +1,36 @@
cmake_minimum_required(VERSION 3.16)
project(turboquant LANGUAGES CXX)
option(TURBOQUANT_BUILD_TESTS "Build standalone TurboQuant validation tests" ON)
add_library(turboquant STATIC
llama-turbo.cpp
)
target_include_directories(turboquant PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}
)
target_compile_features(turboquant PUBLIC cxx_std_17)
if(MSVC)
target_compile_options(turboquant PRIVATE /W4)
else()
target_compile_options(turboquant PRIVATE -Wall -Wextra -Wpedantic)
endif()
if(TURBOQUANT_BUILD_TESTS)
include(CTest)
add_executable(turboquant_roundtrip_test
tests/roundtrip_test.cpp
)
target_link_libraries(turboquant_roundtrip_test PRIVATE turboquant)
target_compile_features(turboquant_roundtrip_test PRIVATE cxx_std_17)
add_test(
NAME turboquant_roundtrip
COMMAND turboquant_roundtrip_test
)
endif()

View File

@@ -13,7 +13,7 @@ Unlock 64K-128K context on qwen3.5:27b within 32GB unified memory.
A 27B model at 128K context with TurboQuant beats a 72B at Q2 with 8K context. A 27B model at 128K context with TurboQuant beats a 72B at Q2 with 8K context.
## Status ## Status
See [issues](http://143.198.27.163:3000/Timmy_Foundation/turboquant/issues) for current progress. See [issues](https://forge.alexanderwhitestone.com/Timmy_Foundation/turboquant/issues) for current progress.
## Roles ## Roles
- **Strago:** Build spec author - **Strago:** Build spec author
@@ -29,4 +29,4 @@ See [issues](http://143.198.27.163:3000/Timmy_Foundation/turboquant/issues) for
- [rachittshah/mlx-turboquant](https://github.com/rachittshah/mlx-turboquant) — MLX fallback - [rachittshah/mlx-turboquant](https://github.com/rachittshah/mlx-turboquant) — MLX fallback
## Docs ## Docs
- [BUILD-SPEC.md](BUILD-SPEC.md) — Full build specification (Strago, v2.2) - [Project Status](docs/PROJECT_STATUS.md) — Full project status and build specification

View File

@@ -389,6 +389,40 @@ Step 7: If pass → production. If fail → drop to turbo3 or adjust per-layer p
*Build: /tmp/llama-cpp-turboquant/build/bin/ (all binaries)* *Build: /tmp/llama-cpp-turboquant/build/bin/ (all binaries)*
*Branch: feature/turboquant-kv-cache* *Branch: feature/turboquant-kv-cache*
---
# Weekly Progress Updates
**Tracking issue:** #76
**Process established:** 2026-04-16
## Process
1. **Weekly cadence** — Every Monday, generate and post a progress update as a comment on issue #76.
2. **Benchmark results** — Post as they happen (don't wait for weekly update).
3. **Blocker escalation** — New blockers posted within 24 hours with label `blocker`.
4. **PROJECT_STATUS.md** — Updated weekly with current state.
## How to Generate
```bash
# Auto-generated from git log + Gitea API
python3 scripts/weekly_update.py --post
# Preview first
python3 scripts/weekly_update.py
# Custom date range
python3 scripts/weekly_update.py --since 2026-04-01
# Raw JSON data
python3 scripts/weekly_update.py --json
```
## Template
See `docs/WEEKLY_TEMPLATE.md` for manual updates.
--- ---

26
docs/WEEKLY_TEMPLATE.md Normal file
View File

@@ -0,0 +1,26 @@
# TurboQuant Weekly Update Template
Use this template when posting manual weekly updates. For automated updates, run `scripts/weekly_update.py --post`.
## Week of [START_DATE] to [END_DATE]
### Completed
- [item 1]
- [item 2]
### Benchmark Results
- [key metric or "No new benchmarks this week"]
### In Progress
- [item being worked on — who's on it]
### Blockers
- [blocker — impact — who needs to act]
- _None_ if clear
### Next Week
- [planned item 1]
- [planned item 2]
---
_Generated by `scripts/weekly_update.py` or filled manually._

View File

@@ -135,7 +135,5 @@ llama-server -m model.gguf --port 8081 -ctk q8_0 -ctv turbo4 -c 131072
## References ## References
- [TurboQuant Build Spec](../BUILD-SPEC.md) - [Project Status](../docs/PROJECT_STATUS.md)
- [Phase 1 Report](../PHASE1-REPORT.md)
- [Full Knowledge Transfer](../FULL-REPORT.md)
- [llama.cpp TurboQuant Fork](https://github.com/TheTom/llama-cpp-turboquant) - [llama.cpp TurboQuant Fork](https://github.com/TheTom/llama-cpp-turboquant)

323
scripts/weekly_update.py Normal file
View File

@@ -0,0 +1,323 @@
#!/usr/bin/env python3
"""
TurboQuant Weekly Progress Update Generator
Generates a structured weekly update from:
- Git log (commits since last week)
- Open/closed issues and PRs
- Benchmark results
- Blockers (open issues labeled 'blocker')
Usage:
python3 scripts/weekly_update.py # This week
python3 scripts/weekly_update.py --since 2026-04-08 # Custom range
python3 scripts/weekly_update.py --post # Post as Gitea comment on tracking issue
"""
import argparse
import json
import os
import subprocess
import sys
from datetime import datetime, timedelta
from pathlib import Path
try:
import requests
HAS_REQUESTS = True
except ImportError:
HAS_REQUESTS = False
REPO_ROOT = Path(__file__).resolve().parent.parent
GITEA_URL = "https://forge.alexanderwhitestone.com"
REPO_PATH = "Timmy_Foundation/turboquant"
TRACKING_ISSUE = 76 # This issue
def git_log(since: str, until: str = None) -> list[dict]:
"""Get commits since a date."""
until = until or datetime.now().strftime("%Y-%m-%d")
cmd = [
"git", "-C", str(REPO_ROOT), "log",
f"--since={since}", f"--until={until}",
"--format=%H|%an|%ae|%aI|%s",
"--all"
]
result = subprocess.run(cmd, capture_output=True, text=True)
commits = []
for line in result.stdout.strip().split("\n"):
if not line:
continue
parts = line.split("|", 4)
if len(parts) == 5:
commits.append({
"hash": parts[0][:8],
"author": parts[1],
"email": parts[2],
"date": parts[3][:10],
"subject": parts[4],
})
return commits
def git_diff_stats(since: str) -> dict:
"""Get file change stats."""
cmd = [
"git", "-C", str(REPO_ROOT), "diff",
f"--stat", f"{since}..HEAD"
]
result = subprocess.run(cmd, capture_output=True, text=True)
lines = result.stdout.strip().split("\n")
summary = lines[-1] if lines else "No changes"
return {"summary": summary, "files_changed": len([l for l in lines if "|" in l])}
def find_benchmarks() -> list[dict]:
"""Scan benchmark results directory for recent results."""
bench_dir = REPO_ROOT / "benchmarks"
results = []
if not bench_dir.exists():
return results
for f in bench_dir.glob("*.json"):
try:
data = json.loads(f.read_text())
results.append({"file": f.name, "data": data})
except (json.JSONDecodeError, Exception):
pass
# Also check for markdown reports
for f in bench_dir.glob("*.md"):
if f.name != "README.md":
stat = f.stat()
results.append({
"file": f.name,
"type": "report",
"modified": datetime.fromtimestamp(stat.st_mtime).strftime("%Y-%m-%d"),
"size": stat.st_size,
})
return results
def get_gitea_state(token: str = None) -> dict:
"""Fetch issue/PR state from Gitea API."""
if not HAS_REQUESTS or not token:
return {"available": False}
H = {"Authorization": f"token {token}"}
base = f"{GITEA_URL}/api/v1/repos/{REPO_PATH}"
try:
# Open issues
r = requests.get(f"{base}/issues?state=open&limit=100", headers=H)
open_issues = r.json() if r.status_code == 200 else []
# Closed issues (recent)
r = requests.get(f"{base}/issues?state=closed&limit=50&sort=updated&order=desc", headers=H)
closed_issues = r.json() if r.status_code == 200 else []
# PRs
r = requests.get(f"{base}/pulls?state=open&limit=50", headers=H)
open_prs = r.json() if r.status_code == 200 else []
return {
"available": True,
"open_issues": open_issues,
"closed_issues": closed_issues,
"open_prs": open_prs,
}
except Exception as e:
return {"available": False, "error": str(e)}
def categorize_commits(commits: list[dict]) -> dict:
"""Categorize commits by conventional prefix."""
categories = {
"feat": [], "fix": [], "bench": [], "docs": [],
"test": [], "refactor": [], "chore": [], "other": []
}
for c in commits:
subject = c["subject"].lower()
if subject.startswith("feat") or subject.startswith("feature"):
categories["feat"].append(c)
elif subject.startswith("fix"):
categories["fix"].append(c)
elif subject.startswith("bench") or subject.startswith("perf"):
categories["bench"].append(c)
elif subject.startswith("doc"):
categories["docs"].append(c)
elif subject.startswith("test"):
categories["test"].append(c)
elif subject.startswith("refactor"):
categories["refactor"].append(c)
elif subject.startswith("chore") or subject.startswith("ci"):
categories["chore"].append(c)
else:
categories["other"].append(c)
return {k: v for k, v in categories.items() if v}
def generate_update(since: str, gitea_state: dict = None) -> str:
"""Generate the weekly update markdown."""
now = datetime.now()
until = now.strftime("%Y-%m-%d")
week_label = f"Week of {since} to {until}"
commits = git_log(since, until)
diff_stats = git_diff_stats(since)
categories = categorize_commits(commits)
benchmarks = find_benchmarks()
lines = [
f"## {week_label}",
"",
f"**Generated:** {now.strftime('%Y-%m-%d %H:%M UTC')}",
f"**Commits:** {len(commits)} | **Files changed:** {diff_stats['files_changed']}",
"",
]
# Completed work by category
lines.append("### Completed")
lines.append("")
if commits:
for cat, items in categories.items():
label = {
"feat": "Features", "fix": "Fixes", "bench": "Benchmarks",
"docs": "Documentation", "test": "Tests", "refactor": "Refactoring",
"chore": "Maintenance", "other": "Other"
}.get(cat, cat)
lines.append(f"**{label}:**")
for c in items:
lines.append(f"- `{c['hash']}` {c['subject']} ({c['author']}, {c['date']})")
lines.append("")
else:
lines.append("- No commits this week")
lines.append("")
# Benchmark results
if benchmarks:
lines.append("### Benchmark Results")
lines.append("")
for b in benchmarks:
if b.get("type") == "report":
lines.append(f"- **{b['file']}** (updated {b['modified']}, {b['size']} bytes)")
else:
lines.append(f"- **{b['file']}** — see `benchmarks/{b['file']}`")
lines.append("")
# Gitea state (if available)
if gitea_state and gitea_state.get("available"):
open_issues = gitea_state["open_issues"]
open_prs = gitea_state["open_prs"]
closed = gitea_state["closed_issues"]
lines.append("### In Progress")
lines.append("")
blockers = []
for issue in open_issues:
labels = [l["name"] for l in issue.get("labels", [])]
prefix = ""
if "blocker" in labels:
blockers.append(issue)
prefix = "🚧 BLOCKER — "
assignee = issue.get("assignee", {})
who = assignee.get("login", "unassigned") if assignee else "unassigned"
lines.append(f"- {prefix}#{issue['number']}: {issue['title']} ({who})")
if open_prs:
lines.append("")
lines.append("**Open PRs:**")
for pr in open_prs:
lines.append(f"- #{pr['number']}: {pr['title']} ({pr['user']['login']})")
lines.append("")
# Blockers
if blockers:
lines.append("### Blockers")
lines.append("")
for b in blockers:
lines.append(f"- #{b['number']}: {b['title']}")
if b.get("body"):
snippet = b["body"][:200].replace("\n", " ")
lines.append(f" > {snippet}...")
lines.append("")
# Recently closed
recent_closed = [i for i in closed if i.get("closed_at")]
if recent_closed:
lines.append("### Closed This Period")
lines.append("")
for issue in recent_closed[:10]:
closed_date = issue.get("closed_at", "")[:10]
lines.append(f"- #{issue['number']}: {issue['title']} (closed {closed_date})")
lines.append("")
# Next week
lines.append("### Next Week")
lines.append("")
lines.append("- _TBD — fill in planned work_")
lines.append("")
return "\n".join(lines)
def post_gitea_comment(token: str, body: str, issue: int = TRACKING_ISSUE):
"""Post the update as a comment on the tracking issue."""
if not HAS_REQUESTS:
print("ERROR: requests library not available", file=sys.stderr)
return False
H = {"Authorization": f"token {token}", "Content-Type": "application/json"}
url = f"{GITEA_URL}/api/v1/repos/{REPO_PATH}/issues/{issue}/comments"
r = requests.post(url, headers=H, json={"body": body})
if r.status_code in (200, 201):
print(f"Posted comment on issue #{issue}")
return True
else:
print(f"Failed to post: {r.status_code} {r.text}", file=sys.stderr)
return False
def main():
parser = argparse.ArgumentParser(description="Generate TurboQuant weekly progress update")
parser.add_argument("--since", help="Start date (YYYY-MM-DD), default: 7 days ago")
parser.add_argument("--post", action="store_true", help="Post as Gitea comment on issue #76")
parser.add_argument("--json", action="store_true", help="Output raw data as JSON")
args = parser.parse_args()
since = args.since or (datetime.now() - timedelta(days=7)).strftime("%Y-%m-%d")
# Try to load Gitea token
token = None
token_path = Path.home() / ".config" / "gitea" / "token"
if token_path.exists():
token = token_path.read_text().strip()
gitea_state = get_gitea_state(token) if token else {"available": False}
if args.json:
data = {
"since": since,
"commits": git_log(since),
"benchmarks": find_benchmarks(),
"gitea": {k: v for k, v in gitea_state.items() if k != "available"} if gitea_state.get("available") else None,
}
print(json.dumps(data, indent=2, default=str))
return
update = generate_update(since, gitea_state)
if args.post:
if not token:
print("ERROR: No Gitea token found at ~/.config/gitea/token", file=sys.stderr)
sys.exit(1)
post_gitea_comment(token, update)
else:
print(update)
if __name__ == "__main__":
main()

14
scripts/weekly_update.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# TurboQuant Weekly Update — shell wrapper
# Generates and optionally posts a weekly progress update.
#
# Usage:
# ./scripts/weekly_update.sh # Print to stdout
# ./scripts/weekly_update.sh --post # Post as Gitea comment on #76
# ./scripts/weekly_update.sh --since 2026-04-01 # Custom date range
# ./scripts/weekly_update.sh --json # Raw JSON data
set -euo pipefail
cd "$(dirname "$0")/.."
python3 scripts/weekly_update.py "$@"

104
tests/roundtrip_test.cpp Normal file
View File

@@ -0,0 +1,104 @@
#include "llama-turbo.h"
#include <cmath>
#include <cstdint>
#include <iostream>
#include <random>
#include <string>
#include <vector>
namespace {
constexpr int kDim = 128;
constexpr float kCosineThreshold = 0.99f;
constexpr float kZeroTolerance = 1.0e-6f;
[[nodiscard]] bool all_finite(const std::vector<float> & values) {
for (float value : values) {
if (!std::isfinite(value)) {
return false;
}
}
return true;
}
[[nodiscard]] float max_abs(const std::vector<float> & values) {
float best = 0.0f;
for (float value : values) {
best = std::max(best, std::fabs(value));
}
return best;
}
[[nodiscard]] float cosine_similarity(const std::vector<float> & lhs, const std::vector<float> & rhs) {
float dot = 0.0f;
float lhs_norm = 0.0f;
float rhs_norm = 0.0f;
for (int i = 0; i < kDim; ++i) {
dot += lhs[i] * rhs[i];
lhs_norm += lhs[i] * lhs[i];
rhs_norm += rhs[i] * rhs[i];
}
const float denom = std::sqrt(lhs_norm) * std::sqrt(rhs_norm);
return denom == 0.0f ? 1.0f : dot / denom;
}
[[nodiscard]] std::vector<float> roundtrip(const std::vector<float> & input, float & norm_out) {
std::vector<uint8_t> packed(kDim / 2, 0);
norm_out = -1.0f;
polar_quant_encode_turbo4(input.data(), packed.data(), &norm_out, kDim);
std::vector<float> decoded(kDim, 0.0f);
polar_quant_decode_turbo4(packed.data(), decoded.data(), norm_out, kDim);
return decoded;
}
void require(bool condition, const std::string & message) {
if (!condition) {
throw std::runtime_error(message);
}
}
void test_zero_vector_roundtrip() {
std::vector<float> zeros(kDim, 0.0f);
float norm = -1.0f;
const auto decoded = roundtrip(zeros, norm);
require(norm == 0.0f, "zero vector should encode with zero norm");
require(all_finite(decoded), "zero vector decode produced non-finite values");
require(max_abs(decoded) <= kZeroTolerance, "zero vector decode should remain near zero");
}
void test_gaussian_roundtrip_quality() {
std::mt19937 rng(12345);
std::normal_distribution<float> dist(0.0f, 1.0f);
std::vector<float> input(kDim, 0.0f);
for (float & value : input) {
value = dist(rng);
}
float norm = -1.0f;
const auto decoded = roundtrip(input, norm);
require(norm > 0.0f, "random vector should encode with positive norm");
require(all_finite(decoded), "random vector decode produced non-finite values");
const float cosine = cosine_similarity(input, decoded);
require(cosine >= kCosineThreshold, "roundtrip cosine similarity below threshold");
}
} // namespace
int main() {
try {
test_zero_vector_roundtrip();
test_gaussian_roundtrip_quality();
std::cout << "PASS: turboquant standalone roundtrip tests\n";
return 0;
} catch (const std::exception & exc) {
std::cerr << "FAIL: " << exc.what() << '\n';
return 1;
}
}

View File

@@ -0,0 +1,52 @@
#!/usr/bin/env python3
"""Quick test for weekly_update.py — verifies parsing, output format, and edge cases."""
import subprocess
import sys
import json
from pathlib import Path
SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "weekly_update.py"
def run(args: list[str]) -> str:
result = subprocess.run(
[sys.executable, str(SCRIPT)] + args,
capture_output=True, text=True, cwd=str(SCRIPT.parent.parent)
)
return result.stdout, result.stderr, result.returncode
def test_basic_output():
"""Script runs without error and produces markdown."""
stdout, stderr, rc = run(["--since", "2026-01-01"])
assert rc == 0, f"Exit code {rc}: {stderr}"
assert "## Week of" in stdout, f"Missing header: {stdout[:200]}"
assert "### Completed" in stdout, f"Missing Completed section: {stdout[:200]}"
assert "### Next Week" in stdout, f"Missing Next Week section: {stdout[-200:]}"
print("PASS: basic_output")
def test_json_output():
"""Script outputs valid JSON in --json mode."""
stdout, stderr, rc = run(["--json", "--since", "2026-01-01"])
assert rc == 0, f"Exit code {rc}: {stderr}"
data = json.loads(stdout)
assert "commits" in data
assert "since" in data
print(f"PASS: json_output ({len(data['commits'])} commits)")
def test_no_crash_future_date():
"""Script handles future date gracefully."""
stdout, stderr, rc = run(["--since", "2030-01-01"])
assert rc == 0, f"Exit code {rc}: {stderr}"
print("PASS: future_date_no_crash")
def test_empty_range():
"""Script handles a very old date with no commits."""
stdout, stderr, rc = run(["--since", "2020-01-01", "--since", "2020-01-02"])
assert rc == 0
print("PASS: empty_range")
if __name__ == "__main__":
test_basic_output()
test_json_output()
test_no_crash_future_date()
print("\nAll tests passed.")