Timmy 064ab602dd feat: M1 benchmark suite — throughput, memory, perplexity comparison ...

Implements Issue #80: benchmark turboquant vs llama.cpp baseline on M1.

New files:
- benchmarks/run_m1_benchmark.py — comprehensive benchmark runner
- benchmarks/run_benchmark_m1.sh — shell wrapper for easy execution
- tests/test_m1_benchmark.py — unit tests for benchmark functions

Measures:
- Tokens/sec throughput (f16 vs turbo4, 3-run average)
- Memory usage (RSS monitoring during inference)
- Quality via perplexity (llama-perplexity on wikitext-2)

Generates:
- benchmarks/m1_benchmark_results.json — raw results
- benchmarks/m1_benchmark_report.md — markdown comparison table

Closes #80

2026-04-15 22:10:43 -04:00

..

perplexity_results.json

feat: wikitext-2 corpus + perplexity benchmark script (closes #21)

2026-04-12 00:39:14 -04:00

prompts.json

feat: add standardized benchmarking prompts

2026-03-30 21:14:48 +00:00

run_benchmark_m1.sh

feat: M1 benchmark suite — throughput, memory, perplexity comparison

2026-04-15 22:10:43 -04:00

run_benchmarks.py

feat: multi-backend benchmark suite with TTFT + memory tracking (#37)

2026-04-13 14:05:17 +00:00

run_long_session.py

burn: add long-session quality test (Issue #12) (#39)

2026-04-13 19:59:22 +00:00

run_m1_benchmark.py

feat: M1 benchmark suite — throughput, memory, perplexity comparison

2026-04-15 22:10:43 -04:00

run_perplexity.py

feat: wikitext-2 corpus + perplexity benchmark script (closes #21)

2026-04-12 00:39:14 -04:00

test_prompts.json

Add benchmark test prompts for quality comparison (Issue #22)

2026-03-31 17:31:05 +00:00