Alexander Whitestone 27ebfa3525 Fix #11: Full test matrix — 10 prompts + quality + performance ...

Test matrix runner (benchmarks/run_test_matrix.py) implementing all
acceptance criteria from #11:

Quality Tests:
- 10 practical prompts with expected-pattern matching
- Perplexity proxy (WikiText-2 chunks)
- Needle-in-Haystack at 8K/16K/32K contexts
- Multi-turn context retention (prompt #7)

Performance Tests:
- tok/s at 4K/8K/16K context
- TTFT proxy measurement
- Peak memory (macOS/Linux)
- Context ceiling binary search

Outputs:
- JSON: reports/test-matrix-YYYY-MM-DD.json
- Markdown: reports/test-matrix-YYYY-MM-DD.md
- Go/No-Go assessment with issue list

Smoke test: 10/10 quality, 3/3 needle-in-haystack on qwen2.5:7b.

Refs: Timmy_Foundation/turboquant#11

2026-04-14 22:10:39 -04:00

..

__pycache__

Fix #11: Full test matrix — 10 prompts + quality + performance

2026-04-14 22:10:39 -04:00

perplexity_results.json

feat: wikitext-2 corpus + perplexity benchmark script (closes #21)

2026-04-12 00:39:14 -04:00

prompts.json

feat: add standardized benchmarking prompts

2026-03-30 21:14:48 +00:00

run_benchmarks.py

feat: multi-backend benchmark suite with TTFT + memory tracking (#37)

2026-04-13 14:05:17 +00:00

run_long_session.py

burn: add long-session quality test (Issue #12) (#39)

2026-04-13 19:59:22 +00:00

run_perplexity.py

feat: wikitext-2 corpus + perplexity benchmark script (closes #21)

2026-04-12 00:39:14 -04:00

run_test_matrix.py

Fix #11: Full test matrix — 10 prompts + quality + performance

2026-04-14 22:10:39 -04:00

test_prompts.json

Add benchmark test prompts for quality comparison (Issue #22)

2026-03-31 17:31:05 +00:00