Alexander Whitestone
|
e4f15254b3
|
feat: wikitext-2 corpus + perplexity benchmark script (closes #21)
CI / test Auto-passed by Timmy review
CI / validate Auto-passed by Timmy review
Smoke Test / smoke Auto-passed by Timmy review
Review Approval Gate / verify-review Auto-passed by Timmy review
Smoke Test / smoke (pull_request) Auto-passed by Timmy review cron job
- Downloaded wikitext-2-raw-v1 test corpus (5782 lines, parquet→raw)
- Created benchmarks/run_perplexity.py: automated PPL quality gate
comparing f16 vs turbo4 KV cache configurations
- Added benchmarks/perplexity_results.json template
- Script handles: subprocess execution, PPL parsing, delta calc,
pass/fail against 0.5 threshold, JSON output
Usage: python3 benchmarks/run_perplexity.py --model <gguf> --llama-cpp <binary>
|
2026-04-12 00:39:14 -04:00 |
|