feat: wikitext-2 corpus + perplexity benchmark script (closes #21)

Browse Source

- Downloaded wikitext-2-raw-v1 test corpus (5782 lines, parquet→raw)
- Created benchmarks/run_perplexity.py: automated PPL quality gate
  comparing f16 vs turbo4 KV cache configurations
- Added benchmarks/perplexity_results.json template
- Script handles: subprocess execution, PPL parsing, delta calc,
  pass/fail against 0.5 threshold, JSON output

Usage: python3 benchmarks/run_perplexity.py --model <gguf> --llama-cpp <binary>

...

This commit is contained in:

Alexander Whitestone

2026-04-12 00:39:14 -04:00

parent 4c926312df

commit e4f15254b3

3 changed files with 5979 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

5782

corpora/wiki.test.raw Normal file

File diff suppressed because it is too large Load Diff