step35-cli
|
03cf42e29d
|
feat(benchmarks): multi-config comparison suite (refs #29)
Smoke Test / smoke (pull_request) Successful in 11s
Add compare_configs.py — orchestrates running 4 Gemma4 configs in
parallel and produces side-by-side comparison table with aggregated
metrics (TTFT, tokens/sec, latency, peak memory). Picks winner by
highest tokens/sec.
Configurations:
1. Ollama gemma4 (baseline)
2. llama-server gemma4 f16 KV
3. llama-server gemma4 turbo4 KV
4. llama-server gemma4 turbo4 + layer-adaptive
Also adds comprehensive test suite (13 tests) covering ConfigEntry,
aggregation, table building, demo mode, and prompt loading.
Closes #29
Co-authored-by: step35-cli <step35-cli@timmy.foundation>
|
2026-04-26 00:05:24 -04:00 |
|