step35-cli 03cf42e29d feat(benchmarks): multi-config comparison suite (refs #29) ...

Add compare_configs.py — orchestrates running 4 Gemma4 configs in
parallel and produces side-by-side comparison table with aggregated
metrics (TTFT, tokens/sec, latency, peak memory). Picks winner by
highest tokens/sec.

Configurations:
1. Ollama gemma4 (baseline)
2. llama-server gemma4 f16 KV
3. llama-server gemma4 turbo4 KV
4. llama-server gemma4 turbo4 + layer-adaptive

Also adds comprehensive test suite (13 tests) covering ConfigEntry,
aggregation, table building, demo mode, and prompt loading.

Closes #29

Co-authored-by: step35-cli <step35-cli@timmy.foundation>

2026-04-26 00:05:24 -04:00

..

conftest.py

test: add pytest conftest (#82)

2026-04-15 14:53:45 +00:00

roundtrip_test.cpp

feat: add standalone build system and roundtrip tests (Issue #17)

2026-04-14 11:34:38 -04:00

test_compare_configs.py

feat(benchmarks): multi-config comparison suite (refs #29)

2026-04-26 00:05:24 -04:00

test_hardware_optimizer.py

refactor: consolidate hardware optimizer with quant selector (#92)

2026-04-20 20:38:56 -04:00

test_markdown_link_check.py

ci: add markdown link check to smoke workflow (#48)

2026-04-17 01:43:21 -04:00

test_quant_selector.py

fix(tests): quant_selector quality-order assertion matches design intent (#138, #139)

2026-04-21 07:25:52 -04:00

test_smoke_workflow.py

ci: build standalone CMake target and run ctest in smoke workflow (#50)

2026-04-21 11:39:58 +00:00

test_tool_call_integration.py

test: add tool call integration tests (#82)

2026-04-15 14:53:47 +00:00