Timmy
1fd3aca367
The document was written to the worktree on 2026-10-20 and never committed, so the ledger
carried docs/deepseek-cost-routing.md as NOT_ADDED (54961d4) and --fail-on-missing returned
rc 0 at a head where the path resolved in no ref. A live deliverable of a present lane is not
a row to tolerate: it is a file to commit. This commits it, appended with a verification
section that grades what holds and what does not -- the arithmetic reproduces from the quoted
fields (Documented), the DeepSeek direct rates reproduce from three independent secondary
sources (Corroborated), the primary rate card is Unrecovered because no fetch backend here can
read it, and the OpenRouter billed field is one live response never re-fetched (Testimony
only). The routing conclusion is unchanged and implies no new gate.
Volt's ledger row is not edited here; the tool now reports it CLEARED, which is his to retire.
4.9 KiB
4.9 KiB
DeepSeek v4-pro: direct vs OpenRouter — measurement + routing plan
Date: 2026-10-20. Live A/B, same prompt both sides (measure-v4pro-cost.py).
Measured (one representative reasoning task)
| DeepSeek direct | OpenRouter | |
|---|---|---|
| prompt tokens | 198 | 119 |
| completion tokens | 17,181 | 6,644 |
| reasoning tokens | 16,008 | 5,738 |
| actual billed | (computed) | $0.0100731 |
OpenRouter's own cost field → effective rates:
- output: $0.009966 / 6,644 = ~$1.50/M (NOT the $0.75 headline)
- input: $0.0001071 / 119 = ~$0.90/M (NOT the $0.25 headline)
Per-token comparison (our workload is ~90% output/reasoning)
| scheme | in/M | out/M |
|---|---|---|
| DS direct off-peak | $0.66 | $1.98 |
| DS direct peak | $1.32 | $3.96 |
| OpenRouter (measured) | ~$0.90 | ~$1.50 |
| OpenRouter (headline, misleading) | $0.25 | $0.75 |
Conclusions
- OpenRouter's headline price is misleading — the real billed rate is ~$0.90/$1.50, not $0.25/$0.75.
- Per-token: OpenRouter beats DeepSeek on OUTPUT (~24% cheaper off-peak, ~62% cheaper peak), but LOSES on INPUT (~36% dearer than off-peak).
- For our output-heavy workload, OpenRouter is a net win mainly at PEAK (dodges the 2× surcharge).
- YELLOW FLAG: OpenRouter emitted 2.6× fewer reasoning tokens (5,738 vs 16,008) for the identical prompt — possibly shallower reasoning (or stochastic). Quality unverified.
Plan
- Quality gate (blocking): 3-task benchmark, both routes, judged on answer quality. If OpenRouter answers are ≥ our bar, proceed; else drop the idea.
- Peak-routing: route peak-hours v4-pro through OpenRouter (flat ~$1.50/M out vs $3.96/M peak); keep off-peak direct (our $0.66 input is cheaper, and $1.98 off-peak output is close enough to not justify the quality risk).
- Free-tier flash remains primary for wizard work (opencode deepseek-v4.1-flash-free + free models) — the routing only concerns v4-pro (main agent + dreams).
- Track OR spend in the existing balance watch if we route there.
Decision gate owner: Timmy. Approver: Alexander.
Nous Portal check (2026-10-20)
- Flash discount (real): DeepSeek V4.1 Flash 1017 is 77% off → $0.07 in / $0.50 out (vs our direct $0.15/$0.60 off-peak, $0.30/$1.20 peak). Plus a partner promo: "V4 Flash 0731 90% off for 7 days."
- No Pro deal: Nous Portal V4 Pro 0813 = $0.26 in / $8.00 out — output is ~4× our direct off-peak ($1.98). Worse for reasoning-heavy work.
- No Nous key configured in env (would need a subscription to use the flash promo).
Revised conclusion (post quality gate)
- Quality: comparable. OpenRouter's v4-pro answered all 3 tasks correctly and completely; the earlier 2.6× reasoning-token drop was run-to-run variance (on the planning task OR emitted more tokens than DS).
- Cost: a wash. OR was cheaper on 2 tasks, ~2.5× dearer on 1 (it emitted more tokens). No reliable off-peak saving.
- V4 Pro: direct DeepSeek off-peak remains the best real deal. OpenRouter's only edge is dodging the 2× peak surcharge — but we already moved heavy v4-pro work off-peak, so little peak usage remains to capture. Not worth auto-flipping the main agent's provider (risk > reward).
- Flash: Nous Portal (77% off) / Deep Infra ($0.06) are cheaper, but we already run flash work on FREE models — keep as fallback only.
- Captured wins remain: off-peak scheduling + free wizard models + free opencode flash route.
Verification status (appended 2026-10-24 by Timmy, control plane)
This file stayed untracked until today — it was written to the worktree on 2026-10-20 and
never committed, so no reader of the repository could open the document those rates came
from. It is committed now, and the ledger row that tolerated it
(tools/path_claim_baseline.tsv, class NOT_ADDED, added by Volt at 54961d4) now reads
CLEARED.
- Arithmetic: Documented. The billed field is the sum of its two parts ($0.009966 + $0.0001071 = $0.0100731 exactly), and both effective rates reproduce from it: 0.009966 / 6,644 = $1.5000/M output, 0.0001071 / 119 = $0.9000/M input. Every figure in the two tables above follows from the quoted fields.
- DeepSeek direct rates: Corroborated. $0.66 input / $1.98 output off-peak and $1.32 /
$3.96 peak, with peak at 01:00-04:00 and 06:00-10:00 UTC Monday-Friday, reproduced
2026-10-24 from three independent secondary sources (mercatus-ai.com, deepseekprice.com,
cabina.ai). The primary rate card is Unrecovered: the web-extract backend on this host
is search-only, so
api-docs.deepseek.com/quick_start/pricingitself was never fetched. - OpenRouter billed field: Testimony only. One live API response on 2026-10-20, not re-fetched since; the $1.50/$0.90 effective rates are derived from that response, not from any published OpenRouter rate sheet.
- The quality gate above ran on 2026-10-20 and was not re-run. The conclusion it produced is unchanged: direct off-peak for v4-pro, free flash for wizard work, no auto-flip.