fix(pricing): update DeepSeek V4 card and add a peak/off-peak axis - #905
Merged
Conversation
`_TIER_DEEPSEEK_PRO` and `_TIER_DEEPSEEK_FLASH` held DeepSeek's pre-2026-08-16 rate card, which was itself a time-limited promotion. The card that replaced it is published as peak / off-peak, and `get_pricing` had no axis that could reach it: `input_tokens` is prompt size, and `service_tier` comes from the response (DeepSeek declares none, so it always resolved to "standard"). Peak/off-peak is a function of the wall-clock time the request was sent, not of anything in it. So this adds the third axis rather than picking one side of the schedule and documenting the gap. `get_pricing` and `compute_cost` take `request_time` (POSIX epoch seconds, `None` = now), and a new public `is_deepseek_peak` owns the schedule: 01:00-04:00 and 06:00-10:00 UTC, Monday through Friday, windows half-open so 03:59:59 is peak and 04:00:00 is not. Defaulting to "now" is correct for the live path without touching it — `cost_tracker.record_api_usage` prices a response the moment it arrives. Rates, USD per 1M tokens, checked 2026-08-25 against api-docs.deepseek.com/quick_start/pricing/ (peak is exactly 2x): deepseek-v4-flash 0.22 in / 0.66 out / 0.007 cache read deepseek-v4-pro 0.66 in / 1.98 out / 0.022 cache read `cache_read` is what drives the size of the error: at ~96% of agentic input tokens, 0.003625 against a real 0.022 understated a deepseek-v4-pro session 2.3x off-peak and 4.5x peak — $0.0237 per 1M tokens reported against a real $0.0536, and $0.1073 inside a peak window. Also here: - `CostTracker.get_cache_savings` prices each event at `event.timestamp` rather than at display time, so opening /cost after 01:00 UTC no longer restates an off-peak session's savings at peak rates. - A `# checked YYYY-MM-DD` line on the DeepSeek block, per the issue's suggestion — it is the row most likely to move again. - The README's DeepSeek headline quoted `~$0.0435` / 1M cache-hit input, derived from the same stale card (and never matching the table's own 0.003625). Now $0.022 off-peak, with the peak window stated. - `deepseek-v4-flash-vision-exp` is deliberately left unregistered: it shares the flash card on the vendor's page but has no row in models/configs.py and is unreachable through the provider. Tests: 52 new in tests/test_deepseek_peak_pricing.py covering the schedule (window boundaries, the Mon/Fri fencepost, weekends inside the windows, a 168-hour sweep pinning 35 peak hours), the published card as absolutes on both sides, and the scope gate that no other model is time-tiered at any hour of the week. The existing DeepSeek and status-bar pricing assertions now pin an explicit instant instead of depending on when CI runs. Closes #904 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The default-to-now test only asserted `is_deepseek_peak()` agreed with `is_deepseek_peak(time.time())` — self-consistent, and equally true of a default that had stopped reaching the clock entirely. Since every production call site relies on that default, a regression there would park all of them on one card with nothing failing. Now fakes the clock in the pricing module's own namespace (not `time.time` itself, which pytest also reads) and pins the default through `is_deepseek_peak`, `get_pricing` and `compute_cost` on both sides of the schedule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Test Results 5 files 997 suites 29m 20s ⏱️ For more details on these failures, see this check. Results for commit a1c01ac. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #904.
The problem
_TIER_DEEPSEEK_PROand_TIER_DEEPSEEK_FLASHheld DeepSeek's pre-2026-08-16 rate card — which, as @xyzs996 points out, was itself a time-limited promotion. The card that replaced it is published as peak / off-peak, andget_pricinghad no axis that could reach it:input_tokensis prompt size. DeepSeek's tier does not depend on prompt size.service_tiercomes from the response. DeepSeek declares none, so it always resolved to"standard"— and peak/off-peak is not something a caller or provider declares per request anyway.The modelling around it was right and survives unchanged: a cache miss is still just input, so
cache_creationstill mirrorsinput.The fix: the third axis, not a documented gap
The issue offered a contained alternative — put the off-peak card in the constants and say so in a comment. I went with the timestamp axis instead. Off-peak is 133 of every 168 hours, not 143 as the issue estimates (the windows are 3h + 4h on each of 5 weekdays = 35 peak hours), so the single-card version would still be wrong ~21% of the time, by exactly 2x, on the axis that is cheapest to just model.
get_pricingandcompute_costnow takerequest_time(POSIX epoch seconds,None= now), and a new publicis_deepseek_peakowns the schedule: 01:00–04:00 and 06:00–10:00 UTC, Monday through Friday, windows half-open so 03:59:59 is peak and 04:00:00 is not.Defaulting to "now" is what keeps this from touching any call site: the live path prices a response the moment it arrives, so
record_api_usageis already correct without a change. Callers holding a real timestamp should pass it — one now does (below).Rates
USD per 1M tokens, checked 2026-08-25 against https://api-docs.deepseek.com/quick_start/pricing/. Peak is exactly 2x off-peak.
input/cache_creationoutputcache_readdeepseek-v4-flashoff-peakdeepseek-v4-flashpeakdeepseek-v4-prooff-peakdeepseek-v4-propeakcache_readdrives the size of the error, as the issue says: at ~96% of agentic input tokens,0.003625against a real0.022understated adeepseek-v4-prosession 2.3x off-peak and 4.5x peak — $0.0237 per 1M tokens reported against a real $0.0536, or $0.1073 inside a peak window.Worth noting for anyone touching these rows later: pro is exactly 3x flash on input and output, but not on
cache_read(0.022 against 0.007 is 22/7). That near-ratio invites deriving one row from the other; there is a test pinning that it does not hold.Also in here
CostTracker.get_cache_savingsprices each event atevent.timestampinstead of at display time. It runs when/costis opened, arbitrarily later than the requests it sums, so at "now" it would restate an off-peak session's savings at peak rates the moment the clock crossed 01:00 UTC.# checked 2026-08-25line on the DeepSeek block, per the issue's suggestion. It is the row most likely to move again, and a promo card rotting into silent under-reporting is the same failure thegpt-5.6-lunarow above it was written about.~$0.0435/ 1M cache-hit input — derived from the same stale card, and never matching the pricing table's own0.003625either. Now$0.022off-peak, with the peak window and its ~227x stated underneath.deepseek-v4-flash-vision-expis deliberately left unregistered. It shares the flash card on the vendor's page, but it has no row inmodels/configs.pyand is unreachable through the provider (supports_vision=False), so pricing it would be a row nothing can exercise.Aggregate-replay paths, stated rather than fixed
cost_restore.build_cost_blockandeval/harbor/advisor_cost.pyrecompute from an already-aggregatedmodel_usageblock, so they have no per-request time to pass. They inherit exactly the inexactness they already carry for the context tier (gpt-5.6-lunadoubling above 272K), and both are commented to that effect. A session spanning a peak boundary gets priced entirely on one side of it; fixing that needs per-request cost records, not a different default incompute_cost. In practiceadvisor_cost.pynever reaches it for DeepSeek — that path prefers the billed figure wherevercost_usd > 0, which for a metered provider is always.Tests
tests/test_deepseek_peak_pricing.py, 52 new tests in three groups:weekday() >= 5test and a1 <= weekday() <= 5test differ), weekends inside the windows, UTC-not-local, and a 168-hour sweep pinning 35 peak hours so a window cannot silently widen.cache_creation == inputon both, peak exactly 2x. Absolutes rather than ratios on purpose: the values this PR replaces were internally consistent (right ratios, mirroredcache_creation) and still 3x low, so only an external number catches that class of bug.compute_costdoubling in a peak window on the issue's measured agent token mix, its $0.0536 / $0.1073 figures pinned as dollars, the vendor-prefix strip carrying the timestamp,get_cache_savingshonouring event timestamps, and a scope gate that every other model returns an identical card at all 168 hours of the week.The existing DeepSeek and status-bar pricing assertions now pin an explicit instant rather than depending on when CI happens to run.
Full suite: 10,402 passed, 17 skipped. The one failure locally is
TestBuildAppSmoke::test_pip_install_editable, a known dirty-working-tree timeout unrelated to pricing.🤖 Generated with Claude Code