Skip to content

fix(ai): distinguish automatic prompt-cache lifetimes - #835

Draft
Altairpaca wants to merge 3 commits into
code-yeongyu:mainfrom
Altairpaca:fix/831-deepseek-cache-lifetime
Draft

fix(ai): distinguish automatic prompt-cache lifetimes#835
Altairpaca wants to merge 3 commits into
code-yeongyu:mainfrom
Altairpaca:fix/831-deepseek-cache-lifetime

Conversation

@Altairpaca

@Altairpaca Altairpaca commented Aug 12, 2026

Copy link
Copy Markdown

Summary

Direct DeepSeek API sessions are currently treated as if they had a deterministic 5-minute prompt-cache TTL. That propagates into Goal monitor scheduling (a continuation every 270s — 5m minus the default 30s safety buffer, or the 240s fallback) and into TUI copy claiming the wake stays inside a "5m prompt-cache TTL" and keeps tokens warm with estimated savings.

DeepSeek's official context cache is automatic and best-effort: enabled by default for all users, no client-visible TTL contract, and unused entries are cleared only after hours to days (api-docs.deepseek.com/guides/kv_cache). This PR fixes the cache-lifetime classification rather than tuning DeepSeek-specific numbers.

Root cause

resolvePromptCacheTtlSeconds() (packages/ai/src/utils/prompt-cache-ttl.ts) returned 300s for every openai-completions model without Anthropic-style cache control — including direct DeepSeek. PR #767 then consumed that budget: resolvePromptCacheSafeWaitSeconds() derived 270s, and the Goal monitor scheduled its continuation there. Returning undefined alone would not fix it: Goal maps unknown budgets to the legacy 240s fallback, which is still a cache-preservation framing.

Behavior before/after

Before (direct DeepSeek, Goal with a live wake source):

  • TTL estimate: 300s (fabricated 5m TTL)
  • Goal monitor wake: every 4m 30s (or the 240s fallback)
  • TUI: "Continuation expected ready ... - the timed wake stays inside the 5m prompt-cache TTL." / "~120K tokens kept warm · est. $0.324 saved vs a cold re-read"

After:

  • New browser-safe PromptCacheLifetime semantic: fixed(ttlSeconds) | automatic | disabled | unknown, resolved by resolvePromptCacheLifetime(); resolvePromptCacheTtlSeconds() becomes a backwards-compatible wrapper (public signature unchanged, every non-DeepSeek result identical).
  • Direct built-in DeepSeek classifies as automatic — no fixed TTL (300 or 3600) is ever reported, including under PI_CACHE_RETENTION=long.
  • Goal schedules automatic-cache lanes at the configured liveness backstop (promptCache.goalBackstopMaxSeconds, default 3570s → 59m30s wake) instead of the 270s/240s cache-preservation wakes.
  • Cache-warm entries for automatic lanes carry cacheLifetime: "automatic", omit ttlSeconds/estimatedSavedUsd, and render neutrally: "Continuation expected ready ... - provider caching is automatic; the timed wake only keeps the goal alive." / "~120K tokens cached after the prior turn".
  • Fixed-TTL lanes (Anthropic 300/3600, Bedrock, OpenRouter cache-control, other openai-completions) keep their exact existing scheduling, metrics, and rendering.

Rebuilt on latest main (2026-08-14)

This PR was force-pushed with a fresh implementation based on current main (0f651bd4a, 387 commits past the previous base c67dd6906), not a mechanical rebase:

  • Upstream renamed the cache-warm renderer copy from "Continuation deferred X" to "Continuation expected ready (X)"; the automatic branch is written against the new copy.
  • Checked the intervening 387 commits for a provider cache-lifetime capability/type abstraction to reuse: none exists (prompt-cache-ttl.ts only gained chatTemplateArgs/supportsThinkingTokenBudget compat fields). The discriminated-union design is the minimal shape, and it reuses the existing detectOpenAICompletionsCompat DeepSeek detection convention (provider === "deepseek" || baseUrl.includes("deepseek.com")).
  • prompt_cache_retention: "24h" behavior with cacheRetention: "long" on DeepSeek is a separate finding, deliberately NOT fixed here (own follow-up).

RED -> GREEN

All captures under local-ignore/qa-evidence/20260814-issue-831-deepseek-cache-lifetime-rebuild/:

  • RED red-01-ai-prompt-cache-lifetime.txt: 6 failing new cases — resolvePromptCacheLifetime is not a function (API contract absent)
  • RED red-02-prompt-cache-budget.txt: expected undefined to be 270 — DeepSeek had a 270s budget
  • RED red-03-issue-831-regression.txt: 4 failing — 240s fallback delay; "~120K tokens kept warm" / "stayed warm" copy
  • GREEN green-01..04: ai lifetime+TTL 38/38, ai adjacent cache suites 58 passed/12 skipped (1 pre-existing live E2E billing failure, see below), goal/budget suites 72/72, cache-keepalive 7/7

Verification

  • npm run check — Biome + pinned-deps + ts-imports + shrinkwrap + install-lock + claude-sdk-platform-lock + tsc + browser-smoke, exit 0
  • npm test — workspace suites pass; the only failure is a pre-existing release-tooling script test (scripts/prepare-senpi-bundled-workspaces.test.mjs "issue 446 ...", npm pack --dry-run output parsing on npm 12) that fails identically on pristine main — reproduced and recorded
  • senpi-qa channels: mock-loop self-test 48/48, cli-smoke self-test 8/8, rpc-drive state + mock openai-completions turn (marker delivered), tui-smoke 5/5 — evidence in the same directory
  • git diff --check clean; package CHANGELOG.md untouched (maintainer-owned per CONTRIBUTING.md); required changes.md entries added (packages/ai/src, coding-agent core, goal)

Scope/non-goals

  • Classification change is limited to direct built-in DeepSeek; every other lane keeps its previous conservative classification (remaining openai-completions lanes stay 300s).
  • The cache-keepalive extension is untouched (Anthropic-first-party-only, out of scope).
  • Separate finding, deliberately NOT fixed here: with cacheRetention: "long", openai-completions buildParams may send prompt_cache_key and prompt_cache_retention: "24h" to DeepSeek (its compat allows long retention). Default short retention sends neither field. This deserves its own focused follow-up.

Evidence

local-ignore/qa-evidence/20260814-issue-831-deepseek-cache-lifetime-rebuild/ — RED and GREEN vitest captures, npm check/test receipts, senpi-qa channel outputs. No credentials or environment dumps are included.

Fixes #831

Maintainer note (CI)

  • Fork PR: the CI / Changelog gate / Publish Model Catalog workflows triggered by the force-push are awaiting first-time-contributor approval before they execute.
  • The Changelog gate will fail until this PR gets the no-changelog label or maintainer-added CHANGELOG entries: this PR intentionally does not edit CHANGELOG.md (CONTRIBUTING.md: "Do not edit CHANGELOG.md. Changelog entries are added by maintainers."), and changes.md entries are provided at the fork contract.

@Altairpaca
Altairpaca force-pushed the fix/831-deepseek-cache-lifetime branch from 6a7bb25 to 0b0432a Compare August 14, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(prompt-cache): direct DeepSeek is treated as a fixed 5m TTL, forcing unnecessary 4m30 Goal cache-warm wakes

1 participant