Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions packages/ai/src/changes.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,32 @@
# AI Source Changes

## 2026-08-14 - Distinguish automatic prompt-cache lifetimes (DeepSeek)

### What changed and why

- New browser-safe `PromptCacheLifetime` semantic and `resolvePromptCacheLifetime()`:
`fixed(ttlSeconds) | automatic | disabled | unknown`. `resolvePromptCacheTtlSeconds()` is now a
backwards-compatible wrapper over it, returning the TTL only for `fixed` lifetimes.
- Direct DeepSeek (`provider: "deepseek"` or a `deepseek.com` base URL on the `openai-completions`
lane) classifies as `automatic`: DeepSeek's context cache is enabled by default, best-effort, and
exposes no client-visible deterministic TTL (api-docs.deepseek.com/guides/kv_cache). It no longer
reports a fabricated 300s TTL, and `PI_CACHE_RETENTION=long` no longer fabricates one either.
- Every other lane keeps its exact previous classification (Anthropic 300/3600, Bedrock 300/3600,
OpenRouter cache-control 300/3600, conservative 300 for the remaining `openai-completions` lanes,
Responses lanes 300, unknown lanes `undefined`).

### Why this cannot be expressed externally

- The classification lives in the browser-safe TTL resolver that cache-aware tool waits and Goal
monitor scheduling consume. Extensions observe only higher-level requests and cannot rewrite the
provider-agnostic cache-lifetime contract consumed by runtime scheduling.

### Expected merge conflict zones

- MEDIUM: `utils/prompt-cache-ttl.ts` around `resolvePromptCacheTtlSeconds()` / the new
`resolvePromptCacheLifetime()`.
- LOW: `index.ts` root export block.

## 2026-08-13 - Preserve explicit request compatibility fields

### What changed and why
Expand Down
2 changes: 2 additions & 0 deletions packages/ai/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,12 @@ export * from "./utils/event-stream.ts";
export * from "./utils/json-parse.ts";
export { extractOpenAiCodexAccountId } from "./utils/openai-codex-auth.ts";
export * from "./utils/overflow.ts";
export type { PromptCacheLifetime } from "./utils/prompt-cache-ttl.ts";
export {
isAnthropicApiBaseUrl,
PROMPT_CACHE_TTL_LONG_SECONDS,
PROMPT_CACHE_TTL_SHORT_SECONDS,
resolvePromptCacheLifetime,
resolvePromptCacheTtlSeconds,
} from "./utils/prompt-cache-ttl.ts";
export * from "./utils/retry.ts";
Expand Down
56 changes: 42 additions & 14 deletions packages/ai/src/utils/prompt-cache-ttl.ts
Original file line number Diff line number Diff line change
Expand Up @@ -332,48 +332,76 @@ function resolveOpenAIResponsesCacheRetention(cacheRetention?: CacheRetention, e
return "short";
}

export function resolvePromptCacheTtlSeconds(model: Model<Api>, env?: ProviderEnv): number | undefined {
export type PromptCacheLifetime =
| { readonly kind: "fixed"; readonly ttlSeconds: number }
| { readonly kind: "automatic" }
| { readonly kind: "disabled" }
| { readonly kind: "unknown" };

function isDeepSeekOpenAICompletionsModel(model: Model<"openai-completions">): boolean {
return model.provider === "deepseek" || model.baseUrl.includes("deepseek.com");
}

/**
* Classify a model's prompt-cache lifetime semantics.
*
* `fixed` carries the deterministic client-visible TTL used by cache-aware
* scheduling; `automatic` means the provider caches best-effort without a
* client-visible TTL contract (direct DeepSeek); `disabled` means caching is
* turned off; `unknown` means the lane has no known cache contract.
*/
export function resolvePromptCacheLifetime(model: Model<Api>, env?: ProviderEnv): PromptCacheLifetime {
switch (model.api) {
case "claude-sdk-oauth":
// The Claude SDK owns prompt caching for this lane and uses Anthropic's default 5m TTL.
return PROMPT_CACHE_TTL_SHORT_SECONDS;
return { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_SHORT_SECONDS };
case "anthropic-messages": {
const anthropicModel = model as Model<"anthropic-messages">;
const retention = resolveAnthropicCacheRetention(anthropicModel.cacheRetention, env, "short");
if (retention === "none") return undefined;
if (retention === "none") return { kind: "disabled" };
return retention === "long" &&
isAnthropicApiBaseUrl(anthropicModel.baseUrl) &&
getAnthropicCompat(anthropicModel).supportsLongCacheRetention
? PROMPT_CACHE_TTL_LONG_SECONDS
: PROMPT_CACHE_TTL_SHORT_SECONDS;
? { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_LONG_SECONDS }
: { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_SHORT_SECONDS };
}
case "bedrock-converse-stream": {
const bedrockModel = model as Model<"bedrock-converse-stream">;
const retention = resolveBedrockCacheRetention(bedrockModel.cacheRetention, env);
if (retention === "none" || !supportsPromptCaching(bedrockModel, env)) return undefined;
if (retention === "none" || !supportsPromptCaching(bedrockModel, env)) return { kind: "disabled" };
return retention === "long" && supportsOneHourCacheTtl(bedrockModel)
? PROMPT_CACHE_TTL_LONG_SECONDS
: PROMPT_CACHE_TTL_SHORT_SECONDS;
? { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_LONG_SECONDS }
: { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_SHORT_SECONDS };
}
case "openai-completions": {
const completionsModel = model as Model<"openai-completions">;
const retention = resolveOpenAICompletionsCacheRetention(completionsModel.cacheRetention, env);
if (retention === "none") return undefined;
if (retention === "none") return { kind: "disabled" };
// Direct DeepSeek caches automatically and best-effort with no client-visible TTL
// (api-docs.deepseek.com/guides/kv_cache), so it must not get a fabricated 5m TTL.
if (isDeepSeekOpenAICompletionsModel(completionsModel)) return { kind: "automatic" };
const compat = getOpenAICompletionsCompat(completionsModel);
if (compat.cacheControlFormat === "anthropic") {
return retention === "long" && compat.supportsLongCacheRetention
? PROMPT_CACHE_TTL_LONG_SECONDS
: PROMPT_CACHE_TTL_SHORT_SECONDS;
? { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_LONG_SECONDS }
: { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_SHORT_SECONDS };
}
return PROMPT_CACHE_TTL_SHORT_SECONDS;
return { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_SHORT_SECONDS };
}
case "openai-responses":
case "openai-codex-responses":
case "azure-openai-responses": {
const retention = resolveOpenAIResponsesCacheRetention(model.cacheRetention, env);
return retention === "none" ? undefined : PROMPT_CACHE_TTL_SHORT_SECONDS;
return retention === "none"
? { kind: "disabled" }
: { kind: "fixed", ttlSeconds: PROMPT_CACHE_TTL_SHORT_SECONDS };
}
default:
return undefined;
return { kind: "unknown" };
}
}

export function resolvePromptCacheTtlSeconds(model: Model<Api>, env?: ProviderEnv): number | undefined {
const lifetime = resolvePromptCacheLifetime(model, env);
return lifetime.kind === "fixed" ? lifetime.ttlSeconds : undefined;
}
110 changes: 110 additions & 0 deletions packages/ai/test/prompt-cache-lifetime.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { type Api, type Model, resolvePromptCacheLifetime, resolvePromptCacheTtlSeconds } from "../src/index.ts";

function createModel<TApi extends Api>(api: TApi, overrides: Partial<Model<TApi>> = {}): Model<TApi> {
return {
id: "test-model",
name: "Test Model",
api,
provider: "test-provider",
baseUrl: "https://example.com/v1",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 4096,
...overrides,
} as Model<TApi>;
}

const anthropicModel = createModel("anthropic-messages", {
provider: "anthropic",
baseUrl: "https://api.anthropic.com/v1",
});

const anthropicCompletionsModel = createModel("openai-completions", {
provider: "custom-proxy",
compat: {
cacheControlFormat: "anthropic",
supportsLongCacheRetention: true,
},
});

const cacheableBedrockModel = createModel("bedrock-converse-stream", {
id: "anthropic.claude-3-7-sonnet-20250219-v1:0",
provider: "amazon-bedrock",
});

const openAIResponsesModel = createModel("openai-responses", {
provider: "openai",
baseUrl: "https://api.openai.com/v1",
});

const deepseekModel = createModel("openai-completions", {
provider: "deepseek",
baseUrl: "https://api.deepseek.com",
});

const originalCacheRetention = process.env.PI_CACHE_RETENTION;

beforeEach(() => {
delete process.env.PI_CACHE_RETENTION;
});

afterEach(() => {
if (originalCacheRetention === undefined) {
delete process.env.PI_CACHE_RETENTION;
} else {
process.env.PI_CACHE_RETENTION = originalCacheRetention;
}
});

describe("DeepSeek automatic cache lifetime (issue #831)", () => {
it("classifies direct DeepSeek as automatic, not a fixed 5m TTL", () => {
expect(resolvePromptCacheLifetime(deepseekModel)).toEqual({ kind: "automatic" });
});

it("detects DeepSeek through a deepseek.com base URL", () => {
const urlModel = createModel("openai-completions", {
provider: "custom-proxy",
baseUrl: "https://api.deepseek.com/v1",
});
expect(resolvePromptCacheLifetime(urlModel)).toEqual({ kind: "automatic" });
});

it("reports no fixed TTL for direct DeepSeek via the legacy wrapper", () => {
expect(resolvePromptCacheTtlSeconds(deepseekModel)).toBeUndefined();
});

it("keeps long retention from fabricating a fixed TTL for DeepSeek", () => {
expect(resolvePromptCacheLifetime(deepseekModel, { PI_CACHE_RETENTION: "long" })).toEqual({
kind: "automatic",
});
expect(resolvePromptCacheTtlSeconds(deepseekModel, { PI_CACHE_RETENTION: "long" })).toBeUndefined();
});
});

describe("prompt-cache lifetime classification", () => {
it("maps every other lane exactly as the legacy TTL resolver did", () => {
expect(resolvePromptCacheLifetime(anthropicModel)).toEqual({ kind: "fixed", ttlSeconds: 300 });
expect(resolvePromptCacheLifetime({ ...anthropicModel, cacheRetention: "long" })).toEqual({
kind: "fixed",
ttlSeconds: 3600,
});
expect(resolvePromptCacheLifetime({ ...anthropicModel, cacheRetention: "none" })).toEqual({ kind: "disabled" });
expect(resolvePromptCacheLifetime(anthropicCompletionsModel)).toEqual({ kind: "fixed", ttlSeconds: 300 });
expect(resolvePromptCacheLifetime(cacheableBedrockModel)).toEqual({ kind: "fixed", ttlSeconds: 300 });
expect(resolvePromptCacheLifetime(openAIResponsesModel)).toEqual({ kind: "fixed", ttlSeconds: 300 });
expect(resolvePromptCacheLifetime(createModel("openai-completions", { cacheRetention: "none" }))).toEqual({
kind: "disabled",
});
expect(resolvePromptCacheLifetime(createModel("google-generative-ai"))).toEqual({ kind: "unknown" });
});

it("keeps the legacy wrapper identical to the fixed lifetime", () => {
const lifetime = resolvePromptCacheLifetime(anthropicModel);
expect(resolvePromptCacheTtlSeconds(anthropicModel)).toBe(
lifetime.kind === "fixed" ? lifetime.ttlSeconds : undefined,
);
});
});
2 changes: 1 addition & 1 deletion packages/coding-agent/docs/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ If your Claude Pro/Max subscription usage through `claude-sdk-oauth` feels unexp
| Anthropic-compatible providers (kimi-coding, fireworks, gateways) | 5 minutes | The 1h TTL is gated on the native `api.anthropic.com` base URL, so these lanes stay short. | `cacheRetention` |

Override precedence: `cacheRetention` in `models.json` / the model catalog wins over everything. `PI_CACHE_RETENTION=long` selects long; any other set value forces short; unset falls back to the lane default above.
5. **Goal-monitor timing.** The goal monitor's continuation backstop is derived from the model's cache-safe wait (TTL minus `promptCache.safetyBufferSeconds`, default 30), capped by `promptCache.goalBackstopMaxSeconds` (default 3570), instead of a fixed 4 minutes. The default 5m lanes wake every ~4m30s; a supported lane explicitly configured for 1h retention can wait up to 59m30s. Cache-warm notices show which warm iteration you are on.
5. **Goal-monitor timing.** The goal monitor's continuation backstop is derived from the model's cache-safe wait (TTL minus `promptCache.safetyBufferSeconds`, default 30), capped by `promptCache.goalBackstopMaxSeconds` (default 3570), instead of a fixed 4 minutes. The default 5m lanes wake every ~4m30s; a supported lane explicitly configured for 1h retention can wait up to 59m30s. Automatic-cache lanes (e.g. direct DeepSeek) have no client-visible TTL, so they skip the TTL-derived wake entirely and schedule their liveness wake at `promptCache.goalBackstopMaxSeconds` instead. Cache-warm notices show which warm iteration you are on.
6. **Wake sources that hold the goal backstop.** Anything that can wake a parked session publishes a `wake_source_state` event (`{source, activeCount}`): terminal monitors (`terminal-monitors`), background bash sessions including auto-detached and killed ones (`terminal-background-sessions`), detached `eval` cells (`senpi-codemode`), and omo-senpi background task children plus owned team members (`senpi-task`). The goal extension sums every source, so a goal waits inside the prompt-cache TTL while ANY of them is on duty instead of continuing immediately. The legacy `terminal_monitor_state` event is still emitted for external consumers and is folded onto the same `terminal-monitors` count.
7. **Directive-block deduplication.** As of v2026.8.4, the flatten serialization collapses repeated `<ultrawork-mode>` directive blocks to a single copy, preventing the issue-#494 scenario where duplicated ~17KB directive blocks consumed up to 73% of the re-sent prompt. The continuity observation reports how many were collapsed and the payload size.

Expand Down
4 changes: 3 additions & 1 deletion packages/coding-agent/docs/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,12 +329,14 @@ When unset, senpi leaves provider payloads unchanged. This setting currently app

Sizes how long foreground tools may block on the active model's prompt-cache lifetime, so a long
`bash` call never straddles cache expiry and forces a full re-read. When the model's cache TTL is
unknown (e.g. Google models) or caching is off, no budget applies and timeout behavior is unchanged.
unknown (e.g. Google models), caching is off, or the provider caches automatically without a
client-visible TTL (e.g. direct DeepSeek), no budget applies and timeout behavior is unchanged.

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `promptCache.cacheAwareTimeouts` | boolean | `true` | Cap foreground tool waits at the model's prompt-cache TTL minus the safety buffer; `false` restores the fixed legacy ceilings |
| `promptCache.safetyBufferSeconds` | number | `30` | Headroom subtracted from the cache TTL (a 5m TTL yields a 270s ceiling). If it consumes the whole TTL, no budget applies |
| `promptCache.goalBackstopMaxSeconds` | number | `3570` | Maximum Goal monitor continuation backstop. Automatic-cache lanes (e.g. direct DeepSeek) schedule their liveness wake at this value instead of a TTL-derived delay |

A foreground `bash` command still running at the budget is handed to a live background session
instead of being killed; its explicit `timeout` remains the kill deadline. See
Expand Down
19 changes: 19 additions & 0 deletions packages/coding-agent/src/core/changes.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,24 @@
# changes

## Automatic prompt-cache lifetimes yield no cache-derived budget (2026-08-14)

### What changed

- `core/prompt-cache-budget.ts` now derives no budget for automatic-cache providers (direct
DeepSeek): `resolvePromptCacheSafeWaitSeconds()` returns `undefined` because pi-ai's TTL wrapper
reports no fixed TTL for `automatic` lifetimes. Cache-aware foreground tool waits fall back to
their legacy ceilings, and the advisory `PI_PROMPT_CACHE_SAFE_WAIT_SECONDS` env mirror is cleared.

### Why

- Providers like DeepSeek cache automatically and best-effort with no client-visible TTL, so a
TTL-derived wait budget would be fabricated. The resolver contract stays "fixed-TTL waits only"
(issue #831).

### Expected merge conflict zones

- LOW: `core/prompt-cache-budget.ts` docs/behavior note and its budget test suite.

## Admit provider-owned compaction lanes (2026-08-14)

### What changed
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ function whyLine(data: GoalCacheWarmupEntryData): string {
switch (data.phase) {
case "scheduled": {
const expected = `Continuation expected ${formatExpectedWake(data.dueAtMs, data.delayMs)}`;
if (data.cache?.cacheLifetime === "automatic") {
return `${expected} - provider caching is automatic; the timed wake only keeps the goal alive.`;
}
if (data.cache?.ttlSeconds === undefined) {
return `${expected} - the monitor wakes the goal the moment decisive output lands.`;
}
Expand Down Expand Up @@ -69,6 +72,9 @@ function warmLine(data: GoalCacheWarmupEntryData): string | undefined {
const cache = data.cache;
if (cache === undefined || cache.cachedTokens <= 0) return undefined;
const tokens = `~${formatWarmTokenCount(cache.cachedTokens)} tokens`;
if (cache.cacheLifetime === "automatic") {
return `${tokens} cached after the prior turn`;
}
const ttlMayHaveElapsed =
cache.ttlSeconds !== undefined && (data.waitedMs ?? data.delayMs) >= cache.ttlSeconds * 1000;
if (ttlMayHaveElapsed) {
Expand Down
Loading