Set prompt_cache_key on the Grok and OpenAI Responses adapters - #543
Merged
TheGreatAxios merged 9 commits intoAug 23, 2026
Merged
Conversation
Codex's Responses adapter already keys prompt_cache_key by session id; grok-responses-adapter.ts and openai-responses-adapter.ts never set it, so xAI/OpenCode-Go Responses calls had no cache-routing signal beyond the implicit prefix hash. Threads the session id (parent session's sessionId, or a fresh id per subagent thread) through buildXaiSource/buildGoSource into each adapter's providerOptions, and sets prompt_cache_key in the request body when present. Per xAI's docs (docs.x.ai/developers/advanced-api-usage/prompt-caching), the Responses API honors prompt_cache_key the same way OpenAI does; verified this isn't a no-op before shipping it.
…rompt_cache_key-xai Amp-Thread-ID: https://ampcode.com/threads/T-01a02c68-0d8d-777b-b717-81fb9a282023 Co-authored-by: Amp <amp@ampcode.com>
…rompt_cache_key-xai
Amp-Thread-ID: https://ampcode.com/threads/T-01a02c68-0d8d-777b-b717-81fb9a282023 Co-authored-by: Amp <amp@ampcode.com>
…rompt_cache_key-xai
…rompt_cache_key-xai
…rompt_cache_key-xai
…rompt_cache_key-xai
…rompt_cache_key-xai
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
codex-responses-adapter.tssetsprompt_cache_keyper session (~line 304).grok-responses-adapter.tsandopenai-responses-adapter.tsnever set it, and no session id reached the grok adapter at all. Grok threads were caching at 66-72% vs codex's 92%+, on roughly 340M grok input tokens per 4 days.What the docs say
OpenAI (platform.openai.com/docs/guides/prompt-caching, now redirected to developers.openai.com/api/docs/guides/prompt-caching):
prompt_cache_keyoperates at request-routing granularity: requests are routed to a machine "based onprompt_cache_key, with a hash of the initial prefix of the prompt as a secondary key."prompt_cache_keymust be set for reliable matching.store, so there's no documented interaction withstore:false— the existing codex-adapter comment ("store:false has no other signal") is our own inference (no response-id chaining is possible with store:false, so prompt_cache_key is the only cache-locality lever), not a documented interaction.xAI (docs.x.ai/developers/advanced-api-usage/prompt-caching, how-it-works/best-practices pages) — this was the critical check:
prompt_cache_key— it is not OpenAI-only. Docs state: "Always setx-grok-conv-id(orprompt_cache_keyfor Responses API)" to route requests to the same server and maximize cache hits.prompt_cache_key/x-grok-conv-idonly affects routing consistency, not whether caching happens.cached_tokens— persistent zeros mean the key or message ordering needs checking.prompt_cache_keyon the grok adapter is not a no-op; it's the documented mechanism.What changed
grok-responses-adapter.ts: newGROK_SESSION_ID_OPTIONproviderOptions key;buildRequestsetsbody.prompt_cache_keyfrom it when present.openai-responses-adapter.ts: newOPENAI_SESSION_ID_OPTIONproviderOptions key; same pattern (used by OpenCode Go models on the Responses protocol, e.g. gpt-5.6-luna).config/index.ts:buildXaiSourcenow requiressessionIdand stashes it asGROK_SESSION_ID_OPTION;buildGoSourcetakes an optionalsessionIdand stashes it asOPENAI_SESSION_ID_OPTIONwhen the endpoint speaks Responses.config/inference-sources.ts:buildInferenceSourceForRefthreadsctx.sessionIdinto bothbuildXaiSourceandbuildGoSourcecalls. The parent session gets its own real session id; each subagent thread already gets its own id viabuildSubagentSources's existingsessionId ?? randomUUID()fallback (same pattern codex already used), so every thread's key is stable across turns and distinct from every other thread.exec/runner.ts/tui/runner.ts: the initial xAI source build now passes the livesessionIdthrough, matching the existing codex source build call right next to it.Tests
tests/unit/grok-responses-adapter.test.ts: prompt_cache_key set and stable across twobuildRequestcalls for the same session id; distinct across two different session ids; absent from the body when no session id is present.tests/unit/openai-responses-adapter.test.ts(new): same three cases for the OpenAI Responses adapter.src/config.test.ts:buildXaiSourcenow requiressessionIdin every call site; added a case asserting the session id lands inproviderOptions.grokSessionId.Gates
bunx prettier --check/bunx eslinton touched files: both report only pre-existing issues already present onmainbefore this change (whole-file indentation drift, non-null assertions, type-vs-interface, unused vars) — none on the lines this PR adds. The two provider-adapter files and the new test file are fully clean on both. This is the known repo-wide prettier/eslint gap (seeci-split-prettier-eslint-required-checks), not something reformatted here.bun run typecheck: clean.bun run build: clean.bun test ./src ./tests ./evals: 5121 pass, 1 pre-existing unrelated failure (src/agent/lsp-availability.test.tsfails identically on unmodifiedmainin this worktree — missingtypescript-language-serverbinary, not a code regression).Fixes CL-6904