Skip to content

Record: provider prompt cache never matches the DM packet by design (checkpoint-only cache, ceiling ~5% of a turn) #421

Description

@MoonlightByte

Record only. Owner ruling 2026-09-16: the provider prompt cache will not match under the game's design and we should not chase it. This issue exists so the measurement and the reason are on file, not as work to do.

What was measured (real OpenAI, gpt-5.6-luna, headless, WSL, 2026-09-16)

With PR #418 in place the capture layer now reports cached_tokens per call. Two ordinary player turns on the restored Keep of Doom save:

Call Turn 2 prompt tokens Turn 2 cached
T067 main DM 27.4K 0
T065 validator 29.3K 6.7K (its leading system run)
T105 companion voices (6 calls) 5.5K each 0 (each under the 1024-token minimum)

Six controlled replays of the recorded requests established the provider's rule: a hit requires an exact match up to one of two checkpoints, (A) the end of the leading run of system messages, or (B) the end of the last assistant message. There is no longest-prefix matching: an identical 9K-token prefix inside a longer leading run cached 0, while the exact same request replayed cached 24.9K of 26.7K.

Why the DM packet never matches

Checkpoint (B) cannot match in live play because every turn adds one exchange. Checkpoint (A) requires the leading system run to be byte-identical between turns, and by design it is not: the companion context (fresh recall every turn), the plot status ("current situation" line moves with the story) and the character/NPC sheets (any HP or effect change) all live in that run. This is the game's design, not a defect: dynamic state is meant to reach the DM every turn.

Ceiling, for the record

Replaying two real turns with only the truly static blocks leading (DM prompt, world state, chronicle, location) and every volatile block moved behind the conversation cached 8.1K of 27.7K on turn 2, about 29% of the DM call and roughly 5% of a turn's total tokens. It would need the packet assembled in that order at request-build time (main.py plus the compression integration), and the branch perf/dm-packet-cache-order (the simpler one-block move) changed nothing and is left unpushed.

Not to be confused with the game's own compression layer (T084 request-time derivative with its rolling cache, character-sheet and location compressors), which is a separate mechanism and was not part of this measurement.

Evidence: session captures capture-turns-before/after, probes cache_probe3..6.py (session scratchpad, not preserved).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions