Skip to content

fix: disable automatic cache affinity for Azure OpenAI - #3521

Open
koriyoshi2041 wants to merge 1 commit into
vectorize-io:mainfrom
koriyoshi2041:fix/azure-cache-affinity
Open

fix: disable automatic cache affinity for Azure OpenAI#3521
koriyoshi2041 wants to merge 1 commit into
vectorize-io:mainfrom
koriyoshi2041:fix/azure-cache-affinity

Conversation

@koriyoshi2041

Copy link
Copy Markdown
Contributor

Problem

With the openai provider and an *.openai.azure.com base URL, automatic cache affinity currently adds OpenAI's prompt_cache_key field. Azure Foundry rejects that field with unrecognized_request_argument, breaking LLM-backed operations.

Fixes #3518.

Fix

Remove Azure OpenAI hosts from the automatic prompt_cache_key allowlist. Native OpenAI and openai.com hosts keep the existing behavior, while Azure hosts now resolve auto to none.

The configuration and Azure setup docs now describe the same behavior.

Test

  • uv run pytest tests/test_cache_affinity.py -q — 67 passed
  • Ruff check and format check on the changed Python files — passed
  • uv run ty check hindsight_api/engine/cache_affinity.py hindsight_api/config.py — passed
  • git diff --check — passed

The repository-wide lint script reached the TypeScript lint step but could not load @eslint/js in this fresh worktree. No TypeScript files are changed by this PR.

Risk

Low. This only narrows the auto host allowlist. Operators who explicitly select openai_prompt_cache_key retain that override, and non-Azure host behavior is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

azure-openai base_url triggers prompt_cache_key, returns 400 on Azure Foundry

1 participant