Skip to content

feat: add Azure AI Foundry provider support (CPD) - #2248

Open
ricofurtado wants to merge 11 commits into
release-cpd-0.1from
azure-ai-foundry-cpd
Open

feat: add Azure AI Foundry provider support (CPD)#2248
ricofurtado wants to merge 11 commits into
release-cpd-0.1from
azure-ai-foundry-cpd

Conversation

@ricofurtado

Copy link
Copy Markdown
Collaborator

What

Adds Azure AI Foundry as a model provider on the CPD release line, for both LLM and embedding roles.

Ported from azure-ai-rebase-main. The two lines diverged well before this feature — CPD forked roughly 234 main commits back — so this is a rebuilt port rather than a cherry-pick: the Azure-specific changes were kept and adapted to CPD's conventions, while unrelated main-side drift that happened to sit in the same hunks was left out.

Included

  • Azure AI Foundry provider config, credential handling, and LiteLLM routing (azure_ai/ prefix, api-version baked into the base URL), plus Azure OpenAI env plumbing.
  • POST /models/azure-ai-foundry with lightweight credential validation and optional real-inference testing.
  • Settings/onboarding acceptance of Azure providers, provider removal with embedding-in-use conflict handling, and the OPENRAG_AZURE_AI_ENABLED feature flag (default on) gating the UI, endpoints, and credential acceptance.
  • Frontend: provider tile, settings dialog/form, onboarding card, logo, and Azure entries in the LLM/embedding selectors.

Adapted to CPD

  • Auth uses get_current_user, not main's require_permission — CPD has no providers:write RBAC scope.
  • Provider removal clears the model to "", matching how the other providers behave on this branch, rather than main's _default_*_model.
  • Kept CPD's plain expressions over main's useMemo/render-phase refactors, and its simpler models-route error handling.

Deliberately excluded: embedding_model_provider

Upstream adds this to the OpenSearch component so the embedding identifier is prefixed with its provider (OpenAI:text-embedding-3-small). That value is what gets written to the embedding_model doc field and the dynamic chunk_embedding_* vector field, so it changes the format of indexed data.

Search resolves models by aggregating embedding_model off the index and matching those values against the embedding object's raw deployment/model/model_id/model_name identifiers, which are never provider-prefixed. Prefixed documents therefore miss the lookup and get skipped, and if every document is prefixed the search raises.

Upstream changes only the write side and has no test coverage for it. Doing this properly needs prefix-aware matching plus a backfill for existing documents (scripts/migrate_embedding_model_field.py), which belongs in its own change. Leaving it out keeps CPD's current behaviour for same-named models across providers unchanged.

Also excluded, as CPD-inapplicable: the Langflow 1.11.2 → 1.11.3 flow re-export and langflowai base-image bump (CPD stays on its own Langflow); the component_index.json / custom_components/ restructure, which does not exist here; and VLM settings, workspace OAuth overrides, RBAC providers:write, the local provider, and main's provider-error sanitizers.

flows/ is untouched by this PR.

Testing

  • 545 unit tests pass, up from 530, with the same 16 pre-existing failures as the untouched branch (all in test_settings_refresh_endpoint.py / unrelated suites — present before this change).
  • New tests/unit/test_azure_ai_foundry.py covers credential validation, routing, the feature flag, and provider removal.
  • Frontend typecheck, lint, and production build clean.
  • docker-compose.yml parses.

Manual verification against a live Azure AI Foundry deployment has not been done — worth a pass before merge.

🤖 Generated with Claude Code

Ports the Azure AI Foundry work from azure-ai-rebase-main onto
release-cpd-0.1. The two lines diverged well before this feature
(CPD forked ~234 main commits back), so this is a rebuilt port
rather than a cherry-pick: Azure-specific changes were kept and
adapted to CPD's conventions, while main-side drift that arrived
in the same hunks was deliberately left out.

Included:
- Azure AI Foundry provider config, credential handling and
  LiteLLM routing (azure_ai/ prefix, api-version baked into the
  base URL), plus Azure OpenAI env plumbing.
- POST /models/azure-ai-foundry endpoint with lightweight
  credential validation and optional real-inference testing.
- Settings/onboarding acceptance of Azure providers, provider
  removal with embedding-in-use conflict handling, and the
  OPENRAG_AZURE_AI_ENABLED feature flag (default on) gating the
  UI, endpoints and credential acceptance.
- Frontend: Azure provider tile, settings dialog/form, onboarding
  card, logo, and Azure entries in the LLM/embedding selectors.

Adapted to CPD:
- Auth uses get_current_user, not require_permission.
- Provider removal clears the model to "" like the other
  providers here, rather than using main's _default_*_model.
- Kept CPD's plain expressions over main's useMemo/render-phase
  refactors, and its simpler models-route error handling.

Deliberately excluded:
- embedding_model_provider. Upstream adds this to the OpenSearch
  component so it prefixes the embedding identifier with the
  provider ("OpenAI:text-embedding-3-small"). That value is what
  gets written to the embedding_model doc field and the dynamic
  chunk_embedding_* vector field, so it changes indexed data
  format. Search resolves models by aggregating embedding_model
  from the index and matching those against the embedding
  object's raw deployment/model/model_id/model_name identifiers,
  which are never provider-prefixed -- so prefixed documents miss
  the lookup and get skipped, and if every document is prefixed
  the search raises. Upstream changes only the write side and has
  no test coverage for it. Doing this properly needs prefix-aware
  matching plus a backfill for existing documents (see
  scripts/migrate_embedding_model_field.py), which belongs in its
  own change. Not porting it leaves CPD's existing behaviour for
  same-named models across providers unchanged.
- Langflow 1.11.2 -> 1.11.3 flow re-export and the langflowai
  base-image bump; CPD stays on its own Langflow.
- component_index.json / custom_components/ restructure, which
  does not exist on this branch.
- VLM settings, workspace OAuth overrides, RBAC providers:write,
  the "local" provider, and main's provider-error sanitizers.

Verified: 545 unit tests pass (up from 530) with the same 16
pre-existing failures as the untouched branch; frontend
typecheck, lint and production build clean.
@github-actions github-actions Bot added frontend 🟨 Issues related to the UI/UX backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) docker tests labels Aug 14, 2026
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 142deb3c-1f4e-47c9-9695-c6b2653ff7a3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the enhancement 🔵 New feature or request label Aug 14, 2026
@github-actions github-actions Bot added enhancement 🔵 New feature or request and removed enhancement 🔵 New feature or request labels Aug 14, 2026
@github-actions github-actions Bot added enhancement 🔵 New feature or request and removed enhancement 🔵 New feature or request labels Aug 14, 2026
Ingestion and search with Langflow disabled always called OpenAI,
regardless of knowledge.embedding_provider: patched_embedding_client was
an alias for patched_async_client, a raw AsyncOpenAI client bound to
api.openai.com. With Langflow enabled the ingest flow calls the provider
itself, so this only surfaced when ingesting with Langflow off, where it
failed against the configured provider's credentials (e.g. an OpenAI 429
for an account with no credits while Azure AI Foundry was selected).

Route that client through litellm.aembedding instead. The resolved model
prefix from get_litellm_model_name (already computed and then discarded)
now actually determines routing.

Azure AI Foundry's OpenAI-compatible endpoint (.../openai/v1) must not use
LiteLLM's azure_ai provider: that handler always inserts the Azure OpenAI
deployment path, producing
.../openai/v1/openai/deployments/<model>/embeddings, which 404s with
"Resource not found". Route it through the plain openai provider with an
explicit api_base so it hits .../openai/v1/embeddings, the route this
endpoint form actually serves. Credentials are passed per call rather than
via env vars so the real OpenAI provider is unaffected.

Also fixes two latent bugs found along the way:

- settings.py read config.providers.azure_openai, a field removed from
  ProvidersConfig. The AttributeError fired inside the credential-loading
  try, silently aborting env export for WatsonX, Ollama and Foundry and
  falling back to provider="openai", which also made the startup HTTP/2
  probe issue a doomed OpenAI request.
- ModelsService.__init__ never sets _config_manager, so its hasattr guard
  was always false and the endpoint-form branch was dead in production
  (tests set the attribute, so it passed). Fall back to the live config.

Verified against a live Foundry endpoint: resolves to
openai/text-embedding-3-small and POSTs .../openai/v1/embeddings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added enhancement 🔵 New feature or request and removed enhancement 🔵 New feature or request labels Aug 15, 2026
@github-actions github-actions Bot added enhancement 🔵 New feature or request and removed enhancement 🔵 New feature or request labels Aug 15, 2026
@github-actions github-actions Bot added enhancement 🔵 New feature or request and removed enhancement 🔵 New feature or request labels Aug 15, 2026
@github-actions github-actions Bot added enhancement 🔵 New feature or request and removed enhancement 🔵 New feature or request labels Aug 16, 2026
@github-actions github-actions Bot removed the enhancement 🔵 New feature or request label Aug 16, 2026
@github-actions github-actions Bot added the enhancement 🔵 New feature or request label Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) docker enhancement 🔵 New feature or request frontend 🟨 Issues related to the UI/UX tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant