feat: add Azure AI Foundry provider support (CPD) - #2248
Open
ricofurtado wants to merge 11 commits into
Open
Conversation
Ports the Azure AI Foundry work from azure-ai-rebase-main onto
release-cpd-0.1. The two lines diverged well before this feature
(CPD forked ~234 main commits back), so this is a rebuilt port
rather than a cherry-pick: Azure-specific changes were kept and
adapted to CPD's conventions, while main-side drift that arrived
in the same hunks was deliberately left out.
Included:
- Azure AI Foundry provider config, credential handling and
LiteLLM routing (azure_ai/ prefix, api-version baked into the
base URL), plus Azure OpenAI env plumbing.
- POST /models/azure-ai-foundry endpoint with lightweight
credential validation and optional real-inference testing.
- Settings/onboarding acceptance of Azure providers, provider
removal with embedding-in-use conflict handling, and the
OPENRAG_AZURE_AI_ENABLED feature flag (default on) gating the
UI, endpoints and credential acceptance.
- Frontend: Azure provider tile, settings dialog/form, onboarding
card, logo, and Azure entries in the LLM/embedding selectors.
Adapted to CPD:
- Auth uses get_current_user, not require_permission.
- Provider removal clears the model to "" like the other
providers here, rather than using main's _default_*_model.
- Kept CPD's plain expressions over main's useMemo/render-phase
refactors, and its simpler models-route error handling.
Deliberately excluded:
- embedding_model_provider. Upstream adds this to the OpenSearch
component so it prefixes the embedding identifier with the
provider ("OpenAI:text-embedding-3-small"). That value is what
gets written to the embedding_model doc field and the dynamic
chunk_embedding_* vector field, so it changes indexed data
format. Search resolves models by aggregating embedding_model
from the index and matching those against the embedding
object's raw deployment/model/model_id/model_name identifiers,
which are never provider-prefixed -- so prefixed documents miss
the lookup and get skipped, and if every document is prefixed
the search raises. Upstream changes only the write side and has
no test coverage for it. Doing this properly needs prefix-aware
matching plus a backfill for existing documents (see
scripts/migrate_embedding_model_field.py), which belongs in its
own change. Not porting it leaves CPD's existing behaviour for
same-named models across providers unchanged.
- Langflow 1.11.2 -> 1.11.3 flow re-export and the langflowai
base-image bump; CPD stays on its own Langflow.
- component_index.json / custom_components/ restructure, which
does not exist on this branch.
- VLM settings, workspace OAuth overrides, RBAC providers:write,
the "local" provider, and main's provider-error sanitizers.
Verified: 545 unit tests pass (up from 530) with the same 16
pre-existing failures as the untouched branch; frontend
typecheck, lint and production build clean.
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Ingestion and search with Langflow disabled always called OpenAI, regardless of knowledge.embedding_provider: patched_embedding_client was an alias for patched_async_client, a raw AsyncOpenAI client bound to api.openai.com. With Langflow enabled the ingest flow calls the provider itself, so this only surfaced when ingesting with Langflow off, where it failed against the configured provider's credentials (e.g. an OpenAI 429 for an account with no credits while Azure AI Foundry was selected). Route that client through litellm.aembedding instead. The resolved model prefix from get_litellm_model_name (already computed and then discarded) now actually determines routing. Azure AI Foundry's OpenAI-compatible endpoint (.../openai/v1) must not use LiteLLM's azure_ai provider: that handler always inserts the Azure OpenAI deployment path, producing .../openai/v1/openai/deployments/<model>/embeddings, which 404s with "Resource not found". Route it through the plain openai provider with an explicit api_base so it hits .../openai/v1/embeddings, the route this endpoint form actually serves. Credentials are passed per call rather than via env vars so the real OpenAI provider is unaffected. Also fixes two latent bugs found along the way: - settings.py read config.providers.azure_openai, a field removed from ProvidersConfig. The AttributeError fired inside the credential-loading try, silently aborting env export for WatsonX, Ollama and Foundry and falling back to provider="openai", which also made the startup HTTP/2 probe issue a doomed OpenAI request. - ModelsService.__init__ never sets _config_manager, so its hasattr guard was always false and the endpoint-form branch was dead in production (tests set the attribute, so it passed). Fall back to the live config. Verified against a live Foundry endpoint: resolves to openai/text-embedding-3-small and POSTs .../openai/v1/embeddings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ion and error handling
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds Azure AI Foundry as a model provider on the CPD release line, for both LLM and embedding roles.
Ported from
azure-ai-rebase-main. The two lines diverged well before this feature — CPD forked roughly 234 main commits back — so this is a rebuilt port rather than a cherry-pick: the Azure-specific changes were kept and adapted to CPD's conventions, while unrelated main-side drift that happened to sit in the same hunks was left out.Included
azure_ai/prefix, api-version baked into the base URL), plus Azure OpenAI env plumbing.POST /models/azure-ai-foundrywith lightweight credential validation and optional real-inference testing.OPENRAG_AZURE_AI_ENABLEDfeature flag (default on) gating the UI, endpoints, and credential acceptance.Adapted to CPD
get_current_user, not main'srequire_permission— CPD has noproviders:writeRBAC scope."", matching how the other providers behave on this branch, rather than main's_default_*_model.useMemo/render-phase refactors, and its simpler models-route error handling.Deliberately excluded:
embedding_model_providerUpstream adds this to the OpenSearch component so the embedding identifier is prefixed with its provider (
OpenAI:text-embedding-3-small). That value is what gets written to theembedding_modeldoc field and the dynamicchunk_embedding_*vector field, so it changes the format of indexed data.Search resolves models by aggregating
embedding_modeloff the index and matching those values against the embedding object's rawdeployment/model/model_id/model_nameidentifiers, which are never provider-prefixed. Prefixed documents therefore miss the lookup and get skipped, and if every document is prefixed the search raises.Upstream changes only the write side and has no test coverage for it. Doing this properly needs prefix-aware matching plus a backfill for existing documents (
scripts/migrate_embedding_model_field.py), which belongs in its own change. Leaving it out keeps CPD's current behaviour for same-named models across providers unchanged.Also excluded, as CPD-inapplicable: the Langflow 1.11.2 → 1.11.3 flow re-export and
langflowaibase-image bump (CPD stays on its own Langflow); thecomponent_index.json/custom_components/restructure, which does not exist here; and VLM settings, workspace OAuth overrides, RBACproviders:write, thelocalprovider, and main's provider-error sanitizers.flows/is untouched by this PR.Testing
test_settings_refresh_endpoint.py/ unrelated suites — present before this change).tests/unit/test_azure_ai_foundry.pycovers credential validation, routing, the feature flag, and provider removal.docker-compose.ymlparses.Manual verification against a live Azure AI Foundry deployment has not been done — worth a pass before merge.
🤖 Generated with Claude Code