Conversation
Draft the full-flow-agentic Responses gateway so a single pipeline adapts to whichever inference upstream the load balancer selects, using the `selected_upstream` filter condition from praxis-proxy/praxis#1241. Inside the IRR inference step the protocol-adaptive body builders now run AFTER the step's unconditional load_balancer (praxis#1241 rejects a selected_upstream-gated filter unless a load balancer is guaranteed to run before it on every reachable path; each IRR step is validated as its own isolated linear pipeline): - openai_responses_proxy when application_protocol=openai_responses - responses_to_chat_completions when application_protocol=openai_chat_completions - path_rewrite when application_protocol=openai_chat_completions The inference cluster declares http.application_protocol/application_provider (default native openai_responses/openai, commented openai_chat_completions/vllm variant); flip that block to switch protocols without editing the pipeline. selected_upstream fails closed, so exactly one body builder fires per request. openai_stream_events stays first and unconditional, with limits pinned at or above the converter's bounds so no client-visible frame is dropped from the store. Header comments (canonical order, step 11, a new protocol-adaptive note, and the direct-OpenAI variant) are updated to stay truthful. WIP: praxis-proxy/praxis#1241 is not yet merged, so until the pinned praxis-core rev in the root Cargo.toml includes it the selected_upstream condition is unknown to the parser and CI will fail. Draft only. Signed-off-by: Sébastien Han <seb@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Draft the
full-flow-agenticResponses gateway so one pipeline adapts to whichever inference upstream the load balancer selects, using the newselected_upstreamfilter condition from praxis-proxy/praxis#1241 (matches the selected cluster's typedhttp.application_protocol/http.application_provider).Only
examples/configs/openai/responses/full-flow-agentic.yamlchanges.Tracking: praxis-proxy/praxis#1141 — this PR is the Praxis AI functional example the issue calls for. Note it is cross-repo, so merging this PR will link but not auto-close #1141; that issue must be closed manually once the remaining acceptance criteria (integration test, generated-doc sync, streaming/agentic coverage) also land.
How
Inside the IRR
inferencestep, the protocol-adaptive body builders now run after the step's unconditionalload_balancer, each gated on the selected upstream's protocol:openai_responses_proxywhen application_protocol: openai_responses(native)responses_to_chat_completionswhen application_protocol: openai_chat_completions(vLLM/chat)path_rewrite(/v1/responses→/v1/chat/completions)when application_protocol: openai_chat_completions(+ POST path)http: { application_protocol: openai_responses, application_provider: openai }by default, with a commentedopenai_chat_completions/vllmvariant — flip that block to switch the whole gateway's protocol without touching the pipeline.selected_upstreamfails closed, so exactly one body builder fires per request (declare the protocol or neither runs).openai_stream_eventsstays first and unconditional, with limits pinned at/above the converter's bounds so no client-visible frame is dropped from the store.Why this ordering
praxis#1241 rejects a
selected_upstream-gated filter unless an unconditional load balancer is guaranteed to run before it on every reachable path. Each IRR step is validated as its own isolated linear pipeline, and the step'sload_balancermust live inside the step — so the gated builders go after it. At runtime the body builders'on_request_bodyruns after upstream selection and before dispatch, the correct window to read the selected cluster's protocol; keepingopenai_stream_eventsfirst means the Chat→Responses translator runs before it on the response phase (which it requires).Status — DRAFT / WIP
praxis-proxy/praxis#1241 is not yet merged. Until the pinned
praxis-corerev in the rootCargo.tomlis bumped to include it, theselected_upstreamcondition is unknown to the config parser and CI will fail. This PR is intentionally forward-looking and should land only after the praxis bump.