Skip to content

fix(openai): treat explicit null tool_choice as absent/default in Responses translation - #1338

Merged
shaneutt merged 5 commits into
praxis-proxy:mainfrom
Artemon-line:1266-responses-null-tool-choice
Sep 25, 2026
Merged

shaneutt merged 5 commits into
praxis-proxy:mainfrom
Artemon-line:1266-responses-null-tool-choice

Conversation

@Artemon-line

@Artemon-line Artemon-line commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1266

Summary

The Responses-to-Chat translator previously rejected requests containing an explicit "tool_choice": null with error 400 Bad Request: unsupported Responses tool_choice type for Chat Completions translation: null.

In OpenAI's Responses API, sending "tool_choice": null behaves identically to omitting the tool_choice field (defaulting to "auto" when tools are present).

This change updates build_chat_tool_choice to treat Value::Null as Ok(None), omitting tool_choice from the translated Chat Completions request and allowing requests with "tool_choice": null to succeed.

Testing Instructions (Real OpenAI)

  1. Set your OpenAI API key in your environment:

    export OPENAI_API_KEY="sk-..."
  2. Run the proxy with an OpenAI-backed configuration:

    cargo run -p praxis-ai-proxy --features full,store-sqlite -- --config <config.yaml>
  3. Send a request with explicit tool_choice: null:

    curl -i http://localhost:8080/v1/responses \
      -H "Content-Type: application/json" \
      -d '{
        "model": "gpt-4o-mini",
        "input": "Hello",
        "tools": [
          {
            "type": "function",
            "name": "get_weather",
            "parameters": {"type": "object", "properties": {}}
          }
        ],
        "tool_choice": null
      }'

Expected Results:

  • Returns an HTTP/1.1 200 OK response from OpenAI.
  • The JSON response body echoes "tool_choice": null and returns a completed assistant response item.
  • Requests with explicit tool_choice: null no longer return 400 Bad Request: unsupported Responses tool_choice type for Chat Completions translation: null.

@Artemon-line
Artemon-line requested review from a team and rhdedgar September 23, 2026 17:18
@Artemon-line
Artemon-line force-pushed the 1266-responses-null-tool-choice branch from 0cd62b1 to 7e171b7 Compare September 23, 2026 17:20
@Artemon-line

Copy link
Copy Markdown
Contributor Author

@leseb it's green, pls take a look 👍

@leseb leseb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Null leaks into synthesized Responses output
File: chat_completions.rs:1258
Explicit null is omitted upstream, but remains Null in ResponsesState. Buffered and streaming response builders then emit "tool_choice": null instead of "auto".
This violates the OpenResponses response schema, which requires a string/object tool choice.
Normalize the response-side value and test both finite and SSE output.
[MAJOR] Required inference-fixture coverage is missing
File: test_openai_responses_vllm.py:6738
Repository instructions require transformation changes to update the inference scenario, recording, coverage manifest, and generated README.
Existing tests only verify success/status, so they miss the invalid echoed value.

@Artemon-line

Copy link
Copy Markdown
Contributor Author

[MAJOR] Null leaks into synthesized Responses output File: chat_completions.rs:1258 Explicit null is omitted upstream, but remains Null in ResponsesState. Buffered and streaming response builders then emit "tool_choice": null instead of "auto". This violates the OpenResponses response schema, which requires a string/object tool choice. Normalize the response-side value and test both finite and SSE output. [MAJOR] Required inference-fixture coverage is missing File: test_openai_responses_vllm.py:6738 Repository instructions require transformation changes to update the inference scenario, recording, coverage manifest, and generated README. Existing tests only verify success/status, so they miss the invalid echoed value.

Addressed in 2fcb5ac

@leseb

leseb commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

@Artemon-line ResponsesState normalizes null to "auto", but openai_client_tool_compat snapshots the raw null and later removes tool_choice from responses containing rich tools such as custom or shell. That still violates the required response schema in the composed finite/streaming pipeline. I left an inline Conductor comment at state.rs:926.

@Artemon-line

Artemon-line commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor Author

Good catch @leseb! Addressed in 9dc9e8b -- restore_snapshot_tools now normalizes null snapshot tool_choice to "auto" instead of removing tool_choice from the response.

@leseb
leseb force-pushed the 1266-responses-null-tool-choice branch from 5e9e70b to a5c1e8d Compare September 25, 2026 15:40
Artemon-line and others added 5 commits September 25, 2026 17:40
…ponses translation

Signed-off-by: Artemy <ahladenk@redhat.com>
Signed-off-by: Artemy <ahladenk@redhat.com>
…add inference scenario

- Filter out null tool_choice when constructing ResponseContext and ResponsesState so response builders emit "tool_choice": "auto" instead of null.
- Add inference scenario responses/chat-null-tool-choice and synthetic fixture.
- Update SDK/raw HTTP tests to assert tool_choice='auto' in response resources.

Signed-off-by: Artemy <ahladenk@redhat.com>
… restoration

- Update restore_snapshot_tools to insert tool_choice='auto' when the snapshot tool_choice is null, rather than removing tool_choice from the response resource.
- Update unit tests and synthetic client-tool-compat fixture recordings to expect tool_choice='auto'.

Signed-off-by: Artemy <ahladenk@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
@shaneutt shaneutt removed this from the v0.4.0 milestone Sep 25, 2026
@shaneutt shaneutt added this to the v0.5.0 milestone Sep 25, 2026
@leseb
leseb force-pushed the 1266-responses-null-tool-choice branch from a5c1e8d to 2b39f1b Compare September 25, 2026 15:45
@shaneutt
shaneutt merged commit 7c00920 into praxis-proxy:main Sep 25, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

Responses-to-Chat rejects explicit null tool_choice accepted by OpenAI

3 participants