fix(openai): treat explicit null tool_choice as absent/default in Responses translation - #1338
Conversation
0cd62b1 to
7e171b7
Compare
|
@leseb it's green, pls take a look 👍 |
leseb
left a comment
There was a problem hiding this comment.
[MAJOR] Null leaks into synthesized Responses output
File: chat_completions.rs:1258
Explicit null is omitted upstream, but remains Null in ResponsesState. Buffered and streaming response builders then emit "tool_choice": null instead of "auto".
This violates the OpenResponses response schema, which requires a string/object tool choice.
Normalize the response-side value and test both finite and SSE output.
[MAJOR] Required inference-fixture coverage is missing
File: test_openai_responses_vllm.py:6738
Repository instructions require transformation changes to update the inference scenario, recording, coverage manifest, and generated README.
Existing tests only verify success/status, so they miss the invalid echoed value.
Addressed in 2fcb5ac |
|
@Artemon-line ResponsesState normalizes null to "auto", but openai_client_tool_compat snapshots the raw null and later removes tool_choice from responses containing rich tools such as custom or shell. That still violates the required response schema in the composed finite/streaming pipeline. I left an inline Conductor comment at state.rs:926. |
5e9e70b to
a5c1e8d
Compare
…ponses translation Signed-off-by: Artemy <ahladenk@redhat.com>
Signed-off-by: Artemy <ahladenk@redhat.com>
…add inference scenario - Filter out null tool_choice when constructing ResponseContext and ResponsesState so response builders emit "tool_choice": "auto" instead of null. - Add inference scenario responses/chat-null-tool-choice and synthetic fixture. - Update SDK/raw HTTP tests to assert tool_choice='auto' in response resources. Signed-off-by: Artemy <ahladenk@redhat.com>
… restoration - Update restore_snapshot_tools to insert tool_choice='auto' when the snapshot tool_choice is null, rather than removing tool_choice from the response resource. - Update unit tests and synthetic client-tool-compat fixture recordings to expect tool_choice='auto'. Signed-off-by: Artemy <ahladenk@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
a5c1e8d to
2b39f1b
Compare
Fixes #1266
Summary
The Responses-to-Chat translator previously rejected requests containing an explicit
"tool_choice": nullwith error400 Bad Request: unsupported Responses tool_choice type for Chat Completions translation: null.In OpenAI's Responses API, sending
"tool_choice": nullbehaves identically to omitting thetool_choicefield (defaulting to"auto"when tools are present).This change updates
build_chat_tool_choiceto treatValue::NullasOk(None), omittingtool_choicefrom the translated Chat Completions request and allowing requests with"tool_choice": nullto succeed.Testing Instructions (Real OpenAI)
Set your OpenAI API key in your environment:
Run the proxy with an OpenAI-backed configuration:
Send a request with explicit
tool_choice: null:Expected Results:
HTTP/1.1 200 OKresponse from OpenAI."tool_choice": nulland returns a completed assistant response item.tool_choice: nullno longer return400 Bad Request: unsupported Responses tool_choice type for Chat Completions translation: null.