feat: OpenAI-compatible tool calling - #6
Open
maxff77 wants to merge 1 commit into
Open
Conversation
`tools` is destructured in both chat handlers and then ignored, and Qwen Web exposes no native function-calling API, so every OpenAI-protocol agent harness gets prose back instead of `tool_calls` and the agent loop cannot run. Elicit calls by injecting Qwen's own Hermes `<tool_call>` prompt format, then sieve them back into OpenAI-shaped `tool_calls`. - One grammar definition shared by the streaming and buffered paths, so they cannot drift - Incremental hold-back sieve: normal text streams live, only a tail that could prefix a marker is withheld; markers inside code fences and inline spans never fire, so the model can document the format - Round-trip of `assistant.tool_calls` and `role:"tool"` messages, which is what closes multi-turn agent loops - `tool_choice` auto / none / required / named function - Parallel calls, with the call id carried through the prompt so results map back - Qwen Web runs its own tool layer that emits `Tool NAME does not exists.` when it cannot resolve our names; that upstream noise is stripped rather than passed off as an answer - SSE keepalive while a call is being captured, tunable with SSE_KEEPALIVE_MS, since nothing is written to the socket during capture and proxies cut idle connections Verified against the live endpoint: 20/20 calls across streaming and non-streaming probes, full round trips through an agent harness, no raw markers ever reaching the client. 50 assert-based checks in scripts/test-tool-calls.js, plus a CI workflow. Every sieve case runs at every chunk size, streaming compared against buffered, which is how two chunk-boundary bugs were caught during development. No new dependencies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Someone is attempting to deploy a commit to the smanx's projects Team on Vercel. A member of the Team first needs to authorize it. |
❌ Deploy Preview for qwen2api failed. Why did it fail? →
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
toolsis destructured in both chat handlers and then ignored, and Qwen Web exposes no native function-calling API. Every OpenAI-protocol agent harness (Cline, Roo, OpenClaw, …) gets prose back instead oftool_calls, so the agent loop cannot run at all.This elicits calls by injecting Qwen's own Hermes
<tool_call>prompt format, then sieves them back into OpenAI-shapedtool_calls. The injected syntax is internal and never reaches the client.What it does
assistant.tool_callsandrole:"tool"messages — this is what closes multi-turn loopstool_choiceauto/none/required/ named functionSSE_KEEPALIVE_MS, default 15s) — nothing is written to the socket while a call is being captured, and proxies cut idle connectionsOne upstream quirk worth knowing
Qwen Web runs its own tool layer. It tries to resolve your tool names against its registry, fails, and injects
Tool NAME does not exists.into the output stream ahead of the model's real<tool_call>. That prose was reaching clients as message content. The sieve now strips it.Verification
Against the live endpoint: 20/20 calls across streaming and non-streaming probes, full round trips driven by a real agent harness, no raw markers ever reaching the client.
50 assert-based checks in
scripts/test-tool-calls.js, plus a CI workflow. No new dependencies. Every sieve case runs at every chunk size with streaming compared against buffered — that is how two chunk-boundary bugs were caught during development.Known limitations, stated plainly
Prompt injection gives no guarantee a native function-calling API would. The model can simply not follow the format; measured hit rate was high but it is not 100%. Failures degrade safely: no call, and no raw markers leak.
Tool names and arguments are not validated against your schemas — that is left to the caller's executor.
tool_choice: "required"is a prompt-level instruction, not enforced with a 422.Overlaps with #5 on one line of
package.json(thetestscript). If you merge #5 first I will rebase this to run both suites.