Add Gemini provider tools#1578
Conversation
🦋 Changeset detectedLatest commit: b9cb3a7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 31 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if (geminiTools !== undefined) { | ||
| providerTools.push(geminiTools); | ||
| } | ||
|
|
||
| if (toolCtx) { | ||
| for (const tool of toolCtx.providerTools) { | ||
| if (tool instanceof GeminiTool) { | ||
| providerTools.push(tool.toToolConfig()); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (tools.length > 0 && providerTools.length > 0) { | ||
| throw new Error('Gemini does not support mixing function tools and provider tools'); | ||
| } |
There was a problem hiding this comment.
🔴 Regression: geminiTools combined with function tools now throws instead of merging into a single Tool object
The old code in buildConnectConfig merged geminiTools (e.g., {googleSearch: {}}) and functionDeclarations into a single Tool object via the spread operator: [{ functionDeclarations: [...], ...this.options.geminiTools }]. The new toToolsConfig function treats geminiTools as a separate "provider tool" (plugins/google/src/utils.ts:192-193) and then throws an error at plugins/google/src/utils.ts:204-206 if both function declarations and provider tools are present. This breaks any existing user who configures both geminiTools (e.g., {googleSearch: {}}) and function tools in a realtime session — a combination that worked before and is valid in the Google API (a single Tool object can contain both functionDeclarations and other tool types like googleSearch). The fix should merge geminiTools into the function declarations Tool object (matching the old behavior) rather than pushing it into the separate providerTools array.
Prompt for agents
The toToolsConfig function in plugins/google/src/utils.ts incorrectly treats the geminiTools option as a separate provider tool, causing a runtime error when users combine geminiTools (e.g. {googleSearch: {}}) with function tools. The old behavior in plugins/google/src/beta/realtime/realtime_api.ts merged geminiTools into the same Tool object as functionDeclarations using spread: [{functionDeclarations: [...], ...geminiTools}].
The fix: instead of pushing geminiTools into the providerTools array (line 192-194), merge it into the function declarations Tool object when function declarations exist, or push it as a standalone Tool when no function declarations are present. This maintains backward compatibility while still throwing when new GeminiTool provider tool instances (from toolCtx.providerTools) are mixed with function tools.
Approach: When building the function declarations Tool object (lines 177-189), spread geminiTools into it (similar to the old realtime code). Remove the geminiTools push from providerTools (line 192-194). If there are no function declarations but geminiTools is defined, push it as a standalone Tool object in the tools array.
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
Summary
Stacked on #1576.
Test plan
Note: pnpm --filter @livekit/agents-plugin-google api:check currently fails on this branch with API Extractor rejecting the existing export-star-as-beta declaration syntax in dist/index.d.ts.