feat(runtime): exa_search FunctionTool — make the tool Flash hallucinates real#145
Merged
Conversation
…ates real 30 days of audit logs show Exa itself never failed (8/8 API calls healthy); every 'exa breaks' incident was agent-side: 1. The skill catalog advertises exa-search in every system prompt but no exa_search tool existed — Flash pattern-matched the name to a native tool and emitted calls the runner had no handler for, killing the whole session with exit 1 (3 crashes: 2026-07-09 x2, 2026-07-13; the last one misdiagnosed itself to the user). Sam journaled the root cause on 07-09 and re-hit it on 07-13 — prose can't patch a model prior, so the tool now exists with the exact name and query arg Flash already tries to call. 2. The skill's only curl template piped to jq, which isn't in the container — searches succeeded, got billed, and the output was eaten (2 occurrences). The tool formats results in Python; the rewritten skill bans jq by name and documents the python3 fallback. 3. Skill body drifted from the current Exa API (stale type enum, response fields promised unconditionally) — corrected against the 2026-07 docs. Registered in worker_tools (main/worker/pro_executor) and the parallel_workers fanout; mentor stays read-only. Live-verified against the real API: happy path + include_domains + 401-as-string (~$0.02). Diagnosis by a 4-agent research workflow, 2026-07-14 maintenance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The finding
30 days of audit logs: Exa's API never failed once (8/8 calls healthy). Every "exa breaks" incident was agent-side, in two modes:
exa-searchin every system prompt, but noexa_searchtool was registered — Flash pattern-matched the name and emittedexa_search(...)calls the ADK runner had no handler for → unhandled exception → whole session dies with exit 1, no reply to the user (Jul 9 ×2, Jul 13 ×1 — the last one misdiagnosed itself to Cesare as a "document retrieval parsing error"). Sam journaled the root cause on Jul 9 and re-hit it Jul 13: prose can't patch a model prior.jq, which isn't installed in the container — searches succeeded, got billed, output vanished.The fix
exa_searchFunctionTool inadk_runner.pywith the exact name +queryarg Flash already tries to call. httpx (existing dep), results formatted in Python (no shell parsing), Exa error tags surfaced as strings (429/400/402 branchable), empty results phrased as guidance not failure, per-query cost reported. Registered inworker_tools(→ main, worker, pro_executor) and theparallel_workersfanout; mentor stays read-only.typeenum, conditional response fields, category conflicts).Verification
include_domains, and invalid key →error: Exa HTTP 401 [INVALID_API_KEY]as a string.exa_searchcall → session errored) should be extinct intool_calls/*.jsonl, and no bash entry should containapi.exa.ai … | jq.Diagnosis produced by a 4-agent research workflow during the 2026-07-14 maintenance follow-up; full write-up available on request.
Tier 3 (runtime). Note: CI will stay red until #143 (trivy/apt-cache fix) merges — same as the rest of the queue.
🤖 Generated with Claude Code