Fix tool-call extraction and populate events on the run() path - #504
Open
ambiorix2099 wants to merge 2 commits into
Open
Fix tool-call extraction and populate events on the run() path#504ambiorix2099 wants to merge 2 commits into
ambiorix2099 wants to merge 2 commits into
Conversation
Tool tasks were identified by a `call_` reference-name prefix. That prefix is
the LLM provider's tool-call id format, not something the server adds, so an
Anthropic-backed agent (`toolu_…`) recorded no tool calls at all. Five tool
kinds were also filtered out as system tasks and five more were named from the
task type, and `run()` never passed `events=` at all.
Identify a tool by the task type it compiled to, per the server's
`ToolCompiler.TYPE_MAP`, and resolve its name from `_agent_tool_name` — which
the tool-dispatch script sets on every tool kind — falling back to `method` and
the task definition. Nothing reads the reference name for identity, so nothing
depends on provider-controlled data. Names are no longer case-folded, so
`getWeather` stops arriving as `getweather`.
`agent_tool` and a strategy handoff both compile to SUB_WORKFLOW; only the tool
carries `_agent_tool_name`, which is what tells them apart. The agent's own
statically-compiled workers — guardrails, callbacks, gates, routing — compile to
SIMPLE tasks like a worker tool does, so they are excluded by name; that check
is consulted only after the dispatch key has failed to settle the question.
`run()` and `run_async()` now derive `events` from the execution they already
fetch for `tool_calls`, with no extra call. Eight assertions in
`conductor.ai.agents.testing` read `events`, and four of them check for absence —
including `assert_no_errors`, which the eval runner runs on every case — so they
were passing without evidence. `expect_handoff_to`, used by the eval runner's
own documented example, was failing every time.
The task-to-event mapping was copy-pasted across the sync and async polling
streams; both now share `_task_events`, which is also what `run()` uses, so an
assertion reads the same whichever way the agent was run.
Verified against a live server: before, an Anthropic-backed agent with one tool
returned `tool_calls: []` and `events: []`; after, `[('getWeather', {'city':
'Tokyo'})]` and a full event list. The OpenAI path is unchanged apart from the
name no longer being lowercased.
ambiorix2099
force-pushed
the
agentspan-evals
branch
from
September 8, 2026 22:46
b7fe5ba to
2a3e64d
Compare
Contributor
Author
|
Framework agents: the Claude Agent SDK injects a task per tool call ( LangChain and LangGraph publish their steps to the event stream rather than as tasks, so their per-step detail still reaches |
…-task detection
The Claude Agent SDK injects one task per tool call into the running
execution (POST /agent/{id}/tasks) using the Anthropic tool-use id as the
reference name, so those tool calls sit in the polled task list. But run()
routes every framework agent to _run_framework, which built its result with
neither tool_calls nor events, so the very case #501 captured its payloads
from still came back empty. All four framework result paths now derive both
fields from the execution, as the native path does. LangChain and LangGraph
publish their steps to the event stream rather than as tasks, so their
per-step detail is still stream-only.
_is_agent_internal_task matched its suffixes against the reference name or
the task definition name, which dropped an injected tool of the user's own
named, say, refresh_gate. It now requires both names to look internal, and
strips Conductor's __N turn counter from the reference first — without which
an internal worker inside the agent loop was read as a tool call, since only
the task definition name saved it.
Drops _transfer_check, a suffix nothing mints. Renames the new test doubles
to FakeTask/FakeWorkflowRun, so Workflow no longer stands for an execution.
Comments on the new code rewritten to the module's own style.
ambiorix2099
marked this pull request as ready for review
September 9, 2026 17:03
NicholasDCole
approved these changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
run(),AgentResult.tool_callsdropped or misnamed most tool kinds andAgentResult.eventswas always empty. 13 of the 17 assertions inconductor.ai.agents.testingread one of the two; three pass on the empty list rather than fail._extract_tool_callskeyed on acall_reference-name prefix, which is the provider's tool-call id format and not Conductor's;_SYSTEM_TASK_TYPESskipped five task types thatToolCompiler.TYPE_MAPcompiles tool kinds to; survivors were namedtask_type.lower().run()never passedevents=._agent_tool_namekey the dispatch script sets, and derive events from the execution already fetched. Polled task list throughout; the SSE path is untouched.User impact
One
runtime.run()per tree, same agent, agetWeatherworker tool and an HTTP tool, Conductor 3.32.0-rc18 onopenai/gpt-4o-mini:result.tool_callsresult.events[('getweather', {'city': 'Tokyo', '_agent_tool_name': 'getWeather'})][][('getWeather', {'city': 'Tokyo'}), ('serverHealth', …)]thinking, tool_call, tool_result, tool_call, tool_result, thinking, doneBoth runs scheduled the same tasks.
assert_no_errorsruns on every eval case and was green without evidence;expect_handoff_to, the runner's documented example, failed every time.Changes
runtime.py:_is_tool_task/_tool_name/_tool_argsreplace the prefix check, aSUB_WORKFLOWcounting as a tool only with that key, which separates anagent_toolfrom a handoff;events=at the fiveAgentResultsites;_task_eventsshared with both polling streamsruntime.py: framework agents derive both fields from their execution too, in_run_framework,_run_framework_with_eventsand both async branches, since the Claude Agent SDK injects a task per tool callresult.py: one internal-argument key set, soAgentEvent.argsandtool_callsagreetests/unit/ai/test_tool_extraction.py: new, per tool kind and both provider id formatstests/unit/ai/test_runtime.py:test_non_tool_tasks_ignoredused aSIMPLEtask the server counts as a tool; nowINLINEFixes #501