Skip to content

fix(agents): name tool calls from the payload, and populate AgentResult.Events - #174

Open
ambiorix2099 wants to merge 3 commits into
mainfrom
agentspan-evals
Open

fix(agents): name tool calls from the payload, and populate AgentResult.Events#174
ambiorix2099 wants to merge 3 commits into
mainfrom
agentspan-evals

Conversation

@ambiorix2099

@ambiorix2099 ambiorix2099 commented Aug 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • AgentResult.Events was null after WaitAsync, so enumerating it threw. ToolCalls named every tool after its task type, and came back empty on any non-OpenAI provider.
  • Root cause: BuildResult never assigned Events; ExtractToolCalls named tools from taskType (the tool's own name only for a worker task, per SimpleTaskMapper) and selected them by a call_ prefix on the reference name, which is the provider's tool-call id.
  • Fix: select by task type off ToolCompiler.TYPE_MAP, name from inputData._agent_tool_name, then method, then taskDefName. Events comes from the same walk.

User impact

Same agent and call, an HTTP tool and a worker tool, run on conductor-oss 3.32.1 with openai/gpt-4o-mini each side of the change:

ToolCalls Events
before [{name="HTTP"}, {name="get_weather"}], neither with a result null, foreach threw
after [{name="lookup_order", result=…}, {name="get_weather", result=…}] ToolCall, ToolResult, ToolCall, ToolResult, Done

Scope is the polled result path; StreamAsync is unchanged.

Not fixed: the server's dynamic dispatch script sets no _agent_tool_name, so on that path an agent-as-tool is missed and a media tool still reports its task type.

Changes

  • Conductor.AI/Result.cs: IsToolTask/ResolveToolName replace the reference-name check; results fall back to the whole task output when there is no result key; Events built alongside ToolCalls
  • Conductor.AI.Tests/AgentToolCallExtractionTests.cs, StubAgentServer.cs: new. Naming per tool kind, Anthropic reference names, agent structure excluded, Events shape
  • Conductor.AI.Tests/AgentStatusMappingTests.cs: fixture gains the dispatch markers and a realistic reference name
  • docs/agents/concepts/streaming-hitl.md, docs/agents/reference/api.md: Events on a result is a reconstruction from completed tasks

Fixes #173

@ambiorix2099
fix(agents): name tool calls from the payload, and populate AgentResu…
a8c14dd 
…lt.Events

`ToolCalls` named every tool after its Conductor task type. That is correct only
for a worker tool, because Conductor sets an executed SIMPLE task's `taskType` to
the task's own name; every other kind carries its system task type there, so an
HTTP tool was named "HTTP", an MCP tool "CALL_MCP_TOOL", an agent used as a tool
"SUB_WORKFLOW". Resolve the name from `inputData._agent_tool_name`, then
`inputData.method`, then `taskDefName`.

Detection required a `call_` reference-name prefix. The server seeds the reference
from the provider's tool-call id, so that prefix is OpenAI's format and an
Anthropic-backed agent recorded no tool calls at all. Identify a tool task by task
type instead, allowlisted off the server's `ToolCompiler.TYPE_MAP`. Two cases need
corroborating evidence that the LLM dispatched the task, and take a dispatch marker
(`_agent_tool_name` or `_agent_state`) as well: the worker kind, whose task type is
the tool's own name and so cannot be allowlisted, and `SUB_WORKFLOW`/`HUMAN`, which
the agent layer emits for sub-agents, strategy workflows, routers and plan approval
steps far more often than for a tool. Without that second requirement every
multi-agent handoff would be reported as a tool call that never happened.

`AgentResult.Events` was never assigned, so enumerating it threw. Build it from the
same task walk: a `ToolCall`/`ToolResult` pair per tool call, closed by a terminal
`Done`, or `Error` for a run that did not complete. It is a reconstruction from the
finished execution's tasks and is narrower than the live stream, which is documented
rather than implied.

Also read a tool's result from the whole task output when there is no `result` key,
since an HTTP tool answers under `response`, and include `GENERATE_PDF`, which
`TYPE_MAP` omits although the server compiles a `generate_pdf` tool to it.

The existing tool-call test passed for the right reason but on a fixture that
omitted `method` and `_agent_tool_name` and carried a reference name without the
fork index and loop suffix real payloads have, so it exercised neither name
resolution nor detection. Its fixture now has the shape a real payload does.
@ambiorix2099

ambiorix2099 commented Sep 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

StreamAsync is unchanged, so these events don't reach it. When SSE is off or drops, it falls back to polling and yields one Done, discarding what WaitAsync now builds.

Fixing that isn't a one-liner: the same fallback runs after a mid-stream reconnect failure, so a replay would have to skip events already sent. No issue open.

@ambiorix2099
docs(agents): trim tool-extraction comments to the file's convention
3571c1f 
The comment layer added with the tool-call fix explained how the code works
where the code already shows it, and ran well past what this file does
elsewhere: its 25 existing doc blocks average 2.5 lines, 17 of them are
one-liners, and its section banners carry no prose at all.

Docs now state what each member is, and why it exists only where that is not
evident from the code: GENERATE_PDF is absent from the server's TYPE_MAP, the
dispatch marker is what keeps the agent's own structure out of ToolCalls, and
an HTTP tool answers under `response`. The fallback order, the return shape and
the per-kind task type lists are left to the code.

Test comments come down to the 5-6% of lines the sibling suites carry; the
fixtures already show the payload shapes the prose was restating.

Nothing executable changed.
@ambiorix2099
fix(agents): report a dispatched tool call that recorded no output
c9b0da1 
Review follow-ups on the tool-extraction change.

ExtractToolActivity skipped any task without outputData before deciding whether
it was a tool task at all, so a tool the LLM dispatched that failed or never
finished left no ToolCalls entry and no ToolCall event. The call demonstrably
happened; it is now reported with its name and arguments and no result. Servers
send outputData as an empty object rather than omitting it, so this is a
narrow path, but the ordering hid it rather than handling it.

AmbiguousToolTaskTypes named the ambiguity rather than the rule it encodes and
becomes MarkerRequiredToolTaskTypes, which is what the call site asks. The
internal-input-key list joins the two dispatch markers as a named set.

Test fixtures built by positional Replace over JSON, which would rewrite any
later occurrence of TYPE or NAME, become interpolated raw strings.

Docs: the new concepts section leads with its example, per
docs/documentation-standard.md rule 1, and the reference entry links rather
than repeating the gloss, per rule 5.
@ambiorix2099
ambiorix2099 marked this pull request as ready for review September 9, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@NicholasDCole NicholasDCole Awaiting requested review from NicholasDCole
@kowser-orkes kowser-orkes Awaiting requested review from kowser-orkes
@ling-senpeng13 ling-senpeng13 Awaiting requested review from ling-senpeng13
@dfont-orkes dfont-orkes Awaiting requested review from dfont-orkes

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AgentResult.Events is never assigned, and every non-worker tool is named after its task type

1 participant

@ambiorix2099