Skip to content

ollama-adapter: stop leaking raw chain-of-thought into chat replies - #323

Merged
TheGreatAxios merged 4 commits into
mainfrom
cl-6654-reasoning-leak
Aug 22, 2026
Merged

ollama-adapter: stop leaking raw chain-of-thought into chat replies#323
TheGreatAxios merged 4 commits into
mainfrom
cl-6654-reasoning-leak

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Contributor

Summary

CL-6654: an agent's visible reply leaked the model's raw reasoning/thinking
text and internal tool identifiers into the timeline (screenshots on the
ticket). Root cause: Ollama's OpenAI-compatible endpoint never populates
the reasoning_content field @intx/inference's OpenAI provider checks
for reasoning — gpt-oss/qwen instead wrap their chain-of-thought inline in
<think>...</think> inside the ordinary content stream, so it rode
through every hop as genuine reply text: harness → inferenceDoneBlocks
(correctly typed "text", since that's what it actually was) →
chat-orchestrator.ts's TextParttimeline.tsx's plain bubble.

The fix is contained entirely in packages/ollama-adapter, which already
wraps @intx/inference's built-in OpenAI adapter for this provider: it now
also wraps parseResponse/parseJSONResponse to reclassify a <think>
span's tokens as inference.thinking.delta before anything downstream
ever sees them. Once correctly typed:

  • The vendored harness (vendor/intx/hub-sessions/event-collector.ts)
    already turns a type: "thinking" content block into a "reasoning"
    part, and packages/agent-events's inferenceDoneBlocks already drops
    it from the finalized reply (see its existing "drops block kinds with no
    chat-part equivalent" test) — no visible chain-of-thought disclosure by
    default, per the product call on this ticket.
  • The existing live TurnActivityStrip "Thinking…" indicator
    (packages/chat-ui/src/turn-activity.tsx) starts correctly firing for
    Ollama models for the first time — it already listens for
    inference.thinking.delta and previously never received one from this
    provider, so gpt-oss/qwen turns had no working-indicator at all.
  • Tool calls the model narrates in its own reasoning ("I should call the
    Giphy tool...") are dropped along with the rest of the reasoning span,
    so no raw @corbits/... paths reach the transcript either.

This closes the leak without inventing a new UI surface — react-ui's
ReasoningPartView disclosure and tool-activity.ts's natural-language
tool rows already exist and are simply routed to correctly now.

Everything upstream (@intx/inference) is unmodified; the fix is a
wrapper in our own package, consistent with how it already wraps
buildRequest for request-body overrides.

Scope cut under timebox

A named, per-agent typing indicator ("Myra is typing…") and
connector-logo tool-activity rows were both raised as follow-on design
asks on this ticket. The generic "Thinking…" strip and text-only tool
rows already ship safely (no leak, no raw identifiers) as of this PR;
the identity/logo polish is filed as a follow-up rather than blocking this
fix.

Live browser verification (agent-browser against a running Ollama
model) was not completed in this pass — the fix is validated at the unit
level against the actual vendored harness's block-indexing behavior
(vendor/intx/hub-sessions) and @intx/inference's real event shapes.
Recommend a manual pass with a real gpt-oss/qwen model before merge.

Test plan

  • packages/ollama-adapter/src/think-tags.test.ts — whole/split
    <think> spans reclassified, ordinary text and tool-call events
    pass through untouched, thinking events never collide with the
    text index the harness's block map keys on
  • WORKBENCH_CHECK_SINCE=origin/main bun run test
  • WORKBENCH_CHECK_SINCE=origin/main bun run typecheck
  • bun run lint
  • Manual: real Ollama gpt-oss/qwen session, confirm transcript shows
    only the polished reply, no <think> text, no raw tool paths

Ollama's OpenAI-compatible endpoint never populates the reasoning_content
field @intx/inference's OpenAI provider checks for reasoning; gpt-oss and
qwen instead wrap their chain-of-thought inline in <think>...</think>
inside the ordinary content stream, so it rides through as plain reply
text. This module splits those spans out into inference.thinking.delta
events instead, at the one place that knows the tokens came from Ollama.
Wraps parseResponse/parseJSONResponse so a <think>...</think> span in a
local model's output is reclassified as thinking rather than text before
anything downstream sees it. Once correctly typed, the harness already
keeps thinking content out of the finalized turn's visible text (see
agent-events's existing "drops block kinds with no chat-part equivalent"
behavior), and the existing live "Thinking..." strip in turn-activity.tsx
now activates for the whole reasoning window instead of never firing —
this closes CL-6654's leak of raw model reasoning and internal tool
identifiers into the visible transcript.

Reclassified thinking events use a fixed sentinel index distinct from the
adapter's own text/tool-call indices, since the harness's per-index block
map throws a ProtocolMismatchError if two block kinds share an index.
@TheGreatAxios
TheGreatAxios merged commit 85fffea into main Aug 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant