Skip to content

CL-6204: real tool-payload budget estimation + default Ollama adapter - #264

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-6204-budget-fix
Aug 21, 2026
Merged

CL-6204: real tool-payload budget estimation + default Ollama adapter#264
TheGreatAxios merged 3 commits into
mainfrom
cl-6204-budget-fix

Conversation

@TheGreatAxios

@TheGreatAxios TheGreatAxios commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Fixes CL-6535

Summary

CL-6204 (the context-compaction work) merged as Done, but two of its
core pieces were inert on main:

  • Budget estimator blind to tool payloads. estimateTurnsChars
    measured excerptBlock's human-readable placeholder
    ([tool_result callId], ~20 chars) instead of a turn's real payload.
    10 turns each carrying a 20,000-char tool_result measured as ~160
    chars against a 32,000-char hard limit — overflow detection and
    compaction never fired, and Ollama silently truncated server-side.
  • @corbits/ollama-adapter never loaded by default.
    SIDECAR_ADAPTER_MANIFEST defaulted to [], so a default deployment
    ran the built-in OpenAI adapter against Ollama and never sent
    options.num_ctx, even though the seed already writes per-model
    numCtx quirks onto seeded Ollama offerings.

Changes

  • estimateTurnsChars now measures real payload size: tool_call
    argument JSON, tool_result content (including media source sizes),
    thinking/citation/safety/code-execution blocks — kept separate from
    the human-readable excerpt used in the recap summary.
  • createBudgetedContextCompactor reserves the summary turn's own
    worst-case size out of the budget before selecting kept turns, so
    folded output can't exceed the budget it folded to (previously the
    summary could land on top of an already-full budget).
  • SIDECAR_ADAPTER_MANIFEST's default now registers
    @corbits/ollama-adapter for the "ollama" provider key instead of
    an empty manifest. An operator-set value still fully replaces the
    default rather than merging. This also required hoisting
    @corbits/ollama-adapter to the workspace root — @intx/inference's
    dynamic import() of the specifier, invoked from deep inside its own
    node_modules, cannot otherwise resolve a package only linked into
    individual consumers' node_modules.
  • Removed workbench-director.ts's dead duplicate director factory
    (defined/workbenchDirectorFactory/buildWorkbenchDirectorRef): it
    never received a contextBudget and had zero non-test call sites.

Follow-ups filed, not fixed here

  • context-budget.ts's hard limit reserves no headroom for the system
    prompt or tool definitions, though its header comment claims to.
    Needs the actual system-prompt size threaded through at that layer.

Test plan

  • apps/sidecar package test suite: 186 pass, 0 fail
  • packages/ollama-adapter package test suite: 14 pass, 0 fail
  • apps/sidecar package-scoped tsc --noEmit: clean
  • New test reproduces the reviewer's exact repro (10× 20,000-char
    tool_result turns) and proves the honest-overflow path now
    fires
  • New test proves a default boot (no SIDECAR_ADAPTER_MANIFEST)
    resolves ollama and that quirks.numCtx reaches
    options.num_ctx in the built request

…adapter registration

Reproduces the CL-6204 gaps: estimateTurnsChars undercounting tool_call/
tool_result payloads by orders of magnitude (the reviewer's 10-turn,
20,000-char tool_result repro), compaction output exceeding the budget
it folded to once the summary turn's own size is counted, and
SIDECAR_ADAPTER_MANIFEST defaulting to no adapters so a seeded Ollama
model's num_ctx never reaches the request.
…Ollama adapter by default

The context-budget estimator (estimateTurnsChars) measured excerptBlock's
human-readable placeholder ("[tool_result callId]", ~20 chars) instead of
a turn's real payload size, so tool-heavy history silently undercounted
by orders of magnitude, missed the hard limit, and let Ollama truncate
server-side with no error. Compaction now also reserves the summary
turn's own worst-case size out of the budget, so a fold no longer lands
a summary on top of an already-full budget.

SIDECAR_ADAPTER_MANIFEST defaulted to [], so @corbits/ollama-adapter
(shipped in this repo) was only ever wired up by hand-editing an env var
-- a default deployment ran the built-in OpenAI adapter against Ollama
and never sent options.num_ctx. The manifest now defaults to registering
the Ollama adapter for the "ollama" provider key; an operator-set
manifest still fully replaces it rather than merging. The adapter package
is added as a dependency of the sidecar app and hoisted to the workspace
root (matching the other @corbits/* packages already hoisted there) --
without that, @intx/inference's dynamic import of the specifier from
deep inside its own node_modules cannot resolve it at all.

Also removes workbench-director.ts's dead duplicate director factory
(defined/workbenchDirectorFactory/buildWorkbenchDirectorRef): it never
received a contextBudget and had zero non-test call sites, a legacy path
left beside the one createWorkbenchDirectorRegistry actually uses.
…ama adapter

Documents that the sidecar registers @corbits/ollama-adapter out of the
box and that setting the variable replaces the default wholesale rather
than merging with it.
@TheGreatAxios
TheGreatAxios merged commit 592cb28 into main Aug 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant