feat(dte): dynamic thinking effort (series PRs 1-5 + e2e addenda + F7) - #29
Open
easonLiangWorldedtech wants to merge 124 commits into
Open
feat(dte): dynamic thinking effort (series PRs 1-5 + e2e addenda + F7)#29easonLiangWorldedtech wants to merge 124 commits into
easonLiangWorldedtech wants to merge 124 commits into
Conversation
…nd adaptive effort envelope DTE series 2/5 (part of Zoo-Code-Org#1329). - ApiHandlerCreateMessageMetadata.reasoningEffort: per-request override channel - resolveEffectiveReasoningEffort: single shared resolution point (override > settings > model default) - AnthropicHandler: adaptive output_config.effort envelope in both requestParams branches (in-range only) - Task: setRuntimeThinkingEffort/getRuntimeThinkingEffort with in-memory apiConfiguration merge/restore, per-request metadata at all four createMessage sites, dispose() reset; never persisted
DTE series 2/5 — addresses the CodeRabbit review finding on Zoo-Code-Org#1338: when a task-local thinking-effort override is active, updateApiConfiguration() now re-captures the incoming profile's reasoningEffort as the restore value and re-applies the override on top of the new in-memory copy, so clearing the override restores the NEW profile value instead of the stale one. Additive: activation and clearing semantics are otherwise unchanged. Adds two regression tests (override active + profile switch restores new value; inactive updateApiConfiguration unchanged behavior).
DTE series 2/5 — addresses the CodeRabbit docstring-coverage warning on Zoo-Code-Org#1338 (33.33% < 80% across the functions touched by the diff): - AnthropicHandler.createMessage: documents the shared effective-effort resolution and the adaptive output_config.effort envelope (in-range only). - Task.dispose: documents centralized teardown incl. the transient task-local override reset. - Task.updateApiConfiguration: documents the override-preservation behavior (re-captured restore value + re-applied override on the new in-memory copy). Comment-only change: 30/30 patch lines and 10/10 branches unchanged; 317/317 tests and tsc --noEmit re-verified green.
…/dte-3-native-tool
Add the set_thinking_effort native tool (DTE series 3/5): the model adjusts its own per-turn thinking effort mid-task with no approval gate. - Guardrails: one-line chat notification (success or refusal), escalation cap (max 3 upward changes per task), A->B->A oscillation refusal, hard clamp to the model capability array (ties toward the lower level). - Gating: dynamicThinkingEffort experiment + model supportsReasoningEffort (non-empty array or true), evaluated at task start so the tool list stays stable within a task (prompt-cache safety). - Display: webview ChatRow one-line row (applied / oscillation / escalation refusal), i18n keys in all 17 locales; partial streaming updates the same line. - Tests: executor (clamp/cap/oscillation/no-op/no-approval/display), parser (partial + complete), dispatch, gating matrix, schema wiring, ChatRow display. Stacked on DTE PR-1 (experiment flag) and PR-2 (task-local runtime effort state). Closes Zoo-Code-Org#1330.
Address PR review feedback on set_thinking_effort (DTE series 3/5): - Executor: seed the per-task guard history with the task's effective baseline so returning from a changed value to the original baseline is refused as oscillation (A -> B -> A); existing no-op behavior preserved. - Parser: only build nativeArgs when effort AND reason are strings; a non-string payload now fails at parse time and cannot reach the executor. - Gating: a supportsReasoningEffort array that only lists 'disable' no longer exposes the tool (it could apply no level). - i18n: translate the new thinkingEffort chat strings into all 17 non-English webview locales (placeholders preserved). - Tests: regression tests for each change plus branch-coverage for the previously partial lines (non-string args, 'disable'-only capability, baseline oscillation, partial streaming without params, description fallback, capability robustness). All touched patch lines are now fully branch-covered (codecov patch partials resolved). CodeRabbit: Zoo-Code-Org#1354
… post-mode-switch revalidation, ask prefill normalization)
…e, in-chat display
…tadata in say-rejection test
easonLiangWorldedtech
changed the title
easonLiangWorldedtech
changed the title
Open
7 tasks
taskMetadata.ts L123-124 spread ...(thinkingEffort && {...}) / ...(thinkingEffortSource && {...}); the existing real-implementation call only exercised one side of each logical AND (codecov reported 2 partials on the combined trial tree). Drive the real taskMetadata() with both truthy and falsy effort values.
…-spread coverage test)
taskMetadata() now always carries thinkingEffort/thinkingEffortSource keys (even while undefined) instead of conditionally spreading them. The TaskHistoryStore upsert merges {...disk, ...delta} and buildDelta only propagates keys present in the incoming item, so an absent key left the stale persisted effort in place after the override was cleared. The spec pins the key-presence contract. (CodeRabbit on trial PR Zoo-Code-Org#1379)
abortTask() called dispose() (which clears the task-local effort state) before the final saveClineMessages(), so the history write for an aborted task carried thinkingEffort: undefined and the history-restore path could not recover the effort. Snapshot getRuntimeThinkingEffort() before dispose and pass it into saveClineMessages() (other callers read the live state). Two new abort tests cover the snapshot path and the inactive path. (CodeRabbit on trial PR 1379)
When a model capability array contains only "disable", the filtered start-level list is empty and the old hint read "the target model only supports: none". "none" is a valid NEW_TASK_EFFORT_LEVELS value, so the hint could be read as an instruction to send thinking_effort: "none", which fails validation again. Use the unsupported-model wording for that case instead. (CodeRabbit on trial PR 1379)
The new supportedReasoningEfforts hint in the nl locale said "OpenAI-compatibiele"; the correct spelling is "OpenAI-compatibele". (CodeRabbit on trial PR 1379)
…a think param getOllamaThinkParam() only read this.options.reasoningEffort, so a task-local per-request override (metadata.reasoningEffort) was ignored by the native Ollama provider while every other provider resolves it via the shared override-first order. Thread the override through buildChatRequestOptions() into getOllamaThinkParam() (override ?? settings fallback); completePrompt keeps the settings-derived value since it has no per-request metadata. Two new provider request tests cover the override-beats-settings and override-without-settings paths. (CodeRabbit on trial PR 1379)
Replace the any-typed test doubles in the thinking-effort chip spec: mockState now carries ClineMessage[]/HistoryItem[] arrays and a full HistoryItem currentTaskItem (typed factory), the context provider and findLastIndex mocks are parameter-typed/generic, mockModelInfo is a ModelInfo built from a shared fixture, and the ProviderSettings fixtures come from a typed factory instead of as-assertions. Test data and behavior unchanged. (CodeRabbit on trial PR 1379)
aimock JSON matchers cannot bind a post-tool turn to a toolCallId in this pipeline: the OpenRouter provider converts the post-tool continuation to a separate role:user environment-details message (mergeToolResultText is not set for openai/* models), and aimock toolCallId matching only inspects the LAST message, so it can never fire. Scope the fixtures with the flow keys a JSON fixture can express instead: the first turn of each flow binds to its unique prompt marker (DTE_E2E_SWITCH / the existing DTE_E2E_EFFORT_APPLY marker), and every post-tool turn additionally matches the fresh environment-details user message (userMessage "<environment_details>"), on top of the existing model + hasToolResult + turnIndex keys. Verified offline against aimock 1.35.0 loadFixtureFile/validateFixtures/matchFixture: all seven scripted requests serve the intended fixture, and the two suites still cannot cross-match. (CodeRabbit on trial PR 1379)
Registry capability arrays are never trusted blindly: supportedLevels now filters through isNewTaskEffortLevel (keeping "disable" for settings-specific handling), so an unknown string can no longer reach the ask payload's supportedThinkingEfforts or be delegated as a child start effort. Adds regression tests for unknown-value filtering and the unknown-only unsupported wording.
…ngWorldedtech/Zoo-Code into feat/dte-trial-all
Replaces the two JSON fixtures with a JS module (src/fixtures/thinking-effort.ts, same pattern as deepseek-v4.ts) after CodeRabbit confirmed the JSON scoping was insufficient: post-tool requests end with a role:user environment-details message, so aimock's toolCallId matcher (last message must be role:tool) cannot bind the continuation turns and a JSON fixture cannot carry a predicate. Each turn is now scoped to its own flow: the baseline turn binds to the suite's unique prompt marker, and every continuation binds to the previous turn's unique tool call id (call_dte_sw_001..004, call_dte_e2e_001), so no other suite can serve these responses and the DTE suites cannot match unrelated turns. Verified locally with the real extension host + aimock (USE_MOCK): both DTE suites pass (2 passing, exit 0).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dynamic Thinking Effort — reviewer guide (trial VSIX included)
1. What's in the build
feat/dte-1-experimentdynamicThinkingEffortexperimental settingfeat/dte-2-task-statefeat/dte-3-native-toolset_thinking_effortnative tool (model-driven, no approval)feat/dte-4-webview-uifeat/dte-5-orchestratornew_taskthinking_effort (subtask start effort)feat/dte-3-e2eset_thinking_effortmid-task workflowfeat/dte-5-e2enew_taskthinking_effortfeat/dte-7-f7-openai-compatTrial branch:
feat/dte-trial-all@310f0da97(all 8 heads merged, additive). A prebuilt VSIX is attached to the fork release for reviewer convenience (§2).2. Install (5 minutes)
Download the build from the release:
zoo-code-3.80.0.vsix(34,613,972 bytes · SHA25604F6B33B847101E203CA098B2381087EA4C324B9DF904F90CDDADB007D9E7594).Install it:
…or in VS Code: Extensions view → ⋯ menu → Install from VSIX… → pick the file.
Reload the window (Command Palette →
Developer: Reload Window).Open the Zoo Code panel — you're ready.
3. How to use DTE
A. Managed providers (Anthropic / OpenAI / DeepSeek / …)
set_thinking_effortto change the effort itself (you can also change it at any time with the composer toggle). The change is transient — it applies from the next request only and is never persisted to settings.\n\n > Note: the model can only call the tool when the Dynamic Thinking Effort experimental setting is enabled (see 4.5) — the composer toggle, header chip and in-chat display do not require it.new_taskcan be given a starting effort; if you leave it unset it inherits the parent's effective effort.B. Self-hosted / OpenAI-compatible (F7)
supportedEffortLevelsmulti-select to declare which effort levels your model supports.C. Effort resolution (precedence, strongest first)
task-local override(composer selection,set_thinking_effort, subtask start effort fromnew_task— all transient) →settings.reasoningEffort→model default.disableis a UI off-switch (not a level, not sent).adaptiveis Anthropic's model-decides soft signal.4. New UI surfaces — screenshots & guide
Every new DTE surface, what it is, where it appears, and how to use it. (Captured from the real webview components; light + dark themes.)
4.1 Composer thinking-effort toggle (PR 4/5)
Bottom bar of the chat composer, next to the API-config selector. Renders only for models that advertise per-request effort support (registry capability, or an F7 declaration — see 4.6). The chip shows the current effective effort; open it to switch the task-local effort — the change applies from the next request and is never persisted to settings. If no task is open yet, the selection is parked and applied to the next top-level task you start (when that task's model supports the level; a stale selection never leaks into a later task).
Dark: menu · resting
4.2 Task header chip (PR 4/5)
Sits in the task header (top of the chat, next to the task title). Shows the current effective effort plus a source badge so you can tell where it came from:
Dark: you · default
4.3 In-chat effort display (PR 3/5 + 4/5)
One-line entries in the chat whenever the effort changes, so the switch is visible in the conversation:
set_thinking_effortwith a reason)Dark: in-chat lines
4.4 Settings — supported reasoning effort levels (F7, self-hosted)
New control in Settings → your OpenAI-compatible profile (custom OpenAI endpoint, LM Studio, Ollama, vLLM, SGLang). Self-hosted models don't advertise effort capability, so declare the levels your model accepts — this unlocks the whole DTE feature set for that profile. Values buffer in the settings form and persist on Save.
Dark: declared · undeclared
4.5 Settings — experimental toggle (PR 1/5)
Settings → Experimental → Dynamic Thinking Effort. This toggle gates model-driven mid-task switching only — when it is on, capable models are given the
set_thinking_efforttool so the model itself can change the effort during a task. The UI surfaces (composer toggle, header chip, in-chat display,new_taskselector, F7 settings) are gated by model capability, not by this experiment — they render without it. Turn it on to try the model-driven flow (step 4 of §3).Dark: experimental
4.6 Composer toggle — self-hosted model with F7 declaration (F7 + 4/5)
The same composer toggle for a self-hosted OpenAI-compatible model (Qwen3.8:27B) whose effort levels were declared in 4.4 — the declared levels appear in the menu (webview capability fill-in).
Dark: menu · resting
4.7 new_task start-effort selector (PR 5/5)
When the orchestrator asks to start a subtask (
new_task), the ask block shows a small Brain-icon effort selector next to the prompt (only when the target model supports effort). It is pre-filled with the parent task's effective effort and can be switched before entering the subtask; leave it as-is to inherit. (Rendered inline in the ask bar — see the e2e addendum Zoo-Code-Org#1356 for the wire-level proof.)5. Provider support & thinking-effort options
Which providers wire thinking effort into the request, the levels they accept, and the wire shape:
low · medium · high · xhigh · max(out-of-range → omitted → API default)output_config.effort(adaptive envelope)low · medium · high(model default + selected)reasoning_effort{ effort }supported_parametersminimal · low · medium · high(effort) /thinkingBudget(2.5)thinkingConfig.thinkingLevellow · high · maxreasoning_effortlow · medium · highreasoning: { effort }none · minimal · low · medium · high · xhigh · maxreasoning_effort+thinkingtogglereasoning: { enabled, effort }low · medium · high(or model-declared)reasoning_effortlow · medium · highreasoning_effortminimal · low · medium · high · xhigh · maxreasoning_effort(extended enum)low · medium · highreasoning_effortlow · medium · high(allow-listed)reasoning_effortlow · medium · high(allow-listed)reasoning_effortlow/medium/high(xhigh/max→high)think(bool or level)6. What to verify during the trial
set_thinking_effortmid-task: applied / no-op / refused transitions behave correctly (escalation cap = max 3 upward changes per task + oscillation refusal). Requires the DTE experiment enabled (4.5).new_task: explicit start effort honored; unset inherits the parent's effective effort.supportedEffortLevelson a self-hosted profile unlocks the UI; undeclared profiles are unchanged.Known limitations (by design, for this trial):
thinkwire — see §5).adaptiveis a soft guidance signal, not a concrete level.7. Provenance
feat/dte-trial-all@310f0da97(fork).zoo-code-3.80.0.vsix— the fork's base package version; artifact identifier only) — SHA25604F6B33B847101E203CA098B2381087EA4C324B9DF904F90CDDADB007D9E7594.