diff --git a/docs/advanced-usage/available-tools/new-task.md b/docs/advanced-usage/available-tools/new-task.md index afae4f21..048da12e 100644 --- a/docs/advanced-usage/available-tools/new-task.md +++ b/docs/advanced-usage/available-tools/new-task.md @@ -25,6 +25,7 @@ The tool accepts these parameters: - `mode` (required): The slug of the mode to start the new task in (e.g., "code", "ask", "architect") - `message` (required): The initial user message or instructions for this new task - `todos` (optional): Initial todo list in markdown checklist format +- `thinking_effort` (optional): The thinking effort to start the subtask at. Must be a level the target model supports. Omit it to inherit the parent task's effective effort. When the approval block appears, an effort selector next to the prompt is pre-filled with this value and can be changed before approval. See [Thinking Effort](/features/thinking-effort) --- diff --git a/docs/advanced-usage/available-tools/set-thinking-effort.md b/docs/advanced-usage/available-tools/set-thinking-effort.md new file mode 100644 index 00000000..d1cabd79 --- /dev/null +++ b/docs/advanced-usage/available-tools/set-thinking-effort.md @@ -0,0 +1,110 @@ +--- +description: Discover how the set_thinking_effort tool lets the model adjust its own thinking effort mid-task, with guardrails that keep changes bounded. +keywords: + - set_thinking_effort + - Zoo Code tools + - thinking effort + - reasoning effort + - mid-task adjustment + - dynamic thinking effort +--- + +# set_thinking_effort + +The `set_thinking_effort` tool adjusts the current task's thinking effort while the task is running. The model calls it when it judges that the task needs more or less reasoning than the current effort provides. The change applies from the next API request. + +--- + +## Parameters + +The tool accepts these parameters: + +- `effort` (required): The effort level to apply. Valid levels: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`. A requested level the model does not support is clamped to the nearest supported level. +- `reason` (required): Why the change is needed. The reason appears in the in-chat display line. + +--- + +## What It Does + +This tool sets the task-local thinking effort override for the running task. The override applies to the next API request and stays in effect for the rest of the task until it is changed again. It is never written to the provider settings. The override is stored with the task, so reopening the task from history restores it. + +The tool does not require user approval. The model decides when to call it, and the user can still change the effort at any time with the composer thinking effort selector. + +--- + +## When is it used? + +- When the model starts a task at a moderate effort and later determines that a specific step needs deeper analysis +- When a long task contains phases that need less reasoning, to keep the task fast and within budget +- When the task direction changes and the previous effort no longer fits the work + +--- + +## Key Features + +- Model-initiated, with a required reason shown in the chat +- No approval gate; the user retains control through the composer toggle +- Applies from the next request, never mid-stream +- Task-local: it does not modify the provider settings or other tasks; the override is stored with the task and restored when the task is reopened from history +- Guardrails keep the effort bounded within a single task + +--- + +## Limitations + +- Requires the **Dynamic Thinking Effort** experimental setting to be enabled; otherwise the tool is not exposed to the model +- Requires a model that advertises per-request thinking effort; otherwise the call returns an error +- A requested level that the model does not support is clamped to the nearest supported level; if the model advertises no usable levels, the call is refused +- Oscillation is refused: returning to the level the task just moved away from (A → B → A) keeps the current effort +- Upward changes are capped at 3 per task; further upward changes are refused +- Setting `effort` to `disable` is not supported; use the provider settings to turn reasoning off + +--- + +## How It Works + +When the model calls `set_thinking_effort`, the tool follows this process: + +1. **Validation**: + - Rejects missing `effort` or `reason` + - Rejects levels outside the valid set + - Clamps the requested level to the nearest level in the model's capability array + +2. **Guardrails**: + - If the task is already at the requested level, the call is a no-op and the tool confirms the current effort + - If the requested level would oscillate back to the level the task just left, the change is refused and a refusal line appears in the chat + - If the change is upward and the task has already made 3 upward changes, the change is refused and a refusal line appears in the chat + +3. **Application**: + - Sets the task-local runtime effort with the source marked as model-driven + - Appends one line to the chat: `Thinking effort: (Zoo) — ` + - The task header chip updates to show the new effort with the **Zoo (auto)** source badge + - The next API request carries the new effort; the provider settings are unchanged + +--- + +## Usage Examples + +Raising the effort before a complex refactoring step: + +``` + +high +The next step refactors the authentication module across 6 files; deeper analysis reduces the chance of breaking edge cases. + +``` + +Lowering the effort for a repetitive batch step: + +``` + +low +The remaining 40 files need the same one-line change; fast responses keep the task within budget. + +``` + +Refused oscillation: the task moves from `high` to `medium`, then requests `high` again. The tool refuses with: + +``` +Thinking effort change refused: oscillation between 'high' and 'medium' detected. Keep the current effort. +``` diff --git a/docs/basic-usage/the-chat-interface.md b/docs/basic-usage/the-chat-interface.md index 05e4f062..21406bd7 100644 --- a/docs/basic-usage/the-chat-interface.md +++ b/docs/basic-usage/the-chat-interface.md @@ -34,6 +34,8 @@ The chat interface consists of the following main elements: 7. **Mode Selector:** The mode selector is a dropdown located to the left of the chat input field. It is used for selecting which mode Zoo should use for your tasks. Its settings gear opens the Modes tab, not general settings. +8. **Thinking Effort Selector:** The thinking effort selector (brain icon) appears in the composer bottom bar when the selected model supports per-request thinking effort. It sets the effort for the next request without touching your settings. See [Thinking Effort](/features/thinking-effort). + The chat composer, mode and API selectors, auto-approval controls, and todo-delete confirmation follow your active IDE appearance. Their borders, hover states, keyboard focus rings, and error text remain distinguishable in VS Code light, dark, and high-contrast themes. Zoo Code chat composer using the VS Code light theme diff --git a/docs/features/experimental/experimental-features.md b/docs/features/experimental/experimental-features.md index 068287c0..cf39d7fe 100644 --- a/docs/features/experimental/experimental-features.md +++ b/docs/features/experimental/experimental-features.md @@ -33,6 +33,7 @@ The following experimental features are currently available: - [Custom Tools](/features/experimental/custom-tools) - Define TypeScript/JavaScript tools that Zoo can call like built-in tools - [Background Editing](/features/experimental/background-editing) - Work uninterrupted while Zoo edits files in the background - [Image Generation](/features/image-generation) - Generate images from text prompts and save them to your workspace +- [Dynamic Thinking Effort](/features/thinking-effort) - Let the model adjust its own thinking effort mid-task with the `set_thinking_effort` tool. The chat UI surfaces (composer toggle, header chip, in-chat display) are gated by model capability and do not require this setting - [Run Slash Command](/advanced-usage/available-tools/run-slash-command) - Execute predefined slash commands for templated instructions and workflow automation --- diff --git a/docs/features/index.md b/docs/features/index.md index 8e0aefb1..3e2cb04f 100644 --- a/docs/features/index.md +++ b/docs/features/index.md @@ -28,6 +28,7 @@ Discover the powerful features that make Zoo Code your ultimate AI-powered codin - [**Enhance Prompt**](/features/enhance-prompt) - Automatically improve your prompts for better results - [**Suggested Responses**](/features/suggested-responses) - Context-aware follow-up suggestions - [**Model Temperature**](/features/model-temperature) - Fine-tune AI creativity and consistency +- [**Thinking Effort**](/features/thinking-effort) - Control how hard a capable model thinks, per task and per request ### Workflow Management - [**Task Todo List**](/features/task-todo-list) - Track progress on complex multi-step tasks diff --git a/docs/features/thinking-effort.md b/docs/features/thinking-effort.md new file mode 100644 index 00000000..046fada0 --- /dev/null +++ b/docs/features/thinking-effort.md @@ -0,0 +1,154 @@ +--- +description: Control how hard a capable model thinks. Set the thinking effort for a task, have the model adjust it mid-task, and declare supported levels for self-hosted models. +keywords: + - thinking effort + - reasoning effort + - dynamic thinking effort + - model reasoning + - per-request effort + - self-hosted models + - supported reasoning effort levels +--- + +# Thinking Effort + +Zoo Code sends a **thinking effort** value with each request to models that support it. The value controls how much reasoning the model applies before answering. Higher levels take longer and cost more. Lower levels respond faster. + +The effort resolves at two levels: + +- **Profile default** — the "Model Reasoning Effort" value in the provider settings. Stored per API configuration profile and sent with every request. +- **Task-local override** — set from the chat during a task. It applies from the next request and is never written to settings. The override is stored with the task, so reopening the task from history restores it; a new task starts from the profile default. + +--- + +## Setting the effort for a task + +### Composer toggle + +The composer bottom bar shows a thinking effort selector (brain icon) when the selected model supports per-request effort. + +Composer thinking effort menu listing the levels the model supports + +*The composer menu lists only the levels the model advertises. Selecting a level applies it from the next request.* + +- Selecting a level sets the task-local override. The change takes effect from the next API request, not mid-stream. +- If no task is open, the selection is kept and applied to the next task you start, when that task's model supports the level. +- The selection is stored with the task (and restored when the task is reopened from history) but is never persisted to settings. + +### Task header chip + +The task header shows the current effective effort with a source badge: + +| Badge | Meaning | +|---|---| +| **you** | Set with the composer toggle, and different from the profile default | +| **Zoo (auto)** | Set by the model with the `set_thinking_effort` tool, or inherited from a parent task | +| **default** | The profile default or the model default | + +Task header chip showing the effective effort and its source badge + +*The source badge shows where the current effort came from.* + +### In-chat display + +Each effort change appends one line to the chat: + +- Model-driven: `Thinking effort: high (Zoo) — ` +- User-driven: `Thinking effort set to: low` +- Refused: a refusal line (see [Model-driven changes](#model-driven-changes)) + +--- + +## Effort levels and precedence + +Not every level is available for every model. The composer menu and the settings dropdown show only what the model advertises. + +| Level | Settings label | +|---|---| +| `none` | None | +| `minimal` | Minimal (Fastest) | +| `low` | Low | +| `medium` | Medium | +| `high` | High | +| `xhigh` | Extra High | +| `max` | Max | + +The effective effort resolves in this order, strongest first: + +1. Task-local override — composer toggle, `set_thinking_effort`, or the subtask start effort +2. Profile default — "Model Reasoning Effort" in the provider settings +3. Model default + +--- + +## Model Reasoning Effort setting + +The "Model Reasoning Effort" dropdown in the provider settings sets the profile default. It appears for providers whose models support reasoning effort, for example [OpenAI](/providers/openai), [DeepSeek](/providers/deepseek), [xAI](/providers/xai), and [Kimi Code](/providers/kimi-code). + +- Selecting **None** turns reasoning off for that profile. +- The selection is stored with the profile and survives reloads. +- The per-provider pages list the exact levels and wire details for each provider. + +--- + +## Model-driven changes + +With the **Dynamic Thinking Effort** experimental setting enabled, a capable model can call the `set_thinking_effort` tool to adjust its own effort mid-task, for example after it judges that the task needs deeper analysis. + +- The tool is model-initiated. It does not require user approval, and the model supplies a reason that appears in the in-chat line. +- Guardrails apply: a requested level is clamped to the nearest level the model supports, a level change that ping-pongs back (A → B → A) is refused, and at most 3 upward changes per task are allowed. +- Refused changes show a refusal line in the chat and leave the effort unchanged. + +The experimental setting gates model-driven changes only. The composer toggle, header chip, and in-chat display are gated by model capability and work with the experimental setting off. See [Experimental Features](/features/experimental/experimental-features) and the [set_thinking_effort](/advanced-usage/available-tools/set-thinking-effort) tool reference. + +--- + +## Subtask start effort + +When the orchestrator starts a subtask with `new_task`, the approval block shows an effort selector next to the prompt, pre-filled with the parent task's effective effort. + +- Change the selection before approving to start the subtask at a different effort. +- Leave it as-is to inherit the parent's effort. +- The selection applies to the subtask only. The parent task keeps its own effort. + +--- + +## Self-hosted and OpenAI-compatible models + +Self-hosted models do not advertise thinking effort support, so the controls are hidden by default. Declare the levels your model accepts on the [OpenAI Compatible](/providers/openai-compatible) provider page: + +1. Open Settings and select your **OpenAI Compatible** profile. +2. Tick **Enable Reasoning Effort**. +3. Under **Supported Reasoning Effort Levels**, tick the levels your model accepts (None → Max). +4. Select the default level in **Model Reasoning Effort**. +5. Click **Save**. + +Supported Reasoning Effort Levels declaration on an OpenAI Compatible profile + +*Declaring the levels your model accepts unlocks the thinking effort controls for that profile.* + +After saving, the composer toggle, header chip, and in-chat display render for that profile, and the declared levels appear in the menu. A declaration with no levels selected keeps the controls off. + +**Scope note:** on the OpenAI Compatible provider page, the declaration unlocks the controls and the profile default. The selected level is not yet sent to the server with each request. The [Ollama](/providers/ollama) provider already sends the selected level through Ollama's `think` parameter. + +--- + +## Provider support + +Per-request thinking effort (the task-local surfaces above) is available where the model advertises the capability: + +| Provider | Levels | Notes | +|---|---|---| +| [Anthropic](/providers/anthropic) / [Vertex](/providers/vertex) | low, medium, high, Extra High, Max | Adaptive-thinking models | +| [OpenAI](/providers/openai) | low, medium, high | o-series and GPT-5 families | +| [OpenRouter](/providers/openrouter) | per model | Capability comes from the model metadata | +| [Google Gemini](/providers/gemini) | Minimal, Low, Medium, High | `thinkingBudget` on 2.5 models | +| [DeepSeek](/providers/deepseek) | low, high, max | | +| [xAI](/providers/xai) | low, medium, high | Grok reasoning models | +| [Z.ai](/providers/zai) | None → Max | GLM models with a `thinking` toggle | +| [Friendli](/providers/friendli), [Requesty](/providers/requesty), Poe, Unbound | low, medium, high (extended on Friendli) | | +| [Kimi Code](/providers/kimi-code), nanoGPT, opencode-go | low, medium, high (varies) | | +| [Ollama](/providers/ollama) | low, medium, high | Sent through the `think` parameter. Extra High and Max map to high | +| [OpenAI Compatible](/providers/openai-compatible) (self-hosted) | user-declared | Declaration unlocks the controls. The selected level is not yet sent with each request | + +Profiles that do not advertise the capability (and have no declaration) keep their current behavior. No thinking effort parameter is sent. diff --git a/docs/providers/lmstudio.md b/docs/providers/lmstudio.md index 76cc4dea..7beed1d2 100644 --- a/docs/providers/lmstudio.md +++ b/docs/providers/lmstudio.md @@ -51,3 +51,4 @@ Zoo Code supports running models locally using LM Studio. LM Studio provides a * **Troubleshooting:** If you see a "Please check the LM Studio developer logs to debug what went wrong" error, you may need to adjust the context length settings in LM Studio. * **Token Tracking:** Zoo Code tracks token usage for models run via LM Studio, helping you monitor consumption. * **Reasoning Support:** For models that support it, Zoo Code can parse "think" tags or similar reasoning indicators in LM Studio responses, offering more insight into the model's process. +* **Thinking Effort Levels:** The LM Studio provider page does not expose the "Supported Reasoning Effort Levels" declaration. To declare the levels your model accepts and use the thinking effort controls, point Zoo Code at LM Studio's OpenAI-compatible endpoint with the [OpenAI Compatible](/providers/openai-compatible) provider. See [Thinking Effort](/features/thinking-effort). diff --git a/docs/providers/ollama.md b/docs/providers/ollama.md index 7100125a..f57829c0 100644 --- a/docs/providers/ollama.md +++ b/docs/providers/ollama.md @@ -110,6 +110,18 @@ Zoo Code supports running models locally using Ollama. This provides privacy, of --- +## Reasoning Effort + +Models that support reasoning effort expose a **Model Reasoning Effort** dropdown in the Ollama provider settings. The selected level is sent with each request through Ollama's `think` parameter: + +- **None** turns the parameter off. +- Low, Medium, and High map 1:1 to the Ollama parameter. +- Extra High and Max map to High. + +Selecting a level in the composer thinking effort selector overrides the profile default for the running task. See [Thinking Effort](/features/thinking-effort) for the task-local behavior. + +--- + ## Tips and Notes * **Resource Requirements:** Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose. diff --git a/docs/providers/openai-compatible.md b/docs/providers/openai-compatible.md index 2307884f..e98db599 100644 --- a/docs/providers/openai-compatible.md +++ b/docs/providers/openai-compatible.md @@ -48,6 +48,22 @@ You'll find these settings in the Zoo Code settings panel (click the