From fa25cc0fc6734efdc48678d61fcb923b5350086a Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Tue, 22 Sep 2026 13:01:22 +0900 Subject: [PATCH 01/12] docs: document classification and mcpSettings.deferSchemaBytes --- .../object_structure/classification.mdx | 156 ++++++++++++++++++ .../object_structure/mcp_settings.mdx | 25 +++ .../librechat_yaml/object_structure/meta.json | 1 + 3 files changed, 182 insertions(+) create mode 100644 content/docs/configuration/librechat_yaml/object_structure/classification.mdx diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx new file mode 100644 index 000000000..065d43eab --- /dev/null +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -0,0 +1,156 @@ +--- +title: "Classification Object Structure" +icon: SlidersHorizontal +--- + +## Overview + +The `classification` configuration connects LibreChat to a classifier: a model that +answers small typed questions about text and returns probabilities, rather than +writing prose. Features use it where a decision has to be made before or around a +model call, such as picking which tools a turn needs. + +Three question shapes are available: + +- **boolean** asks whether something holds and returns the probability of yes. +- **choice** picks one option from a defined set and returns the full distribution. +- **score** rates against ordered levels and returns a weighted position. + +Classification is disabled by default. When it is off, or when no provider is +configured, every feature built on it behaves exactly as it does without it. + +## Example + +```yaml filename="Classification Object Structure" +classification: + enabled: true + provider: http + providers: + http: + baseURL: https://classifier.example.com/v1/classify + apiKeyEnv: CLASSIFIER_API_KEY + timeoutMs: 4000 + maxRetries: 2 + + toolSelection: + enabled: true + shortlist: 5 + needsToolThreshold: 0.15 + lowConfidenceExtra: 3 + + memoryGate: + enabled: true + threshold: 0.25 +``` + +## Configuration + +### Subkeys + + + +## providers + +Each provider reads its API key from an environment variable. The key is never +written in `librechat.yaml`; only the name of the variable is. + + + +## toolSelection + +An agent holding a large MCP catalog pays for every tool's argument schema on every +turn. Deferring those schemas solves the cost, but the model then spends a +`tool_search` round trip discovering the tool it needs, on the turn it needs it. + +Tool selection ranks the deferred tools against the request before the model runs +and surfaces the few it is likely to need, so their schemas arrive with the first +call. It requires tools that are already deferred, whether by the per-tool toggle in +the agent builder or by [`mcpSettings.deferSchemaBytes`](/docs/configuration/librechat_yaml/object_structure/mcp_settings). + +Selection only ever adds. A tool it passes over stays listed by name in the +`tool_search` description and remains one search away, so a wrong prediction costs a +round trip rather than an answer. + + + +## memoryGate + +The memory model runs after every turn, and most turns carry nothing durable. The +gate asks one yes/no question first and skips that call when the answer is no. + +A gate that cannot reach its provider answers yes, so an outage never causes a +memory to be lost. + + + + + `whenTrue` and `whenFalse` are named that way because YAML reads bare `true:` and + `false:` keys as booleans rather than strings. + + +## Customizing the questions + +Every question ships with default wording, and an operator can replace it without +touching code. A support deployment can tell the ranker what its tools are for, and +a deployment with its own idea of what is worth remembering can say so: + +```yaml filename="Custom question wording" +classification: + toolSelection: + instructions: Pick the tool for this support ticket. + guidance: Prefer billing tools for anything about an invoice. + memoryGate: + instructions: Is there a dietary requirement here? + whenTrue: An allergy or a standing preference. + whenFalse: Anything about one meal only. +``` + +## Notes on accuracy + +A classifier returns probabilities, and the thresholds above decide what to do with +them. Two properties are worth understanding before enabling either capability: + +- Thresholds are domain specific. Start with the defaults, observe your own traffic, + and adjust rather than assuming the defaults transfer. +- Probabilities are only useful when the provider produces calibrated ones. A + provider that cannot measure confidence reports it as absent, and tool selection + treats that as unconfident and widens the shortlist instead of narrowing it. diff --git a/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx b/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx index da823e629..184ce6ae5 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx @@ -24,6 +24,10 @@ mcpSettings: - "host.docker.internal:8080" # Permit one private host on one port - "10.0.0.5:8000" # Permit one private IP on one port + # Withhold an MCP tool's argument schema from the model when it is larger than + # this many bytes. 0, the default, disables the rule. + deferSchemaBytes: 4096 + # Optional MCP OAuth catalog-recovery tuning; defaults are shown. catalogRecovery: discoveryBackoffMs: [300000, 600000, 1200000, 1800000] @@ -45,6 +49,7 @@ mcpSettings: options={[ ['allowedDomains', 'Array of Strings', 'A list specifying allowed domains for MCP server connections.', 'When configured, only listed domains are allowed. When not configured, SSRF targets are blocked but all other domains are allowed.'], ['allowedAddresses', 'Array of Strings', 'An SSRF exemption list, scoped to private IP space. Hostname/IP + port pairs listed here bypass the default-deny SSRF block when `allowedDomains` is not configured.', 'Use when you want default SSRF protection AND specific internal MCP servers, without flipping `allowedDomains` into strict-whitelist mode.'], + ['deferSchemaBytes', 'Number', 'Argument-schema size, in bytes, above which an MCP tool starts deferred: the model receives its name and description and loads the schema through `tool_search` when it needs it. A per-tool `defer_loading` toggle still takes precedence in either direction.', '0, which disables the rule and leaves every tool exactly as its own options configure it.'], ['catalogRecovery', 'Object', 'Optional bounds for passive MCP discovery, OAuth reauthentication recovery, and cross-replica authorization fencing.', 'Uses the defaults shown in the example.'], ]} /> @@ -197,3 +202,23 @@ A hostname entry trusts whatever IP that hostname resolves to at runtime on the - [MCP Servers Configuration](/docs/configuration/librechat_yaml/object_structure/mcp_servers) - [MCP Features](/docs/features/mcp) - [Actions allowedAddresses](/docs/configuration/librechat_yaml/object_structure/actions#allowedaddresses) (similar concept for Actions) + +## deferSchemaBytes + +A small number of MCP tools can account for most of a catalog's size. A tool whose +argument schema is very large costs those tokens on every turn, even when the +conversation has nothing to do with it, while its description may be only a line. + +Setting `deferSchemaBytes` withholds the argument schema of any tool above that +size. The model still sees the tool's name and description in the `tool_search` +listing and loads the schema when it decides to use it, so nothing becomes +unreachable. + +Precedence, highest first: + +1. A per-tool `defer_loading` toggle in the agent builder, in either direction. +2. `mcpSettings.deferSchemaBytes`, when the tool has no explicit toggle. +3. Otherwise the tool is loaded in full. + +Because an explicit per-tool choice always wins, the rule is safe to apply +instance-wide without overriding tools an author deliberately pinned open. diff --git a/content/docs/configuration/librechat_yaml/object_structure/meta.json b/content/docs/configuration/librechat_yaml/object_structure/meta.json index aa15aff93..bf10d6b5b 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/meta.json +++ b/content/docs/configuration/librechat_yaml/object_structure/meta.json @@ -26,6 +26,7 @@ "mcp_settings", "skill_sync", "memory", + "classification", "summarization", "message_filter", "web_search", From 2f5d902066d49a3e0f55e98cb74dead65596fd6c Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Tue, 22 Sep 2026 13:23:40 +0900 Subject: [PATCH 02/12] docs: document memory categorization options --- .../object_structure/classification.mdx | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index 065d43eab..927564a21 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -119,9 +119,24 @@ memory to be lost. ['instructions', 'String', 'Replaces the wording of the durability question.', 'A built-in question.'], ['whenTrue', 'String', 'Describes what a yes means.', 'Built-in wording.'], ['whenFalse', 'String', 'Describes what a no means.', 'Built-in wording.'], + ['categorize', 'Boolean', 'Also ask which of `memory.validKeys` the turn belongs under, and suggest it to the memory model. Requires `validKeys`; ignored without them.', 'false'], + ['categoryThreshold', 'Number', 'Below this probability the suggested key is dropped rather than shown.', '0.4'], + ['detectUpdates', 'Boolean', 'Also ask whether the turn changes something already stored, rather than adding something new.', 'false'], ]} /> +`categorize` and `detectUpdates` ride in the request the gate already makes, so +they cost no additional round trip. The result reaches the memory model as a +suggestion it may ignore, not as an override: + +```text + +The durable part of this turn most likely belongs under `work_context`. It looks +like a change to what is already stored there, not a new fact. +Ignore this if it does not fit what the user actually said. + +``` + `whenTrue` and `whenFalse` are named that way because YAML reads bare `true:` and `false:` keys as booleans rather than strings. From f1c4e86962feac03c0617688898763fac672b581 Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Tue, 22 Sep 2026 13:55:44 +0900 Subject: [PATCH 03/12] docs: document endpoints.agents.maxToolResultChars --- .../object_structure/agents.mdx | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/agents.mdx b/content/docs/configuration/librechat_yaml/object_structure/agents.mdx index 2bda69bde..322a65950 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/agents.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/agents.mdx @@ -90,6 +90,35 @@ endpoints: > This configuration enables the builder interface for agents. +## maxToolResultChars + + + +**Default:** `0`, which keeps every result whole. + +**Example:** + +```yaml filename="endpoints / agents / maxToolResultChars" +maxToolResultChars: 28800 +``` + +Tool output is often the largest thing in a conversation, and the size is +usually concentrated in a few calls rather than spread evenly. Because the cap +is applied before the result is stored, every later turn re-sends the capped +version, so the saving repeats for the life of the conversation. + +The model is told the result was cut, so it can call the tool again more +narrowly instead of working from partial data without knowing it. + ## recursionLimit Date: Tue, 22 Sep 2026 16:22:25 +0900 Subject: [PATCH 04/12] docs: document built-in providers and gate prerequisite --- .../object_structure/classification.mdx | 29 +++++++++++++++++++ .../object_structure/memory.mdx | 1 + 2 files changed, 30 insertions(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index 927564a21..2a59a64ea 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -72,6 +72,30 @@ written in `librechat.yaml`; only the name of the variable is. ]} /> +### Built-in providers + +`http` posts the question set to whatever endpoint `baseURL` names and expects the +answers back in the same vocabulary the table above uses. A deployment running its +own classifier, or one fronted by a small adapter service, needs nothing more than +this. + +`typesafe` talks to the TypeSafe System One API, which uses its own name for a +yes/no question and appends `/systemone` to the base URL. It defaults to +`https://api.typesafe.ai/v1` and the `jev-latest` model, so only the key variable +has to be set: + +```yaml filename="TypeSafe provider" +classification: + enabled: true + provider: typesafe + providers: + typesafe: + apiKeyEnv: TYPESAFE_API_KEY +``` + +Adding another provider is one file implementing the same interface plus one +registry entry; nothing in the features above changes. + ## toolSelection An agent holding a large MCP catalog pays for every tool's argument schema on every @@ -109,6 +133,11 @@ round trip rather than an answer. The memory model runs after every turn, and most turns carry nothing durable. The gate asks one yes/no question first and skips that call when the answer is no. +It sits in front of the automatic memory agent, so it requires +[`memory`](/docs/configuration/librechat_yaml/object_structure/memory) to be +configured with `memory.agent.enabled: true`. Without automatic extraction there is +no call to skip, and the gate never runs. + A gate that cannot reach its provider answers yes, so an outage never causes a memory to be lost. diff --git a/content/docs/configuration/librechat_yaml/object_structure/memory.mdx b/content/docs/configuration/librechat_yaml/object_structure/memory.mdx index 5fd04c908..a15d96b32 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/memory.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/memory.mdx @@ -370,3 +370,4 @@ memory: - Agent configuration allows customization of automatic memory processing behavior - When disabled, all memory features are turned off regardless of other settings - The message window size affects how much recent context is considered for memory updates +- [`classification.memoryGate`](/docs/configuration/librechat_yaml/object_structure/classification#memorygate) can skip the automatic memory agent on turns that carry nothing durable, and suggest which valid key a memory belongs under From c29f88563ec5bb9fa792f36a42606bbfbd9fad0f Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Tue, 22 Sep 2026 16:22:25 +0900 Subject: [PATCH 05/12] docs: name the models that can classify --- .../object_structure/classification.mdx | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index 2a59a64ea..1f8a7a979 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -96,6 +96,54 @@ classification: Adding another provider is one file implementing the same interface plus one registry entry; nothing in the features above changes. +### What can answer these questions + +This expects a classification model, not a chat model. The difference is not the +protocol but what comes back: a classifier returns a probability it was trained to +calibrate, and every threshold on this page reads one. A chat model asked to report +its own confidence returns a number that looks the same and means much less. + +That narrows the field considerably. Two model families answer caller-defined +questions with calibrated probabilities: + + + +Laya ships as weights, so it needs a small service in front of it that speaks the +shape above. Once that exists it is the `http` provider and nothing else: + +```yaml filename="A self-hosted classifier" +classification: + enabled: true + provider: http + providers: + http: + baseURL: http://laya.internal:8080/classify + apiKeyEnv: CLASSIFIER_API_KEY +``` + +Three things that look like they should work and do not: + +- **Chat completion endpoints**, including OpenAI, Anthropic and the general + OpenRouter catalog. No provider here speaks chat completions, and nothing reads + logprobs. +- **Moderation endpoints** such as OpenAI's or Mistral's. They return scores over a + fixed taxonomy and cannot answer a question you define. +- **Zero-shot classification by embedding similarity**, such as Jina's classifier or + a hosted NLI model. These do take caller-defined labels, but the scores are + similarities and nobody claims they are calibrated. They will work in the + mechanical sense and quietly make the thresholds meaningless. + + + If a provider cannot measure confidence it should report it as absent rather than + as a number. Tool selection treats absent confidence as unconfident and widens the + shortlist, which is the safe direction; a fabricated `0` or `1.0` defeats that. + + ## toolSelection An agent holding a large MCP catalog pays for every tool's argument schema on every From 91a3a94220a11e09ebbc67b870a7065e6f658510 Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Tue, 22 Sep 2026 21:16:30 +0900 Subject: [PATCH 06/12] docs: describe classification providers as presets --- .../object_structure/classification.mdx | 82 ++++++++++++++----- 1 file changed, 63 insertions(+), 19 deletions(-) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index 1f8a7a979..fd9bef98b 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -50,7 +50,7 @@ classification: -### Built-in providers +### Presets -`http` posts the question set to whatever endpoint `baseURL` names and expects the -answers back in the same vocabulary the table above uses. A deployment running its -own classifier, or one fronted by a small adapter service, needs nothing more than -this. +The hosts that serve a classification model over HTTP differ only in their URL, +their model name and how they wrap the body. That is configuration rather than +behaviour, so each one ships as a preset: a set of defaults you can name, and +override field by field. -`typesafe` talks to the TypeSafe System One API, which uses its own name for a -yes/no question and appends `/systemone` to the base URL. It defaults to -`https://api.typesafe.ai/v1` and the `jev-latest` model, so only the key variable -has to be set: + + +Naming a preset is usually the whole configuration: -```yaml filename="TypeSafe provider" +```yaml filename="A preset, with nothing to add" classification: enabled: true provider: typesafe @@ -93,8 +102,41 @@ classification: apiKeyEnv: TYPESAFE_API_KEY ``` -Adding another provider is one file implementing the same interface plus one -registry entry; nothing in the features above changes. +`cloudflare` is the exception, because its URL carries your account id. Without +one it stays off and says so rather than guessing: + +```yaml filename="Cloudflare" +classification: + enabled: true + provider: cloudflare + providers: + cloudflare: + baseURL: https://api.cloudflare.com/client/v4/accounts//ai/run + apiKeyEnv: CLOUDFLARE_API_TOKEN +``` + +### A host with no preset + +A host nobody has written a preset for needs no code either, only its shape: + +```yaml filename="Any other host" +classification: + enabled: true + provider: inhouse + providers: + inhouse: + baseURL: https://classify.internal/v1/run + model: your-model + dialect: systemone + requestKey: input + responseKey: result + apiKeyEnv: INHOUSE_CLASSIFIER_KEY +``` + + + A provider entry rejects keys it does not recognise, so `requestkey` fails at + config load instead of being dropped and surfacing later as a missing setting. + ### What can answer these questions @@ -113,16 +155,18 @@ questions with calibrated probabilities: ]} /> -Laya ships as weights, so it needs a small service in front of it that speaks the -shape above. Once that exists it is the `http` provider and nothing else: +Laya ships as weights, so it needs a small service in front of it. It implements +the same three question shapes, so `dialect: systemone` describes it and no code +is involved: ```yaml filename="A self-hosted classifier" classification: enabled: true - provider: http + provider: laya providers: - http: + laya: baseURL: http://laya.internal:8080/classify + dialect: systemone apiKeyEnv: CLASSIFIER_API_KEY ``` From f7f9e3111216c4e891c70449269da0d0b2bb2fec Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Wed, 23 Sep 2026 09:32:47 +0900 Subject: [PATCH 07/12] docs: document the per-capability classifier timeouts --- .../librechat_yaml/object_structure/classification.mdx | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index fd9bef98b..cedc9b914 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -206,6 +206,7 @@ round trip rather than an answer. `categorize` and `detectUpdates` ride in the request the gate already makes, so -they cost no additional round trip. The result reaches the memory model as a +they cost no additional round trip. + +Each capability can set its own `timeoutMs` because one ceiling does not suit +both. Ranking a large catalog is a request of several thousand tokens; the gate +asks one short question about the recent turn. The result reaches the memory model as a suggestion it may ignore, not as an override: ```text From 1e3f639c2b68e332a3204bb24e8720987eaa20bd Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Wed, 23 Sep 2026 09:45:21 +0900 Subject: [PATCH 08/12] docs: explain what makes ranking confidence low --- .../librechat_yaml/object_structure/classification.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index cedc9b914..aeef11b46 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -296,3 +296,9 @@ them. Two properties are worth understanding before enabling either capability: - Probabilities are only useful when the provider produces calibrated ones. A provider that cannot measure confidence reports it as absent, and tool selection treats that as unconfident and widens the shortlist instead of narrowing it. +- Low confidence usually means two candidates are close, not that the answer is + poor. A catalog holding several tools that genuinely could serve the same request + will report low confidence often, which is the case `lowConfidenceExtra` exists + for: surfacing both costs one extra schema, while surfacing the wrong one alone + costs a `tool_search` round trip. Raise `lowConfidenceBelow` if your catalog has + many near-duplicates, lower it if its tools are sharply distinct. From aa26860817a12a2e256e44c5f471358a194d7df6 Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Thu, 24 Sep 2026 12:28:04 +0900 Subject: [PATCH 09/12] docs: rename deferSchemaChars and point tool result caps at the existing setting --- .../object_structure/agents.mdx | 29 ------------------- .../object_structure/classification.mdx | 19 ++++++++---- .../object_structure/mcp_settings.mdx | 20 +++++++------ .../shared_endpoint_settings.mdx | 10 ++++--- 4 files changed, 30 insertions(+), 48 deletions(-) diff --git a/content/docs/configuration/librechat_yaml/object_structure/agents.mdx b/content/docs/configuration/librechat_yaml/object_structure/agents.mdx index 322a65950..2bda69bde 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/agents.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/agents.mdx @@ -90,35 +90,6 @@ endpoints: > This configuration enables the builder interface for agents. -## maxToolResultChars - - - -**Default:** `0`, which keeps every result whole. - -**Example:** - -```yaml filename="endpoints / agents / maxToolResultChars" -maxToolResultChars: 28800 -``` - -Tool output is often the largest thing in a conversation, and the size is -usually concentrated in a few calls rather than spread evenly. Because the cap -is applied before the result is stored, every later turn re-sends the capped -version, so the saving repeats for the life of the conversation. - -The model is told the result was cut, so it can call the tool again more -narrowly instead of working from partial data without knowing it. - ## recursionLimit @@ -197,7 +201,7 @@ turn. Deferring those schemas solves the cost, but the model then spends a Tool selection ranks the deferred tools against the request before the model runs and surfaces the few it is likely to need, so their schemas arrive with the first call. It requires tools that are already deferred, whether by the per-tool toggle in -the agent builder or by [`mcpSettings.deferSchemaBytes`](/docs/configuration/librechat_yaml/object_structure/mcp_settings). +the agent builder or by [`mcpSettings.deferSchemaChars`](/docs/configuration/librechat_yaml/object_structure/mcp_settings). Selection only ever adds. A tool it passes over stays listed by name in the `tool_search` description and remains one search away, so a wrong prediction costs a @@ -206,7 +210,7 @@ round trip rather than an answer. @@ -203,21 +203,23 @@ A hostname entry trusts whatever IP that hostname resolves to at runtime on the - [MCP Features](/docs/features/mcp) - [Actions allowedAddresses](/docs/configuration/librechat_yaml/object_structure/actions#allowedaddresses) (similar concept for Actions) -## deferSchemaBytes +## deferSchemaChars A small number of MCP tools can account for most of a catalog's size. A tool whose argument schema is very large costs those tokens on every turn, even when the conversation has nothing to do with it, while its description may be only a line. -Setting `deferSchemaBytes` withholds the argument schema of any tool above that -size. The model still sees the tool's name and description in the `tool_search` +Setting `deferSchemaChars` withholds the argument schema of any tool whose JSON is +longer than that many characters. Characters track the tokens the rule saves more +closely than bytes would. The model still sees the tool's name in the `tool_search` listing and loads the schema when it decides to use it, so nothing becomes -unreachable. +unreachable. The rule only applies when the agents `deferred_tools` capability is +enabled; with it off, every tool loads in full. Precedence, highest first: 1. A per-tool `defer_loading` toggle in the agent builder, in either direction. -2. `mcpSettings.deferSchemaBytes`, when the tool has no explicit toggle. +2. `mcpSettings.deferSchemaChars`, when the tool has no explicit toggle. 3. Otherwise the tool is loaded in full. Because an explicit per-tool choice always wins, the rule is safe to apply diff --git a/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx b/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx index 169472939..e803af7b0 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx @@ -490,18 +490,20 @@ When Langfuse tracing is enabled, the bounded reasoning snapshot is recorded as [ 'maxToolResultChars', 'Number', - 'Limits the maximum number of characters in tool call results sent to the model. Must be a positive number.', + 'Limits the number of characters of a tool call result kept in the conversation. Must be a positive number.', 'maxToolResultChars: 50000', ], ]} /> -**Default:** No limit +**Default:** 30% of the model's context window, at most 400,000 characters **Notes:** -- Helps prevent excessively large tool outputs from consuming too many tokens -- Applies to all tool call results for the endpoint +- A longer result is cut before the model sees it, with a marker telling the model it was cut +- The cut result is what gets stored, so every later turn re-sends the shorter version +- Applies to all tool call results for the endpoint, including MCP tools +- Set under `all` for every endpoint, or on one endpoint to override `all` **Example:** From 6f1672c28b599a7c67b204d5f8e0e8ad86111181 Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Thu, 24 Sep 2026 13:40:10 +0900 Subject: [PATCH 10/12] docs: gate memory on explicit requests and show size-deferred tools in the builder --- .../object_structure/classification.mdx | 40 ++++++++++++++----- .../object_structure/mcp_settings.mdx | 4 ++ .../object_structure/memory.mdx | 2 +- 3 files changed, 35 insertions(+), 11 deletions(-) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index 50b872c48..c85ec735f 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -44,7 +44,7 @@ classification: memoryGate: enabled: true - threshold: 0.25 + threshold: 0.5 ``` ## Configuration @@ -57,7 +57,7 @@ classification: ['provider', 'String', 'Which entry under `providers` answers. A name with a preset needs no more than its key; a name with no preset works too, as long as its entry gives a `baseURL`.', 'http'], ['providers', 'Object', 'Settings per provider, keyed by provider name.', '{}'], ['toolSelection', 'Object', 'Surfaces the deferred MCP tools a turn is likely to need.', 'Disabled.'], - ['memoryGate', 'Object', 'Skips the memory model on turns that carry nothing worth remembering.', 'Disabled.'], + ['memoryGate', 'Object', 'Skips the memory model on turns that do not ask to remember, update or forget anything.', 'Disabled.'], ]} /> @@ -227,9 +227,17 @@ round trip rather than an answer. ## memoryGate -The memory model runs after every turn, and most turns carry nothing durable. The +The memory model runs after every turn, and most turns ask it for nothing. The gate asks one yes/no question first and skips that call when the answer is no. +By default the question is whether the user's newest message asks the assistant to +remember, update or forget something. That matches the default memory +instructions, which only store what the user asks to keep: a preference mentioned +in passing is skipped by both, while "please remember...", "forget where I live" +and a short yes to an offer to remember all go through. If your +`memory.instructions` also keep facts the user did not ask to keep, replace the +question to match (see [Customizing the questions](#customizing-the-questions)). + It sits in front of the automatic memory agent, so it requires [`memory`](/docs/configuration/librechat_yaml/object_structure/memory) to be configured with `memory.agent.enabled: true`. Without automatic extraction there is @@ -238,18 +246,19 @@ no call to skip, and the gate never runs. A gate that cannot reach its provider answers yes, so an outage never causes a memory to be lost. -The gate reads the recent messages newest first, so the turn that just happened -is always part of what it judges, even after a long tool result. +The gate judges only the user's newest message. The few messages before it are sent +along as context, so "yes, remember that" is read against the offer it answers, but +an earlier request is not judged again on the next turn. -The durable part of this turn most likely belongs under `work_context`. It looks +This request most likely belongs under `work_context`. It looks like a change to what is already stored there, not a new fact. Ignore this if it does not fit what the user actually said. @@ -288,11 +297,22 @@ classification: instructions: Pick the tool for this support ticket. guidance: Prefer billing tools for anything about an invoice. memoryGate: - instructions: Is there a dietary requirement here? + instructions: Does `latest` mention a dietary requirement? whenTrue: An allergy or a standing preference. whenFalse: Anything about one meal only. ``` +A deployment whose `memory.instructions` keep anything worth remembering, not only +what the user asks to keep, can ask about that instead: + +```yaml filename="Gate on anything worth remembering" +classification: + memoryGate: + instructions: Does `latest` say something about the user that would still matter weeks from now? + whenTrue: A lasting preference, or a fact about who they are or what they work on. + whenFalse: Small talk, or a detail that only matters inside this task. +``` + ## Notes on accuracy A classifier returns probabilities, and the thresholds above decide what to do with diff --git a/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx b/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx index 079c6a256..f7380b3b2 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/mcp_settings.mdx @@ -224,3 +224,7 @@ Precedence, highest first: Because an explicit per-tool choice always wins, the rule is safe to apply instance-wide without overriding tools an author deliberately pinned open. + +In the agent builder, a tool deferred by this rule shows its defer toggle as on, +with a tooltip saying it was deferred automatically. Switching it off keeps that +tool loaded for the agent; switching it back on returns the tool to the rule. diff --git a/content/docs/configuration/librechat_yaml/object_structure/memory.mdx b/content/docs/configuration/librechat_yaml/object_structure/memory.mdx index a15d96b32..69b5b18fc 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/memory.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/memory.mdx @@ -370,4 +370,4 @@ memory: - Agent configuration allows customization of automatic memory processing behavior - When disabled, all memory features are turned off regardless of other settings - The message window size affects how much recent context is considered for memory updates -- [`classification.memoryGate`](/docs/configuration/librechat_yaml/object_structure/classification#memorygate) can skip the automatic memory agent on turns that carry nothing durable, and suggest which valid key a memory belongs under +- [`classification.memoryGate`](/docs/configuration/librechat_yaml/object_structure/classification#memorygate) can skip the automatic memory agent on turns that do not ask to remember, update or forget anything, and suggest which valid key a request concerns From 816d28c8f3dc6f2a1959308077e285b7eb6d5fb0 Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Thu, 24 Sep 2026 14:11:12 +0900 Subject: [PATCH 11/12] docs: note what too low a tool result cap costs --- .../librechat_yaml/object_structure/shared_endpoint_settings.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx b/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx index e803af7b0..fc8b58353 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx @@ -504,6 +504,7 @@ When Langfuse tracing is enabled, the bounded reasoning snapshot is recorded as - The cut result is what gets stored, so every later turn re-sends the shorter version - Applies to all tool call results for the endpoint, including MCP tools - Set under `all` for every endpoint, or on one endpoint to override `all` +- Too low a cap costs calls instead of saving tokens: a model that cannot see the part of a result it needs searches again in smaller pieces. For tools that return whole pages, such as browser tools, start around 28,800 rather than a few thousand **Example:** From 6d9621e044d9b93027831f8a4c80ef5931dbf978 Mon Sep 17 00:00:00 2001 From: Ali Gulzar Date: Thu, 24 Sep 2026 14:24:21 +0900 Subject: [PATCH 12/12] docs: note how classifier usage is billed --- .../librechat_yaml/object_structure/classification.mdx | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx index c85ec735f..e9fd9a022 100644 --- a/content/docs/configuration/librechat_yaml/object_structure/classification.mdx +++ b/content/docs/configuration/librechat_yaml/object_structure/classification.mdx @@ -23,6 +23,11 @@ selection sends the latest request and the deferred tools' descriptions; the mem gate sends the last few messages. Point `baseURL` at a host you are allowed to send that data to. +Classifier usage is recorded as transactions with the context `classification`, so it +appears in token usage and draws on the balance like any other model call. TypeSafe's +Jev is priced at $0.042 per million input tokens with free output; a model with no +known price is charged at the default rate. + ## Example ```yaml filename="Classification Object Structure"