Skip to content

fix(tools): type blueprint/scheduling schemas and document state traps in tool contracts (WM-4172) - #73

Merged
Andrii Chumak (andriichumak) merged 6 commits into
mainfrom
fix/mcp-level1-tool-contracts
Jul 20, 2026
Merged

fix(tools): type blueprint/scheduling schemas and document state traps in tool contracts (WM-4172)#73
Andrii Chumak (andriichumak) merged 6 commits into
mainfrom
fix/mcp-level1-tool-contracts

Conversation

@andriichumak

@andriichumak Andrii Chumak (andriichumak) commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Level 1 tool-contract fixes: typed blueprint/scheduling, state-trap docs, tool differentiation (2026-07-17, agent-interfaces-self-improve iteration 5, case SX-006)

Finding

type: doc-gap
surface(s): mcp | sdk (MakeTools metadata consumed by mcp.make.com and the CLI)

The six highest-failing MCP tools all carry "legacy dialect" contracts: scenarios_create/scenarios_update accept blueprint and scheduling with no type key at all and one-line descriptions, scenarios_run says nothing about the activation requirement, and executions_get vs executions_get-detail are indistinguishable. Production telemetry (Datadog APM spans, service:mcp-server-host operation_name:mcp.tool_call status:error, 14d windows ending 2026-07-17) shows agents paying for every one of these gaps.

Evidence

  • "Scenario is not activated" on scenarios_run: 10,315 (~28% of all 36.7K run calls) — the natural create→run loop fails because nothing in the contract says runs require activation.
  • Idempotency traps: "Scenario is already running" 2,711 (scenarios_activate), "Scenario is not running" 2,863 (scenarios_deactivate), "Scenario is already being executed" 2,292 (scenarios_run retry loops).
  • Blueprint/scheduling guessing on create/update: invalid scheduling.type 967 (agents invent cron, hour); blueprint missing required metadata 540; "Value exceeded maximum length of 240 chars in parameter 'description'" 1,043; a systematic long tail of invented module slugs (builtin:TextAggregator, util:SetVariables2, google-sheets:searchRows@2, …).
  • executions_get/executions_get-detail: "Value doesn't match pattern in parameter 'executionId'" 673 (format undocumented).
  • teams_list: 676 errors of 1,402 calls (~48%) — org-scoping guidance absent.
  • credential-requests_extend-connection 96.8% error rate (30d): errors when scopes are already present (error-as-answer); credential-requests_delete requestId provenance undocumented.

Full research: make-ai-test-harness/mcp-authoring-failure-modes.md (FM-A, FM-B, FM-C, FM-D).

Mapping

All in src/endpoints/*.tools.ts (the MakeTools metadata that powers tools/list on mcp.make.com via sdk.module auto-registration):

  • scenarios.tools.ts — untyped blueprint/scheduling on create (was line 84-85) and update (was 139-140); one-line descriptions on create/update/run/activate/deactivate.
  • executions.tools.ts — identical descriptions for get vs get-detail (was lines 37/64); untyped executionId.
  • teams.tools.ts:8, folders.tools.ts:8 — no Access-denied recovery.
  • credential-requests.tools.ts (extend-connection was line 430, delete was line 99).

Fix

Description/schema-only changes; no execute behavior touched:

  • blueprint and scheduling get full JSON Schemas mirroring the SDK's own Blueprint/Scheduling TS types (shared consts): scheduling.type enum, interval minimum, additionalProperties: false on scheduling; blueprint requires name/flow/metadata, flow items require id/module/version, module-id format guidance ("verify via app-modules_list — never invent"). The blueprint object and its flow items carry additionalProperties: true — load-bearing, see Host dependency below.
  • Intentional breaking change on the MCP surface: the old examples taught agents to pass blueprint/scheduling as JSON strings; with type: 'object' declared, the host's validation now rejects the string form before it reaches the API (the API's own normalizePayload still accepts strings, but MCP tool calls never get that far). The rejection error ("Expected object") is clear enough to self-correct in one turn, and objects are the contract we want agents on. Examples are converted to objects accordingly.
  • scenarios_update: wholesale-replace warning (fetch with scenarios_get first), tool-scenario exclusion, maxLength: 240 on description.
  • scenarios_run: activation requirement, executionId → executions_get-detail before retrying, concurrent-run trap, data-keys-must-match-interface, responsive semantics.
  • scenarios_activate/deactivate: already-in-state errors documented as success — do not retry.
  • executions_get vs executions_get-detail: differentiated (metadata-only vs per-module I/O + "ALWAYS call after a failed run"); executionId documented with pattern: ^[0-9a-f]{32}$.
  • teams_list/folders_list: Access-denied recovery via users_me. (Making organizationId optional requires backend support in imt-web-api — out of scope here, tracked as a Level-3 item.)
  • credential-requests_extend-connection: "all scopes already present" error documented as satisfied-requirement; exact scope strings via credential-requests_list-app-modules-with-creds. credential-requests_delete: requestId provenance (string ID from create/list, NOT a connectionId).
  • scheduling.type's behavioral guidance lives on the enum property's own description. The host's schema conversion used to drop annotations from enum properties; make-mcp-server-host#338 fixes that, and since #338 is a hard prerequisite of this release anyway (see below), no duplication into the parent description is needed. The enum values always survived conversion (as anyOf consts), so they aren't repeated in prose either.

Host dependency & release ordering (from review)

The host validates every tool call against these schemas via FromSchemaValue.CleanValue.Check. Value.Clean strips undeclared object properties, and real blueprints carry more properties than any schema will ever declare (e.g. a webhook node's listener) — without additionalProperties: true the recommended scenarios_get → edit → scenarios_update loop would silently corrupt blueprints. TypeBox only honors the boolean form after make-mcp-server-host#338 (maps additionalProperties: trueType.Unknown(); also preserves enum-property descriptions).

Ordering: #338 must be deployed on the host before or together with the @makehq/sdk bump that picks up this PR. #338 is a no-op until then, so it can merge anytime; plan is to land #338 now and do the SDK bump in the same release once the npm package is published.

Delivery chain: this PR carries the version bump to 1.6.5 — npm release on merge → make-mcp-server-host bumps @makehq/sdk (with #338 already in) → host deploy. Quick prod mitigation meanwhile: GrowthBook mcp-tool-description_{toolName} overrides.
Companion PR (host-only tools, same finding class): make-mcp-server-host #337. Each PR is independently useful; this one only reaches production through the SDK release + host bump.

Validation

  • npm run lint (tsc + eslint) green; npm test (full jest unit suite, 261 tests) green on the final state of the branch.
  • End-to-end against the host's real validation pipeline (branch dist/tools.cjs + #338's FromSchema + TypeBoxValidationUtil): webhook-blueprint round-trip preserves listener and undeclared top-level keys; invalid scheduling.type, missing blueprint metadata, over-long description, and malformed executionId are all still rejected with actionable errors; scheduling.type description survives onto the converted schema.
  • Post-deploy check: the error signatures above should decay in service:mcp-server-host operation_name:mcp.tool_call status:error spans (re-runnable queries in mcp-authoring-failure-modes.md).

🤖 Generated with Claude Code

Jira: WM-4172

…s in tool contracts

Level 1 contract fixes for the highest-failing MCP tools (Datadog APM,
14d ending 2026-07-17 — see PR body for per-signature volumes):

- scenarios_create/update: full JSON Schemas for blueprint (required
  name/flow/metadata, module-id format guidance) and scheduling (type
  enum, additionalProperties: false); maxLength 240 on description;
  wholesale-replace warning + tool-scenario exclusion on update;
  examples converted from JSON strings to objects
- scenarios_run: activation requirement, executions_get-detail before
  retry, concurrent-run trap, data/interface and responsive semantics
- scenarios_activate/deactivate: already-in-state errors documented as
  success (do not retry)
- executions_get vs executions_get-detail: differentiated (metadata vs
  per-module I/O); executionId format documented with pattern
- teams_list/folders_list: Access-denied recovery via users_me
- credential-requests_extend-connection/delete: scopes-already-present
  semantics, exact-scope-string source, requestId provenance

Descriptions/schemas only — no execute() behavior changed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@andriichumak
Andrii Chumak (andriichumak) requested a review from a team as a code owner July 17, 2026 16:47
Copilot AI review requested due to automatic review settings July 17, 2026 16:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the SDK’s tool contract metadata (schemas + descriptions) for MCP/CLI consumers, focusing on reducing common agent authoring failures around scenarios (blueprint/scheduling), executions, teams/folders access scoping, and credential-request behaviors.

Changes:

  • Add structured JSON Schemas for scenarios_create / scenarios_update blueprint and scheduling, plus stronger “state trap” documentation for scenario lifecycle tools.
  • Differentiate executions_get vs executions_get-detail behavior in tool descriptions and document executionId format for scenario executions.
  • Document access-denied recovery guidance for teams_list / folders_list and clarify credential-request ID/scope semantics.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/endpoints/scenarios.tools.ts Adds reusable blueprint/scheduling JSON Schemas and expands lifecycle/tool-contract descriptions (activate/deactivate/run/update).
src/endpoints/executions.tools.ts Clarifies get vs get-detail and adds an executionId pattern/description (with one DLQ-specific mismatch noted in review).
src/endpoints/teams.tools.ts Improves teams_list description with org-scoping and access-denied recovery guidance.
src/endpoints/folders.tools.ts Improves folders_list description with access-denied recovery guidance.
src/endpoints/credential-requests.tools.ts Clarifies requestId provenance and “scopes already present” handling guidance.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/endpoints/executions.tools.ts Outdated
…om DLQ executionId

- The extracted scheduling/blueprint schema consts lost contextual
  typing, widening 'object' to string — annotate both as JSONSchema
  and add the missing additionalProperties field to the type (CI
  ts-jest/typedoc caught this; local pipe masked the exit code)
- executions_get-for-incomp-exec: DLQ execution IDs are UUID-shaped
  with dashes (test/mocks/incomplete-executions/get.json), so the
  32-hex pattern would reject valid IDs — removed, format documented
  instead (Copilot review)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jorgecollinet-make

Copy link
Copy Markdown
Contributor

Review — validated locally against the real host validation pipeline

How this was validated: every claim below was reproduced with local runs using only real components — the built dist/tools.cjs of this branch, the host's actual FromSchema (make-mcp-server-host lib/libs/typebox/from-schema.ts, compiled as-is) and the actual TypeBoxValidationUtil from @integromat/api-utils in the host's node_modules. Baseline ("before") = @makehq/sdk 1.6.2 as currently installed in the host. Lint and the full jest suite were also re-run green on this branch (34 suites / 261 tests). Runnable harness at the bottom.

The one piece of context everything hinges on: mcp.make.com doesn't just show these schemas to agents — it validates every tool call against them before executing. For each static tool the host runs (sdk.module.ts:58 → executor in conversation-module.ts:756):

FromSchema(inputSchema)  →  Clone → Convert → Default → Clean → Check

Today blueprint/scheduling have no type, so they convert to Type.Unknown and pass through untouched. This PR gives them real object schemas — which switches all of the behaviors below ON the moment the host bumps the SDK.


🔴 Finding 1 (blocker): Value.Clean silently deletes blueprint keys the schema doesn't list

Plain version: TypeBox's Value.Clean step removes every property that isn't declared in the schema's properties. The new blueprint schema declares 9 node properties — but real blueprints contain more. So the exact workflow this PR's own description recommends becomes lossy:

  1. Agent calls scenarios_get → gets a blueprint whose webhook trigger node has "listener": true (a real BlueprintNode field, src/endpoints/blueprints.ts:82).
  2. Agent edits one mapper field and calls scenarios_update with the complete blueprint — exactly as the new description instructs ("always fetch the current blueprint with scenarios_get first, edit that JSON, and send the complete result").
  3. The host validation silently deletes listener (and any other undeclared key, at blueprint top level too) before the API ever sees it.
  4. The call succeeds — but the stored blueprint is not the one the agent sent. No error, no warning.

Validated run (same round-tripped webhook blueprint through both schema versions):

[BEFORE / sdk 1.6.2 in prod] accepted: true | trigger node keys reaching the API: id, module, version, parameters, mapper, metadata, listener
[AFTER  / PR #73 branch    ] accepted: true | trigger node keys reaching the API: id, module, version, parameters, mapper, metadata
=> listener flag survived? before: true / after: false

The obvious fix doesn't work. Adding additionalProperties: true to the blueprint/node schemas does NOT stop Clean — TypeBox only preserves extra keys when additionalProperties is a real TypeBox schema (e.g. Type.Unknown()), which the host's FromSchema never produces from a boolean. Validated:

with additionalProperties: true on blueprint + flow items — listener survived? false
host FromSchema patched (boolean true -> Type.Unknown())  — listener survived? true

Suggested resolution (two-sided, order matters):

  1. Host: teach FromSchema to map additionalProperties: trueType.Unknown() (fits naturally into #337 or a follow-up). It's a no-op until a schema uses the flag, so it can land anytime.
  2. This PR: add additionalProperties: true to blueprintInputSchema and its flow.items.
  3. Only then release the SDK / bump the host. If this SDK version reaches the host before the FromSchema change, the silent stripping goes live.

(Alternative: keep blueprint description-only in this PR and land the typed schema together with the host fix. The scheduling schema is safe as-is — it's a closed type, stripping unknown keys there is correct.)

🟠 Finding 2: JSON-string blueprint/scheduling now rejected at the host — contradicts the PR body

Plain version: the old examples on these very tools taught agents to pass scheduling/blueprint as JSON strings. With type: 'object' in the schema, the host's Check step now rejects that form before it reaches the API. The PR body says "both remain accepted by normalizePayload" — true at the API layer, but on the MCP surface validation runs first, so strings never get there.

Validated run:

call: scenarios_create with scheduling: '{"type":"indefinitely","interval":900}' (string, per the old example)
[BEFORE / sdk 1.6.2 in prod] accepted: true  (string passes through to the API, which parses it)
[AFTER  / PR #73 branch    ] accepted: false
agent sees: Invalid arguments:
            '/scheduling' - Expected object
            '/blueprint' - Expected object

This is probably a desirable forcing move (objects are the better contract, and the error is clear enough to self-correct in one turn), but it's a breaking change for any agent/integration built on the old string examples — it should be a conscious, documented decision in the PR body rather than an accident, and the "normalizePayload accepts both" claim should be corrected.

✅ Finding 3: the intended fixes genuinely work (validated)

The two big production failure modes this PR targets now fail instantly with actionable errors instead of burning an API round-trip:

scheduling {type:'cron', hour:9}  → rejected at host: "'/scheduling/type' - Expected union value"   (was: accepted, failed at API)
blueprint without metadata        → rejected at host: "'/blueprint/metadata' - Expected required property"
241-char description              → "'/description' - Expected string length less or equal to 240"
UUID execId on executions_get     → "'/executionId' - Expected string to match '^[0-9a-f]{32}$'"
UUID execId on DLQ variant        → accepted ✔ (Copilot's catch, fixed correctly in da756b7)
interval sent as string "900"     → coerced to 900 by Value.Convert (free bonus)

Schema fidelity also checks out: both consts mirror the SDK's own Scheduling/BlueprintNode types exactly (enum values, interval min 60, days/months semantics, node required fields, filter/routes shapes), and teams_list's "requires an organizationId" matches its schema (already required).

🟡 Finding 4 (minor): the scheduling.type guidance never reaches agents

The host's FromSchema converts enum properties via FromEnum, which drops the property's other keywords — including description. So the carefully written guidance is lost on the advertised surface:

SDK source:  "Type of scheduling. 'indefinitely' runs on an interval; 'on-demand' only runs when triggered manually or via scenarios_run."
tools/list:  {"anyOf":[{"const":"immediately","type":"string"},{"const":"indefinitely","type":"string"}, ...]}   ← no description

The enum values themselves survive (validation works — see Finding 3), only the text is lost. Host-side one-liner (FromEnum passing options through), or cheap SDK-side mitigation: fold the type-specific guidance into the parent scheduling description, which does survive.

🟡 Finding 5 (minor): additionalProperties: false on scheduling never actually errors

Clean strips unknown keys before Check runs, so the strictness is silent:

scheduling {type:'daily', time:'09:00', timezone:'Europe/Prague'}  → accepted: true
scheduling passed to the API: {"type":"daily","time":"09:00"}      ← 'timezone' silently dropped

Harmless here (the description text covers it), just be aware it doesn't produce the error it looks like it should.


Summary: descriptions/state-trap text, maxLength, executionId patterns, teams/folders/credential-requests changes and the scheduling schema are ✅ ready. The blueprint schema's properties block is ⛔ until the host FromSchema handles additionalProperties: true — otherwise the scenarios_get→edit→scenarios_update loop silently corrupts blueprints. Companion host PR #337 currently does not touch from-schema.ts (verified against its diff).

Repro harness (node, run from anywhere; adjust the two repo paths)
// NODE_PATH=<host>/node_modules node repro.cjs
// 1. build this branch:                cd make-typescript-sdk && npm run build
// 2. compile the host's FromSchema:    cd make-mcp-server-host && ./node_modules/.bin/esbuild \
//      lib/libs/typebox/from-schema.ts --bundle --format=cjs --platform=node \
//      --external:@sinclair/typebox --outfile=/tmp/host-from-schema.cjs
const HOST = '<path-to>/make-mcp-server-host';
const SDK  = '<path-to>/make-typescript-sdk';
const { FromSchema } = require('/tmp/host-from-schema.cjs');
const { TypeBoxValidationUtil } = require(`${HOST}/node_modules/@integromat/api-utils`);
const { MakeTools } = require(`${SDK}/dist/tools.cjs`);                       // this PR
const { MakeTools: Prod } = require(`${HOST}/node_modules/@makehq/sdk/dist/tools.cjs`); // 1.6.2 baseline

const blueprint = {
    name: 'Webhook to Slack',
    flow: [{ id: 1, module: 'gateway:CustomWebHook', version: 1, parameters: { hook: 12345 },
             mapper: {}, metadata: { designer: { x: 0, y: 0 } }, listener: true }],
    metadata: { version: 1 },
};
const run = (tools, args) => {
    const t = tools.find((t) => t.name === 'scenarios_update');
    try { return { ok: true, payload: TypeBoxValidationUtil.validate(FromSchema(t.inputSchema), args) }; }
    catch (e) { return { ok: false, err: (e.suberrors ?? []).map((s) => `'${s.path}' - ${s.message}`) }; }
};
console.log('before:', Object.keys(run(Prod,      { scenarioId: 925, blueprint }).payload.blueprint.flow[0]));
console.log('after :', Object.keys(run(MakeTools, { scenarioId: 925, blueprint }).payload.blueprint.flow[0]));
// before: [... 'listener'] / after: listener is gone, call still succeeds

🤖 Generated with Claude Code — reviewed via the agent-interfaces-self-improve harness.

….type guidance on the parent description

Blueprints carry more properties than the schema declares (e.g. a webhook
node's `listener`), and the MCP host's validation pipeline strips
undeclared properties from a scenarios_get → edit → scenarios_update
round-trip unless the schema explicitly allows them — silently, with the
call still succeeding. `additionalProperties: true` on the blueprint
object and its flow items keeps those properties intact while declared
properties and required keys stay enforced. Requires host support for the
boolean form (make-mcp-server-host#338) before this reaches production.

The scheduling.type enum values are also spelled out in the parent
scheduling description, since the host's schema conversion drops
annotations from enum properties.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bump version to 1.6.5 for release.

Changes since v1.6.4:
- fix(tools): type blueprint/scheduling schemas and document state traps in tool contracts (#73)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The enum values always reach agents as anyOf consts in the advertised
schema, and the type-specific behavioral guidance on the enum property's
description survives host conversion once make-mcp-server-host#338 (a
hard prerequisite of this release) is deployed — repeating either in the
parent description only adds tokens to every tools/list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…facts

Generic recovery coaching ('instead of retrying with guessed IDs',
spelled-out error-message matching) is behavior the model infers on its
own; the descriptions keep only what it cannot: where the teamId comes
from, and that a team-scoped token cannot list an organization's teams
(teams_get is the alternative).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@andriichumak

Copy link
Copy Markdown
Collaborator Author

jorgecollinet (@jorgecollinet-make) Thanks for the review. The comments are addressed, also added version bump for the SDK. There is a counterpart PR in MCP repo https://github.com/integromat/make-mcp-server-host/pull/338 - once this one is merged - I'll bump the SDK version there and merge it all together.

@andriichumak
Andrii Chumak (andriichumak) merged commit d2431da into main Jul 20, 2026
4 checks passed
@andriichumak
Andrii Chumak (andriichumak) deleted the fix/mcp-level1-tool-contracts branch July 20, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants