Skip to content

Register WebMCP tools with cache-stable schemas and frozen targets - #9

Merged
jo32 merged 2 commits into
mainfrom
codex/webmcp-direct-tools
Sep 23, 2026
Merged

jo32 merged 2 commits into
mainfrom
codex/webmcp-direct-tools

Conversation

@jo32

@jo32 jo32 commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Problem and behavior

Follow-up to #8, which is already merged. WebMCP calls previously required the model to copy a session-local toolRef into a generic execution wrapper. This change registers each discovered capability as a scoped webmcp__ tool with its business parameter schema, so the model supplies the business arguments directly.

Tool names derive from canonical capability content rather than random session/document IDs. Existing declarations survive navigation and loading; currently available tools are reported through append-only runtime context. A bounded historical catalog avoids accumulating unlimited obsolete declarations.

Each model step separately freezes its execution bindings at agent/pre-step. Reading browser context or previewing a prompt cannot retarget a call already generated for another page. Dispatch still validates tab, document, frame, revision and schema; stale calls are rejected before dispatch, and uncertain actions are never automatically replayed. The runtime implementation stays in the Cordis browser plugin; upstream Harness source and unrelated Jev work are excluded.

Compatibility: browser_webmcp_call and its legacy identity arguments are removed. Browser discovery now returns callName; Builder guidance and DevTools guidance use the registered business tools. Benchmark diagnostics count both historical wrapper calls and new direct calls.

Benchmark evidence and limitations

Six selected WebMCP ON cases, three attempts per case/model. These are historical comparisons, not a full-suite or concurrent randomized A/B result. USD estimates use actual uncached input, cache reads, cache writes and output separately; Luna is API-equivalent pricing.

Compared with the short-reference implementation from #8:

Per-six-case mean unless stated MiMo: short reference → final Luna: short reference → final
Passes across 18 attempts 18/18 → 17/18 18/18 → 18/18
Weighted cache hit rate 90.5% → 79.0% 76.1% → 54.5%
Estimated USD $0.026331 → $0.020288 $0.036244 → $0.030326
Agent time 345.2 → 323.3 sec 181.3 → 91.3 sec
Agent steps 60.3 → 34.3 41.7 → 24.7

The intermediate direct-tool experiment had cache hit rates of 68.8%/53.7%; stable declarations recover these to 79.0%/54.5%, not to the short-reference baseline. Newly discovered capabilities still change the top-level tool list. MiMo also becomes slower than that intermediate experiment (194.3 → 323.3 sec), mainly in the order task.

MiMo's hev-1 third attempt returned an empty tool name/ID, then the next request failed with HTTP 400. It remains in the results without replacement, and early termination makes the aggregate cost unsuitable as a same-success-quality claim. Across the other five tasks, all three repeats pass; mean cost is $0.026222 → $0.018658 compared with the intermediate direct-tool run. The malformed-call handling is not fixed here.

The report also preserves individual cost/cache regressions, two interrupted infrastructure starts before any model attempt, frozen model/configuration/source checks, and the unused historical Jev-tool schema difference. All 12 course/order completion audits verify successful tool results rather than relying on final prose.

Sanitized per-attempt datasets are included; raw conversations, credentials and generated build outputs are not committed.

Validation and review

  • pnpm check and pnpm build passed.
  • Full pnpm test passed with NODE_OPTIONS=--no-experimental-webstorage. Default Node 25 initially failed an unrelated jsdom localStorage.clear test; the environment flag resolves that conflict without changing business code.
  • Browser plugin: 172 tests across 19 files, including real scoped Cordis assembly and model-step boundaries.
  • Real Electron: native/generated tool execution, stale-target rejection after navigation, stable names across same-schema revision changes.
  • All 36 formal attempts have complete usage; runtime source/build hashes stayed unchanged throughout and still match the committed source. All owned benchmark instances stopped.

Code/tests and benchmark documentation are separate commits for review. The benchmark JSON is verbose; start with the runtime/bindings diff and the Markdown report. This PR is for review, with no main merge or deployment performed.

@vercel

vercel Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
deepdeck Ready Ready Preview Sep 23, 2026 4:12pm UTC

Request Review

@jo32
jo32 merged commit 43363dc into main Sep 23, 2026
4 checks passed

This branch was successfully deployed

1 active deployment
Preview — 5bc0e57a Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant