Skip to content

fix(idempotency): define payload identity per operation class - #782

Open
vishkaty wants to merge 4 commits into
Universal-Commerce-Protocol:mainfrom
vishkaty:fix/idempotency-payload-identity
Open

fix(idempotency): define payload identity per operation class#782
vishkaty wants to merge 4 commits into
Universal-Commerce-Protocol:mainfrom
vishkaty:fix/idempotency-payload-identity

Conversation

@vishkaty

@vishkaty vishkaty commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Motivation

#664: idempotency payload matching is defined as a SHA-256 hash over the raw HTTP body bytes. Over MCP the signed body is the full JSON-RPC envelope, and the MCP base protocol requires the request id to not repeat within a session, so a compliant client retry can never be byte identical to the original request. Raw-body hashing therefore treats every legitimate retry of complete_checkout and cancel_checkout as a mismatched payload and rejects it without executing, on exactly the two operations idempotency exists to protect. This is not an edge case with a crafted client. It follows from the base MCP protocol as written, for any conforming client.

A second, related gap: the idempotency key can appear in two places on a signed MCP request, the covered Idempotency-Key header and meta["idempotency-key"] in the tool arguments, with no stated rule for which one governs when they disagree.

The resolution

This PR implements the class split proposed by YanisMtcr in the #664 thread, building on the raw-arguments direction from westonale-facet (envelope excluded):

  1. Operations whose arguments carry nothing beyond meta and the target resource id are decidable as a class from the operation's input schema alone. cancel_checkout is in this class: its OpenRPC params are exactly meta and id. For this class, payload identity is the pair (idempotency key, target resource id). A matching pair returns the stored result; a matching key with a different resource id is rejected without executing. No hashing, no canonicalization.
  2. Operations whose arguments carry anything more are the payload-carrying class. complete_checkout is in this class: its OpenRPC params add a checkout object. For this class, payload matching stays a SHA-256 hash, now scoped to the operation's arguments with the transport envelope excluded (for MCP the params.arguments object with its top level meta member removed, serialized with JCS per RFC 8785; for REST the request body), not the JSON-RPC message bytes. For REST the hashed input remains the raw body bytes, so the existing Content-Digest linkage is unchanged.
  3. When the request is signed, the covered Idempotency-Key header is authoritative and any payload copy of the key must equal it; a mismatch is rejected without executing. On unsigned requests, the payload copy alone is authoritative. This rule lives in signatures.md (Idempotency Key Placement), because a payload copy of the key exists wherever a signed MCP request carries meta["idempotency-key"] (cancel_cart requires it too, not just the checkout operations); the checkout MCP binding restates it where both locations are visible in the examples.

Exact changes

  • docs/specification/signatures.md (Idempotency Key Placement, under Replay Protection): adds the key location rule, binding general.
  • docs/specification/signatures.md (Payload Matching, same section): replaces the single raw-body-hash rule with the two-class definition above; the fresh-key-on-payload-change guidance is retained, scoped to the payload-carrying class.
  • docs/specification/shopping/checkout/mcp.md (Request Metadata): restates the key location rule for this binding.
  • docs/specification/shopping/checkout/rest.md and docs/specification/shopping/cart/rest.md (Idempotency-Key header items): the duplicate-key items now say payload rather than request body. This is a consequence of the class split, not new scope: those items defer to signatures.md for the full payload-matching contract, and for cancel operations (no request body) the old body-matching summary would contradict the pair rule — a reused key with a different target previously counted as a matching payload because two empty bodies hash equal, and returned the cached response of a different resource; under the class split it is rejected without executing.

No schema files change. cancel_cart shares the same params shape as cancel_checkout (meta, id) in source/services/shopping/mcp.openrpc.json, so it falls under the target-only class through the same schema-general rule in signatures.md, with no separate edit needed in the cart docs.

Related work

#679 documents the MCP server contract by reference to the REST binding and the overview error registry, at docs/specification/checkout-mcp.md, a path removed by the #723 specification reorg (current location is docs/specification/shopping/checkout/mcp.md). Because #679 documents by reference rather than restating the contract, it stays compatible with this PR once rebased onto the current paths: its reference to Replay Protection picks up the class split here.

Testing

  • python3 scripts/validate_examples.py --schema-base source/schemas/: 343 passed, 0 failed, 0 errors, 50 skipped on this branch, identical to a clean checkout of main at the base SHA.
  • ucp-schema lint source/: 124 files checked, all passed. No schema files were touched by this PR.
  • Relative links added in the diff checked by hand against the built anchors: docs/specification/shopping/checkout/mcp.md#message-signing and docs/specification/signatures.md#replay-protection, both existing headings, unchanged by this PR.
  • Full-tree conflict marker sweep: none found.

Amendment: the key binding

Probing both reference implementations against a literal reading of the class rules surfaced two gaps in the first commit, both closed by the second: the class 2 identity omitted the REST path identifier, which would have recreated for updates the cross resource replay the class 1 rule fixes for cancels, and cross operation key reuse was undefined while update_checkout and complete_checkout share identical argument shapes. The section now opens with a binding rule, a stored key binds the operation it was first used with and the target resource identifier where one exists, and any reuse across that binding is rejected without executing, before payload comparison. Both reference implementations already persist exactly this operation and resource scoping for checkout operations, so the binding codifies existing practice rather than adding new state.

Amendment: JCS for the payload-carrying MCP hash

The review from westonale-facet is right that removing meta is an operation on a parsed value, so an unnamed per implementation re serialization was standing in for a canonical form, and member order variance across attempts would recreate the guaranteed retry mismatch on the MCP side. The payload carrying MCP hash input is now the params.arguments object with its top level meta member removed, serialized with JCS (RFC 8785), the canonicalization the specification already uses for AP2 mandate artifacts; the no canonicalization statement is scoped to the REST arm, where the hashed input really is the received byte string. Checked against an executable implementation of the full rule: member order permutations, whitespace variants, unicode escapes versus literals, and number forms (1 versus 1.0 versus 1e0) hash identically, while changed payloads, nested meta members inside checkout, cross operation reuse, and cross target reuse are all rejected.

Raw-body SHA-256 matching treats every MCP retry as a payload mismatch,
because the JSON-RPC envelope (id, and meta when unpinned) varies per
request while the logical operation does not. This affects exactly the
operations idempotency exists to protect: complete_checkout and
cancel_checkout.

Replaces the single raw-body-hash rule in signatures.md (Payload
Matching, under Replay Protection) with two classes decided from the
operation's input schema:

- Target-only operations (arguments carry nothing beyond meta and the
  target resource identifier, e.g. cancel_checkout, cancel_cart):
  identity is (idempotency key, target resource identifier); no
  hashing.
- Payload-carrying operations (e.g. complete_checkout): hash over the
  operation's arguments with the transport envelope excluded, not the
  JSON-RPC message bytes. For REST the hashed input remains the raw
  body bytes, the same digest RFC 9530 mandates as Content-Digest.

States, in signatures.md (Idempotency Key Placement) and restated in
the checkout MCP binding, which key location is authoritative when a
signed request carries the key in both the covered Idempotency-Key
header and a payload copy such as meta["idempotency-key"].

Aligns the duplicate-key items in the two REST binding summaries
(checkout, cart) with the class split: they now say payload rather
than request body, deferring to the payload-matching contract they
already reference.

Refs Universal-Commerce-Protocol#664.
Wire probes of both reference implementations against a literal reading
of the class rules surfaced two gaps: the class 2 identity omitted the
REST path identifier, recreating for updates the cross resource replay
the class 1 rule fixes for cancels, and cross operation key reuse was
undefined while update and complete share identical argument shapes.
A stored key now binds its operation and target resource identifier,
both classes resolve reuse mismatches by rejecting before any payload
comparison, and the class definitions are unchanged. Both references
already persist exactly this binding, so the rule codifies existing
practice.
@westonale

Copy link
Copy Markdown

The class split is right and the covered header as authoritative is the correct resolution of the two-location problem. One thing in the payload-carrying clause does not hold up, and it reintroduces the failure #664 was filed against.

The text defines the MCP hash over "the params.arguments object with meta removed" while claiming that "no cross-party canonical form is defined or needed, which preserves the spec's no-canonicalization posture". Those two are in tension. You cannot remove a member from a byte string; you can only remove it from a parsed value and then re-serialize. That re-serialization is a canonical form, just an unnamed and per-implementation one.

The cross-party argument does not rescue it, because the Business is not comparing against another party. It is comparing two of its own hashes computed from two different received byte streams, the original and the retry. Those streams can legitimately differ inside params.arguments: member order is not preserved by serializers that build from a hash map, and a client that reconstructs the arguments per attempt has no obligation to emit them in the same order. Same logical arguments, different member order, different hash, 409 Conflict on a valid retry. That is the guaranteed-retry-mismatch bug the issue opened on, moved from the envelope into the arguments.

The REST arm does not have this problem, since it hashes the raw body bytes as received, which is genuinely byte-level. The asymmetry is only on the MCP side, which is the side the issue was about.

Two ways out, and either is fine as long as the text says which. Hash the raw bytes of the params.arguments value as received and handle meta by excluding it at the sender's framing rather than by receiver-side removal, which keeps the no-canonical claim true. Or adopt JCS explicitly for this class and drop the no-canonicalization claim for it, which is the stronger variant and tolerates re-serialization across client stacks. What does not work is receiver-side member removal described as a raw-byte operation.

Review on the PR identified that removing meta from params.arguments is
an operation on a parsed value, not on bytes, so describing the hash
input in raw byte terms hid an unnamed per implementation canonical
form, and member order variance across retries would reproduce the
guaranteed retry mismatch the issue opened on. The payload carrying
class now serializes the arguments object, with its top level meta
member removed, using the JSON Canonicalization Scheme per RFC 8785,
the canonicalization this specification already uses for AP2 mandate
artifacts. The no canonicalization statement is scoped to REST, where
the hashed input really is the single byte string received.
@vishkaty

Copy link
Copy Markdown
Contributor Author

Adopted the stronger variant: the MCP hashed input is now the params.arguments object with its top level meta member removed, serialized with JCS per RFC 8785, the canonicalization the specification already uses for AP2 mandate artifacts, and the no canonicalization claim is scoped to REST where the body really is a single byte string as received. You are right that receiver side member removal was a parsed value operation wearing raw byte language, and the per attempt member order case makes it the same guaranteed retry mismatch the issue opened on, so this class now says what it does.

The MCP Transport section stated no JSON canonicalization is required
without qualification. That is true for signing, where Content-Digest
binds the raw JSON-RPC body, but this PR defines JCS canonicalization
for payload matching in the payload carrying class one section above.
Scoping the sentence to signing removes the apparent contradiction and
points at the Replay Protection rules for the matching path.
@vishkaty

Copy link
Copy Markdown
Contributor Author

Scoped the no canonicalization sentence in the MCP Transport section so it reads against signing rather than payload matching: Content-Digest still binds the raw JSON-RPC body with no canonicalization, while the payload-carrying class uses JCS as defined under Replay Protection. Without that scope the two statements sat one section apart and read as contradictory.

@carolinerg1 carolinerg1 added status:under-review and removed status:needs-triage Signal that the PR is ready for human triage labels Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants