feat: add response-carried request constraints - #655
Conversation
7521e4d to
3105ad5
Compare
|
Consolidating the concerns raised across this constraints arc (#288 → #424 → #580 → #626) and where each lands under Legend: ✅ resolved / moot ·
Net: this arc has genuinely come a long way. The debates that took the most back-and-forth — genericity (C6), where enforcement lives (C5), the |
…try ordering (#657) * reserve `ucp` namespace, add map_order for registry ordering UCP registries are reverse-DNS keyed JSON maps, and JSON object member order is not a protocol contract: RFC 8785 (JCS) canonicalization, used by UCP signing, sorts object member names. Businesses have no reliable way to declare preferred traversal order for registry keys, e.g. payment handler presentation order (#170, design discussion in #525). The model: the member name `ucp` is reserved at every object scope as the protocol namespace. The top-level envelope is the root manifestation of that reservation, not a special wrapper. The namespace is ambient: any object scope MAY carry a `ucp` member, domain schemas never declare it (it is document grammar, like the reservation itself), and its contents are defined exclusively by the vocabulary registered in ucp.json#/$defs/members. Consumers process the members they recognize and ignore the rest; a member is admitted to the vocabulary only if it is safe to ignore, so no member can be load-bearing for correctness. One exception: an object closed with additionalProperties:false must declare the optional `ucp` property explicitly. Future members register once and work at every scope immediately -- request constraints (#655) can rebase onto this vocabulary. The first member, map_order, declares key-traversal order for sibling map-valued fields, carried in an array because JCS preserves array element order. Partial lists are valid, unlisted keys remain valid and follow, and the list is not an allowlist. At the root envelope it orders the registries beside it: "ucp": { "payment_handlers": { "com.google.pay": [ ... ], "dev.shopify.shop_pay": [ ... ] }, "map_order": { "payment_handlers": ["dev.shopify.shop_pay", "com.google.pay"] } } At any deeper scope the same member rides the ambient `ucp` member, with no schema change to the host object -- e.g. ordering an identity provider registry inside a capability config: "config": { "providers": { "app.example.login": [ ... ], "com.google": [ ... ] }, "ucp": { "map_order": { "providers": ["app.example.login", "com.google"] } } } Refs #525, #170 * explain within-handler presentation preference Handler-level presentation preference is carried by map_order.payment_handlers; review on #657 noted the instrument grain was uncovered. The order of the business's advertised available_instruments array now carries the same suggestive within-handler preference, earliest first: platforms SHOULD consider it and MAY apply their own ordering, and the buyer-side context.payment[] preference remains distinct with the platform arbitrating. Completes handler-major presentation ordering at both grains. * exclude dictionaries from the ambient namespace The ambient `ucp` reservation applies to structured objects whose members are schema-defined fields. Dictionary keys are application data, so reserving `ucp` there conflicts with schemas such as `attribution`, where all values are strings. Limit the reservation to structured object scopes and treat `ucp` as ordinary data in dictionary containers. Keep structured objects used as dictionary values eligible. Narrow the closed-object authoring rule to the same boundary and align the central vocabulary description, overview, and glossary. * prohibit direct namespace recursion The existing idempotence rule prohibited any nested `ucp` member, which also excluded valid protocol namespaces on structured objects located beneath the root envelope, such as a capability's `config`. Limit the prohibition to a direct `ucp.ucp` child while keeping deeper structured objects eligible for their own protocol namespace. * define the ambient namespace validation contract UCP schemas are open by default, so ordinary JSON Schema validation accepts ambient `ucp` at eligible structured scopes without applying vocabulary rules. Recommend that UCP-aware tooling recognize and apply the central vocabulary. Require UCP-aware strict validators to continue allowing and validating ambient `ucp` while rejecting other unknown domain fields. Note that non-UCP-aware strict validators may reject it; this is an accepted compatibility limit. * clarify complete checkout signature coverage AP2 merchant authorization covers the JCS-canonicalized checkout without `ap2`, rather than the raw HTTP body bytes. Projecting the checkout through schema-recognized fields can therefore change the signed logical content. Clarify that verification and mandate construction use the complete checkout JSON and that removing any covered member invalidates the signature. * omit map_order from operation requests Mark both `map_order` properties with `ucp_request: "omit"`. Document per-member applicability and require repeated exposures to carry the same annotation. Request resolution now removes `map_order` while retaining the containing `ucp` envelope where its schema requires it. * define map_order target resolution Nested `map_order` lives inside `ucp` while its target map sits on the object annotated by `ucp`. At the root, registries and `map_order` instead share the root envelope. Calling both relationships sibling fields left root domain maps ambiguous. Define nested lookup on the object containing `ucp` and root lookup inside the root envelope. Leave root domain fields such as `actions` out of scope and align the schema descriptions and processing rules. * recover from duplicate map_order entries `uniqueItems` allowed a repeated key in advisory ordering metadata to invalidate an entire profile or response, even though deterministic recovery is available. Allow repeated keys at the schema layer while prohibiting producers from emitting them. Require consumers to keep the containing document valid, honor the first occurrence, and ignore later repetitions. * move ambient validation to schema resolution Define UCP-aware schema resolution as the boundary that applies the central namespace vocabulary at eligible structured scopes and emits ordinary JSON Schema for standard validators and code generators. Keep the authoring guide focused on source-schema rules rather than repeating the resolution contract. * tighten ambient namespace and ordering contracts Clarify the boundary between explicit root `ucp` declarations and ambient nested placement, including closed-object authoring and tolerant handling of an invalid direct `ucp.ucp` child. Make partial `map_order` declarations deterministic with RFC 8785 fallback ordering, require producers to name present maps and keys, and keep unusable advisory entries from invalidating the containing document. Keep central schema descriptions focused on local shape and document the existing within-handler instrument presentation order. * clarify AP2 signature coverage
Static request schemas describe the inputs a Business accepts generally, but cannot express narrower requirements for specific transaction. Platforms therefore cannot know before submission that a negotiated quantity is fixed or that an otherwise optional field is required. Add response-only `$requestConstraints` as a bounded Draft 2020-12 fragment that a Business emits and enforces against the corresponding later request representation. A Platform may use the fragment when forming or validating input, or ignore it and rely on the Business's existing validation errors. Keep the structural member ambient rather than adding an ordinary property to every carrier schema. Each adopting contract defines correspondence and lifecycle, while the shared type closes the vocabulary to `required`, `properties`, `enum`, and `const`, with optional display text and `$comment`. Examples demonstrate a Cart Line Item fixed at quantity 100, preservation of the company-scoped `ACME-X7Q9-L2M4` discount code, and a submitted card instrument that requires `billing_address`.
Static request schemas cannot express transaction-specific narrowing, forcing Platforms to discover negotiated requirements only after submission fails. Register response-only `ucp.request_constraints` in the central protocol vocabulary and define a closed Draft 2020-12 fragment for object presence and value constraints. Attach constraints through the containing `ucp` scope so host schemas do not need carrier-specific declarations. Keep Platform processing optional and Business validation authoritative, preserving the namespace requirement that registered members remain safe to ignore.
The TC discussion sharpened the intended split: Request Constraints narrow the resolved request schema, Business enforcement is authoritative, and Platform preflight is an optional optimization rather than a second conformance boundary. Define effective request validation as the conjunction of the resolved request schema and emitted constraints. Require Businesses to emit conforming constraints and enforce every rule, while allowing Platforms to evaluate any supported subset after validating the member's closed shape. A successful preflight can therefore avoid doomed requests without claiming that every emitted constraint has passed. Keep the wire language focused on required, properties, enum, and const by removing display annotations and $comment. Clarify that constraint objects are embedded JSON Schema rather than ambient UCP objects, and make each operation responsible for scope and lifecycle.
3105ad5 to
c7698c9
Compare
|
@igrigorik @raginpirate — overall this is feeling good to me. Two things I'd love to get thoughts on before stamping though:
{
"ucp": {
"request_constraints": {
"anyOf": [
{ "properties": { "card_number_type": { "enum": ["fpan", "dpan"] } }, "required": ["cvc"] },
{ "properties": { "card_number_type": { "const": "network_token" } }, "required": ["cryptogram"] }
]
}
}
}The closed subset ( Honestly I'd rather land a solution that finally does address that case which was the impetus for this journey. But I'm OK taking this as-is for the incremental progress it brings. IIUC, either path to closing the gap layers on as a clean backwards-compatible change: structural separation of card credential types (#424) OR extending the Nit for if we do the vocabulary route: since the meta-schema is
WDYT on these two topics? CC @gsmith85 |
Request Constraints currently combines two independent concerns: a closed
predicate language and a binding that applies that language to subsequent
request data. Keeping both in request_constraints.json makes the grammar
appear request-specific and prevents other contracts from referencing it
without inheriting that name and semantic scope.
Extract the Object and Value Constraint grammar into the direction-agnostic
constraint_expression.json schema. Keep request_constraints.json as the
binding to subsequent UCP request data. Each containing UCP property owns
its request and response applicability; the shared grammar has no direction
of its own.
A domain reuses the grammar by defining a target model and a constraint
binding for that target. For example, payment could define:
PaymentAcceptanceFacts {
brand,
is_po_box,
credential_type
}
Payment Acceptance Constraints would reference Constraint Expression and
state that emitted expressions are evaluated against PaymentAcceptanceFacts:
{
"$id": "https://example.com/payment_acceptance_constraints.json",
"title": "Payment Acceptance Constraints",
"$ref":
"https://ucp.dev/schemas/common/types/constraint_expression.json"
}
A payment handler could then expose that binding through its own carrier:
"acceptance_constraints": {
"$ref": "payment_acceptance_constraints.json",
"ucp_request": "omit"
}
An emitted value could constrain the derived facts using only the shared
operations:
{
"required": ["brand"],
"properties": {
"brand": {
"enum": ["visa", "mastercard"]
},
"is_po_box": {
"const": false
}
}
}
The payment contract remains responsible for deriving
PaymentAcceptanceFacts and defining authority and lifecycle. The binding
schema validates that the emitted expression uses the shared grammar, and
the emitted expression is then evaluated against the derived facts.
Constraint Expression admits two recursive positions. Object Constraints
use required and properties to constrain field presence and nested objects.
Value Constraints use enum and const to constrain JSON values. Target field
names remain domain-defined while the executable operation vocabulary stays
closed.
This boundary gives Platforms one coherent evaluator contract. N domains
can bind the grammar to their own target models, and M Platforms can
implement the grammar once, producing an N + M integration model. Allowing
each handler to define executable keywords would instead require bespoke
semantics for every Platform x handler pair and trend toward N x M
integrations.
The refactor does not add payment acceptance facts or another domain
binding. It establishes the reusable grammar those contracts can adopt
later. The existing Request Constraints wire language, response-only
applicability, validation behavior, and Business enforcement remain
unchanged.
|
@jamesandersen @raginpirate ptal at 30b0b3e. That refactor extracts the closed executable language from its first binding and enables additional target-specific bindings: In other words: different targets and carriers, but one executable constraint language. A payment-owned binding could reuse the same grammar without requiring its target to appear on the wire. Payment might define a logical target derived from an instrument or credential: It could then express accepted brands using the shared grammar: {
"properties": {
"brand": {
"enum": ["visa", "mastercard"]
}
}
}The payment contract would own how those facts are derived, where the expression is carried, and its authority and lifecycle; Constraint Expression would own the executable operations and evaluation semantics. Request Constraints and payment acceptance constraints can therefore remain separate because they have different targets and lifecycles while sharing one evaluator contract. This preserves an N + M model—N domains define target bindings and M Platforms implement the grammar once—instead of every handler introducing executable keywords that every Platform must separately understand. This PR does not define Polymorphic branching and
|
Model Constraint Expression directly as its root Object Constraint instead of delegating through the repository's only top-level local reference. Preserve recursive Object and Value Constraint semantics while remaining compatible with the existing schema reference renderer.
| member into eligible response scopes so ordinary resolved-schema validation | ||
| applies the shared schema. | ||
|
|
||
| The member's value is defined in two layers. |
There was a problem hiding this comment.
small nit: With 30b0b3e separating the constraints schema from request_constraints (for potential reuse) should it be bumped out to be described outside of the request_constraints section? ... IMO probably best left until there's a second use case for it (e.g. follow-up on payment facts)
There was a problem hiding this comment.
As a follow up... do we actually need the intent of why we separated the layers described in this document? I just find this paragraph is telling me more about the history of this PR rather than what I need to know to author using the constraints concept.
gsmith85
left a comment
There was a problem hiding this comment.
LGTM! 🚀
The composition model (allOf intersection over the resolved request schema) is solid and keeps validation strictly within standard JSON Schema. The grammar/carrier separation sets up future domain bindings nicely.
| evaluated against that file. `request_constraints.json` is a target-binding | ||
| wrapper: syntax added there applies only at the outermost position, so do not | ||
| use it for a change that has to hold at nested positions. | ||
|
|
There was a problem hiding this comment.
It will be useful for schema authors to have clear guidelines on whether a business requirement should be modeled proactively as a request_constraints rule or left to reactive messages validation. This prevents authors from misusing request_constraints for volatile or runtime-evaluated state (such as live inventory or fraud scoring).
Should we add something like below:
| ### When to use `request_constraints` vs. `messages` | |
| Capability designers should use the following heuristic when deciding how to | |
| model business requirements: | |
| | Criterion | Use `request_constraints` | Use standard `messages` | | |
| | :-- | :-- | :-- | | |
| | **Temporal Role** | Proactive preflight for subsequent request *N+1*. | Reactive diagnostic for submitted request *N*. | | |
| | **Rule Stability** | Deterministic and known in current response. | Evaluated dynamically at execution time. | | |
| | **Primary Goal** | Machine preflight and round-trip elimination. | Explaining execution outcomes or failure reasons. | | |
| Use `request_constraints` for rules known in advance (e.g., fixed purchase lot | |
| quantities, locked discount codes, or required billing fields). Use `messages` | |
| when validation depends on real-time execution state (e.g., live inventory | |
| availability, fraud checks, payment authorization) or when reporting why a | |
| submission failed. |
There was a problem hiding this comment.
I don't think this concern lands on schema authors. request_constraints is registered once in ucp.json#/$defs/members and no capability schema declares it — there's no authoring-time decision to guide. messages, by contrast, is declared per-capability. The choice isn't for that party.
Aligning with you though, it might be worth a simple line or two in the overview to inform businesses that it is detrimental for them to enforce requirements on volatile, changing properties in schemas.
There was a problem hiding this comment.
One wiring note on the registration: nothing in source/schemas references ucp.json#/$defs/members, so resolved schemas never apply the member grammar. At 14df500 with ucp-schema 1.4.1 a Cart response carrying request_constraints {"bogus": 1} validates in default mode, and strict mode rejects the documented Cart example since the member is unevaluated. Exposing the member through #/$defs/base the way map_order is, or shipping the materialization described in schema-authoring.md with the release, would close both.
amithanda
left a comment
There was a problem hiding this comment.
Added minor non-blocking documentation suggestions to make the distinction clearer between request constraints and existing messages based mechanism. PTAL.
raginpirate
left a comment
There was a problem hiding this comment.
I think we're missing some fundamental prose around how to handle the path to constrain against as a platform, lmk your thoughts
| member into eligible response scopes so ordinary resolved-schema validation | ||
| applies the shared schema. | ||
|
|
||
| The member's value is defined in two layers. |
There was a problem hiding this comment.
As a follow up... do we actually need the intent of why we separated the layers described in this document? I just find this paragraph is telling me more about the history of this PR rather than what I need to know to author using the constraints concept.
| evaluated against that file. `request_constraints.json` is a target-binding | ||
| wrapper: syntax added there applies only at the outermost position, so do not | ||
| use it for a change that has to hold at nested positions. | ||
|
|
There was a problem hiding this comment.
I don't think this concern lands on schema authors. request_constraints is registered once in ucp.json#/$defs/members and no capability schema declares it — there's no authoring-time decision to guide. messages, by contrast, is declared per-capability. The choice isn't for that party.
Aligning with you though, it might be worth a simple line or two in the overview to inform businesses that it is detrimental for them to enforce requirements on volatile, changing properties in schemas.
|
Also wanted to add; thank you so much for continuing to iterate here @igrigorik. |
Replace overloaded grammar and host terminology with actionable guidance on where expression syntax and request-binding changes belong. Distinguish stable, proactive preflight from runtime outcomes reported through messages. Advise Businesses not to advertise volatile constraints unless they can continue enforcing them, and describe quantity locks as exact sale-basis steps rather than lot semantics.
* add path targeting to Request Constraints Request Constraints described what to validate but did not define how a nested response occurrence corresponded to request data. Leaving that mapping to each domain made array positions fragile and could not express cross-shape bindings, such as an available payment instrument constraining a submitted instrument. Bind each value to request data with an optional, outer-only `path`. An explicit path is a complete RFC 9535 JSONPath query evaluated against the next request; when omitted, the path is derived from the Normalized Path of the structured response object containing the constraint. This preserves concise ambient authoring while supporting stable-identity queries and multi-object selection. Validate every selected object against the complete closed Constraint Expression in conjunction with the resolved request schema. Zero matches have no effect, all matches must pass, and overlapping expressions compose without precedence. Businesses must ensure potentially overlapping expressions are jointly satisfiable. Keep Business evaluation authoritative while making Platform preflight optional and advisory. A Platform may evaluate a supported subset, but must process each chosen value completely and skip malformed, unsupported, non-object, or resource-limited values rather than partially interpreting them. Existing operation outcomes and messages continue to govern runtime failures. Each authoritative resource response supplies the advertised constraints for the next request to that resource; the following response replaces that set, omission clears it, and sets are never merged by path. Require explicit payment instrument paths to bridge response and request shapes by matching handler and instrument identity. * Update source/schemas/common/types/request_constraints.json Co-authored-by: Daniel Wyckoff <daniel.wyckoff@shopify.com>
|
@igrigorik one quick note before this merges, PTAL at #424. |
The path-targeting change referenced Constraint Expression's `required` property directly, and a follow-up applied the same factoring to `properties`. Although both references are valid Draft 2020-12, the documentation renderer turns both anonymous targets into nonexistent anchors. The `properties` target also contains relative nested references whose source-resource scope the released bundler does not preserve. Spell out `required` and `properties` at the Request Constraints root while continuing to reference the shared grammar for nested Object and Value Constraints. This preserves the wire contract and keeps the PR compatible with today's toolchain. The property-level references can replace these duplicated root shapes after resource-aware fragment bundling and anonymous-target documentation rendering are both supported and released.
|
@raginpirate you walked right into the design question I wrestled for hours while iterating on options for #744. Short answer: the array works, but I don't think it is the right primitive for #424, and I would not fold it into #655. JSON Schema can express the contract you're after. The reason the array feels necessary is that our current grammar does not admit As a side note, yes, an array can also produce a more compact representation for some rule sets. The explicit trade-off is that we double down on UCP-specific constraint resolution: JSONPath filters become conditional control flow, and the member becomes a list of independently targeted rules with its own composition and partial-evaluation semantics. The invariant behind #744 was “one binding carries one schema”: prefer standard schema structure before introducing a rule sheet. Crucially, the array does not replace vocabulary evolution; it only routes around
The benefit of this path is that we retain a simple evaluator contract: UCP owns the binding step through In the #424 case, one path selects the submitted card instrument and a nested So my steer and preference is: keep |
raginpirate
left a comment
There was a problem hiding this comment.
Re-approving; thanks for the feedback on array form. Happy to align in this direction 😄
Depends on #657.
Static request schemas describe the inputs a Business generally accepts, but not narrower requirements for a particular transaction. Without response-time constraints, a Platform learns that a negotiated quantity is fixed or an otherwise optional field is required only after submission fails—or through domain-specific constraint formats.
PR #657 establishes the ambient
ucpprotocol namespace and central member registry. This PR addsrequest_constraintsto that vocabulary instead of inventing a parallel structural namespace.ucp.request_constraintsinucp.json#/$defs/members.required,properties,enum, andconst.allOfwithout schema merging.ucpscope to identify the constrained parent object; adopting contracts define correspondence and lifecycle when existing operation semantics do not.Examples
A Cart Line Item fixed at quantity
100:{ "id": "line_123", "quantity": 100, "ucp": { "request_constraints": { "properties": { "quantity": {"const": 100} } } } }A submitted card instrument that requires a billing address for this transaction:
{ "type": "card", "ucp": { "request_constraints": { "required": ["billing_address"] } } }Boundaries
This PR does not define payment availability, custom assertion keywords, schema merging, or new outcomes and error codes. Payment-specific correspondence and availability remain downstream.