fix(catalog): make the approval signing input the JCS it claims to be (#517)

[zohebk8s]

Contributes to #517. Independent of #531 and reviewable on its own; see Interaction with #531 for what landing both costs. The two items from that review that need a maintainer decision rather than a fix are #533; neither touches canonicalization, so nothing here waits on them.

Why

canonical_json is documented as "the RFC 8785-compatible JSON form used by cMCP records" and serialized with ensure_ascii=True, which is the one thing JCS does not do. The standard emits UTF-8 and escapes only what ECMAScript JSON.stringify escapes:

cmcp  : b'{"principal_id":"jos\u00e9","role":"s\u00e9curit\u00e9"}'
jcs   : b'{"principal_id":"jos\xc3\xa9","role":"s\xc3\xa9curit\xc3\xa9"}'

Any record carrying a non-ASCII principal_id, issuer, role, catalog_id, or policy_id is therefore signed over different bytes than a conforming producer signs. A record produced by any other implementation fails here with "approval signature is invalid", which points nowhere near the encoding, and a reviewer whose name is not ASCII cannot approve a catalog change at all.

sort_keys is the second divergence. It orders members by code point where JCS orders by UTF-16 code unit. The two disagree for any key outside the BMP, since a surrogate pair leads with 0xD800 and sorts below a BMP character above 0xE000.

#517 asks for RFC 8785 reuse specifically so these records can cross implementations, so this is a conformance defect rather than a preference.

What

Members are ordered on their UTF-16BE bytes, which is the same comparison JCS specifies since every code unit occupies two bytes, and the output is UTF-8.

Values JCS cannot pin down are refused rather than serialized into a signing input two implementations would read differently:

• floating point numbers, whose JCS serialization is the ECMAScript number algorithm and is not implemented here;
• integers beyond 2**53 - 1, which JCS treats as IEEE 754 doubles;
• non-string object keys, which have no canonical order;
• unpaired surrogates, which previously escaped as UnicodeEncodeError.

Each raises CatalogApprovalError, matching how the rest of the module reports malformed input. Approval records carry none of these, so refusing them only closes a door.

b'{"principal_id":"jos\xc3\xa9","role":"s\xc3\xa9curit\xc3\xa9"}'   non-ASCII stays UTF-8
b'{"\xf0\x90\x80\x80":2,"\xef\xbf\xbf":1}'                          U+10000 sorts below U+FFFF
rejected: canonical JSON does not accept floating point numbers
rejected: integer is outside the range RFC 8785 serializes exactly
rejected: canonical JSON object keys must be strings
rejected: canonical JSON cannot encode an unpaired surrogate

The spec doc now states the canonicalization rules and what is refused, which it did not before.

Compatibility

ASCII-only records serialize to exactly the same bytes as before. A pinned digest and a byte-for-byte assertion cover it, so nothing that verifies today stops verifying. Nothing in the tree consumes these records at runtime yet, so there is no deployed producer to migrate; the encoding will stop being free to correct as soon as there is.

Tests

13 new tests in tests/unit/test_catalog_canonical_json.py, kept in a separate file so they do not collide with #531, which rewrites test_catalog_approval.py substantially.

tests/unit/test_catalog_canonical_json.py   13 passed
with the existing approval tests            17 passed
ruff                                         All checks passed
mypy --strict                                no issues, 1 source file

Verified by restoring the old one-line implementation: 11 of the 13 fail. The two that survive are the ASCII byte-stability pin and the largest exact integer, both of which are meant to be invariant across this change.

Interaction with #531

Both branches are cut from main, neither needs the other, and either can be reviewed first. They do overlap textually. I applied this branch on top of #531 rather than guess, so the cost of landing both is measured:

• CHANGELOG.md, conflict. Both add an entry at the same anchor, the top of Unreleased/Fixed. Keep both, in whichever order reads better.
• src/cmcp_runtime/catalog/approval.py, conflict. Both add a module constant directly after PROFILE, _MAX_EXACT_INT here and _B64URL there. Keep both.
• tests/unit/test_catalog_canonical_json.py, three lines. The end to end non-ASCII test calls verify_catalog_change, whose signature fix(catalog): pin the approval policy verifier-side and bind the chain fields (#517) #531 changes: it needs expected_policy_hash and expected_catalog_id, and compute_policy_hash in place of digest_json for the policy.

The spec doc merges cleanly. Resolved as above, with both branches applied: 39 passed, 1 xfailed, ruff clean.

Merging #531 first is the tidier order. The third item is a test file this PR owns but #531 breaks, so in the other order #531 has to edit a file it does not otherwise touch, or main goes red.

Not covered

audit/trace_claim.py, cmcp_verify/embodied_action.py, and catalog/loader.py each carry their own copy of the same sort_keys and separators idiom with the same two divergences. Only this one claims RFC 8785 in its docstring, and only this one signs reviewer identities, so the others are left alone here. Whether they collapse into a shared helper is a separate question, and the answer decides whether their wire formats can move at all.

[github-actions]

🟡 Contributor Check: MEDIUM

Check	Result
Profile	MEDIUM
Credential	LOW
Overall	MEDIUM

Automated check by AgenTrust Contributor Check.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!