trace-verify verifies TRACE signatures with json.dumps(sort_keys=True), which spec section 3.2.2 names as insufficient

[lywinged]

Raised first on #134, which merged before anyone had a chance to look at it. Moving it here rather than leaving it on a closed PR, because it is not a defect in that document: spec/registry-anchor-v1.md §7 is what makes the package a reference implementation, and §5.2 explicitly hands signature verification to v0.2 §3.3, which the package implements.

Filed here because there is nowhere else public to file it. trace-verify's PyPI metadata points Bug Tracker at github.com/agentrust-io/trace-registry, which returns 404 to an anonymous reader.

What it does

trace_verify._signature.canonical_body_bytes builds the signature pre-image as:

body = {k: v for k, v in claim.items() if k != "signature"}
json.dumps(body, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode("ascii")

spec/trace-v0.2.md §3.2.2 names that exact construction:

Implementations MUST use an RFC 8785-conformant library. Using json.dumps(sort_keys=True) (Python) or equivalent ad-hoc sorting is insufficient: it diverges from RFC 8785 for non-ASCII strings and for IEEE 754 number serialization.

agentrust_trace.sign_record signs the same object -- the claim with signature removed -- through rfc8785.dumps. Same scheme, different canonicalizer, so the two agree on ASCII and disagree everywhere else.

Reproduction

agentrust-trace 0.6.0 and trace-verify 0.2.0, both from PyPI. Escapes throughout, so nothing here depends on how a terminal renders it.

from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from agentrust_trace.sign import sign_record, key_to_jwk
import trace_verify._signature as tvs

key = Ed25519PrivateKey.generate()
record = {
    "eat_profile": "https://trace.agentrust-io.com/profiles/v0.2",
    "sub": "spiffe://trust.example.org/agent/payments",
    "policy_label": "Zahlungsverkehr f\u00fcr M\u00fcnchen",  # escapes; any non-ASCII does it
}
signed = sign_record(record, key)
print(tvs.verify_claim_signature(signed, key_to_jwk(key)))   # False

record contents                                agentrust-trace   trace-verify
ASCII only                                     valid             valid
any string field containing U+00FC             valid             INVALID
any string field containing a CJK character    valid             INVALID

The CLI verifies signatures by default and exits 1 when the check fails, so an external verifier following §7, on a valid record carrying a German policy label or a non-Latin data class, is told the signature is invalid. verify_claim_against_registry's docstring describes it as "the fail-closed check used by the anchoring paths"; if that is accurate, such a record also cannot be anchored, but that path is not visible from outside.

Two readings

The public material does not distinguish them, so I am asking rather than asserting:

1. the registry deliberately signs over a different pre-image than v0.2 §3.2.2, in which case that belongs in spec/registry-anchor-v1.md §0 as a third row -- and it is the most consequential one, since §0 currently states that the signing layer is JCS; or
2. it is a defect in the package.

Either way I would not have §7 point an external verifier at it before that is settled, since §7 exists precisely so that people run it.

Severity, honestly

Nobody is hurt today. This is a Developer Preview, ADOPTERS.md lists one organisation, and a record containing a non-ASCII character may not exist in the wild yet. Nothing is failing right now.

What makes it worth fixing while it is cheap is the failure mode. A valid, correctly signed record is reported as having an invalid signature. For evidence whose whole purpose is that a third party can check it without trusting the operator, a false rejection is the worst direction to fail in, and the trigger is not exotic: one accented character in a policy label.

It is also the mistake spec/registry-anchor-v1.md §0 warns about, in the opposite direction. §0 says do not reuse the signing canonicalizer at the leaf. What is in the shipped code is the leaf canonicalizer reused at the signing layer. The warning is right and one-directional, and the missing direction is the one that has already happened.

Two smaller things in the same package

• PyPI metadata points Homepage, Documentation and Bug Tracker at github.com/agentrust-io/trace-registry. All three return 404 to an anonymous reader, which is the audience spec/registry-anchor-v1.md exists to serve.
• The installed distribution is 0.2.0 while trace_verify.__version__ is 0.1.0, and --version prints the latter. Same drift chore(release): 0.6.0, and stop __version__ drifting #131 fixed here.

Happy to open a PR against whichever repository ends up owning the fix, if that is useful and if it becomes reachable.

[github-actions]

❔ Contributor Check: UNKNOWN

Check	Result
Profile	UNKNOWN
Credential	LOW
Overall	UNKNOWN

Automated check by AgenTrust Contributor Check.

[imran-siddique]

Confirmed, and thank you for both halves of this.

The signature pre-image was wrong. canonical_body_bytes used json.dumps(sort_keys=True, ensure_ascii=True) while agentrust_trace.sign_record signs through rfc8785.dumps, so the two agreed on ASCII and diverged on everything else. Any Trust Record carrying non-ASCII text was verified against bytes the producer never signed. Fixed in agentrust-io/trace-registry#38, with a regression test that signs a claim the way a producer signs it and verifies it.

One correction worth recording, because I nearly got it wrong. Grepping for the pattern turns up four more sites in that repo, and they are all correct: docs/anchor-format.md section 1 defines the anchor leaf as sorted-keys ASCII JSON over the complete signed claim, and excludes non-integer numbers so the IEEE 754 divergence cannot arise there. Two canonicalizations, two layers, on purpose. Your report named exactly the one function that was wrong.

On the tracker: you are right that there was nowhere to file. trace-verify is public on PyPI and pointed Bug Tracker at a private repo, so the link 404s for every reader. That is repointed at this repo in the same PR. Homepage and Documentation still point into the private repo, which means anchor-format.md is unreadable to anyone using the package despite being its normative spec; that one needs a decision rather than a patch and is not fixed yet.

The needs-review:UNKNOWN label on your issues is a broken bot, not a judgement about you. Tracked as agentrust-io/.github#27.

[imran-siddique]

Closing the loop on the half of this I left as a decision rather than a patch.

I said Homepage and Documentation still pointed into the private repo and needed a decision. Checking again today, [project.urls] on main was already fixed in #38 and points every entry at public trace-spec, including a dedicated Anchor format link to spec/registry-anchor-v1.md. What was wrong is narrower and more annoying: that fix landed after v0.3.0 was tagged and carried no version bump, so PyPI is still serving the pre-fix metadata. trace-verify 0.3.0 today:

Bug Tracker    -> trace-spec/issues                              reachable
Homepage       -> agentrust-io/trace-registry                    private
Documentation  -> trace-registry/.../docs/anchor-format.md       404 anonymously
Changelog      -> trace-registry/.../CHANGELOG.md                private

So the fix has existed for twelve days and no installer has seen it. A version bump and a release is the whole remedy, and that is agentrust-io/trace-registry#43.

There was a second thing underneath, which your original report is what surfaced. The anchor format existed in two places: the private docs/anchor-format.md and the public spec/registry-anchor-v1.md. The public one is the more developed of the two, with the canonicalization warning in §0, the transparency claim relationship in §6, reference implementations in §7 and conformance in §8 that the private copy does not carry. So the copy a package reader could reach was the weaker one, and it was already drifting from the normative one. #43 replaces it with a pointer.

That matters beyond this issue. agentrust-io/agent-manifest#322, confirmed today, is the same two-canonicalizations mistake in a different repository: canonicalize() sorts keys by Unicode code point where RFC 8785 §3.2.3 requires UTF-16 code units, so agent-manifest rejects TRACE records that trace-spec signed. §0 of the anchor document is precisely the text that names that trap, and it has been sitting behind a 404 on the package that implements it.

Leaving this open until #43 is merged and 0.3.1 is on PyPI, since the published metadata is the thing you actually reported.