Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ All TRACE test failures emit a structured error code of the form `TR-<MODULE>-<N
| TR-ENV-001 | Missing or invalid `eat_profile` URI | Set `eat_profile` to `"tag:agentrust-io.com,2026:trace-v0.2"` |
| TR-ENV-002 | `iat` is missing, not an integer, or out of range | Set `iat` to a Unix timestamp integer (e.g. `int(time.time())`) |
| TR-ENV-003 | `subject` does not match SPIFFE URI or DID pattern | Use `spiffe://<trust-domain>/<path>` or a `did:` URI |
| TR-ENV-004 | One or more required fields are absent | Add the missing field(s); check the [Schema Reference](https://trace.agentrust-io.com/docs/schema/) for the full required set |
| TR-ENV-004 | `cnf` is absent or not an object, `cnf.jwk` is absent or not an object, or `cnf.jwk.kty` is absent | Populate `cnf.jwk` with at least `kty`. This checks that one field, not the schema's full required set, which structural validation covers |

## TR-SIG — Signature

Expand All @@ -18,22 +18,23 @@ All TRACE test failures emit a structured error code of the form `TR-<MODULE>-<N
| TR-SIG-001 | Signature algorithm is not Ed25519 | Generate an Ed25519 key (`generate_key()`) and re-sign; ES256 and RS256 are not accepted |
| TR-SIG-002 | `cnf.jwk` missing or malformed | Populate `cnf.jwk` with the OKP public key `{"kty":"OKP","crv":"Ed25519","x":"..."}` — `sign_record()` does this automatically |
| TR-SIG-003 | Signature verification failed | Re-sign the record with `sign_record(record, key)`; the record fields must not have changed after signing |
| TR-SIG-004 | Private key material (`d` member) found in `cnf.jwk` | Remove the `d` field before embedding the JWK; `key_to_jwk()` returns the public-only form |
| TR-SIG-004 | `cnf.jwk` carries private key material (a `d` member), or `cnf.jwk.kty` is missing or names an unsupported key type (`OKP` and `EC` are accepted) | Remove `d` and embed only the public form of the JWK; `key_to_jwk()` returns it. For key type, use `OKP` or `EC`; Ed25519 signature verification additionally requires `kty: "OKP"` with `crv: "Ed25519"`, and a supported key that is not that pair passes this check and fails TR-SIG-005 |
| TR-SIG-005 | The signature check outcome: the Ed25519 verification result, a signature that cannot be verified, a signature left unchecked because `cnf.jwk` carried private key material, or no signature at all. With no signature it is FAIL at Level 1 and above and `UNVERIFIED` at Level 0, which is not a pass | Sign the record with `sign_record(record, key)` and do not change the signed fields afterwards. An unsigned record is reported as unverified rather than skipped, so it cannot be read as a benign omission |

## TR-RTE — Runtime

| Code | Description | How to fix |
|------|-------------|------------|
| TR-RTE-001 | `runtime.platform` is not a recognised TEE enum value | Use one of: `software-only`, `tpm2`, `sev-snp`, `tdx`, `opaque` |
| TR-RTE-001 | `runtime` is missing or not an object, or `runtime.platform` is not in the registered set, or is `software-only` at Level 1 and above | Use a value from the `runtime.platform` enum in `schemas/trace-claim.json`. `software-only` carries no hardware attestation evidence and is accepted only at Level 0 |
| TR-RTE-002 | `runtime.measurement` is not a valid `sha256:` digest | Provide a 64-character hex digest prefixed with `sha256:`; for Level 0 all-zeros is conventional |
| TR-RTE-003 | RIM URI present but does not resolve to a valid reference image | Remove `runtime.rim_uri` if not using a RIM, or ensure the URI returns a valid reference manifest over HTTPS |
| TR-RTE-003 | `runtime.rim_uri` is present and is not an `https://` URI | Remove `runtime.rim_uri` if not using a RIM, or set it to an `https://` URI. The URI is not resolved and the manifest behind it is not checked; this is a format check |

## TR-POL — Policy

| Code | Description | How to fix |
|------|-------------|------------|
| TR-POL-001 | `policy.bundle_hash` is not a valid `sha256:` digest | Compute `sha256:` + hex digest of your Cedar policy bundle bytes |
| TR-POL-002 | `policy.enforcement_mode` is not `enforce`, `advisory`, or `silent` | Replace `"strict"` or `"monitor"` with `"enforce"`, `"advisory"`, or `"silent"` |
| TR-POL-002 | `policy.enforcement_mode` is not `enforce`, `advisory`, `silent`, or `declared` | Replace `"strict"` or `"monitor"` with one of the four accepted values; `"declared"` is the honest value for a producer that binds a policy without evaluating it |

## TR-TXN — Transcript

Expand All @@ -46,8 +47,7 @@ All TRACE test failures emit a structured error code of the form `TR-<MODULE>-<N

| Code | Description | How to fix |
|------|-------------|------------|
| TR-ANC-001 | `transparency` field missing or empty | Submit the record to a SCITT transparency log and set `transparency` to the returned receipt URI |
| TR-ANC-002 | `transparency` URI does not use `https://` scheme | Only `https://` URIs are accepted; update the URI or use the agentrust registry at `https://registry.agentrust-io.com` |
| TR-ANC-001 | `transparency` is absent or empty, is not a string, or is not an `https://` URI with a host | Submit the record to a SCITT transparency log and set `transparency` to the returned receipt URI. The URI is not resolved and the receipt behind it is not fetched; this is a format check |

## TR-SCA — Provenance

Expand Down
17 changes: 6 additions & 11 deletions docs/levels.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Level 0 records are signed with a software key. The `runtime.platform` must be `
**What causes a Level 0 failure:**

- `eat_profile` missing or wrong value — TR-ENV-001
- `runtime.platform` is a TEE value (e.g. `sev-snp`) but Level 0 is requested — TR-RTE-001 does not apply, but TR-ENV still checks the envelope
- `runtime.platform` is a TEE value (e.g. `amd-sev-snp`) but Level 0 is requested — TR-RTE-001 does not apply, but TR-ENV still checks the envelope
- `policy.enforcement_mode` is `"strict"` or `"monitor"` — TR-POL-002
- `cnf.jwk` missing or contains private key material (`d` field) — TR-SIG-002, TR-SIG-004
- Signature does not verify against `cnf.jwk` — TR-SIG-003
Expand All @@ -67,7 +67,7 @@ Level 0 records are signed with a software key. The `runtime.platform` must be `

## Level 1 — TEE Attestation

Level 1 adds hardware attestation. `runtime.platform` must be one of: `tpm2`, `sev-snp`, `tdx`, `opaque`. The measurement must be non-zero. `appraisal.status` must be `"affirming"`.
Level 1 adds hardware attestation. `runtime.platform` must be a value from the `runtime.platform` enum in `schemas/trace-claim.json` other than `software-only`, which carries no hardware attestation evidence. The measurement must be non-zero. `appraisal.status` must be `"affirming"`.

**Minimum conformant Level 1 record** (changes from Level 0 in bold context):

Expand All @@ -82,7 +82,7 @@ Level 1 adds hardware attestation. `runtime.platform` must be one of: `tpm2`, `s
"version": "20251001"
},
"runtime": {
"platform": "sev-snp",
"platform": "amd-sev-snp",
"measurement": "sha256:a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2"
},
"policy": {
Expand Down Expand Up @@ -123,7 +123,7 @@ Level 1 adds hardware attestation. `runtime.platform` must be one of: `tpm2`, `s

## Level 2 — Transparency Anchoring

Level 2 adds tool transcript and transparency anchor requirements. The `transparency` field must be a resolvable HTTPS URI pointing to a SCITT receipt, not the placeholder value.
Level 2 adds tool transcript and transparency anchor requirements. The `transparency` field must be an HTTPS URI with a host, pointing to a SCITT receipt. The suite checks that it parses and that the scheme and host are present; it does not resolve the URI, and it does not recognise a placeholder value.

**Minimum conformant Level 2 record** (additional fields over Level 1):

Expand All @@ -133,11 +133,7 @@ Level 2 adds tool transcript and transparency anchor requirements. The `transpar
"hash": "sha256:c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4",
"call_count": 4
},
"transparency": "https://registry.agentrust-io.com/claim/01J3XKWP4NQZ8R5HT6YD7VMBCE",
"anchor": {
"log_id": "https://registry.agentrust-io.com",
"leaf_hash": "sha256:f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5"
}
"transparency": "https://registry.agentrust-io.com/claim/01J3XKWP4NQZ8R5HT6YD7VMBCE"
}
```

Expand All @@ -147,8 +143,7 @@ Level 2 adds tool transcript and transparency anchor requirements. The `transpar

- `tool_transcript.hash` missing or not a valid `sha256:` digest — TR-TXN-001
- `tool_transcript.call_count` negative or not an integer — TR-TXN-002
- `transparency` is the placeholder URI, missing, or not HTTPS — TR-ANC-001
- `anchor.leaf_hash` missing or not a valid `sha256:` digest — TR-ANC-002
- `transparency` is absent or empty, is not a string, or is not an `https://` URI with a host — TR-ANC-001

---

Expand Down
10 changes: 5 additions & 5 deletions docs/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ The TRACE conformance suite is divided into seven modules. Each module maps to a

| Module | ID Prefix | Spec Section | What It Tests |
|--------|-----------|--------------|---------------|
| [Envelope](modules/tr-env.md) | TR-ENV | §3.2 | EAT envelope structure, `eat_profile` URI, required fields, `iat` validity |
| [Signature](modules/tr-sig.md) | TR-SIG | §3.2.1 | Algorithm conformance (Ed25519), key binding, private key leak detection |
| [Runtime](modules/tr-rte.md) | TR-RTE | §3.1 | TEE platform enum, measurement format, RIM URI resolution |
| [Envelope](modules/tr-env.md) | TR-ENV | §3.2 | `eat_profile` URI, `iat` validity, `subject` form, presence of `cnf.jwk.kty` |
| [Signature](modules/tr-sig.md) | TR-SIG | §3.2.1 | Private key leak detection, key type support, and the Ed25519 signature verification outcome |
| [Runtime](modules/tr-rte.md) | TR-RTE | §3.1 | TEE platform enum, measurement format, RIM URI scheme |
| [Policy](modules/tr-pol.md) | TR-POL | §3.1 | Policy bundle hash format, enforcement mode values |
| [Transcript](modules/tr-txn.md) | TR-TXN | §3.1 | Tool-call transcript hash binding |
| [Transparency](modules/tr-anc.md) | TR-ANC | §3.2 | SCITT receipt URI format, inclusion proof structure |
| [Provenance](modules/tr-sca.md) | TR-SCA | §3.1 | SLSA provenance level, builder URI, digest format |
| [Transparency](modules/tr-anc.md) | TR-ANC | §3.2 | SCITT receipt URI form. The URI is not resolved and no inclusion proof is checked |
| [Provenance](modules/tr-sca.md) | TR-SCA | §3.1 | SLSA provenance level and digest format |
3 changes: 1 addition & 2 deletions docs/modules/tr-anc.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,4 @@ Tests transparency anchoring via SCITT.

| Test ID | Description | Positive Case | Negative Case |
|---------|-------------|---------------|---------------|
| TR-ANC-001 | `transparency` is a non-empty URI | `https://transparency.example/entries/abc123` | missing field, empty string |
| TR-ANC-002 | URI scheme is `https://` | `https://` prefix | `http://`, bare path, `ipfs://` |
| TR-ANC-001 | `transparency` is an `https://` URI with a host. Not resolved | `https://transparency.example/entries/abc123` | missing field, empty string, non-string, `http://`, bare path, `ipfs://` |
2 changes: 1 addition & 1 deletion docs/modules/tr-env.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ Tests the top-level EAT envelope structure of a TRACE Trust Record.
| TR-ENV-001 | `eat_profile` present and correct URI | `tag:agentrust-io.com,2026:trace-v0.2` | missing or wrong |
| TR-ENV-002 | `iat` is a valid Unix timestamp | integer, reasonable range | string, future date |
| TR-ENV-003 | `subject` matches SPIFFE URI or DID | `spiffe://trust.example/agent/x` or `did:key:z6Mk...` | bare string |
| TR-ENV-004 | Required fields present | all of: eat_profile, iat, subject, model, runtime, policy, data_class, build_provenance, appraisal | missing any |
| TR-ENV-004 | `cnf.jwk.kty` is present. This is not a gate over the schema's required set | `cnf.jwk.kty` set to any value | `cnf` absent, `cnf.jwk` absent or not an object, `kty` absent |
2 changes: 1 addition & 1 deletion docs/modules/tr-pol.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ Tests Cedar policy bundle binding.
| Test ID | Description | Positive Case | Negative Case |
|---------|-------------|---------------|---------------|
| TR-POL-001 | `policy.bundle_hash` is a valid `sha256:` digest | `sha256:` followed by 64 hex chars | missing, wrong prefix, wrong length |
| TR-POL-002 | `policy.enforcement_mode` is `enforce`, `advisory`, or `silent` | `enforce` | `strict`, `monitor`, absent |
| TR-POL-002 | `policy.enforcement_mode` is `enforce`, `advisory`, `silent`, or `declared` | `enforce` | `strict`, `monitor`, absent |
2 changes: 1 addition & 1 deletion docs/modules/tr-rte.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ Tests TEE platform attestation in the `runtime` field.

| Test ID | Description | Positive Case | Negative Case |
|---------|-------------|---------------|---------------|
| TR-RTE-001 | `runtime.platform` is a known TEE enum value | `tpm2`, `sev-snp`, `tdx`, `opaque` | `"software"`, `"cloud"`, absent |
| TR-RTE-001 | `runtime.platform` is in the registered set, and is not `software-only` above Level 0 | `intel-tdx`, `amd-sev-snp`, `nvidia-h100`, `tpm2` | `"software"`, `"cloud"`, `"sev-snp"`, absent |
| TR-RTE-002 | `runtime.measurement` is a valid `sha256:` digest | `sha256:` followed by 64 hex chars | missing, wrong prefix, all zeros |
| TR-RTE-003 | RIM URI (if present) resolves to a valid reference image | valid `https://` URI returning a reference manifest | non-HTTPS URI, 404 response |
3 changes: 2 additions & 1 deletion docs/modules/tr-sig.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ Tests Ed25519 signature binding on the TRACE Trust Record.
| TR-SIG-001 | Signature algorithm is Ed25519 (OKP crv=Ed25519) | `{"kty":"OKP","crv":"Ed25519"}` | ES256, RS256, missing `alg` |
| TR-SIG-002 | `cnf.jwk` present and carries the public key | JWK with `x` member set | missing `cnf`, missing `jwk`, missing `x` |
| TR-SIG-003 | Signature verifies over the canonical record bytes (RFC 8785 JCS) | valid Ed25519 signature | bit-flipped signature, wrong key |
| TR-SIG-004 | `cnf.jwk` does not contain private key material (`d` member absent) | JWK with only `x` | JWK with `d` present |
| TR-SIG-004 | `cnf.jwk` carries no private key material, and `cnf.jwk.kty` is a supported key type | JWK with `x` only, `kty` of `OKP` or `EC` | JWK with `d` present, missing `kty`, `RSA` |
| TR-SIG-005 | The signature check outcome: verified, unverifiable, not checked, or absent | valid Ed25519 signature | bit-flipped signature, no signature, JWK carrying `d` |
4 changes: 2 additions & 2 deletions docs/tutorials/writing-conformance-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ def test_md5_bundle_hash_fails_tr_pol_001():
def test_unknown_enforcement_mode_fails_tr_pol_002():
trace = _policy_trace(
bundle_hash="sha256:" + "b" * 64,
enforcement_mode="strict", # not in {enforce, advisory, silent}
enforcement_mode="strict", # not in {enforce, advisory, silent, declared}
)
codes = {f.code for f in tr_pol.check(trace) if f.failed()}
assert "TR-POL-002" in codes
Expand Down Expand Up @@ -233,7 +233,7 @@ Common codes you will encounter:
| TR-SIG-001 | `signature` | Signature missing or does not verify |
| TR-SIG-002 | `cnf.jwk` | Key must be OKP/Ed25519 |
| TR-POL-001 | `policy.bundle_hash` | Must match `sha256:<64 hex chars>` |
| TR-POL-002 | `policy.enforcement_mode` | Must be `enforce`, `advisory`, or `silent` |
| TR-POL-002 | `policy.enforcement_mode` | Must be `enforce`, `advisory`, `silent`, or `declared` |
| TR-RTE-001 | `runtime.platform` | Must be a registered TEE platform enum |

When a finding carries `status == Status.UNVERIFIED`, the record has no signature. This is not a benign skip at Level 1 or above.
Expand Down
39 changes: 34 additions & 5 deletions src/trace_tests/modules/tr_sig.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,27 @@ def _verify_ed25519(pub_x: str, sig_b64: str, body: bytes) -> tuple[bool, str]:
return False, "TR-SIG-001: signature verification failed"


def _jwk_of(container: Any) -> dict[str, Any]:
"""The JWK under ``container["cnf"]["jwk"]``, or ``{}`` when it is not an object.

A malformed record must produce a finding, not an exception. ``runner.run`` calls
every module without a ``try``, so anything raised here ends the run rather than
failing the record, and the caller sees a traceback where a verdict belongs.

The outer ``isinstance`` is for ``check_cmcp_runtime``, which passes
``record["trace"]`` and so can hand this anything at all. ``check`` passes the
``trace`` it was given, which for the plain format is the record itself and is
already a dict; that function reads ``trace`` directly elsewhere and is not
hardened against a non-dict ``trace``. Whether one can reach it is a question
about ``loader.extract_trace``, not about this helper.
"""
if not isinstance(container, dict):
return {}
cnf = container.get("cnf")
jwk = cnf.get("jwk") if isinstance(cnf, dict) else None
return jwk if isinstance(jwk, dict) else {}


def check_cmcp_runtime(record: dict[str, Any]) -> list[Finding]:
"""Verify the Ed25519 signature on a cmcp RuntimeClaim."""
findings: list[Finding] = []
Expand All @@ -84,7 +105,7 @@ def check_cmcp_runtime(record: dict[str, Any]) -> list[Finding]:
findings.append(Finding("TR-SIG-001", Status.FAIL, "TR-SIG-001: signature field is missing or empty"))
return findings

jwk = record.get("trace", {}).get("cnf", {}).get("jwk", {})
jwk = _jwk_of(record.get("trace"))
kty = jwk.get("kty")
crv = jwk.get("crv")
x = jwk.get("x")
Expand Down Expand Up @@ -117,16 +138,24 @@ def check(trace: dict[str, Any], record: dict[str, Any], fmt: str, level: int =
return check_cmcp_runtime(record)

findings: list[Finding] = []
jwk = trace.get("cnf", {}).get("jwk", {})
jwk = _jwk_of(trace)
kty = jwk.get("kty")
crv = jwk.get("crv")
x = jwk.get("x")

if "d" in jwk:
findings.append(Finding(
rule="TR-SIG-002",
status=Status.FAIL,
message="cnf.jwk must not contain private key material ('d' field present in JWK)",
"TR-SIG-004", Status.FAIL,
"TR-SIG-004: cnf.jwk must not contain private key material "
"('d' member present in the JWK)",
))
# The signature is not checked against a key the record should never have
# carried. Say so rather than returning nothing: a consumer reading TR-SIG-005
# to learn whether the signature was verified would otherwise find no finding
# at all, which is the benign-omission reading UNVERIFIED exists to prevent.
findings.append(Finding(
"TR-SIG-005", Status.UNVERIFIED,
"TR-SIG-005: signature not checked; cnf.jwk carries private key material",
))
return findings

Expand Down
Loading