fix(tr-sig): report a malformed record instead of raising on it

[lywinged]

Stacked on docs/error-code-registry, and opened as a draft until that lands. This branch contains that commit as well, so until it merges the diff here shows both: 12 files, not the 5 this change touches. GitHub re-scopes it automatically once the other PR is in. The TR-SIG-004 and TR-SIG-005 rows this change adds conditions back to are corrected there.

runner.run calls every module directly, with no try. A module that raises ends the run: the caller gets a traceback where a verdict belongs, and the record is neither passed nor failed. tr_sig did that in five ways.

$ python -m trace_tests.cli verify --record leaked.json
  File "src/trace_tests/runner.py", line 38, in run
    results["TR-SIG"] = tr_sig.check(trace, record, fmt, level)
  File "src/trace_tests/modules/tr_sig.py", line 126, in check
    findings.append(Finding(
TypeError: Finding.__init__() got an unexpected keyword argument 'rule'

The crash

Finding is a dataclass over code, status, message. There is no rule parameter, so that call has never produced a finding.

It is the check meant to catch a record that embeds its own private key, and it is the only such check in the suite. Nothing rejected the record earlier either: the packaged schema sets no additionalProperties on cnf.jwk and carries no guard against a d member. No test anywhere placed a d in a JWK, so the path had never run.

The other four

Same shape, one level down. trace.get("cnf", {}).get("jwk", {}) raises AttributeError whenever cnf or cnf.jwk is present but is not an object, and check_cmcp_runtime reads the same chain three deep. Measured before the change: a string, a number, null, a list or true in either position ends the run.

_jwk_of reads the JWK through isinstance guards and returns an empty dict when the record does not carry one as an object, which lands on the existing "kty is missing" finding. Both entry points use it, and its docstring says which of the two the outer guard is for: check_cmcp_runtime passes record["trace"] and can hand it anything, while check passes a trace that is already a dict and reads it directly elsewhere. That function is not hardened against a non-dict trace, and the helper does not pretend otherwise.

Which code the leak is reported under

TR-SIG-004, where docs/error-codes.md documents this condition, rather than TR-SIG-002, which the raising call named. TR-SIG-002 in this module belongs to check_cmcp_runtime and means a key that is not OKP/Ed25519; reusing it here would give one code two meanings across two formats.

A state this fix makes reachable for the first time

The leak branch returns early, so nothing else in the module runs. That path used to raise, so its outcome had never been observed. Making it reachable as it stood would publish a record carrying no signature verdict of any kind: not pass, not fail, not unverified. That is precisely the benign-omission reading UNVERIFIED exists to prevent, and one test in this suite already reads that finding with a bare next.

The branch now also emits TR-SIG-005 UNVERIFIED, "signature not checked; cnf.jwk carries private key material".

The tests

tests/test_modules_never_raise.py asserts that no module raises, and that none returns an empty finding list, over every top-level field replaced with each of nine junk values, every top-level field removed, the same two classes for cnf.jwk, and a JWK carrying d. The module list and each module's parameters are read from the signature rather than written down, so a module that gains a parameter is still exercised instead of dropping out quietly. False and 0 are among the junk values because a bare truthiness test reads them as absent, which is a different branch from a wrong type. Six of the seven modules already guarded their inputs with isinstance and pass unchanged; tr_sig was the outlier.

That parametrised test passes fmt="trace" throughout, so it never reaches check_cmcp_runtime. A separate case walks a cmcp envelope with the same junk values at trace, trace.cnf, trace.cnf.jwk and signature; without it, half of this change was hardened and untested.

Four further tests assert behaviour rather than the absence of an exception, because the generic test would pass if the checks were deleted outright:

• a record embedding its own private key fails under TR-SIG-004;
• the same record still reports whether its signature was checked;
• runner.run completes on it at every level, which is the path the traceback came out of and which a module-only test would not cover;
• no finding from any module repeats the key it found. report.py publishes every message into a JSON and an HTML artifact meant to be forwarded, so a message that quoted the offending value to be helpful would copy the private key into the thing the reader sends on. This runs over all seven modules rather than only the one that reports the leak, because a message anywhere could echo the value. The report itself carries only the record's digest, checked, so a message is the only place a key could be copied out.

Verified by reverting each change in turn; each has a test that fails without it and passes with it, including the cmcp read on its own. The report path was exercised end to end on the new status combination: JSON, HTML and the badge all build, level 0 reports 2 failures with 1 unverified, level 1 reports 4 with the unverified folded in, and no level is marked passed. The CLI now reports the record instead of ending the run.

TR-SIG  FAIL        TR-SIG-004: cnf.jwk must not contain private key material
                    ('d' member present in the JWK)
TR-SIG  UNVERIFIED  TR-SIG-005: signature not checked; cnf.jwk carries
                    private key material
Result: FAIL  (8 checks, 2 failure(s), 0 skipped)

Scope, and what is left

These tests mutate the fields of a record that is itself a dict. Three things nearby are not repaired here, each because it changes behaviour or decides what a new case reports, rather than fixing a call that never worked:

• A record whose trace is not a dict at all is a hole one layer up. loader.extract_trace returns record["trace"] unchecked for cmcp-runtime, so an absent trace raises KeyError there, and a non-dict trace raises AttributeError in the first module to read it.
• check_cmcp_runtime has no private-key check at all, so a cmcp runtime claim embedding d is detected by nothing.
• A signature that fails verification is carried as a TR-SIG-005 finding whose message names TR-SIG-003, so the code in the forwarded report and the code in the message disagree.

---

217 passed, 5 xpassed. ruff reports six findings in tr_sig.py, all of them identical on main and none in the lines this changes.

---

Generated by Claude Code

[github-actions]

❔ Contributor Check: UNKNOWN

Check	Result
Profile	UNKNOWN
Credential	LOW
Overall	UNKNOWN

Automated check by AgenTrust Contributor Check.