docs: align the published error codes and record samples with the modules - #74
Open
lywinged wants to merge 1 commit into
Open
docs: align the published error codes and record samples with the modules#74lywinged wants to merge 1 commit into
lywinged wants to merge 1 commit into
Conversation
…ules
Ten statements the documentation makes that the code does not do, across nine
pages. All live on main, none checked by anything.
Codes:
- TR-SIG-005 is carried by every signature finding tr_sig.py produces and is
documented nowhere. It is the signature check outcome: the Ed25519 result, a
signature that cannot be verified, or no signature at all, which is FAIL at
Level 1 and above and UNVERIFIED at Level 0. UNVERIFIED is deliberately not
SKIP, so an unsigned record cannot read as a benign omission.
- TR-SIG-004 was documented as private key material in cnf.jwk. It reports key
type: kty missing, or not in {OKP, EC}. The private-key condition is not
reported under this code, or any code, because the module raises before
producing a finding for it. That is a module bug, fixed separately.
- TR-ANC-002 is documented in three files and named by no module, with two
descriptions that disagree: docs/error-codes.md and docs/modules/tr-anc.md
call it the https-scheme check, which tr_anc.py performs under TR-ANC-001,
while docs/levels.md calls it a missing anchor.leaf_hash. Folded into
TR-ANC-001. Adding a second ANC code instead is a behaviour change.
- TR-ANC-001 was documented as requiring a resolvable URI and as rejecting a
placeholder value. Nothing resolves it, and no placeholder check exists
anywhere under src/. It checks presence, string type, https scheme and a
host. An earlier revision of this branch carried the placeholder claim
forward; it is removed.
- TR-RTE-001 documented `sev-snp`, `tdx` and `opaque` as valid platforms. None
is in _VALID_PLATFORMS, so a reader following the page produced a record
TR-RTE-001 rejects, while eight of the ten registered platforms went
unmentioned. The wrong list appeared in three pages, and docs/levels.md
carried a sample record using one of them.
- TR-RTE-003 was documented as resolving the RIM URI and checking the manifest
behind it. It checks that the string starts with https://, and nothing else.
- TR-ENV-004 was documented as a required-fields gate over the schema's full
required set, positive case "all of: eat_profile, iat, subject, ...". It
checks cnf.jwk.kty, and nothing else.
- TR-POL-002 lost track of `declared`, in four pages. Four values, not three.
- docs/modules.md summarises five of the seven modules as doing work they do
not do: private key leak detection (the path that raises), RIM URI
resolution, SCITT inclusion proof structure, builder URI, and required
fields. Each row now says what its module checks.
Samples:
- docs/levels.md shows an anchor object in its minimum conformant Level 2
record. schemas/trace-claim.json is additionalProperties: false and defines
no anchor property. Measured: valid_level0.json validates, and the same
record with the documented anchor block fails with "Additional properties are
not allowed ('anchor' was unexpected)". anchor.leaf_hash appears nowhere else
in the repository.
tests/test_docs_match_the_modules.py adds two guards.
The code set named by the modules must equal the code set with a row in
docs/error-codes.md. A row, not a mention: a code named in passing is not
documented, and accepting a mention would let the check be satisfied by prose
that tells a reader nothing. Every TR- code in this package lives under
modules/, so that scope is complete.
And every JSON sample under docs/ must agree with the packaged schema. Three
qualify today, all in docs/levels.md, but every .md is scanned rather than a
list kept by hand, because a hand-maintained list of what gets checked is the
same defect this exists to catch. The check also fails if it validated nothing,
so it cannot degrade to a pass over no work when the samples or the fence
change.
Both guards fail on main's documentation and pass on this one. Verified
individually: a code moved from a row into prose fails the first, a changed
fence fails the second, and each sample drift reintroduced alone fails alone.
What the guards do not do, stated in their docstrings. The code check matches
on codes named in module source, not on Finding.code: TR-SIG-003 appears only
inside a message string a TR-SIG-005 finding carries, so matching on
Finding.code would demand deleting a row that documents a real condition. And
neither guard can tell whether a row describes what its code reports, which was
six of the ten defects above; those were checked by reading the rows against
the module.
The sample guard drops `required` before validating, since the samples are
fragments, but leaves `if` and `not` intact. Stripping `required` from an `if`
makes it vacuously true and fires the matching `then` against records the
condition never meant to reach; the schema's `origin` rule does exactly that,
and an earlier draft of this guard reported two valid samples as broken. String
values containing an ellipsis are dropped before validation: a sample signature
written as eyJhbGciOiJFZERTQSJ9... is a reader's placeholder, not a claim about
the format.
Not touched: two sample CI transcripts under docs/tutorials/ quote output the
code no longer produces. Regenerating a transcript is a different kind of
change and they are left rather than edited by hand.
203 passed, 5 xpassed. ruff clean. No broken relative links, nav unchanged.
CI runs neither ruff nor mypy today.
Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
Contributor
|
❔ Contributor Check: UNKNOWN
Automated check by AgenTrust Contributor Check. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ten statements the published documentation makes that the code does not do, across nine pages. All of them are live on
main, and nothing checks any of it.Codes
TR-SIG-005is carried by every signature finding and documented nowhere. It is the signature check outcome: the Ed25519 verification result, a signature that cannot be verified, or no signature at all, which is FAIL at Level 1 and above andUNVERIFIEDat Level 0.UNVERIFIEDis deliberately held apart fromSKIPso an unsigned record cannot read as a benign omission, and that was undiscoverable from the page.TR-SIG-004was documented as private key material incnf.jwk. It reports key type:ktymissing, or not in{OKP, EC}. The private-key condition is not reported under this code, or under any code, because the module raises before producing a finding for it. That is a module bug and is fixed separately, not here.TR-ANC-002is documented in three files and named by no module, and two of them disagree about what it is.docs/error-codes.mdanddocs/modules/tr-anc.mdcall it thehttps://scheme check, whichtr_anc.pyperforms under TR-ANC-001;docs/levels.mdcalls it a missinganchor.leaf_hash. Folded into TR-ANC-001. Adding a second ANC code instead would be a behaviour change and is a separate decision.TR-ANC-001was documented as requiring a resolvable URI and as rejecting a placeholder value. Nothing resolves it, and no placeholder check exists anywhere undersrc/. It checks presence, string type,httpsscheme, and a host.TR-RTE-001documentedsev-snp,tdxandopaqueas valid platform values. None of the three is in_VALID_PLATFORMS, so a reader following the page produced a record TR-RTE-001 rejects, while eight of the ten registered platforms went unmentioned. The wrong list appeared in three pages, anddocs/levels.mdcarried a sample record using one of them.TR-RTE-003was documented as resolving the RIM URI and checking the manifest behind it. It checks that the string starts withhttps://, and nothing else.TR-ENV-004was documented as a required-fields gate over the schema's full required set, with a positive case reading "all of: eat_profile, iat, subject, ...". It checkscnf.jwk.kty, and nothing else.TR-POL-002lost track ofdeclared, in four pages. Four values, not three.docs/modules.mdsummarises five of the seven modules as doing work they do not do: private key leak detection (the path that raises), RIM URI resolution, SCITT inclusion proof structure, builder URI, and required fields. Each row now says what its module checks.Samples
The minimum conformant Level 2 record is not conformant.
docs/levels.mdshows ananchorobject;schemas/trace-claim.jsonisadditionalProperties: falseand defines noanchorproperty. Measured:valid_level0.jsonvalidates, and the same record with the documentedanchorblock fails withAdditional properties are not allowed ('anchor' was unexpected).anchor.leaf_hashappears nowhere else in the repository.The guards
tests/test_docs_match_the_modules.pyadds two.The code set named by the modules must equal the code set with a row in
docs/error-codes.md. A row, not a mention: a code named in passing is not documented, and accepting a mention would let the check be satisfied by prose that tells a reader nothing. EveryTR-code in this package lives undermodules/, so that scope is complete.Every JSON sample under
docs/must agree with the packaged schema. Three qualify today, all indocs/levels.md, but every.mdis scanned rather than a list kept by hand, because a hand-maintained list of what gets checked is the same defect this exists to catch. The check also fails if it validated nothing, so it cannot degrade into a pass over no work when the samples or the fence change.Verification
Both guards fail against
main's documentation and pass against this branch. Individually: a code moved out of its row and into prose fails the first; a changed code fence fails the second; and each sample drift reintroduced on its own fails on its own.What these guards do not do
Stated in their docstrings, because a guard that is trusted for more than it checks is the defect this PR is about.
The code check matches on codes named in module source, not on
Finding.code.TR-SIG-003appears only inside a message string that aTR-SIG-005finding carries, so matching onFinding.codewould demand deleting a row that documents a real condition.Neither guard can tell whether a row describes what its code reports, which was six of the ten defects above. Those were found by reading each row against its module, and the same reading is what a future row needs.
Implementation notes worth a reviewer's eye
The sample guard drops
requiredbefore validating, because the samples are fragments rather than whole records, but it leavesifandnotintact. Strippingrequiredfrom anifmakes it vacuously true and fires the matchingthenagainst records the condition was never meant to reach; the schema'soriginrule does exactly that, and an earlier draft of this guard reported two valid samples as broken.String values containing an ellipsis are dropped before validation. A sample signature written as
eyJhbGciOiJFZERTQSJ9...is a reader's placeholder, not a claim about the format.Not touched
Two sample CI transcripts under
docs/tutorials/quote output the code no longer produces. Regenerating a transcript is a different kind of change, so they are left as they are rather than edited by hand.Docs and tooling only. No module, schema or behaviour change. 203 passed, 5 xpassed (the TEE-hardware smoke set,
strict=Falseby design, unrelated).ruffclean, no broken relative links, nav unchanged. CI runs neitherruffnormypytoday.Generated by Claude Code