refactor(snapshot): type receipt entries while preserving the content_id invariant - #493
Conversation
…rrowed Typing the entries changed the digest from 'every key the entry carried' to 'path, size_bytes, sha256'. An entry with an extra key now hashes the same where it used to hash differently: verified against main, which raises on that marker while this branch certifies it. That is the right trade, because it lets a later format revision add metadata without invalidating every snapshot already exported, and the receipt was never a tamper defence. But it was silent in both the code and the tests, so it gets a case and a paragraph.
There was a problem hiding this comment.
LGTM, merging.
Verified across versions rather than within one: exported a copy-mode snapshot on main, then ran this branch's verifier against it. Digest identical.
Pushed one commit. The digest now covers path, size_bytes, sha256 and nothing else, so an entry with an extra key hashes the same where it used to differ. That is the right trade (a later revision can add metadata without invalidating existing snapshots) but it was silent in the code and the tests.
|
Fair point on the cross-version check. I proved the two paths agreed on the branch, which only proves internal consistency, not stability. Exporting on main to verify against the branch is exactly the blind spot I missed. The extra-key delta is a sharp catch too. I was so focused on byte-identity for today's shape that I didn't map out the input space for future metadata. I will carry that four-point checklist (added, removed, reordered, retyped) into anything involving serialization from now on. I am starting the lane-entry validation for #474 now. |
2 participants
refactor(snapshot): type receipt entries while preserving the content_id invariant
Closes #489.
What changed
Receipt entries no longer travel as
dict[str, str | int]through the verifier. A frozenFileIntegrityRecord(path: str, size_bytes: int, sha256: str) is parsed once per entry at the marker-reading boundary, next to the #473 content_id gate, and everything after the boundary is typed: the inventory gate, the per-file loop, and the sha256 comparison at what was :942.The boundary refuses without coercion, naming the exact field: a receipt whose
sha256arrived as a JSON number used to reach a comparison that can never succeed and was reported as damaged bytes (exit 1, "your data is damaged"); the truth is that the receipt is malformed, which is unreadable input (exit 2, "your receipt is written wrong"). The same refusal covers a non-string path, a non-int size_bytes, and bool masquerading as int.The invariant, proven
The hard constraint:
_inventory_content_idserializes entries withjson.dumps(sorted-by-path, sort_keys=True, separators=(",", ":")), and #483 made every verify compare that digest against the storedcontent_id. Moving one byte breaks every snapshot ever exported.Proof from this branch: the exact serialized string over an 8-entry inventory (7 tables, 1 asset, 1000 characters) and its digest are byte-identical before and after the refactor. Digest before and after:
c117d138cfdd8cd4c613906b8813aa92a89a96192c89ee6522e7a30fc9b69b9a. The bridge isto_dict_for_hashing(), which rebuilds exactly the dict shape the exporter has always serialized, and the digest is computed over that dict form, never over the dataclass.Durable pins in the suite: an invariant test that computes the digest the old raw-dict way and the new records way and asserts both equal a computed golden value, and a refusal test feeding a numeric sha256 and asserting the boundary raises naming the field.
Exporter unchanged, proven
The exporter now builds records and serializes them through
to_dict_for_hashingwhen writing the marker, so the emittedformat.jsoncarries the identical dict shapes. The dataset-snapshot suite, which pins the marker's structure and recomputes itscontent_id, passes unchanged apart from parsing records where it recomputes.Mutation
With the parser changed to accept an int
sha256, exactly the refusal test fails and the other 42 stay green. Restored, all green.Validation
43 passed across tests/test_snapshot_verify.py and tests/test_dataset_snapshot.py, ruff check, ruff format, ty clean on the changed files, zero non-ASCII and zero em dashes. Boundary-family note: #486 (format identity) and #483 (inventory gate) are merged; #469 (path containment) stays with VARUN3WARE; #474 (post-sync CRC reads) is claimed and next.