feat(snapshot): verify delivered snapshots against integrity receipts - #457
Merged
kstonekuan merged 1 commit intoSep 7, 2026
Merged
Conversation
5 tasks
kstonekuan
approved these changes
Sep 7, 2026
Contributor
There was a problem hiding this comment.
LGTM, merging.
You carried the #454 finding across before I asked. Both branches hold under mutation.
VARUN3WARE
added a commit
to VARUN3WARE/hflow
that referenced
this pull request
Sep 7, 2026
Import VerificationReport from the landed verification module (Hebbian-Robotics#457) and keep the import-delivery half in lerobot_verify: resolve landing/<basename> under the verified root, treat empty episodes as ok, and cover Kevin's copy cases.
This was referenced Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(snapshot): verify delivered snapshots against integrity receipts
Closes #428.
Problem
#401 stamps every exported snapshot with an integrity receipt (per-file size and sha256, plus an inventory content_id), but nothing checks a delivered snapshot against it. A truncated transfer looks exactly like a clean one.
What this adds
verify_dataset_snapshot(<directory>)andhflow verify snapshot <directory>re-read every table and copied asset named in theintegrityblock and compare size first, then sha256 (the hash read is skipped when the size already differs).0clean,1damaged,2unreadable input,3unverifiable.format.jsonwith nointegritykey is reported as ano-receiptfinding (unverifiable), not corruption. A missing directory or an unparsableformat.jsonraises to exit2: the wrong input, not damage.format.jsonit describes.Shared types, landed here first
The report shape lives in
src/hflow/verification.py:VerificationStatus(ok / damaged / unverifiable),VerificationFinding(uri, reason, detail),VerificationReportwith.okderived from the status,exit_code_for, and the reason constantsmissing,size-mismatch,content-id-mismatch,no-receipt. This is the shape agreed in the #432 contract with VARUN3WARE; #454 imports these types rather than redefining them.Recorded receipt paths are joined only onto the handed directory, so a copied delivery verifies in place.
Moved-root proof
test_moved_root_verifies_from_the_new_root_aloneexports a snapshot to root A, copies the whole delivery to root B, deletes A entirely, and verifies B clean, then damages one file under B and verifies B damaged. With A gone, any read outside the handed root would see nothing, so both reports can only come from B's own bytes. A variant keeps A alive while B is damaged and asserts the damage is still reported from B, not masked by A.Cost
One streaming sha256 pass per receipted file, plus the size stats. A size mismatch short-circuits that file's hash read. No catalog, storage, or media fetch traffic: verification reads only the handed directory.
Tests
13 tests in
tests/test_snapshot_verify.py: clean verifies in both media modes, same-size content damage, missing file, truncation reported by size with the hash read skipped (spy on_sha256_hex), copied-asset damage, pre-#401 no-receipt, unlisted extras ignored, multi-finding partial transfer, CLI exit codes 0/1/3/2 including empty and binary-garbageformat.json, read-only verification, and the two moved-root tests.Mutation check: with the size and sha256 comparisons disabled, the damage tests go red, including the spy assertion; restored, all green.