Context
Env::copy_to_file's raw mode (crates/zerodb/src/copy.rs), used for Meilisearch's live-index snapshots, copies every source page through userspace into a buffer then writes it in one std::fs::write — minutes for a large database, and the open read transaction it holds meanwhile blocks garbage collection of pages newer commits freed.
Copy-on-write filesystems can clone a file near-instantly by duplicating block metadata: FICLONE (XFS/btrfs), clonefile (APFS), ZFS block cloning. This is safe: commits already fsync changed pages before writing and fsyncing the meta page recording the new state, so a meta page whose checksum passes on open only references durable pages (docs/SPEC/06-recovery.md) — a block clone just captures the file at an instant, exactly what recovery already tolerates.
Task
After making the transaction fully durable — force_sync, or a brief writer pause spanning the whole commit sequence (justify which suffices) — attempt a reflink clone; fall back to copy_file_range (in-kernel, not atomic, still needs the pause), then to today's streaming copy. Probe filesystem support at runtime, since it varies by filesystem, not OS. As a beyond-LMDB change, it needs a short approved design document (ADR) in docs/adr/ first.
Why it matters
Minutes-to-milliseconds snapshots shrink the reader-pinning window that keeps old pages unreclaimed — a real cost today for Meilisearch's snapshot/backup workflow.
Risk
A writer pause that doesn't cover the whole commit sequence can leave a meta page whose checksum passes but whose data was torn mid-write; recovery catches a torn meta, not valid metadata over torn data, so a wrong argument ships a silently corrupt snapshot.
See also: docs/SPEC/06-recovery.md — commit ordering and meta selection; crates/zerodb/src/copy.rs — current streaming copy implementation.
Context
Env::copy_to_file's raw mode (crates/zerodb/src/copy.rs), used for Meilisearch's live-index snapshots, copies every source page through userspace into a buffer then writes it in onestd::fs::write— minutes for a large database, and the open read transaction it holds meanwhile blocks garbage collection of pages newer commits freed.Copy-on-write filesystems can clone a file near-instantly by duplicating block metadata:
FICLONE(XFS/btrfs),clonefile(APFS), ZFS block cloning. This is safe: commits already fsync changed pages before writing and fsyncing the meta page recording the new state, so a meta page whose checksum passes on open only references durable pages (docs/SPEC/06-recovery.md) — a block clone just captures the file at an instant, exactly what recovery already tolerates.Task
After making the transaction fully durable —
force_sync, or a brief writer pause spanning the whole commit sequence (justify which suffices) — attempt a reflink clone; fall back tocopy_file_range(in-kernel, not atomic, still needs the pause), then to today's streaming copy. Probe filesystem support at runtime, since it varies by filesystem, not OS. As a beyond-LMDB change, it needs a short approved design document (ADR) indocs/adr/first.Why it matters
Minutes-to-milliseconds snapshots shrink the reader-pinning window that keeps old pages unreclaimed — a real cost today for Meilisearch's snapshot/backup workflow.
Risk
A writer pause that doesn't cover the whole commit sequence can leave a meta page whose checksum passes but whose data was torn mid-write; recovery catches a torn meta, not valid metadata over torn data, so a wrong argument ships a silently corrupt snapshot.
See also:
docs/SPEC/06-recovery.md— commit ordering and meta selection;crates/zerodb/src/copy.rs— current streaming copy implementation.