Context
ZeroDB is a pure-Rust embedded key-value store: a transactional B+tree over a memory-mapped file, built as a drop-in replacement for LMDB at the API level used by heed (the Rust LMDB wrapper Meilisearch uses). Its two consumers are milli (Meilisearch's indexing/search core) and hannoy (Meilisearch's HNSW vector index).
docs/PERF-GAP-VS-LMDB.md tracks every known time/memory cost where ZeroDB is currently slower or heavier than LMDB, plus a section titled "Deliberate — keep, do not optimize" that records costs kept on purpose (e.g. the per-commit checksum on the meta page, kept because crash-detection depends on it).
During the broader performance investigation, several well-known storage-engine techniques were evaluated for ZeroDB's design and explicitly rejected — but the reasoning was never written down anywhere. Without a record, a future contributor (or a future session) is likely to re-propose one of these and re-derive the same conclusion from scratch.
Task
Add one paragraph per technique to the "Deliberate — keep, do not optimize" section of docs/PERF-GAP-VS-LMDB.md, each stating the technique and the specific reason it doesn't fit ZeroDB:
O_DIRECT commit writes (a Linux flag that makes writes bypass the OS page cache): incompatible with ZeroDB's read path, which maps the file with MAP_SHARED and reads straight out of the page cache. Linux does not guarantee invalidating already-mapped pages when a file is written with O_DIRECT, so readers could see stale data. The "double buffering" problem O_DIRECT normally avoids (one copy in the app's write buffer, one in the page cache) doesn't exist here — the mmap'd pages are the page-cache pages the write call fills.- Bε-trees / buffered writes (buffering writes in internal nodes and flushing them down in batches, as in BetrFS/TokuDB, to turn random writes into sequential ones): would force every read to merge in pending buffered writes, destroying the zero-copy reads mmap currently gives and adding read latency.
milli's workload is read-bound, so this trade is wrong for ZeroDB's main consumer.
- Per-page compression: the two dominant value types ZeroDB stores don't compress well — Roaring-bitmap-encoded document ID sets and float32 vectors for
hannoy's HNSW index. It would also require a buffer pool (a scratch area to decompress into) that ZeroDB doesn't have yet.
- A hash-table sub-database type (as an alternative to the B+tree, for O(1) lookup): the actual cost of walking down the tree has already been sharply reduced by other read-path fixes landed in this repo (per-transaction caching of resolved records and validated pages). A hash table would make the real remaining costs — page faults and memory locality — worse, and would lose ordered iteration, which both
milli and hannoy rely on for range scans.
- Bloom filter sidecars (a probabilistic structure some engines use to skip lookups for keys that don't exist):
milli already gates word existence with its own FSTs (finite-state transducers used for fast prefix/existence lookups) before ever touching the store, so a bloom filter would be redundant — and keeping one transactionally consistent under MVCC costs more than it saves.
- Pointer swizzling (replacing on-disk page IDs with in-memory pointers once a page is loaded, used by buffer-pool engines like LeanStore's vmcache): ZeroDB's mapped pages are immutable, shared, and already have stable addresses via mmap — exactly the property swizzling exists to provide, and ZeroDB gets it for free from the OS.
MDB_FIXEDMAP (an LMDB flag that maps the database at a fixed virtual address instead of letting the OS choose): experimental and unstable upstream, fights address-space-layout randomization, doesn't handle file resizes cleanly, and no consumer needs it.
Why it matters
This is a documentation-only change, but it protects real engineering time: without it, the same seven ideas are likely to resurface during Phase 3 exploration (ZeroDB's roadmap phase for improvements beyond plain LMDB parity) and get re-evaluated from zero each time. Writing the disqualifying argument down once makes the performance document a complete decision record instead of a partial one.
See also: docs/PERF-GAP-VS-LMDB.md — performance inventory, target file for this change.
Context
ZeroDB is a pure-Rust embedded key-value store: a transactional B+tree over a memory-mapped file, built as a drop-in replacement for LMDB at the API level used by
heed(the Rust LMDB wrapper Meilisearch uses). Its two consumers aremilli(Meilisearch's indexing/search core) andhannoy(Meilisearch's HNSW vector index).docs/PERF-GAP-VS-LMDB.mdtracks every known time/memory cost where ZeroDB is currently slower or heavier than LMDB, plus a section titled "Deliberate — keep, do not optimize" that records costs kept on purpose (e.g. the per-commit checksum on the meta page, kept because crash-detection depends on it).During the broader performance investigation, several well-known storage-engine techniques were evaluated for ZeroDB's design and explicitly rejected — but the reasoning was never written down anywhere. Without a record, a future contributor (or a future session) is likely to re-propose one of these and re-derive the same conclusion from scratch.
Task
Add one paragraph per technique to the "Deliberate — keep, do not optimize" section of
docs/PERF-GAP-VS-LMDB.md, each stating the technique and the specific reason it doesn't fit ZeroDB:O_DIRECTcommit writes (a Linux flag that makes writes bypass the OS page cache): incompatible with ZeroDB's read path, which maps the file withMAP_SHAREDand reads straight out of the page cache. Linux does not guarantee invalidating already-mapped pages when a file is written withO_DIRECT, so readers could see stale data. The "double buffering" problemO_DIRECTnormally avoids (one copy in the app's write buffer, one in the page cache) doesn't exist here — the mmap'd pages are the page-cache pages the write call fills.milli's workload is read-bound, so this trade is wrong for ZeroDB's main consumer.hannoy's HNSW index. It would also require a buffer pool (a scratch area to decompress into) that ZeroDB doesn't have yet.milliandhannoyrely on for range scans.millialready gates word existence with its own FSTs (finite-state transducers used for fast prefix/existence lookups) before ever touching the store, so a bloom filter would be redundant — and keeping one transactionally consistent under MVCC costs more than it saves.MDB_FIXEDMAP(an LMDB flag that maps the database at a fixed virtual address instead of letting the OS choose): experimental and unstable upstream, fights address-space-layout randomization, doesn't handle file resizes cleanly, and no consumer needs it.Why it matters
This is a documentation-only change, but it protects real engineering time: without it, the same seven ideas are likely to resurface during Phase 3 exploration (ZeroDB's roadmap phase for improvements beyond plain LMDB parity) and get re-evaluated from zero each time. Writing the disqualifying argument down once makes the performance document a complete decision record instead of a partial one.
See also: docs/PERF-GAP-VS-LMDB.md — performance inventory, target file for this change.