Skip to content

No support for nested write transactions (child txns with partial rollback) #71

Description

@qdequele

Context

LMDB (specifically the Meilisearch fork of LMDB that Meilisearch actually ships, mdb.master.nested-rtxns, vendored via lmdb-master-sys 0.2.6 / heed 0.22.1 — this is the parity target, not stock LMDB) supports nested write transactions: you can open a child write transaction inside a parent write transaction. The child's page modifications are "shadowed" (kept separate from the parent's), so if the child is aborted, only its changes are discarded and the parent's transaction continues unaffected. This gives partial-rollback / sub-batching within a single write transaction.

ZeroDB does not support this. It intentionally makes nested write transactions unrepresentable in its public API: there is no Env::nested_write_txn and no RwTxn::nested method. This isn't an oversight — heed itself (the typed Rust LMDB wrapper Meilisearch uses, and the API ZeroDB is a drop-in replacement for) only exposes RwTxn::nested as a private (pub(crate)) method, so no code outside heed can call it anyway. This decision is recorded as a short design document (ADR) in docs/adr/0007-nested-read-txns.md.

To be clear about what is supported: nested read transactions (a read-only transaction opened inside an active write transaction, which can see the write transaction's uncommitted state) work fully in ZeroDB. That's a different, Meilisearch-fork-specific feature that both milli (Meilisearch's core indexing engine) and hannoy (Meilisearch's HNSW vector-search index) already rely on. Only nested write transactions are the gap.

This gap is tracked as a documented, approved divergence from LMDB behavior (see docs/DIVERGENCES.md, the entry documenting that ZeroDB does not support nested write transactions) and has been flagged critical by maintainer directive as of 2026-07-22.

Task

Design and, if approved, implement nested write transaction support: a child write transaction whose page modifications are tracked separately from its parent's, such that aborting the child discards only the child's writes while the parent transaction remains valid and can continue or commit normally.

This requires a short design document (ADR) written and approved before any implementation, covering at minimum:

  • The public API shape (how a consumer would open/commit/abort a nested write transaction).
  • How per-child modified-page tracking interacts with ZeroDB's existing copy-on-write dirty-page store (the in-memory structure a write transaction uses to hold modified pages before they're flushed to disk).
  • Interaction with garbage collection / free-page reuse (pages freed inside a child that gets aborted must not be treated as freed).
  • Interaction with cursor tracking across parent/child boundaries.

Given that no current consumer uses this feature — zero call sites across all five pinned Meilisearch-side consumers, and heed itself doesn't expose it publicly — the repository's stated scope rule applies: start with a spike that reveals the actual blast radius (how much of the write path has to change) rather than a full staged implementation. Decide whether to continue based on what the spike finds.

Why it matters

This is a real capability gap versus LMDB, not just a missing convenience. Two ways it could bite:

  1. If a future version of heed exposes RwTxn::nested publicly, ZeroDB would be unable to serve it as a drop-in replacement.
  2. If milli or hannoy ever wants partial-rollback batching within one write transaction (e.g., try a batch of changes, discard just that batch on failure, without aborting the whole transaction), ZeroDB has no way to provide it today.

Retrofitting page shadowing into ZeroDB's existing write path (which currently assumes one flat dirty-page store per write transaction, not a nested hierarchy) is expected to be substantial, invasive work — hence the spike-first approach.

Risk

No consumer currently depends on this, so there's no urgency from breakage — the risk is entirely in scope: a naive implementation could balloon into a large change touching the dirty-page store, garbage collection, and cursor handling simultaneously (this has happened before on other features in this codebase). The spike-first approach exists specifically to avoid that.

See also

  • docs/adr/0007-nested-read-txns.md — nested read txn design, closest prior art
  • crates/zerodb-core/src/rwtxn.rs — current write transaction implementation

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:apiNew engine/adapter API surfacearea:write-pathPuts, COW, dirty store, splitscriticalCritical: must-track gap vs LMDB paritydivergenceSanctioned behavior divergence vs LMDB (docs/DIVERGENCES.md)needs-adrRequires a short approved design doc (ADR in docs/adr/) before implementationsize:XLExtra large: multi-week, ADR-gated

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions