Support multiple processes accessing the same database

[qdequele]

Context

LMDB (the C storage engine ZeroDB is API-compatible with, via heed, the Rust wrapper Meilisearch uses) lets multiple OS processes open the same database file at the same time. It does this with a companion lock.mdb file: a shared-memory table listing active read transactions, plus a mutex that serializes the single writer, with checks that a listed process is still alive (by PID) and cleanup of slots left behind by processes that crashed while holding one.

ZeroDB currently supports only one process at a time, by design. There is no lock file. Opening the same database path twice within one process is rejected with an EnvAlreadyOpened error, and a second process cannot open the file at all — external tools refuse outright rather than attach to a database another process has open (see crates/zerodb-tools/tests/live_env_refusal.rs). This is a deliberate, already-approved decision documented in docs/DIVERGENCES.md (ZeroDB's list of intentional behavior differences from LMDB) — not an oversight. It is currently flagged as the largest operational gap versus LMDB and marked critical priority by the maintainers, so this issue tracks what closing it would take.

Task

This is a scoping/tracking issue, not a request to implement cross-process access immediately, and it does not reverse the single-process decision on its own. Before any implementation work starts, a short design document (ADR) must be written in docs/adr/ describing the cross-process concurrency protocol and get maintainer approval — this affects commit durability and how ZeroDB decides which transactions can see which data, so it needs to be designed deliberately rather than bolted on.

The design needs to cover, at minimum:

• A shared-memory reader table usable across processes (LMDB-style), extending or replacing the current in-process registry in crates/zerodb-core/src/readers.rs. docs/adr/0006-reader-table.md documents the existing single-process reader table design and is the starting point.
• A lock-file protocol for cross-process coordination: detecting that a process listed as a reader or writer has died (PID liveness), and a safe way to reclaim its slot/lock without letting two writers proceed at once.
• A cross-process write mutex — today ZeroDB's single-writer rule is enforced only with an in-process mutex, which provides no protection across separate processes.

Once approved, the implementation would touch crates/zerodb-core/src/readers.rs plus the environment-open/registry code path.

Why it matters

Without this, ZeroDB cannot support deployment patterns LMDB users rely on: running a stat/check/backup tool against a live database from a separate process, or an external tool doing an LMDB-style hot copy while the main process keeps writing. For Meilisearch specifically, this blocks tooling that would inspect or back up a running instance's storage without stopping it first.

Risk

High. This changes the concurrency protocol underneath commit durability and read-transaction visibility guarantees — exactly the kind of change where a subtle bug (e.g. a reclaimed lock slot that lets two writers run concurrently, or a crashed reader that isn't correctly excluded from garbage collection) causes data corruption rather than a crash. That is why a design document and explicit maintainer approval are required before writing any code, and this issue alone should not be treated as authorization to start.

See also:

• docs/DIVERGENCES.md — full list of sanctioned LMDB behavior differences
• docs/adr/0006-reader-table.md — existing single-process reader table design