Skip to content

[POC] In-memory lmdb adapter for Pyodide/WASM - #16

Draft
jaelliot wants to merge 1 commit into
WebOfTrust:mainfrom
jaelliot:feature/lmdb-inmemory-adapter
Draft

[POC] In-memory lmdb adapter for Pyodide/WASM#16
jaelliot wants to merge 1 commit into
WebOfTrust:mainfrom
jaelliot:feature/lmdb-inmemory-adapter

Conversation

@jaelliot

@jaelliot jaelliot commented Apr 7, 2026

Copy link
Copy Markdown

Proof of Concept / Demo

This is a volatile, in-memory lmdb API adapter that allows keripy's LMDBer to boot inside Pyodide (WASM) where the real lmdb C extension cannot load.

Status: Proof of concept — not production-ready. Data is lost on page refresh.

What it does

  • Provides a dict-backed implementation of the lmdb API subset that keripy actually calls
  • Enables LMDBer(name=..., temp=True, reopen=True) to succeed in Pyodide
  • Covers: Environment, _Transaction, _Cursor, _Database, BadValsizeError
  • Supports dupsort databases with sorted duplicate values

What it does NOT do

  • No persistence — all data is in-memory Python dicts, lost on page refresh
  • No IndexedDB integration — that's Phase 2 (wrapping this adapter with async load/save)
  • No MVCC — single-threaded Pyodide doesn't need it

Validated against

  • 44 unit tests covering all keripy dbing.py cursor patterns
  • LMDBer integration test: instantiation, setVer/getVer, putVal/getVal/delVal, setVal, getTopItemIter, cntTop, remTop, cntAll, dupsort putVals/getVals/cntVals/getValLast/delVals, close

Bug found and fixed during integration

txn.delete(key, b'') on dupsort databases: real lmdb treats empty val as "delete all dups at key." The adapter initially treated b'' as a specific value lookup. Fixed to match real lmdb semantics.

Files changed

  • python/lmdb.py — single new file, ~499 lines, zero external dependencies (only bisect from stdlib)

- Dict-backed implementation of lmdb API subset used by keripy
- Covers Environment, Transaction, Cursor, and Database classes
- Supports dupsort databases with sorted duplicate values
- Matches real lmdb delete semantics (empty val deletes all dups)
- Passes 44 unit tests and LMDBer integration test
- Data is volatile (in-memory only), persistence deferred to Phase 2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant