Skip to content

perf: O(1) select-by-ID in DBSQLMemoryAdapter; add JSON and DB benchmarks - #151

Merged
gmpassos merged 4 commits into
masterfrom
perf/db-json-optimizations
Aug 12, 2026
Merged

perf: O(1) select-by-ID in DBSQLMemoryAdapter; add JSON and DB benchmarks#151
gmpassos merged 4 commits into
masterfrom
perf/db-json-optimizations

Conversation

@gmpassos

@gmpassos gmpassos commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Adds JSON and DB benchmark suites, and one optimization they turned up.

The optimization: O(1) select-by-ID in DBSQLMemoryAdapter

_selectEntries answered every condition by scanning the table Map and evaluating the condition per row — including a ConditionID, even though that Map is keyed by ID and the adapter already has an O(1) lookup helper. So selectByID was O(rows):

rows before after
10 7.80 µs 7.13 µs
50 9.73 µs 7.16 µs
400 25.26 µs 7.22 µs

Now flat. A lookup miss falls through to the original scan, so a ConditionID whose value doesn't match a key exactly (a String '7' against an int 7) resolves exactly as before — just as slowly as it always did. Results are unchanged either way.

This mostly speeds up the test suite and development, since the memory adapter is where those run.

A correction

An earlier commit on this branch claimed the cost of a query "is the repository/transaction machinery". That was wrong, and I'd published it before checking. Scaling the row count shows selectByQuery is linear:

rows selectByQuery
10 10.9 µs
50 20.4 µs
400 105.9 µs

≈ 8.5 µs fixed + ~0.24 µs/row. The number was dominated by the memory adapter's scan, which a real SQL adapter never pays. Only the fixed part is shared framework cost. benchmark/README.md, the CHANGELOG and the in-file comments now say so, and the DB suite takes --rows=N so the two can be separated.

What the suites show otherwise

JSON needs nothing. Json.encodeToSink — the response path — runs close to a bare dart:convert encode (~1.5 µs vs ~1.2 µs, small map) and is faster on larger payloads, since it writes bytes to a sink instead of building a String. Request bodies go through dart:convert directly, so there's no layer to remove.

DB, at 50 rows:

µs/op
ConditionParseCache.parseQuery (cached) 0.010
Entity.toJson 0.074
generateSelectSQL 0.83
EntityHandler.createFromMap 1.12
ConditionParser.parse (shared parser) 2.74
Transaction.executeBlock (empty) 2.43
repository.selectByID 7.26

The two things a query is assumed to be expensive for are not: parsing is cached at ~300× cheaper than parsing, and SQL generation is under a microsecond.

Two hypotheses that failed

Recorded so they aren't re-tried:

  1. Per-response JsonEncoder. Json._buildJsonEncoder returns the cached defaultEncoder only when every argument is defaulted — and the response path always passes toEncodable, so it builds a fresh encoder each time. Memoizing it (safe: autoResetEntityCache defaults to true) moved the small-map encode 1.555 → 1.522 µs and made the 50-map case slightly worse. Noise.
  2. Eager Completers. Transaction creates three in its constructor despite the fields being late final. Making them lazy left an empty executeBlock at 2.40 µs, unchanged.

Neither is included.

Another measurement error worth flagging

My first DB run reported ConditionParser.parse at 125 µs, which looked like a major find. It was my benchmark constructing a ConditionParser per iteration — the cost is PetitParser building the grammar lazily. bones_api holds it in a static final, so it's a one-off startup cost and there's no bug. The corrected benchmark reuses the parser (2.74 µs), with a note in the README since a per-query parser would be catastrophic.

Where to look next

Transaction.executeBlock — 2.4 µs for an empty block is the floor under every DB operation and the largest remaining fixed cost. That wants a real profiler (dart run --observe), not more micro-benchmarks.

The DB suite carries a small self-contained BenchUser entity so it doesn't depend on the test fixtures.

Gate Result
dart test --exclude-tags docker 783 passed
dart analyze --fatal-infos --fatal-warnings . clean
dart format -o none --set-exit-if-changed . clean

🤖 Generated with Claude Code

@gmpassos
gmpassos changed the base branch from perf/request-optimizations to master August 12, 2026 06:59
Measurement only. I went looking for optimizations in the DB and JSON paths
and did not find one worth shipping, so this adds the suites and what they
say, rather than a change that does not survive being measured.

`json_benchmark.dart` — encoding and decoding, in the shapes the server uses.
`db_benchmark.dart` — the entity path against `DBSQLMemoryAdapter`, whose
storage is a `Map`, so the numbers are the framework around the query rather
than I/O. It carries a small self-contained entity so it does not depend on
the test fixtures.

What they show:

- JSON is already fine. `Json.encodeToSink` (the response path) runs close to
  a bare `dart:convert` encode of the same value (~1.5us vs ~1.2us for a small
  map) and is *faster* on larger payloads, since it writes bytes to a sink
  instead of building a `String`. Request bodies go through `dart:convert`
  directly, so there is no layer to remove there.

- On the DB side the two things a query is assumed to be expensive for are
  not. Query parsing is cached (0.009us, ~300x cheaper than parsing) and SQL
  generation is 0.81us. The cost is the machinery around them: an *empty*
  `Transaction.executeBlock` is 2.4us, and a `selectByQuery` against an
  in-memory `Map` is 20.6us.

Two hypotheses that failed, recorded so they are not re-tried:

- `Json._buildJsonEncoder` builds a fresh `JsonEncoder` whenever a
  `toEncodable` is given, which the response path always does. Memoizing it
  moved the small-map encode 1.555us -> 1.522us and made the 50-map case
  slightly worse — noise.
- `Transaction` creates three `Completer`s in its constructor despite the
  fields being `late final`. Making them lazy left an empty
  `executeBlock` at 2.40us, unchanged.

Neither is in this commit. The next pass should start at the
repository/transaction layer, and with a profiler (`dart run --observe`)
rather than more micro-benchmarks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gmpassos
gmpassos force-pushed the perf/db-json-optimizations branch from f3b1dd9 to f657126 Compare August 12, 2026 07:02
Follows the benchmark suites added in the previous commit, and corrects a
conclusion I drew from them.

`DBSQLMemoryAdapter._selectEntries` answered every condition by scanning the
table `Map` and evaluating the condition per row — including a `ConditionID`,
even though that `Map` is keyed by ID and the adapter already has an O(1)
lookup helper. `selectByID` was therefore O(rows):

    rows      before     after
      10      7.80us     7.13us
      50      9.73us     7.16us
     400     25.26us     7.22us

Now flat. A lookup miss falls through to the original scan, so a `ConditionID`
whose value does not match a key exactly (a `String` '7' against an `int` 7)
resolves exactly as before, just as slowly as it always did. Results are
unchanged either way.

This mostly benefits the test suite and development, which is where the memory
adapter runs.

Correction: the previous commit claimed the cost of a query "is the
repository/transaction machinery". That was wrong. Scaling the row count shows
`selectByQuery` is linear — 10.9us at 10 rows, 20.4us at 50, 105.9us at 400,
about 8.5us fixed plus ~0.24us per row — so the number was dominated by the
memory adapter's scan, which a real SQL adapter does not pay. Only the fixed
part is shared framework cost, and an empty `Transaction.executeBlock` (2.4us)
is most of it. `benchmark/README.md` and the CHANGELOG now say so, and the DB
suite takes `--rows=N` so the two can be separated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.16%. Comparing base (f169496) to head (2be651a).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #151      +/-   ##
==========================================
+ Coverage   68.14%   68.16%   +0.02%     
==========================================
  Files          66       66              
  Lines       22129    22137       +8     
==========================================
+ Hits        15080    15090      +10     
+ Misses       7049     7047       -2     
Flag Coverage Δ
unittests 68.16% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The file-level and section comments still said the gap between the layered
measurements and the repository calls was framework overhead. It is mostly the
memory adapter's per-row scan. Also drops the hardcoded row count from a label
now that `--rows=N` exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gmpassos gmpassos changed the title bench: add JSON and DB benchmark suites perf: O(1) select-by-ID in DBSQLMemoryAdapter; add JSON and DB benchmarks Aug 12, 2026
Chased the ~2.3us empty `Transaction.executeBlock` — the largest remaining
fixed cost on the DB path — and did not find a hotspot to remove. Recording
the eliminations so the next attempt does not start from zero:

  Transaction() ctor                        0.068us
  Zone.current.fork()                       0.032us
  asyncTry (sync, onError + onFinally)      0.030us
  Completer()                               0.011us
  commit logging (root=INFO vs OFF)         ~0.23us
  executeBlock (empty)                      2.3us

A nested `executeBlock` adds ~0.04us, so the short-circuit to an enclosing
transaction is already optimal.

Notably the commit log — guarded by `isLoggable`, which is true by default —
is only ~10% here, unlike the request path where the same pattern was ~75%.
No single piece accounts for the total; the remainder is spread across the
async plumbing of the commit path, where an `await` of an already-completed
value alone costs ~0.15us. Removing that means restructuring the synchronous
path so it stops allocating futures, which is a deliberate refactor rather
than an incremental win.

No code change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gmpassos
gmpassos merged commit 82751af into master Aug 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant