refactor: consolidate commit ingest, subscribe auth, and bind-twin fixtures - #1274
Draft
joepio wants to merge 7 commits into
Draft
refactor: consolidate commit ingest, subscribe auth, and bind-twin fixtures#1274joepio wants to merge 7 commits into
joepio wants to merge 7 commits into
Conversation
cursor
Bot
changed the title
Map duplicated logic across Rust, TypeScript, and Flutter, and classify what should stay dual-maintained versus what should merge. Links existing plans instead of restating them. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
A cleanup is only allowed to land if characterization tests pin the old paths first, then scripts/consolidation-measure.py shows production lines in scope went down. Policy-splits (same shape, different CommitOpts) are called out so they cannot hide a behavior change. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
Flutter catch-up applies a signed commit without re-checking ACL. Hub and peer ingest still reject the same unauthorized signer. Characterization for folding ws_apply::apply_commit_json into ingest_commit_json. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
Fold Flutter WS apply_commit_json into ingest_commit_json via CommitIngestOpts::replica() (signature on, rights/timestamp off). Hub and peer keep enforcing rights. Named presets replace repeated struct literals. Subscribe, SubscribeDrive, and SubscribeQuery share authorize_read so the three check_read copies cannot drift. Measure: 2333 → 2257 production non-blank in scope; largest file engine.rs 1084 → 918. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
Mark the two landed items in the audit and add an UNRELEASED changelog entry so the remaining work list stays accurate. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
Rust, TypeScript, and Dart now load testdata/search-query.json and testdata/server-url.json instead of each suite inventing the same expected bytes. The File-picker repro looks up the escaped isA key instead of shipping a third escape implementation. CI mounts the whole testdata directory so new fixtures do not need a one-off file mount. Empty server-URL input is left unbound: TS returns https://, Dart returns empty. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
flutter/AGENTS.md still said Loro was missing; Phase A already landed. scripts/consolidation-measure.py --kind bind-twins allows a flat production line count so a shared fixture is not rejected as a no-op. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com>
joepio
force-pushed
the
cursor/consolidate-ingest-subscribe-auth-88f8
branch
from
August 18, 2026 15:49
abe4f0a to
4a8f8aa
Compare
Member
Author
|
Decision (accepted 2026-09-01, |
This was referenced Sep 1, 2026
joepio
added a commit
that referenced
this pull request
Sep 1, 2026
Add RFC-style decision documents, each with a "Decision needed by maintainer" box and consequences for open PRs: - runtime-boundary-decision.md: Rust decides, TS displays; twins only for pure fixtured functions; first AtomicNode slice. - authority-unit-decision.md: drive stays the identity/replication unit; additive zone chain is the rights unit; what #1254 must change. - commit-retention-floor-decision.md: envelope-on-resource floor; sequence #1274 -> #1313 -> #1254. - trust-model-decision.md: the node that owns the URL is trusted with plaintext; anything that only stores is blind. - schema-routes-decision.md: did:ad:frozen is the on-ramp; optional schema is policy; defaults fingerprint closes the repopulate gap. Index them under "Decisions pending" in planning/README.md. Claude-Session: https://claude.ai/code/session_019asLKBrBWY5ovyeCgtmdSd
joepio
added a commit
that referenced
this pull request
Sep 1, 2026
* docs(planning): five architecture decision documents Add RFC-style decision documents, each with a "Decision needed by maintainer" box and consequences for open PRs: - runtime-boundary-decision.md: Rust decides, TS displays; twins only for pure fixtured functions; first AtomicNode slice. - authority-unit-decision.md: drive stays the identity/replication unit; additive zone chain is the rights unit; what #1254 must change. - commit-retention-floor-decision.md: envelope-on-resource floor; sequence #1274 -> #1313 -> #1254. - trust-model-decision.md: the node that owns the URL is trusted with plaintext; anything that only stores is blind. - schema-routes-decision.md: did:ad:frozen is the on-ramp; optional schema is policy; defaults fingerprint closes the repopulate gap. Index them under "Decisions pending" in planning/README.md. Claude-Session: https://claude.ai/code/session_019asLKBrBWY5ovyeCgtmdSd * docs(planning): match the decisions index to the documents Claude-Session: https://claude.ai/code/session_019asLKBrBWY5ovyeCgtmdSd * docs(planning): record the five architecture decisions as accepted All five RFCs accepted 2026-09-01. Status boxes record the outcome, the README table is renamed to "Decisions", and encryption.md is closed to "at-rest + vault" with an explicit reopen test. Claude-Session: https://claude.ai/code/session_019asLKBrBWY5ovyeCgtmdSd
joepio
added a commit
that referenced
this pull request
Sep 1, 2026
Adds `atomic_lib::runtime::AtomicNode`, the thin node surface proposed in
planning/runtime-boundary-decision.md. It wraps `Db` and delegates every
method to code that already existed, so there is no behaviour change:
- `open(NodeConfig)` -> `Db::init_memory/init_redb/init_redb_file/init_redb_opfs`
- `get` -> `get_resource_extended`, `query` -> `Storelike::query`
- `apply_commit(json, IngestPolicy)`:
Hub/Peer -> `sync::engine::ingest_commit` (extracted from
`ingest_commit_json`, which now wraps it and serialises the response),
Replica -> `sync::ws_apply::apply_commit_json` (now returns
`CommitResponse`), LocalCache -> the WASM `applyCommit` path.
- `mutate(ResourceEdit)` -> `Resource::save_locally` / `save_as_genesis`
- `subscribe()` -> `Db::subscribe_events`
- `sync_with_peer` (iroh) -> `sync::peer::sync_drive_with_peer_outcome`
`IngestPolicy::{Hub, Peer, Replica, LocalCache}` mirrors the
`CommitIngestOpts::{hub, peer, replica}` presets of #1274 plus the WASM
no-validation path, so that PR can map onto it 1:1 once merged.
The WASM `ClientDb` is the first consumer: it now holds an `AtomicNode`
and routes `applyCommit` and `query` through it.
Tests: two nodes in one process, `mutate` on one, `apply_commit(Peer)`
on the other, query and `DbEvent` reflect it; LocalCache accepts a
tampered signature while Peer rejects it; mutate without agent errors.
Claude-Session: https://claude.ai/code/session_019asLKBrBWY5ovyeCgtmdSd
joepio
added a commit
that referenced
this pull request
Sep 1, 2026
Adds `atomic_lib::runtime::AtomicNode`, the thin node surface proposed in
planning/runtime-boundary-decision.md. It wraps `Db` and delegates every
method to code that already existed, so there is no behaviour change:
- `open(NodeConfig)` -> `Db::init_memory/init_redb/init_redb_file/init_redb_opfs`
- `get` -> `get_resource_extended`, `query` -> `Storelike::query`
- `apply_commit(json, IngestPolicy)`:
Hub/Peer -> `sync::engine::ingest_commit` (extracted from
`ingest_commit_json`, which now wraps it and serialises the response),
Replica -> `sync::ws_apply::apply_commit_json` (now returns
`CommitResponse`), LocalCache -> the WASM `applyCommit` path.
- `mutate(ResourceEdit)` -> `Resource::save_locally` / `save_as_genesis`
- `subscribe()` -> `Db::subscribe_events`
- `sync_with_peer` (iroh) -> `sync::peer::sync_drive_with_peer_outcome`
`IngestPolicy::{Hub, Peer, Replica, LocalCache}` mirrors the
`CommitIngestOpts::{hub, peer, replica}` presets of #1274 plus the WASM
no-validation path, so that PR can map onto it 1:1 once merged.
The WASM `ClientDb` is the first consumer: it now holds an `AtomicNode`
and routes `applyCommit` and `query` through it.
Tests: two nodes in one process, `mutate` on one, `apply_commit(Peer)`
on the other, query and `DbEvent` reflect it; LocalCache accepts a
tampered signature while Peer rejects it; mutate without agent errors.
Claude-Session: https://claude.ai/code/session_019asLKBrBWY5ovyeCgtmdSd
joepio
added a commit
that referenced
this pull request
Sep 5, 2026
… History Every applied signed commit leaves its JSON-AD on the resource it changed, keyed pure_id || createdAt || signature, written in the apply transaction. Not a resource, not indexed. Retention per node: latest (default, the envelope that produced the current state) or all (a signed audit log), via --envelope-retention / ATOMIC_ENVELOPE_RETENTION. Every commit's Loro change carries a token (browser drain token; Rust builder and create_did now tag too), so an envelope maps to the History version it produced. envelopes::attribute_history verifies signatures with the apply code, credits each token to one envelope (the genesis carrier only to a genesis envelope), and reports completeness. Read paths: GET /history-attribution (read-gated), WASM ClientDb.historyAttribution, Store.getHistoryAttribution merging both. History shows by <agent> Verified / Unverified / Unattributed. The destroy envelope on the tombstone value (#1370) is now the subject's latest row in this tree; the tombstone is a marker again. Tests: lib envelopes (8), server it history_attribution, browser lib history-attribution (5), e2e history assertions. Planning: decision doc amended (no #1274 gating, retention knob), auditability doc now Building.
joepio
added a commit
that referenced
this pull request
Sep 5, 2026
…on, attributable History (#1313) * refactor(lib): persist only critical and creation commit envelopes Ordinary content commits are applied and then discarded. Genesis, rights, parent, and destroy stay, including unflagged creations. Drop previous-commit validation, stop indexing Loro binaries as KV keys, and stop creating the /commits collection. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com> * refactor(browser): stop treating discarded commit envelopes as resources UI reads author and dates from the resource. History no longer offers Show Commit, the Sync page does not link commit ids, and sequential saves no longer set previousCommit. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com> * docs: describe commits as signed envelopes, not an event log Align public docs, README, changelogs, and test-coverage notes with apply-and-discard for ordinary content commits. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com> * docs(planning): verifiable History must survive clone Proofs have to replicate with the resource. Preferred path is a sibling Loro envelopes container so snapshot and SYNC_PUSH carry the signed log. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com> * style(lib): collapse store.ts Commit import to satisfy oxfmt CI failed on @tomic/lib format-check: the multi-line import of CommitBuilder / commitIdOf / Commit is a single-line import. Co-authored-by: joepmeindertsma <joepmeindertsma@gmail.com> * feat: keep signed envelopes per resource (Tree::Envelopes), attribute History Every applied signed commit leaves its JSON-AD on the resource it changed, keyed pure_id || createdAt || signature, written in the apply transaction. Not a resource, not indexed. Retention per node: latest (default, the envelope that produced the current state) or all (a signed audit log), via --envelope-retention / ATOMIC_ENVELOPE_RETENTION. Every commit's Loro change carries a token (browser drain token; Rust builder and create_did now tag too), so an envelope maps to the History version it produced. envelopes::attribute_history verifies signatures with the apply code, credits each token to one envelope (the genesis carrier only to a genesis envelope), and reports completeness. Read paths: GET /history-attribution (read-gated), WASM ClientDb.historyAttribution, Store.getHistoryAttribution merging both. History shows by <agent> Verified / Unverified / Unattributed. The destroy envelope on the tombstone value (#1370) is now the subject's latest row in this tree; the tombstone is a marker again. Tests: lib envelopes (8), server it history_attribution, browser lib history-attribution (5), e2e history assertions. Planning: decision doc amended (no #1274 gating, retention knob), auditability doc now Building. * style: oxfmt history-attribution test with the lib config * fix(history): bind envelopes to deltas by version range, flat i18n-safe VersionTitle - attribute_history read each envelope's tokens by importing its update into an empty doc; a browser delta has dependencies, sits pending there and lists no changes, so every browser edit came back with no tokens. Tokens are now read from the stored doc over the update's [start, end) range (AtomicLoroDoc::change_messages_in / update_range). Regression test with a real delta. - getLoroHistory strips the drain token from Version.message for display; expose it as Version.token so attributionForVersion can match it. - VersionTitle is split into flat pieces: the i18n extractor turned the element-spanning ternary into one placeholder message that rendered as [i18n-404:…] (the CI failure). Catalogues re-extracted. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
Follows the duplication audit in #1273. This branch includes that planning work plus the first consolidations that pass the contract.
Checklist
Consolidation 1 — commit ingest + subscribe auth
check_read)Db; authorize a WS subscribe.lib/src/sync/engine.rs,lib/src/sync/ingest.rs,lib/src/sync/ws_apply.rs,lib/src/sync/mod.rs,server/src/commit_monitor.rs,server/src/handlers/commit.rstest(sync): pin replica WS commit ingest skipping write rights)scripts/consolidation-measure.py --baselineexits 0: yesengine.rs1084engine.rs918CommitIngestOpts::replica()still skips write-rights and timestamp. WASMapplyCommitis not folded in (signature off — a fourth policy).Remaining path:
ingest_commit_jsonwithCommitIngestOpts::{hub,peer,replica}. Subscribe handlers callauthorize_read.Consolidation 2 — bind-twins (search + server URL)
normalizeServerUrlproducing the same bytes.lib/src/client/search.rs,browser/lib/src/search.ts,browser/data-browser/src/helpers/serverUrl.ts,flutter/lib/atomic/server_url.darttestdata/search-query.json,testdata/server-url.jsonscripts/consolidation-measure.py --kind bind-twins --baselineexits 0: yes (365 → 365, no third copy)file_search_repro.rs(looks up the fixture)https://, Dart returns''. Not "fixed" here.Also
flutter/AGENTS.mdno longer claims Loro is missing (Phase A landed).--kind bind-twinsso a flat line count is not a failed gate.testdata/directory (JS, Rust, Flutter).