Skip to content

fix(sync): fan out the server's stamp with a commit; keep the author's doc, cursor and history consistent - #1377

Merged
joepio merged 2 commits into
developfrom
fix/fanout-server-stamped-ops
Sep 6, 2026
Merged

fix(sync): fan out the server's stamp with a commit; keep the author's doc, cursor and history consistent#1377
joepio merged 2 commits into
developfrom
fix/fanout-server-stamped-ops

Conversation

@joepio

@joepio joepio commented Sep 6, 2026

Copy link
Copy Markdown
Member

What broke

Live collaboration stopped for the author of a document once a collaborator edited it. The server stamps lastCommit on the resource after applying a commit; Resource::set_unsafe is doc-first, so the stamp is a Loro op under the server's own peer. The websocket fan-out forwarded only commit.loro_update and skipped the author's connection, so no live subscriber ever got the stamp. A collaborator who loaded the stored snapshot built the next edit on top of it; the author parked that edit as pending and every later delta parked behind it. recoverFromIncompleteImport then refetched a snapshot and replaced the doc under loro-prosemirror, which threw Unreachable code from its cursor mapping.

This is the "ephemeral cursor" spec that has been red on CI since 2026-09-02, and locally documents › create document, edit, page title, websockets as well.

Server

  • CommitResponse::broadcast_update: everything the stored doc gained during the apply (author ops plus the stamp), exported from the version captured before the import. fanout_delta() is what DbEvent::Changed and commit_monitor::encode_commit_frame send.
  • Commit frames reach the author's connection too. skip_same_source stays for non-commit changes.
  • Regression test: fanout_delta_lets_the_author_apply_a_peer_edit_built_on_the_stored_snapshot.

Client (@tomic/lib)

Delivering the echo exposed three assumptions, each with a unit test in gap-recovery.test.ts:

  1. Echo dedup still imports. applyIncoming dedups by commit id but imports the bytes so the stamp lands.
  2. Remote ops are not unsaved work. absorbImportedOpsIntoSaveCursor folds imported peers into the save cursor and markLoroSavedAt merges rather than replaces. Without this the drain re-committed an empty delta every 130 ms (syncInProgress never cleared; the quick-edit spec).
  3. A pending edit keeps its history token whoever seals it. Loro commits pending ops without a message on import, export, and from loro-prosemirror's plugin init, which a re-render can trigger between two keystrokes. The staged token is armed as setNextCommitMessage on every write and used by sealPendingEdits() before imports and exports, and the drain reuses it. Without this the History page showed "Second Title" for both versions.

Verification

  • cargo test -p atomic_lib --lib: 247 passed. cargo test -p atomic-server: 65 + 43 passed. cargo fmt clean.
  • @tomic/lib vitest: 352 passed. lint, format, typecheck clean.
  • Local e2e against this branch (server + vite dev): 174 passed, 4 failed, all four environmental on this machine (macOS ControlOrMeta+m, the sveltekit template port, and two Cloud Vault specs that talk to a stale control plane on :3030). history page, quick edit text typing ux, documents ×3, rename-regression re-run serially ×5: green.

Notes

  • docs/src/websockets.md echo section updated.
  • cargo clippy -p atomic_lib --all-targets -- -D warnings reports two pre-existing lints on develop (insert_propval_raw never used, an unused Result in expression.rs), untouched here.
  • browser/data-browser typecheck has a pre-existing error in upgradeDocument.ts:221 on develop, untouched here.

CI follow-up

The first run failed only sync::iroh_e2e::e2e_new_resource_after_bulk_resync, which asserted that a new resource does not reach the other peer until a bulk resync. With the fan-out delta the live push now applies for a genesis too, so the test treats the bulk path as the fallback. The whole iroh_e2e module passes locally with --features iroh,db-redb (15 tests).

…or's doc, cursor and history consistent

The server stamps `lastCommit` on a resource after applying a commit. That
stamp is a Loro op under the server's own peer, and anyone who loads the
stored snapshot builds their next edit on top of it. The live fan-out only
forwarded the author's own bytes and never to the author, so the author
parked that edit as pending and every later delta parked behind it: the
document quietly stopped being live for the person who created it (the
"ephemeral cursor" e2e red since 2026-09-02, and the documents websockets
spec locally).

Server: `CommitResponse::broadcast_update` is everything the doc gained
during the apply; `fanout_delta()` feeds both `DbEvent::Changed` and the
websocket `UPDATE`, which now reaches the author's connection too.

Client, three invariants the echo exposed:
- `applyIncoming` still imports an echo it dedups by commit id.
- Ops that arrive from the server are folded into the Loro save cursor at
  import and kept there by the ack; otherwise the drain re-committed an
  empty delta every 130 ms until reload.
- A pending edit keeps its history token whoever seals it: the staged
  commit token is armed as Loro's next commit message on every write and
  used by explicit seals before imports and exports. loro-prosemirror's
  plugin init commits the shared doc from a re-render, which put the title
  edit into the untagged history bucket.
…sync

`e2e_new_resource_after_bulk_resync` asserted that B did not have a freshly
created canvas until a resync, because the live delta was the commit's own
bytes and a peer could not always apply a genesis from those. With
`CommitResponse::fanout_delta` the live push carries everything the apply
added, B has the canvas before anyone asks for a resync, and the bulk sync
correctly imports nothing. The bulk path is now the fallback the test takes
only when live delivery did not happen.
@joepio
joepio merged commit e5dd05a into develop Sep 6, 2026
7 checks passed
@joepio
joepio deleted the fix/fanout-server-stamped-ops branch September 6, 2026 14:04
joepio added a commit that referenced this pull request Sep 6, 2026
… ack; waitForSynced flushes the ClientDb (#1378)

* fix(browser): recognise an own commit's echo by signature, before the ack [full-e2e]

Since #1377 the server echoes a commit to its author. `applyIncoming` deduped
that echo by comparing the wire commit id with the cached `lastCommit`, which
the ack stamps — and under load the echo lands first. It was then treated as
a peer's push: imported (fine) and notified (not fine): the page re-rendered
right as the user opened the next cell's tag picker, and develop's full e2e
lost `tables › create and fill`.

The drain now registers the signature in `appliedCommitSignatures` before the
POST (the genesis too), and `applyIncoming` treats an id carrying a signature
this client signed as its own: import silently, no notify, whatever the order.

* ci: second full-e2e sample [full-e2e]

* e2e: waitForSynced also waits for the ClientDb flush [full-e2e]

pendingDirtyCount === 0 means the server has the edit; the OPFS write that
mirrors it sits on a worker that the next navigation destroys, and a folder
page loaded from OPFS then lacks the child that was just made. That gap is
what made folder @smoke flap on the runner.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant