diff --git a/.changeset/spring-cleaning-browser-host.md b/.changeset/spring-cleaning-browser-host.md new file mode 100644 index 0000000..9c94e10 --- /dev/null +++ b/.changeset/spring-cleaning-browser-host.md @@ -0,0 +1,42 @@ +--- +"@mcp-b/do-runtime": minor +--- + +Ship the Worker half of the browser alarm protocol. `createBrowserAlarmProjector()` +supplies the `AlarmScheduler`'s `projectWake` and an `acknowledge()` for the +`BrowserAlarmCoordinator`'s `deliver()`. Projections leave one at a time, and each draws +its generation only after the previous one was sent. A consumed wake is acknowledged only +after the latest projection is accepted, no delivery or cleanup is active, and the next +wake is absent or later. If the latest projection failed, `acknowledge()` sends it again +first, so a scheduler with nothing new to project cannot leave the wake retrying forever. +`nextGeneration()` must be durable across Worker restarts: the coordinator silently drops +any projection older than the generation it journaled, so an in-memory counter would stall +every wake after a restart. `parseBrowserAlarmProjection()` is now exported. + +Add `connectMessagePortWebSocket()` to `@mcp-b/do-runtime/browser`. It routes one +MessagePort socket through a Workers-style `fetch` such as the Agents SDK's +`routeAgentRequest()`. A socket nothing routes closes with 1011, and a refused upgrade +closes with 1008 and the refusal's text when it is printable ASCII of at most 123 bytes. +Previously every failure closed with a generic 1011 that dropped the Agent's reason. +`serveMessagePortWebSockets()` now takes `(bridge, url) => Promise`, such as +`(bridge, url) => connectMessagePortWebSocket(bridge, url, route)`, instead of a function +resolving a URL to a socket. It reports a connection failure after closing the client, +and a socket that finishes connecting after `stop()` now closes with +"MessagePort transport closed" instead of "host stopped". + +Gate a hibernatable socket that is a host transport, such as a `MessagePortWebSocket` +rehydrated after a Worker restart. The actor used to receive the transport itself, so its +`send()` and `close()` could leave before a preceding storage write was confirmed. It now +receives one stable socket per transport that waits for the output gate like a +`WebSocketPair` half; hibernation hosts still see the transport. + +`OffscreenDocumentAdapter` gains optional `ready()` and `replaceUnready()` hooks. The +coordinator runs readiness in the same single flight as creation, for new and existing +documents, and replaces a document that fails it at most once. + +Fix two `MessagePortWebSocket` hangs. A throwing `onmessage`, `onopen` or `onclose` +handler skipped the `addEventListener` listeners behind it; during the open flush it also +dropped the queued frames and held every later frame in the queue. Handler errors are now +reported the way `EventTarget` reports a throwing listener. A throw while bridging a +connected socket, such as a second `accept()`, left the brokered client connecting; it now +closes with 1011. diff --git a/.changeset/spring-cleaning-maintenance.md b/.changeset/spring-cleaning-maintenance.md new file mode 100644 index 0000000..78e6af1 --- /dev/null +++ b/.changeset/spring-cleaning-maintenance.md @@ -0,0 +1,24 @@ +--- +"@mcp-b/do-runtime": patch +--- + +Validate both `withEnvAndExports()` scopes before installing either. A non-object `exports` +argument previously left the `env` scope installed for the whole realm, so every later `env` +read resolved against it. + +Treat a `Timer.afterDelay` that rejects on abort, as `node:timers/promises` does, as +cancellation. Replacing or deleting a waiting alarm previously raised an unhandled +`AbortError`, which terminates a Node host by default. The `Timer` contract now states that +an aborted wait may stay pending or reject, but must not resolve. + +Close the databases `createActorContainer()` opened when an open-time check refuses, such as +storage written by a newer release. Each refused attempt previously leaked its handles: a +retry opened another connection to the same OPFS file, and on Node the provider's +`exportSnapshot()` refused from then on. A `FacetHost.abort` that throws is now recorded like a +failed facet deletion instead of becoming an unhandled rejection. + +Report `rowsWritten` as 0 for statements without result columns that write nothing, in both +SQLite backends. DDL previously repeated the last write's count, because SQLite does not +reset `sqlite3_changes()` for it. `SqliteWasmActorStorage.copyFrom()` now reads every source +file in the same task as its recovery-sidecar check, so a source that is still running cannot +open a write transaction between the two. diff --git a/.changeset/spring-cleaning-opfs-pool.md b/.changeset/spring-cleaning-opfs-pool.md new file mode 100644 index 0000000..e55c144 --- /dev/null +++ b/.changeset/spring-cleaning-opfs-pool.md @@ -0,0 +1,15 @@ +--- +"@mcp-b/do-runtime": minor +--- + +Add `installSqliteWasmHost()` to `@mcp-b/do-runtime/backends/sqlite-wasm`. It installs an OPFS SAH +pool through sqlite-wasm's `installOpfsSAHPoolVfs()` with the `name`, `directory`, `clearOnInit` +and `initialCapacity` options, and returns the `{ pool, capi }` host the sqlite-wasm provider +takes. Concurrent calls for one pool share a single install. Install pools through it: + +- It waits up to 10 seconds for a terminated worker to release the pool before installing, then + rejects with a `NoModificationAllowedError`. Installing while the previous worker was still + shutting down could delete the pool's directory and every database in it. +- A transaction interrupted by worker termination is now rolled back when its database is + reopened. Previously, reopening could expose the interrupted transaction's writes in place of + committed rows and left the recovery journal behind, so snapshot export stayed refused. diff --git a/.changeset/spring-cleaning-scope-and-vite.md b/.changeset/spring-cleaning-scope-and-vite.md new file mode 100644 index 0000000..ee39312 --- /dev/null +++ b/.changeset/spring-cleaning-scope-and-vite.md @@ -0,0 +1,14 @@ +--- +"@mcp-b/do-runtime": minor +--- + +Export `ACTOR_SCOPE_GLOBALS` from `@mcp-b/do-runtime` and `facetScopeBanner()` from +`@mcp-b/do-runtime/vite`. The banner binds every name `installActorScope()` writes to a +facet bundle's own scope. Banners that bound only timers, `fetch` and `crypto` left +`WebSocketPair` on the root actor's global, so a facet's socket frames waited on the root's +output gate and could leave before the facet's own write committed. + +`doRuntimeAwaitTransform()` now resolves the imports it injects to the package's own files, +ahead of any host alias for them, so hosts no longer alias `@mcp-b/do-runtime/gate` or +`@mcp-b/do-runtime/browser/async-hooks`. Add a `@mcp-b/do-runtime/cloudflare-email` export, +a data-only `EmailMessage` for hosts to alias `cloudflare:email` to. diff --git a/.changeset/spring-cleaning-transformed-await-gate.md b/.changeset/spring-cleaning-transformed-await-gate.md new file mode 100644 index 0000000..661401d --- /dev/null +++ b/.changeset/spring-cleaning-transformed-await-gate.md @@ -0,0 +1,21 @@ +--- +"@mcp-b/do-runtime": patch +--- + +Keep the input lock and the implicit transaction across transformed awaits that settle while +the actor still holds its lock. `doRuntimeAwaitTransform()` previously resumed every await +through a macrotask hop and a fresh input lock queued behind waiting events, including awaits +of storage calls and plain values. Another event could then run between `await +storage.get()` and the following `put()`, so two concurrent read-modify-writes lost an update. +The implicit transaction also committed at the await, so writes survived an abort that workerd +rolls back. A transformed `await ctx.blockConcurrencyWhile()` let queued events run before its +caller resumed. Every consumer that transforms the Agents SDK was exposed on its storage +sequences. + +A transformed await now continues in the same checkpoint when its promise settles while the +actor holds its lock: storage calls, plain values, and resumptions the runtime already admitted. +Settled awaits no longer pay a macrotask each. An await on foreign I/O still re-enters through +a fresh lock. When another actor's continuation owns the checkpoint, the await waits for a later +task without releasing its lock, and its implicit transaction commits at that hand-off. Code that +relied on a storage-only await to let other events in now blocks them, as on workerd. The Node +and browser conformance lanes now also run the suite with the probe compiled by the transform. diff --git a/.github/dependabot.yml b/.github/dependabot.yml index fed14f4..ffd65d3 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -18,10 +18,5 @@ updates: actions: patterns: - "*" - ignore: - # changesets/action v2 requires Changesets CLI v3; we are on CLI v2. - # Lift this when the CLI is upgraded. - - dependency-name: changesets/action - update-types: ["version-update:semver-major"] commit-message: prefix: "ci(deps)" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8cb5c0b..be579ec 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -69,8 +69,15 @@ jobs: - name: Test browser conformance run: pnpm test:conformance-browser - - name: Build examples - run: pnpm examples:build + - name: Test transformed Node conformance + run: pnpm test:conformance-node-transformed + + - name: Test transformed browser conformance + run: pnpm test:conformance-browser-transformed + + # The extension e2e builds the extension itself; vibe's e2e uses the dev server. + - name: Build vibe-platform for production + run: pnpm --filter do-runtime-example-vibe-platform build - name: Test examples run: pnpm test:examples diff --git a/.gitignore b/.gitignore index d9505f0..353a57a 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ node_modules/ dist/ +.sdk-pack/ .tsbuild/ .wrangler/ __screenshots__/ diff --git a/README.md b/README.md index b43dfb5..4185a69 100644 --- a/README.md +++ b/README.md @@ -335,7 +335,7 @@ measured failure in the browser test lane: `globalThis.sqlite3ApiConfig = { disable: { vfs: { opfs: true, "opfs-wl": true } } }` before initializing sqlite. The host uses the SAH pool; the other OPFS VFSes spawn workers and arm watchdogs of their own. -3. Run `sqlite3InitModule()` and `installOpfsSAHPoolVfs(...)` before +3. Run `sqlite3InitModule()` and `installSqliteWasmHost(sqlite3, ...)` before `installActorScope`. The pool installer uses global timers during startup. 4. Install the actor scope with a resolver that throws when its container is gone. A torn-down worker must refuse new work instead of falling through to @@ -375,7 +375,9 @@ The runtime's own `_cf_` tables are versioned per database file in brings older files forward before any event enters and refuses storage written by a newer package version. Application SQL cannot access that runtime-owned stamp. -The browser provider takes an already-installed OPFS SAH pool (`installOpfsSAHPoolVfs`; sync access handles in a dedicated worker — no cross-origin isolation or `SharedArrayBuffer` needed). One pool per worker; the root and each local facet get separate prefixes inside it. `SqliteWasmActorStorage` adds the close, physical delete, and clone operations a local placement host needs around one prefix. The Node provider uses in-memory databases by default and a directory when asked. +The browser provider takes an already-installed OPFS SAH pool (`installSqliteWasmHost`; sync access handles in a dedicated worker — no cross-origin isolation or `SharedArrayBuffer` needed). One pool per worker; the root and each local facet get separate prefixes inside it. `SqliteWasmActorStorage` adds the close, physical delete, and clone operations a local placement host needs around one prefix. The Node provider uses in-memory databases by default and a directory when asked. + +Install every pool with `installSqliteWasmHost(sqlite3, options)` from `@mcp-b/do-runtime/backends/sqlite-wasm`, not with the driver's `installOpfsSAHPoolVfs`. It takes the same `name`, `directory`, `clearOnInit` and `initialCapacity` options and returns the `{ pool, capi }` host the provider expects. It also fixes two driver behaviours that lose data when the browser terminates a worker. A failed install deletes the pool's directory, so the helper first waits up to 10 seconds for the previous owner to release every file. The driver also never rolls back a transaction a terminated worker left open, so its uncommitted pages can overwrite committed rows; the helper makes SQLite roll the transaction back when the database is reopened. That fix applies to every SAH pool in the same sqlite3 instance and assumes one connection per database file, which is how this backend opens them. Both concrete providers also implement `SqlDatabaseSnapshotProvider`. After the host has stopped the actor, `provider.close()` releases every database handle; `exportSnapshot()` then returns the SQLite images for the whole actor storage scope, and `importSnapshot()` replaces an idle scope. The same snapshot can seed a cold local replica because SQLite images are portable between these providers. Node snapshots require a dedicated directory-backed provider. This is backup/restore and replica seeding, not Cloudflare's time-indexed PITR or continuously updated read replication. @@ -385,9 +387,9 @@ The browser provider attempts to restore the original images when a snapshot imp Construct one `AlarmScheduler` per namespace over a `SqlDatabase` of its own. It owns `_cf_ALARM`, delivery, retry counts (`ALARM_RETRY_MAX_TRIES`), exponential backoff with jitter, and abandonment. Pass `scheduler.hooks(id)` as a root actor's `ports.alarms`, and give the scheduler a `getActor(id)` that places the actor if it is not running — an alarm is a reason to wake a Durable Object, not something that needs one awake already. -A suspending browser host can project the scheduler's next wake through `BrowserAlarmCoordinator` from `@mcp-b/do-runtime/browser/alarm-coordinator`. The coordinator journals the physical hop, rejects stale projections, rearms a consumed watchdog, and reconciles after background-worker restart. The host supplies durable journal storage, the physical alarm calls, and delivery back into its scheduler; logical delivery policy remains in `AlarmScheduler`. +A suspending browser host projects the scheduler's next wake through both halves of `@mcp-b/do-runtime/browser/alarm-coordinator`. In the background worker, `BrowserAlarmCoordinator` journals the physical hop, drops stale projections, rearms a consumed watchdog, and reconciles after restart; the host supplies durable journal storage, the physical alarm calls, and `deliver()`. Beside the scheduler, `createBrowserAlarmProjector()` supplies the scheduler's `projectWake` and the `acknowledge()` that `deliver()` returns across the host's transport; the background passes whatever arrives over that transport through `parseBrowserAlarmProjection()`, which returns `null` for a malformed projection, before `coordinator.project()`. Logical delivery policy remains in `AlarmScheduler`. -The scheduler's `projectWake(when, activeDeliveries)` callback reports the earliest pending wake and the number of active delivery or cleanup attempts. Active attempts keep their original deadline projected through retry persistence and awaited abandonment, even after cancellation removes an entry. This keeps recovery armed when the scheduler's timer fires before the browser's physical alarm. A host can acknowledge a consumed watchdog only after its latest projection is durably acknowledged, activity is zero, and the pending wake is absent or later than the consumed deadline. If bookkeeping fails, the scheduler retries it with bounded backoff without redelivering a completed handler. The retained alarm stays projected as due so physical recovery can also reconstruct the scheduler from its durable row. +The scheduler's `projectWake(when, activeDeliveries)` callback reports the earliest pending wake and the number of active delivery or cleanup attempts. Active attempts keep their original deadline projected through retry persistence and awaited abandonment, even after cancellation removes an entry. This keeps recovery armed when the scheduler's timer fires before the browser's physical alarm. `acknowledge()` releases a consumed watchdog only after its latest projection is durably acknowledged, activity is zero, and the pending wake is absent or later than the consumed deadline; if that projection failed, it is sent again first. The `nextGeneration()` given to `createBrowserAlarmProjector()` must be durable and strictly increasing across Worker restarts: `project()` resolves even when the coordinator silently drops a projection older than the generation it journaled, so an in-memory counter would stall every wake after a restart. Keep it in a host table beside `_cf_ALARM`. If bookkeeping fails, the scheduler retries it with bounded backoff without redelivering a completed handler. The retained alarm stays projected as due so physical recovery can also reconstruct the scheduler from its durable row. ### Facets @@ -395,9 +397,9 @@ The scheduler's `projectWake(when, activeDeliveries)` callback reports the earli ### Actor-scoped I/O, and the one trap -On workerd every awaitable thing is an io-context primitive, so "resuming from an await re-enters with a fresh input lock" never needs saying. Here it does. A raw `setTimeout` resolves a promise the runtime does not own; the continuation resumes with an empty invocation stack and the next `ctx.storage` call throws `no input lock available in this context`. That is by design — the alternative is a continuation that silently writes outside the gate. +On workerd every awaitable thing is an io-context primitive, so "an await resumes under an input lock" never needs saying: a storage await keeps the lock it started with, and outbound I/O re-enters with a fresh one. Here it does. A raw `setTimeout` resolves a promise the runtime does not own; the continuation resumes with an empty invocation stack and the next `ctx.storage` call throws `no input lock available in this context`. That is by design — the alternative is a continuation that silently writes outside the gate. -`container.globals` is the complete gated set, bound to that container: `setTimeout`/`clearTimeout`/`setInterval`/`clearInterval` capture the critical section when armed and re-enter when fired; `scheduler.wait()` and `scheduler.yield()` resume under the actor; `fetch()` waits for output locks and releases the input gate while in flight; `crypto` re-enters on async completion; and `WebSocketPair` creates runtime-owned socket halves. Install it as the worker's globals (`installActorScope`) when one worker hosts one root, or hand it to application code explicitly when it must not. +`container.globals` is the gated platform set, bound to that container: `setTimeout`/`clearTimeout`/`setInterval`/`clearInterval` capture the critical section when armed and re-enter when fired; `scheduler.wait()` and `scheduler.yield()` resume under the actor; `fetch()` waits for output locks and releases the input gate while in flight; `crypto` re-enters on async completion; and `WebSocketPair` creates runtime-owned socket halves. Install it as the worker's globals (`installActorScope`) when one worker hosts one root. When it must not, hand application code `actorScopeBindings(() => container.globals)` instead: it adds the `ReadableStream` and `TransformStream` constructors whose callbacks re-enter the actor, which `container.globals` itself does not carry. ### Hibernatable WebSockets @@ -411,7 +413,10 @@ raw socket reference plus copied tags and attachment bytes, drops the old placement, and supplies that snapshot as `webSockets` on the replacement. The registry is populated before the constructor, so SDKs can lazily rebuild their connection wrappers without another upgrade or connect hook. Closed sockets are -removed before `webSocketClose` runs. +removed before `webSocketClose` runs. A host transport in that snapshot, such as +a fresh `MessagePortWebSocket` after a Worker restart, reaches the actor as one +stable socket whose sends wait for storage confirmation, while host callbacks +keep naming the transport. `HibernationMirror` is the package's in-memory reference implementation. Seed a replacement mirror with the prior socket snapshot and auto-response pair, then @@ -422,12 +427,21 @@ Hosts whose actor lives in another Worker can use `MessagePortWebSocket`, `createMessagePortWebSocketConstructor`, and `serveMessagePortWebSockets` from `@mcp-b/do-runtime/browser/message-port-websocket`; binary frames stay in structured clone and each socket gets one dedicated `MessagePort`. +`connectMessagePortWebSocket(bridge, url, fetch)` from `@mcp-b/do-runtime/browser` +routes one socket through a Workers-style router such as `routeAgentRequest()`. +A missing route closes the client 1011, and a refused upgrade closes it 1008 +with the refusal's text when it is printable ASCII of at most 123 bytes. It does +not tell the peer the socket opened: `serveMessagePortWebSockets(port, (bridge, url) => …)` +sends that to clients from `createMessagePortWebSocketConstructor`, and a host +with its own port protocol opens its end itself. `container.quiescence()` reports armed timers, pending `waitUntil` work, input lock state, and output-gate breakage without waiting. `drainWaitUntil()` is for shutdown and intentionally never settles while a live interval remains armed. -Actor bundles can also install `doRuntimeAwaitTransform()` from `@mcp-b/do-runtime/vite`. A production build checks the final module graph and fails with transformed/total counts for any included module with an uncovered await; the development transform warns once per module if a transformed await reaches its fail-open path without an actor lock. +Actor bundles can also install `doRuntimeAwaitTransform()` from `@mcp-b/do-runtime/vite`. Transformed awaits follow workerd's rule, with the exceptions in [gating coverage](docs/gating-coverage.md#transform). An await that settles while the actor still holds its input lock continues in the same checkpoint and keeps the lock and the implicit transaction: storage calls, plain values, and resumptions the runtime already admitted. An await on foreign I/O re-enters through a fresh input lock. A production build checks the final module graph and fails with transformed/total counts for any included module with an uncovered await; the development transform warns once per module if a transformed await reaches its fail-open path without an actor lock. + +The plugin resolves the imports it injects (`@mcp-b/do-runtime/gate`, `@mcp-b/do-runtime/browser/async-hooks`) to this package's own files, ahead of any host alias for them. `asyncContext: true` lowers async functions for the browser `AsyncLocalStorage`. Any dependency importing `cloudflare:workers`, `cloudflare:email` (a data-only `EmailMessage`) or `node:async_hooks`, including one Vite pre-bundles, still needs host aliases to `@mcp-b/do-runtime/cloudflare-workers`, `/cloudflare-email` and `/browser/async-hooks`. A host that loads facet bundles by URL into its root's realm prefixes each with `facetScopeBanner({ registry })`, which binds every name `installActorScope` writes (`ACTOR_SCOPE_GLOBALS`) to `globalThis[registry][scope]`, where `scope` is the bundle URL's `?scope=` parameter. ## What is not supported @@ -442,7 +456,7 @@ The browser cannot reproduce every workerd facility. Unsupported runtime APIs th | Outbound `new WebSocket(url)` inside actor globals | Refused before opening a connection because the native handshake cannot wait for the output gate. `WebSocketPair` and host-owned transports remain supported. | | `cloudflare:workers` tracing | No-op spans expose the current API. Active span identity follows synchronous callbacks only; there is no observer or propagation across awaits. | | Stored value wire bytes | Browser-safe versioned structured-clone encoding rather than V8's private format; public value types align and legacy JSON rows remain readable. | -| SQL row counters | Local `rowsRead`/`rowsWritten`, including `sql.ingest()`, use returned rows and SQLite changes; workerd uses unavailable libsql billing counters. | +| SQL row counters | Local `rowsRead`/`rowsWritten`, including `sql.ingest()`, use returned rows and SQLite's `changes()`, so schema, index, trigger and FK-action writes are not counted; workerd's libsql counters include them. | | Reserved SQL names | `_cf_` detected from tokenized SQL text, which can reject more than workerd's authorizer. `ANALYZE` on a reserved table is refused where workerd allows it. | | Authorizer-only SQL forms | `ATTACH`, `DETACH`, the temp-schema creations (both `CREATE TEMP …` and the `temp.` qualifier), `VACUUM`, and virtual-table modules outside upstream's four (`fts5`, `fts5vocab`, `rtree`, `rtree_i32`) are refused from the leading keyword, with workerd's own messages. These reach the authorizer's own decisions — action codes and its temp-schema rule — rather than `SqlStorageRegulator` callbacks, so porting the regulator did not carry them. The refused and allowed forms are matched through SQLite's identifier quoting, whitespace or none. `EXPLAIN` in front of a refused form still compiles here, where workerd's authorizer refuses it. | | SQL function allowlist | Not enforced. Workerd's authorizer denies any function outside its 138-name `ALLOWED_SQLITE_FUNCTIONS` list; this runtime allows every function the backend compiled, including build-detail readers such as `sqlite_version()` and `sqlite_source_id()`. | @@ -466,7 +480,7 @@ This is `0.x`. The public surface is what [`src/index.ts`](src/index.ts) and the | `src/io/` | Gates, invocation context, actor storage engine, ids, Worker channels | | `src/api/` | Workers-facing APIs: `DurableObjectState`, SQL, WebSocket, Worker Loader, `cloudflare:workers` | | `src/server/` | Actor containers, facet lifecycle, deletion recovery, alarm scheduling | -| `src/browser/` | Physical alarm projection, offscreen-document recovery, and MessagePort-backed WebSockets for browser hosts | +| `src/browser/` | Physical alarm projection, offscreen-document recovery, MessagePort-backed WebSockets, and the opt-in `async_hooks` shim ([browser async context](docs/browser-async-context.md)) for browser hosts | | `src/transport/` | The one `MessagePort` Cap'n Web session adapter | | `backends/` | `node:sqlite` and sqlite-wasm/OPFS `SqlDatabaseProvider`s | | `conformance/` | One suite, three hosts: workerd, Node, browser; plus the probe fixture and benchmarks | @@ -479,14 +493,16 @@ The `util → io → api → server` direction follows workerd's own layering, e ## Tests ```bash -pnpm test:unit # workerd's own unit tests, ported module by module -pnpm test:conformance-workerd # the oracle: the suite on real workerd, importing nothing from src/ -pnpm test:conformance-node # the suite on this runtime over node:sqlite -pnpm test:conformance-browser # the suite in headless Chromium over sqlite-wasm + OPFS, with a real Cap'n Web session -pnpm test # all of the above +pnpm test:unit # workerd's own unit tests, ported module by module +pnpm test:conformance-workerd # the oracle: the suite on real workerd, importing nothing from src/ +pnpm test:conformance-node # the suite on this runtime over node:sqlite +pnpm test:conformance-browser # the suite in headless Chromium over sqlite-wasm + OPFS, with a real Cap'n Web session +pnpm test:conformance-node-transformed # the Node suite with the probe compiled by the await transform, as a consumer builds actors +pnpm test:conformance-browser-transformed # the browser suite, compiled the same way +pnpm test # all of the above ``` -The workerd lane is what makes the others mean something: every row it passes is a contract the Node and browser lanes must also pass, including cross-root RPC gate release and resumption. The browser smoke lane also fills the real OPFS SAH pool to capacity and proves visible failure, no leaked slot, and recovery. A substrate that lacks a feature asserts the named refusal instead of skipping the row. `pnpm bench:node` and `pnpm bench:browser` measure `sql.exec` latency over a realistic message store on each substrate. +The workerd lane is what makes the others mean something: every row it passes is a contract the Node and browser lanes must also pass, including cross-root RPC gate release and resumption. The browser smoke lane also fills the real OPFS SAH pool to capacity and proves visible failure, no leaked slot, and recovery. Three of its smoke specs terminate a real worker mid-operation: `actor-crash` (the hot journal is rolled back and acknowledged writes survive), `alarm-recovery` (an interrupted alarm is redelivered exactly once), and `hibernation-worker-restart` (hibernated sockets survive through transferred `MessagePort`s). A substrate that lacks a feature asserts the named refusal instead of skipping the row. `pnpm bench:node` and `pnpm bench:browser` measure `sql.exec` latency over a realistic message store on each substrate. ## Development @@ -506,7 +522,7 @@ and Rook's six-package Agents SDK fork in built `agents` package through a `file:` dependency, which resolves their own peer dependencies without installing a second SDK implementation. -`pnpm sdk:pack` builds and packs the six SDK packages into `dist/sdk/` with +`pnpm sdk:pack` builds and packs the six SDK packages into `.sdk-pack/` with their upstream names. After the SDK and consumer gates pass, attach these tarballs to a GitHub release tagged `rook-sdk-` at the tested commit. Rook pins those release asset URLs and their lockfile integrity; diff --git a/backends/node-sqlite.test.ts b/backends/node-sqlite.test.ts index ec15d17..e62932e 100644 --- a/backends/node-sqlite.test.ts +++ b/backends/node-sqlite.test.ts @@ -57,6 +57,11 @@ describe("node-sqlite backend", () => { expect(db.exec("DELETE FROM things WHERE id = ?", [99]).rowsWritten).toBe(0); expect(db.exec("DELETE FROM things WHERE id = ?", [1]).rowsWritten).toBe(1); expect(db.exec("DELETE FROM things", []).rowsWritten).toBe(2); + expect(db.exec("CREATE TABLE other (id INTEGER)", []).rowsWritten).toBe(0); + // FTS5 writes its shadow tables too; total_changes() counts those, changeCount does not. + db.exec("CREATE VIRTUAL TABLE docs USING fts5(body)", []); + expect(db.exec("INSERT INTO docs VALUES ('one')", []).rowsWritten).toBe(1); + expect(db.exec("INSERT INTO docs VALUES ('two') RETURNING rowid", []).rowsWritten).toBe(1); }); test("the backend accepts only SQLite's four binding value kinds", async () => { diff --git a/backends/node-sqlite.ts b/backends/node-sqlite.ts index 700e50a..8c2b5e3 100644 --- a/backends/node-sqlite.ts +++ b/backends/node-sqlite.ts @@ -1,8 +1,7 @@ /** * ← workerd `NO upstream correspondence (storage-backend adaptation)` * - * `SqlDatabaseProvider` over `node:sqlite`. Promoted out of - * `host/fixtures/storage-node.ts`, which is already this adapter. + * `SqlDatabaseProvider` over `node:sqlite`. * * Upstream's equivalent is `SqliteDatabase`'s binding to the SQLite C API plus * its kj-filesystem VFS — 3,768 lines this package deliberately does not port, @@ -11,8 +10,7 @@ * write, and the four operations they need from a database. * * This is the substrate the unit lane runs on. It is also decision 11's Node - * conformance lane, and `fixtures/storage-node.ts` already proves the seam - * across 20 of the extension's 24 Node-lane test files. + * conformance lane. */ import { randomUUID } from "node:crypto"; @@ -35,6 +33,9 @@ import { } from "../src/util/sqlite"; import { requireImportableRuntimeStorage } from "../src/util/sqlite-migrations"; +const TOTAL_CHANGES = "SELECT total_changes() AS value"; +const CHANGES = "SELECT changes() AS value"; + export type NodeSqlProviderOptions = { /** * Dedicated directory for one actor's database files. Omit for in-memory @@ -127,19 +128,28 @@ export function createNodeSqlProvider( export class NodeSqlDatabase implements SqlDatabase { readonly #path: string; #database: DatabaseSync; + /** Prepared once per connection, because every statement reads them. */ + #totalChangesQuery: StatementSync; + #changesQuery: StatementSync; #closed = false; constructor(path: string, private readonly onClose: () => void = () => {}) { this.#path = path; this.#database = new DatabaseSync(path); + this.#totalChangesQuery = this.#database.prepare(TOTAL_CHANGES); + this.#changesQuery = this.#database.prepare(CHANGES); } prepare(sql: string): SqlDatabaseStatement { const statement = this.#database.prepare(sql); const source = statement.sourceSQL; - return new NodeSqlStatement(statement, source, parameterLayout(source), () => - this.#totalChanges(), + return new NodeSqlStatement( + statement, + source, + parameterLayout(source), + () => readCount(this.#totalChangesQuery, "total_changes()"), + () => readCount(this.#changesQuery, "changes()"), ); } @@ -174,6 +184,8 @@ export class NodeSqlDatabase implements SqlDatabase { } } this.#database = new DatabaseSync(this.#path); + this.#totalChangesQuery = this.#database.prepare(TOTAL_CHANGES); + this.#changesQuery = this.#database.prepare(CHANGES); } close(): void { @@ -191,15 +203,14 @@ export class NodeSqlDatabase implements SqlDatabase { } return value; } +} - #totalChanges(): number { - const row = this.#database.prepare("SELECT total_changes() AS value").get(); - const value = row?.value; - if (typeof value !== "number" && typeof value !== "bigint") { - throw new Error("total_changes() did not return a number."); - } - return Number(value); +function readCount(query: StatementSync, name: string): number { + const value = query.get()?.value; + if (typeof value !== "number" && typeof value !== "bigint") { + throw new Error(`${name} did not return a number.`); } + return Number(value); } type ParameterLayout = { @@ -216,6 +227,7 @@ class NodeSqlStatement implements SqlDatabaseStatement { readonly sql: string, layout: ParameterLayout, private readonly totalChanges: () => number, + private readonly changes: () => number, ) { this.#statement = statement; this.#layout = layout; @@ -236,9 +248,13 @@ class NodeSqlStatement implements SqlDatabaseStatement { const columns = statement.columns(); if (columns.length === 0) { + // `changes` survives a statement that writes nothing (DDL), so it counts only if + // total_changes() moved. It excludes trigger and FTS5/R-Tree shadow-table writes. + const before = this.totalChanges(); const { changes } = named === undefined ? statement.run(...anonymous) : statement.run(named, ...anonymous); - return { columnNames: [], rawRows: [], rowsWritten: Number(changes) }; + const wrote = this.totalChanges() !== before; + return { columnNames: [], rawRows: [], rowsWritten: wrote ? Number(changes) : 0 }; } statement.setReadBigInts(true); @@ -246,13 +262,13 @@ class NodeSqlStatement implements SqlDatabaseStatement { const changesBefore = this.totalChanges(); const rows: unknown[] = named === undefined ? statement.all(...anonymous) : statement.all(named, ...anonymous); + // `node:sqlite` exposes no sqlite3_stmt_readonly(). A moved total_changes() tells DML + // RETURNING from SELECT without parsing SQL; changes() is then this statement's own count. + const wrote = this.totalChanges() !== changesBefore; return { columnNames: columns.map((column) => column.name), rawRows: rows.map(asRow), - // `node:sqlite` exposes no sqlite3_stmt_readonly() or per-statement write - // counter. The total-change delta distinguishes SELECT from DML RETURNING - // without parsing SQL or executing the statement twice. - rowsWritten: this.totalChanges() - changesBefore, + rowsWritten: wrote ? this.changes() : 0, }; } diff --git a/backends/sqlite-wasm.test.ts b/backends/sqlite-wasm.test.ts index 675e2d1..9c66a38 100644 --- a/backends/sqlite-wasm.test.ts +++ b/backends/sqlite-wasm.test.ts @@ -1,7 +1,9 @@ import { Buffer } from "node:buffer"; -import { expect, test } from "vitest"; +import sqlite3InitModule from "@sqlite.org/sqlite-wasm"; +import { expect, test, vi } from "vitest"; import { createSqliteWasmProvider, + installSqliteWasmHost, SqliteWasmActorStorage, SqliteWasmDatabase, SqliteWasmRestoreError, @@ -24,6 +26,23 @@ class FakeDatabase implements SqliteWasmDatabaseHandle { close(): void {} } +test("a column-less statement reports its own changeCount: 0 after DDL, 1 for an FTS5 row", async () => { + // The real engine over Emscripten's in-memory filesystem; the pool is not involved. + const sqlite3 = await sqlite3InitModule(); + const pool = { ...memoryHost(new Map()).pool, OpfsSAHPoolDb: sqlite3.oo1.DB }; + const database = new SqliteWasmDatabase({ pool, capi: sqlite3.capi }, "/actor.root.sqlite"); + database.exec("CREATE TABLE things (id INTEGER PRIMARY KEY)", []); + + expect(database.exec("INSERT INTO things VALUES (1), (2), (3)", []).rowsWritten).toBe(3); + // sqlite3_changes() is not reset by DDL, so it would still say 3 here. + expect(database.exec("CREATE TABLE other (id INTEGER)", []).rowsWritten).toBe(0); + // total_changes() also counts FTS5's shadow-table writes. + database.exec("CREATE VIRTUAL TABLE docs USING fts5(body)", []); + expect(database.exec("INSERT INTO docs VALUES ('one')", []).rowsWritten).toBe(1); + expect(database.exec("INSERT INTO docs VALUES ('two') RETURNING rowid", []).rowsWritten).toBe(1); + database.close(); +}); + test("reset fails closed when the SAH pool does not remove the database", () => { const opened: string[] = []; const host = { @@ -256,6 +275,77 @@ test("snapshot import copies Buffer inputs before exporting existing storage", a expect(files.get("/destination.root.sqlite")).toEqual(expected); }); +test("a pool is installed only after its previous owner has released every file", async () => { + // A failed install deletes the pool directory, so the driver must not see a held file. + let held = 2; + const file = { + kind: "file", + createSyncAccessHandle: async () => { + if (held === 0) return { close: () => {} }; + held -= 1; + throw new DOMException("held by a terminated worker", "NoModificationAllowedError"); + }, + }; + const root = opfsDirectory({ ".pool": opfsDirectory({ ".opaque": opfsDirectory({ slot: file }) }) }); + vi.stubGlobal("navigator", { storage: { getDirectory: async () => root } }); + const heldAtInstall: number[] = []; + try { + const installing = installSqliteWasmHost( + { + capi: { ...memoryHost(new Map()).capi, sqlite3_vfs_find: () => 0 }, + wasm: {}, + installOpfsSAHPoolVfs: async () => { + heldAtInstall.push(held); + throw new Error("installed"); + }, + }, + { name: "pool" }, + ); + await expect(installing).rejects.toThrow("installed"); + } finally { + vi.unstubAllGlobals(); + } + expect(heldAtInstall).toEqual([0]); +}); + +test("concurrent installs of one pool share a single install", async () => { + // Otherwise the second call's wait would see the first call's handles as held. + vi.stubGlobal("navigator", { storage: { getDirectory: async () => opfsDirectory({}) } }); + let installs = 0; + const sqlite3 = { + capi: { ...memoryHost(new Map()).capi, sqlite3_vfs_find: () => 0 }, + wasm: {}, + installOpfsSAHPoolVfs: async () => { + installs += 1; + return memoryHost(new Map()).pool; + }, + }; + try { + const [first, second] = await Promise.all([ + installSqliteWasmHost(sqlite3, { name: "pool" }), + installSqliteWasmHost(sqlite3, { name: "pool" }), + ]); + expect(second).toBe(first); + } finally { + vi.unstubAllGlobals(); + } + expect(installs).toBe(1); +}); + +/** An OPFS directory handle over `entries`, as much of one as the pool wait reads. */ +function opfsDirectory(entries: Record) { + return { + getDirectoryHandle: async (name: string) => { + const entry = entries[name]; + if (entry === undefined) throw new DOMException(name, "NotFoundError"); + return entry; + }, + values: async function* () { + yield* Object.values(entries); + }, + }; +} + function memoryHost(files: Map): SqliteWasmHost { return { capi: { diff --git a/backends/sqlite-wasm.ts b/backends/sqlite-wasm.ts index 26836aa..fe77b73 100644 --- a/backends/sqlite-wasm.ts +++ b/backends/sqlite-wasm.ts @@ -3,10 +3,11 @@ * * `SqlDatabaseProvider` over the browser's OPFS SAH pool. * - * The pool is a parameter, not something this module goes and gets. Installing - * the VFS is the host's job — `installOpfsSAHPoolVfs` decides the OPFS - * directory, the pool capacity and whether to clear on init, all of which are - * layout questions this package deliberately knows nothing about. What arrives + * The pool is a parameter, not something this module goes and gets. The host + * decides the OPFS directory, the pool capacity and whether to clear on init, + * all of which are layout questions this package deliberately knows nothing + * about; it installs the pool through `installSqliteWasmHost`, which corrects + * two driver behaviours a terminated worker turns into data loss. What arrives * here is the already-installed pool, and with it the two things a backend * needs that a bare `sqlite3` module cannot give: a database constructor bound * to that VFS, plus the pool's file export/import/unlink operations used by @@ -18,10 +19,10 @@ * is the subset of `SAHPoolUtil` and `oo1.DB` that is used, copied from the * driver's own `.d.mts`. * - * NOT exercised by the unit lane: it needs OPFS, which means a browser. It is - * exercised twice in the browser lane — by `sqlite-wasm.smoke.spec.ts`, which - * drives this file directly, and by the conformance suite, which runs the whole - * package over it. + * The unit lane runs this file over fake pools and the real engine; only the + * OPFS pool itself needs a browser. The pool is exercised twice in the browser + * lane — by `sqlite-wasm.smoke.spec.ts`, which drives this file directly, and + * by the conformance suite, which runs the whole package over it. */ import { @@ -89,8 +90,8 @@ export interface SqliteWasmCapi { /** * What the host hands over: the pool it installed, and the C-API namespace it - * already holds. Both come off the same `sqlite3` object the caller used to - * call `installOpfsSAHPoolVfs`, so this asks for nothing it does not have. + * already holds. Both come off the same `sqlite3` object the caller passed to + * `installSqliteWasmHost`, which returns exactly this. */ export interface SqliteWasmHost { readonly pool: OpfsSahPool; @@ -102,6 +103,189 @@ export type SqliteWasmProviderOptions = { prefix: string; }; +/** + * The four `installOpfsSAHPoolVfs` options this module supports: `name`, + * `directory`, `clearOnInit` and `initialCapacity`. + */ +export type SqliteWasmPoolOptions = { + /** The VFS name. The driver's default is "opfs-sahpool". */ + readonly name?: string; + /** The pool's OPFS directory. The driver's default is `.${name}`. */ + readonly directory?: string; + readonly clearOnInit?: boolean; + readonly initialCapacity?: number; +}; + +/** ← `Sqlite3Static`, the members `installSqliteWasmHost` reads. */ +export interface SqliteWasmModule { + readonly capi: SqliteWasmCapi & { sqlite3_vfs_find(name: string): number }; + /** Read by the lock patch; see `rollBackHotJournals`. */ + readonly wasm: object; + installOpfsSAHPoolVfs(options: SqliteWasmPoolOptions): Promise; +} + +/** + * Installs an OPFS SAH pool for this backend through the driver's + * `installOpfsSAHPoolVfs`, returning the host `createSqliteWasmProvider` takes. + * Install every pool through it; both of its additions are load-bearing for a + * worker the browser can terminate: + * + * - It waits, up to 10 s, until no file in the pool is still held by a previous + * owner, then rejects with a `NoModificationAllowedError` DOMException. A + * failed install runs the driver's `removeVfs()`, which deletes the pool + * directory, and a worker terminated mid-slice keeps its handles for about + * 2 s: retrying the install across that release lost the whole pool in 3 of 24 + * measured recoveries. + * - It makes SQLite roll back the hot journal a terminated worker left. The + * driver reports a reserved lock on every file, so the journal was never + * replayed: 284 of 2304 acknowledged rows reopened overwritten by the + * transaction the worker died in. + * + * Concurrent calls for one pool share one install, and a later call for a pool + * this worker has installed returns it at once. + */ +export function installSqliteWasmHost( + sqlite3: SqliteWasmModule, + options: SqliteWasmPoolOptions = {}, +): Promise { + const name = options.name || "opfs-sahpool"; + const flights = installing.get(sqlite3) ?? new Map>(); + installing.set(sqlite3, flights); + // One `sqlite3` instance's install of one pool name always yields that instance's pool type. + const current = flights.get(name) as Promise | undefined; + if (current !== undefined) return current; + const flight = install(sqlite3, name, options); + flights.set(name, flight); + const land = (): void => { + flights.delete(name); + }; + void flight.then(land, land); + return flight; +} + +/** Installs in flight, per `sqlite3` instance and pool name. */ +const installing = new WeakMap>>(); + +async function install( + sqlite3: SqliteWasmModule, + name: string, + options: SqliteWasmPoolOptions, +): Promise { + const installed = sqlite3.capi.sqlite3_vfs_find(name) !== 0; + if (!installed) await released(options.directory || `.${name}`); + const pool = await sqlite3.installOpfsSAHPoolVfs(options); + if (!installed) rollBackHotJournals(sqlite3, name); + return { pool, capi: sqlite3.capi }; +} + +/** How long a previous owner may hold the pool; a busy terminated worker measured about 2 s. */ +const POOL_RELEASE_TIMEOUT_MS = 10_000; + +/** Captured at import, because a host installs its actor scope over `setTimeout` afterwards. */ +const platformSetTimeout = globalThis.setTimeout.bind(globalThis); + +/** + * Resolves once no file in the pool directory is held by another context, or at + * once when the pool does not exist yet. Opening and closing a sync access + * handle changes nothing, and a terminated worker never takes one back. + */ +async function released(directory: string): Promise { + let files = await navigator.storage.getDirectory(); + try { + // The driver keeps a pool's files in `.opaque` under its directory. + for (const part of [...directory.split("/"), ".opaque"]) { + if (part !== "") files = await files.getDirectoryHandle(part); + } + } catch (error) { + if (error instanceof DOMException && error.name === "NotFoundError") return; + throw error; + } + for (const deadline = Date.now() + POOL_RELEASE_TIMEOUT_MS; ; ) { + try { + for await (const file of files.values()) { + if (file.kind === "file") (await file.createSyncAccessHandle()).close(); + } + return; + } catch (error) { + if (!(error instanceof DOMException && error.name === "NoModificationAllowedError")) { + throw error; + } + if (Date.now() > deadline) { + throw new DOMException( + `OPFS SAH pool ${directory} is still held by another context.`, + "NoModificationAllowedError", + ); + } + await new Promise((resolve) => platformSetTimeout(resolve, 20)); + } + } +} + +/** + * The struct-binder and heap members the lock patch uses. The driver's + * declarations omit the `$`-prefixed pointer members, so they are read here + * rather than required of the host's `sqlite3` type. + */ +type WasmStruct = { installMethod(name: string, func: (...args: number[]) => number): unknown }; +type SqliteWasmInternals = { + readonly capi: { + readonly sqlite3_vfs: new (pointer: number) => WasmStruct & { $xOpen: number }; + readonly sqlite3_io_methods: new (pointer: number) => WasmStruct; + sqlite3_vfs_find(name: string): number; + }; + readonly wasm: { + functionEntry(pointer: number): ((...args: number[]) => number) | null | undefined; + peekPtr(pointer: number): number; + poke32(pointer: number, value: number): unknown; + }; +}; + +/** + * Makes `xCheckReservedLock` report no reserved lock for every SAH pool in this + * sqlite3 instance. + * + * The driver's `opfs-sahpool` reports one for every file, which tells SQLite + * another connection is mid-write, so `hasHotJournal()` never replays a journal + * a terminated worker left. The driver's other OPFS VFSes report none. + * + * The driver keeps one io-methods struct (`opfsIoMethods`) for every file of + * every SAH pool in this sqlite3 instance, reachable only through an open file's + * `pMethods`. So this pool's `xOpen` is wrapped until its first successful open, + * which patches that shared struct before the connection's first read (where + * SQLite checks for a hot journal) and restores `xOpen`. + */ +function rollBackHotJournals( + sqlite3: { readonly capi: object; readonly wasm: object }, + vfsName: string, +): void { + const { capi, wasm } = sqlite3 as unknown as SqliteWasmInternals; + const pointer = capi.sqlite3_vfs_find(vfsName); + // A paused pool the driver returned from its cache kept the hook it was installed with. + if (pointer === 0) return; + const vfs = new capi.sqlite3_vfs(pointer); + const xOpen = vfs.$xOpen; + const open = wasm.functionEntry(xOpen); + if (!open) throw new Error(`The SAH pool VFS ${vfsName} has no xOpen.`); + vfs.installMethod("xOpen", (pVfs, zName, pFile, flags, pOutFlags) => { + const result = open(pVfs, zName, pFile, flags, pOutFlags); + if (result !== 0) return result; + // ponytail: every SAH pool in this sqlite3 instance now answers "none", which is exact + // while each database file has one connection, do-runtime's contract. The driver tracks + // each file's `lockType` only in private state; answer from it once upstream exposes it or + // fixes `xCheckReservedLock`, or wrap `xLock`/`xUnlock` to track locks per path if a host + // ever opens one file twice. + new capi.sqlite3_io_methods(wasm.peekPtr(pFile)).installMethod( + "xCheckReservedLock", + (_file, pOut) => { + wasm.poke32(pOut, 0); + return 0; + }, + ); + vfs.$xOpen = xOpen; + return result; + }); +} + /** * One actor's named databases and their file lifecycle inside an OPFS SAH pool. * @@ -158,21 +342,16 @@ export class SqliteWasmActorStorage implements SqlDatabaseProvider { if (sidecar !== undefined) { throw new Error(`Cannot clone actor storage with a SQLite recovery sidecar: ${sidecar}`); } - const images: Array<{ name: string; bytes: Uint8Array }> = []; - for (const file of files) { - const name = file.slice(source.#prefix.length + 1, -".sqlite".length); - requireSafeDatabaseName(name); - images.push({ - name, - bytes: new Uint8Array(await source.#host.pool.exportFile(file)), - }); - } - this.close(); - await replaceDatabases( - this.#host.pool, - this.#prefix, - images.map(({ name, bytes }) => ({ name, image: bytes })), + // Every export starts in the check's task, before a running source can open a journal. + const images = await Promise.all( + files.map(async (file) => { + const name = file.slice(source.#prefix.length + 1, -".sqlite".length); + requireSafeDatabaseName(name); + return { name, image: new Uint8Array(await source.#host.pool.exportFile(file)) }; + }), ); + this.close(); + await replaceDatabases(this.#host.pool, this.#prefix, images); } #ownedFiles(): string[] { @@ -432,8 +611,12 @@ class WasmSqlStatement implements SqlDatabaseStatement { const columnCount = this.#statement.columnCount; if (columnCount === 0) { + // `changes(false)` survives a statement that writes nothing (DDL), so it counts only if + // total_changes() moved. It excludes trigger and FTS5/R-Tree shadow-table writes. + const before = this.#database.changes(true); this.#statement.step(); - return { columnNames: [], rawRows: [], rowsWritten: this.#database.changes(false) }; + const wrote = this.#database.changes(true) !== before; + return { columnNames: [], rawRows: [], rowsWritten: wrote ? this.#database.changes(false) : 0 }; } const changesBefore = this.#database.changes(true); @@ -446,12 +629,13 @@ class WasmSqlStatement implements SqlDatabaseStatement { } rawRows.push(row); } - // A SELECT leaves total_changes() untouched; DML RETURNING advances it. - // The delta avoids a SQL classifier and matches the public cursor contract. + // A SELECT leaves total_changes() untouched; DML RETURNING advances it, which avoids a + // SQL classifier. `changes(false)` is then this statement's own count, as above. + const wrote = this.#database.changes(true) !== changesBefore; return { columnNames, rawRows, - rowsWritten: this.#database.changes(true) - changesBefore, + rowsWritten: wrote ? this.#database.changes(false) : 0, }; } diff --git a/conformance/bench/await-transform.bench.ts b/conformance/bench/await-transform.bench.ts index ad6ca33..38b0868 100644 --- a/conformance/bench/await-transform.bench.ts +++ b/conformance/bench/await-transform.bench.ts @@ -1,5 +1,5 @@ import { expect, test } from "vitest"; -import { __gate } from "../../src/gate"; +import { __gateAwait, __resumeAwait } from "../../src/gate"; import { InputGate, OutputGate } from "../../src/io/io-gate"; import { IoContext, type Actor, type Timer } from "../../src/io/io-context"; @@ -38,7 +38,8 @@ async function measure(kind: "plain" | "awaitIo" | "transformed"): Promise) => reports.push(event.data)); + worker.addEventListener("error", (event) => reports.push({ kind: "error", error: event.message })); + worker.postMessage(boot); + return { worker, reports }; +} + +/** The first report of a kind, polled until a deadline; a worker error fails at once. */ +async function next( + reports: CrashReport[], + kind: K, +): Promise> { + const deadline = Date.now() + 15_000; + for (;;) { + const failure = reports.find((report) => report.kind === "error"); + if (failure?.kind === "error") throw new Error(failure.error); + const found = reports.find((report): report is Extract => report.kind === kind); + if (found !== undefined) return found; + if (Date.now() > deadline) throw new Error(`timed out waiting for ${kind}`); + await new Promise((resolve) => setTimeout(resolve, 10)); + } +} + +test( + "worker termination mid-slice rolls back the open implicit transaction, the first reopen removes the hot journal, and acknowledged writes survive", + async () => { + const poolName = `do-runtime-actor-crash-${Math.random().toString(36).slice(2)}`; + try { + const crashed = start({ poolName, phase: "crash" }); + await next(crashed.reports, "acknowledged"); + await next(crashed.reports, "started"); + crashed.worker.terminate(); + + const reopened = start({ poolName, phase: "reopen" }); + expect(await next(reopened.reports, "reopened")).toEqual({ + kind: "reopened", + beforeReopen: { + files: ["/actor.facets.sqlite", "/actor.root.sqlite", "/actor.root.sqlite-journal"], + hotJournal: true, + exported: + "Cannot export a snapshot with a SQLite recovery sidecar: /actor.root.sqlite-journal", + }, + // One open and close of the actor database, which is the first read after the crash. + afterReopen: { + files: ["/actor.facets.sqlite", "/actor.root.sqlite"], + rows: [["call-1", 2_304]], + exported: "facets,root", + }, + }); + } finally { + for (const worker of workers.splice(0)) worker.terminate(); + } + }, + 30_000, +); diff --git a/conformance/browser/actor-crash.worker.ts b/conformance/browser/actor-crash.worker.ts new file mode 100644 index 0000000..c737ade --- /dev/null +++ b/conformance/browser/actor-crash.worker.ts @@ -0,0 +1,116 @@ +/** + * An actor whose worker is terminated in the middle of a synchronous slice, and + * the fresh worker that inspects its pool afterwards. + * + * The crash slice rewrites every acknowledged row instead of appending rows, + * which is measured rather than stylistic. SQLite can spill a page it appended + * without syncing the journal, so an append-only slice leaves a journal whose + * header was never synced, which SQLite ignores, and nothing to roll back. A + * rewrite journals every page it touches, and once it dirties more pages than + * the cache holds (`cache_spill` is 2026 pages of 8 KiB in this build) SQLite + * syncs the journal header and writes uncommitted pages into the database + * file. Only then does recovery depend on replaying the journal. + */ + +import { createActorContainer, DEFAULT_ALARM_OUTLET, noFacets } from "../../src/index"; +import { createSqliteWasmProvider, SqliteWasmActorStorage } from "../../backends/sqlite-wasm"; +import { installPool, timer, UNIQUE_KEY } from "./substrate"; + +export type CrashBoot = { readonly poolName: string; readonly phase: "crash" | "reopen" }; + +type Files = { readonly files: readonly string[] }; + +export type CrashReport = + | { readonly kind: "acknowledged" } + | { readonly kind: "started" } + | { + readonly kind: "reopened"; + readonly beforeReopen: Files & { readonly hotJournal: boolean; readonly exported: string }; + readonly afterReopen: Files & { + readonly rows: readonly (readonly unknown[])[]; + readonly exported: string; + }; + } + | { readonly kind: "error"; readonly error: string }; + +/** One row per 8 KiB page, and more pages than the cache can hold dirty. */ +const ROWS = 2_304; +const PAD = "x".repeat(7_000); +/** SQLite's rollback-journal magic, which only a synced (hot) journal header carries. */ +const JOURNAL_MAGIC = [0xd9, 0xd5, 0x05, 0xf9, 0x20, 0xa1, 0x63, 0xd7]; + +class Ledger { + constructor(private readonly ctx: DurableObjectState) {} + + acknowledge(): void { + const sql = this.ctx.storage.sql; + sql.exec("CREATE TABLE ledger (id INTEGER PRIMARY KEY, value TEXT NOT NULL)"); + for (let id = 1; id <= ROWS; id++) sql.exec("INSERT INTO ledger VALUES (?, ?)", id, `call-1${PAD}`); + } + + crash(): void { + const sql = this.ctx.storage.sql; + for (let id = 1; id <= ROWS; id++) { + sql.exec("UPDATE ledger SET value = ? WHERE id = ?", `call-2${PAD}`, id); + } + self.postMessage({ kind: "started" } satisfies CrashReport); + // The slice never ends, so its implicit transaction can end only with the worker. + for (;;); + } +} + +self.addEventListener("message", (event: MessageEvent) => { + void run(event.data).catch((error: unknown) => { + self.postMessage({ kind: "error", error: String(error) } satisfies CrashReport); + }); +}); + +async function run({ poolName, phase }: CrashBoot): Promise { + if (phase === "crash") { + const container = await createActorContainer({ + id: "ledger", + uniqueKey: UNIQUE_KEY, + exports: {}, + env: {}, + ports: { + sql: new SqliteWasmActorStorage(await installPool(poolName), "/actor"), + alarms: DEFAULT_ALARM_OUTLET, + facets: noFacets, + timer, + }, + }); + const ledger = container.entry(await container.start((ctx) => new Ledger(ctx))); + // The entry answers only after the output gate releases, so this is a committed write. + await ledger.acknowledge(); + self.postMessage({ kind: "acknowledged" } satisfies CrashReport); + await ledger.crash(); + return; + } + + const host = await installPool(poolName, { clearOnInit: false }); + const files = (): string[] => host.pool.getFileNames().sort(); + const provider = createSqliteWasmProvider(host, { prefix: "/actor" }); + const journal = await host.pool.exportFile("/actor.root.sqlite-journal"); + const beforeReopen = { + files: files(), + hotJournal: JOURNAL_MAGIC.every((byte, index) => journal[index] === byte), + exported: await exported(provider), + }; + const db = await provider.open("root"); + const rows = db.exec( + "SELECT substr(value, 1, 6), count(*) FROM ledger GROUP BY 1 ORDER BY 1", + [], + ).rawRows; + provider.close(); + const afterReopen = { files: files(), rows, exported: await exported(provider) }; + self.postMessage({ kind: "reopened", beforeReopen, afterReopen } satisfies CrashReport); +} + +/** The exported database names, or the refusal. */ +async function exported(provider: ReturnType): Promise { + try { + return (await provider.exportSnapshot()).databases.map(({ name }) => name).join(","); + } catch (error) { + return error instanceof Error ? error.message : String(error); + } +} diff --git a/conformance/browser/actor.worker.ts b/conformance/browser/actor.worker.ts index 0ba4447..f0f009a 100644 --- a/conformance/browser/actor.worker.ts +++ b/conformance/browser/actor.worker.ts @@ -47,6 +47,7 @@ */ import { + ACTOR_SCOPE_GLOBALS, actorScopeBindings, createActorContainer, HibernationMirror, @@ -547,20 +548,6 @@ const facetScopes: Record = {}; (globalThis as Record)[FACET_SCOPE_GLOBAL] = facetScopes; let facetScopeCounter = 0; -/** The actor globals the dynamic facet module binds to its own container. */ -const FACET_SCOPE_NAMES = [ - "scheduler", - "setTimeout", - "clearTimeout", - "setInterval", - "clearInterval", - "fetch", - "crypto", - "WebSocket", - "WebSocketPair", - "WebSocketRequestResponsePair", -] as const; - async function facetModule(className: string, gate: FacetGate): Promise { const hash = className.lastIndexOf("#"); if (hash < 0) throw new Error(`Browser lane cannot resolve facet class ${className}.`); @@ -583,7 +570,7 @@ async function facetModule(className: string, gate: FacetGate): Promise { facets, timer, hibernation, + // "fetched" in two chunks, the second after a timer, as every lane's outbound answers. fetch: async () => { await timer.afterDelay(60); - return new Response("fetched"); + const body = new ReadableStream({ + async start(controller) { + controller.enqueue(new TextEncoder().encode("fet")); + await timer.afterDelay(20); + controller.enqueue(new TextEncoder().encode("ched")); + controller.close(); + }, + }); + return new Response(body); }, }, webSockets: hibernation.snapshot(), diff --git a/conformance/browser/alarm-recovery.smoke.spec.ts b/conformance/browser/alarm-recovery.smoke.spec.ts new file mode 100644 index 0000000..d97d6c7 --- /dev/null +++ b/conformance/browser/alarm-recovery.smoke.spec.ts @@ -0,0 +1,111 @@ +/** + * An alarm whose scheduler worker is terminated mid-delivery, recovered by a + * real restart over the same OPFS pool. + * + * `alarm-scheduler.test.ts` proves the recovery against a fake timer and a + * handler that never returns. This is the same interruption produced the way + * Chrome produces it — `Worker.terminate()` while `deliverAlarm` is pending — + * with the row read back through the scheduler's own connection each time. + */ + +import { expect, test } from "vitest"; +import type { AlarmBoot, AlarmReport } from "./alarm-recovery.worker"; + +type Kind = AlarmReport["kind"]; + +/** Every worker the test started, terminated however it ends. */ +const workers: Worker[] = []; + +function start(boot: AlarmBoot): { worker: Worker; reports: AlarmReport[] } { + const worker = new Worker(new URL("./alarm-recovery.worker.ts", import.meta.url), { + type: "module", + }); + workers.push(worker); + const reports: AlarmReport[] = []; + worker.addEventListener("message", (event: MessageEvent) => reports.push(event.data)); + worker.addEventListener("error", (event) => reports.push({ kind: "error", error: event.message })); + worker.postMessage(boot); + return { worker, reports }; +} + +/** + * The first report that matches, polled until a deadline; a worker error fails at once. Longer + * than the 10 s a replacement's pool install may wait, so a slow release reports as itself. + */ +async function next( + reports: AlarmReport[], + kind: K, + matches: (report: Extract) => boolean = () => true, +): Promise> { + const deadline = Date.now() + 15_000; + for (;;) { + const failure = reports.find((report) => report.kind === "error"); + if (failure?.kind === "error") throw new Error(failure.error); + const found = reports.find( + (report): report is Extract => + report.kind === kind && matches(report as Extract), + ); + if (found !== undefined) return found; + if (Date.now() > deadline) throw new Error(`timed out waiting for ${kind}`); + await new Promise((resolve) => setTimeout(resolve, 10)); + } +} + +test( + "an alarm interrupted by scheduler-worker termination is redelivered exactly once after restart", + async () => { + const poolName = `do-runtime-alarm-recovery-${Math.random().toString(36).slice(2)}`; + try { + const hung = start({ poolName, phase: "hang" }); + const interrupted = await next(hung.reports, "delivering"); + hung.worker.terminate(); + expect(interrupted.retryCount).toBe(0); + + const restarted = start({ poolName, phase: "count" }); + // The constructor's projection: recovery has already turned the running mark into a + // retry that moved the backoff and not the counted retries. + const recovered = await next(restarted.reports, "projected"); + expect(recovered.rows).toEqual([ + { + actor_id: "actor", + scheduled_time: interrupted.scheduledTime, + retry_time: expect.any(Number), + backoff: 1, + counted_retry: 0, + previous_retry_counted: 0, + running: 0, + }, + ]); + // The first rung, two seconds, plus at most a quarter of it in jitter. + const retryTime = recovered.rows[0]?.retry_time as number; + expect(retryTime).toBeGreaterThanOrEqual(recovered.before + 2_000); + expect(retryTime).toBeLessThanOrEqual(recovered.at + 2_500); + + // Uncounted, so the handler is told retryCount 0, exactly as for a first delivery. + expect(await next(restarted.reports, "delivered")).toEqual({ + kind: "delivered", + scheduledTime: interrupted.scheduledTime, + retryCount: 0, + }); + const settled = await next( + restarted.reports, + "projected", + (report) => report.wake === null && report.active === 0, + ); + expect(settled.rows).toEqual([]); + expect(restarted.reports.filter((report) => report.kind === "delivered")).toHaveLength(1); + restarted.worker.terminate(); + + const third = start({ poolName, phase: "count" }); + expect(await next(third.reports, "projected")).toMatchObject({ wake: null, rows: [] }); + // As long as the recovered rung could take: a delivery still owed would have arrived by now. + for (const deadline = Date.now() + 2_500; Date.now() < deadline; ) { + expect(third.reports.filter((report) => report.kind !== "projected")).toEqual([]); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + } finally { + for (const worker of workers.splice(0)) worker.terminate(); + } + }, + 30_000, +); diff --git a/conformance/browser/alarm-recovery.worker.ts b/conformance/browser/alarm-recovery.worker.ts new file mode 100644 index 0000000..bd3664f --- /dev/null +++ b/conformance/browser/alarm-recovery.worker.ts @@ -0,0 +1,66 @@ +/** + * The namespace's `AlarmScheduler` in a worker that is terminated while it is + * delivering an alarm, and the schedulers that replace it over the same pool. + * + * `projectWake` is the scheduler's own seam for a browser watchdog, and every + * projection is reported with the `_cf_ALARM` rows as the scheduler's + * connection reads them at that moment. The first one comes from the + * constructor, after recovery has rewritten any interrupted delivery. + */ + +import { AlarmScheduler } from "../../src/index"; +import { createSqliteWasmProvider } from "../../backends/sqlite-wasm"; +import { installPool, timer } from "./substrate"; + +export type AlarmBoot = { readonly poolName: string; readonly phase: "hang" | "count" }; + +type Delivery = { readonly scheduledTime: number; readonly retryCount: number }; + +export type AlarmReport = + | ({ readonly kind: "delivering" } & Delivery) + | ({ readonly kind: "delivered" } & Delivery) + | { + readonly kind: "projected"; + readonly wake: number | null; + readonly active: number; + readonly rows: readonly Record[]; + /** Read before the scheduler was constructed, and when this projection was reported. */ + readonly before: number; + readonly at: number; + } + | { readonly kind: "error"; readonly error: string }; + +const report = (value: AlarmReport): void => self.postMessage(value); + +self.addEventListener("message", (event: MessageEvent) => { + void boot(event.data).catch((error: unknown) => report({ kind: "error", error: String(error) })); +}); + +async function boot({ poolName, phase }: AlarmBoot): Promise { + const host = await installPool(poolName, { clearOnInit: phase === "hang" }); + const db = await createSqliteWasmProvider(host, { prefix: "/namespace" }).open("alarms"); + const rows = (): Record[] => { + const { columnNames, rawRows } = db.exec("SELECT * FROM _cf_ALARM", []); + return rawRows.map((row) => Object.fromEntries(columnNames.map((name, i) => [name, row[i]]))); + }; + const before = Date.now(); + const scheduler = new AlarmScheduler({ + timer, + db, + getActor: () => ({ + deliverAlarm: (scheduledTime, retryCount) => { + if (phase === "hang") { + report({ kind: "delivering", scheduledTime, retryCount }); + return new Promise(() => {}); + } + report({ kind: "delivered", scheduledTime, retryCount }); + return Promise.resolve({ outcome: "ok", retry: false, retryCountsAgainstLimit: true }); + }, + abandonAlarm: () => Promise.resolve(null), + }), + projectWake: (wake, active) => { + report({ kind: "projected", wake, active, rows: rows(), before, at: Date.now() }); + }, + }); + if (phase === "hang") scheduler.setAlarm("actor", Date.now()); +} diff --git a/conformance/browser/await-publication.smoke.spec.ts b/conformance/browser/await-publication.smoke.spec.ts index 40a63d7..df80ea4 100644 --- a/conformance/browser/await-publication.smoke.spec.ts +++ b/conformance/browser/await-publication.smoke.spec.ts @@ -64,6 +64,8 @@ test("only one actor publishes into the await-to-resume gap", async () => { void secondPublication.then(() => { secondPublished = true; }); + // Settle both outside either actor's checkpoint; a promise settled inside one continues there. + await portHop(); firstSource.resolve(); secondSource.resolve(); await Promise.resolve(); diff --git a/conformance/browser/hibernation-worker-restart.smoke.spec.ts b/conformance/browser/hibernation-worker-restart.smoke.spec.ts new file mode 100644 index 0000000..89df89b --- /dev/null +++ b/conformance/browser/hibernation-worker-restart.smoke.spec.ts @@ -0,0 +1,127 @@ +/** + * A hibernatable socket across a real actor-worker termination. + * + * The suite's eviction rows rebuild the container inside a worker that stays + * alive, so the socket object survives with it. A Chrome-extension host + * discards the worker instead: the port the worker held dies with it, the host + * keeps the client's side of the socket and the mirrored state, and hands a + * fresh port for the same socket to the replacement. This spec is that cycle + * over one OPFS pool, with the page as the host. + */ + +import { expect, test } from "vitest"; +import type { MessagePortWebSocketWireMessage } from "../../src/browser/message-port-websocket"; +import type { SocketBoot, SocketReport } from "./hibernation-worker-restart.worker"; + +type Kind = SocketReport["kind"]; + +/** Every worker the test started, terminated however it ends. */ +const workers: Worker[] = []; + +function start(boot: SocketBoot): { worker: Worker; reports: SocketReport[] } { + const worker = new Worker(new URL("./hibernation-worker-restart.worker.ts", import.meta.url), { + type: "module", + }); + workers.push(worker); + const reports: SocketReport[] = []; + worker.addEventListener("message", (event: MessageEvent) => reports.push(event.data)); + worker.addEventListener("error", (event) => reports.push({ kind: "error", error: event.message })); + worker.postMessage(boot, [boot.port]); + return { worker, reports }; +} + +/** The client's side of one worker generation's port: what it sends, and what reaches it. */ +function client(port: MessagePort): { + send: (frame: MessagePortWebSocketWireMessage) => void; + frames: MessagePortWebSocketWireMessage[]; +} { + const frames: MessagePortWebSocketWireMessage[] = []; + port.onmessage = (event: MessageEvent) => frames.push(event.data); + return { send: (frame) => port.postMessage(frame), frames }; +} + +/** + * Polls until `read` answers, failing fast on a worker error. Longer than the 10 s a + * replacement's pool install may wait, so a slow release reports as itself. + */ +async function until(reports: SocketReport[], read: () => T | undefined, what: string): Promise { + const deadline = Date.now() + 15_000; + for (;;) { + const failure = reports.find((report) => report.kind === "error"); + if (failure?.kind === "error") throw new Error(failure.error); + const value = read(); + if (value !== undefined) return value; + if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`); + await new Promise((resolve) => setTimeout(resolve, 10)); + } +} + +const next = (reports: SocketReport[], kind: K) => + until( + reports, + () => reports.find((report): report is Extract => report.kind === kind), + kind, + ); + +test("hibernated sockets survive actor-worker termination through transferred MessagePorts", async () => { + const poolName = `do-runtime-hibernation-restart-${Math.random().toString(36).slice(2)}`; + const url = "wss://room.invalid/"; + try { + const first = new MessageChannel(); + const before = client(first.port1); + const accepting = start({ poolName, url, port: first.port2 }); + await next(accepting.reports, "accepted"); + before.send({ type: "message", data: "ping" }); + expect(await until(accepting.reports, () => before.frames[0], "pong")).toEqual({ + type: "message", + data: "pong", + }); + accepting.worker.postMessage("hibernate"); + const { kind: _hibernated, ...hibernated } = await next(accepting.reports, "hibernated"); + expect(hibernated).toEqual({ + socket: { + tags: ["room:lobby", "user:ada"], + attachment: expect.any(Uint8Array), + autoResponseTimestamp: expect.any(Number), + }, + autoResponse: { request: "ping", response: "pong" }, + }); + accepting.worker.terminate(); + + const second = new MessageChannel(); + const after = client(second.port1); + const restarted = start({ poolName, url, port: second.port2, hibernated }); + await next(restarted.reports, "ready"); + after.send({ type: "message", data: "hello" }); + expect(await next(restarted.reports, "message")).toEqual({ + kind: "message", + message: "hello", + tags: ["room:lobby", "user:ada"], + attachment: { user: "ada", since: new Date(0), roles: new Set(["admin"]) }, + autoResponseTimestamp: hibernated.socket.autoResponseTimestamp, + stored: "ada", + }); + expect(await until(restarted.reports, () => after.frames[0], "echo")).toEqual({ + type: "message", + data: "echo:hello", + }); + after.send({ type: "message", data: "ping" }); + expect(await until(restarted.reports, () => after.frames[1], "pong")).toEqual({ + type: "message", + data: "pong", + }); + after.send({ type: "close", code: 4000, reason: "bye" }); + expect(await next(restarted.reports, "close")).toEqual({ + kind: "close", + code: 4000, + reason: "bye", + wasClean: true, + }); + + // Both pings were answered without waking a handler in either worker. + expect(accepting.reports.map(({ kind }) => kind)).toEqual(["accepted", "hibernated"]); + expect(restarted.reports.map(({ kind }) => kind)).toEqual(["ready", "message", "close"]); + } finally { + for (const worker of workers.splice(0)) worker.terminate(); + } +}, 30_000); diff --git a/conformance/browser/hibernation-worker-restart.worker.ts b/conformance/browser/hibernation-worker-restart.worker.ts new file mode 100644 index 0000000..e007fee --- /dev/null +++ b/conformance/browser/hibernation-worker-restart.worker.ts @@ -0,0 +1,156 @@ +/** + * An actor worker that accepts a hibernatable socket over a transferred + * `MessagePort`, and the replacement worker that rehydrates it after + * `terminate()`. The two halves follow the Chrome-extension host this runtime + * serves: the first connect is the actor's own `fetch`, bridged to the port; + * the rehydration wraps the replacement's port directly and hands it to the + * container with the state the first worker mirrored. + * + * The boot order is the README's: the pool, then the actor scope, then the + * container. + */ + +import { + createActorContainer, + DEFAULT_ALARM_OUTLET, + HibernationMirror, + installActorScope, + noFacets, + WebSocketRequestResponsePair, + type ActorContainer, + type HibernationAutoResponse, +} from "../../src/index"; +import { installWebSocketUpgradeGlobals, upgradeWebSocket } from "../../src/browser"; +import { bridgeWebSocket, MessagePortWebSocket } from "../../src/browser/message-port-websocket"; +import { SqliteWasmActorStorage } from "../../backends/sqlite-wasm"; +import { installPool, timer, UNIQUE_KEY } from "./substrate"; + +/** A mirrored socket without its transport, which is what outlives the worker. */ +export type HibernatedSocket = { + readonly tags?: readonly string[]; + readonly attachment?: Uint8Array; + readonly autoResponseTimestamp?: number; +}; + +export type Hibernated = { + readonly socket: HibernatedSocket; + readonly autoResponse: HibernationAutoResponse | null; +}; + +export type SocketBoot = { + readonly poolName: string; + readonly url: string; + /** This worker's end of the socket's port. */ + readonly port: MessagePort; + /** Absent for the worker that accepts the socket; the mirror for its replacement. */ + readonly hibernated?: Hibernated; +}; + +export type SocketReport = + | { readonly kind: "accepted" } + | { readonly kind: "ready" } + | ({ readonly kind: "hibernated" } & Hibernated) + | { + readonly kind: "message"; + readonly message: string; + readonly tags: readonly string[]; + readonly attachment: unknown; + readonly autoResponseTimestamp: number | null; + readonly stored: unknown; + } + | { + readonly kind: "close"; + readonly code: number; + readonly reason: string; + readonly wasClean: boolean; + } + | { readonly kind: "error"; readonly error: string }; + +const report = (value: SocketReport): void => self.postMessage(value); + +class Room { + constructor(private readonly ctx: DurableObjectState) {} + + async fetch(): Promise { + const pair = new WebSocketPair(); + this.ctx.acceptWebSocket(pair[1], ["room:lobby", "user:ada"]); + pair[1].serializeAttachment({ user: "ada", since: new Date(0), roles: new Set(["admin"]) }); + this.ctx.setWebSocketAutoResponse(new WebSocketRequestResponsePair("ping", "pong")); + await this.ctx.storage.put("user", "ada"); + return new Response(null, { status: 101, webSocket: pair[0] }); + } + + async webSocketMessage(ws: WebSocket, message: string | ArrayBuffer): Promise { + report({ + kind: "message", + message: String(message), + tags: this.ctx.getTags(ws), + attachment: WebSocket.prototype.deserializeAttachment.call(ws), + autoResponseTimestamp: this.ctx.getWebSocketAutoResponseTimestamp(ws)?.getTime() ?? null, + stored: await this.ctx.storage.get("user"), + }); + ws.send(`echo:${String(message)}`); + } + + webSocketClose(_ws: WebSocket, code: number, reason: string, wasClean: boolean): void { + report({ kind: "close", code, reason, wasClean }); + } +} + +let container: ActorContainer | undefined; +let mirror: HibernationMirror | undefined; + +self.addEventListener("message", (event: MessageEvent) => { + void (event.data === "hibernate" ? hibernate() : boot(event.data)).catch((error: unknown) => { + report({ kind: "error", error: String(error) }); + }); +}); + +async function boot({ poolName, url, port, hibernated }: SocketBoot): Promise { + const host = await installPool(poolName, { clearOnInit: hibernated === undefined }); + installActorScope(globalThis, () => { + if (container === undefined) throw new Error("no live room container"); + return container.globals; + }); + installWebSocketUpgradeGlobals(); + + if (hibernated === undefined) { + mirror = new HibernationMirror(); + } else { + const socket = new MessagePortWebSocket(url, port, false); + socket.open(); + mirror = new HibernationMirror([{ socket, ...hibernated.socket }], hibernated.autoResponse); + } + container = await createActorContainer({ + id: "room", + uniqueKey: UNIQUE_KEY, + exports: {}, + env: {}, + ports: { + sql: new SqliteWasmActorStorage(host, "/actor"), + alarms: DEFAULT_ALARM_OUTLET, + facets: noFacets, + timer, + hibernation: mirror, + }, + webSockets: mirror.snapshot(), + }); + const room = container.entry(await container.start((ctx) => new Room(ctx))); + if (hibernated !== undefined) { + report({ kind: "ready" }); + return; + } + + const server = upgradeWebSocket(await room.fetch()); + if (server === undefined) throw new Error("the room did not upgrade"); + bridgeWebSocket(server, new MessagePortWebSocket(url, port, false)); + report({ kind: "accepted" }); +} + +/** What a host keeps before it discards the worker: the mirror, minus the transport. */ +async function hibernate(): Promise { + const [mirrored, ...others] = mirror?.snapshot() ?? []; + if (mirrored === undefined || others.length > 0) throw new Error("expected one mirrored socket"); + const { socket: _transport, ...socket } = mirrored; + report({ kind: "hibernated", socket, autoResponse: mirror?.autoResponsePair ?? null }); +} diff --git a/conformance/browser/host.ts b/conformance/browser/host.ts index 7b03c3f..b13953b 100644 --- a/conformance/browser/host.ts +++ b/conformance/browser/host.ts @@ -53,11 +53,13 @@ * * **A real OPFS crash stays outside the shared oracle.** Worker termination asks * whether SQLite's rollback journal survives a process disappearing mid-write on - * OPFS, which workerd cannot answer because it has no OPFS. The browser-only - * `sqlite-wasm-crash.smoke.spec.ts` therefore owns that proof: it commits one row, - * terminates the worker with a second row in an open transaction, retries bounded - * replacement workers until the browser releases the pool handles, and observes - * only the committed row. The shared §1.6 crash row keeps the narrower portable + * OPFS, which workerd cannot answer because it has no OPFS. The browser-only smoke + * specs therefore own that proof. `sqlite-wasm-crash.smoke.spec.ts` commits one + * row, terminates the worker with a second row in an open transaction, and starts + * one replacement, whose `installSqliteWasmHost` waits for the pool handles to be + * released; it observes only the committed row. `actor-crash.smoke.spec.ts` + * terminates an actor after SQLite has spilled journaled pages, so recovery has to + * replay the hot journal. The shared §1.6 crash row keeps the narrower portable * contract: volatile instance state disappears while output-gated storage stays. * * `crash()` below is the same thing the node lane means by it: drop the container diff --git a/conformance/browser/sqlite-wasm-crash.smoke.spec.ts b/conformance/browser/sqlite-wasm-crash.smoke.spec.ts index 5ecbc25..a829a35 100644 --- a/conformance/browser/sqlite-wasm-crash.smoke.spec.ts +++ b/conformance/browser/sqlite-wasm-crash.smoke.spec.ts @@ -4,6 +4,12 @@ import type { CrashReport, } from "./sqlite-wasm-crash.worker"; +/** + * Longer than `installSqliteWasmHost` waits for a terminated worker's handles (10 s), so a slow + * release fails as the helper's refusal rather than as a timeout here. + */ +const WORKER_TIMEOUT_MS = 12_000; + function runWorker(command: CrashCommand): Promise<{ worker: Worker; report: CrashReport }> { const worker = new Worker(new URL("./sqlite-wasm-crash.worker.ts", import.meta.url), { type: "module", @@ -13,7 +19,7 @@ function runWorker(command: CrashCommand): Promise<{ worker: Worker; report: Cra const timeout = setTimeout(() => { worker.terminate(); resolve({ worker, report: { kind: "error", error: "worker timed out" } }); - }, 2_000); + }, WORKER_TIMEOUT_MS); worker.addEventListener("message", (event: MessageEvent) => { clearTimeout(timeout); resolve({ worker, report: event.data }); @@ -34,22 +40,11 @@ test( dirty.worker.terminate(); expect(dirty.report).toEqual({ kind: "dirty" }); - const deadline = Date.now() + 10_000; - let lastError = "replacement worker did not start"; - while (Date.now() < deadline) { - const recovered = await runWorker({ mode: "recover", poolName }); - recovered.worker.terminate(); - if (recovered.report.kind === "recovered") { - expect(recovered.report.rows).toEqual(["committed"]); - return; - } - lastError = - recovered.report.kind === "error" - ? recovered.report.error - : `unexpected ${recovered.report.kind} report`; - await new Promise((resolve) => setTimeout(resolve, 50)); - } - throw new Error(`replacement worker could not reacquire OPFS: ${lastError}`); + // One replacement: its install waits for the terminated worker's handles, and killing it + // mid-wait to try again would only leave another set of handles held. + const recovered = await runWorker({ mode: "recover", poolName }); + recovered.worker.terminate(); + expect(recovered.report).toEqual({ kind: "recovered", rows: ["committed"] }); }, 20_000, ); diff --git a/conformance/browser/sqlite-wasm-crash.worker.ts b/conformance/browser/sqlite-wasm-crash.worker.ts index 0ce43dc..6d5cce9 100644 --- a/conformance/browser/sqlite-wasm-crash.worker.ts +++ b/conformance/browser/sqlite-wasm-crash.worker.ts @@ -1,5 +1,5 @@ import sqlite3InitModule from "@sqlite.org/sqlite-wasm"; -import { createSqliteWasmProvider, type SqliteWasmHost } from "../../backends/sqlite-wasm"; +import { createSqliteWasmProvider, installSqliteWasmHost } from "../../backends/sqlite-wasm"; export type CrashCommand = { readonly mode: "dirty" | "recover"; @@ -18,16 +18,12 @@ self.addEventListener("message", (event: MessageEvent) => { }); async function run({ mode, poolName }: CrashCommand): Promise { - const sqlite3 = await sqlite3InitModule(); - const pool = await sqlite3.installOpfsSAHPoolVfs({ + const host = await installSqliteWasmHost(await sqlite3InitModule(), { name: poolName, clearOnInit: mode === "dirty", initialCapacity: 4, }); - const database = await createSqliteWasmProvider( - { pool, capi: sqlite3.capi } satisfies SqliteWasmHost, - { prefix: "/crash" }, - ).open("root"); + const database = await createSqliteWasmProvider(host, { prefix: "/crash" }).open("root"); if (mode === "dirty") { database.exec("CREATE TABLE recovery (value TEXT NOT NULL)", []); @@ -44,6 +40,6 @@ async function run({ mode, poolName }: CrashCommand): Promise { return value; }); database.close(); - pool.pauseVfs(); + host.pool.pauseVfs(); self.postMessage({ kind: "recovered", rows } satisfies CrashReport); } diff --git a/conformance/browser/substrate.ts b/conformance/browser/substrate.ts index c28b0f9..65e30d6 100644 --- a/conformance/browser/substrate.ts +++ b/conformance/browser/substrate.ts @@ -22,11 +22,13 @@ * `clearOnInit` is safe for the same reason: it runs once, when the worker * installs its pool, and never again for the life of that actor. What it buys is * that a browser profile carrying pool files from an earlier run cannot make a - * later one pass or fail for reasons the run itself did not create. + * later one pass or fail for reasons the run itself did not create. The restart + * specs are the exception by design: their replacement worker reopens the pool a + * terminated worker owned, with `clearOnInit: false`. */ import sqlite3InitModule from "@sqlite.org/sqlite-wasm"; -import type { SqliteWasmHost } from "../../backends/sqlite-wasm"; +import { installSqliteWasmHost, type SqliteWasmHost } from "../../backends/sqlite-wasm"; import type { Timer } from "../../src/index"; /** @@ -37,11 +39,13 @@ import type { Timer } from "../../src/index"; */ export const UNIQUE_KEY = "do-runtime-conformance-browser"; -export async function installPool(name: string): Promise { - const sqlite3 = await sqlite3InitModule(); - const pool = await sqlite3.installOpfsSAHPoolVfs({ +export async function installPool( + name: string, + { clearOnInit = true }: { readonly clearOnInit?: boolean } = {}, +): Promise { + return await installSqliteWasmHost(await sqlite3InitModule(), { name, - clearOnInit: true, + clearOnInit, // One pool now holds the whole actor tree: the root's own database, the facet // tree index, one database per facet placed, and a rollback journal beside // each of those as a further file. The default of six is enough only until it @@ -71,7 +75,6 @@ export async function installPool(name: string): Promise { // decides whether the tree fits. initialCapacity: 64, }); - return { pool, capi: sqlite3.capi }; } /** diff --git a/conformance/browser/vitest.transformed.config.ts b/conformance/browser/vitest.transformed.config.ts new file mode 100644 index 0000000..f6458c3 --- /dev/null +++ b/conformance/browser/vitest.transformed.config.ts @@ -0,0 +1,26 @@ +import { fileURLToPath } from "node:url"; +import { defineConfig, mergeConfig } from "vitest/config"; +import { doRuntimeAwaitTransform } from "../../src/vite"; +import lane from "./vitest.config"; + +const packageRoot = fileURLToPath(new URL("../../", import.meta.url)); + +/** + * The same suite with the probe compiled the way a consumer compiles actor code. The workerd + * lane stays untransformed: it is the oracle this lane's transformed awaits must agree with. + */ +export default mergeConfig( + lane, + defineConfig({ + plugins: [ + doRuntimeAwaitTransform({ include: "**/conformance/fixtures/probe.ts", asyncContext: true }), + ], + resolve: { + alias: { + "@mcp-b/do-runtime/gate": `${packageRoot}src/gate.ts`, + "@mcp-b/do-runtime/browser/async-hooks": `${packageRoot}src/browser/async-hooks.ts`, + }, + }, + test: { name: "browser-transformed" }, + }), +); diff --git a/conformance/fixtures/probe.ts b/conformance/fixtures/probe.ts index 968ed6f..a007ba8 100644 --- a/conformance/fixtures/probe.ts +++ b/conformance/fixtures/probe.ts @@ -126,7 +126,6 @@ export class Probe extends DurableObject { >(); #handlerEvents: Record[] = []; #handlerTrace: string[] = []; - #handlerTimes: { event: string; at: number }[] = []; #listenerMessages = 0; #latePair: [WebSocket, WebSocket] | undefined; #capacityClients: WebSocket[] = []; @@ -207,6 +206,31 @@ export class Probe extends DurableObject { this.trace.push("fetch:exit"); return { marker: this.marker, status: response.status, body }; } + /** + * A fetched body read the three ways SDK code reads one, with a storage write after every + * chunk. Every lane's outbound streams its second chunk after a timer, so that read resumes + * from a later task: ungated, its write has no input lock. Measured: each chunk resumes gated. + */ + async readFetchedBody(): Promise> { + const url = "https://conformance.invalid/body"; + const decoder = new TextDecoder(); + const out = { piped: [] as string[], reader: [] as string[], iterated: [] as string[] }; + const piped = (await fetch(url)).body!.pipeThrough(new TextDecoderStream()).getReader(); + for (let next = await piped.read(); !next.done; next = await piped.read()) { + out.piped.push(next.value); + await this.ctx.storage.put("piped", out.piped); + } + const reader = (await fetch(url)).body!.getReader(); + for (let next = await reader.read(); !next.done; next = await reader.read()) { + out.reader.push(decoder.decode(next.value)); + await this.ctx.storage.put("reader", out.reader); + } + for await (const chunk of (await fetch(url)).body!) { + out.iterated.push(decoder.decode(chunk)); + await this.ctx.storage.put("iterated", out.iterated); + } + return out; + } /** Local storage. Measured: HOLDS, so this returns "A". */ async gateViaStorage(): Promise { this.marker = "A"; @@ -214,6 +238,12 @@ export class Probe extends DurableObject { await this.ctx.storage.get("probe"); return this.marker; } + /** A plain value. Measured: a microtask never returns to the event loop, so this returns "A". */ + async gateViaPlainValue(): Promise { + this.marker = "A"; + await 42; + return this.marker; + } // -- §1.2 host-provided async primitives resume gated ---------------------- // @@ -268,6 +298,29 @@ export class Probe extends DurableObject { return `${(await this.ctx.storage.get("afterDigest")) ?? "MISSING"}`; } + /** + * A stream this actor creates and only an outside consumer pulls (`highWaterMark: 0`). Storage + * comes first in `pull`, so a callback that did not re-enter the actor has no input lock. + * Measured: `"123"`, then 3 pulls. + */ + pullStream(): ReadableStream { + let n = 0; + const storage = this.ctx.storage; + return new ReadableStream( + { + async pull(controller) { + await storage.put("pulls", ++n); + controller.enqueue(new TextEncoder().encode(String(await storage.get("pulls")))); + if (n === 3) controller.close(); + }, + }, + { highWaterMark: 0 }, + ); + } + async readPulls(): Promise { + return (await this.ctx.storage.get("pulls")) ?? 0; + } + /** * A `setTimeout` callback runs gated. Upstream captures the critical section * at the call and re-enters through `context.run(cb, cs)` @@ -329,10 +382,11 @@ export class Probe extends DurableObject { // // Writes here are deliberately un-awaited: the question is what survives when // the actor dies before the implicit transaction commits. - /** Neither survives: one transaction spans the storage await. */ + /** Neither survives: one transaction spans both storage awaits. */ async txAcrossStorageAwait(): Promise { void this.ctx.storage.put("p1", 1); await this.ctx.storage.get("p1"); + await this.ctx.storage.get("p1"); void this.ctx.storage.put("p2", 2); this.ctx.abort("conformance: kill before commit"); throw new Error("unreachable after conformance: kill before commit"); @@ -378,17 +432,40 @@ export class Probe extends DurableObject { // -- §1.5 critical sections ------------------------------------------------ flag = "init"; async setFlag(): Promise { + this.trace.push("setFlag"); this.flag = "B"; return "set"; } /** Measured: genuinely blocks, so this returns "A". */ async blockConcurrency(): Promise { await this.ctx.blockConcurrencyWhile(async () => { + this.trace.push("section:enter"); this.flag = "A"; await scheduler.wait(60); + this.trace.push("section:exit"); }); return this.flag; } + /** + * A section that throws. Measured: the caller sees the callback's error and the object is + * reset; the put before the section survives and the one inside it is rolled back. + */ + async failSection(): Promise { + this.marker = "dirty"; + await this.ctx.storage.put("before", 1); + await this.ctx.blockConcurrencyWhile(async () => { + await this.ctx.storage.put("inside", 1); + throw new Error("conformance: section failed"); + }); + return "returned"; + } + async readSectionFailure(): Promise> { + return { + marker: this.marker, + before: (await this.ctx.storage.get("before")) ?? null, + inside: (await this.ctx.storage.get("inside")) ?? null, + }; + } /** Measured: nests without deadlocking. */ async nestedBlockConcurrency(): Promise { return await this.ctx.blockConcurrencyWhile(async () => @@ -590,19 +667,25 @@ export class Probe extends DurableObject { * actually is instead of what it was assumed to be. */ async reservedNames(): Promise> { + // Classified by message, as `sqlPragmas` does: "no such table" is not the refusal. SQLite words + // a denied column read "access to … is prohibited", so both authorizer wordings count. const attempt = (sql: string): string => { try { this.ctx.storage.sql.exec(sql); return "allowed"; - } catch { - return "refused"; + } catch (error) { + return error instanceof Error && /not authorized|SQLITE_AUTH/.test(error.message) + ? "refused" + : `refused otherwise: ${String(error)}`; } }; this.ctx.storage.sql.exec("CREATE TABLE IF NOT EXISTS names(callback TEXT)"); + // `_cf_KV` exists once a KV value does, so the SELECT below reads a real reserved table. + this.ctx.storage.kv.put("reserved-names", 1); return { // A reserved identifier, which is the whole point of the rule. createTable: attempt("CREATE TABLE IF NOT EXISTS _cf_probe(x)"), - selectFrom: attempt("SELECT * FROM _cf_probe"), + selectFrom: attempt("SELECT * FROM _cf_KV"), // A quoted identifier is still an identifier. quotedIdentifier: attempt('CREATE TABLE IF NOT EXISTS "_cf_quoted"(x)'), // Data. The `agents` statement above, with its own table name. @@ -1001,6 +1084,25 @@ export class Probe extends DurableObject { return { asyncRead, syncRead, listed, deleted, missing: typeof kv.get("shared:sync") }; } + /** + * `list()` range options, and the count a multi-key `delete` returns. `p;` is the first key + * after the `p:` prefix range, so a reverse listing that overshot the range would return it. + */ + async listOptions(): Promise> { + const storage = this.ctx.storage; + await storage.put({ a: 1, b: 2, c: 3, d: 4, "p:1": 1, "p:2": 2, "p:3": 3, "p;": 0 }); + const keys = async (options?: DurableObjectListOptions) => [ + ...(await storage.list(options)).keys(), + ]; + return { + lastWithPrefix: await keys({ prefix: "p:", reverse: true, limit: 1 }), + startEnd: await keys({ start: "b", end: "d" }), + startAfterEnd: await keys({ startAfter: "b", end: "d" }), + deleted: await storage.delete(["a", "b", "missing"]), + remaining: await keys(), + }; + } + /** `deleteAll()` resets the actor database, including alarm metadata. */ async deleteAllState(): Promise> { await this.ctx.storage.put("delete-me", "present"); @@ -1013,8 +1115,8 @@ export class Probe extends DurableObject { } // -- §2.4 the value codec -------------------------------------------------- - /** Rich values retain the same public structured-clone types in every lane. */ - async richValueRoundTrip(): Promise> { + /** Rich values retain the same public structured-clone types and contents in every lane. */ + async richValueRoundTrip(): Promise> { await this.ctx.storage.put("codec", { when: new Date(0), map: new Map([["k", 1]]), @@ -1026,12 +1128,12 @@ export class Probe extends DurableObject { const read = await this.ctx.storage.get>("codec"); if (read === undefined) throw new Error("Stored rich value disappeared."); return { - when: read.when instanceof Date ? "Date" : typeof read.when, - map: read.map instanceof Map ? "Map" : typeof read.map, - set: read.set instanceof Set ? "Set" : typeof read.set, - bytes: read.bytes instanceof ArrayBuffer ? "ArrayBuffer" : typeof read.bytes, - re: read.re instanceof RegExp ? "RegExp" : typeof read.re, - err: read.err instanceof Error ? "Error" : typeof read.err, + when: read.when instanceof Date ? read.when.getTime() : typeof read.when, + map: read.map instanceof Map ? [...read.map] : typeof read.map, + set: read.set instanceof Set ? [...read.set] : typeof read.set, + bytes: read.bytes instanceof ArrayBuffer ? [...new Uint8Array(read.bytes)] : typeof read.bytes, + re: read.re instanceof RegExp ? String(read.re) : typeof read.re, + err: read.err instanceof Error ? String(read.err) : typeof read.err, }; } @@ -1077,6 +1179,12 @@ export class Probe extends DurableObject { async readAlarmLog(): Promise { return (await this.ctx.storage.get("alarmLog")) ?? []; } + /** Measured: `getAlarm()` answers null, and the handler never runs (`alarmLog` stays empty). */ + async armThenDeleteAlarm(): Promise { + await this.ctx.storage.setAlarm(Date.now() + 200); + await this.ctx.storage.deleteAlarm(); + return await this.ctx.storage.getAlarm(); + } // -- §1.8 a failed alarm is retried, and told how many times --------------- /** @@ -1347,19 +1455,24 @@ export class Probe extends DurableObject { this.#throwNextMessage = true; } + /** + * Missing for exactly one dispatch: the lookup that finds nothing also restores the method, so + * the next frame reaches it with no race between frame delivery and a restoring call. + */ removeSocketMessageHandler(): void { - Object.defineProperty(this, "webSocketMessage", { configurable: true, value: undefined }); - } - - restoreSocketMessageHandler(): void { - Reflect.deleteProperty(this, "webSocketMessage"); + Object.defineProperty(this, "webSocketMessage", { + configurable: true, + get: () => { + Reflect.deleteProperty(this, "webSocketMessage"); + return undefined; + }, + }); } socketJournal(): Record { return { events: this.#handlerEvents, trace: this.#handlerTrace, - times: this.#handlerTimes, listenerMessages: this.#listenerMessages, clients: Object.fromEntries( [...this.#clientMessages].map(([id, messages]) => [ @@ -1592,20 +1705,16 @@ export class Probe extends DurableObject { if (typeof message === "string" && message.startsWith("slow:")) { this.#handlerTrace.push(`start:${message}`); - this.#handlerTimes.push({ event: `start:${message}`, at: Date.now() }); await scheduler.wait(200); this.#handlerTrace.push(`end:${message}`); - this.#handlerTimes.push({ event: `end:${message}`, at: Date.now() }); return; } if (typeof message === "string" && message.startsWith("block:")) { this.#handlerTrace.push(`start:${message}`); - this.#handlerTimes.push({ event: `start:${message}`, at: Date.now() }); await this.ctx.blockConcurrencyWhile(async () => { await scheduler.wait(200); }); this.#handlerTrace.push(`end:${message}`); - this.#handlerTimes.push({ event: `end:${message}`, at: Date.now() }); return; } if (message === "echo") ws.send("echoed"); diff --git a/conformance/node/host.ts b/conformance/node/host.ts index 9241993..4fdb6cc 100644 --- a/conformance/node/host.ts +++ b/conformance/node/host.ts @@ -36,9 +36,10 @@ import { AlarmScheduler, createActorContainer, HibernationMirror, - installWebSocketGlobals, + installActorScope, type ActorContainer, type ActorEntry, + type ActorGlobalScope, type FacetHandle, type FacetHost, type FacetId, @@ -127,90 +128,28 @@ const current = new AsyncLocalStorage(); /** Captured before anything is installed, so the fall-through below cannot recurse. */ const nodeSetTimeout = globalThis.setTimeout; const nodeClearTimeout = globalThis.clearTimeout; -const nodeSetInterval = globalThis.setInterval; -const nodeClearInterval = globalThis.clearInterval; -const nodeFetch = globalThis.fetch; - -type SchedulerGlobal = { - wait(ms: number, options?: { signal?: AbortSignal }): Promise; - yield(): Promise; -}; - -const scheduler: SchedulerGlobal = { - wait: (ms, options) => { - const container = current.getStore(); - if (container === undefined) { - return new Promise((resolve) => { - nodeSetTimeout(resolve, ms); - }); - } - return container.globals.scheduler.wait(ms, options); - }, - yield: () => scheduler.wait(0), -}; -(globalThis as { scheduler?: SchedulerGlobal }).scheduler ??= scheduler; - -globalThis.setTimeout = ((callback: (...args: never[]) => void, ms?: number, ...args: never[]) => { - const container = current.getStore(); - if (container === undefined) return nodeSetTimeout(callback, ms, ...args); - return container.globals.setTimeout(callback, ms, ...args); -}) as typeof globalThis.setTimeout; - -globalThis.clearTimeout = ((id?: number) => { - const container = current.getStore(); - if (container === undefined) return nodeClearTimeout(id); - return container.globals.clearTimeout(id); -}) as typeof globalThis.clearTimeout; - -globalThis.setInterval = ((callback: (...args: never[]) => void, ms?: number, ...args: never[]) => { - const container = current.getStore(); - if (container === undefined) return nodeSetInterval(callback, ms, ...args); - return container.globals.setInterval(callback, ms, ...args); -}) as typeof globalThis.setInterval; - -globalThis.clearInterval = ((id?: number) => { - const container = current.getStore(); - if (container === undefined) return nodeClearInterval(id); - return container.globals.clearInterval(id); -}) as typeof globalThis.clearInterval; - -globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => { - const container = current.getStore(); - if (container === undefined) return nodeFetch(input, init); - return container.globals.fetch(input, init); -}) as typeof globalThis.fetch; - -const ActorWebSocketPair: typeof WebSocketPair = new Proxy( - class WebSocketPair { - declare readonly 0: WebSocket; - declare readonly 1: WebSocket; - }, - { - construct() { - const Pair = current.getStore()?.globals.WebSocketPair; - if (Pair === undefined) { - throw new Error("Node lane: WebSocketPair was constructed outside an actor event."); - } - return new Pair(); - }, - }, -); -installWebSocketGlobals(globalThis, ActorWebSocketPair); -installWebSocketUpgradeGlobals(); /** - * `crypto`, on the same ambient as the timers above. - * - * A getter rather than an assigned value, because `crypto.subtle` is read at the - * call site and the answer depends on which actor is running — the whole point of - * `current` here. Outside an actor it is the platform's, which is what `nodeCrypto` - * keeps. + * What the installed names reach with an empty ambient: Node's own, captured before the install. + * The rest of `installActorScope`'s set — `WebSocketPair`, `scheduler` — has no caller outside an + * actor here, so it has no fall-through either. One actor caller does arrive with an empty store: + * a stream callback pulled from an outside consumer's async context re-enters its actor's lock but + * not this lane's store, so a scope primitive called there is not the actor's. */ -const nodeCrypto = globalThis.crypto; -Object.defineProperty(globalThis, "crypto", { - configurable: true, - get: () => (current.getStore()?.globals.crypto as Crypto | undefined) ?? nodeCrypto, -}); +const platform = { + setTimeout: nodeSetTimeout, + clearTimeout: nodeClearTimeout, + setInterval: globalThis.setInterval, + clearInterval: globalThis.clearInterval, + fetch: globalThis.fetch, + crypto: globalThis.crypto, + WebSocket: globalThis.WebSocket, +} as unknown as ActorGlobalScope; + +// The package's own set, so a binding added there (the actor stream constructors were the first +// this lane missed) reaches this lane without a line here. +installActorScope(globalThis, () => current.getStore()?.globals ?? platform); +installWebSocketUpgradeGlobals(); /** * Runs every method of an actor instance inside that actor's ambient, so a @@ -791,9 +730,18 @@ async function place(name: string): Promise { facets: host, timer, hibernation, + // "fetched" in two chunks, the second after a timer, as every lane's outbound answers. fetch: async () => { await timer.afterDelay(60); - return new Response("fetched"); + const body = new ReadableStream({ + async start(controller) { + controller.enqueue(new TextEncoder().encode("fet")); + await timer.afterDelay(20); + controller.enqueue(new TextEncoder().encode("ched")); + controller.close(); + }, + }); + return new Response(body); }, }, webSockets: hibernation.snapshot(), diff --git a/conformance/node/vitest.transformed.config.ts b/conformance/node/vitest.transformed.config.ts new file mode 100644 index 0000000..c10b67a --- /dev/null +++ b/conformance/node/vitest.transformed.config.ts @@ -0,0 +1,26 @@ +import { fileURLToPath } from "node:url"; +import { defineConfig, mergeConfig } from "vitest/config"; +import { doRuntimeAwaitTransform } from "../../src/vite"; +import lane from "./vitest.config"; + +const packageRoot = fileURLToPath(new URL("../../", import.meta.url)); + +/** + * The same suite with the probe compiled the way a consumer compiles actor code. The workerd + * lane stays untransformed: it is the oracle this lane's transformed awaits must agree with. + */ +export default mergeConfig( + lane, + defineConfig({ + plugins: [ + doRuntimeAwaitTransform({ include: "**/conformance/fixtures/probe.ts", asyncContext: true }), + ], + resolve: { + alias: { + "@mcp-b/do-runtime/gate": `${packageRoot}src/gate.ts`, + "@mcp-b/do-runtime/browser/async-hooks": `${packageRoot}src/browser/async-hooks.ts`, + }, + }, + test: { name: "node-transformed" }, + }), +); diff --git a/conformance/suite/alarms.spec.ts b/conformance/suite/alarms.spec.ts index 1f8cbc5..c60f756 100644 --- a/conformance/suite/alarms.spec.ts +++ b/conformance/suite/alarms.spec.ts @@ -13,13 +13,25 @@ import { host } from "conformance:host"; it("§1.8 an alarm re-armed from inside its own handler does not re-enter", async () => { const probe = await host.spawn("alarm-overlap"); await probe.call("armAlarm"); - await new Promise((resolve) => setTimeout(resolve, 1_200)); - expect(await probe.call("readAlarmLog")).toEqual([ - "enter:1", - "exit:1", - "enter:2", - "exit:2", - ]); + const deadline = Date.now() + 5_000; + let log: string[] = []; + while (log.length < 4 && Date.now() < deadline) { + await new Promise((resolve) => setTimeout(resolve, 50)); + log = await probe.call("readAlarmLog"); + } + expect(log).toEqual(["enter:1", "exit:1", "enter:2", "exit:2"]); +}); + +it("§1.8 deleteAlarm cancels a pending alarm before it fires", async () => { + const probe = await host.spawn("alarm-delete"); + expect(await probe.call("armThenDeleteAlarm")).toBeNull(); + // Nothing marks an absence, so poll through the window the alarm was due in and fail on the + // first delivery. Load can hide a late delivery; it cannot fail a correct lane. + const deadline = Date.now() + 1_000; + while (Date.now() < deadline) { + expect(await probe.call("readAlarmLog")).toEqual([]); + await new Promise((resolve) => setTimeout(resolve, 50)); + } }); /** diff --git a/conformance/suite/critical-sections.spec.ts b/conformance/suite/critical-sections.spec.ts index 2d9792b..0d7ce89 100644 --- a/conformance/suite/critical-sections.spec.ts +++ b/conformance/suite/critical-sections.spec.ts @@ -14,6 +14,27 @@ it("§1.5 blockConcurrencyWhile blocks a concurrent event", async () => { const other = probe.post("setFlag"); await other.settled; expect(await blocking.settled).toBe("A"); + // The two posts are separate requests on the workerd lane, so `setFlag` may land first; what + // must never happen is `setFlag` running between the section's entry and its exit. + expect([ + ["section:enter", "section:exit", "setFlag"], + ["setFlag", "section:enter", "section:exit"], + ]).toContainEqual(await probe.call("readTrace")); +}); + +it("§1.5 a throwing blockConcurrencyWhile rejects its caller and resets the object over committed storage", async () => { + const probe = await host.spawn("cs-throws"); + // The message only: the name differs by lane (`BrokenActorError` on this runtime, documented). + // Settled with both handlers because `expect().rejects` leaves workerd's reset RPC unhandled. + const call = await probe.call("failSection").then( + (value) => `fulfilled: ${String(value)}`, + (error: unknown) => String(error), + ); + expect(call).toContain("conformance: section failed"); + // By identity, as a later event would: workerd breaks the old stub, and `respawn` would replace + // the instance itself on the other lanes, hiding a runtime that never reset it. + const after = await host.spawn(probe.name); + expect(await after.call("readSectionFailure")).toEqual({ marker: "init", before: 1, inside: null }); }); it("§1.5 a nested blockConcurrencyWhile nests rather than deadlocking", async () => { diff --git a/conformance/suite/gates.spec.ts b/conformance/suite/gates.spec.ts index 97e9900..433439c 100644 --- a/conformance/suite/gates.spec.ts +++ b/conformance/suite/gates.spec.ts @@ -62,6 +62,15 @@ it("§1.2 fetch releases the input gate and its continuations resume gated", asy expect(await probe.call("readTrace")).toEqual(["fetch:enter", "setMarker", "fetch:exit"]); }); +it("§1.3 a fetched body read via pipeThrough(TextDecoderStream), getReader() and for-await resumes gated after every chunk", async () => { + const probe = await host.spawn("gate-fetched-body"); + expect(await probe.call("readFetchedBody")).toEqual({ + piped: ["fet", "ched"], + reader: ["fet", "ched"], + iterated: ["fet", "ched"], + }); +}); + it("§1.2 a local storage await HOLDS the input gate", async () => { // The asymmetry that shrinks the whole hazard surface: most awaits in agent // code are storage, and none of them is an interleaving point. @@ -72,6 +81,14 @@ it("§1.2 a local storage await HOLDS the input gate", async () => { expect(await slow.settled).toBe("A"); }); +it("§1.2 a plain-value await HOLDS the input gate", async () => { + const probe = await host.spawn("gate-plain-value"); + const slow = probe.post("gateViaPlainValue"); + const fast = probe.post("setMarker"); + await fast.settled; + expect(await slow.settled).toBe("A"); +}); + /** * §1.2 — the host-provided async primitives, one row each. * @@ -106,6 +123,13 @@ it("§1.2 a continuation after crypto.subtle.digest can still touch storage", as expect(await probe.call("storageAfterDigest")).toBe("32"); }); +it("§1.2 an actor-created ReadableStream re-enters its creator when an outside consumer pulls it", async () => { + const probe = await host.spawn("gate-stream-pull"); + const stream = await probe.call>("pullStream"); + expect(await new Response(stream).text()).toBe("123"); + expect(await probe.call("readPulls")).toBe(3); +}); + it("§1.2 a setTimeout callback runs gated and can touch storage", async () => { const probe = await host.spawn("gate-timer-callback"); expect(await probe.call("armTimer")).toBe("armed"); diff --git a/conformance/suite/hibernation.spec.ts b/conformance/suite/hibernation.spec.ts index 30dfe6b..b1fb738 100644 --- a/conformance/suite/hibernation.spec.ts +++ b/conformance/suite/hibernation.spec.ts @@ -23,7 +23,6 @@ async function eventually(read: () => Promise, ready: (value: T) => boolea async function journal(actor: ProbeActor): Promise<{ events: Record[]; trace: string[]; - times: { event: string; at: number }[]; listenerMessages: number; clients: Record; closes: Record; @@ -167,13 +166,11 @@ describe("handler dispatch and close state", () => { it("D2 silently drops missing and throwing handlers and keeps later delivery alive", async () => { const actor = await host.spawn("ws-handler-failures"); await actor.call("openSelfSocket", "socket", ["socket"]); + // Missing for one dispatch, then restored by that dispatch itself; frames arrive in order. await actor.call("removeSocketMessageHandler"); await actor.call("sendSelf", "socket", "missing"); - await new Promise((resolve) => setTimeout(resolve, 50)); - await actor.call("restoreSocketMessageHandler"); await actor.call("throwOnNextSocketMessage"); await actor.call("sendSelf", "socket", "throws"); - await new Promise((resolve) => setTimeout(resolve, 50)); await actor.call("sendSelf", "socket", "survives"); const result = await eventually( @@ -188,10 +185,10 @@ describe("handler dispatch and close state", () => { }); it("D4 overlaps handler promises but waits for blockConcurrencyWhile", async () => { + // Frames dispatch in order, so the trace alone says overlap or serialization. const concurrent = await host.spawn("ws-concurrent"); await concurrent.call("openSelfSocket", "socket", ["socket"]); await concurrent.call("sendSelf", "socket", "slow:one"); - await new Promise((resolve) => setTimeout(resolve, 60)); await concurrent.call("sendSelf", "socket", "slow:two"); const overlapping = await eventually( () => journal(concurrent), @@ -203,12 +200,10 @@ describe("handler dispatch and close state", () => { "end:slow:one", "end:slow:two", ]); - expect(overlapping.times[1]!.at - overlapping.times[0]!.at).toBeLessThan(180); const blocked = await host.spawn("ws-blocked"); await blocked.call("openSelfSocket", "socket", ["socket"]); await blocked.call("sendSelf", "socket", "block:one"); - await new Promise((resolve) => setTimeout(resolve, 60)); await blocked.call("sendSelf", "socket", "block:two"); const serialized = await eventually( () => journal(blocked), @@ -220,7 +215,6 @@ describe("handler dispatch and close state", () => { "start:block:two", "end:block:two", ]); - expect(serialized.times[2]!.at - serialized.times[0]!.at).toBeGreaterThanOrEqual(190); }); it("D5/B4 reports peer close while listed, tolerates reciprocity, then evicts", async () => { @@ -484,3 +478,26 @@ it("preserves tags and attachment across a real eviction without reconnecting", connects: 1, }); }); + +it("a socket rehydrated after eviction can send to its client and delivers the client's close to webSocketClose", async () => { + const actor = await host.spawn("ws-eviction-close"); + const client = await host.connect(actor, ["connection-id"]); + await host.evict(actor); + await client.send("echo"); + expect(await client.nextMessage()).toBe("echoed"); + + await client.close(4000, "bye"); + const result = await eventually( + () => journal(actor), + (value) => value.events.some((event) => "close" in event), + ); + expect(result.events.at(-1)).toEqual({ + id: "connection-id", + close: { code: 4000, reason: "bye", wasClean: true }, + readyState: 2, + listedDuringHandler: false, + tagsDuringHandler: ["connection-id"], + sendAfterPeerClose: null, + reciprocalClose: null, + }); +}); diff --git a/conformance/suite/storage.spec.ts b/conformance/suite/storage.spec.ts index 94d5f9e..8a1ab36 100644 --- a/conformance/suite/storage.spec.ts +++ b/conformance/suite/storage.spec.ts @@ -29,12 +29,23 @@ it("§2.4 deleteAll removes ordinary values and the stored alarm", async () => { it("§2.4 rich values round-trip with their workerd types", async () => { const probe = await host.spawn("codec"); expect(await probe.call("richValueRoundTrip")).toEqual({ - when: "Date", - map: "Map", - set: "Set", - bytes: "ArrayBuffer", - re: "RegExp", - err: "Error", + when: 0, + map: [["k", 1]], + set: [1], + bytes: [1, 2, 3], + re: "/pattern/g", + err: "Error: boom", + }); +}); + +it("§2.4 list() honours prefix with reverse and limit, start/startAfter/end, and delete([...]) returns the count", async () => { + const probe = await host.spawn("list-options"); + expect(await probe.call("listOptions")).toEqual({ + lastWithPrefix: ["p:3"], + startEnd: ["b", "c"], + startAfterEnd: ["c"], + deleted: 2, + remaining: ["c", "d", "p:1", "p:2", "p:3", "p;"], }); }); diff --git a/conformance/workerd/vitest.config.ts b/conformance/workerd/vitest.config.ts index 40a6686..7f3a8e3 100644 --- a/conformance/workerd/vitest.config.ts +++ b/conformance/workerd/vitest.config.ts @@ -13,9 +13,18 @@ export default defineConfig({ cloudflareTest({ wrangler: { configPath: `${here}wrangler.test.jsonc` }, miniflare: { + // "fetched" in two chunks, the second after a timer, as every lane's outbound answers. outboundService: async () => { await new Promise((resolve) => setTimeout(resolve, 60)); - return new Response("fetched"); + const body = new ReadableStream({ + async start(controller) { + controller.enqueue(new TextEncoder().encode("fet")); + await new Promise((resolve) => setTimeout(resolve, 20)); + controller.enqueue(new TextEncoder().encode("ched")); + controller.close(); + }, + }); + return new Response(body); }, }, }), diff --git a/docs/browser-async-context.md b/docs/browser-async-context.md index dcc05cb..7d70bee 100644 --- a/docs/browser-async-context.md +++ b/docs/browser-async-context.md @@ -27,7 +27,7 @@ Oxc transformer to lower async functions and generators to Promise callbacks. Importing the shim installs context binding on `Promise.prototype.then` once per realm. Importing the runtime alone does not patch Promise. -The opt-in transform also corrects Vite's bundled Oxc 0.144.0 generator helper: +The opt-in transform also corrects the generator helper from Vite's bundled Oxc (≥ 0.144.0): an early return must resume awaited `finally` cleanup with `next`, while a delegated `yield*` may still require `return`. This follows the [upstream Babel helper](https://github.com/babel/babel/blob/main/packages/babel-helpers/src/helpers/wrapAsyncGenerator.ts). diff --git a/docs/decisions.md b/docs/decisions.md index df7e14f..afa210a 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -158,6 +158,8 @@ await is wrapped once at its shared owner with `awaitIo`; it is not repaired at every caller or by patching the realm. The await transform's tokenized continuation marker is the sole exception: it republishes the context captured from the exact slice, and readers ignore it once that context's lock is gone. +The opt-in browser async-context import patches `Promise.prototype.then` for +`AsyncLocalStorage` only, not for actor scope. ### §2.4 Storage contract @@ -165,8 +167,10 @@ The public storage surface is the Workers TypeScript contract. Runtime internals use a small synchronous `SqlDatabase` seam, with `node:sqlite` and sqlite-wasm backends. A versioned browser-safe encoding preserves structured- clone value semantics and remains backward-readable with legacy JSON rows. A -present browser SAH pool accepts only current actor/facet logical database names -and SQLite-owned companions; an unknown name fails startup. +browser SAH pool holds only current actor/facet logical database names and +SQLite-owned companions. The runtime validates each name it opens but never +scans the pool at startup, so failing startup on an unknown name is the host's +obligation. Concrete local providers expose a host-owned, versioned snapshot of every database in one actor scope. Export and import require all handles closed; import @@ -189,6 +193,8 @@ The projection retains active delivery deadlines through retry persistence and abandonment, including deliveries whose entry was cancelled. Hosts acknowledge a consumed wake only after the latest projection is accepted and runtime activity has settled; timer-first delivery must leave a wake for worker recovery. +The library's projector waits on the latest projection and resends it first if +it failed, so an idle scheduler cannot leave an old failure standing. Failed start or completion bookkeeping retries on that same scheduler's timer with bounded backoff. A completed handler result stays with the pending cleanup, @@ -230,9 +236,11 @@ consumer peer dependencies retain one identity. gate. 8. Carry explicit actor scope through application-owned code and route raw host promises through `awaitIo`. Do not add a generic async-context shim or - patch the realm with zones. The compile-time await transform may republish - only the exact context it captured, tokenized for one checkpoint and valid - only while that context still holds an input lock. + patch the realm with zones, beyond the opt-in + [browser async context](browser-async-context.md) import. The compile-time + await transform may republish only the exact context it captured, tokenized + for one checkpoint and valid only while that context still holds an input + lock. 9. No stream pump or generic remote-facet protocol in the runtime. Facet placement is the host's; direct actor hops use native capabilities. 10. Store per-connection host bridges by connection id. Never use the diff --git a/docs/future-durability.md b/docs/future-durability.md index dfa4e52..c9599a9 100644 --- a/docs/future-durability.md +++ b/docs/future-durability.md @@ -88,11 +88,12 @@ at our SQLite seam, cheapest first: deterministically only if the statements are deterministic (`random()`, `CURRENT_TIMESTAMP`), and must also cover the runtime's own internal writes, which never pass the application regulator. -- **(c) Session extension / preupdate hook — the right primitive, likely - unavailable.** `sqlite3session_changeset` is an LTX-shaped logical delta. - Needs compile-time `SQLITE_ENABLE_SESSION`; **verify before planning anything - on it** that the shipped `@sqlite.org/sqlite-wasm` exports `sqlite3session_*`. - If absent this means a custom wasm build and browser/Node divergence. +- **(c) Session extension / preupdate hook — the right primitive, and + available.** `sqlite3session_changeset` is an LTX-shaped logical delta. It + needs compile-time `SQLITE_ENABLE_SESSION`, and both shipped engines have it + (verified September 2026): `@sqlite.org/sqlite-wasm` 3.53.4 exports the 16 + `sqlite3session_*` functions, and Node 24.11's `DatabaseSync` has + `createSession()` and `applyChangeset()`. No custom wasm build is needed. - **(d) A VFS that tees `xWrite`.** Genuine page-level frames with no build flags, at the cost of owning a VFS. @@ -142,8 +143,7 @@ whatever seam a future capture would attach to. Revisit when: a user loses actor data and it matters (OPFS is evictable — the likeliest first trigger); users want the same agent on two devices; a server component appears for any other reason, at which point a conditional-write -record is marginal; a wasm build with the session extension becomes routine; or -an actor must continue on another host while its device is offline. That last +record is marginal; or an actor must continue on another host while its device is offline. That last requirement needs coordinated ownership and replication rather than snapshot backup. Local OPFS already survives ordinary tab closure. If it all stays single-device and the only worry is loss, start with snapshot backup. diff --git a/docs/gating-coverage.md b/docs/gating-coverage.md index fb316d1..ebf5802 100644 --- a/docs/gating-coverage.md +++ b/docs/gating-coverage.md @@ -16,7 +16,7 @@ enumerates it. Every row is one of: - **Open hole** — reachable from gated flows today; needs a seam. - **Fail-closed** — refused loudly rather than passed through ungated. - **Foreign by design** — no runtime seam can exist; actor code must use - `awaitIo` / `makeReentryCallback` discipline (or, if the tail grows, the + `awaitIo` discipline (or, if the tail grows, the compile-time await transform below). - **Not in contract** — realm globals a Durable Object should never touch; listed so the fall-through is a decision, not an accident. @@ -32,7 +32,7 @@ enumerates it. Every row is one of: | `body.values()` / async iteration | iterator reads through the gated reader; early return preserves native cancel and lock-release semantics | `api/http.ts` | | Reader/stream lifecycle (`reader.closed`, both `cancel()` methods) | settlement uses `awaitIo`; `closed` is gated and registered once | `api/http.ts` | | `body.tee()` | both halves re-gated | `api/http.ts` | -| Actor-created `ReadableStream` / `TransformStream` callbacks | constructor captures the current actor and async stores; `pull`, `transform`, `flush`, and `cancel` use `makeReentryCallback`; synchronous `start` retains its native timing and receiver | `api/global-scope.ts`; delayed input and external consumer regressions in `global-scope.test.ts` | +| Actor-created `ReadableStream` / `TransformStream` callbacks | constructor captures the current actor and async stores; `pull`, `transform`, `flush`, and `cancel` use `makeReentryCallback`; synchronous `start` retains its native timing and receiver | `api/global-scope.ts`; delayed input and external consumer regressions in `global-scope.test.ts`; the §1.2 outside-consumer conformance row | | `body.pipeThrough()` / `pipeTo()` | returned readable re-gated (recurses through chains); settlement `awaitIo`d — native pipe machinery bypasses the `getReader` override and would launder the stream | `api/http.ts`, 0.2.2 | | `setTimeout` / `setInterval` | arming captures the critical section; firing re-enters via `ctx.run` | `api/global-scope.ts` | | `scheduler.wait()` / `scheduler.yield()` | scoped `Scheduler` over the same timer path | `api/global-scope.ts` | @@ -49,15 +49,39 @@ through `@mcp-b/do-runtime/gate`, and wraps every `for await` source so The gate helper fails open outside actor code. A development transform supplies the module id and warns once if that path is reached; production keeps the helper -silent. Inside an actor it publishes each continuation through a fresh -input-gated slice, including awaits of plain values. -The slice preserves a surrounding `blockConcurrencyWhile` critical section so -the section can await its own continuation without deadlocking. The tokenized -actor identity is realm-shared so separately bundled actor and host copies agree. -It exists only while the captured context still holds its input lock, the -synchronous current slice always wins, and the marker clears at that context's -checkpoint boundary. Publications are serialized across actors so two promises -settling in the same checkpoint cannot overwrite each other's identity. +silent. Inside an actor, an await resumes where workerd would, except as below: + +- **Settled while the actor still holds its input lock**, in the critical section + the await ran under — a storage call, a plain value, or a resumption the runtime + already admitted (a timer, `fetch`, actor RPC, the `blockConcurrencyWhile` + hand-back). The continuation runs in that checkpoint, so it keeps the lock and + the implicit transaction (§1.2, §1.7.1). +- **Settled anywhere else** — foreign I/O, a raw timer. The continuation re-enters + through a fresh input-gated slice queued behind waiting events (§1.3). That slice preserves a surrounding `blockConcurrencyWhile` + critical section so the section can await its own continuation without + deadlocking. + +The checkpoint ends at the runtime's `MessageChannel` hand-off, not at the end of +the microtask drain. A foreign promise that settles in the gap before that +hand-off therefore continues under the still-held lock, as an untransformed +continuation would. It can run ahead of this actor's earlier foreign +continuations already queued at the gate. A promise another actor resolves +resumes inline if this actor holds a lock; workerd defers it to this actor's own +turn. A promise resolved inside `blockConcurrencyWhile` resumes an await captured +outside the section only after the section ends, where workerd resumes it inside; +a section that waits on that continuation stalls until its 30-second deadline +breaks the actor. + +The tokenized actor identity is realm-shared so separately bundled actor and host +copies agree. It exists only while the captured context still holds its input +lock, the synchronous current slice always wins, and the marker clears at that +context's checkpoint boundary. One actor's continuations own a checkpoint, so two +actors' promises settling in the same checkpoint cannot overwrite each other's +identity. The other actor's continuation waits for a later task; if it had its +lock, it keeps holding it, so none of that actor's other events can run first. Its +implicit transaction still commits at the hand-off. It happens whenever another +actor's continuation ran earlier in the same task, with or without a call between +them. At build end the plugin reads the final Rollup module graph, after later transforms, and compares fully wrapped awaits with total awaits per included @@ -78,9 +102,12 @@ promise continuations that do not pass through syntax the transform can rewrite. requires wrapping `get`, `getAll`, `entries`, `values`, `forEach`, and both iteration protocols; no current consumer calls `formData()`, so that proxy is deferred rather than silently claiming the Files are covered. -2. **`WritableStream` seams** — none handed out by the runtime today, so no - hole yet; the moment an API returns one, `writer.write()` / `ready` / - `close()` need the same treatment. This row exists so that PR adds the seam. +2. **Actor-created `WritableStream` sinks** — actor-constructed stream callbacks + are in scope (the Gated row above), but only `ReadableStream` and + `TransformStream` are wrapped. A sink fed by native pipe machinery, such as + `gatedBody.pipeTo(new WritableStream({ write(chunk) { … } }))`, runs `write`, + `close` and `abort` with no input lock, so a storage call inside them throws. + The seam is the same constructor proxy, extended to those three callbacks. ## Fail-closed @@ -92,15 +119,18 @@ promise continuations that do not pass through syntax the transform can rewrite. ## Foreign by design For modules outside the transform, no runtime seam can exist for promises the -actor manufactures itself. The discipline: resolve them through -`ctx.awaitIo(...)`, or deliver events through `makeReentryCallback`. Provenance -(0.2.1) names the window when the discipline slips. +actor manufactures itself. The discipline: resolve them through the `awaitIo` +the host hands actor code — `actorScopeBindings(...).awaitIo`, or the host's own +wrapper over `container.awaitIo()`. Provenance (0.2.1) names the window when the +discipline slips. - `new Promise` resolved from an event: `MessagePort.onmessage`, `addEventListener`, `FileReader`, `AbortSignal` `"abort"`. -- User-constructed streams read outside a gated chain: `new ReadableStream`, - a `TransformStream` / `TextDecoderStream` / `CompressionStream` read directly - rather than via a gated body's `pipeThrough`. +- Direct reads of user-constructed streams outside a gated chain: a + `new ReadableStream`, `TransformStream`, `TextDecoderStream` or + `CompressionStream` read directly rather than via a gated body's + `pipeThrough`. An actor-created stream's callbacks are gated (the row above); + reads from it are not. - `AbortSignal.timeout()` — a platform timer; use `scheduler.wait` + an `AbortController` instead. - One-shot platform promises: dynamic `import()`, `WebAssembly.instantiate`, @@ -123,7 +153,7 @@ this one is absent rather than ungated). 1. **Same-PR rule**: any change that exposes a new async platform surface to actor code adds or moves a row here in the same commit, the way vendored - edits carry their `upstream-diff.md` entry. + edits carry their `vendor/agents/docs/fork-diff.md` row. 2. **Provenance is the tripwire**: every escape now reports the last gated site and the milliseconds elapsed — it points at the row to file. 3. **Dist audits find holes before production does**: grep consumer bundles for diff --git a/docs/migrations.md b/docs/migrations.md index 83e9ebf..d41bae2 100644 --- a/docs/migrations.md +++ b/docs/migrations.md @@ -134,7 +134,7 @@ only when no persisted state exists. Give the state an integer version and handle every shipped shape. The `migratePersistedState()` hook is currently supplied by the Rook Agents SDK fork; -it is not part of upstream Agents 0.22. +it is not part of upstream Agents 0.23. ```ts import { Agent } from "agents"; diff --git a/docs/workerd-sync.md b/docs/workerd-sync.md index 00ea559..e7af0de 100644 --- a/docs/workerd-sync.md +++ b/docs/workerd-sync.md @@ -1,6 +1,15 @@ # workerd sync: September 7, 2026 -The oracle and Workers types are pinned to `1.20260907.1` and `5.20260907.1`. +**September 11 re-pin.** The oracle is now pinned to `v1.20260911.1` +([`925464ba9fe5`](https://github.com/cloudflare/workerd/tree/925464ba9fe5751e4468626ce77f7a5810df274f)), +with Workers types `5.20260911.1` and `@sqlite.org/sqlite-wasm` `3.53.4-build1` +(SQLite 3.53.4), since 0.8.2. Conformance was re-validated on those pins, but the +[`v1.20260907.1...v1.20260911.1`](https://github.com/cloudflare/workerd/compare/v1.20260907.1...v1.20260911.1) +upstream range has not been audited the way this document audits the July range. +A direct probe on SQLite 3.53.4 reproduced the default-expression findings below, +so the [engine work](#sqlite-engine-work-still-required) is unchanged. + +The oracle and Workers types were pinned to `1.20260907.1` and `5.20260907.1`. This audit compares the source-comment baseline [`v1.20260713.1`](https://github.com/cloudflare/workerd/tree/03c396e9b14ea5644dfcfb696086d8df040a4efc) with [`v1.20260907.1`](https://github.com/cloudflare/workerd/tree/beb7bd5c370d898e5ea81947aaa80ba5f48cd47e). diff --git a/examples/extension/README.md b/examples/extension/README.md index 422e7b4..44ca5c5 100644 --- a/examples/extension/README.md +++ b/examples/extension/README.md @@ -56,10 +56,18 @@ popup.html ──sendMessage──▶ service worker ──chrome.offscreen.crea exponential backoff, abandonment — is rows rather than process memory, which is the divergence from workerd that exists precisely because MV3 evicts its contexts. -- **A physical MV3 wake.** The scheduler projects only its earliest durable wait - through the offscreen supervisor onto `chrome.alarms`. The e2e destroys the - offscreen document before that alarm is due and proves Chrome wakes the service - worker, recreates the host, and lets the scheduler deliver the stored event. +- **A physical MV3 wake.** The worker's `createBrowserAlarmProjector()` projects + the scheduler's earliest durable wait through the offscreen supervisor to the + service worker's `BrowserAlarmCoordinator`, which arms `chrome.alarms` and + journals each hop in `chrome.storage.local`. The projection generation lives in + a host table beside `_cf_ALARM`, because the coordinator ignores any generation + older than the last it journaled and a restarted worker must count on from + there. The worker boots its scheduler with no client attached. The e2e + destroys the offscreen document and sends no host operation until the + coordinator has acknowledged the wake: Chrome recreates the host, and the new + scheduler delivers the stored event on its own. The e2e also stops the service + worker in the middle of a held delivery; the coordinator's journaled watchdog + brings it back, and the delivery completes exactly once. - **The Agents SDK queue.** The e2e enqueues an increment and observes its state write through `snapshot()`, exercising the SDK's SQLite-backed queue rather than a host callback. @@ -70,10 +78,13 @@ popup.html ──sendMessage──▶ service worker ──chrome.offscreen.crea Chrome recreates an evicted host. - **A hibernatable `AgentClient` connection across container eviction.** The offscreen page opens the SDK client over a `MessagePort`-backed WebSocket, - while the actor receives the server half through `routeAgentRequest()` and - `ctx.acceptWebSocket()`. The e2e replaces only the root actor container and - proves that the same client receives a new state broadcast and writes state - back to the replacement without reconnecting. It also covers standard named + while the actor receives the server half through `ctx.acceptWebSocket()`. The + worker's `serveMessagePortWebSockets` connects each socket through + `connectMessagePortWebSocket()` and `routeAgentRequest()`, which close an + unrouted socket with 1011 and a refused upgrade with 1008 and the Agent's + reason. The e2e replaces only the root actor container and proves that the + same client receives a new state broadcast and writes state back to the + replacement without reconnecting. It also covers standard named routing, `getAgentByName()` direct stubs, a decorated `@callable()` method, and a streaming callable's chunks and final value. - **A real network relay boundary.** The e2e boots an authless, hibernatable @@ -90,11 +101,13 @@ popup.html ──sendMessage──▶ service worker ──chrome.offscreen.crea carries an in-memory `ForwardableEmailMessage` through `routeAgentEmail()` to the actor's `onEmail()` hook. Forwarding and replies still refuse because this host has no outbound email binding. -- **Offscreen corpse recovery.** A crashed offscreen document disappears from - `chrome.runtime.getContexts` while still holding the one offscreen slot. - `src/background.ts` catches the resulting "single offscreen document" error — - the error string is Chrome's only report of the corpse — closes it, and retries - once. +- **Offscreen corpse recovery and readiness.** A crashed offscreen document + disappears from `chrome.runtime.getContexts` while still holding the one + offscreen slot. The package's `OffscreenDocumentCoordinator` closes it and + retries creation once; `src/background.ts` supplies only the adapter, including + the "single offscreen document" substring that is Chrome's only report of the + corpse. The adapter's `ready()` pings the document until its listener answers, + so `ensure-host` resolves only when a `host-op` will be heard. ## Load it unpacked @@ -154,7 +167,7 @@ only as a competing supervisor and asserts that Web Locks refuse it before OPFS. | `src/protocol.ts` | The types both TypeScript projects compile. It imports nothing. | | `@mcp-b/do-runtime/browser` | The browser Request/`Response`-101 upgrade adapter; the runtime supplies `WebSocketPair`. | | `@mcp-b/do-runtime` `HibernationMirror` | The process-local `HibernationHost` record shared by this example and the conformance embedders. | -| `../platform-shims/message-port-websocket.ts` | The client-side WebSocket adapter carried over a `MessagePort`. | +| `@mcp-b/do-runtime/browser/message-port-websocket` | The WebSocket adapter carried over a `MessagePort`: the offscreen client and the worker's server side. | | `public/manifest.json` | Copied verbatim into `dist/` by Vite's `publicDir`. | | `wrangler.relay.jsonc` | Local-workerd configuration for the relay proof. | @@ -180,10 +193,13 @@ Every step is where it is because moving it was measured to fail. once, at bootstrap. Disabling the `opfs` and `opfs-wl` VFSes keeps the proxy workers this host does not use out of the picture; `opfs-sahpool` must stay enabled. -3. **Install sqlite and the pool *before* `installActorScope`.** - `installOpfsSAHPoolVfs` probes the other OPFS VFSes on the way in and those - probes arm watchdogs through the global `setTimeout`. Installing the actor - scope first hands the actor's gate to a storage library. +3. **Install sqlite and the pool with `installSqliteWasmHost`, *before* + `installActorScope`.** The helper waits for a terminated predecessor to + release the pool and makes SQLite roll back the transaction it left open. + The driver's `installOpfsSAHPoolVfs`, which it calls, probes the other OPFS + VFSes on the way in, and those probes arm watchdogs through the global + `setTimeout`. Installing the actor scope first hands the actor's gate to a + storage library. 4. **`installActorScope(globalThis, resolve)` where `resolve` throws.** One worker hosts one root, so "no container" cannot mean "outside any actor" — it can only mean the container was torn down mid-flight, and handing that @@ -267,10 +283,10 @@ substrate. Cloudflare-managed products remain explicit integration boundaries: Written down because this example exists partly to find them. - **The Agents SDK root entry eagerly imports Workers-only Node and email modules.** - Vite maps the Node imports through `unenv`; `cloudflare:email` remains a - fail-closed shim. The inbound test supplies a host-created - `ForwardableEmailMessage`; its forwarding and reply methods refuse because the - demo has no outbound Email Routing binding. + Vite maps the Node imports through `unenv`; `cloudflare:email` resolves to the + package's `EmailMessage` data constructor, which sends nothing. The inbound test + supplies a host-created `ForwardableEmailMessage`; its forwarding and reply + methods refuse because the demo has no outbound Email Routing binding. - **The hibernation mirror is not process durability.** It survives a container replacement inside this Worker. It cannot survive destruction of the Worker that owns the raw `MessagePort` socket; that lifecycle reconnects instead. @@ -286,20 +302,25 @@ Written down because this example exists partly to find them. though `sqlite3ApiConfig` turns those VFSes off. The `sqlite3.wasm` binary is emitted correctly with no plugin and no `locateFile` override, which is the good half of the same mechanism. -- **A failed pool install tries to delete the pool directory.** The second holder - above also logs `removeVfs() failed with no recovery strategy: … 'removeEntry' …`. - It fails, because the first holder has the directory open — but a cleanup path - that reaches for `removeEntry` on shared storage after a failed acquisition is - worth knowing about before it succeeds on some other platform. -- **Two `ensureOffscreen()` callers can close a healthy document.** Not the - runtime's, but a trap for anyone copying this shape: `onInstalled` and the +- **A failed pool install deletes the pool directory when it can.** The driver's + `installOpfsSAHPoolVfs` runs `removeVfs()` after a failed acquisition. Against a + live holder the delete fails (`removeVfs() failed with no recovery strategy: … + 'removeEntry' …`), but on Chrome it succeeds when the holder is a terminated + worker still releasing its handles: retrying the driver's install across that + release lost the whole pool in 3 of 24 measured recoveries. `installSqliteWasmHost` + waits until every file is released before installing, and after 10 s rejects + with a `NoModificationAllowedError` instead of reaching the driver's cleanup. +- **Two uncoordinated creators can close a healthy document.** A trap for anyone + who replaces the package's coordinator with their own: `onInstalled` and the popup's first message arrive together, both see no document, the loser gets - "single offscreen document", and the corpse-recovery path then closes the - winner's live document. `src/background.ts` serialises on an in-flight promise - for exactly this reason. + "single offscreen document", and corpse recovery then closes the winner's live + document. `OffscreenDocumentCoordinator.ensure()` single-flights creation and + recovery for exactly this reason. - **`chrome.offscreen.createDocument` can resolve before the document listens.** - A message sent right after it is answered `undefined` rather than queued, so - `src/popup/popup.ts` retries once — and only for that outcome. + `offscreen.ts` registers its listener behind a top-level `await`, and a message + sent in that gap is answered `undefined` rather than queued. The adapter's + `ready()` pings until the listener answers, inside the same single flight, and + replaces a document that stays mute past its timeout once. - **`chrome-types` models `chrome.offscreen.Reason` and `chrome.runtime.ContextType` as types, not runtime enums.** The dotted form Chrome's own docs use does not compile; string literals do. diff --git a/examples/extension/scripts/e2e.mjs b/examples/extension/scripts/e2e.mjs index 3cb39a1..9a9cd29 100644 --- a/examples/extension/scripts/e2e.mjs +++ b/examples/extension/scripts/e2e.mjs @@ -22,8 +22,15 @@ const repoRoot = fileURLToPath(new URL("../../../", import.meta.url)); const dist = `${example}dist`; const profile = `${example}.e2e-profile`; -/** How long to wait for the alarm to be delivered. It is armed for 2s. */ +/** How long to wait for an alarm to recreate the host or finish its delivery. */ const ALARM_TIMEOUT_MS = 15_000; +/** + * How long the held wake's handler runs. It covers the alarm's latency, the + * journal poll and the CDP calls that stop the service worker mid-delivery. + */ +const HELD_DELIVERY_MS = 10_000; +/** The coordinator's `chrome.storage.local` journal, owned by `src/background.ts`. */ +const WAKE_JOURNAL = "do-runtime-wake-journal"; /** How long to wait for the first op, which pays for wasm init and the OPFS pool. */ const BOOT_TIMEOUT_MS = 30_000; @@ -183,6 +190,49 @@ async function pollOp(page, name, args, predicate, timeoutMs = BOOT_TIMEOUT_MS) ); } +/** Poll a value read inside an extension context without sending the host anything. */ +async function pollEvaluate(target, read, predicate, timeoutMs, arg) { + const deadline = Date.now() + timeoutMs; + let value; + while (Date.now() < deadline) { + value = await target.evaluate(read, arg); + if (predicate(value)) return value; + await new Promise((resolve) => setTimeout(resolve, 100)); + } + throw new Error(`the extension did not reach the expected state: ${JSON.stringify(value)}`); +} + +async function offscreenDocuments() { + return (await chrome.runtime.getContexts({ contextTypes: ["OFFSCREEN_DOCUMENT"] })).length; +} + +async function wakeJournal(key) { + return (await chrome.storage.local.get(key))[key] ?? null; +} + +/** + * Stop the extension service worker the way Chrome's idle timeout does. Chrome + * keeps the worker's target across the restart, so Playwright's existing handle + * reaches the new instance and no new `serviceworker` event fires. + */ +async function stopServiceWorker(context, extensionId) { + const page = await context.newPage(); + const session = await context.newCDPSession(page); + try { + const { targetInfos } = await session.send("Target.getTargets"); + const target = targetInfos.find( + ({ type, url }) => + type === "service_worker" && url.startsWith(`chrome-extension://${extensionId}/`), + ); + if (target === undefined) throw new Error("the extension service worker target is missing"); + const { success } = await session.send("Target.closeTarget", { targetId: target.targetId }); + if (!success) throw new Error("Chrome did not stop the extension service worker"); + } finally { + await session.detach(); + await page.close(); + } +} + /** Poll the popup's output pane until it shows something matching `pattern`. */ async function waitForOutput(page, pattern, timeoutMs) { const deadline = Date.now() + timeoutMs; @@ -415,8 +465,10 @@ async function main() { ); // --------------------------------------------------------------------- - // 3. A real alarm: armed in the actor's storage, delivered by the - // AlarmScheduler's own database in the same worker. + // 3. A real alarm with only chrome.alarms to deliver it. The wake is armed + // in Agent storage, then the whole host is removed, and no host + // operation is sent until well past its time: the physical alarm must + // recreate the host, and the new Worker's scheduler must deliver it. const childArmedFor = await op(popup, "armSubAgentWake", [5000]); if (typeof childArmedFor !== "number") { fail("a sub-agent answers its scheduled time", String(childArmedFor)); @@ -441,19 +493,37 @@ async function main() { await worker.evaluate(async () => chrome.offscreen.closeDocument()); pass("the offscreen host was removed before the durable wake"); - const deadline = Date.now() + ALARM_TIMEOUT_MS; - let alarms = 0; - while (Date.now() < deadline) { - try { - snapshot = await op(popup, "snapshot"); - alarms = snapshot.events.filter((event) => event.kind === "sdk-schedule").length; - if (alarms > 0) break; - } catch { - // Expected until chrome.alarms wakes the service worker and it recreates the host. - } - await new Promise((resolve) => setTimeout(resolve, 250)); + try { + await pollEvaluate(worker, offscreenDocuments, (count) => count === 1, ALARM_TIMEOUT_MS); + pass("chrome.alarms recreated the offscreen host"); + } catch (error) { + fail("chrome.alarms recreated the offscreen host", error.message); + } + // Reading the journal from the service worker sends the host nothing. + const latestWake = Math.max(armedFor, childArmedFor); + try { + await pollEvaluate( + worker, + wakeJournal, + (journal) => + journal?.delivery === null && + (journal.projection.when === null || journal.projection.when > latestWake), + BOOT_TIMEOUT_MS, + WAKE_JOURNAL, + ); + pass("the coordinator acknowledged both wakes"); + } catch (error) { + fail("the coordinator acknowledged both wakes", error.message); } - check("chrome.alarms recreated the host and delivered the alarm", alarms, 1); + const firstOperationAt = Date.now(); + snapshot = await op(popup, "snapshot"); + const wakes = snapshot.events.filter((event) => event.kind === "sdk-schedule"); + check("the recreated host delivered the alarm", wakes.length, 1); + check( + "the alarm was delivered before any host operation", + wakes.length > 0 && wakes.every((event) => event.at < firstOperationAt), + true, + ); check("the alarm handler's write landed", snapshot.value, 21); check( "the recreated host delivered durable work to the sub-agent", @@ -724,6 +794,66 @@ async function main() { (await chrome.runtime.getContexts({ contextTypes: ["OFFSCREEN_DOCUMENT"] })).length, ); check("the service worker created one offscreen document", offscreenContexts, 1); + + // --------------------------------------------------------------------- + // 8. Chrome stops the service worker in the middle of an alarm delivery. + // The coordinator armed a watchdog before handing the wake over. Until + // the delivery is acknowledged the test only reads extension storage + // from the popup, which wakes nothing, so only the watchdog alarm can + // restart the coordinator's `fire()`, the one path that clears it. + const beforeHeld = await op(popup, "snapshot"); + await worker.evaluate(() => { + globalThis.__stoppedServiceWorker = true; + }); + const heldFor = await op(popup, "armWake", [2000, HELD_DELIVERY_MS]); + const delivering = await pollEvaluate( + popup, + wakeJournal, + (journal) => journal?.delivery != null, + ALARM_TIMEOUT_MS, + WAKE_JOURNAL, + ); + const watchdog = await popup.evaluate( + async () => (await chrome.alarms.get("do-runtime-wake"))?.scheduledTime ?? null, + ); + check( + "the coordinator armed its watchdog before delivering", + watchdog, + delivering.delivery.wake, + ); + await stopServiceWorker(context, extensionId); + // The running delivery stays projected at its alarm's own time, which the + // SDK rounds down to the second, so it bounds the hold better than heldFor. + check( + "the service worker stopped while the delivery was held", + Date.now() < delivering.projection.when + HELD_DELIVERY_MS, + true, + ); + try { + await pollEvaluate( + popup, + wakeJournal, + (journal) => journal?.delivery === null, + BOOT_TIMEOUT_MS, + WAKE_JOURNAL, + ); + pass("the watchdog completed the held delivery's acknowledgement"); + } catch (error) { + fail("the watchdog completed the held delivery's acknowledgement", error.message); + } + check( + "a restarted service worker acknowledged it", + await worker.evaluate(() => globalThis.__stoppedServiceWorker === undefined), + true, + ); + const afterHeld = await op(popup, "snapshot"); + check("the held alarm ran exactly once", afterHeld.value - beforeHeld.value, 1); + check( + "the held alarm recorded one event", + afterHeld.events.filter((event) => event.kind === "sdk-schedule" && event.at >= heldFor) + .length, + 1, + ); } catch (error) { fail("the run threw", error?.stack ?? String(error)); } finally { diff --git a/examples/extension/src/background.ts b/examples/extension/src/background.ts index 79c4d25..74eba0e 100644 --- a/examples/extension/src/background.ts +++ b/examples/extension/src/background.ts @@ -10,11 +10,28 @@ * for alarm identity, retry policy, and delivery. */ +import { + BrowserAlarmCoordinator, + parseBrowserAlarmProjection, + parseBrowserAlarmTransportJournal, +} from "@mcp-b/do-runtime/browser/alarm-coordinator"; import { OffscreenDocumentCoordinator } from "@mcp-b/do-runtime/browser/offscreen-document"; -import { WAKE_ALARM, type ExtensionResponse } from "./protocol"; +import { + parseExtensionResponse, + WAKE_ALARM, + type ExtensionMessage, + type ExtensionResponse, +} from "./protocol"; const OFFSCREEN_URL = "offscreen.html"; +/** The alarm coordinator's journal, which outlives every service worker. */ +const WAKE_JOURNAL = "do-runtime-wake-journal"; + +/** How long a document may take to answer its first ping before it is replaced. */ +const READY_TIMEOUT_MS = 10_000; +const READY_POLL_MS = 50; + const JUSTIFICATION = "Hosts the Durable Object runtime's actor worker, which needs OPFS synchronous access " + "handles and therefore a dedicated Worker that outlives the service worker."; @@ -40,9 +57,9 @@ const OFFSCREEN_CONTEXT: chrome.runtime.ContextType = "OFFSCREEN_DOCUMENT"; const OFFSCREEN_REASON: chrome.offscreen.Reason = "WORKERS"; /** - * The runtime coalesces concurrent creation and recovers Chrome's hidden, - * occupied offscreen slot. This adapter supplies only the Chrome operations and - * its string-only occupied-slot signal. + * The runtime coalesces concurrent creation, recovers Chrome's hidden, occupied + * offscreen slot, and waits for the document to answer. This adapter supplies + * only the Chrome operations, its string-only occupied-slot signal, and a ping. */ const offscreenDocument = new OffscreenDocumentCoordinator({ async exists() { @@ -60,28 +77,68 @@ const offscreenDocument = new OffscreenDocumentCoordinator({ }), async close() { console.warn( - "[do-runtime example] an offscreen document held the slot but was not listed; " + - "closing it and retrying once.", + "[do-runtime example] replacing an offscreen document that is unlisted or not answering.", ); - await chrome.offscreen.closeDocument(); + // A document that is already gone refuses to close; creating one is still right. + await chrome.offscreen.closeDocument().catch(() => {}); }, isOccupiedError: (error) => String(error).includes(SINGLE_DOCUMENT_ERROR), + /** + * `createDocument` resolves before `offscreen.ts` registers its listener + * behind a top-level await, and Chrome answers a message sent in that gap + * with nothing rather than queueing it. + */ + async ready() { + const deadline = Date.now() + READY_TIMEOUT_MS; + for (;;) { + const answer: unknown = await chrome.runtime + .sendMessage({ type: "host-ping" } satisfies ExtensionMessage) + .catch(() => undefined); + if (answer !== undefined) return; + if (Date.now() >= deadline) throw new Error("the offscreen document did not answer"); + await new Promise((resolve) => setTimeout(resolve, READY_POLL_MS)); + } + }, + /** A document still mute after the timeout is replaced once, not pinged forever. */ + replaceUnready: () => true, }); -export function ensureOffscreen(): Promise { - return offscreenDocument.ensure(); -} - -async function projectWake(scheduledTime: number | null): Promise { - if (scheduledTime === null) { - await chrome.alarms.clear(WAKE_ALARM); - return; - } - if (!Number.isSafeInteger(scheduledTime) || scheduledTime < 0) { - throw new TypeError("projected alarm time must be a non-negative safe integer"); - } - await chrome.alarms.create(WAKE_ALARM, { when: scheduledTime }); -} +/** + * The physical half of the worker's `AlarmScheduler`. The runtime journals each + * hop, so a service worker stopped mid-delivery leaves a watchdog that resumes + * it; this adapter supplies Chrome's alarm and storage calls and the delivery. + */ +const alarms = new BrowserAlarmCoordinator({ + // Recreate the host if Chrome removed it, then wait until its scheduler has + // finished everything due by the consumed wake. + async deliver(scheduledTime) { + await offscreenDocument.ensure(); + const response: unknown = await chrome.runtime.sendMessage({ + type: "host-op", + op: "fireAlarm", + args: [scheduledTime], + } satisfies ExtensionMessage); + const result = parseExtensionResponse(response); + if (!result.ok) throw new Error(result.error); + const projection = parseBrowserAlarmProjection(result.value); + if (projection === null) throw new TypeError("the host answered an invalid wake projection"); + return projection; + }, + physical: { + async clear() { + await chrome.alarms.clear(WAKE_ALARM); + }, + create: (when) => chrome.alarms.create(WAKE_ALARM, { when }), + }, + store: { + load: async () => + parseBrowserAlarmTransportJournal((await chrome.storage.local.get(WAKE_JOURNAL))[WAKE_JOURNAL]), + save: (journal) => chrome.storage.local.set({ [WAKE_JOURNAL]: journal }), + }, +}); +void alarms.reconcile().catch((error: unknown) => { + console.error("[do-runtime example] the alarm journal could not be reconciled:", error); +}); function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); @@ -89,10 +146,10 @@ function isRecord(value: unknown): value is Record { /** * `ensure-host` is the popup asking for a host before it sends any operation; - * `project-wake` mirrors the scheduler's earliest durable wait into Chrome. - * `host-op` messages are NOT answered here — the offscreen document receives - * them directly — so this listener returns `false` for them and lets the channel - * belong to whoever will actually reply. + * `project-wake` carries the scheduler's latest projection to the coordinator. + * `host-ping` and `host-op` messages are NOT answered here — the offscreen + * document receives them directly — so this listener returns `false` for them + * and lets the channel belong to whoever will actually reply. */ chrome.runtime.onMessage.addListener( ( @@ -105,11 +162,13 @@ chrome.runtime.onMessage.addListener( } let operation: Promise; if (message.type === "ensure-host") { - operation = ensureOffscreen(); - } else if (message.scheduledTime === null || typeof message.scheduledTime === "number") { - operation = projectWake(message.scheduledTime); + operation = offscreenDocument.ensure(); } else { - operation = Promise.reject(new TypeError("invalid projected wake message")); + const projection = parseBrowserAlarmProjection(message.projection); + operation = + projection === null + ? Promise.reject(new TypeError("invalid projected wake message")) + : alarms.project(projection); } void operation.then( () => { @@ -125,8 +184,8 @@ chrome.runtime.onMessage.addListener( chrome.alarms.onAlarm.addListener((alarm) => { if (alarm.name !== WAKE_ALARM) return; - void ensureOffscreen().catch((error: unknown) => { - console.error("[do-runtime example] alarm wake could not recreate the host:", error); + void alarms.fire(alarm.scheduledTime).catch((error: unknown) => { + console.error("[do-runtime example] the alarm wake was not delivered:", error); }); }); @@ -134,8 +193,8 @@ chrome.alarms.onAlarm.addListener((alarm) => { * Start the host on install and on browser startup, without waiting for a popup. */ chrome.runtime.onInstalled.addListener(() => { - void ensureOffscreen(); + void offscreenDocument.ensure(); }); chrome.runtime.onStartup.addListener(() => { - void ensureOffscreen(); + void offscreenDocument.ensure(); }); diff --git a/examples/extension/src/offscreen/offscreen.ts b/examples/extension/src/offscreen/offscreen.ts index 86a387a..0d4252a 100644 --- a/examples/extension/src/offscreen/offscreen.ts +++ b/examples/extension/src/offscreen/offscreen.ts @@ -44,6 +44,7 @@ import { type SupervisorRpc, type ThinkProbeStatus, type ThinkProbeSubmission, + type WakeProjection, type WorkerBoot, } from "../protocol"; @@ -98,16 +99,15 @@ worker.postMessage( * The actor worker uses the runtime's `newRpcSession`, which applies the * identity graft before it exposes its Workers targets. * - * No local main is passed because nothing calls back: this example's alarm - * scheduler lives in the actor's own worker, so the worker never needs to reach - * the supervisor. A host with more than one actor would pass a target here, the - * way the runtime's conformance page does. + * The worker calls back for one thing: its alarm scheduler lives in the actor's + * own worker, and each projected wake has to reach the service worker's alarm + * coordinator, because `chrome.alarms` is not exposed here. */ class SupervisorTarget extends RpcTarget implements SupervisorRpc { - async projectWake(scheduledTime: number | null): Promise { + async projectWake(projection: WakeProjection): Promise { const response: unknown = await chrome.runtime.sendMessage({ type: "project-wake", - scheduledTime, + projection, } satisfies ExtensionMessage); const result = parseExtensionResponse(response); if (!result.ok) throw new Error(result.error); @@ -351,7 +351,10 @@ const ops = { ): Promise => await host.submitThink(name, text, idempotencyKey), thinkStatus: async (name: string): Promise => await host.thinkStatus(name), stopThink: async (name: string): Promise => await host.stopThink(name), - armWake: async (delayMs: number): Promise => await host.armWake(delayMs), + armWake: async (delayMs: number, holdMs = 0): Promise => + await host.armWake(delayMs, holdMs), + fireAlarm: async (scheduledTime: number): Promise => + await host.fireAlarm(scheduledTime), status: async (): Promise => await host.status(), sdkIncrement: async (): Promise => await (await connectedAgent()).call("increment"), sdkSetLegacyState: async (value: number): Promise => { @@ -446,7 +449,9 @@ async function runOp(op: HostOp, args: readonly unknown[]): Promise { case "stopThink": return await ops.stopThink(stringArg(args, 0)); case "armWake": - return await ops.armWake(integerArg(args, 0)); + return await ops.armWake(integerArg(args, 0), args.length > 1 ? integerArg(args, 1) : 0); + case "fireAlarm": + return await ops.fireAlarm(integerArg(args, 0)); case "status": return await ops.status(); case "sdkIncrement": @@ -497,6 +502,9 @@ window.__host = ops; * `return true` keeps `sendResponse` alive across the await, and it is returned * ONLY for a message this listener will actually answer — a listener that claims * every message holds the channel open for messages meant for someone else. + * + * `host-ping` is the service worker's readiness probe: registering this listener + * is the last thing the document does, so an answer means it can take `host-op`s. */ if (typeof chrome !== "undefined" && chrome.runtime?.id !== undefined) { chrome.runtime.onMessage.addListener( @@ -505,6 +513,10 @@ if (typeof chrome !== "undefined" && chrome.runtime?.id !== undefined) { _sender: chrome.runtime.MessageSender, sendResponse: (response: ExtensionResponse) => void, ): boolean => { + if (isRecord(message) && message.type === "host-ping") { + sendResponse({ ok: true, value: null }); + return false; + } if (!isRecord(message) || message.type !== "host-op") return false; const operation = isHostOp(message.op) && Array.isArray(message.args) diff --git a/examples/extension/src/popup/popup.ts b/examples/extension/src/popup/popup.ts index cbd3f2b..c2b2680 100644 --- a/examples/extension/src/popup/popup.ts +++ b/examples/extension/src/popup/popup.ts @@ -8,12 +8,7 @@ * closes. */ -import { - parseExtensionResponse, - type ExtensionMessage, - type ExtensionResponse, - type HostOp, -} from "../protocol"; +import { parseExtensionResponse, type ExtensionMessage, type HostOp } from "../protocol"; function mustFind(selector: string): T { const element = document.querySelector(selector); @@ -33,35 +28,18 @@ function print(label: string, value: unknown): void { * `undefined` — the raw value, not a result — is how Chrome says "no listener * claimed this message". Every other answer is an `ExtensionResponse`, because * `sendResponse` cannot reject and the two sides settle rather than throw. - */ -async function sendOnce(message: ExtensionMessage): Promise { - const response: unknown = await chrome.runtime.sendMessage(message); - return response === undefined ? undefined : parseExtensionResponse(response); -} - -/** - * One retry, and only for "nobody answered". * - * `chrome.offscreen.createDocument` resolves when the document has loaded, but - * an extension message sent immediately after can still arrive before that - * page's `onMessage` listener is registered, and Chrome answers `undefined` - * rather than queueing. Measured: the popup's first operation failed this way - * while every later one succeeded. - * - * A real error from the other side is NOT retried — it comes straight out of - * here, because retrying a call that failed on its merits would only hide it. + * Nothing is retried: `ensure-host` resolves only once the offscreen document + * answers, so a `host-op` sent after it always has a listener. */ async function send(message: ExtensionMessage): Promise { - let response = await sendOnce(message); - if (response === undefined) { - await new Promise((resolve) => setTimeout(resolve, 100)); - response = await sendOnce(message); - } + const response: unknown = await chrome.runtime.sendMessage(message); if (response === undefined) { throw new Error("no extension context answered; is the offscreen document up?"); } - if (!response.ok) throw new Error(response.error); - return response.value; + const result = parseExtensionResponse(response); + if (!result.ok) throw new Error(result.error); + return result.value; } /** diff --git a/examples/extension/src/protocol.ts b/examples/extension/src/protocol.ts index f24400a..a6c752a 100644 --- a/examples/extension/src/protocol.ts +++ b/examples/extension/src/protocol.ts @@ -21,9 +21,12 @@ export const WAKE_ALARM = "do-runtime-wake"; /** The authless relay fixture tells its host when an Agents client is paired. */ export const RELAY_CLIENT_READY = "do-runtime:relay-client-ready"; +/** A `BrowserAlarmProjection`: the scheduler's earliest wake and its durable generation. */ +export type WakeProjection = { readonly generation: number; readonly when: number | null }; + /** What the actor worker can ask its offscreen supervisor to project. */ export interface SupervisorRpc { - projectWake(scheduledTime: number | null): Promise; + projectWake(projection: WakeProjection): Promise; } /** The state shape the real Agents client receives over its socket. */ @@ -142,7 +145,10 @@ export interface HostRpc { submitThink(name: string, text: string, idempotencyKey: string): Promise; thinkStatus(name: string): Promise; stopThink(name: string): Promise; - armWake(delayMs: number): Promise; + /** `holdMs` keeps the wake's handler running, so a test can interrupt its delivery. */ + armWake(delayMs: number, holdMs?: number): Promise; + /** Resolves once the scheduler has finished every wake due by `scheduledTime`. */ + fireAlarm(scheduledTime: number): Promise; status(): Promise; } @@ -160,15 +166,17 @@ export type HostOp = /** * `chrome.runtime.sendMessage` payloads. * - * `ensure-host` is answered by the service worker; `host-op` is answered by the - * offscreen document, which receives extension messages directly. They are one - * union because both travel the same channel and every listener has to be able - * to say "not mine" — a listener that returns `true` for a message it will never - * answer holds `sendResponse` open until the channel closes. + * `ensure-host` and `project-wake` are answered by the service worker; + * `host-ping` and `host-op` by the offscreen document, which receives extension + * messages directly. They are one union because all of them travel the same + * channel and every listener has to be able to say "not mine" — a listener that + * returns `true` for a message it will never answer holds `sendResponse` open + * until the channel closes. */ export type ExtensionMessage = | { readonly type: "ensure-host" } - | { readonly type: "project-wake"; readonly scheduledTime: number | null } + | { readonly type: "project-wake"; readonly projection: WakeProjection } + | { readonly type: "host-ping" } | { readonly type: "host-op"; readonly op: HostOp; readonly args: readonly unknown[] }; /** Every answer is a settled result rather than a throw: `sendResponse` cannot reject. */ diff --git a/examples/extension/src/worker/actor.worker.ts b/examples/extension/src/worker/actor.worker.ts index f43dda6..ad21ca3 100644 --- a/examples/extension/src/worker/actor.worker.ts +++ b/examples/extension/src/worker/actor.worker.ts @@ -45,6 +45,7 @@ import { } from "@mcp-b/do-runtime"; import { createSqliteWasmProvider, + installSqliteWasmHost, SqliteWasmActorStorage, type SqliteWasmHost, } from "@mcp-b/do-runtime/backends/sqlite-wasm"; @@ -52,11 +53,11 @@ import sqlite3InitModule from "@sqlite.org/sqlite-wasm"; import { getAgentByName, routeAgentEmail, routeAgentRequest } from "agents"; import { RpcTarget } from "cloudflare:workers"; import { + connectMessagePortWebSocket, installWebSocketUpgradeGlobals, - upgradeWebSocket, withWebSocketUpgrade, - type UpgradeWebSocket, } from "@mcp-b/do-runtime/browser"; +import { createBrowserAlarmProjector } from "@mcp-b/do-runtime/browser/alarm-coordinator"; import { serveMessagePortWebSockets } from "@mcp-b/do-runtime/browser/message-port-websocket"; import type { CounterSnapshot, @@ -67,6 +68,7 @@ import type { SupervisorRpc, ThinkProbeStatus, ThinkProbeSubmission, + WakeProjection, WorkerBoot, } from "../protocol"; import { Counter, type CounterEnv } from "./counter"; @@ -439,6 +441,8 @@ type Substrate = { readonly host: SqliteWasmHost; /** The namespace's one scheduler. It owns `_cf_ALARM`, the retry ladder, and delivery. */ readonly scheduler: AlarmScheduler; + /** Resolves a consumed Chrome wake once the scheduler has finished the work due by then. */ + readonly acknowledgeWake: (scheduledTime: number) => Promise; }; let substrate: Promise | undefined; @@ -475,8 +479,12 @@ async function installSubstrate(): Promise { // the actor's gate to a storage library. Measured on the runtime's own browser // lane, that produced `Ignoring inability to install the … sqlite3_vfs` // warnings carrying the actor scope's refusal text. + // + // Through `installSqliteWasmHost`, never the driver directly: it waits for a + // terminated predecessor to release the pool, and makes SQLite roll back the + // transaction that predecessor left open. const sqlite3 = await sqlite3InitModule(); - const pool = await sqlite3.installOpfsSAHPoolVfs({ + const host: SqliteWasmHost = await installSqliteWasmHost(sqlite3, { name: POOL_NAME, // NOT the conformance lane's `true`. That lane wipes the pool so a stale // browser profile cannot make a run pass or fail; an extension wiping its @@ -484,7 +492,6 @@ async function installSubstrate(): Promise { clearOnInit: false, initialCapacity: POOL_CAPACITY, }); - const host: SqliteWasmHost = { pool, capi: sqlite3.capi }; // --------------------------------------------------------------------------------- // 3. Now install the actor scope: gated `setTimeout`, `clearTimeout`, @@ -512,22 +519,44 @@ async function installSubstrate(): Promise { // running. That is upstream's `getActorContainer(id)` contract and it is the // whole point of an alarm: a wake is a reason to start a Durable Object, not // something that requires one to be started already. + // + // Each wake reaches the service worker's `BrowserAlarmCoordinator` with a + // generation, and the coordinator ignores any older than the last it + // journaled. So the counter must outlive this worker: it is this host's own + // table, beside the runtime's `_cf_ALARM` in the scheduler's database. + const db = await createSqliteWasmProvider(host, { prefix: ALARM_PREFIX }).open(ALARM_DATABASE); + db.exec( + "CREATE TABLE IF NOT EXISTS wake_generation " + + "(id INTEGER PRIMARY KEY CHECK (id = 1), generation INTEGER NOT NULL)", + [], + ); + const wakes = createBrowserAlarmProjector({ + nextGeneration: () => + Number( + db.exec( + "INSERT INTO wake_generation VALUES (1, 1) " + + "ON CONFLICT (id) DO UPDATE SET generation = generation + 1 RETURNING generation", + [], + ).rawRows[0]?.[0], + ), + project: async (projection) => { + if (peer === undefined) throw new Error("cannot project an alarm before the supervisor connects"); + await peer.projectWake(projection); + }, + }); const scheduler = new AlarmScheduler({ timer, - db: await createSqliteWasmProvider(host, { prefix: ALARM_PREFIX }).open(ALARM_DATABASE), + db, getActor: () => ({ deliverAlarm: async (scheduledTime: number, retryCount: number): Promise => await (await placed()).container.deliverAlarm(scheduledTime, retryCount), abandonAlarm: async (scheduledTime: number): Promise => await (await placed()).container.abandonAlarm(scheduledTime), }), - projectWake: async (scheduledTime) => { - if (peer === undefined) throw new Error("cannot project an alarm before the supervisor connects"); - await peer.projectWake(scheduledTime); - }, + projectWake: wakes.projectWake, }); - return { host, scheduler }; + return { host, scheduler, acknowledgeWake: wakes.acknowledge }; } // ======================================================================================= @@ -819,8 +848,17 @@ class HostTarget extends RpcTarget implements HostRpc { await (await placed()).entry.stopThink(name); } - async armWake(delayMs: number): Promise { - return await (await placed()).entry.armWake(delayMs); + async armWake(delayMs: number, holdMs = 0): Promise { + return await (await placed()).entry.armWake(delayMs, holdMs); + } + + /** + * The service worker's delivery of a consumed Chrome wake. The scheduler + * delivers on its own timer from the moment this worker boots; this answers + * once that work is done, with the next wake to arm. + */ + async fireAlarm(scheduledTime: number): Promise { + return await (await installedSubstrate()).acknowledgeWake(scheduledTime); } /** @@ -851,22 +889,6 @@ class HostTarget extends RpcTarget implements HostRpc { /** Held so the session is not collected while the worker lives. */ let peer: ReturnType> | undefined; -async function connectAgentSocket(url: string): Promise { - await placed(); - const request = withWebSocketUpgrade(new Request(url.replace(/^ws/, "http"))); - const response = await routeAgentRequest( - request, - { Counter: rootNamespace }, - { onBeforeConnect: withWebSocketUpgrade }, - ); - if (response == null) throw new Error(`No Agent route matched ${request.url}`); - const socket = upgradeWebSocket(response); - if (response.status !== 101 || socket === undefined) { - throw new Error(`Agent WebSocket upgrade failed with ${response.status}`); - } - return socket; -} - function isWorkerBoot(value: unknown): value is WorkerBoot { return ( typeof value === "object" && @@ -887,5 +909,21 @@ self.addEventListener("message", (event: MessageEvent) => { // directly: it applies the `RpcTarget` prototype graft that makes the class // above recognisable to capnweb, immediately before opening the session. peer = newRpcSession(event.data.port, new HostTarget()); - serveMessagePortWebSockets(event.data.sockets, connectAgentSocket); + // The runtime turns a missing route or a refused upgrade into the close the + // client reads; the Agent's own refusal text becomes a 1008 reason. + serveMessagePortWebSockets(event.data.sockets, (bridge, url) => + connectMessagePortWebSocket(bridge, url, async (request) => { + await placed(); + return await routeAgentRequest( + request, + { Counter: rootNamespace }, + { onBeforeConnect: withWebSocketUpgrade }, + ); + }), + ); + // Start the scheduler now, not at the first operation: a worker recreated by + // a Chrome wake has no client, and its due alarm must still be delivered. + void installedSubstrate().catch((error: unknown) => { + console.error("[do-runtime example] the alarm scheduler could not start:", error); + }); }); diff --git a/examples/extension/src/worker/counter.ts b/examples/extension/src/worker/counter.ts index 9d6831c..f3acb34 100644 --- a/examples/extension/src/worker/counter.ts +++ b/examples/extension/src/worker/counter.ts @@ -245,11 +245,14 @@ export class Counter extends Agent { * physical Durable Object alarm. The storage engine then tells the host's * alarm outlet (`ports.alarms`) before the local commit lands. The retry * ladder, backoff and abandonment are the host `AlarmScheduler`'s. + * + * `holdMs` keeps the wake's handler running that long, so the e2e can stop + * the service worker in the middle of a delivery. */ - async armWake(delayMs: number): Promise { + async armWake(delayMs: number, holdMs = 0): Promise { this.#schema(); const at = Date.now() + Math.max(0, delayMs); - await this.schedule(new Date(at), "scheduledIncrement"); + await this.schedule(new Date(at), "scheduledIncrement", { holdMs }); return at; } @@ -262,7 +265,9 @@ export class Counter extends Agent { * across a service-worker eviction, because the ladder is rows rather than * process memory. */ - async scheduledIncrement(): Promise { + async scheduledIncrement(payload?: { readonly holdMs?: number }): Promise { + const holdMs = payload?.holdMs ?? 0; + if (holdMs > 0) await scheduler.wait(holdMs); this.#schema(); this.#record("sdk-schedule"); this.setState({ ...this.state, value: this.state.value + 1 }); diff --git a/examples/extension/vite.config.ts b/examples/extension/vite.config.ts index 222fea7..dcca948 100644 --- a/examples/extension/vite.config.ts +++ b/examples/extension/vite.config.ts @@ -1,6 +1,6 @@ import { createRequire } from "node:module"; import { fileURLToPath } from "node:url"; -import { doRuntimeAwaitTransform } from "@mcp-b/do-runtime/vite"; +import { doRuntimeAwaitTransform, facetScopeBanner } from "@mcp-b/do-runtime/vite"; import agents from "agents/vite"; import { defaultClientConditions, defineConfig } from "vite"; @@ -30,10 +30,7 @@ const actorPlugins = () => [ const facetBanner = (chunk: { name: string }): string => chunk.name === "counter-child" || chunk.name === "think-probe" - ? `const __facetKey = new URL(import.meta.url).searchParams.get("scope"); -const __facetScope = globalThis.__doRuntimeExtensionFacetScopes?.[__facetKey]; -if (__facetScope === undefined) throw new Error(\`facet module has no scope named \${__facetKey}\`); -const { scheduler, setTimeout, clearTimeout, setInterval, clearInterval, fetch, crypto } = __facetScope;` + ? facetScopeBanner({ registry: "__doRuntimeExtensionFacetScopes" }) : ""; export default defineConfig(({ mode }) => ({ @@ -116,8 +113,6 @@ export default defineConfig(({ mode }) => ({ resolve: { conditions: ["worker", ...defaultClientConditions], alias: { - "@mcp-b/do-runtime/browser/async-hooks": `${packageRoot}dist/browser/async-hooks.js`, - "@mcp-b/do-runtime/gate": `${packageRoot}dist/gate.js`, /** * The specifier a Workers module imports `DurableObject` and `RpcTarget` * from. No browser resolves it, so the host supplies it — exactly as @@ -130,7 +125,7 @@ export default defineConfig(({ mode }) => ({ * class beside the package build, and capnweb would refuse its instances. */ "cloudflare:workers": cloudflareWorkersModule, - "cloudflare:email": `${packageRoot}examples/platform-shims/cloudflare-email.ts`, + "cloudflare:email": `${packageRoot}dist/cloudflare-email.js`, ...(mode === "think-probe" ? { "@cloudflare/shell": cloudflareShellModule, diff --git a/examples/platform-shims/cloudflare-email.ts b/examples/platform-shims/cloudflare-email.ts deleted file mode 100644 index bdb2e4c..0000000 --- a/examples/platform-shims/cloudflare-email.ts +++ /dev/null @@ -1,6 +0,0 @@ -/** `agents` imports this eagerly; email delivery is outside these browser demos. */ -export class EmailMessage { - constructor() { - throw new Error("cloudflare:email is not available in this browser host."); - } -} diff --git a/examples/vibe-platform/README.md b/examples/vibe-platform/README.md index b6d2057..1d04866 100644 --- a/examples/vibe-platform/README.md +++ b/examples/vibe-platform/README.md @@ -57,7 +57,8 @@ The workspace starter is also seeded exactly once, under boot semantics. dependency. `server/*` and `src/*` are byte-for-byte the workspace rows; generated `worker.ts` routes `/api/*` to the Durable Object and other requests to the built front-end assets. `wrangler.jsonc` contains the Durable Object binding, `new_sqlite_classes` migration, assets configuration, and the -Agents SDK's `nodejs_compat` flag; `package.json` pins the same Agents SDK version tested here. +Agents SDK's `nodejs_compat` flag; `package.json` pins the Agents SDK release the vendored fork +tested here carries, and the e2e fails if the two differ. ## Shape @@ -99,9 +100,10 @@ set the headers directly. **One tab at a time.** Each OPFS SAH pool takes exclusive sync access handles — that exclusivity is what makes SQLite synchronous here — so a second tab cannot install the workspace or user-agent -pool. An agent edit closes SQLite and pauses its VFS before replacing the worker; both installers -still retry for reloads and crashes, where the old worker cannot acknowledge release. Close the -other tab and reload; measured, it recovers. +pool. An agent edit closes SQLite and pauses its VFS before replacing the worker; for reloads and +crashes, where the old worker cannot acknowledge release, both installers go through +`installSqliteWasmHost`, which waits up to 10 seconds for the pool to be released and then reports +it locked. Close the other tab and reload; measured, it recovers. **The preview needs the network.** React comes from esm.sh at preview time. Offline, the bundle still builds and the workspace still saves and persists — the iframe just renders nothing. The e2e @@ -109,9 +111,11 @@ detects this and prints `SKIP` for the three steps that need a rendered React ap failing; `VIBE_E2E_OFFLINE=1 node scripts/e2e.mjs` takes that path on purpose. **This is the Agents SDK's HTTP state path, not its whole platform.** The SDK eagerly imports Node -and email modules, so Vite maps the Node imports through `unenv` and a fail-closed email shim. The -starter disables Agent WebSocket hibernation because every source edit deliberately terminates the -Worker and its local transport; an in-Worker mirror would disappear with both. The [MV3 extension +and email modules, so Vite maps the Node imports through `unenv` and `cloudflare:email` to the +package's data-only export. The +starter keeps the SDK's default hibernatable sockets, but this host mirrors none of them +(`ports.hibernation`): every source edit deliberately terminates the Worker and its local transport, +and an in-Worker mirror would disappear with both. The [MV3 extension example](../extension/README.md) is the broader compatibility harness: it keeps the transport alive across container eviction and runs the SDK client, hibernatable socket server, bidirectional state sync, callable and streaming RPC, the SDK queue and scheduler, stateless MCP, and inbound email diff --git a/examples/vibe-platform/scripts/e2e.mjs b/examples/vibe-platform/scripts/e2e.mjs index 0f71f69..b022525 100644 --- a/examples/vibe-platform/scripts/e2e.mjs +++ b/examples/vibe-platform/scripts/e2e.mjs @@ -126,7 +126,7 @@ if (forcedOffline) await page.route("https://esm.sh/**", (route) => route.abort( const pageErrors = []; page.on("pageerror", (error) => pageErrors.push(String(error))); page.on("console", (message) => { - if (message.type() === "error") pageErrors.push(message.text()); + if (message.type() === "error") pageErrors.push(`${message.text()} (${message.location().url})`); }); const preview = () => page.frameLocator("#preview"); @@ -383,8 +383,11 @@ try { const exportedPackage = JSON.parse( await readFile(path.join(exportDirectory, "package.json"), "utf8"), ); - if (exportedPackage.dependencies?.agents !== "0.22.0") { - throw new Error("exported package.json does not pin agents@0.22.0"); + const tested = JSON.parse( + await readFile(path.join(root, "node_modules/agents/package.json"), "utf8"), + ).version; + if (exportedPackage.dependencies?.agents !== tested) { + throw new Error(`exported package.json does not pin the tested agents@${tested}`); } await symlink(path.join(root, "node_modules"), path.join(exportDirectory, "node_modules")); const result = await execFileAsync( @@ -403,6 +406,14 @@ try { console.log(` ${transcript.split("\n").slice(-4).join("\n ")}`); }); + await step("the page raised no errors beyond the ones the steps cause on purpose", async () => { + // The deliberate syntax and constructor failures reach the UI log, not the console. Offline, + // the preview's React imports from esm.sh fail, and nothing else may. + const expected = online ? [] : [/^Failed to load resource: .*\(https:\/\/esm\.sh\//]; + const unexpected = pageErrors.filter((error) => !expected.some((pattern) => pattern.test(error))); + if (unexpected.length > 0) throw new Error(unexpected.join("\n")); + }); + if (failures > 0) { console.log("\n--- page log ---"); console.log(await page.locator("#log").innerText()); diff --git a/examples/vibe-platform/src/main.ts b/examples/vibe-platform/src/main.ts index 883f913..33fd323 100644 --- a/examples/vibe-platform/src/main.ts +++ b/examples/vibe-platform/src/main.ts @@ -273,7 +273,7 @@ async function stopAgent(): Promise { async function restartAgent(initial = false): Promise { // A normal replacement closes SQLite and pauses the VFS before termination. - // A crashed/reloaded worker cannot acknowledge teardown, so installPool still retries. + // A crashed/reloaded worker cannot acknowledge teardown, so installPool waits for its release. await stopAgent(); const source = await bundleWorkspace( AGENT_ENTRY, @@ -564,7 +564,8 @@ Run \`pnpm exec wrangler deploy\` when you are ready to deploy. private: true, type: "module", scripts: { deploy: "wrangler deploy", "deploy:dry": "wrangler deploy --dry-run" }, - dependencies: { agents: "0.22.0" }, + // The vendored fork's version; scripts/e2e.mjs checks it against the installed package. + dependencies: { agents: "0.23.0" }, devDependencies: { wrangler: "^4.114.0" }, }, null, diff --git a/examples/vibe-platform/src/worker/agent.worker.ts b/examples/vibe-platform/src/worker/agent.worker.ts index 731623a..35f4603 100644 --- a/examples/vibe-platform/src/worker/agent.worker.ts +++ b/examples/vibe-platform/src/worker/agent.worker.ts @@ -14,6 +14,7 @@ import { type Timer, } from "@mcp-b/do-runtime"; import { + installSqliteWasmHost, SqliteWasmActorStorage, type SqliteWasmHost, } from "@mcp-b/do-runtime/backends/sqlite-wasm"; @@ -29,7 +30,6 @@ import { type WireRequest, type WireResponse, } from "../wire"; -import type { RetryablePoolOptions } from "./sqlite-storage"; // Stable forever: changing either value orphans the authored actor's data. const UNIQUE_KEY = "do-runtime-example-vibe-user-agent"; @@ -59,22 +59,18 @@ async function installPool(): Promise { value: { disable: { vfs: { opfs: true, "opfs-wl": true } } }, }); const sqlite3 = await sqlite3InitModule(); - const options: RetryablePoolOptions = { - name: POOL_NAME, - clearOnInit: false, - initialCapacity: 8, - forceReinitIfPreviouslyFailed: true, - }; - - for (let attempt = 1; ; attempt += 1) { - try { - const pool = await sqlite3.installOpfsSAHPoolVfs(options); - return { pool, capi: sqlite3.capi }; - } catch (error) { - if (attempt === 1) report("the agent storage is still releasing; waiting", false); - if (attempt >= 20) throw new Error("The user agent's storage stayed locked.", { cause: error }); - await new Promise((resolve) => rawSetTimeout(resolve, 150)); + try { + // Waits up to 10 s for a terminated predecessor to release the pool. + return await installSqliteWasmHost(sqlite3, { + name: POOL_NAME, + clearOnInit: false, + initialCapacity: 8, + }); + } catch (error) { + if (error instanceof DOMException && error.name === "NoModificationAllowedError") { + throw new Error("The user agent's storage stayed locked.", { cause: error }); } + throw error; } } diff --git a/examples/vibe-platform/src/worker/host.worker.ts b/examples/vibe-platform/src/worker/host.worker.ts index 76dd971..1998360 100644 --- a/examples/vibe-platform/src/worker/host.worker.ts +++ b/examples/vibe-platform/src/worker/host.worker.ts @@ -48,6 +48,7 @@ import { type Timer, } from "@mcp-b/do-runtime"; import { + installSqliteWasmHost, SqliteWasmActorStorage, type SqliteWasmHost, } from "@mcp-b/do-runtime/backends/sqlite-wasm"; @@ -61,7 +62,6 @@ import { type WorkspaceRpc, } from "../wire"; import { Workspace, type WorkspaceEnv } from "./workspace"; -import type { RetryablePoolOptions } from "./sqlite-storage"; // --------------------------------------------------------------------------- // Names that must never change @@ -110,20 +110,6 @@ const timer: Timer = { // --------------------------------------------------------------------------- // 2 + 3. sqlite and the pool, before any actor scope exists -const sleep = (ms: number): Promise => - new Promise((resolve) => rawSetTimeout(resolve, ms)); - -/** - * The driver's option bag, plus the one option it implements and does not - * declare. Without `forceReinitIfPreviouslyFailed` the driver caches the - * REJECTED promise under this pool name (`installOpfsSAHPoolVfs`, in - * `dist/index.mjs`), so every retry below would return the first failure - * forever. - */ -/** How long to keep trying for the pool before telling the user it is locked. */ -const POOL_ATTEMPTS = 20; -const POOL_RETRY_MS = 150; - async function installPool(): Promise { // The driver reads this once at bootstrap and then `delete`s it, which is why // the property has to be configurable — a plain non-configurable definition @@ -145,38 +131,28 @@ async function installPool(): Promise { const sqlite3 = await sqlite3InitModule(); - const options: RetryablePoolOptions = { - name: POOL_NAME, - // NOT the conformance lane's `true`. That lane wants a pristine profile on - // every run; this one is a workspace, and clearing on init would delete the - // user's files on every page load. - clearOnInit: false, - // Two databases per root actor — its own storage and the facet-tree index — - // plus a rollback journal beside each, is four files. Eight is that with - // headroom; the pool cannot grow past its capacity without an explicit - // `reserveMinimumCapacity`, and running out surfaces as `SQLITE_CANTOPEN` - // from whichever open happens to be unlucky. - initialCapacity: 8, - forceReinitIfPreviouslyFailed: true, - }; - - for (let attempt = 1; ; attempt += 1) { - try { - const pool = await sqlite3.installOpfsSAHPoolVfs(options); - return { pool, capi: sqlite3.capi }; - } catch (error) { - // Two situations produce this failure and only one is worth waiting out. - // A page RELOAD leaves the previous worker's sync access handles held - // until the browser gets round to releasing them, and there is no signal - // for that, so retrying is the only strategy available. (Measured on - // Chromium, 2026-08: a reload never actually needed a retry. This is for - // when it does, and it costs nothing when it does not.) A second TAB holds - // the handles for as long as it stays open, and no amount of retrying will - // help — so the loop ends by naming that case. - if (attempt === 1) report("the workspace storage is locked; waiting for it", false); - if (attempt >= POOL_ATTEMPTS) throw new Error(WORKSPACE_LOCKED_MESSAGE, { cause: error }); - await sleep(POOL_RETRY_MS); + try { + return await installSqliteWasmHost(sqlite3, { + name: POOL_NAME, + // NOT the conformance lane's `true`. That lane wants a pristine profile on + // every run; this one is a workspace, and clearing on init would delete the + // user's files on every page load. + clearOnInit: false, + // Two databases per root actor — its own storage and the facet-tree index — + // plus a rollback journal beside each, is four files. Eight is that with + // headroom; the pool cannot grow past its capacity without an explicit + // `reserveMinimumCapacity`, and running out surfaces as `SQLITE_CANTOPEN` + // from whichever open happens to be unlucky. + initialCapacity: 8, + }); + } catch (error) { + // The helper has already waited 10 s for a reloaded or crashed worker to + // release the pool. A second TAB holds it for as long as it stays open, and + // no amount of waiting helps, so name that case. + if (error instanceof DOMException && error.name === "NoModificationAllowedError") { + throw new Error(WORKSPACE_LOCKED_MESSAGE, { cause: error }); } + throw error; } } diff --git a/examples/vibe-platform/src/worker/sqlite-storage.ts b/examples/vibe-platform/src/worker/sqlite-storage.ts deleted file mode 100644 index 6290433..0000000 --- a/examples/vibe-platform/src/worker/sqlite-storage.ts +++ /dev/null @@ -1,6 +0,0 @@ -import type { Sqlite3Static } from "@sqlite.org/sqlite-wasm"; - -/** sqlite-wasm documents and implements the retry flag but omits it from its declaration. */ -export type RetryablePoolOptions = Parameters[0] & { - forceReinitIfPreviouslyFailed?: boolean; -}; diff --git a/examples/vibe-platform/vite.config.ts b/examples/vibe-platform/vite.config.ts index bda4442..b7dfe74 100644 --- a/examples/vibe-platform/vite.config.ts +++ b/examples/vibe-platform/vite.config.ts @@ -36,7 +36,7 @@ export default defineConfig({ // one DurableObject/RpcTarget identity, while authored source keeps the // exact platform specifier it will deploy with. "cloudflare:workers": cloudflareWorkersModule, - "cloudflare:email": `${repoRoot}examples/platform-shims/cloudflare-email.ts`, + "cloudflare:email": `${repoRoot}dist/cloudflare-email.js`, "node:async_hooks": "unenv/node/async_hooks", "node:diagnostics_channel": "unenv/node/diagnostics_channel", "node:os": "unenv/node/os", @@ -53,7 +53,7 @@ export default defineConfig({ server: { headers: crossOriginIsolation, fs: { - // The platform shims above live outside this example's root. + // The package's `dist/` files above live outside this example's root. allow: [repoRoot], }, }, diff --git a/package.json b/package.json index 04bfeef..c05d7f8 100644 --- a/package.json +++ b/package.json @@ -76,6 +76,10 @@ "types": "./dist/cloudflare-workers.d.ts", "import": "./dist/cloudflare-workers.js" }, + "./cloudflare-email": { + "types": "./dist/src/api/cloudflare-email.d.ts", + "import": "./dist/cloudflare-email.js" + }, "./gate": { "types": "./dist/src/gate.d.ts", "import": "./dist/gate.js" @@ -103,26 +107,27 @@ "build": "pnpm build:package", "typecheck": "pnpm build:package && tsc -b tsconfig.json && tsc -p conformance/tsconfig.json && tsc -p examples/extension/tsconfig.page.json && tsc -p examples/extension/tsconfig.worker.json && tsc -p examples/vibe-platform/tsconfig.page.json && tsc -p examples/vibe-platform/tsconfig.worker.json", "test": "pnpm test:unit && pnpm test:conformance", - "examples:build": "pnpm sdk:sync && pnpm build:package && pnpm --filter \"do-runtime-example-*\" build", "test:examples": "pnpm sdk:sync && pnpm build:package && node examples/extension/scripts/e2e.mjs && node examples/vibe-platform/scripts/e2e.mjs", "test:unit": "vitest run --config vitest.unit.config.ts", - "test:conformance": "pnpm test:conformance-workerd && pnpm test:conformance-node && pnpm test:conformance-browser", + "test:conformance": "pnpm test:conformance-workerd && pnpm test:conformance-node && pnpm test:conformance-browser && pnpm test:conformance-node-transformed && pnpm test:conformance-browser-transformed", "test:conformance-workerd": "vitest run --config conformance/workerd/vitest.config.ts", "test:conformance-node": "vitest run --config conformance/node/vitest.config.ts", "test:conformance-browser": "vitest run --config conformance/browser/vitest.config.ts", + "test:conformance-node-transformed": "vitest run --config conformance/node/vitest.transformed.config.ts", + "test:conformance-browser-transformed": "vitest run --config conformance/browser/vitest.transformed.config.ts conformance/suite", "check:oracle": "node scripts/check-workerd-oracle.mjs", "check:package": "pnpm build:package && publint && attw --pack . --profile esm-only && node scripts/check-package.mjs", "build:package": "node scripts/build-package.mjs && tsc -p tsconfig.publish.json && node scripts/fix-declaration-imports.mjs", "changeset": "changeset", "changeset:version": "changeset version", - "changeset:publish": "pnpm check:package && changeset publish", + "changeset:publish": "changeset publish", "prepare": "pnpm build:package", "prepack": "pnpm build:package", "prepublishOnly": "pnpm check:package", "publish:dry": "pnpm publish --access public --dry-run", "bench:node": "vitest run --config conformance/bench/vitest.node.config.ts", "bench:browser": "vitest run --config conformance/bench/vitest.browser.config.ts", - "sdk:pack": "pnpm sdk:build && rm -rf dist/sdk && pnpm --dir vendor/agents --filter agents --filter \"@cloudflare/*\" --fail-if-no-match -r pack --pack-destination \"$PWD/dist/sdk\"" + "sdk:pack": "pnpm sdk:build && rm -rf .sdk-pack && pnpm --dir vendor/agents --filter agents --filter \"@cloudflare/*\" --fail-if-no-match -r pack --pack-destination \"$PWD/.sdk-pack\"" }, "dependencies": { "@ungap/structured-clone": "1.4.0", @@ -146,7 +151,6 @@ "@types/node": "^26.5.1", "@types/ungap__structured-clone": "1.2.0", "@vitest/browser-playwright": "4.1.11", - "@vitest/coverage-v8": "4.1.11", "drizzle-orm": "^0.45.2", "playwright": "1.63.0", "publint": "0.3.24", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9e9cf9f..339f443 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -45,9 +45,6 @@ importers: '@vitest/browser-playwright': specifier: 4.1.11 version: 4.1.11(playwright@1.63.0)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(vitest@4.1.11) - '@vitest/coverage-v8': - specifier: 4.1.11 - version: 4.1.11(@vitest/browser@4.1.11)(vitest@4.1.11) drizzle-orm: specifier: ^0.45.2 version: 0.45.2(@cloudflare/workers-types@5.20260911.1)(sql.js@1.14.2) @@ -62,7 +59,7 @@ importers: version: 6.0.3 vitest: specifier: 4.1.11 - version: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-v8@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) + version: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) workerd: specifier: 1.20260911.1 version: 1.20260911.1 @@ -74,7 +71,7 @@ importers: dependencies: '@cloudflare/think': specifier: file:../../vendor/agents/packages/think - version: file:vendor/agents/packages/think(@ai-sdk/provider@4.0.9)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(agents@file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4) + version: file:vendor/agents/packages/think(@ai-sdk/provider@4.0.9)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(agents@file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4) '@mcp-b/do-runtime': specifier: workspace:* version: link:../.. @@ -86,7 +83,7 @@ importers: version: 3.53.4-build1 agents: specifier: file:../../vendor/agents/packages/agents - version: file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4) + version: file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4) ai: specifier: 7.0.18 version: 7.0.18(zod@4.6.4) @@ -126,7 +123,7 @@ importers: version: 3.53.4-build1 agents: specifier: file:../../vendor/agents/packages/agents - version: file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4) + version: file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4) capnweb: specifier: 0.12.0 version: 0.12.0 @@ -255,18 +252,10 @@ packages: resolution: {integrity: sha512-xmCA9kP3IhySsqhzwIdWGlDN/1A4cCKNBO/uwZx/3YzmDoMePwno2Q5/Bq0q+tYaKbeF940YiKV/kaW8Mzvpjw==} engines: {node: ^22.18.0 || >=24.11.0} - '@babel/helper-string-parser@7.29.7': - resolution: {integrity: sha512-Pb5ijPrZ89GDH8223L4UP8i6QApWxs04RbPQJTeWDV0/keR2E36MeKnyr6LYmUUvqRRI+Iv87SuF1W6ErINzYw==} - engines: {node: '>=6.9.0'} - '@babel/helper-string-parser@8.0.0': resolution: {integrity: sha512-6mJgmFFFIIO82vvoLt9XtRC7/TkzXfts1t/SpRX4IHSzMgqoPYCWesVu1udUPUWioAE/2fcG6WuI8zrkE1gwrg==} engines: {node: ^22.18.0 || >=24.11.0} - '@babel/helper-validator-identifier@7.29.7': - resolution: {integrity: sha512-qehxGkRj55h/ff8EMaJ+cYhyaKlHIxqYDn682wQD7RNp9UujOQsHog2uS0r2vzr4pW+sXf90NeeayjcNaX3fFg==} - engines: {node: '>=6.9.0'} - '@babel/helper-validator-identifier@8.0.4': resolution: {integrity: sha512-4wFaiLd0bVo4cIoTXI3zKI038NIWE/cr3jvBjejOVYVxV/m8Ltav1USiGzG1fmS5J2RhgEOgXNNK46cRPnRsrg==} engines: {node: ^22.18.0 || >=24.11.0} @@ -279,11 +268,6 @@ packages: resolution: {integrity: sha512-fQtPOXjYOYv85PIdwotp2TJGVYOycX0PQq+l844fFAxOULtBy8BVF35GyeueX0r4KvDthqPH5xAI1clQPk/2uA==} engines: {node: ^22.18.0 || >=24.11.0} - '@babel/parser@7.29.8': - resolution: {integrity: sha512-E8lTAYNB1KW+FH+VGJuZM1ioAx2E6oVlvQFRrf5P8ZZmsiJXYAD9vTFV7yyEURNzgh1dFqMZuO6tUwcARbqFCA==} - engines: {node: '>=6.0.0'} - hasBin: true - '@babel/parser@8.0.4': resolution: {integrity: sha512-srpptsAkEbbNIC/q8nT7o+m6CQe8CJUTV/t7MYc9NnWlgYVtHOb7JH6SorxMhN0kuRJjVqXbKClG6xSbPtzz+g==} engines: {node: ^22.18.0 || >=24.11.0} @@ -326,10 +310,6 @@ packages: resolution: {integrity: sha512-XFfnuvapSc/vJOcUO7kwORSvpBIvraofKEZ2dhT0PjiF21BRCD7YbAFC8UEeDJNeLoQz82/gVqzgX5hCzkCbdg==} engines: {node: ^22.18.0 || >=24.11.0} - '@babel/types@7.29.8': - resolution: {integrity: sha512-Vj1jF3cPfxg7OAfoI7QnVKLoILlm2JF9pnVHrX8qx7AHMiYWT+NDAA7jChlNgRS4WTLc/fD1lXLmPixluj+3Gg==} - engines: {node: '>=6.9.0'} - '@babel/types@8.0.4': resolution: {integrity: sha512-eY+Yn3dCqTGmyiq2QRU66lA5FL8lqqqvecHt0fF3uHONIa7ToYsaCiWV8lOKqAs0Rb2SjixiKFROngnulPtt2g==} engines: {node: ^22.18.0 || >=24.11.0} @@ -338,10 +318,6 @@ packages: resolution: {integrity: sha512-eVdMqi3ej5aHhyQ2Si6yD2cAWeV8FJK9UrhK5aL0Sd8hu5GhT+YswhVNbVheOGVYMg8kuGuMaUpkB3stjj4z8A==} engines: {node: ^22.18.0 || >=24.11.0} - '@bcoe/v8-coverage@1.0.2': - resolution: {integrity: sha512-6zABk/ECA/QYSCQ1NGiVwwbQerUCZ+TQbp64Q3AgmfNvurHH0j8TtXa1qbShXA6qqkpAj4V5W8pP6mLe1mcMqA==} - engines: {node: '>=18'} - '@blazediff/core@1.9.1': resolution: {integrity: sha512-ehg3jIkYKulZh+8om/O25vkvSsXXwC+skXmyA87FFx6A/45eqOkZsBltMw/TVteb0mloiGT8oGRTcjRAz66zaA==} @@ -451,9 +427,9 @@ packages: resolution: {directory: vendor/agents/packages/think, type: directory} peerDependencies: '@ai-sdk/react': ^3.0.0 || ^4.0.0 - '@chat-adapter/discord': 4.40.0 - '@chat-adapter/slack': 4.40.0 - '@chat-adapter/telegram': ^4.29.0 + '@chat-adapter/discord': 4.38.0 + '@chat-adapter/slack': 4.38.0 + '@chat-adapter/telegram': 4.38.0 agents: '>=0.23.0 <1.0.0' ai: ^6.0.0 || ^7.0.0 react: ^19.0.0 @@ -1523,15 +1499,6 @@ packages: peerDependencies: vitest: 4.1.11 - '@vitest/coverage-v8@4.1.11': - resolution: {integrity: sha512-8MVGEFnJIcdGjcbfKmeq8z0pZHH0JlVtoVZH9Q/qwUp6wyFnEJUBMrw9DCaj+ra3vShGmhavjalMIhPNxZAUcw==} - peerDependencies: - '@vitest/browser': 4.1.11 - vitest: 4.1.11 - peerDependenciesMeta: - '@vitest/browser': - optional: true - '@vitest/expect@4.1.11': resolution: {integrity: sha512-VX2x5vNJXET47KAFzwERI+KRMtTTCSWTfSMKsW7JsUsXV4psq++e3DvZpuTDOpHcxytiDs6p2nhVb2tVDiiUYw==} @@ -1592,7 +1559,7 @@ packages: '@x402/core': ^2.0.0 '@x402/evm': ^2.0.0 ai: ^6.0.0 || ^7.0.0 - chat: ^4.29.0 + chat: 4.38.0 just-bash: ^3.0.0 react: ^19.0.0 vite: '>=6.0.0 <9.0.0' @@ -1662,9 +1629,6 @@ packages: resolution: {integrity: sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==} engines: {node: '>=12'} - ast-v8-to-istanbul@1.0.5: - resolution: {integrity: sha512-UPAgKJFSEGMWSDr3LX4tqnAb4f7KGT8O40Tyx8wbYmmZ/yn58lNCm8h3svs3eXgiGd5AXxz8NDOvXWvicq+rJA==} - async-lock@1.4.1: resolution: {integrity: sha512-Az2ZTpuytrtqENulXwO3GGv1Bztugx6TT37NIo7imr/Qo0gsYiGtSdBa2B6fsXhTpVZDNfu1Qn3pk531e3q+nQ==} @@ -1760,8 +1724,8 @@ packages: character-entities@2.0.2: resolution: {integrity: sha512-shx7oQ0Awen/BRIdkjkvz54PnEEI/EjwXDSIZp86/KKdbafHh1Df/RYGBhn4hbe2+uKC9FnT5UCEdyPz3ai9hQ==} - chat@4.40.0: - resolution: {integrity: sha512-slu3VDxItlelEZ8A5vqzlmtT2WErqj2YCGuhhcNx+Ev0+wHeBUqUDUA7hXkca+BfFtW1ZX7ECcySCTG2q2gefg==} + chat@4.38.0: + resolution: {integrity: sha512-begN9W19/kSfMPz7hl0q8CNufs6TE7ddvIERhBPjShxWRvwI6+KGVGlr9cJsNoppjE3JywvHUYUuifZa5uj6IA==} engines: {node: '>=20'} peerDependencies: ai: ^6.0.182 || ^7.0.0 @@ -2230,9 +2194,6 @@ packages: resolution: {integrity: sha512-c8/gF9ac8Y78/agExVocyLevgR+JlpNB444Py0FSX8pJoPdYUfUzRcXtYEYGwt6l19qIlVZPN5Mfsw9jFShmQQ==} engines: {node: '>=16.9.0'} - html-escaper@2.0.2: - resolution: {integrity: sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==} - http-errors@2.0.1: resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==} engines: {node: '>= 0.8'} @@ -2306,18 +2267,6 @@ packages: engines: {node: '>=14.17'} hasBin: true - istanbul-lib-coverage@3.2.2: - resolution: {integrity: sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg==} - engines: {node: '>=8'} - - istanbul-lib-report@3.0.1: - resolution: {integrity: sha512-GCfE1mtsHGOELCU8e/Z7YWzpmybrx/+dSTfLrvY8qRmaY6zXTKWn6WQIjaAFw069icm6GVMNkgu0NzI4iPZUNw==} - engines: {node: '>=10'} - - istanbul-reports@3.2.0: - resolution: {integrity: sha512-HGYWWS/ehqTV3xN10i23tkPkpH46MLCIMFNCaaKNavAXTF1RkqxawEPtnjnGZ6XKSInBKkiOA5BKS+aZiY3AvA==} - engines: {node: '>=8'} - jju@1.4.0: resolution: {integrity: sha512-8wb9Yw966OSxApiCt0K3yNJL8pnNeIv+OEq2YMidz4FKP6nonSRoOXc80iXY4JaN2FC11B9qsNmDsm+ZOfMROA==} @@ -2451,13 +2400,6 @@ packages: magic-string@1.3.1: resolution: {integrity: sha512-rm91zr2Ou+XueDTohjQQjdQEcYM6zVi8KVUCG8Ec3vHwUEKrhSdCNyfuIywkA6hcCAteIn0ZOtAHA6eGpiX+Pg==} - magicast@0.5.4: - resolution: {integrity: sha512-llBEhWm1SacoRwgHUoQJYtwp4PBLF4faQi5TCpIGyGs9n4y5+juI0tDgyKIfpqxckRHaHzouUEph3THklWh03w==} - - make-dir@4.0.0: - resolution: {integrity: sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==} - engines: {node: '>=10'} - markdown-table@3.0.4: resolution: {integrity: sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw==} @@ -3590,12 +3532,8 @@ snapshots: '@babel/traverse': 8.0.4 '@babel/types': 8.0.4 - '@babel/helper-string-parser@7.29.7': {} - '@babel/helper-string-parser@8.0.0': {} - '@babel/helper-validator-identifier@7.29.7': {} - '@babel/helper-validator-identifier@8.0.4': {} '@babel/helper-validator-option@8.0.0': {} @@ -3605,10 +3543,6 @@ snapshots: '@babel/template': 8.0.0 '@babel/types': 8.0.5 - '@babel/parser@7.29.8': - dependencies: - '@babel/types': 7.29.8 - '@babel/parser@8.0.4': dependencies: '@babel/types': 8.0.4 @@ -3661,11 +3595,6 @@ snapshots: '@babel/types': 8.0.5 obug: 2.2.1 - '@babel/types@7.29.8': - dependencies: - '@babel/helper-string-parser': 7.29.7 - '@babel/helper-validator-identifier': 7.29.7 - '@babel/types@8.0.4': dependencies: '@babel/helper-string-parser': 8.0.0 @@ -3676,8 +3605,6 @@ snapshots: '@babel/helper-string-parser': 8.0.0 '@babel/helper-validator-identifier': 8.0.4 - '@bcoe/v8-coverage@1.0.2': {} - '@blazediff/core@1.9.1': {} '@borewit/text-codec@0.2.2': {} @@ -3823,15 +3750,15 @@ snapshots: - ai - zod - '@cloudflare/think@file:vendor/agents/packages/think(@ai-sdk/provider@4.0.9)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(agents@file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4)': + '@cloudflare/think@file:vendor/agents/packages/think(@ai-sdk/provider@4.0.9)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(agents@file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4)': dependencies: '@ai-sdk/anthropic': 4.0.46(zod@4.6.4) '@ai-sdk/openai': 4.0.52(zod@4.6.4) '@cloudflare/codemode': 0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4) '@cloudflare/shell': 0.4.3(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4) - agents: file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4) + agents: file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4) ai: 7.0.18(zod@4.6.4) - chat: 4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4) + chat: 4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4) just-bash: 3.4.2 workers-ai-provider: 4.0.0(@ai-sdk/anthropic@4.0.46(zod@4.6.4))(@ai-sdk/openai@4.0.52(zod@4.6.4))(@ai-sdk/provider@4.0.9)(ai@7.0.18(zod@4.6.4)) zod: 4.6.4 @@ -3862,7 +3789,7 @@ snapshots: cjs-module-lexer: 1.2.3 esbuild: 0.28.1 miniflare: 5.20260815.0-alpha - vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-v8@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) + vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) wrangler: 4.124.0(@cloudflare/workers-types@5.20260911.1) zod: 4.4.3 transitivePeerDependencies: @@ -4334,7 +4261,7 @@ snapshots: '@jridgewell/trace-mapping@0.3.31': dependencies: '@jridgewell/resolve-uri': 3.1.2 - '@jridgewell/sourcemap-codec': 1.5.5 + '@jridgewell/sourcemap-codec': 1.6.0 '@jridgewell/trace-mapping@0.3.9': dependencies: @@ -4569,7 +4496,7 @@ snapshots: '@vitest/mocker': 4.1.11(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) playwright: 1.63.0 tinyrainbow: 3.1.1 - vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-v8@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) + vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) transitivePeerDependencies: - bufferutil - msw @@ -4585,7 +4512,7 @@ snapshots: pngjs: 7.0.0 sirv: 3.0.2 tinyrainbow: 3.1.1 - vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-v8@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) + vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) ws: 8.21.3 transitivePeerDependencies: - bufferutil @@ -4593,22 +4520,6 @@ snapshots: - utf-8-validate - vite - '@vitest/coverage-v8@4.1.11(@vitest/browser@4.1.11)(vitest@4.1.11)': - dependencies: - '@bcoe/v8-coverage': 1.0.2 - '@vitest/utils': 4.1.11 - ast-v8-to-istanbul: 1.0.5 - istanbul-lib-coverage: 3.2.2 - istanbul-lib-report: 3.0.1 - istanbul-reports: 3.2.0 - magicast: 0.5.4 - obug: 2.1.4 - std-env: 4.2.0 - tinyrainbow: 3.1.1 - vitest: 4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-v8@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) - optionalDependencies: - '@vitest/browser': 4.1.11(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(vitest@4.1.11) - '@vitest/expect@4.1.11': dependencies: '@standard-schema/spec': 1.1.0 @@ -4665,7 +4576,7 @@ snapshots: acorn@8.18.0: {} - agents@file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4): + agents@file:vendor/agents/packages/agents(@babel/core@8.0.1)(@babel/runtime@7.29.7)(@cloudflare/codemode@0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(@modelcontextprotocol/client@2.0.0)(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(@modelcontextprotocol/server@2.0.0)(ai@7.0.18(zod@4.6.4))(chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4))(just-bash@3.4.2)(rolldown@1.2.8)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(zod@4.6.4): dependencies: '@babel/plugin-proposal-decorators': 8.0.2(@babel/core@8.0.1) '@cfworker/json-schema': 4.1.1 @@ -4685,7 +4596,7 @@ snapshots: optionalDependencies: '@cloudflare/codemode': 0.5.1(@modelcontextprotocol/sdk@1.30.0(@cfworker/json-schema@4.1.1)(zod@4.6.4))(ai@7.0.18(zod@4.6.4))(zod@4.6.4) ai: 7.0.18(zod@4.6.4) - chat: 4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4) + chat: 4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4) just-bash: 3.4.2 vite: 8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1) transitivePeerDependencies: @@ -4730,12 +4641,6 @@ snapshots: assertion-error@2.0.1: {} - ast-v8-to-istanbul@1.0.5: - dependencies: - '@jridgewell/trace-mapping': 0.3.31 - estree-walker: 3.0.3 - js-tokens: 10.0.0 - async-lock@1.4.1: {} available-typed-arrays@1.0.7: @@ -4836,7 +4741,7 @@ snapshots: character-entities@2.0.2: {} - chat@4.40.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4): + chat@4.38.0(ai@7.0.18(zod@4.6.4))(zod@4.6.4): dependencies: '@workflow/serde': 4.1.0-beta.2 mdast-util-to-string: 4.0.0 @@ -5246,8 +5151,6 @@ snapshots: hono@4.13.7: {} - html-escaper@2.0.2: {} - http-errors@2.0.1: dependencies: depd: 2.0.0 @@ -5311,19 +5214,6 @@ snapshots: sha.js: 2.4.12 simple-get: 4.0.1 - istanbul-lib-coverage@3.2.2: {} - - istanbul-lib-report@3.0.1: - dependencies: - istanbul-lib-coverage: 3.2.2 - make-dir: 4.0.0 - supports-color: 7.2.0 - - istanbul-reports@3.2.0: - dependencies: - html-escaper: 2.0.2 - istanbul-lib-report: 3.0.1 - jju@1.4.0: {} jose@6.2.12: {} @@ -5436,16 +5326,6 @@ snapshots: dependencies: '@jridgewell/sourcemap-codec': 1.6.0 - magicast@0.5.4: - dependencies: - '@babel/parser': 7.29.8 - '@babel/types': 7.29.8 - source-map-js: 1.2.1 - - make-dir@4.0.0: - dependencies: - semver: 7.8.5 - markdown-table@3.0.4: {} marked-terminal@7.3.0(marked@9.1.6): @@ -6487,7 +6367,7 @@ snapshots: fsevents: 2.3.3 yaml: 2.9.1 - vitest@4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(@vitest/coverage-v8@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)): + vitest@4.1.11(@types/node@26.5.1)(@vitest/browser-playwright@4.1.11)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)): dependencies: '@vitest/expect': 4.1.11 '@vitest/mocker': 4.1.11(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1)) @@ -6512,7 +6392,6 @@ snapshots: optionalDependencies: '@types/node': 26.5.1 '@vitest/browser-playwright': 4.1.11(playwright@1.63.0)(vite@8.3.0(@types/node@26.5.1)(esbuild@0.28.2)(yaml@2.9.1))(vitest@4.1.11) - '@vitest/coverage-v8': 4.1.11(@vitest/browser@4.1.11)(vitest@4.1.11) transitivePeerDependencies: - msw diff --git a/scripts/build-package.mjs b/scripts/build-package.mjs index 5f73cb1..0613774 100644 --- a/scripts/build-package.mjs +++ b/scripts/build-package.mjs @@ -34,6 +34,7 @@ await build({ "backends/sqlite-wasm": new URL("backends/sqlite-wasm.ts", root).pathname, "backends/node-sqlite": new URL("backends/node-sqlite.ts", root).pathname, "cloudflare-workers": new URL("src/api/cloudflare-workers.ts", root).pathname, + "cloudflare-email": new URL("src/api/cloudflare-email.ts", root).pathname, gate: new URL("src/gate.ts", root).pathname, vite: new URL("src/vite.ts", root).pathname, conformance: new URL("conformance/host.ts", root).pathname, diff --git a/scripts/check-package.mjs b/scripts/check-package.mjs index 1f49eb4..79e293b 100644 --- a/scripts/check-package.mjs +++ b/scripts/check-package.mjs @@ -77,16 +77,19 @@ if (typeof runtime.BrokenActorError !== "function" || typeof runtime.CanceledErr } if ( typeof alarmCoordinator.BrowserAlarmCoordinator !== "function" || - typeof alarmCoordinator.parseBrowserAlarmTransportJournal !== "function" + typeof alarmCoordinator.parseBrowserAlarmTransportJournal !== "function" || + typeof alarmCoordinator.createBrowserAlarmProjector !== "function" || + typeof alarmCoordinator.parseBrowserAlarmProjection !== "function" ) { - throw new Error("packed browser alarm entry does not export its coordinator and parser"); + throw new Error("packed browser alarm entry does not export its coordinator, projector and parsers"); } if ( typeof messagePortWebSocket.MessagePortWebSocket !== "function" || typeof messagePortWebSocket.createMessagePortWebSocketConstructor !== "function" || - typeof messagePortWebSocket.serveMessagePortWebSockets !== "function" + typeof messagePortWebSocket.serveMessagePortWebSockets !== "function" || + typeof modules.get("./browser").connectMessagePortWebSocket !== "function" ) { - throw new Error("packed MessagePort WebSocket entry does not export its host helpers"); + throw new Error("packed MessagePort WebSocket entries do not export their host helpers"); } if (typeof offscreenDocument.OffscreenDocumentCoordinator !== "function") { throw new Error("packed offscreen document entry does not export its coordinator"); @@ -97,8 +100,20 @@ if (typeof nodeBackend.createNodeSqlProvider !== "function") { if (typeof gate.__gate !== "function" || typeof gate.__gateAsyncIterable !== "function") { throw new Error("packed gate entry does not export its helpers"); } -if (typeof vite.doRuntimeAwaitTransform !== "function") { - throw new Error("packed Vite entry does not export doRuntimeAwaitTransform"); +if (typeof vite.doRuntimeAwaitTransform !== "function" || typeof vite.facetScopeBanner !== "function") { + throw new Error("packed Vite entry does not export doRuntimeAwaitTransform and facetScopeBanner"); +} +if (new (modules.get("./cloudflare-email").EmailMessage)("a", "b", "c").to !== "b") { + throw new Error("packed cloudflare-email entry does not export its EmailMessage constructor"); +} +// The plugin resolves what it injects to the very files the export map names, so an application +// import of the same subpath is the same module instance. +const { resolveId } = vite.doRuntimeAwaitTransform({ asyncContext: true }); +for (const subpath of ["./gate", "./browser/async-hooks"]) { + const target = fileURLToPath(new URL(manifest.exports[subpath].import, root)); + if (resolveId.handler(`@mcp-b/do-runtime${subpath.slice(1)}`) !== target) { + throw new Error(`packed Vite plugin does not resolve its injected ${subpath} import to the export`); + } } // Compile and run the documented host against only the files npm will ship. diff --git a/scripts/check-workerd-oracle.mjs b/scripts/check-workerd-oracle.mjs index 256b557..6f23b0a 100644 --- a/scripts/check-workerd-oracle.mjs +++ b/scripts/check-workerd-oracle.mjs @@ -24,11 +24,15 @@ for (const name of platforms) { const readme = await readFile(new URL("README.md", root), "utf8"); const decisions = await readFile(new URL("docs/decisions.md", root), "utf8"); +const sync = await readFile(new URL("docs/workerd-sync.md", root), "utf8"); if (!readme.includes(`conformance oracle is pinned to \`v${version}\``)) { throw new Error(`README.md does not name oracle v${version}`); } if (!decisions.includes(`oracle is pinned separately to release \`v${version}\``)) { throw new Error(`docs/decisions.md does not name oracle v${version}`); } +if (!sync.includes(`oracle is now pinned to \`v${version}\``)) { + throw new Error(`docs/workerd-sync.md does not record the re-pin to oracle v${version}`); +} console.log(`workerd oracle pins agree on v${version}`); diff --git a/src/api/actor-scope-globals.ts b/src/api/actor-scope-globals.ts new file mode 100644 index 0000000..0397000 --- /dev/null +++ b/src/api/actor-scope-globals.ts @@ -0,0 +1,23 @@ +// `.js`: see the import of this file in src/vite.ts. +import type { ActorScopeBindings } from "./global-scope.js"; + +/** + * The names `installActorScope` writes. A facet bundle that binds its own scope must bind all + * of them: a name it leaves out resolves to the root actor's installed global. + * + * A module of its own so the Vite plugin reads it without loading the runtime. + */ +export const ACTOR_SCOPE_GLOBALS = Object.freeze([ + "scheduler", + "setTimeout", + "clearTimeout", + "setInterval", + "clearInterval", + "fetch", + "crypto", + "WebSocket", + "WebSocketPair", + "WebSocketRequestResponsePair", + "ReadableStream", + "TransformStream", +] as const satisfies readonly (keyof ActorScopeBindings)[]); diff --git a/src/api/cloudflare-email.test.ts b/src/api/cloudflare-email.test.ts new file mode 100644 index 0000000..8dcdac9 --- /dev/null +++ b/src/api/cloudflare-email.test.ts @@ -0,0 +1,16 @@ +/** + * NO upstream test file. workerd's `cloudflare:email` (`src/cloudflare/email.ts`) re-exports + * `EmailMessage` from `cloudflare-internal:email`, which the embedder supplies — Miniflare on + * the oracle lane. These are that module's answers, measured on workerd 1.20260911.1. + */ + +import { expect, test } from "vitest"; +import { EmailMessage } from "./cloudflare-email"; + +test("EmailMessage holds its arguments the way workerd's does, unchecked", () => { + expect(JSON.stringify(new EmailMessage("a@example.com", "b@example.com", "raw text"))).toBe( + '{"from":"a@example.com","to":"b@example.com","EmailMessage::raw":"raw text"}', + ); + const unchecked = new EmailMessage(1 as never, 2 as never, "raw text"); + expect([unchecked.from, unchecked.to]).toEqual([1, 2]); +}); diff --git a/src/api/cloudflare-email.ts b/src/api/cloudflare-email.ts new file mode 100644 index 0000000..5372b5a --- /dev/null +++ b/src/api/cloudflare-email.ts @@ -0,0 +1,21 @@ +/** + * ← workerd `src/cloudflare/email.ts` — the built-in `cloudflare:email` module, which + * re-exports `EmailMessage` from the embedder's `cloudflare-internal:email`. + * + * A data constructor and nothing more: sending belongs to the `send_email` binding and to + * `ForwardableEmailMessage.reply()`, which the host supplies. The Agents SDK imports this module + * eagerly and constructs one in `reply()`. The fields are what the oracle's module holds + * (workerd 1.20260911.1 under Miniflare): `from` and `to` as given, unchecked, and the body under + * the key its `send_email` binding reads. + */ +export class EmailMessage { + readonly from: string; + readonly to: string; + readonly "EmailMessage::raw": ReadableStream | string; + + constructor(from: string, to: string, raw: ReadableStream | string) { + this.from = from; + this.to = to; + this["EmailMessage::raw"] = raw; + } +} diff --git a/src/api/cloudflare-workers.test.ts b/src/api/cloudflare-workers.test.ts index d5a7dca..77c24be 100644 --- a/src/api/cloudflare-workers.test.ts +++ b/src/api/cloudflare-workers.test.ts @@ -160,6 +160,17 @@ test("withEnvAndExports installs both at once", () => { clearEnv(); }); +test("withEnvAndExports refuses a bad exports value without leaving the env scope installed", () => { + clearEnv(); + Object.assign(env, { REAL: "base" }); + expect(() => withEnvAndExports({ LEAKED: "scoped" }, "not an object", () => 1)).toThrow( + TypeError, + ); + expect((env as Record).REAL).toBe("base"); + expect((env as Record).LEAKED).toBeUndefined(); + clearEnv(); +}); + // ======================================================================================= // The entrypoint classes diff --git a/src/api/cloudflare-workers.ts b/src/api/cloudflare-workers.ts index 39f65f6..e35a3a4 100644 --- a/src/api/cloudflare-workers.ts +++ b/src/api/cloudflare-workers.ts @@ -234,6 +234,8 @@ function runInScopes( pushes: readonly { readonly scopes: Bindings[]; readonly value: unknown }[], fn: () => unknown, ): unknown { + // Validate every scope before pushing any: a refusal must not leave one installed realm-wide. + for (const push of pushes) asBindings(push.value); for (const push of pushes) push.scopes.push(asBindings(push.value)); try { const result = fn(); diff --git a/src/api/global-scope.test.ts b/src/api/global-scope.test.ts index 8ed532c..120b129 100644 --- a/src/api/global-scope.test.ts +++ b/src/api/global-scope.test.ts @@ -24,8 +24,10 @@ import { isAlarmFailureUserError, NO_GLOBAL_OUTBOUND_MESSAGE, } from "./global-scope"; +import { ACTOR_SCOPE_GLOBALS } from "./actor-scope-globals"; import { HibernatableWebSocketRegistry } from "./web-socket"; import { AsyncLocalStorage } from "../browser/async-hooks"; +import { facetScopeBanner } from "../vite"; test("readable stream callbacks re-enter their creator when consumed outside its actor", async () => { const { ctx, scope } = newScope(); @@ -524,6 +526,51 @@ describe("installActorScope", () => { expect(bound.currentExternalEntry).toBe(currentExternalEntry); }); + test("§1.7 a facet bundle's WebSocketPair sends behind the facet's commit, not the root's", async () => { + // A same-realm host installs the root's scope as the realm's globals and prefixes each + // facet bundle with `facetScopeBanner`. A name the banner leaves unbound resolves to the + // root, so a facet's pair would wait on the root's output gate and send ahead of the + // facet's own write. + const root = newScope(); + const facet = newScope(); + const registry = "__doRuntimeBannerTestScopes"; + Reflect.set(globalThis, registry, { facet: actorScopeBindings(() => facet.scope) }); + // A data: URL's query is part of its body; the trailing `//` keeps that body a comment. + const source = `${facetScopeBanner({ registry })}\nexport const pair = () => new WebSocketPair();\n//`; + const bundle = (await import( + /* @vite-ignore */ `data:text/javascript,${encodeURIComponent(source)}?scope=facet` + )) as { pair(): InstanceType }; + const received: unknown[] = []; + const commit = Promise.withResolvers(); + void facet.ctx.lockOutputWhile(commit.promise); + + const realm = ACTOR_SCOPE_GLOBALS.map( + (name) => [name, Object.getOwnPropertyDescriptor(globalThis, name)] as const, + ); + installActorScope(globalThis, () => root.scope); + try { + await facet.ctx.run(() => { + const { 0: client, 1: server } = bundle.pair(); + client.accept(); + server.accept(); + client.addEventListener("message", (event) => received.push(event.data)); + server.send("frame"); + }); + } finally { + for (const [name, descriptor] of realm) { + if (descriptor === undefined) Reflect.deleteProperty(globalThis, name); + else Object.defineProperty(globalThis, name, descriptor); + } + Reflect.deleteProperty(globalThis, registry); + } + + await quiesce(); + expect(received).toEqual([]); + commit.resolve(); + await quiesce(); + expect(received).toEqual(["frame"]); + }); + test("writes all actor globals onto a scope object, bound", async () => { // Bound, because a dynamically-loaded Worker source destructures them: `const // { scheduler, setTimeout } = …` would lose `this` on a method. @@ -531,20 +578,11 @@ describe("installActorScope", () => { const target: Record = {}; installActorScope(target, () => scope); - expect(Object.keys(target).sort()).toEqual([ - "ReadableStream", - "TransformStream", - "WebSocket", - "WebSocketPair", - "WebSocketRequestResponsePair", - "clearInterval", - "clearTimeout", - "crypto", - "fetch", - "scheduler", - "setInterval", - "setTimeout", - ]); + expect(Object.keys(target).sort()).toEqual( + Object.keys(actorScopeBindings(() => scope)) + .filter((n) => n !== "awaitIo" && n !== "currentExternalEntry") + .sort(), + ); const { setTimeout: armed, scheduler: sched } = target as unknown as ActorScopeBindings; const seen: string[] = []; diff --git a/src/api/global-scope.ts b/src/api/global-scope.ts index 781ab80..e4ec99b 100644 --- a/src/api/global-scope.ts +++ b/src/api/global-scope.ts @@ -52,6 +52,7 @@ import { type IoContext, } from "../io/io-context"; import { onAbort } from "../io/io-gate"; +import { ACTOR_SCOPE_GLOBALS } from "./actor-scope-globals"; import { gateResponseBody } from "./http"; import { installWebSocketGlobals, @@ -700,13 +701,8 @@ const ASYNC_SUBTLE_METHODS = [ * exceeded` on the first row. */ export function installActorScope(target: object, resolve: () => ActorGlobalScope): void { - const bindings = actorScopeBindings(resolve); - // Descriptors, not values: `crypto` is a getter, and reading it here would resolve the scope - // at install time — which is before the container exists on the facet path, where the whole - // arrangement is a late binding. - for (const [name, descriptor] of Object.entries(Object.getOwnPropertyDescriptors(bindings))) { - // These are explicit actor capabilities, not web-platform globals. - if (name === "awaitIo" || name === "currentExternalEntry") continue; - Object.defineProperty(target, name, { ...descriptor, configurable: true }); + const descriptors = Object.getOwnPropertyDescriptors(actorScopeBindings(resolve)); + for (const name of ACTOR_SCOPE_GLOBALS) { + Object.defineProperty(target, name, { ...descriptors[name], configurable: true }); } } diff --git a/src/api/hibernatable-web-socket.test.ts b/src/api/hibernatable-web-socket.test.ts index 84676e0..a33a8ac 100644 --- a/src/api/hibernatable-web-socket.test.ts +++ b/src/api/hibernatable-web-socket.test.ts @@ -9,6 +9,7 @@ import { type ActorContainerOptions, type HibernationHost, } from "../server/actor-container"; +import { MessagePortWebSocket } from "../browser/message-port-websocket"; import { HibernationMirror } from "../server/hibernation-mirror"; import type { RawWebSocket } from "./web-socket"; @@ -340,6 +341,41 @@ describe("hibernation embedder contract", () => { expect(secondActor.messages).toEqual(["after-rebuild"]); }); + test("gates a rehydrated host transport's sends behind storage confirmation", async () => { + // A Worker restart rehydrates the host's own transport, not a pair half. + const channel = new MessageChannel(); + const transport = new MessagePortWebSocket("ws://actor.test", channel.port1, false); + transport.open(); + const frames: unknown[] = []; + channel.port2.addEventListener("message", (event) => frames.push(event.data)); + channel.port2.start(); + const confirmation = Promise.withResolvers(); + const container = await createActorContainer( + options({ + ports: { ...options().ports, alarms: { scheduleRun: () => confirmation.promise } }, + webSockets: [{ socket: transport, tags: ["id"] }], + }), + ); + await container.start(() => ({ alarm() {} })); + + const handed = await container.run(() => { + // Moving the alarm earlier holds this write's commit until scheduleRun confirms. + void container.state.storage.setAlarm(Date.now() + 60_000); + const [socket] = container.state.getWebSockets(); + socket?.send("after the write"); + return socket; + }); + await quiesce(); + expect(frames).toEqual([]); + + confirmation.resolve(); + await quiesce(); + expect(frames).toEqual([{ type: "message", data: "after the write" }]); + expect(container.state.getWebSockets()).toEqual([handed]); + expect(container.state.getWebSockets()[0]).toBe(handed); + channel.port2.close(); + }); + test("pins cross-accept, attachment, and synchronous close errors", async () => { const { container } = await started(); await container.run(() => { diff --git a/src/api/web-socket.ts b/src/api/web-socket.ts index 5bc3bfd..365bea7 100644 --- a/src/api/web-socket.ts +++ b/src/api/web-socket.ts @@ -125,7 +125,8 @@ type SocketDelivery = type SocketMetadata = { accepted?: SocketAcceptance; attachment?: Uint8Array; - rawListenersInstalled?: true; + /** The actor's gated side of a host transport, kept across registries. */ + wrapper?: AcceptedWebSocket; }; /** Raw listeners follow transport ownership without retaining a released actor wrapper. */ @@ -514,7 +515,10 @@ type HandlerDispatch = { }; type RegistryEntry = { - readonly socket: RawWebSocket; + /** What the actor holds: a pair half, or the gated wrapper of a host transport. */ + readonly socket: AcceptedWebSocket; + /** The identity hibernation hosts mirror. */ + readonly raw: RawWebSocket; readonly tags: string[]; autoResponseTimestamp?: number; }; @@ -569,10 +573,10 @@ export class HibernatableWebSocketRegistry { ); } const normalizedTags = normalizeTags(tags); - if (socket instanceof AcceptedWebSocket) socket.acceptHibernation(this); - else this.#listenRaw(socket); + const accepted = socket instanceof AcceptedWebSocket ? socket : this.#wrap(socket); + if (accepted === socket) accepted.acceptHibernation(this); state.accepted = { mode: "hibernatable", registry: this, tags: normalizedTags }; - this.#entries.push({ socket, tags: normalizedTags }); + this.#entries.push({ socket: accepted, raw: socket, tags: normalizedTags }); this.#host?.accepted(socket, normalizedTags); } @@ -629,7 +633,7 @@ export class HibernatableWebSocketRegistry { "Failed to execute 'getWebSocketAutoResponseTimestamp' on 'DurableObjectState': parameter 1 is not of type 'WebSocket'.", ); } - const timestamp = this.#entries.find((entry) => entry.socket === socket)?.autoResponseTimestamp; + const timestamp = this.#entryFor(socket)?.autoResponseTimestamp; return timestamp === undefined ? null : new Date(timestamp); } @@ -662,9 +666,8 @@ export class HibernatableWebSocketRegistry { } attachmentChanged(socket: RawWebSocket, bytes: Uint8Array): void { - if (this.#entries.some((entry) => entry.socket === socket)) { - this.#host?.attachment(socket, bytes); - } + const entry = this.#entryFor(socket); + if (entry !== undefined) this.#host?.attachment(entry.raw, bytes); } receive(socket: RawWebSocket, type: SocketEvent, event: Event): void { @@ -674,11 +677,13 @@ export class HibernatableWebSocketRegistry { const data = (event as MessageEvent).data as unknown; if (typeof data === "string" && data === this.#autoResponse?.request) { entry.autoResponseTimestamp = this.#ctx.now(); - this.#host?.autoResponseTimestamp?.(socket, entry.autoResponseTimestamp); - if (socket instanceof AcceptedWebSocket) { - socket.sendAutoResponse(this.#autoResponse.response); - } else if ((socket.readyState ?? AcceptedWebSocket.OPEN) < AcceptedWebSocket.CLOSING) { - socket.send(this.#autoResponse.response); + this.#host?.autoResponseTimestamp?.(entry.raw, entry.autoResponseTimestamp); + // A host transport answers directly, as it does while the actor is evicted. + const raw = entry.raw; + if (raw instanceof AcceptedWebSocket) { + raw.sendAutoResponse(this.#autoResponse.response); + } else if ((raw.readyState ?? AcceptedWebSocket.OPEN) < AcceptedWebSocket.CLOSING) { + raw.send(this.#autoResponse.response); } return; } @@ -713,21 +718,24 @@ export class HibernatableWebSocketRegistry { const index = this.#entries.indexOf(entry); if (index === -1) return; this.#entries.splice(index, 1); - this.#host?.closed(entry.socket); + this.#host?.closed(entry.raw); } - #listenRaw(socket: RawWebSocket): void { - const state = socketMetadata(socket); - if (state.rawListenersInstalled === true) return; - state.rawListenersInstalled = true; - for (const type of ["message", "close", "error"] as const) { - socket.addEventListener(type, (event) => { - const accepted = socketMetadata(socket).accepted; - if (accepted?.mode === "hibernatable") { - accepted.registry.receive(socket, type, event); - } - }); - } + #entryFor(socket: RawWebSocket): RegistryEntry | undefined { + return this.#entries.find((entry) => entry.socket === socket || entry.raw === socket); + } + + /** + * The actor's side of a host transport, output-gated like a pair half. One + * wrapper per transport shares its metadata and moves to a replacement + * registry, so the actor sees one stable socket and frames arrive once. + */ + #wrap(raw: RawWebSocket): AcceptedWebSocket { + const state = socketMetadata(raw); + state.wrapper ??= new AcceptedWebSocket(this.#ctx, raw); + metadata.set(state.wrapper, state); + state.wrapper.rehydrateHibernation(this, this.#ctx); + return state.wrapper; } #rehydrate(value: RehydratedWebSocket): void { @@ -742,8 +750,8 @@ export class HibernatableWebSocketRegistry { state.attachment = value.attachment.slice(); } if (socket instanceof AcceptedWebSocket) socket.rehydrateHibernation(this, this.#ctx); - else this.#listenRaw(socket); - const entry: RegistryEntry = { socket, tags }; + const accepted = socket instanceof AcceptedWebSocket ? socket : this.#wrap(socket); + const entry: RegistryEntry = { socket: accepted, raw: socket, tags }; if (value.autoResponseTimestamp !== undefined) { entry.autoResponseTimestamp = value.autoResponseTimestamp; } diff --git a/src/browser.ts b/src/browser.ts index 08f05ae..32f3bcf 100644 --- a/src/browser.ts +++ b/src/browser.ts @@ -1,4 +1,5 @@ import { AcceptedWebSocket, markWebSocketUsed, type RawWebSocket } from "./api/web-socket"; +import { bridgeWebSocket, type MessagePortWebSocket } from "./browser/message-port-websocket"; export type UpgradeWebSocket = EventTarget & RawWebSocket & { @@ -57,6 +58,49 @@ export function upgradeWebSocket(response: Response): UpgradeWebSocket | undefin return host; } +/** + * Route one MessagePort socket through a Workers-style `fetch`, such as the + * Agents SDK's `(request) => routeAgentRequest(request, env, { onBeforeConnect: withWebSocketUpgrade })`, + * and bridge the socket it upgrades to. Install the upgrade globals first so + * the handler can answer 101. + * + * The bridge closes 1011 when nothing routes the request or routing throws, + * and 1008 when the upgrade is refused, carrying the response text as the + * reason when it is printable ASCII of at most 123 bytes. The returned promise + * rejects in each of those cases. + * + * It does not tell the peer that the socket opened. `serveMessagePortWebSockets` + * sends that signal to clients made by `createMessagePortWebSocketConstructor`; + * a host running its own port protocol opens its end itself. + */ +export async function connectMessagePortWebSocket( + bridge: MessagePortWebSocket, + url: string, + fetch: (request: Request) => Promise, +): Promise { + try { + const request = withWebSocketUpgrade(new Request(url.replace(/^ws/, "http"))); + const response = await fetch(request); + if (response == null) { + bridge.close(1011, "No WebSocket route matched"); + throw new Error(`No WebSocket route matched ${request.url}`); + } + const socket = upgradeWebSocket(response); + if (response.status !== 101 || socket === undefined) { + const detail = await response.text().catch(() => ""); + const reason = /^[\x20-\x7E]{1,123}$/u.test(detail) ? detail : "WebSocket upgrade rejected"; + bridge.close(1008, reason); + throw new Error( + `WebSocket upgrade failed with ${response.status} for ${request.url}${detail ? `: ${detail}` : ""}`, + ); + } + bridgeWebSocket(socket, bridge); + } catch (error) { + bridge.close(1011, "WebSocket connection failed"); + throw error; + } +} + /** Preserve the upgrade signal across browser `Request.clone()` calls. */ export function withWebSocketUpgrade(request: T): T { if (upgradeRequests.has(request)) return request; diff --git a/src/browser/alarm-coordinator.test.ts b/src/browser/alarm-coordinator.test.ts index b3c9eeb..29991f6 100644 --- a/src/browser/alarm-coordinator.test.ts +++ b/src/browser/alarm-coordinator.test.ts @@ -1,7 +1,17 @@ import { describe, expect, it, vi } from "vitest"; -import { ALARM_RETRY_MAX_TRIES, alarmRetryDelayMs } from "../server/alarm-scheduler"; +import { createNodeSqlProvider } from "../../backends/node-sqlite"; +import type { Timer } from "../io/io-context"; +import { + ALARM_RETRY_MAX_TRIES, + AlarmScheduler, + alarmRetryDelayMs, + type AlarmResult, +} from "../server/alarm-scheduler"; +import type { SqlDatabase } from "../util/sqlite"; import { BrowserAlarmCoordinator, + createBrowserAlarmProjector, + parseBrowserAlarmProjection, parseBrowserAlarmTransportJournal, type BrowserAlarmProjection, type BrowserPhysicalAlarm, @@ -38,6 +48,127 @@ class MemoryPhysicalAlarm implements BrowserPhysicalAlarm { } } +const START = 1_000_000; +const OK: AlarmResult = { outcome: "ok", retry: false, retryCountsAgainstLimit: false }; +const USER_FAILURE: AlarmResult = { outcome: "exception", retry: true, retryCountsAgainstLimit: true }; +const PARKED = Symbol("parked"); + +/** A scheduler clock that moves only when a test advances it. */ +class ManualTimer implements Timer { + #now = START; + #pending: { readonly at: number; readonly resolve: () => void }[] = []; + + now(): number { + return this.#now; + } + + afterDelay(ms: number, signal?: AbortSignal): Promise { + return new Promise((resolve) => { + const entry = { at: this.#now + ms, resolve }; + this.#pending.push(entry); + signal?.addEventListener("abort", () => { + this.#pending = this.#pending.filter((pending) => pending !== entry); + }); + }); + } + + async advance(ms: number): Promise { + this.#now += ms; + const due = this.#pending.filter((entry) => entry.at <= this.#now); + this.#pending = this.#pending.filter((entry) => entry.at > this.#now); + for (const entry of due) entry.resolve(); + await settle(); + } +} + +/** Every alarm hop here is a promise continuation, so one task drains them all. */ +function settle(): Promise { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +function afterQueuedWork(promise: Promise): Promise { + return Promise.race([promise, settle().then((): typeof PARKED => PARKED)]); +} + +/** A durable generation in a host table beside `_cf_ALARM`, as a browser host keeps it. */ +function nextDurableGeneration(db: SqlDatabase): number { + db.exec( + "CREATE TABLE IF NOT EXISTS alarm_projection " + + "(id INTEGER PRIMARY KEY CHECK (id = 1), generation INTEGER NOT NULL)", + [], + ); + const row = db.exec( + "INSERT INTO alarm_projection VALUES (1, 1) " + + "ON CONFLICT (id) DO UPDATE SET generation = generation + 1 RETURNING generation", + [], + ).rawRows[0]; + return Number(row?.[0]); +} + +type HostBoundary = { + deliver?(scheduledTime: number, retryCount: number): Promise; + abandon?(): Promise; + project?(projection: BrowserAlarmProjection): Promise; +}; + +/** + * One Worker lifetime: a scheduler over `db` projecting through + * `createBrowserAlarmProjector` into a service-worker coordinator. Pass the + * same database, store and physical alarm to model a Worker restart. + */ +async function startWorker( + options: HostBoundary & { + db?: SqlDatabase; + nextGeneration?: () => number | Promise; + physical?: MemoryPhysicalAlarm; + store?: MemoryTransportStore; + } = {}, +) { + const db = options.db ?? (await createNodeSqlProvider().open("alarms")); + const physical = options.physical ?? new MemoryPhysicalAlarm(); + const store = options.store ?? new MemoryTransportStore(); + const timer = new ManualTimer(); + const projections: BrowserAlarmProjection[] = []; + const wakes = createBrowserAlarmProjector({ + nextGeneration: options.nextGeneration ?? (() => nextDurableGeneration(db)), + project: async (projection) => { + projections.push(projection); + await options.project?.(projection); + await coordinator.project(projection); + }, + }); + const coordinator = new BrowserAlarmCoordinator({ + deliver: wakes.acknowledge, + now: () => timer.now(), + physical, + store, + }); + const scheduler = new AlarmScheduler({ + timer, + db, + random: () => 0, + getActor: () => ({ + deliverAlarm: async (scheduledTime, retryCount) => + (await options.deliver?.(scheduledTime, retryCount)) ?? OK, + abandonAlarm: async () => (await options.abandon?.()) ?? null, + }), + projectWake: wakes.projectWake, + }); + // Land the constructor's projection before a test controls later ones. + await settle(); + return { + acknowledge: wakes.acknowledge, + coordinator, + db, + physical, + projections, + store, + timer, + schedule: (actorId: string, when: number | null) => + scheduler.hooks(actorId).scheduleRun(when, Promise.resolve()), + }; +} + describe("BrowserAlarmCoordinator", () => { it("parses a loose durable journal and rejects invalid storage", () => { const journal = { @@ -67,6 +198,11 @@ describe("BrowserAlarmCoordinator", () => { projection: { generation: 1, when: 1 }, }), ).toBeNull(); + expect(parseBrowserAlarmProjection({ generation: 2, when: null })).toEqual({ + generation: 2, + when: null, + }); + expect(parseBrowserAlarmProjection({ generation: 2 })).toBeNull(); }); it("acknowledges a projection only after the physical alarm operation finishes", async () => { @@ -323,3 +459,274 @@ describe("BrowserAlarmCoordinator", () => { }); }); }); + +describe("createBrowserAlarmProjector", () => { + it("acknowledges an early wake that the projected future wake already covers", async () => { + const worker = await startWorker(); + const future = START + 60_000; + await worker.schedule("future", future); + + await expect(worker.acknowledge(START)).resolves.toEqual(worker.projections.at(-1)); + expect(worker.projections.at(-1)?.when).toBe(future); + }); + + it("acknowledges a cancelled wake without waiting for a delivery", async () => { + const worker = await startWorker(); + const scheduledTime = START + 60_000; + await worker.schedule("cancelled", scheduledTime); + const acknowledged = worker.acknowledge(scheduledTime); + expect(await afterQueuedWork(acknowledged)).toBe(PARKED); + + await worker.schedule("cancelled", null); + + await expect(acknowledged).resolves.toEqual(worker.projections.at(-1)); + expect(worker.projections.at(-1)?.when).toBeNull(); + }); + + it("completes a coordinator delivery once a failed attempt's retry is projected", async () => { + const release = Promise.withResolvers(); + const worker = await startWorker({ + deliver: async () => { + await release.promise; + return USER_FAILURE; + }, + }); + await worker.schedule("retry", START); + await worker.timer.advance(0); + + const firing = worker.coordinator.fire(START); + expect(await afterQueuedWork(firing)).toBe(PARKED); + release.resolve(); + + const retry = START + alarmRetryDelayMs(0); + const projection = await firing; + expect(projection).toEqual(worker.projections.at(-1)); + expect(projection?.when).toBe(retry); + expect(worker.store.journal).toEqual({ + delivery: null, + projection: worker.projections.at(-1), + }); + expect(worker.physical.creates.at(-1)).toBe(retry); + }); + + it("keeps a running delivery unacknowledged through an unrelated projection", async () => { + const release = Promise.withResolvers(); + const worker = await startWorker({ + deliver: async () => { + await release.promise; + return OK; + }, + }); + await worker.schedule("held", START); + await worker.timer.advance(0); + const delivery = worker.acknowledge(START); + const earlierWake = worker.acknowledge(START - 1); + const nextWake = START + 60_000; + + await worker.schedule("unrelated", nextWake); + + // The active alarm stays due for crash recovery even with a later wake. + expect(worker.projections.at(-1)?.when).toBe(START); + expect(await afterQueuedWork(delivery)).toBe(PARKED); + expect(await afterQueuedWork(earlierWake)).toBe(PARKED); + release.resolve(); + await expect(delivery).resolves.toMatchObject({ when: nextWake }); + await expect(earlierWake).resolves.toMatchObject({ when: nextWake }); + }); + + it("sends a failed projection again before acknowledging a consumed wake", async () => { + let unreachable = false; + const worker = await startWorker({ + project: async () => { + if (unreachable) throw new Error("the service worker was unreachable"); + }, + }); + await worker.schedule("due", START + 10_000); + unreachable = true; + await expect(worker.schedule("due", null)).rejects.toThrow("the service worker was unreachable"); + await expect(worker.acknowledge(START + 10_000)).rejects.toThrow( + "the service worker was unreachable", + ); + unreachable = false; + + // The idle scheduler will never project again, so the wake must resend it. + const projection = await worker.coordinator.fire(START + 10_000); + + expect(projection).toEqual(worker.projections.at(-1)); + expect(worker.store.journal).toEqual({ + delivery: null, + projection: { generation: projection?.generation, when: null }, + }); + }); + + it("does not let an older projection's acceptance acknowledge a newer refusal", async () => { + const held = Promise.withResolvers(); + const release = Promise.withResolvers(); + const scheduledTime = START + 60_000; + let refuseCancellation = false; + const worker = await startWorker({ + project: async (projection) => { + if (projection.when === scheduledTime) { + held.resolve(); + await release.promise; + } else if (refuseCancellation && projection.when === null) { + throw new Error("Chrome refused the newer projection"); + } + }, + }); + const scheduled = worker.schedule("held-projection", scheduledTime); + await held.promise; + const delivery = worker.acknowledge(scheduledTime); + refuseCancellation = true; + const cancelled = worker.schedule("held-projection", null); + + release.resolve(); + + await scheduled; + await expect(delivery).rejects.toThrow("Chrome refused the newer projection"); + await expect(cancelled).rejects.toThrow("Chrome refused the newer projection"); + }); + + it("does not let an older projection's refusal reject a newer one's acknowledgement", async () => { + const held = Promise.withResolvers(); + const release = Promise.withResolvers(); + const worker = await startWorker({ + project: async (projection) => { + if (projection.when !== START + 10_000) return; + held.resolve(); + await release.promise; + throw new Error("Chrome refused the older projection"); + }, + }); + const older = worker.schedule("moved", START + 10_000); + await held.promise; + const newer = worker.schedule("moved", START + 20_000); + const acknowledged = worker.acknowledge(START); + expect(await afterQueuedWork(acknowledged)).toBe(PARKED); + + release.resolve(); + + await expect(older).rejects.toThrow("Chrome refused the older projection"); + await newer; + await expect(acknowledged).resolves.toEqual(worker.projections.at(-1)); + expect(worker.projections.at(-1)?.when).toBe(START + 20_000); + }); + + it("parks acknowledgement behind a refused abandonment until its cleanup retry lands", async () => { + const retryCounts: number[] = []; + let abandonments = 0; + const worker = await startWorker({ + deliver: async (_scheduledTime, retryCount) => { + retryCounts.push(retryCount); + return USER_FAILURE; + }, + abandon: async () => { + abandonments += 1; + if (abandonments === 1) throw new Error("The actor refused the abandonment"); + return null; + }, + }); + await worker.schedule("abandoned", START); + for (let second = 0; abandonments === 0 && second < 1_000; second += 1) { + await worker.timer.advance(1_000); + } + expect(retryCounts.at(-1)).toBe(ALARM_RETRY_MAX_TRIES); + const delivered = retryCounts.length; + + // The failed cleanup keeps the alarm projected at its own time: the work + // is unfinished, so acknowledging its wake would drop the last recovery. + const acknowledged = worker.acknowledge(START); + expect(worker.projections.at(-1)?.when).toBe(START); + expect(await afterQueuedWork(acknowledged)).toBe(PARKED); + + // One bookkeeping backoff later the scheduler retries the abandonment. + await worker.timer.advance(alarmRetryDelayMs(0)); + await expect(acknowledged).resolves.toMatchObject({ when: null }); + expect(abandonments).toBe(2); + expect(retryCounts).toHaveLength(delivered); + }); + + it("sends projections one at a time and acknowledges the latest of a churning queue", async () => { + const held = Promise.withResolvers(); + const release = Promise.withResolvers(); + let holdNext = false; + const worker = await startWorker({ + project: async () => { + if (!holdNext) return; + holdNext = false; + held.resolve(); + await release.promise; + }, + }); + holdNext = true; + const earliest = worker.schedule("churn-a", START + 10_000); + await held.promise; + const churn = [ + worker.schedule("churn-b", START + 20_000), + worker.schedule("churn-c", START + 30_000), + worker.schedule("churn-d", START + 40_000), + ]; + const acknowledged = worker.acknowledge(START); + + expect(await afterQueuedWork(acknowledged)).toBe(PARKED); + expect(worker.projections.map(({ when }) => when)).toEqual([null, START + 10_000]); + release.resolve(); + await Promise.all([earliest, ...churn]); + + await expect(acknowledged).resolves.toMatchObject({ when: START + 10_000 }); + expect(worker.projections.map(({ generation }) => generation)).toEqual([1, 2, 3, 4, 5]); + }); + + it("re-arms the physical alarm after a Worker restart with a durable generation", async () => { + const first = await startWorker(); + await first.schedule("before-restart", START + 10_000); + await first.schedule("before-restart", null); + const journaled = first.store.journal?.projection.generation ?? 0; + + const durableAtSend: number[] = []; + const restarted = await startWorker({ + db: first.db, + physical: first.physical, + store: first.store, + project: async () => { + const row = first.db.exec("SELECT generation FROM alarm_projection", []).rawRows[0]; + durableAtSend.push(Number(row?.[0])); + }, + }); + await restarted.schedule("after-restart", START + 20_000); + + expect(restarted.projections[0]?.generation).toBe(journaled + 1); + expect(durableAtSend).toEqual(restarted.projections.map(({ generation }) => generation)); + expect(first.store.journal).toEqual({ + delivery: null, + projection: { generation: journaled + 2, when: START + 20_000 }, + }); + expect(first.physical.creates.at(-1)).toBe(START + 20_000); + }); + + it("draws an asynchronous generation only after the previous projection is sent", async () => { + // This counter numbers a draw when its write lands, so overlapping draws + // could number a newer wake below an older one and have it dropped. + const draws: PromiseWithResolvers[] = []; + let landed = 0; + const worker = await startWorker({ + nextGeneration: async () => { + const draw = Promise.withResolvers(); + draws.push(draw); + await draw.promise; + return (landed += 1); + }, + }); + const older = worker.schedule("moved", START + 10_000); + const newer = worker.schedule("moved", START + 20_000); + + // Land the newest pending draw first. + while (draws.length > 0) { + draws.pop()?.resolve(); + await settle(); + } + + await Promise.all([older, newer]); + expect(worker.store.journal?.projection).toEqual({ generation: 3, when: START + 20_000 }); + }); +}); diff --git a/src/browser/alarm-coordinator.ts b/src/browser/alarm-coordinator.ts index b045799..860e50a 100644 --- a/src/browser/alarm-coordinator.ts +++ b/src/browser/alarm-coordinator.ts @@ -1,6 +1,7 @@ import { ALARM_RETRY_MAX_TRIES, alarmRetryDelayMs, + type AlarmSchedulerOptions, } from "../server/alarm-scheduler"; type LooseRecord = Record; @@ -159,7 +160,99 @@ export class BrowserAlarmCoordinator { } } -function parseBrowserAlarmProjection(value: unknown): BrowserAlarmProjection | null { +/** + * The Worker half of the browser alarm protocol. Pass `projectWake` to the + * `AlarmScheduler` and return `acknowledge()` from the coordinator's + * `deliver()`. + * + * Projections leave one at a time, in call order; each draws its generation + * only after the previous one was sent, so generations rise in send order even + * when `nextGeneration()` is asynchronous. `acknowledge(scheduledTime)` resolves + * with the latest projection once the coordinator has accepted it, no delivery + * or cleanup is active, and its wake is absent or later than `scheduledTime`. + * Only the latest projection counts. If it failed, `acknowledge()` sends it + * again before waiting, so an idle scheduler does not leave the wake stuck on + * an old failure; waiters reject only when the latest projection fails. + */ +export function createBrowserAlarmProjector(options: { + /** + * Must be durable and strictly increasing across Worker restarts. The + * coordinator drops every projection older than the generation it journaled, + * so a counter that restarts lower silently stalls each later wake. Keep it + * in a host table beside the scheduler: a runtime `_cf_` table would need a + * storage-version bump and would restart below generations already journaled. + */ + nextGeneration(): number | Promise; + /** Carries one projection to `BrowserAlarmCoordinator.project()`. */ + project(projection: BrowserAlarmProjection): Promise; +}): { + projectWake: NonNullable; + acknowledge(scheduledTime: number): Promise; +} { + type Sent = { + readonly when: number | null; + readonly activeDeliveries: number; + result?: PromiseSettledResult; + }; + const waiters = new Set<{ + readonly after: number; + resolve(projection: BrowserAlarmProjection): void; + reject(reason: unknown): void; + }>(); + let latest: Sent | undefined; + let tail: Promise = Promise.resolve(); + + const settle = (): void => { + const result = latest?.result; + if (result?.status === "rejected") { + for (const waiter of waiters) waiter.reject(result.reason); + waiters.clear(); + return; + } + if (result === undefined || latest?.activeDeliveries !== 0) return; + for (const waiter of waiters) { + if (result.value.when !== null && result.value.when <= waiter.after) continue; + waiters.delete(waiter); + waiter.resolve(result.value); + } + }; + + const projectWake = (when: number | null, activeDeliveries: number): Promise => { + const sent: Sent = { when, activeDeliveries }; + latest = sent; + const projected = tail.then(async () => { + const projection = { generation: await options.nextGeneration(), when }; + await options.project(projection); + return projection; + }); + tail = projected.catch(() => {}); + const record = (result: PromiseSettledResult): void => { + sent.result = result; + settle(); + }; + void projected.then( + (value) => record({ status: "fulfilled", value }), + (reason: unknown) => record({ status: "rejected", reason }), + ); + return projected.then(() => {}); + }; + + return { + projectWake, + acknowledge(scheduledTime) { + return new Promise((resolve, reject) => { + waiters.add({ after: scheduledTime, resolve, reject }); + if (latest?.result?.status === "rejected") { + void projectWake(latest.when, latest.activeDeliveries).catch(() => {}); + } else { + settle(); + } + }); + }, + }; +} + +export function parseBrowserAlarmProjection(value: unknown): BrowserAlarmProjection | null { if (!isRecord(value)) return null; if (!isNonnegativeInteger(value.generation)) return null; if (value.when !== null && !isFiniteNumber(value.when)) return null; diff --git a/src/browser/message-port-websocket.test.ts b/src/browser/message-port-websocket.test.ts index dc44518..af21de3 100644 --- a/src/browser/message-port-websocket.test.ts +++ b/src/browser/message-port-websocket.test.ts @@ -2,7 +2,11 @@ import { describe, expect, it, vi } from "vitest"; import { createNodeSqlProvider } from "../../backends/node-sqlite"; import { createActorContainer, noFacets } from "../server/actor-container"; import type { UpgradeWebSocket } from "../browser"; -import { installWebSocketUpgradeGlobals, upgradeWebSocket } from "../browser"; +import { + connectMessagePortWebSocket, + installWebSocketUpgradeGlobals, + upgradeWebSocket, +} from "../browser"; import { MessagePortWebSocket, bridgeWebSocket, @@ -12,6 +16,55 @@ import { type MessagePortWebSocketWireMessage, } from "./message-port-websocket"; +/** Collect errors reported through `queueMicrotask` instead of failing the run with them. */ +function captureReported(): { readonly reported: unknown[]; restore(): void } { + const reported: unknown[] = []; + const nativeQueueMicrotask = globalThis.queueMicrotask; + const queue = vi.spyOn(globalThis, "queueMicrotask").mockImplementation((callback) => + nativeQueueMicrotask(() => { + try { + callback(); + } catch (error) { + reported.push(error); + } + }), + ); + return { reported, restore: () => queue.mockRestore() }; +} + +/** A 101 answer carrying `webSocket`, without installing the upgrade globals. */ +function upgradeResponse(webSocket: object): Response { + return Object.defineProperties(new Response(null), { + status: { value: 101 }, + webSocket: { value: webSocket }, + }); +} + +/** What the far end of a bridge's MessagePort receives. */ +function recordWire(port: MessagePort): MessagePortWebSocketWireMessage[] { + const wire: MessagePortWebSocketWireMessage[] = []; + port.addEventListener("message", (event: MessageEvent) => { + wire.push(event.data); + }); + port.start(); + return wire; +} + +function socketContainer(uniqueKey: string) { + return createActorContainer({ + id: "socket", + uniqueKey, + exports: {}, + env: {}, + ports: { + sql: createNodeSqlProvider(), + facets: noFacets, + alarms: { scheduleRun: async () => {} }, + timer: { now: () => Date.now(), afterDelay: () => new Promise(() => {}) }, + }, + }); +} + class MemorySocket extends EventTarget implements UpgradeWebSocket { readonly sent: MessagePortWebSocketData[] = []; readyState = MessagePortWebSocket.CONNECTING; @@ -80,6 +133,63 @@ describe("MessagePortWebSocket", () => { channel.port2.close(); }); + it("reports a throwing onmessage without skipping listeners or later frames", async () => { + const { reported, restore } = captureReported(); + const channel = new MessageChannel(); + const socket = new MessagePortWebSocket("ws://actor.test", channel.port1, false); + const arrived: unknown[] = []; + channel.port1.addEventListener("message", (event) => arrived.push(event.data)); + const failure = new Error("onmessage failed"); + const handled: MessagePortWebSocketData[] = []; + const listened: MessagePortWebSocketData[] = []; + socket.onmessage = (event) => { + handled.push(event.data); + if (event.data === "first") throw failure; + }; + socket.addEventListener("message", (event) => listened.push(event.data)); + try { + for (const data of ["first", "second"]) { + channel.port2.postMessage({ type: "message", data } satisfies MessagePortWebSocketWireMessage); + } + await vi.waitFor(() => expect(arrived).toHaveLength(2)); + socket.open(); + for (const data of ["third", "fourth"]) { + channel.port2.postMessage({ type: "message", data } satisfies MessagePortWebSocketWireMessage); + } + + const frames = ["first", "second", "third", "fourth"]; + await vi.waitFor(() => expect(listened).toEqual(frames)); + expect(handled).toEqual(frames); + expect(reported).toEqual([failure]); + expect(socket.readyState).toBe(MessagePortWebSocket.OPEN); + } finally { + restore(); + socket.close(); + channel.port2.close(); + } + }); + + it("dispatches a close that arrives while connecting and discards queued frames", async () => { + const channel = new MessageChannel(); + const socket = new MessagePortWebSocket("ws://actor.test", channel.port1, false); + const events: string[] = []; + socket.addEventListener("message", (event) => events.push(`message ${String(event.data)}`)); + socket.addEventListener("close", (event) => events.push(`close ${event.code} ${event.reason}`)); + + channel.port2.postMessage({ type: "message", data: "queued" } satisfies MessagePortWebSocketWireMessage); + channel.port2.postMessage({ + type: "close", + code: 4409, + reason: "worker stopped", + } satisfies MessagePortWebSocketWireMessage); + await vi.waitFor(() => expect(socket.readyState).toBe(MessagePortWebSocket.CLOSED)); + socket.open(); + await new Promise((resolve) => setTimeout(resolve, 0)); + + expect(events).toEqual(["close 4409 worker stopped"]); + channel.port2.close(); + }); + it("carries frames and close state in both directions", async () => { const channel = new MessageChannel(); const left = new MessagePortWebSocket("ws://actor.test", channel.port1); @@ -122,18 +232,7 @@ describe("MessagePortWebSocket", () => { const NativeRequest = globalThis.Request; const NativeResponse = globalThis.Response; installWebSocketUpgradeGlobals(); - const container = await createActorContainer({ - id: "socket", - uniqueKey: "message-port-websocket-test", - exports: {}, - env: {}, - ports: { - sql: createNodeSqlProvider(), - facets: noFacets, - alarms: { scheduleRun: async () => {} }, - timer: { now: () => Date.now(), afterDelay: () => new Promise(() => {}) }, - }, - }); + const container = await socketContainer("message-port-websocket-test"); const channel = new MessageChannel(); const bridge = new MessagePortWebSocket("ws://actor.test", channel.port1, false); const client = new MessagePortWebSocket("ws://actor.test", channel.port2); @@ -172,11 +271,46 @@ describe("MessagePortWebSocket", () => { channel.port2.close(); }); + it("closes the runtime peer with the client's code and drops frames queued behind it", async () => { + const container = await socketContainer("message-port-websocket-close"); + const pair = await container.run(() => { + const sockets = new container.globals.WebSocketPair(); + sockets[1].accept(); + return sockets; + }); + const peerCloses: number[] = []; + pair[1].addEventListener("close", (event) => peerCloses.push(event.code)); + const channel = new MessageChannel(); + const bridge = new MessagePortWebSocket("ws://actor.test", channel.port1, false); + const wire = recordWire(channel.port2); + try { + bridgeWebSocket(upgradeWebSocket(upgradeResponse(pair[0]))!, bridge); + await container.run(() => pair[1].send("before close")); + await vi.waitFor(() => expect(wire).toHaveLength(1)); + await container.run(() => { + pair[1].send("queued behind the close"); + bridge.close(1008, "client left"); + }); + await container.drainWaitUntil(); + + await vi.waitFor(() => expect(peerCloses).toEqual([1008])); + expect(wire).toEqual([ + { type: "message", data: "before close" }, + { type: "close", code: 1008, reason: "client left" }, + ]); + } finally { + container.abort(); + channel.port2.close(); + } + }); + it("serves WebSocket constructors over a broker MessagePort", async () => { const broker = new MessageChannel(); const runtime = new MemorySocket(); const connection = Promise.withResolvers(); - const connect = vi.fn(() => connection.promise); + const connect = vi.fn(async (bridge: MessagePortWebSocket) => { + bridgeWebSocket(await connection.promise, bridge); + }); const stop = serveMessagePortWebSockets(broker.port2, connect); const BrokeredWebSocket = createMessagePortWebSocketConstructor(broker.port1); const client = new BrokeredWebSocket("ws://actor.test"); @@ -198,11 +332,15 @@ describe("MessagePortWebSocket", () => { await vi.waitFor(() => expect(client.readyState).toBe(MessagePortWebSocket.CLOSED)); }); - it("closes a brokered client when its runtime connection fails", async () => { + it("closes a brokered client and reports the failure when connecting throws", async () => { + const { reported, restore } = captureReported(); + const failure = new Error("WebSocket already accepted"); + const runtime = new MemorySocket(); + runtime.accept = () => { + throw failure; + }; const broker = new MessageChannel(); - const connect = vi.fn(async () => { - throw new Error("route failed"); - }); + const connect = vi.fn(async (bridge: MessagePortWebSocket) => bridgeWebSocket(runtime, bridge)); const stop = serveMessagePortWebSockets(broker.port2, connect); const BrokeredWebSocket = createMessagePortWebSocketConstructor(broker.port1); broker.port1.postMessage({ type: "connect", url: "ws://invalid.test" }); @@ -210,21 +348,28 @@ describe("MessagePortWebSocket", () => { const closed = new Promise((resolve) => { client.addEventListener("close", resolve, { once: true }); }); - - await expect(closed).resolves.toMatchObject({ - code: 1011, - reason: "WebSocket connection failed", - wasClean: true, - }); - expect(connect).toHaveBeenCalledOnce(); - expect(connect).toHaveBeenCalledWith("ws://actor.test"); - stop(); + try { + await expect(closed).resolves.toMatchObject({ + code: 1011, + reason: "WebSocket connection failed", + wasClean: true, + }); + expect(connect).toHaveBeenCalledOnce(); + expect(connect).toHaveBeenCalledWith(expect.any(MessagePortWebSocket), "ws://actor.test"); + expect(runtime.readyState).toBe(MessagePortWebSocket.CLOSED); + await vi.waitFor(() => expect(reported).toEqual([failure])); + } finally { + restore(); + stop(); + } }); it("closes a runtime socket that finishes connecting after the server stops", async () => { const broker = new MessageChannel(); const connection = Promise.withResolvers(); - const connect = vi.fn(() => connection.promise); + const connect = vi.fn(async (bridge: MessagePortWebSocket) => { + bridgeWebSocket(await connection.promise, bridge); + }); const stop = serveMessagePortWebSockets(broker.port2, connect); const BrokeredWebSocket = createMessagePortWebSocketConstructor(broker.port1); const client = new BrokeredWebSocket("ws://actor.test"); @@ -237,4 +382,95 @@ describe("MessagePortWebSocket", () => { await vi.waitFor(() => expect(runtime.readyState).toBe(MessagePortWebSocket.CLOSED)); await vi.waitFor(() => expect(client.readyState).toBe(MessagePortWebSocket.CLOSED)); }); + + it("closes 1011 and rejects when no route matches", async () => { + const channel = new MessageChannel(); + const bridge = new MessagePortWebSocket("ws://actor.test/missing", channel.port1, false); + const wire = recordWire(channel.port2); + + await expect( + connectMessagePortWebSocket(bridge, "ws://actor.test/missing", async () => null), + ).rejects.toThrow("No WebSocket route matched http://actor.test/missing"); + await vi.waitFor(() => + expect(wire).toEqual([{ type: "close", code: 1011, reason: "No WebSocket route matched" }]), + ); + channel.port2.close(); + }); + + it.each([ + { name: "keeps a 123-byte refusal", refusal: "x".repeat(123), reason: "x".repeat(123) }, + { name: "drops a 124-byte refusal", refusal: "x".repeat(124), reason: "WebSocket upgrade rejected" }, + { name: "drops a non-ASCII refusal", refusal: "Non autorisé", reason: "WebSocket upgrade rejected" }, + ])( + "closes 1008 and $name as the reason", + async ({ refusal, reason }) => { + const channel = new MessageChannel(); + const bridge = new MessagePortWebSocket("ws://actor.test", channel.port1, false); + const wire = recordWire(channel.port2); + + await expect( + connectMessagePortWebSocket( + bridge, + "ws://actor.test", + async () => new Response(refusal, { status: 403 }), + ), + ).rejects.toThrow("WebSocket upgrade failed with 403 for http://actor.test/"); + await vi.waitFor(() => expect(wire).toEqual([{ type: "close", code: 1008, reason }])); + channel.port2.close(); + }, + ); + + it("closes 1011 and rethrows when routing throws", async () => { + const failure = new Error("route failed"); + const channel = new MessageChannel(); + const bridge = new MessagePortWebSocket("ws://actor.test", channel.port1, false); + const wire = recordWire(channel.port2); + + await expect( + connectMessagePortWebSocket(bridge, "ws://actor.test", async () => { + throw failure; + }), + ).rejects.toBe(failure); + await vi.waitFor(() => + expect(wire).toEqual([{ type: "close", code: 1011, reason: "WebSocket connection failed" }]), + ); + channel.port2.close(); + }); + + it("serves fetch-routed sockets and forwards a refusal reason to the client", async () => { + const { reported, restore } = captureReported(); + const broker = new MessageChannel(); + const runtime = new MemorySocket(); + const requests: (readonly [string, string | null])[] = []; + const route = async (request: Request): Promise => { + requests.push([request.url, request.headers.get("Upgrade")]); + return new URL(request.url).pathname === "/refused" + ? new Response("Unauthorized", { status: 401 }) + : upgradeResponse(runtime); + }; + const stop = serveMessagePortWebSockets(broker.port2, (bridge, url) => + connectMessagePortWebSocket(bridge, url, route), + ); + const BrokeredWebSocket = createMessagePortWebSocketConstructor(broker.port1); + const refused = new BrokeredWebSocket("ws://actor.test/refused"); + const refusal = new Promise((resolve) => { + refused.addEventListener("close", resolve, { once: true }); + }); + const client = new BrokeredWebSocket("wss://actor.test/agents/counter"); + try { + await expect(refusal).resolves.toMatchObject({ code: 1008, reason: "Unauthorized" }); + await vi.waitFor(() => expect(client.readyState).toBe(MessagePortWebSocket.OPEN)); + client.send("request"); + await vi.waitFor(() => expect(runtime.sent).toEqual(["request"])); + expect(requests).toEqual([ + ["http://actor.test/refused", "websocket"], + ["https://actor.test/agents/counter", "websocket"], + ]); + await vi.waitFor(() => expect(reported).toHaveLength(1)); + expect(String(reported[0])).toContain("WebSocket upgrade failed with 401"); + } finally { + restore(); + stop(); + } + }); }); diff --git a/src/browser/message-port-websocket.ts b/src/browser/message-port-websocket.ts index 73de0ae..3a72232 100644 --- a/src/browser/message-port-websocket.ts +++ b/src/browser/message-port-websocket.ts @@ -148,7 +148,15 @@ export class MessagePortWebSocket extends EventTarget implements RawWebSocket { } #emit(event: E, handler: ((event: E) => void) | null): void { - handler?.(event); + // Report a throwing handler the way EventTarget reports a listener, so it + // cannot skip the listeners behind it or abandon a frame flush. + try { + handler?.(event); + } catch (error) { + queueMicrotask(() => { + throw error; + }); + } this.dispatchEvent(event); } } @@ -238,10 +246,16 @@ export function createMessagePortWebSocketConstructor( }; } -/** Serve brokered MessagePort sockets from real in-worker socket endpoints. */ +/** + * Serve brokered MessagePort sockets from real in-worker socket endpoints. + * `connect` bridges each client's server-side endpoint, for example with + * `connectMessagePortWebSocket()` from `@mcp-b/do-runtime/browser`; once it + * resolves, the client is told the socket is open. A bridge that `stop()` + * closed refuses a late socket by closing it with 1001. + */ export function serveMessagePortWebSockets( port: MessagePort, - connect: (url: string) => Promise, + connect: (bridge: MessagePortWebSocket, url: string) => Promise, ): () => void { const bridges = new Set(); const listener = (event: MessageEvent): void => { @@ -250,17 +264,16 @@ export function serveMessagePortWebSockets( const bridge = new MessagePortWebSocket(request.url, request.port, false); bridges.add(bridge); bridge.addEventListener("close", () => bridges.delete(bridge), { once: true }); - void connect(request.url).then( - (socket) => { - if (!bridges.has(bridge)) { - socket.close(1001, "host stopped"); - return; - } - bridgeWebSocket(socket, bridge); + void connect(bridge, request.url) + .then(() => { request.port.postMessage({ type: "open" } satisfies MessagePortWebSocketReadyMessage); - }, - () => bridge.close(1011, "WebSocket connection failed"), - ); + }) + .catch((error: unknown) => { + bridge.close(1011, "WebSocket connection failed"); + queueMicrotask(() => { + throw error; + }); + }); }; port.addEventListener("message", listener); port.start(); diff --git a/src/browser/offscreen-document.test.ts b/src/browser/offscreen-document.test.ts index 7787c40..2993db0 100644 --- a/src/browser/offscreen-document.test.ts +++ b/src/browser/offscreen-document.test.ts @@ -67,4 +67,99 @@ describe("OffscreenDocumentCoordinator", () => { await expect(coordinator.ensure()).rejects.toBe(failure); expect(close).not.toHaveBeenCalled(); }); + + it("probes readiness once inside the shared flight", async () => { + const probe = Promise.withResolvers(); + const ready = vi.fn(() => probe.promise); + const coordinator = new OffscreenDocumentCoordinator({ + close: vi.fn(), + create: vi.fn(async () => {}), + exists: vi.fn(async () => false), + isOccupiedError: vi.fn(() => false), + ready, + }); + let settled = false; + const callers = Promise.all([coordinator.ensure(), coordinator.ensure()]).then(() => { + settled = true; + }); + + await vi.waitFor(() => expect(ready).toHaveBeenCalledOnce()); + await Promise.resolve(); + expect(settled).toBe(false); + probe.resolve(); + await callers; + expect(ready).toHaveBeenCalledOnce(); + }); + + it("replaces an existing document that never answers, once", async () => { + const mute = new Error("the document did not answer"); + const calls: string[] = []; + const coordinator = new OffscreenDocumentCoordinator({ + close: async () => { + calls.push("close"); + }, + create: async () => { + calls.push("create"); + }, + exists: async () => true, + isOccupiedError: () => false, + ready: async () => { + calls.push("ready"); + if (calls.length === 1) throw mute; + }, + replaceUnready: async (error) => { + calls.push(error === mute ? "replace" : "replace?"); + return true; + }, + }); + + await coordinator.ensure(); + + expect(calls).toEqual(["ready", "replace", "close", "create", "ready"]); + }); + + it("propagates a readiness failure after its one replacement", async () => { + const replacementMute = new Error("the replacement did not answer either"); + const ready = vi + .fn() + .mockRejectedValueOnce(new Error("the document did not answer")) + .mockRejectedValueOnce(replacementMute); + const replaceUnready = vi.fn(() => true); + const create = vi.fn(async () => {}); + const coordinator = new OffscreenDocumentCoordinator({ + close: vi.fn(async () => {}), + create, + exists: vi.fn(async () => true), + isOccupiedError: vi.fn(() => false), + ready, + replaceUnready, + }); + + await expect(coordinator.ensure()).rejects.toBe(replacementMute); + expect(ready).toHaveBeenCalledTimes(2); + expect(replaceUnready).toHaveBeenCalledOnce(); + expect(create).toHaveBeenCalledOnce(); + }); + + it("rethrows a readiness failure the host declines to replace", async () => { + const mute = new Error("the document is still booting"); + const close = vi.fn(); + const create = vi.fn(); + const replaceUnready = vi.fn(async () => false); + const coordinator = new OffscreenDocumentCoordinator({ + close, + create, + exists: vi.fn(async () => true), + isOccupiedError: vi.fn(() => false), + ready: vi.fn(async () => { + throw mute; + }), + replaceUnready, + }); + + await expect(coordinator.ensure()).rejects.toBe(mute); + expect(replaceUnready).toHaveBeenCalledWith(mute); + expect(close).not.toHaveBeenCalled(); + expect(create).not.toHaveBeenCalled(); + }); }); diff --git a/src/browser/offscreen-document.ts b/src/browser/offscreen-document.ts index 655144e..d5b91ba 100644 --- a/src/browser/offscreen-document.ts +++ b/src/browser/offscreen-document.ts @@ -3,12 +3,24 @@ export type OffscreenDocumentAdapter = { create(): Promise; close(): Promise; isOccupiedError(error: unknown): boolean; + /** + * Resolves once the document answers, for a document just created and one + * that already existed. A document can load before its listeners register. + * It must settle, rejecting on its own timeout: every `ensure()` caller + * shares the flight it runs in. + */ + ready?(): Promise; + /** + * Decides whether a document that failed `ready()` is closed and recreated. + * The replacement gets one more `ready()`; declining rethrows the failure. + */ + replaceUnready?(error: unknown): boolean | Promise; }; /** * Keeps one browser offscreen document alive across concurrent callers and a - * stale, unlisted document slot. Readiness and application policy stay with - * the embedding host. + * stale, unlisted document slot. Concurrent callers share one creation and one + * readiness probe; when to give up on a mute document stays with the host. */ export class OffscreenDocumentCoordinator { #ensuring: Promise | undefined; @@ -23,7 +35,19 @@ export class OffscreenDocumentCoordinator { } async #ensureOnce(): Promise { - if (await this.adapter.exists()) return; + if (!(await this.adapter.exists())) await this.#create(); + if (this.adapter.ready === undefined) return; + try { + await this.adapter.ready(); + } catch (error) { + if (!(await this.adapter.replaceUnready?.(error))) throw error; + await this.adapter.close(); + await this.#create(); + await this.adapter.ready(); + } + } + + async #create(): Promise { try { await this.adapter.create(); } catch (error) { diff --git a/src/gate.test.ts b/src/gate.test.ts index 5f2d032..b542d05 100644 --- a/src/gate.test.ts +++ b/src/gate.test.ts @@ -5,6 +5,7 @@ import { BrokenActorError, IoContext, requireInputLock, + tryCurrentIoContext, type Actor, type Timer, } from "./io/io-context"; @@ -219,6 +220,81 @@ describe("__gate", () => { }); describe("transformed await resume", () => { + test.each([ + ["a held storage await", (context: IoContext) => context.awaitIoWithInputLock(Promise.resolve(1))], + ["an already-settled storage result", () => Promise.resolve(1)], + ["a plain value", () => 1], + ])("resumes %s twice in one checkpoint before a queued event runs", async (_, awaited) => { + await portHop(); + const context = newContext(); + const order: string[] = []; + const first = context.run(async () => { + order.push("first:start"); + // Both awaits resume in this checkpoint, so they share one implicit transaction. + let crossedTask = false; + void portHop().then(() => { crossedTask = true; }); + __resumeAwait(await __gateAwait(awaited(context))); + __resumeAwait(await __gateAwait(awaited(context))); + expect(crossedTask).toBe(false); + order.push("first:resumed"); + }); + const queued = context.run(() => { + order.push("queued"); + }); + + await Promise.all([first, queued]); + expect(order).toEqual(["first:start", "first:resumed", "queued"]); + }); + + test("waits out a critical section that settles an await captured outside it", async () => { + await portHop(); + const context = newContext(); + const order: string[] = []; + const outside = Promise.withResolvers(); + const waiter = context.run(async () => { + __resumeAwait(await __gateAwait(outside.promise)); + order.push("outside"); + }); + await portHop(); + await context.run(() => + context.blockConcurrencyWhile(async () => { + outside.resolve(); + __resumeAwait(await __gateAwait(portHop())); + order.push("section"); + }), + ); + await waiter; + expect(order).toEqual(["section", "outside"]); + }); + + test("keeps a second actor's lock without taking over another actor's checkpoint", async () => { + const owner = newContext(); + const second = newContext(); + const order: string[] = []; + let secondResumed!: Promise; + let ambient: IoContext | undefined; + await owner.run(async () => { + __resumeAwait(await __gateAwait(Promise.resolve())); + // The owner's continuation is current; the second actor settles a held await inside it. + await second.run(() => { + secondResumed = (async () => { + __resumeAwait(await __gateAwait(Promise.resolve())); + requireInputLock(second, "second actor continuation"); + order.push("second:resumed"); + })(); + }); + void second.run(() => { + order.push("second:queued"); + }); + for (let turn = 0; turn < 4; turn++) await Promise.resolve(); + ambient = tryCurrentIoContext(); + }); + + await secondResumed; + await expect.poll(() => order).toEqual(["second:resumed", "second:queued"]); + expect(ambient).toBe(owner); + }); + test.each(["fulfillment", "rejection"])("stays busy through a foreign await and queued %s", async (outcome) => { const actor = new TestActor(); const context = new IoContext(actor, timer); @@ -241,8 +317,10 @@ describe("transformed await resume", () => { expect(context.waitUntilTaskCount()).toBeGreaterThan(0); expect(drained).toBe(false); - // Another event can enter during the wait, then hold up its publication. + // Another event can enter during the wait, then hold up its publication. The foreign + // promise settles after that event's slice hands off, outside any checkpoint of this actor. const lock = await context.run(() => context.getInputLock()); + await portHop(); try { if (outcome === "fulfillment") pending.resolve(42); else pending.reject(failure); diff --git a/src/gate.ts b/src/gate.ts index 7ee2889..ea7c255 100644 --- a/src/gate.ts +++ b/src/gate.ts @@ -81,8 +81,13 @@ function currentPublication(): PublicationReservation | undefined { return Reflect.get(globalThis, CURRENT_PUBLICATION) as PublicationReservation | undefined; } +/** One actor owns a checkpoint's ambient: refuse only another actor's continuation or publication. */ function reservePublication(context: IoContext): PublicationReservation | undefined { - if (tryCurrentContinuation() !== undefined || currentPublication() !== undefined) return undefined; + const continuation = tryCurrentContinuation(); + const publication = currentPublication(); + if ((continuation ?? context) !== context || (publication?.context ?? context) !== context) { + return undefined; + } const reservation = { context }; Reflect.set(globalThis, CURRENT_PUBLICATION, reservation); atCheckpointEnd(() => clearPublication(reservation)); @@ -100,6 +105,7 @@ function publishOutcome( promise: Promise, finish: (outcome: Outcome, reservation: PublicationReservation) => Result, ): Promise { + const criticalSection = context.getCriticalSection(); const result = new Promise((resolve, reject) => { const publish = context.makeTransformReentryCallback((outcome: Outcome) => { const reservation = reservePublication(context); @@ -109,13 +115,40 @@ function publishOutcome( } resolve(finish(outcome, reservation)); }); + // Settled while this actor still holds a lock in the captured section: a storage call, a + // plain value, or a resumption the runtime already admitted. workerd continues such an await + // inside the same checkpoint, keeping the lock and the implicit transaction (§1.2, §1.7.1). + // Anything else re-enters through a fresh slice. + const settle = (outcome: Outcome): void => { + if (!context.hasCurrent() || context.getCriticalSection() !== criticalSection) { + schedulePublication({ publish: () => publish(outcome), reject }); + return; + } + if (context.isAborted()) { + reject(context.getAbortReason()); + return; + } + const reservation = reservePublication(context); + if (reservation === undefined) { + // Another actor's continuation owns this checkpoint. Wait for the next task without + // releasing the lock, so no other event of this actor runs first; the implicit + // transaction still commits at the hand-off. + const lock = context.getInputLock(); + schedulePublication({ + publish: () => context.run(() => settle(outcome), { input: lock }), + reject, + }); + return; + } + try { + resolve(finish(outcome, reservation)); + } catch (exception) { + reject(exception); + } + }; void promise.then( - (value) => { - schedulePublication({ publish: () => publish({ ok: true, value }), reject }); - }, - (exception: unknown) => { - schedulePublication({ publish: () => publish({ ok: false, exception }), reject }); - }, + (value) => settle({ ok: true, value }), + (exception: unknown) => settle({ ok: false, exception }), ); }); // Keep the actor alive through both the foreign wait and queued publication. @@ -146,9 +179,11 @@ function resumeWithContext(context: IoContext, promise: Promise): Promise< } /** - * Resolve one transformed await per task, inside a fresh actor slice. Admission + * Resolve a transformed await in a later task, inside an actor slice: a fresh one + * when the actor held no lock in the captured section as the promise settled, or + * the retained lock of one that waited out another actor's checkpoint. Admission * attempts are independent so a blocked actor cannot stall the actor that will - * unblock it. The task boundary keeps each continuation ambient isolated. + * unblock it. The task boundary keeps each actor's continuation ambient isolated. */ function schedulePublication(publication: Publication): void { const channel = new MessageChannel(); diff --git a/src/index.ts b/src/index.ts index ff4fada..15b69f4 100644 --- a/src/index.ts +++ b/src/index.ts @@ -17,8 +17,10 @@ * reason: it is what `server/` implements and `api/` consumes, and the seam a * consumer fills is `FacetHost` below it. * - * Refusal messages used by conformance tests are exported so their specified - * fail-closed behavior cannot drift. `createActorContainer` is asynchronous + * Refusal messages are exported as the public names of the runtime's + * fail-closed refusals. Unit tests pin them where they are thrown; the + * conformance suite cannot, because its workerd lane imports nothing from + * `src/`. `createActorContainer` is asynchronous * because `SqlDatabaseProvider.open` is asynchronous; a returned container has * usable state and no hidden half-started storage phase. */ @@ -37,10 +39,12 @@ export { BrokenActorError, type Timer } from "./io/io-context"; export { CanceledError } from "./io/io-gate"; /** * The Worker Loader (§1.11, decision 15). Exported where the `api/` classes are - * not, and for the reason `AlarmScheduler` is: this one is a **binding**, so a - * host has to construct it and put it in `env` — upstream compiles it from - * `Global::WorkerLoader{channel}` the same way (`server/workerd-api.c++:748`) — - * where every other `api/` class reaches a consumer through `container.state`. + * not because this one is a **binding**: the host puts it in `env` — upstream + * compiles it from `Global::WorkerLoader{channel}` (`server/workerd-api.c++:748`) + * — where every other `api/` class reaches a consumer through `container.state`. + * A host does not construct it: the constructor takes the `IoContext` this + * facade withholds, so `container.workerLoader()` builds one bound to the right + * context, and the class is exported as that method's return type. * * What a host supplies is `IsolateChannelFactory`, which is the whole substrate * seam: `loadIsolate` and the calling worker's own outbound. The scaffolding's @@ -199,6 +203,7 @@ export type { Scheduler, SchedulerWaitOptions, } from "./api/global-scope"; +export { ACTOR_SCOPE_GLOBALS } from "./api/actor-scope-globals"; export { actorScopeBindings, FOREIGN_SLICE_MESSAGE, diff --git a/src/io/actor-sqlite.ts b/src/io/actor-sqlite.ts index 495f506..5e1c41c 100644 --- a/src/io/actor-sqlite.ts +++ b/src/io/actor-sqlite.ts @@ -8,7 +8,7 @@ * - `onWrite` taking the output-gate lock at the first must-confirm write, * one lock per flush batch; * - `transactionSync` as SAVEPOINT/RELEASE/ROLLBACK TO with a depth counter, - * plus the async-callback guard today's version lacks; + * plus an async-callback guard upstream has no twin of; * - alarm arm/consume/deferred-deletion, and `deleteAll`. * * Sole `ActorCacheInterface` implementation, exactly as on workerd-with-SQLite. diff --git a/src/io/io-context.ts b/src/io/io-context.ts index 56f8cbc..2e34ea9 100644 --- a/src/io/io-context.ts +++ b/src/io/io-context.ts @@ -122,7 +122,10 @@ import { */ export interface Timer { now(): number; - /** `kj::Timer::afterDelay`. The signal replaces kj's cancel-by-drop. */ + /** + * `kj::Timer::afterDelay`. The signal replaces kj's cancel-by-drop: on abort the + * promise may stay pending or reject, but it must not resolve. + */ afterDelay(ms: number, signal?: AbortSignal): Promise; } diff --git a/src/server/actor-container.test.ts b/src/server/actor-container.test.ts index a21224a..69d08f8 100644 --- a/src/server/actor-container.test.ts +++ b/src/server/actor-container.test.ts @@ -9,6 +9,9 @@ * all because it has no upstream body. */ +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; import { describe, expect, expectTypeOf, test, vi } from "vitest"; import { createNodeSqlProvider } from "../../backends/node-sqlite"; import { __gateAwait, __resumeAwait } from "../gate"; @@ -19,6 +22,7 @@ import { markWebSocketUsed } from "../api/web-socket"; import type { Timer } from "../io/io-context"; import { CanceledError } from "../io/io-gate"; import type { SqlDatabase, SqlDatabaseProvider } from "../util/sqlite"; +import { RUNTIME_STORAGE_VERSION } from "../util/sqlite-migrations"; import { FacetDeletionReceiptStore } from "./facet-deletion"; import type { ActorClassChannel } from "../io/io-channels"; import type { IsolateChannelFactory } from "../api/worker-loader"; @@ -390,6 +394,28 @@ describe("newDatabaseIndexFile", () => { }); }); +describe("createActorContainer", () => { + test.each(["root", "facets"])( + "closes what it opened when %s was written by a newer release", + async (name) => { + const directory = mkdtempSync(join(tmpdir(), "do-runtime-container-")); + try { + const sql = createNodeSqlProvider({ directory }); + const seeded = await sql.open(name); + seeded.exec(`PRAGMA user_version = ${RUNTIME_STORAGE_VERSION + 1}`, []); + seeded.close(); + await expect( + createActorContainer(options({ ports: { ...options().ports, sql } })), + ).rejects.toThrow("written by a newer @mcp-b/do-runtime"); + // `exportSnapshot` refuses while any database handle is still open. + await expect(sql.exportSnapshot()).resolves.toBeDefined(); + } finally { + rmSync(directory, { recursive: true, force: true }); + } + }, + ); +}); + describe("the composition", () => { test.each([ { outcome: "success", fails: false, failure: undefined }, @@ -935,6 +961,24 @@ describe("facets", () => { expect(host.deleted).toEqual([]); }); + test("a host abort that throws is handed to trackFacetTeardown, not left unhandled", async () => { + const host = new RecordingFacetHost(); + host.abort = () => { + throw new Error("host abort exploded"); + }; + const { container, stub } = await counterContainer({ + ports: { sql: createNodeSqlProvider(), alarms, facets: host, timer }, + }); + const track = vi.spyOn( + container as unknown as { trackFacetTeardown(work: Promise): void }, + "trackFacetTeardown", + ); + await openFacet(stub, "child"); + await stub.abortFacet("child"); + await vi.waitFor(() => expect(track).toHaveBeenCalledOnce()); + await expect(track.mock.calls[0]?.[0]).rejects.toThrow("host abort exploded"); + }); + test("delete aborts first, then removes the subtree deepest-first", async () => { const host = new RecordingFacetHost(); const { container, stub } = await counterContainer({ diff --git a/src/server/actor-container.ts b/src/server/actor-container.ts index 5489140..6373ddd 100644 --- a/src/server/actor-container.ts +++ b/src/server/actor-container.ts @@ -243,7 +243,7 @@ export const noFacets: FacetHost = { }; /** - * The five ports. Each one is a seam workerd itself takes as a constructor + * The six ports. Each one is a seam workerd itself takes as a constructor * input; a port that would exist only because our code is currently shaped * badly is an invented seam and was rejected. Rejected, for the record: * a transport port (one implementation per substrate, forever), a logger port @@ -252,13 +252,13 @@ export const noFacets: FacetHost = { * browser), and an addressing-strategy port (unnecessary once the package * speaks facet ids). * - * A fifth, `isolates?: IsolateHost`, was here from the scaffolding and Section 7b + * An `isolates?: IsolateHost` port was here from the scaffolding and Section 7b * removed it. A Worker Loader is a **binding**, not a port: upstream builds it * from `Global::WorkerLoader{channel}` alongside every other binding * (`server/workerd-api.c++:748`) and it reaches an application through `env`, * exactly as `DurableObjectNamespace` and `ctx.exports` already do here. A host - * constructs `WorkerLoader` over its own `IsolateChannelFactory` and puts it in - * `env`; the container never sees one. See `api/worker-loader.ts`'s header. + * gets one over its own `IsolateChannelFactory` from `container.workerLoader()` + * and puts it in `env`. See `api/worker-loader.ts`'s header. */ export type ActorPorts = { sql: SqlDatabaseProvider; @@ -782,9 +782,11 @@ class ActorTree implements FacetTree { // Both outcomes chained: an abort is not conditional on the placement before it succeeding. const done = pending === undefined ? operation() : pending.then(operation, operation); this.#operations.set(id, done); - void done.finally(() => { + // Not `finally`, which would re-reject a failed operation where nothing handles it. + const clear = (): void => { if (this.#operations.get(id) === done) this.#operations.delete(id); - }); + }; + void done.then(clear, clear); } async subtreeOperationBarrier(id: FacetId): Promise { @@ -1175,7 +1177,12 @@ class FacetManagerImpl implements FacetManager { */ #teardown(entry: FacetEntry, description: string): void { this.#tree.runOperation(entry.id, async () => { - this.#host.abort(entry.id, description); + try { + this.#host.abort(entry.id, description); + } catch (error) { + // `abort` should return, not throw; keep a host that throws readable in waitUntilStatus(). + this.#container.trackFacetTeardown(Promise.reject(error)); + } }); } @@ -1745,17 +1752,25 @@ export async function createActorContainer( options: ActorContainerOptions, ): Promise { const actorDb = await options.ports.sql.open(ACTOR_DATABASE_NAME); - ensureRuntimeStorageVersion(actorDb, ACTOR_DATABASE_NAME); - const db = new SqliteDatabase(actorDb); - - // ← `ensureFacetTreeIndex()`'s `KJ_REQUIRE(parent == kj::none, "only 'root' may - // ensureFacetTreeIndex()")` (`server.c++:2704`). A facet is handed the root's rather than - // opening one, which is also why this is the only `open` a facet container makes. - if (options.facet !== undefined) { - return new ActorContainerImpl(options, db, undefined, options.facet.tree); + let facetDb: SqlDatabase | undefined; + // The runtime owns what it opened: a refusal below must not strand the handles. + try { + ensureRuntimeStorageVersion(actorDb, ACTOR_DATABASE_NAME); + const db = new SqliteDatabase(actorDb); + + // ← `ensureFacetTreeIndex()`'s `KJ_REQUIRE(parent == kj::none, "only 'root' may + // ensureFacetTreeIndex()")` (`server.c++:2704`). A facet is handed the root's rather than + // opening one, which is also why this is the only `open` a facet container makes. + if (options.facet !== undefined) { + return new ActorContainerImpl(options, db, undefined, options.facet.tree); + } + facetDb = await options.ports.sql.open(FACET_DATABASE_NAME); + ensureRuntimeStorageVersion(facetDb, FACET_DATABASE_NAME); + const tree = new ActorTree(facetDb, options.ports.facets); + return new ActorContainerImpl(options, db, tree, tree); + } catch (error) { + facetDb?.close(); + actorDb.close(); + throw error; } - const facetDb = await options.ports.sql.open(FACET_DATABASE_NAME); - ensureRuntimeStorageVersion(facetDb, FACET_DATABASE_NAME); - const tree = new ActorTree(facetDb, options.ports.facets); - return new ActorContainerImpl(options, db, tree, tree); } diff --git a/src/server/alarm-scheduler.test.ts b/src/server/alarm-scheduler.test.ts index 0f8ab50..c0ef398 100644 --- a/src/server/alarm-scheduler.test.ts +++ b/src/server/alarm-scheduler.test.ts @@ -12,6 +12,7 @@ * and the jitter source is the `random` constructor option for the same reason. */ +import { setTimeout as sleep } from "node:timers/promises"; import { describe, expect, test } from "vitest"; import { createNodeSqlProvider } from "../../backends/node-sqlite"; import { @@ -1083,6 +1084,20 @@ describe("checkTimestamp", () => { await timer.advance(6_000); expect(actor.deliveries).toHaveLength(1); }); + + test("a timer that rejects on abort is cancellation: no unhandled rejection, no task failure", async () => { + // node:timers/promises rejects an aborted wait with AbortError. + const scheduler = new AlarmScheduler({ + timer: { now: Date.now, afterDelay: (ms, signal) => sleep(ms, undefined, { signal }) }, + db: await newDatabase(), + getActor: () => new FakeActor(), + }); + scheduler.setAlarm("a", Date.now() + 60_000); + scheduler.setAlarm("a", Date.now() + 120_000); // replaces the WAITING entry: aborts its wake + scheduler.deleteAlarm("a"); // aborts the replacement's wake + await settle(); + expect(scheduler.taskFailure()).toBeUndefined(); + }); }); // ======================================================================================= diff --git a/src/server/alarm-scheduler.ts b/src/server/alarm-scheduler.ts index 20c054a..34cdd58 100644 --- a/src/server/alarm-scheduler.ts +++ b/src/server/alarm-scheduler.ts @@ -606,7 +606,11 @@ export class AlarmScheduler { async #checkTimestamp(delay: number, scheduledTime: number, signal: AbortSignal): Promise { let remaining = delay; for (;;) { - await this.#timer.afterDelay(remaining, signal); + // An aborted wake is kj's cancel-by-drop, whether the timer leaves it pending or rejects it. + await this.#timer.afterDelay(remaining, signal).catch((error: unknown) => { + if (!signal.aborted) throw error; + }); + if (signal.aborted) return; // Since we are waiting on timer.afterDelay, it's possible that timer.now() was behind // the real time by a few ms, leading to premature alarm() execution. This checks it the current diff --git a/src/util/sqlite-migrations.ts b/src/util/sqlite-migrations.ts index 49e451b..ad70e47 100644 --- a/src/util/sqlite-migrations.ts +++ b/src/util/sqlite-migrations.ts @@ -54,20 +54,6 @@ export type RuntimeMigration = (db: SqlDatabase) => void; /** `MIGRATIONS[i]` takes a database from storage version `i + 1` to `i + 2`. */ const MIGRATIONS: readonly RuntimeMigration[] = []; -/** - * Bring one just-opened runtime database to `RUNTIME_STORAGE_VERSION`. Called - * by every seam that opens a runtime database, before anything reads it, with - * the database's own name so a refusal says which file it is about. The last - * two parameters exist for the tests in this module's test file; every real - * caller takes the shipped defaults. - * - * A version newer than this release refuses — the analogue of - * `hasCurrentSqliteTable`'s refusal, with the one remedy named. A version 0 - * database is from before versioning existed (the same shape as version 1) or - * a fresh file; both enter the chain at 1. Pending steps and the stamp commit - * as one transaction, so a failed step leaves the file exactly as it was and - * the container placement fails with the step's error. - */ /** * Refuse a snapshot image stamped by a newer release at the import seam, where * the operation that brought the file in is the one that fails — instead of at @@ -91,6 +77,20 @@ export function requireImportableRuntimeStorage( } } +/** + * Bring one just-opened runtime database to `RUNTIME_STORAGE_VERSION`. Called + * by every seam that opens a runtime database, before anything reads it, with + * the database's own name so a refusal says which file it is about. The last + * two parameters exist for the tests in this module's test file; every real + * caller takes the shipped defaults. + * + * A version newer than this release refuses — the analogue of + * `hasCurrentSqliteTable`'s refusal, with the one remedy named. A version 0 + * database is from before versioning existed (the same shape as version 1) or + * a fresh file; both enter the chain at 1. Pending steps and the stamp commit + * as one transaction, so a failed step leaves the file exactly as it was and + * the container placement fails with the step's error. + */ export function ensureRuntimeStorageVersion( db: SqlDatabase, name: string, diff --git a/src/util/sqlite.ts b/src/util/sqlite.ts index f7559e6..21e8a12 100644 --- a/src/util/sqlite.ts +++ b/src/util/sqlite.ts @@ -28,9 +28,7 @@ * `SqliteDatabase`, exactly as upstream's take a `SqliteDatabase&`. * * `transactionSync` is NOT here — it lives in `io/actor-sqlite.ts` as - * SAVEPOINT/RELEASE with a depth counter, exactly as upstream has it. Today - * both browser and Node adapters duplicate `BEGIN IMMEDIATE`, which is why a - * nested call is a live SQLite error (§2.4). Moving it inward fixes that. + * SAVEPOINT/RELEASE with a depth counter, exactly as upstream has it. * * Not ported, because the substrate has no equivalent: the `Regulator` / * authorizer machinery (there is no untrusted-SQL path in `util/`, and @@ -128,8 +126,7 @@ export interface SqlDatabase { * * On the backend rather than above it because only the backend knows how to * recreate its own file, and because the alternative — enumerating and - * dropping every table — is the fragile dance today's `storage.ts` performs, - * complete with an FTS5 shadow-table ordering hazard its comment documents. + * dropping every table — is fragile. * The `SqlDatabase` reference stays valid across the call; what changes is * the file behind it. */ @@ -237,9 +234,9 @@ export type QueryOptions = { * Raised when SQLite has rolled back an open transaction on its own. Upstream * hands this to `ActorSqlite`, which treats it as fatal; §1.6 is why — a * storage failure this severe destroys the object rather than being survived. - * Until `io/actor-sqlite.ts` wires it to `onBroken`, latching it and refusing - * every subsequent statement is what keeps a caller from reading through a - * cache that is knowingly wrong. + * `io/actor-sqlite.ts` breaks the output gate on it; latching it here and + * refusing every subsequent statement keeps a caller from reading through a + * cache that is knowingly wrong in the meantime. */ export class SqliteCriticalError extends Error { override readonly name = "SqliteCriticalError"; diff --git a/src/vite.test.ts b/src/vite.test.ts index 478e72e..7d23ec3 100644 --- a/src/vite.test.ts +++ b/src/vite.test.ts @@ -261,3 +261,13 @@ describe("doRuntimeAwaitTransform", () => { await expect(actorBuild([])).rejects.toThrow("/actor.js: 0/1"); }); }); + +test("from source, the plugin leaves its injected imports to the host's resolution", async () => { + // The package's built siblings resolve these; this repository's own lanes alias them to + // source instead, and must keep that identity. `scripts/check-package.mjs` pins the built side. + const hook = doRuntimeAwaitTransform({ asyncContext: true }).resolveId; + if (typeof hook !== "object") throw new Error("expected an ordered resolveId hook"); + for (const id of ["@mcp-b/do-runtime/gate", "@mcp-b/do-runtime/browser/async-hooks"]) { + expect(await Reflect.apply(hook.handler, {}, [id, undefined, {}])).toBeNull(); + } +}); diff --git a/src/vite.ts b/src/vite.ts index 0b785b4..ba5e6aa 100644 --- a/src/vite.ts +++ b/src/vite.ts @@ -1,3 +1,4 @@ +import { fileURLToPath } from "node:url"; import MagicString from "magic-string"; import { createFilter, @@ -7,6 +8,11 @@ import { type FilterPattern, type Plugin, } from "vite"; +// `.js` here and in actor-scope-globals.ts keeps Vite's bundle config loader from printing its +// four-line "unsupported by `configLoader: 'native'`" warning on every config load from source. +// The native loader cannot load this file from source either way (that needs `.ts` specifiers +// and `allowImportingTsExtensions`), and the built dist is unaffected. +import { ACTOR_SCOPE_GLOBALS } from "./api/actor-scope-globals.js"; const MARKER = "/* @do-runtime-gated */"; const IMPORT = @@ -14,7 +20,7 @@ const IMPORT = const OXC_ASYNC_GENERATOR = /^@oxc-project\+runtime@[^/]+\/helpers\/esm\/wrapAsyncGenerator\.js$/; function correctAsyncGeneratorReturn(code: string, id: string) { - // Oxc 0.144.0 repeats .return() after an await in finally, skipping cleanup. + // Oxc 0.149.0 repeats .return() after an await in finally, skipping cleanup. // Match Babel's distinction between await (k=0) and delegated yield (k=1): // https://github.com/babel/babel/blob/main/packages/babel-helpers/src/helpers/wrapAsyncGenerator.ts // Keep this shape check until Vite's bundled Oxc helper incorporates that fix. @@ -41,6 +47,18 @@ function correctAsyncGeneratorReturn(code: string, id: string) { }; } +/** + * The prelude a same-realm host prefixes to each facet bundle. It binds the actor globals to the + * scope registered at `globalThis[registry][scope]`, where `scope` is the bundle URL's `scope` + * search parameter. + */ +export function facetScopeBanner({ registry }: { registry: string }): string { + return `const __facetKey = new URL(import.meta.url).searchParams.get("scope"); +const __facetScope = globalThis[${JSON.stringify(registry)}]?.[__facetKey]; +if (__facetScope === undefined) throw new Error(\`facet module has no scope named \${__facetKey}\`); +const { ${ACTOR_SCOPE_GLOBALS.join(", ")} } = __facetScope;`; +} + export interface DoRuntimeAwaitTransformOptions { include?: FilterPattern; exclude?: FilterPattern; @@ -48,6 +66,21 @@ export interface DoRuntimeAwaitTransformOptions { asyncContext?: boolean; } +/** + * The package's own files for the imports this plugin injects: siblings of the built + * `dist/vite.js`. Node loads the plugin from its real path, which with Vite's default + * `preserveSymlinks` is the path Vite resolves an application import of the same subpath to, + * so both share one module instance. Run from source, the plugin leaves them to the host. + */ +const INJECTED_MODULES = new Map( + import.meta.url.endsWith(".js") + ? Object.entries({ + "@mcp-b/do-runtime/gate": "./gate.js", + "@mcp-b/do-runtime/browser/async-hooks": "./browser/async-hooks.js", + }).map(([id, path]): [string, string] => [id, fileURLToPath(new URL(path, import.meta.url))]) + : [], +); + function patterns(pattern: FilterPattern | undefined): readonly (string | RegExp)[] { if (pattern === undefined || pattern === null) return []; return typeof pattern === "string" || pattern instanceof RegExp ? [pattern] : pattern; @@ -107,6 +140,9 @@ export function doRuntimeAwaitTransform(options?: DoRuntimeAwaitTransformOptions return { name: "do-runtime-await-transform", enforce: "post", + // Ahead of aliases and Vite's own resolver, which cannot find this package from inside a + // strict pnpm dependency. + resolveId: { order: "pre", handler: (id) => INJECTED_MODULES.get(id) ?? null }, configResolved(config) { development = config.command === "serve"; }, diff --git a/tsconfig.base.json b/tsconfig.base.json index 6d990df..c27e726 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -1,7 +1,7 @@ { // Shared by every src/ project. Two deliberate absences enforce the reverse // boundary structurally rather than by review: - // - no "DOM" in lib, so nothing here can reach document/navigator; + // - no "DOM" in lib, so nothing here can reach document or window; // - no chrome-types in types, so chrome.* does not typecheck. // The runtime has no host-side dependencies in package.json for the same // reason: an import of one does not resolve at all. diff --git a/vendor/agents/docs/rook-0.23-migration.md b/vendor/agents/docs/rook-0.23-migration.md index 9697ad2..ec10115 100644 --- a/vendor/agents/docs/rook-0.23-migration.md +++ b/vendor/agents/docs/rook-0.23-migration.md @@ -49,9 +49,9 @@ there is no partial upgrade. repo root the same lanes are `pnpm sdk:build`, `pnpm sdk:check`, `pnpm sdk:test`. 2. **A `rook-sdk-` pre-release is cut from the merged commit.** - `pnpm sdk:pack` builds and packs the six packages into `dist/sdk/` under + `pnpm sdk:pack` builds and packs the six packages into `.sdk-pack/` under their upstream names; attach those tarballs to a GitHub release tagged - `rook-sdk-` at the tested commit (`do-runtime/README.md:509-515`). + `rook-sdk-` at the tested commit (see the "Development" section of `do-runtime/README.md`). Publish a new tag rather than replacing an existing release's assets. 3. **A rollback lever exists.** That means the previous `rook-sdk-` tarballs _and_ exported OPFS database files for any profile that will wake on