feat: send hydration data as tagged JSON text

[KyleJune]

Summary

Implements the founder decision on udibo/udibo#974 (2026-09-11): the HTML hydration payload becomes JSON text with type tags (payload version: 3); data requests keep CBOR and streaming.

Base64 of CBOR is about a third larger than the bytes it carries and hides the page's text from brotli, so every docs-style page ships its body twice at a poor ratio. Measured on prose pages built from this repo's own docs (brotli quality 5, same markup, only the payload swapped):

page	payload bytes v2 / v3	brotli document v2 / v3	ratio to markup v2 / v3
routing.md	35663 / 28398	22077 / 12218	2.522 / 1.396
state-management.md	36383 / 28711	21501 / 11829	2.546 / 1.401
error-handling.md	18099 / 14443	12706 / 7022	2.493 / 1.378

Changes

• src/_serialization.ts: toTaggedJson / fromTaggedJson wrap the existing processValue / restoreValue, which are unchanged. The JSON form decodes into exactly the tree cbor2 decoding produces, so types behave as they do today. Tags cover Date (epoch ms, invalid dates included), undefined, NaN / Infinity / -Infinity / -0, bigint, and Juniper's CBOR tags 40000–40004 carried by number. Other details:
  • A plain object with its own $t or __proto__ key is written as an escaped entry list, so data can never be read as a tag. The payload is evaluated as a JS object literal, where a "__proto__" key would otherwise set the prototype instead of making a property.
  • bigint collapses exactly as CBOR does: safe-range values become numbers. The rule is taken from cbor2 itself, so the first document load and later data requests give a loader the same types.
  • Lone surrogates become U+FFFD, as CBOR does, which also matches the SSR markup.
  • Functions and symbols throw, as CBOR does.
  • SerializedHydrationData is a union of v2 and v3. deserializeHydrationData decodes both and throws on any other version.
  • processHydrationData is split out so tests can build a v2 payload from the same input.
• toInlineScriptJson (used by src/_server.tsx) keeps escaping < and now also escapes U+2028 / U+2029. Loader data is now readable text in the script, so this escaping is its only defence.
• docs/state-management.md: new "How Values Travel" section describing v3, the escaping, CBOR for data requests, and v2 compatibility. Tightened the rule for what registered serializers may return.

Testing

• src/_serialization_hydration.test.ts:
  • Parity table. One table-driven test puts every value through both v2 and v3, each embedded as the document embeds it (evaluated as a JS literal), and compares the observed results: type, own keys, prototype, error class and exposed message, and promise outcome. It covers 44 cases: Dates, bigint boundaries, non-finite numbers, -0, undefined keys, array holes, errors and HttpError exposure, registerError, registerType, promises, $t and __proto__ escapes, Map/Set/RegExp/URL/typed-array flattening, and lone surrogates.
  • Golden v2 fixture. A payload captured from 0.11.5 still hydrates.
  • Refusals. Unknown versions and unknown tags throw.
• src/_server_hydration.test.tsx:
  • Script breakout. A real rendered page with </script>, <!--, <script>, </SCRIPT >, U+2028 and U+2029 in loader data keeps all of it inside the hydration script. The first </script the HTML parser meets is Juniper's own.
  • Data requests. They still answer application/cbor with native CBOR types, and deferred data still answers application/cbor-stream.
  • Size. The v3 document compresses smaller than the v2 document for the same page.
• Mutation check. Each mutation below was applied alone and made its tests fail:
  • dropping the $t escape
  • dropping the __proto__ escape
  • dropping the U+2028 escape
  • dropping the < escape
  • decoding v3 without the Date tag
  • dropping the bigint collapse
  • dropping v2 decoding
  • switching data requests to JSON
• deno task check and deno task test both green.

Compatibility

The build handshake from #115 does not cover a cached v2 document. X-Juniper-Build is compared only on GET data responses after hydration, while /build/main.js is not fingerprinted and is served private, no-cache, must-revalidate. So a v2 document restored from cache loads the new bundle and decodes its payload before any handshake. The v2 branch of the decoder keeps that page hydrating; the handshake then reloads it on its first data request.

The reverse case is not covered: a new v3 document meeting an old bundle, which can only happen during a rolling deploy when two instances serve different main.js. An old client cannot read v3, so that page stays server-rendered without hydrating until reload.

For the reviewer

• Security review. This changes what reaches an inline script: loader data is now readable text rather than base64. It needs the security review udibo/udibo#974 asked for.

• Open question. v3 flattens Map / Set / URL / typed arrays inside a registered serializer's output (processValue never recurses there), while CBOR data requests keep them. Loader-level values flatten on both paths as before. The docs now tell serializers to return plain data; whether to make v3 refuse such output instead is open.

• Deno coverage bug (for the founder, not filed upstream). Deno 2.9.6 panics while collecting coverage (runtime/coverage.rs:54:44, serde_json "unexpected end of hex escape" on the Profiler.takePreciseCoverage response) for a module where an anonymous class expression comes before an object literal keyed by a lone-surrogate escape. Minimal repro, run with deno test --coverage=cov repro.test.ts:

  const a = [() => new (class {})(), () => ({ "\uD800": 1 })];
  Deno.test("t", () => { a.forEach((f) => f()); });

  Without the class it passes, and so does a lone surrogate in a value or in a key built at runtime. The parity table's lone-surrogate row therefore builds its surrogates with String.fromCharCode (d821733). This could be reported to denoland/deno with that repro.

Closes

Nothing in this repo. udibo/udibo#974 stays open after this merges, until udibo adopts the release.

🤖 Generated with Claude Code

[KyleJune]

test: build lone surrogates at runtime in hydration table — d821733

Fixes the Ubuntu coverage step (run 34596435636), where every test passed and then Deno 2.9.6 panicked at runtime/coverage.rs:54 with unexpected end of hex escape. The trigger was the parity-table row ["a lone surrogate", () => ({ "key\uD800": "value\uDC00" })] placed after the anonymous class Box row: V8's Profiler.takePreciseCoverage response for that module contains an escape serde_json rejects. The row now builds both surrogates with String.fromCharCode, so it round-trips the same key and value.

Evidence (Deno 2.9.6 locally, CI's exact deno task test --coverage):

• PR head b883efc: panics. origin/main 3bf15d8: completes.
• _serialization_hydration.test.ts alone with coverage: panics before the fix, completes after. Deleting only that row also completes; deleting the separator row or the toInlineScriptJson block does not help.
• Counts unchanged: that file 3 passed (53 steps); the full suite 39 passed (443 steps), with and without coverage. deno coverage --detailed and --lcov succeed. deno task check is green.

[KyleJune]

Security review at b883efc: merge from a security standpoint.

• Script breakout: the < and U+2028/U+2029 escape runs on the whole {version, data, publicEnv} string. 22 hostile values (every </script variant, <!--, -->, lone surrogates, NUL/C0, BOM, a 1 MB run of </script>) were placed as values, keys, nested values, registered-type output, deferred promises and error messages, rendered through a real server, parsed with jsdom and evaluated as a module: exactly one hydration script each time, nothing injected, exact round trip.
• Tag confusion: any object with an own $t key is escaped at every level; arrays never decode as tags; unknown tags can only come from the server.
• Prototype keys: the __proto__ escape holds through the embed. A separate, pre-existing problem in the decoders (assigning with =) is filed as its own issue; it does not block this PR.
• CSP/nonce: unchanged.
• Mutations: dropping the < escape, the __proto__ escape or the $t escape each turns the named tests red.

Merge blocker unrelated to security: the Ubuntu job's coverage collection crashes in Deno (unexpected end of hex escape); a fix is in progress on this branch.

Adoption in udibo/udibo owes threat-model updates: T10's "cannot break out because the payload is base64" and T9's "not a demonstrated request-controlled injection path" must be rewritten now that loader data is readable text protected by escaping.