Skip to content

Implement end-to-end encrypted workspace sync - #252

Open
SinaKhalili wants to merge 20 commits into
mainfrom
feat/encrypted-sync
Open

Implement end-to-end encrypted workspace sync#252
SinaKhalili wants to merge 20 commits into
mainfrom
feat/encrypted-sync

Conversation

@SinaKhalili

@SinaKhalili SinaKhalili commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implement the complete first end-to-end encrypted workspace-sync feature in this PR. Work is in progress; user-facing activation is not enabled for general users — the vault is reachable only through the opt-in Settings → Encryption page, the iOS Encryption screen, and zeron vault. No encryption format has shipped, so earlier draft encodings are not compatibility targets.

Implemented so far

  • RFC 0001 and delivery plan (through section 17.11) covering the per-profile vault, trust model, recovery, migration, and release requirements; §7.8 records the implemented control-plane record formats.
  • Rust and native CryptoKit primitive adapters, Ed25519 encoding hardening, strict canonical signed-record wrappers, and per-record-derived-key content sealing (Rust ⇄ Swift verified).
  • Key management (Rust zeron-crypto): RFC 9180 HPKE (X25519/HKDF-SHA256/AES-256-GCM, pinned to the RFC A.1 vectors) on ring + curve25519-dalek 4.1.3 (archive reviewed; cfg-only build script); signed parent-linked membership policy (genesis, add, revoke, recovery rotation/transition) with strict transition rules; epoch keyring; HPKE keyring envelopes; per-object content keys wrapped under epoch keys; grouped checksummed Base32 recovery kit; enrollment proofs and genesis-bound pairing codes.
  • Edge control plane: VaultRoom Durable Object per (org, user) at /vault/:orgId/* — CAS'd signed membership history, keyring/object-key envelopes (first writer wins), device-possession enrollment; every write verified with WebCrypto Ed25519 against the stored head and re-checked after async work. No unlock endpoint, no plaintext key, no administrative override.
  • Legacy fences and framing (RFC §12.2): once a profile has a vault, its plaintext chat/registry/session rooms, blob and device-sidecar writes, and existing sockets refuse further writes; encrypted generations accept only correctly framed sealed records (wrapper + payload format/suite/purpose). The device relay carries only authenticated-channel frames for encrypted profiles and closes any socket that sends plaintext RPC.
  • Engine vault service: AES-GCM-protected local state under a device-held key (macOS Keychain / systemd credential / explicit key file; otherwise Locked, never plaintext); explicit phases; setup with recovery kit and explicit kit confirmation (RecoveryConfirmationRequired until then); comparison-code pairing and approval; revocation with epoch rotation; kit recovery; journaled membership intents and owed envelopes that resume after crashes; seal/open material bound to pinned membership revisions. Vault* RPCs (local-only, never relayed), Settings → Encryption page, zeron vault CLI.
  • Sealed chat transport: sink outcomes (Applied / KeyUnavailable / AuthenticationFailed / Unsupported / StorageFailed) — the cursor never advances over an unverified row; paused/resume instead of redial loops; sealed durable outbox with ack-based retirement and re-sealing of stale-policy batches; a separate encrypted room per chat (chat2/{id}-e1, doc epoch 3); sealed checkpoints, frontiers, tails, tool blobs (writer + reader), and diff sidecars; vault-gated joins.
  • Registry field envelopes (ES-10 first cut): values sealed under one registry object key per epoch with row/field/clock bound in the authenticated plaintext; withheld rows hold the cursor; rejected fields never erase verified ones; separate encrypted registry generation (/registry/{org}/e1).
  • Authenticated row tombstones (ES-10, G4 first half): a delete in the encrypted generation carries a sealed row lifecycle proof (content purpose 9, plaintext {kind, id, op, hlc}); the room refuses bare deletes, stores the proof on the tombstone, serves it with the row, and drops it on revival; readers in Rust, TypeScript, and Swift accept a tombstone only with a verified proof newer than the verified baseline and accept a revival only with a verified field newer than the tombstone — a relay can neither delete nor resurrect a row.
  • Authenticated device channel (ES-11): device RPC for enrolled profiles rides a Noise XX session (Noise_XX_25519_AESGCM_SHA256 via snow 0.10.0, ring-accelerated) between the two devices' vault identities — statics are the membership-published X25519 keys, the prologue binds vault id and generation, device ids travel inside the encrypted handshake, and each side accepts its peer only if the pinned membership head lists that id with that key (re-checked on every inbound frame, so revocation ends the session). The host relay runs the responder per client conn and serves a secured RPC instance (Vault* stays local-only); the link cache never dials plaintext once enrolled; the initiator verifies the host before revealing its own identity. Chunking, an 8 MiB frame cap, and a 2^32 messages-per-direction budget bound the session.
  • iOS: membership verification, CryptoKit HPKE envelopes, recovery kit, and enrollment proofs against the shared fixture; a Keychain-protected mobile vault with journaled recovery; the app decides its transport mode from the vault before any store dials (legacy / encrypted / blocked — never plaintext by default); sealed per-chat outbox, outcome-bearing apply with paused/resume, sealed registry values; an Encryption screen for approval-by-code and kit recovery.
  • Restored the exact pinned GPUI checkout and normal workspace verification; two pre-existing engine test mismatches corrected without changing runtime behavior.

Remaining before completion and activation

  • iOS relay client over the device channel (the phone keeps refusing relay RPC until a Rust bridge exists); device sidecars beyond refusal; iOS readers for sealed tails/blobs; per-session audit of which member issued which command.
  • ES-10 rest: GC/full-sync rules with proofs beyond the gcFloor resync and the reviewed rekey/cutover design; accepted-history closure after revocation (G3); immutable checkpoint provenance.
  • Atomic client-side migration cutover for existing plaintext profiles, authoritative legacy ownership mappings, migration journal, consented cleanup, and retention verification (ES-14). The current fences refuse plaintext writes but do not migrate history.
  • Live two-device pairing on hardware, iOS background/lock lifecycle, performance baselines, end-to-end leakage canaries, and independent security review.

A valid signature alone does not prove encryption, current membership, or freshness. Full key admission and aggregate cryptographic usage/restore bounds remain release-review items. Key roots are freshly generated on rotation, never relabeled. Unrelated local Xcode project/scheme/workspace edits are excluded.

Test plan

  • cargo check --workspace --all-targets passes, including UI/GPUI.
  • cargo test -p zeron-crypto -p zeron-sync -p zeron-engine --features zeron-sync/mock-server passes (crypto 45 + fixture incl. the Noise channel and the lifecycle record; doc 89; sync 39 + 9 + 14 incl. a forged tombstone ignored by the encrypted peer; sync unit/integration incl. pause/resume, sealed-batch acks, registry codec canaries and withhold-then-resume; engine unit suites, crash-recovery fault tests, and every integration suite); cargo test -p zeron-rpc (16 relay scenarios over the in-memory DO, incl. five device-channel cases with zero plaintext frames observed on the relay).
  • npm --prefix edge run typecheck && npm --prefix edge run test (51 Node tests incl. vault-records conformance and gates; 19 workerd tests incl. fencing, framing, chan frames crossing the relay while plaintext closes 4403, and lifecycle-proof gating on the encrypted registry).
  • Live end-to-end against wrangler dev --var AUTH_MODE:dev (crates/engine/tests/vault_e2e.rs): pairing with comparison codes (wrong code refused), cross-device seal/open through control-plane object keys, revocation to epoch 2, kit recovery to epoch 3, catch-up, registry field envelopes with slot-substitution/plaintext rejection, and the device channel through a real DeviceRoom (paired members exchange RPC, a plaintext client is closed by the relay, a device from another vault cannot complete the handshake, a revoked member is cut off and cannot redial).
  • Swift: native fixture probe (scripts/check-vault-fixture.swift, 29 checks) — CryptoKit HPKE opens Rust-sealed keyring envelopes; hashes, epochs, pairing codes agree.
  • iOS: xcodebuild simulator build; ZeronTests 106 unit tests + 11 UI tests pass on an iPhone 17 Pro simulator.
  • Complete native device, migration, and release-security verification.

Generated with Devin


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Devin Review

Suggested .rules additions

  • Crypto retries must resend persisted signed ciphertext bytes, not regenerate signatures: CryptoKit Ed25519 signing is not byte-deterministic on the tested platform.

Release Notes:

  • N/A

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@SinaKhalili

Copy link
Copy Markdown
Collaborator Author

Milestone: Ed25519 input hardening (01eeb0f)

  • Reproduced identity-point signature acceptance in both raw adapters and signed-record verification, then added matching Rust/Swift rejection rules.
  • Added shared point/scalar boundary and rejection vectors. Existing standard and fresh CryptoKit signatures remain valid.
  • Verification: 17 Rust tests, 11 isolated native XCTest cases, shared conformance harness, iOS-target typecheck, Clippy, and formatting passed. Full subgroup/key admission remains a release review item.

Maintainer direction is now recorded: this PR delivers the complete feature as one unit; no compatibility layers for unshipped encryption drafts. Notable milestones will be committed and pushed here automatically. Existing plaintext migration and safety requirements remain in scope.

Next: restore the pinned GPUI checkout and normal workspace verification, then continue encrypted-content and durable-outbox implementation.

@SinaKhalili

Copy link
Copy Markdown
Collaborator Author

Milestone: normal workspace verification restored

The GPUI failure was a missing offline checkout. cargo fetch --locked --target aarch64-apple-darwin restored the exact pinned sources; no versions, security settings, or GPUI code changed.

  • cargo test -p zeron-crypto --locked --offline: 17 passed from the real workspace.
  • cargo check --workspace --all-targets --locked --offline: passed, including the UI/GPUI dependency graph (existing warnings remain).
  • cargo test -p zeron-doc -p zeron-sync -p zeron-rpc -p zeron-engine --locked --offline: passed; live-service tests retain their existing ignored status.

The broader run reproduced a pre-existing heartbeat-test race. Its completion predicate could match the user message before the assistant ran. The committed correction waits for assistant completion without removing any assertion.

Continuing with actual encrypted-content sealing/opening and durable outbox persistence. Earlier draft encryption encodings are not compatibility targets.

@SinaKhalili

Copy link
Copy Markdown
Collaborator Author

Milestone: encrypted content records

Implemented actual Rust/CryptoKit content sealing and opening using scoped root keys and a fresh per-record derived AES-256-GCM key. The record context, key ID, purpose, salt, and revision are bound into HKDF/AAD and the ciphertext is signed. Public sealing APIs do not accept caller-provided nonces/salts; retry bytes are immutable.

  • Swift decrypts the Rust fixture; Rust decrypts fresh Swift ciphertext.
  • 25 Rust tests and 15 native XCTest cases pass, covering wrong keys/scopes/authors/purposes, entropy/size failures, fresh seals, tampering/truncation, and GCM rejection even after a modified payload/context is validly re-signed.
  • The all-target workspace check and iOS-target typecheck pass.
  • No dependency upgrades or compatibility layer for prior drafts.

Next milestone is atomic snapshot/outbox persistence. Key management, all live content paths, pairing/recovery, migration, and release review remain in progress; this is still one feature/PR, not an activation release.

@SinaKhalili

Copy link
Copy Markdown
Collaborator Author

Milestone: atomic encrypted chat outbox

Integrated ciphertext persistence into DocsStore:

  • One IMMEDIATE/FULL SQLite transaction saves the snapshot, cursor, and immutable encrypted batch; callers receive a batch receipt only after commit.
  • Duplicate retries do not rewrite newer snapshots. Replay verifies signatures and the exact caller-supplied current binding; stale epochs/policies remain queued.
  • Exact-byte acknowledgments, chat-purpose/frame limits, bounded reads, and per-profile byte quotas. Corruption or persistence failure preserves pending work and last-good state.

Nine outbox integration tests pass, including restart/ACK, injected mid-transaction failure, cursor overflow/regression, stale/corrupt records, full-width epochs, wrong-purpose/oversized records, and concurrent quota contention. The all-target workspace check and crypto/doc/sync/RPC/engine suites pass; native crypto tests and cross-language encryption checks remain green.

The transport callers still need to consume these receipts under the vault gate. Secure key storage, bootstrap/key envelopes, pairing/recovery/revocation, native durable storage, remaining content paths, and plaintext migration are next in this same PR.

@SinaKhalili SinaKhalili changed the title Add encrypted-sync RFC and cross-language crypto foundations Implement end-to-end encrypted workspace sync Sep 5, 2026
@SinaKhalili
SinaKhalili force-pushed the feat/encrypted-sync branch 2 times, most recently from ec18e0e to f1c5eff Compare September 8, 2026 14:34
@SinaKhalili

Copy link
Copy Markdown
Collaborator Author
image IMG_1088 image

SinaKhalili and others added 20 commits September 9, 2026 11:39
Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add bounded primitive adapters and canonical signed-record wrappers before
connecting encryption to live sync. Share positive and malformed fixtures
between Rust and Swift, and record the remaining protocol review gates.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Prevent identity-point signatures from passing the primitive and signed
record adapters. Share point/scalar encoding checks and regression vectors
across Rust and Swift, retaining full key admission as a release gate.

Record the single-PR delivery and milestone commit workflow.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The previous predicate could finish on the user entry and inspect the
journal before reasoning was emitted. Record restored pinned GPUI sources
and successful normal workspace verification.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add Rust and CryptoKit content sealing/opening with fresh OS entropy,
domain-separated HKDF/AAD, immutable signed ciphertext, and scoped root
keys. Verify cross-language decryption and reject tampering even when a
modified payload is re-signed by an authorized test key.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Return transport receipts only after a FULL-durability SQLite transaction.
Keep ciphertext immutable across retries, verify scoped replay, preserve
stale/corrupt work, and enforce transport and outbox limits without losing
the last durable snapshot.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…overy kit

- RFC 9180 base-mode HPKE (DHKEM X25519/HKDF-SHA256, AES-256-GCM) built on
  ring + curve25519-dalek, pinned to the RFC A.1 vectors.
- Signed parent-linked membership policy records: genesis, add/revoke device,
  recovery rotation, and recovery transitions with strict transition rules.
- Workspace keyring codec, keyring envelopes to devices/recovery authority,
  and per-object content-key wrapping under epoch keys.
- Grouped, checksummed Base32 recovery kit with labeled key derivation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…sealed chat transport

Edge (TypeScript):
- VaultRoom Durable Object (vault1/{org}/{user}): signed membership history
  with parent CAS, keyring envelopes, first-writer-wins object keys, and
  device-possession enrollment requests; every write verified with WebCrypto
  Ed25519 against the stored head. /vault/:orgId/* routes, bindings, v4 migration.
- vault-records.ts: strict deterministic-CBOR codec + transition rules
  mirroring the Rust policy; conformance test against the shared fixture.

Engine (Rust):
- vault::store: AES-GCM-protected local state under a device-held key
  (macOS Keychain, systemd credential, or explicit key file; otherwise
  Locked, never a plaintext fallback).
- vault::service: explicit phases (Unavailable/Locked/NotEnrolled/Pending/
  Ready/KeyUpdateRequired/VerificationFailed/Revoked); setup with recovery
  kit, comparison-code pairing (code bound to the genesis hash), approval,
  revocation with epoch rotation and owed-envelope retry, kit recovery, and
  seal/open material bound to pinned membership revisions.
- chat2: ApplyOutcome-bearing sink with paused/resume semantics (cursor never
  advances over unverified rows), sealed durable outbox with ack-based
  retirement and re-sealing of stale-policy batches, encrypted room
  generation (chat2/{id}-e1, doc epoch 3), sealed checkpoints/frontiers/
  tails/blobs/diffs, vault-gated joins.
- Vault* RPC methods, Settings -> Encryption page, and the vault CLI subcommand.
- Live e2e (crates/engine/tests/vault_e2e.rs) against wrangler dev: two
  devices pair, seal/open, revoke, and a third recovers with the kit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ely; record control-plane decisions

- VaultPolicy.swift: policy payload codec, membership chain verification with
  the same transition rules as policy.rs, enrollment proofs, pairing codes.
- VaultEnvelope.swift: keyring codec, HPKE (CryptoKit) keyring envelope open,
  object-key unwrap, recovery kit parsing and labeled key derivation.
- VaultControlPlaneTests + scripts/check-vault-fixture.swift consume the
  Rust-generated fixture (29 native checks pass; iOS simulator typecheck).
- RFC 0001 §7.8 records the implemented control-plane record formats; plan
  section 17 records the slice, verification, and remaining work.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- zeron-sync RegistryCodec seam: every pushed value is sealed and every
  received value opened (hello state, HTTPS pull, live broadcasts). A batch
  is never half-sealed; rows whose key is not held are withheld and the doc
  cursor is held (RegistryDoc::hold_cursor) so they are re-delivered;
  fields that fail verification are dropped, never the row's verified
  fields or clocks. Mock-server tests cover relay-side canaries, per-field
  merge granularity, and withhold-then-resume.
- Engine VaultRegistryCodec: purpose RegistryField under one registry
  object key per epoch; the authenticated plaintext names row/field/clock
  so a moved value is rejected; null is an authenticated deletion marker.
- Encrypted profiles join the separate registry generation
  /registry/{org}/e1 (reg1e1/{org}/{user}), gated on the vault being
  ready with sealing material prepared before the first push.
- Live e2e: registry envelopes seal/open across devices through the
  control plane; slot substitution and plaintext are rejected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ery kits

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… screen

- AppModel decides the transport mode from the vault before any store
  dials: an explicit absent vault permits the legacy rooms, an approved
  membership with the current epoch key permits the encrypted generations,
  every other state blocks sync. Sign-out invalidates the config; the
  foreground re-checks pending approvals. Mode changes rebuild the stores.
- MobileVault, Keychain-protected VaultPersistence, the per-chat sealed
  outbox (VaultChatDisk), the registry field codec, and the encrypted
  SessionStore/ChatRoomClient/WorkspaceStore paths (outcome-bearing apply,
  paused/resume, ack-retired batches, verified-baseline registry merge).
- EncryptionView (Home → account menu): state, fingerprint, epoch, the
  genesis-bound comparison code, "Approve from another device", and
  "Use recovery key".
- Desktop/CLI expose the vault fingerprint the phone pins during pairing.
- Engine e2e: the transcript reset-frame assertion now includes the
  contextUsage field every update has carried since #264 (pre-existing
  upstream mismatch; failed identically on main).

Verification: xcodebuild simulator build succeeds; ZeronTests 104 unit +
11 UI tests pass on an iPhone 17 Pro simulator; engine e2e suite passes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…elay

Enrolled profiles now speak device RPC through a Noise_XX_25519_AESGCM_SHA256
session between the two devices' vault identities (RFC 0001 section 10,
plan ES-11): statics are the membership-published X25519 keys, the prologue
binds vault id and generation, device ids ride inside the encrypted
handshake payloads, and each side accepts its peer only when the pinned
membership head lists that id with that key. Membership is re-checked on
every inbound sealed frame, so a revocation ends the session.

- crypto: `channel` module (handshake, transport with chunking, frame cap,
  per-direction budget) on `snow` 0.10.0 (ring-accelerated; RustCrypto
  features only satisfy the resolver's compile check).
- rpc: `device_channel` frame shape and the `ChannelAuthority` seam; the
  host relay runs the responder per client conn, `DeviceLink`/`LinkCache`
  run the initiator and never dial plaintext once required; refused links
  surface the host's reason.
- engine: `VaultService` implements `ChannelAuthority`; a secured `RelayRpc`
  instance serves channel conns (Vault* still local-only); the blanket
  relay refusals in `dial_device`/`forward` are replaced by the channel.
- edge: DeviceRoom relays `chan` frames for encrypted profiles and closes
  the socket on plaintext RPC.
- tests: 3 crypto, 5 relay scenarios over the fake DO (zero plaintext
  frames observed), workerd gate update, and a live two-device channel
  scenario in `vault_e2e.rs`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A delete in the encrypted registry generation now carries a row lifecycle
proof: a content record with purpose 9 (RegistryLifecycle) whose
authenticated plaintext is {kind, id, op: "delete", hlc} (RFC 0001
section 9, plan ES-10 / G4). The room refuses a bare delete for an
encrypted generation, stores the proof as the tombstone's delProof, serves
it with the row, replaces it only with a causally newer delete, and drops
it on revival. Readers in Rust, TypeScript, and Swift accept a tombstone
only with a verified proof for exactly that row and clock that is newer
than the verified baseline, and accept a live row over a verified
tombstone only with a verified field newer than it. A relay can neither
delete nor resurrect a row. iOS previously paused on any tombstone in an
encrypted room; it now verifies them.

- crypto: ContentPurpose::RegistryLifecycle; fixture gains a lifecycle
  record (opens as purpose 9, refused as purpose 4).
- doc: proof on delete ops, del_proof on tombstones, seed round-trip.
- sync: RegistryCodec::{seal,open}_lifecycle; open_rows tombstone and
  revival rules; unit tests and a mock-server test where a keyless
  forger's tombstone is merged by the server and ignored by the peer.
- engine: VaultRegistryCodec seals/opens lifecycle proofs.
- edge: del_proof column (added in place), push gate, purpose range 1..9,
  core merge tests, workerd gate test.
- iOS: RegistryCore/VaultRegistryCodec mirror; merge enforces the same
  rules; RegistryCoreTests lifecycle cases; fixture assertion.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add iOS Noise RPC and sealed sidecar reads, approval notifications and recovery UX, and encrypted device aliases. Copy retained chat history and tool output into verified encrypted storage with resumable progress, conflict checks, and reader refresh. Integrate migration gates with main's shared message queue and document local device validation and retained plaintext copies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant