Add Tensorlake as a sandbox provider by diptanu · Pull Request #173 · exoharness/exo

diptanu · 2026-07-27T05:03:26Z

Adds tensorlake alongside Daytona, E2B, Sprites, Vercel, and AgentCore. Tensorlake sandboxes are Firecracker MicroVMs that can suspend and resume with their filesystem intact, which lines up with how Exo conversation sandboxes behave across sessions.

Trying it

Point Exo at a Tensorlake key and configure the provider:

exo secret set TENSORLAKE_API_KEY --env TENSORLAKE_API_KEY
exo provider configure --provider tensorlake --secret TENSORLAKE_API_KEY
exo agent create tl-demo --model gpt --sandbox-provider tensorlake --networking enabled
exo conversation create tl-demo demo

Nothing is provisioned until something first needs the sandbox. A quick test without a model call:

exo conversation sandbox run tl-demo demo "cat /etc/os-release | head -2; uname -m; nproc"
exo conversation sandbox run tl-demo demo "echo 'state survives across processes' > /tmp/note.txt"
exo conversation sandbox run tl-demo demo "cat /tmp/note.txt"

Streaming processes work too, so --shell-program /bin/bash in the REPL behaves as it does on other providers.

Lifecycle

This PR now builds on the attach/detach lifecycle from #161:

ManagedSandboxBackend gains idempotent terminate and optional fork_sandbox APIs.
Deleting a conversation permanently terminates its provider-owned sandboxes; deleting an agent also sweeps its agent-scoped sandbox and every owned conversation sandbox.
Forking a conversation asks the provider to copy each live sandbox under the fork's owner key. Tensorlake implements this with its sandbox copy API; unsupported providers retain the existing cold-start behavior.
Externally attached sandboxes remain borrowed: a fork records them as detached, and owner deletion never terminates the external resource.
In-memory sandbox and process registries are owner-scoped, so a parent and fork cannot accidentally share a live handle merely because their persisted sandbox IDs match.

Every existing backend implements termination. Remote providers use their permanent delete/stop APIs, container backends remove their Exo-owned container, and local-process termination is a no-op.

How Tensorlake resume and fork work

Tensorlake creates have no label or metadata field, and only named sandboxes can suspend and resume. Each sandbox therefore gets a deterministic exo-<hash> name derived from its SandboxKey and spec hash. Agent-scoped and conversation-scoped sandboxes hash differently; changing the image or mounts yields a different name.

stop suspends named sandboxes so filesystem state survives. Snapshots are stored by reference and are not persisted until Tensorlake reports them as restorable. Conversation forks copy a live named sandbox directly to the deterministic name derived for the target conversation, leaving the parent untouched.

Configuration lives on the binding, including optional --cpus and --memory-mb. Credentials resolve lazily from TENSORLAKE_API_KEY, so the provider can be registered before a key is set.

Validation

cargo test -p exo --test tensorlake_backend — 28 passed
cargo test -p exoharness --features basic-backend -- --test-threads=1 — 67 passed, 7 real-provider tests ignored
cargo check --workspace --all-targets
cargo fmt --all -- --check

Known gaps

Tensorlake rejects host bind mounts and durable filesystems; use a local provider when those are required.
Tensorlake's process API frames output as lines, so a trailing partial line arrives only after a newline or process exit, and exec output always ends with one.

61cygni · 2026-07-30T15:58:34Z

Looks good. Thanks for doing this.

Lifecycle goes through the Tensorlake platform API (create/get/suspend/ resume/delete/snapshot); commands go through the per-sandbox proxy, using `/api/v1/processes/run` for exec and `/api/v1/processes` plus the stdin and follow endpoints for streaming processes. Cross-process resume uses a deterministic *named* sandbox derived from SandboxKey + spec hash. Tensorlake creates carry no label or metadata field, and only named sandboxes support suspend/resume, so the name is the only resume handle. A request without an idle TTL gets an ephemeral sandbox. SnapshotKind::TensorlakeSnapshot carries a manifest pointing at a platform snapshot id; restore creates a sandbox from it. Registered on --sandbox-provider and `exo provider configure`, with optional --cpus / --memory-mb. Credentials resolve lazily from TENSORLAKE_API_KEY. Scoped deliberately to the provider: no changes to ManagedSandboxBackend or ManagedSandboxHandle, so this does not collide with the attach/detach lifecycle in exoharness#161. Sandbox reclamation on delete and copying sandboxes into forked conversations both need trait changes and are held back until that lands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

- Wait for a snapshot to become restorable before persisting its id. `POST /sandboxes/{id}/snapshot` returns 202 while the snapshot is still being written, so snapshotting and immediately rewinding could reference something create would reject. `local_ready` and `replicating` both count as restorable; waiting for full durability would stall a rewind for no benefit. A failed snapshot now surfaces its error instead of yielding a manifest. - Reject a snapshot manifest captured from a different sandbox. The name was recorded but never checked, so restoring across identities would silently hand back someone else's filesystem. - Reject `--cpus` / `--memory-mb` for providers that ignore them, matching the existing guard on `--session-storage-mount-path`. - Fold the proxy-liveness flag into the mutex that already guarded it. The target and its liveness are only ever read and written together, so an atomic alongside the lock bought nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

ankrgyl · 2026-08-01T20:03:10Z

+    /// Tensorlake: whole CPU cores per sandbox.
+    #[arg(long)]
+    cpus: Option<u32>,
+    /// Tensorlake: memory per sandbox in MiB.
+    #[arg(long = "memory-mb")]
+    memory_mb: Option<u64>,


if we add these we should propagate them to each sandbox provider

ankrgyl · 2026-08-01T20:04:57Z

+        Ok(false)
+    }


generally prefer not introducing default implementations. also if we have a default implementation, shuoldn't it do snapshot + acquire_from_snapshot?

ankrgyl · 2026-08-01T20:07:53Z

    }
+
+    async fn terminate(&self, request: SandboxRequest) -> Result<()> {
+        let spec_hash = sandbox_spec_hash(&request.spec);


did you run a test for all the sandbox providers you added terminate for? it's a little tricky because it requires API keys for all of them

ankrgyl · 2026-08-01T20:08:24Z

+    /// a later acquire, termination gives up the provider resource and all of
+    /// its retained state. Implementations must be idempotent and must not
+    /// create or resume a sandbox while looking for it.
+    async fn terminate(&self, request: SandboxRequest) -> Result<()>;


if we add these methods we need to add generic tests for them that work across all sandbox providers

diptanu force-pushed the tensorlake-provider branch from 09979cd to f66cea4 Compare July 27, 2026 06:06

diptanu and others added 2 commits July 30, 2026 22:47

diptanu force-pushed the tensorlake-provider branch from f66cea4 to c87d0a2 Compare July 31, 2026 05:50

exoharness: tie sandboxes to owner lifecycle

4b996f3

ankrgyl reviewed Aug 1, 2026

View reviewed changes

Navigation Menu

Uh oh!

Add Tensorlake as a sandbox provider - #173

Add Tensorlake as a sandbox provider#173
diptanu wants to merge 3 commits into
exoharness:mainfrom
diptanu:tensorlake-provider

diptanu commented Jul 27, 2026 •

edited

Loading

Uh oh!

61cygni commented Jul 30, 2026

Uh oh!

ankrgyl Aug 1, 2026

Uh oh!

ankrgyl Aug 1, 2026

Uh oh!

ankrgyl Aug 1, 2026

Uh oh!

ankrgyl Aug 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

diptanu commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Trying it

Lifecycle

How Tensorlake resume and fork work

Validation

Known gaps

Uh oh!

61cygni commented Jul 30, 2026

Uh oh!

ankrgyl Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

ankrgyl Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

ankrgyl Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

ankrgyl Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

diptanu commented Jul 27, 2026 •

edited

Loading