Proposal: suspend a Task automatically when its command exits (spec.onCompletion)

[yairdamri-develeap]

Problem

When a Task's spec.command exits, the runner keeps the sandbox up so it can be inspected (runner.Run waits on ctx.Done() after reportExit). The actor stays Running and keeps its worker until someone calls ax suspend or ax delete.

For batch-style Tasks ("review this PR", "upgrade this dependency") this means a finished agent holds a worker indefinitely. On a pool with fewer workers than Tasks, new Tasks fail to resume with ResourceExhausted: no free workers available while finished Tasks sit idle. The density that suspend/resume is meant to deliver only appears if an external process watches every Task and suspends it.

We hit this while running several agents as Tasks on a small WorkerPool (kind, gVisor): each finished agent had to be suspended by hand before the next one could get a worker.

Existing pieces

• The roadmap lists "Idleness Detection and Automatic Suspension for Density" (docs/roadmap.md).
• runner.Config already has an OnCommandExit func(CommandExit) hook that reportExit calls with the exit code, but nothing sets it today.

Proposal

A small first step that covers command completion only, leaving general idleness detection for later:

1. Runner reports completion. Set OnCommandExit in cmd/ax-task-runner to record the exit, and expose it from the metadata server (for example /metadata/v1alpha1/ax/status returning {"exited": true, "exitCode": 0}).
2. Task API field. Add spec.onCompletion with values Keep (default, today's behavior), Suspend, and Delete.
3. Controller applies it. When the controller sees the command has exited, it applies the policy: for Suspend, call SuspendActor and move the Task to a Completed phase with the exit code in its status, so ax get tasks shows the result and the worker is freed.

Keep as the default preserves current behavior. Suspend keeps /workspace in the snapshot, so results can still be inspected after ax resume.

Questions for maintainers

• Is this in line with how you plan to approach the roadmap item, or would you rather go straight to activity-based idle detection?
• Should the controller poll the runner's metadata endpoint, or should completion be reported another way?
• How should this relate to Substrate's worker-initiated suspend (RequestActorSuspend / internal/ateomsuspend, recently added on Substrate main)? Is AX expected to use that path once it is wired up?
• Would you accept a PR for steps 1–3 (with tests)? Happy to split it into one PR per step.