harness: turn errors must retry the current turn — never drain the queue

[btipling]

Summary

A turn error today looks like a finished turn: the host sets Ready, Wasm treats busy → ready|err as terminal, and tryPromoteQueued pops the queue head into a new user send. The failed work is abandoned; the operator’s follow-up runs instead.

That is wrong. Errors retry the current task (exponential backoff, 5 attempts) or stop. They never consume queue items.

When retries are exhausted and the queue is non-empty, insert a “Continue the current turn” prompt as the new queue head. Do not pop anything already queued.

Current baseline (live code)

Claim	Path / symbol	Notes
Promote on Ready or error	native/harness/src/ui.zig ~227–235	terminal = ready or err; trigger_a = prev busy and terminal → bridge.tryPromoteQueued
Docs agree (and are the bug)	docs/harness-limits.md Submit queue → When it runs	“After the current turn reaches Ready or error, the head submits”
Promote pops head	submit_queue.promoteIf / bridge.tryPromoteQueued	Peek → queueSubmitFromUi → pop
Fail path sets Ready	lib/harnessChat.ts runHarnessChat / agent fail	Error line + setLifecycle(Ready) — same as success, so promote fires
Stop vs error	classifyTurnFailure	499 / abort → stop; timeout / empty / other → error
Existing backoff seam	lib/sandbox/resilience.ts withTransientRetry	5 total attempts (retries default 4 after the first), exponential baseMs / capMs, abort cancels sleep. Used for sandbox attach, not the turn
Queue	submit_queue.zig	FIFO push at tail only — no insert-at-0 yet. Cap 16, Wasm-ephemeral

Operator report: turn errors with queued follow-ups abort the current message and start the queued one.

Goals

#	Goal	Success signal
1	A turn error never promotes the operator queue	Busy + 1+ queued items + /api/agent failure (after retries) → queue depth unchanged; no new user row from the old head
2	Retry the same prompt 5 times with exponential backoff	5 attempts total (1 + 4 retries), same rawPrompt / session, no extra user ring lines. Stay Busy for the whole retry window so promote cannot run
3	Give up after 5 → stop	One Error (or timeout) turn-end line; lifecycle idle; Stop chrome off
4	If the queue is non-empty on give-up, unshift Continue the current turn as the new head	Existing items shift down one; none popped. Queue depth += 1 (unless already full — see below)
5	Operator Stop / abort (499) does not retry and does not insert Continue	Queue stays as today; Stop still drains after a later successful Ready (unchanged). Only errors are banned from consuming the queue
6	Permanent 401/403/validation: no backoff loop; same give-up + Continue-if-queued behavior as exhausted retries	Don’t hammer auth failures 5 times

Non-goals / out of scope

• Do not auto-send the Continue item (park it at the head; the operator edits / removes / sends, or a later successful Ready may promote it like any other head)
• Do not persist the queue or Continue on the cloud session (still Wasm-ephemeral)
• Do not raise MAX_ITEMS 16 / SUBMIT_CAP
• Do not retry Stop
• Do not add a DOM error toast / second composer
• Forbidden wiring: dual DOM chat · secrets in Wasm · laptop-only Production ops

Design

1. Host — retry the in-flight turn (lib/harnessChat.ts / runHarnessTurn)

Stay Busy across attempts. Reuse withTransientRetry (already 5 attempts, abort-aware):

• Retryable: network, 408/429/5xx, timeout, empty model response, other classifyTurnFailure error that isn’t 401/403/validation/stop.
• Not retryable: signal.aborted / 499, 401, 403, prompt validation.
• Do not pushUser on retries (the user line is already on the ring).
• Do not promote or setLifecycle(Ready) between attempts.
• Backoff: existing defaults (baseMs 250, capMs 4000) unless a named NEW cap is added (TURN_RETRY_ATTEMPTS = 5, plus base/cap) and listed in AGENTS.md.

Optional in-canvas system line per retry (Retrying 2/5…) — keep it to one line or none; don’t spam five Error rows.

2. Wasm — promote only on success Ready

tryPromoteQueued must not run on busy → err, nor on busy → ready when the turn failed.

Cleanest: host sets Lifecycle.Error on give-up (the enum already exists) and Ready only on success / after operator Stop. Promote stays prev == busy && cur == ready (drop err from terminal for trigger_a). Trigger B (edit closed while idle) unchanged.

Update docs/harness-limits.md When it runs: after the current turn reaches successful Ready, not error.

3. Queue — insert Continue at head, never pop on error

Add submit_queue.insertFront (or pushFront) + bridge wrapper.

On host give-up, if queuedCount() > 0:

• If count == 16, do not drop operator items; skip the insert (fail closed). Still no promote.
• Else insert the exact prompt: Continue the current turn
• Same ITEM_BYTES / normalize path as a normal enqueue (not blank)

The Continue row is a normal queue row (preview, edit, ✕). It is not auto-submitted.

Layer placement

Concern	Layer	Path(s)
Retry + backoff	DOM host	lib/harnessChat.ts runHarnessTurn (+ withTransientRetry)
Lifecycle Error vs Ready	DOM host + Wasm	HarnessBridge.setLifecycle · ui.zig promote gate
insertFront	Wasm	submit_queue.zig · bridge.zig
Docs	living	docs/harness-limits.md Submit queue · docs/agent-stream.md error · AGENTS.md if new caps

Testing

#	Case	Layer	Expected
1	Busy + queued + retryable 500, then success on attempt 3	host	One user line; assistant from the successful attempt; queue untouched until success Ready, then current promote-head behavior
2	5 retryable failures	host + Wasm	Error turn-end; no queue pop; if queue was non-empty, head is Continue the current turn
3	5 failures, queue empty	host	Stop; no Continue row
4	Queue full (16) on give-up	Wasm	No insert, no pop
5	Operator Stop on attempt 2	host	Immediate abort, no further backoff, no Continue insert; queue intact
6	401/403	host	Single attempt, give-up path, no 5× loop
7	insertFront unit	submit_queue host test	Head is new item; previous head is index 1
8	busy → err does not tryPromoteQueued	Wasm / ui policy	Queue count unchanged

Gates: vitest (harnessChat / host) + zig build test-rich (submit_queue) + zig fmt --check. Agent path is TS+Zig — both layers.

Cloud ops

N/A — no Production mutate. Retry is client-side around the existing /api/agent turn.

Living docs

Surface	Change
docs/harness-limits.md Submit queue	When it runs = successful Ready only. Error: retry 5× then stop; never promote; Continue-at-head if queue non-empty
docs/agent-stream.md	Error / timeout: retries then stop; queue not drained
AGENTS.md	New caps if introduced (TURN_RETRY_ATTEMPTS = 5, backoff)
README / SECURITY / .env.example	N/A

Refs submit-queue plan #664 · withTransientRetry · protocol v18 inv_queued_count

[btipling]

Follow-up on Stop vs queue: #757 — Stop/Esc must not drain the queue (this issue’s “Stop still drains after Ready” is revoked). Idle ▶ with an empty composer and a non-empty queue is what starts the next item.

[btipling]

Closed: shipped in PR #774 (feat(harness): turn errors retry the current turn — never drain the queue, plan #759).