plan: harness — turn errors retry the current turn, never drain the queue (#756) · IMPLEMENTED (PR #774)

[btipling]

Plan header

Field	Value
Status	IMPLEMENTED — PR #774 (not merged; adversarial-review pending)
Review notes	Reviewed 2026-08-22 (plan-review). No blockers. Four findings applied to the body: (1) Major — withTransientRetry hard-codes classifyVercelError and has no injectable classifier in TransientRetryOptions (resilience.ts 260–273, 298), so the plan's "reuse it, only needs a retryability predicate" is not implementable as written — Design §1 + Architecture decision A now specify an additive optional classify seam (defaulting to classifyVercelError) used by the turn path, plus a back-compat test binding that default; (2) new inv_queued_insert_front-style export must be added to native/harness/build.zig export_symlink_names or the linker GCs it (same defect sibling #760 fixed) — added to Implement order step 1 + DoD + Risk; (3) protocol question locked to v18→v19 (REQUIRED_FNS + build.zig + bump both PROTOCOL_VERSION/HARNESS_PROTOCOL_VERSION), coordinating with sibling #760 (union v19/v20 — whichever merges second takes the next free version); (4) both Open questions resolved as decisions (named TURN_RETRY_ATTEMPTS=5 NEW cap; no in-canvas retry line). All baselines verified against main (ui.zig 221–231 terminal-promote, bridge.zig Lifecycle 39–44 / tryPromoteQueued 245–259 / SUBMIT_CAP 76, submit_queue.zig MAX_ITEMS 11 / push 44–52 / promoteIf 113–119, harnessBridge.ts Lifecycle 72–77 / REQUIRED_FNS 208+ / v18, harnessChat.ts classifyTurnFailure 609–632 / Ready-sites 550,558,1492,1523-1533, resilience.ts withTransientRetry 280–317 + classifyVercelError 147–214.
Date	2025-06-25
Type	single
Parent	N/A
Source issue	#756 — "harness: turn errors must retry the current turn — never drain the queue" (bug + operator report)
Branch	plan/turn-retry-never-drain-queue
Layers	DOM host + harness (Wasm)
Reusability impact	none
Production mutate?	no
Cloud ops path	N/A — no Production mutate. Retry is client-side around the existing /api/agent turn
Living docs	docs/harness-limits.md (Submit queue → When it runs), docs/agent-stream.md (Error / timeout), AGENTS.md (protocol v19 row + new caps row)

Summary

A turn error today looks like a finished turn: the host sets Ready, Wasm
treats busy → ready|err as terminal, and tryPromoteQueued pops the queue
head into a new user send. The failed work is abandoned; the operator's
follow-up runs instead. This plan makes errors retry the current task (5
attempts, exponential backoff) or stop, and bans errors from ever consuming
a queued item — on give-up with a non-empty queue it inserts a
"Continue the current turn" prompt as the new queue head, popping nothing.

Goals

#	Goal	Success signal
1	A turn error never promotes the operator queue	Busy + 1+ queued items + /api/agent failure (after retries) → queue depth unchanged; no new user row from the old head
2	Retry the same prompt 5 times with exponential backoff	5 attempts total (1 + 4 retries), same rawPrompt / session, no extra user ring lines; stays Busy for the whole retry window so promote cannot run
3	Give up after 5 → stop	One Error (or timeout) turn-end line; lifecycle idle; Stop chrome off
4	If the queue is non-empty on give-up, unshift Continue the current turn as the new head	Existing items shift down one; none popped; queue depth += 1 (unless full — see below)
5	Operator Stop / abort (499) does not retry and does not insert Continue	Queue stays as today; Stop still drains after a later successful Ready (unchanged). Only errors are banned from consuming the queue
6	Permanent 401/403/validation: no backoff loop; same give-up + Continue-if-queued behavior as exhausted retries	Never hammer auth failures 5×
7	Queue full (16) on give-up	No insert, no pop — operator items untouched; fail closed

Non-goals / out of scope

• Do not auto-send the Continue item (park it at the head; the operator edits / removes / sends, or a later successful Ready may promote it like any other head)
• Do not persist the queue or Continue on the cloud session (still Wasm-ephemeral)
• Do not raise MAX_ITEMS (16) / SUBMIT_CAP (262144)
• Do not retry Stop
• Do not add a DOM error toast / second composer
• Forbidden wiring: dual DOM chat · secrets in Wasm · laptop-only Production ops

Architectural decisions

Decision	Options considered	Choice	Why
Retry seam for the in-flight turn	A) reuse withTransientRetry (lib/sandbox/resilience.ts) — B) hand-rolled nested send loop	A (+ classifier seam)	Already 5-attempt + abort-aware + bounded exponential backoff (baseMs 250, capMs 4000), unit-tested. Baseline truth: withTransientRetry currently hard-codes classifyVercelError internally (resilience.ts 298) with no injectable classifier in TransientRetryOptions (260–273) — it is NOT directly reusable for turn failures. Add a new additive optional classify seam to TransientRetryOptions (defaulting to today's classifyVercelError, so sandbox callers are untouched); the turn path passes a new narrow classifier over classifyTurnFailure + HTTP status, intentionally NOT reusing classifyVercelError (that is the sandbox SDK domain). Decision A becomes viable once the seam lands; B is the fallback if the seam is judged too invasive
Promote gate on error	A) host sets Lifecycle.Error on give-up; Wasm promote stays busy → ready only — B) keep Ready + add an error flag to the promote predicate	A	Lifecycle.Error already exists on both sides (bridge.zig err=3 ↔ harnessBridge.ts Error=3); host is the only lifecycle writer, so simplest correct seam is: host sets Error on give-up and Ready only on success / operator Stop; Wasm's terminal drops err. One source of truth for lifecycle
Continue the current turn insert	A) Wasm submit_queue.insertFront + bridge wrapper — B) host re-sends the failed prompt via the normal submit path	A	Queue is Wasm-ephemeral and only Wasm owns its FIFO/cap; insert-at-front renders like a normal editable queue row. B would re-run a possibly-harmful prompt and duplicates the last user row
Give-up timeout/empty handling	A) timeout/empty are retryable like error — B) treat empty as terminal	A	An empty model response / gateway timeout is a transient inference failure; same as the issue's spec — retry then give up. Permanent 401/403/validation never backoff (goal 6)
Protocol version	A) additive export without bump — B) REQUIRED + v18→v19	B	Repo precedent (v14→v15, inv_set_busy_tick became REQUIRED): a new REQUIRED export bumps the version so a mismatched old wasm/host fails closed with the REQUIRED_FNS diagnostic. Locked in this review. Coordinate with sibling #760 (also adding an export + bumping): union v19/v20 — whichever merges second takes the next free version, referenced in implementation notes

Layer placement

Concern	Layer	Path(s)	Rationale
Retry + backoff + lifecycle Error/Ready	DOM host	lib/harnessChat.ts runHarnessTurn (+ runHarnessChat validation path) · classifier seam + withTransientRetry in lib/sandbox/resilience.ts	Host owns the /api/agent turn, the ring user row, and the lifecycle; Wasm must never see a promote window mid-retry
Promote gate (drop err)	Wasm	native/harness/src/ui.zig ~221–231 (terminal/trigger_a)	Promote stays prev busy && cur ready; err no longer terminal for the queue
insertFront + Continue head + export	Wasm	native/harness/src/submit_queue.zig · native/harness/src/bridge.zig wrapper + new export · native/harness/build.zig export_symlink_names	Queue is Wasm-owned; additive export read by the host (inv_queued_insert_front + existing queuedCount). Every inv_* export MUST be listed in build.zig export_symlink_names or the linker GCs it and JS never sees it (bridge.zig header contract) — see protocol note below
Docs	living	docs/harness-limits.md Submit queue · docs/agent-stream.md error/timeout · AGENTS.md protocol v19 + caps	Timeless "what it does now" wording

Current baseline (live code)

Claim	Path / symbol	Notes
Promote on Ready or error	native/harness/src/ui.zig 221–231	const terminal = cur_lc == .ready or cur_lc == .err; trigger_a = prev_lc == .busy and terminal; → bridge.tryPromoteQueued(false) — verified
Lifecycle enum (Wasm)	native/harness/src/bridge.zig 39–44	boot=0 ready=1 busy=2 err=3
Lifecycle enum (TS)	lib/harnessBridge.ts 72–77	Boot Ready Busy Error — parity with Wasm
Promote pops head	submit_queue.zig promoteIf 113–119 · bridge.zig tryPromoteQueued 245–259	Peek → promoteSubmit → queueSubmitFromUi → has_pending_submit; pop only on accept
Fail path sets Ready	lib/harnessChat.ts runHarnessChat 550,558 · runHarnessTurn agent fail 1492 · chat fail 1523–1533	classifyTurnFailure → Error/System row → setLifecycle(Lifecycle.Ready) — same as success, so promote fires. verified
Stop vs error	lib/harnessChat.ts classifyTurnFailure 609–632	499 / abort / Request cancelled. → stop; timeout / empty / other → retryable error kinds. Note: 401/403/validation are NOT returned as distinct kinds here — the new classifier keyed on classifyTurnFailure + HTTP status must treat these as permanent (goal 6)
Backoff seam hard-codes its classifier	lib/sandbox/resilience.ts withTransientRetry 280–317 · classifyVercelError 147–214 · TransientRetryOptions 260–273	5 attempts (retries default 4), exponential baseMs 250 / capMs 4000, signal aborts sleep. Critical: withTransientRetry calls classifyVercelError(err) internally (line 298) and the options struct has no classify override — it is not directly reusable for turns; an additive classifier seam is required (see Architecture + Design §1). verified
Queue FIFO, no insert-at-0	submit_queue.zig 11 MAX_ITEMS=16, 44–52 push, 113–119	push at tail only; no insertFront yet. MAX_ITEMS 16 · bridge.zig 76 SUBMIT_CAP 262144
Queue count bridge export	lib/harnessBridge.ts 148 queuedCount() · protocol v18 (inv_queued_count)	Host can read depth; no insert-export today
Wasm export whitelist	native/harness/build.zig export_symlink_names (lines 69–89+)	Every export fn inv_* must be listed here or the linker GCs it and JS never sees it (bridge.zig header contract, comment 60–68). A new export requires an entry here in the same change
REQUIRED_FNS	lib/harnessBridge.ts 208–263	Exhaustive list; new export must be appended here + type/method on HarnessExports

Protocol note (locked): adding a Wasm inv_queued_insert_front export is an
additive REQUIRED bridge extension. It must be (a) added to REQUIRED_FNS
(harnessBridge.ts 230+), (b) added to native/harness/build.zig
export_symlink_names (or the linker GCs it), and (c) paired with a v18→v19
protocol bump on both sides (bridge.zig PROTOCOL_VERSION, harnessBridge.ts
HARNESS_PROTOCOL_VERSION) so an old host/wasm fails closed on load instead of
silently missing the export. Sibling #760 also bumps v18→v19 for its own
export — union requires the second one merged to take v20. (A new cap
TURN_RETRY_ATTEMPTS is host-TS only — no Zig parity.)

Design

1. Host — retry the in-flight turn (lib/harnessChat.ts / runHarnessTurn)

Stay Busy across attempts. Reuse withTransientRetry (already 5 attempts,
abort-aware) via a new additive classifier seam:

• Additive seam (required): extend TransientRetryOptions with an optional
  classify?: typeof classifyVercelError parameter in lib/sandbox/resilience.ts.
  Default is today's classifyVercelError (sandbox callers untouched); the turn
  path passes a new narrow classifier that maps classifyTurnFailure result +
  HTTP status → { kind: 'retryable' | 'permanent' }. This keeps the sandbox SDK
  classifier and turn errors from being conflated.
• Retryable: network, 408/429/5xx, timeout, empty model response, other
  classifyTurnFailure error that is not 401/403/validation/stop.
• Not retryable (single attempt, straight to give-up): signal.aborted /
  499, 401, 403, prompt validation.
• Do not pushUser on retries (the user line is already on the ring).
• Do not promote or setLifecycle(Ready) between attempts.
• Backoff: existing defaults (baseMs 250, capMs 4000) unless a named NEW
  cap is added (TURN_RETRY_ATTEMPTS = 5, plus base/cap) — locked: name it
  in the Caps table + AGENTS.md (new cap; not a change to an existing cap).
• Locked decision: no in-canvas per-retry line (Retrying 2/5…) — avoids ring
  noise; the turn already stays Busy.
• The retryability classifier is a new, narrow predicate keyed on
  classifyTurnFailure results + status — the existing classifyVercelError
  (sandbox SDK domain) is intentionally not reused for the turn.

2. Wasm — promote only on success Ready

tryPromoteQueued must not run on busy → err, nor on busy → ready when the
turn failed.

Cleanest: host sets Lifecycle.Error on give-up (the enum already exists on
both sides) and Lifecycle.Ready only on success / after operator Stop.
Promote stays prev == busy && cur == ready — drop err from terminal for
trigger_a. Trigger B (edit closed while idle) unchanged.

Update docs/harness-limits.md When it runs: after the current turn reaches
successful Ready, not error.

3. Queue — insert Continue at head, never pop on error

Add submit_queue.insertFront (or pushFront) + a bridge wrapper + export
inv_queued_insert_front, and append it to native/harness/build.zig
export_symlink_names + REQUIRED_FNS (protocol v19).

On host give-up, if queuedCount() > 0:

• If count == 16, do not drop operator items; skip the insert (fail
  closed). Still no promote.
• Else insert the exact prompt: Continue the current turn (a named TS/Zig
  constant).
• Same ITEM_BYTES / normalize path as a normal enqueue (not blank).

The Continue row is a normal queue row (preview, edit, ✕). It is not
auto-submitted.

Edge cases

• Stop on attempt 2: abort immediately, no further backoff, no Continue.
• Queue full (16) at give-up: no insert, no pop; operator items kept.
• Permanent 401/403/validation: single attempt, give-up + Continue-if-queued
  (no 5× loop).
• Continue the current turn when head was edited: normal queue semantics —
  it promotes like any head on a later successful Ready.
• Empty give-up queue: stop, no Continue row.

Cloud ops path

N/A — no Production mutate. Retry is client-side around the existing
/api/agent turn; no env, no GHA, no data write.

Living docs plan

Surface	Change	Notes
docs/harness-limits.md Submit queue → When it runs	successful Ready only; error: retry 5× then stop; never promote; Continue-at-head if queue non-empty	timeless wording; no issue numbers
docs/agent-stream.md Error / timeout	retries then stop; queue not drained	timeless
AGENTS.md	Protocol v19 row under "Harness stream chrome": inv_queued_insert_front additive REQUIRED export; new TURN_RETRY_ATTEMPTS = 5 cap in Caps table + ownership row	keep parity with HARNESS_PROTOCOL_VERSION
README.md	N/A	no visitor-facing behavior change
SECURITY.md	N/A	no trust-boundary / secrets change
.env.example	N/A	no new env

Implementation order

1. Wasm submit_queue.insertFront + bridge wrapper + export fn inv_queued_insert_front (foundation the host needs); append the new export to native/harness/build.zig export_symlink_names (linker-GC whitelist).
2. Protocol v19: bump PROTOCOL_VERSION (bridge.zig) + HARNESS_PROTOCOL_VERSION (harnessBridge.ts); add the export to REQUIRED_FNS + TS type/method.
3. Add the classify seam to withTransientRetry/TransientRetryOptions (default = classifyVercelError) + the narrow turn classifier.
4. Host retry loop in runHarnessTurn + lifecycle Error-on-give-up.
5. Wasm promote gate (drop err from terminal).
6. Host give-up → Continue the current turn insert (respect full-16).
7. Docs (harness-limits.md, agent-stream.md, AGENTS.md protocol v19 + caps table).
8. Gates: TS + Zig both (see Testing); build-harness for the Wasm change (proves the export reached the artifact).

Testing

#	Case	Layer	Type	Command / method
1	Busy + queued + retryable 500, then success on attempt 3	host	unit	vitest lib/harnessChat.test.ts — one user line; assistant from the successful attempt; queue untouched until success Ready
2	5 retryable failures	host + Wasm	unit	Error turn-end; no queue pop; head becomes Continue the current turn when queue non-empty
3	5 failures, queue empty	host	unit	stop; no Continue row
4	Queue full (16) at give-up	Wasm	unit	zig build test-rich submit_queue — no insert, no pop
5	Operator Stop on attempt 2	host	unit	immediate abort, no backoff, no Continue; queue intact
6	401/403	host	unit	single attempt, give-up path, no 5× loop
7	insertFront unit	Wasm	unit	zig build test-rich — head is new item; previous head is index 1
8	busy → err does NOT tryPromoteQueued	Wasm / ui policy	unit	queue count unchanged on busy → err transition
9	Continue row editable/removable + promotes on later successful Ready	Wasm	unit/integration	normal queue-row semantics on the inserted Continue
10	Retryability predicate (permanent vs retryable statuses)	host	unit	lib/harnessChat.test.ts classify narrow set
11	Classifier seam back-compat	host	unit	lib/sandbox/resilience.test.ts — withTransientRetry with no classify still calls classifyVercelError (sandbox behavior unchanged); passing a classify uses it
12	Protocol v19 parity + REQUIRED_FNS + export in built wasm	host	unit/integration	harnessBridge type + REQUIRED_FNS contains inv_queued_insert_front; wasm-int round-trip proves it reached the artifact via the build.zig whitelist

Gates (both layers): npm run typecheck + node scripts/di-gate.mjs +
vitest run (host) and zig build test-rich + zig build harness -Doptimize=Debug + zig fmt --check (Wasm). CI: wait for build-harness
green (wasm32-freestanding) — local Debug is a host-target smoke only;
build-harness is the release-wasm gate that also proves the new export is in
the shipped artifact.

Caps table

Cap / ceiling	Value	Wire & its ceiling	Defense	Verdict
NEW TURN_RETRY_ATTEMPTS (host)	5 (1 + 4 retries)	Host-side memory/timer; no new transport payload	NEW cap; generous default equal to the existing withTransientRetry default; bounded by signal abort (Stop). No change to an existing cap. Locked in plan-review	OK — new, in table (no human-block; generous default)
Backoff baseMs / capMs (existing)	250 / 4000	host-side timer	unchanged defaults reused; no change	N/A — unchanged
MAX_ITEMS (existing)	16	Wasm ephemeral	not raised / not lowered — explicitly out of scope (goal "do not raise")	N/A — unchanged
SUBMIT_CAP / ITEM_BYTES (existing)	262144	Wasm ephemeral	unchanged	N/A — unchanged
Continue prompt length	"Continue the current turn"	within ITEM_BYTES	fixed constant, trivially below caps	OK
classify seam footprint	one optional fn ref	host memory	additive default preserves sandbox behavior	OK — new, no wire carrier

Definition of done

• Goals 1–7 each have a passing case in the Testing table
• No dual-chat / DOM composer regression (unchanged paths)
• Cloud ops: N/A (no Production mutate) — stated, not silent
• Living docs: harness-limits.md (When it runs = successful Ready only), agent-stream.md, and AGENTS.md protocol v19 row + TURN_RETRY_ATTEMPTS caps row updated with timeless wording
• Tests green in agent workspace / CI: TS + Zig gates above; build-harness green on the branch
• Literal Continue the current turn head inserted only on give-up + non-empty queue; queue full → fail closed (no drop)
• withTransientRetry classifier seam landed (additive classify default = classifyVercelError; sandbox tests unchanged, test 11)
• Protocol v19 on both sides (PROTOCOL_VERSION / HARNESS_PROTOCOL_VERSION) + inv_queued_insert_front in REQUIRED_FNS and native/harness/build.zig export_symlink_names, present in the built wasm (test 12 / wasm-int)
• AGENTS.md / README.md considered (AGENTS v19 + caps; README N/A justified)

Risks & mitigations

Risk	Mitigation
Retrying genuinely repeated side-effects (a post tool already ran server-side before a stream error)	Retry reuses the same session/prompt and the existing "mid-tool cancel re-run" product rule (docs/harness-limits.md) — a continue-after-stall may re-run tools. Same documented stance; do not redact history on retry
A permanent 401/403 mistakenly retried (hammer / spam)	Narrow classifier: 401/403/validation/stop are permanent — single attempt, straight to give-up (test 6)
insertFront when full silently drops operator items	Fail closed: count == 16 → skip insert, no pop, no drop (test 4)
Promote firing between attempts (busy dropped)	Host stays Busy for the whole window; Wasm err no longer terminal. Both locks enforced by test 8
withTransientRetry classified by the wrong domain	Additive classify seam; the turn path passes the narrow turn classifier, sandbox keeps classifyVercelError default (test 11)
New export missing from the linker whitelist → JS never sees it	build.zig export_symlink_names entry added in the same step; test 12 / wasm-int + build-harness prove it (mirrors sibling #760's caught defect)
Protocol-version collision with sibling #760 (both adding exports, both v18→v19)	Locked union: whichever merges second takes the next free version (v19 or v20); reference the merged state in implementation notes
Laptop-only cutover / missing GHA	N/A — no Production mutate / no GHA surface needed (explicit N/A)

Open questions

• Should the retry cap be a named NEW cap (TURN_RETRY_ATTEMPTS = 5, base/cap)
  or silently reuse withTransientRetry defaults? RESOLVED (plan-review): name
  the cap in the Caps table + AGENTS (TURN_RETRY_ATTEMPTS = 5). New cap, no
  human-block, generous default.
• In-canvas per-retry line? RESOLVED (plan-review): none. The turn stays
  Busy; avoids ring noise. No Retrying 2/5… line.

References

• Source issue: #756 — bug + operator report this plan implements.
• Sibling: #760 — plan for harness: Stop/Esc must not drain the queue; idle ▶ promotes when the composer is empty #757 (Stop/Esc never drains; idle ▶ promotes) — shares the promote gate + err-non-terminal lock and the v19/v20 protocol union.
• Prior submit-queue plan #664 ("queue messages while busy (edit + cancel)") — the queue this plan protects.
• withTransientRetry (lib/sandbox/resilience.ts, lines 260–317) — reuse seam + required classify additive seam.
• Protocol v18 inv_queued_count (lib/harnessBridge.ts, bridge.zig) — base for the v19 bump.