agent-lifecycle: bound ensureAwake so a hung wake can't wedge a workbench silently forever - #312
Merged
Merged
Conversation
CL-6643: a run whose deployed record was parked aside can hang inside the injected wake port forever with no ack from the host. Without a bound, that first hung call never leaves pendingWakes, so every later ensureAwake for the same address coalesces onto the same dead promise and hangs too, silently, for the rest of the process's life.
A wake for an address with no sidecar-side deployment record is a real round-trip to the host, not a local check. Previously, if that call never resolved or rejected, its promise sat forever in pendingWakes: ensureAwake's own .finally never ran, so every later call for the same address coalesced onto that dead promise and hung too, with nothing ever bubbling up to sendMail to report as an undelivered notice. ensureAwake now races the injected wake port against wakeTimeoutMs (defaulting to 30s) and always settles, clearing the address out of pendingWakes on either outcome. A wake that never acks now surfaces as a rejection callers can turn into the undelivered notice this class of failure has always needed, instead of leaving the workbench silently dead for the rest of the process's life.
This was referenced Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CL-6643: a message POST was accepted, then silently swallowed — no fanout log, no wake attempt, no error, no undelivered notice — for a workbench whose agent's run had no sidecar-side deployment record (parked aside, cold wake required).
The prime suspect in the ticket was PR #298's wireHash drift-reconcile logic in
packages/chat/src/platform-adapter.ts. That turned out not to be the mechanism: the drift-reconcile path (reconcileDriftedRun) already catches and logs on failure and proceeds ("nothing to reconcile") rather than swallowing, andwakeByAddress's branches all throw or delegate rather than early-returning silently.The actual mechanism is in
packages/agent-lifecycle/src/index.ts'screateAgentLifecycle.ensureAwakecoalesces concurrent callers for the same address onto one in-flightwake()call via apendingWakesmap, releasing the entry in a.finally()oncewake()settles. But nothing bounded how longwake()could take — a cold wake for a run with no sidecar record is a real deploy round-trip to the host, and if that round-trip never acks (confirmed live: reproduced by sending fresh messages into the brokenrun_d661612…workbench — POST accepted every time, then permanent silence, no fanout/wake/error/undelivered-notice lines ever), thewake()promise never settles. Its.finallynever runs, so the address is wedged inpendingWakesforever: every laterensureAwakecall for that address — i.e. every future message sent to that workbench — coalesces onto the same dead promise and hangs too, for the rest of the process's life, with nothing ever rejecting forsendMail/dispatchTurnBatchto turn into an undelivered notice.Fix
ensureAwakenow races the injectedwakeport against a newwakeTimeoutMsoption (default 30s) viawakeWithTimeout, and always settles — clearing the address out ofpendingWakeson either outcome. A wake that never acks now rejects instead of hanging, whichdispatchTurnBatch's existing catch already turns into a real undelivered notice on the timeline.Test plan
packages/agent-lifecycle/test/index.test.ts) reproducing the swallow: awakethat never settles must not permanently wedge the address — a first call rejects on timeout, and a second, later call for the same address gets its own fresh wake attempt rather than coalescing onto the dead promise. Confirmed this test hangs (no output, timeout-killed) against the pre-fix code.WORKBENCH_CHECK_SINCE=origin/main bun run typecheck— passWORKBENCH_CHECK_SINCE=origin/main bun run test— pass (198 tests across affected packages, including the new one)bun run lint— pass (0 errors)wb_verify_0822, workbenchrun_d661612…): sent a fresh message ("Third wake check...") into the already-broken workbench; POST accepted (201), then confirmed the exact same permanent silence the ticket describes, with no fanout/wake/error lines at all — matching the mechanism above.run_d661612…with this fix loaded: not done. The fix lives in this worktree; the shared dev stack (.worktrees/cl-e2e-main, loge2e-final4.log) is running the pre-fix code from its own checkout, and other work depends on it staying up, so I did not restart it to load this branch. Whoever picks this up for review should restart that stack (or a fresh one) on this branch to confirm the end-to-end cold-wake-then-reply path.