Skip to content

Reconcile a routable run's drifted definition before serving it (CL-6588) - #298

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-6588-reconcile
Aug 22, 2026
Merged

Reconcile a routable run's drifted definition before serving it (CL-6588)#298
TheGreatAxios merged 3 commits into
mainfrom
cl-6588-reconcile

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Contributor

Summary

CL-6588: a workbench created before a fix stays broken forever, even after the fix ships, because a launch renders an agent's bytes (workbench_launch.foldedBody) once and nothing re-renders them when the underlying definition changes. The only existing lever that recomputes them (refreshAgentInstanceFromDefinition) fires solely when a human explicitly saves an edit through the settings UI — never when the definition changed for a reason nobody in the room caused (a platform code fix, a redeployed default agent package).

Findings (full write-up posted to CL-6588)

Interchange's docs/ARCHITECTURE.md:47 describes a definition-level "Update Policy" (auto-redeploy/notify/staged/manual) governing how changes reach running agents. That's aspirational, not implemented: docs/IMPLEMENTATION.md's "Workflow Definition Versioning: Pinned-Forever" section documents the real, deliberate behavior — a deployment keeps its deploy-time definition until an explicit undeploy/redeploy, and the one piece of code that would auto-redeploy a stale deployment on reconnect (sidecar-handler.ts's deploy-ref freshness catch-up) is dead: isRunAddress is true for every address workbench uses today, so it always short-circuits before running. Interchange gives us definition versioning (content-identity via workflow_definition.wireHash) and an explicit redeploy primitive — deliberately not automatic reconciliation. This isn't a gap to file upstream; it's Workbench's job.

Workbench already had every piece needed, just not wired together: resolveAuthoredProjectedDefinition (recomputes current content), relaunchTerminalRun + repointBinding (mints a fresh run, swings the room's stable address at it without moving the room), and wireHash as the native content-identity key CL-6452's per-deploy cloning already uses. Nothing here is a parallel mechanism.

Change

  • wakeByAddress's already-routable branch and sendMail's wake-gate choke point (plus the ensureAwake port method) now check a routable run's deployed wire hash against its asset's current hub-authored wire hash, and relaunch through the existing relaunchTerminalRun machinery on a mismatch.
  • lifecycle.ensureAwake short-circuits on routability alone, so the check is also called directly and unconditionally after it — otherwise the exact case this exists for (an already-routable, silently-stale instance) would never reach wakeByAddress at all.
  • Best-effort: a resolution failure (e.g. no stored wire projection) is logged and treated as nothing to reconcile, never as a reason to fail the send.

Test plan

  • WORKBENCH_CHECK_SINCE=origin/main bun run typecheck (packages/chat clean)
  • bun test in packages/chat: 673 pass, 0 fail (3 new tests added for the drift-reconcile path; all 33 pre-existing platform-adapter tests still pass unmodified)
  • bun run lint clean (0 errors)
  • Live verification against the pre-fix broken workbench on the shared :3000 stack — in progress, will report the transcript on CL-6588

Scope notes

  • Does not touch packages/folded-runs structurally (only calls its existing exported functions).
  • Does not require the CL-6581 native-Interchange cutover — the redeploy primitive already exists and is already used for the dead-run case; this only widens the trigger.
  • Scoped to the send/wake choke points (the proven live-repro path). The boot-time sweep (sweepTerminalRuns/listLaunchesBeyondWake) still only catches dead runs, not drifted-but-alive ones — a room recovers the moment someone sends into it, per the same "send-triggered path" precedent sweepTerminalRuns's own doc comment cites for terminal runs. Extending the sweep to also catch drift is a natural, low-risk follow-up, not filed as a blocker here.

A launch renders workflow_run.definitionId / workbench_launch's
foldedBody once; only an explicit refreshAgentInstanceFromDefinition
call (a human saving settings) ever re-reads the definition's asset.
These prove a routable run deployed from a definition that has since
changed for a reason nobody in the room caused gets relaunched on the
next wake or send, and that an unchanged one is left alone.
CL-6588: an agent's launch renders its bytes once, and nothing
re-renders them when the inputs change unless a human explicitly
saves an edit through refreshAgentInstanceFromDefinition. A run that
is alive and routable can still be deployed from a definition whose
current authored content has since changed for a reason nobody in the
room caused: a platform code fix, a redeployed default agent package.
That run stays silently wrong forever, indistinguishable from a
genuinely broken one to anyone using it.

wakeByAddress's already-routable branch and sendMail's wake-gate choke
point now check the routable run's deployed wire hash against the
asset's current hub-authored one (the same content-identity key
CL-6452's per-deploy cloning already keys definitions on) and, on a
mismatch, relaunch it through the existing relaunchTerminalRun /
repointBinding machinery -- the same mint-fresh-run-and-repoint path a
dead run already gets, just triggered by content drift instead of
death. No parallel mechanism, no new redeploy primitive: this only
widens who asks the existing one to run.

lifecycle.ensureAwake short-circuits on routability alone before ever
reaching wakeByAddress, so the check is also called directly and
unconditionally from sendMail's and ensureAwake's lifecycle branch --
otherwise the exact case this exists for (an already-routable
instance) would never reach it.

The check is best-effort: a resolution failure (e.g. a pre-cutover
definition with no stored wire projection at all) is logged and
treated as nothing to reconcile, never as a reason to fail the send.
…hash

The drift check compared a routable run's per-deploy clone wireHash
against its asset's authored wireHash. That clone hash bakes in
per-run values (wf_<runId>, the run's own trigger address), so it is
unique to the run by design and never matches the authored hash, even
when nothing actually changed. Every send re-read as drifted and
relaunched a healthy run mid-turn, which is why three chat e2e tests
timed out waiting for a reply that never arrived because the run
serving it kept getting replaced.

Compare the folded body's content instead (order-independent, via a
canonical-JSON encoding) so staleness reflects real content
differences rather than an always-unique per-run hash. A run whose
asset has no resolvable authored sibling still falls through the
existing catch and is left alone, never treated as drifted.
@TheGreatAxios
TheGreatAxios merged commit 283b32c into main Aug 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant