Reconcile a routable run's drifted definition before serving it (CL-6588) - #298
Merged
Conversation
A launch renders workflow_run.definitionId / workbench_launch's foldedBody once; only an explicit refreshAgentInstanceFromDefinition call (a human saving settings) ever re-reads the definition's asset. These prove a routable run deployed from a definition that has since changed for a reason nobody in the room caused gets relaunched on the next wake or send, and that an unchanged one is left alone.
CL-6588: an agent's launch renders its bytes once, and nothing re-renders them when the inputs change unless a human explicitly saves an edit through refreshAgentInstanceFromDefinition. A run that is alive and routable can still be deployed from a definition whose current authored content has since changed for a reason nobody in the room caused: a platform code fix, a redeployed default agent package. That run stays silently wrong forever, indistinguishable from a genuinely broken one to anyone using it. wakeByAddress's already-routable branch and sendMail's wake-gate choke point now check the routable run's deployed wire hash against the asset's current hub-authored one (the same content-identity key CL-6452's per-deploy cloning already keys definitions on) and, on a mismatch, relaunch it through the existing relaunchTerminalRun / repointBinding machinery -- the same mint-fresh-run-and-repoint path a dead run already gets, just triggered by content drift instead of death. No parallel mechanism, no new redeploy primitive: this only widens who asks the existing one to run. lifecycle.ensureAwake short-circuits on routability alone before ever reaching wakeByAddress, so the check is also called directly and unconditionally from sendMail's and ensureAwake's lifecycle branch -- otherwise the exact case this exists for (an already-routable instance) would never reach it. The check is best-effort: a resolution failure (e.g. a pre-cutover definition with no stored wire projection at all) is logged and treated as nothing to reconcile, never as a reason to fail the send.
…hash The drift check compared a routable run's per-deploy clone wireHash against its asset's authored wireHash. That clone hash bakes in per-run values (wf_<runId>, the run's own trigger address), so it is unique to the run by design and never matches the authored hash, even when nothing actually changed. Every send re-read as drifted and relaunched a healthy run mid-turn, which is why three chat e2e tests timed out waiting for a reply that never arrived because the run serving it kept getting replaced. Compare the folded body's content instead (order-independent, via a canonical-JSON encoding) so staleness reflects real content differences rather than an always-unique per-run hash. A run whose asset has no resolvable authored sibling still falls through the existing catch and is left alone, never treated as drifted.
TheGreatAxios
force-pushed
the
cl-6588-reconcile
branch
from
August 22, 2026 05:31
5d8f4c7 to
e3fbdba
Compare
Merged
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CL-6588: a workbench created before a fix stays broken forever, even after the fix ships, because a launch renders an agent's bytes (
workbench_launch.foldedBody) once and nothing re-renders them when the underlying definition changes. The only existing lever that recomputes them (refreshAgentInstanceFromDefinition) fires solely when a human explicitly saves an edit through the settings UI — never when the definition changed for a reason nobody in the room caused (a platform code fix, a redeployed default agent package).Findings (full write-up posted to CL-6588)
Interchange's
docs/ARCHITECTURE.md:47describes a definition-level "Update Policy" (auto-redeploy/notify/staged/manual) governing how changes reach running agents. That's aspirational, not implemented:docs/IMPLEMENTATION.md's "Workflow Definition Versioning: Pinned-Forever" section documents the real, deliberate behavior — a deployment keeps its deploy-time definition until an explicit undeploy/redeploy, and the one piece of code that would auto-redeploy a stale deployment on reconnect (sidecar-handler.ts's deploy-ref freshness catch-up) is dead:isRunAddressis true for every address workbench uses today, so it always short-circuits before running. Interchange gives us definition versioning (content-identity viaworkflow_definition.wireHash) and an explicit redeploy primitive — deliberately not automatic reconciliation. This isn't a gap to file upstream; it's Workbench's job.Workbench already had every piece needed, just not wired together:
resolveAuthoredProjectedDefinition(recomputes current content),relaunchTerminalRun+repointBinding(mints a fresh run, swings the room's stable address at it without moving the room), andwireHashas the native content-identity key CL-6452's per-deploy cloning already uses. Nothing here is a parallel mechanism.Change
wakeByAddress's already-routable branch andsendMail's wake-gate choke point (plus theensureAwakeport method) now check a routable run's deployed wire hash against its asset's current hub-authored wire hash, and relaunch through the existingrelaunchTerminalRunmachinery on a mismatch.lifecycle.ensureAwakeshort-circuits on routability alone, so the check is also called directly and unconditionally after it — otherwise the exact case this exists for (an already-routable, silently-stale instance) would never reachwakeByAddressat all.Test plan
WORKBENCH_CHECK_SINCE=origin/main bun run typecheck(packages/chat clean)bun testinpackages/chat: 673 pass, 0 fail (3 new tests added for the drift-reconcile path; all 33 pre-existingplatform-adaptertests still pass unmodified)bun run lintclean (0 errors)Scope notes
packages/folded-runsstructurally (only calls its existing exported functions).sweepTerminalRuns/listLaunchesBeyondWake) still only catches dead runs, not drifted-but-alive ones — a room recovers the moment someone sends into it, per the same "send-triggered path" precedentsweepTerminalRuns's own doc comment cites for terminal runs. Extending the sweep to also catch drift is a natural, low-risk follow-up, not filed as a blocker here.