CL-6324: rebase the stack onto main, and what the first real boot found - #101
Closed
TheGreatAxios wants to merge 24 commits into
Closed
CL-6324: rebase the stack onto main, and what the first real boot found#101TheGreatAxios wants to merge 24 commits into
TheGreatAxios wants to merge 24 commits into
Conversation
Upstream's 45 commits since 59f5e7b9 retire the on-disk workflow.json: a deployed workflow's definition is evaluated from its own source closure and re-verified in-child against the approved wire hash, source-ref becomes the only deploy lineage, childWorkflow becomes an owned inline import resolved in memory, and run grants derive from a persisted grant-walk snapshot. Re-applies every workbench-local delta on the new trees, none of which upstream subsumed: the inference.usage forward, the terminal-anchor ownsWorkflowRunRepo gate, the hasConversationText mail drop, and the hub-api needs-you approval route carve-out. Their tests stay green. apps/sidecar keeps its old row: the execution host has not been converted off the retired lineage yet.
Upstream deleted the live-config wrap (wrapHarnessAsSingleStepWorkflow) along with the rest of the live-authored deploy chain. The wrap was a thin adapter over buildSingleStepAgentDefinition, which survives, so the launch now passes the resolved fields directly: the folded run's id, its system prompt, and its catalog-resolved inference preferences. Tools stay empty here — a folded launch pins its tools as packages, not factories.
Records the CL-6324 pin delta, the deltas that survived it, and why the app-side conversion is one migration rather than a per-tree bump: workbench has no code-sourced deploy front, and the retired live-authored chain is what every folded run launches through.
Red/green coverage for the onBodyFailure policy on the re-pinned runtime (CL-6326, CL-6324): default policy unchanged, "continue" re-arms past a failed occurrence while a cancelled one stays terminal-is-final, and crash-recovery honors the same policy. Adds projector coverage asserting a projected onTrigger section carries the authored policy through the live->inert projection, and omits the field when no policy was authored. Fails against the unmodified vendored runtime and projector.
…ection Re-applies the CL-6326 vendored delta on top of the re-vendored runtime: onBodyFailure?: "end" | "continue" on OnTriggerPrimitive/OnTriggerOpts (default "end", byte-compatible with prior behavior), read live by the steady-state drive loop and planOnTriggerResume so a "continue" section re-arms past a failed occurrence instead of ending the run. Cancellation is unaffected and always ends the section. Adds what the delta previously lacked: the live->inert projector's InertOnTrigger and projectOnTrigger now carry the field, so a section's policy survives the child->hub projection instead of being silently dropped before deploy. BodyFailurePolicy is exported from the definition barrel for the projector's type reference.
Red/green coverage for a shared-capacity code-sourced deploy that stamps a pre-existing anchor workflow_run instead of inserting one (CL-6324): the adoption succeeds and issues no INSERT, a definition carrying credential bindings fails closed when no cipher is threaded, and an anchor the tenant does not own is refused before any frame reaches the sidecar. Fails against the two upstream fronts, neither of which accepts a pre-existing anchor.
…hor run Neither code-sourced front could deploy onto a run whose anchor row already exists. deployWorkflowFromSource INSERTs its anchor (a primary-key collision against a folded run's row) and threads no credentialCipher; deployPreparedCodeSourcedWorkflow updates a pre-existing row and threads the cipher, but only under the allocation-ownership lock, so it cannot run on shared capacity. Adds a third front composed from the existing halves -- emitSourceRefDeployFrame and buildInertProjectionStepSources -- following the prepared front's semantics minus the allocation lock: ownership is the anchor row's own tenant plus self-anchoring, checked before the frame so a refused adoption leaves no deployed-but-unanchored agent, and re-asserted on the guarded UPDATE that stamps definitionId and publicKey. No deployer read grant is seeded: the anchor predates the call, so its grants belong to whoever created it.
…loy front Records both vendored deltas in VENDORED.md and each package's VENDORED-FROM, and re-records the workflow and hub-sessions tree hashes so check:killdates matches the edited trees.
Covers the deploy-time config contract (arktype-parsed, env-delivered) and the two definition shapes its mode selects: the folded unbounded step and the per-turn onTrigger section.
…deploy The workflow.json retirement makes source-ref the only deploy lineage: a deployment's definition is evaluated from its own pinned code closure and re-verified against the hub-approved wire hash. Workbench had no code-sourced package to deploy, so this adds the one every agent run will share. The bytes are static and versioned; everything per-run — mailbox, system prompt, inference chain, tool package pins, credential bindings — arrives as deploy-time config in the child's environment and is parsed at the entry module's boundary. The config's mode selects the shape, so the deploy front keeps one parameter set and never branches on step-vs-section.
The first shape here read the config from the child's environment. That cannot work: the approval probe and the run child each evaluate the entry module independently, and the hashed projection covers the trigger address, the system prompt, the (provider, model) pairs, the tool package pins, and the credential bindings — every field of the config. A config read from outside the closure diverges between the two evaluations and fails the re-verify barrier closed. There is also nowhere to read one from: no source variant carries an overlay, the deploy frame carries no config bag, and the probe frame carries no environment at all. So the config becomes the bytes. renderAgentRuntimeSourceTree emits a thin per-run package that pins this versioned one and calls the builder with the run's config as a literal, ready to commit into a workflow-kind asset and deploy as source at a commitSha — the only source variant cheap enough to mint per run.
Red/green coverage for the conversion (CL-6324): deployAtHead renders the run's per-run workflow source package, commits it into the run's own definition asset on a per-run ref, and deploys the resulting commitSha through the adopting code-sourced front against the pre-minted anchor. Covers the whole round trip -- the committed tree's shape, the config rendered into the deployed bytes (address, system prompt, model pairs, tool pins, credential bindings, mode), the adopted deploy's frame, the wake path taking the same route, a caller-supplied section mode riding through untouched, and a run whose definition has no workflow-kind asset failing before any deploy. Section mode is proven to author onBodyFailure "continue" and to keep it through the live->inert projection. Fails against the in-memory synthesize-and-deploy path, which neither renders bytes nor touches an asset.
…nition Cuts deployAtHead over to the code-sourced seam. The in-memory single-step definition it used to build and hand to deploySingleStepAtHead is gone -- that front was retired with the on-disk workflow.json, and a deployment's definition is now whatever its own pinned source closure evaluates to. The run's deploy-time config (trigger address, system prompt, resolved inference chain, tool package pins, credential bindings, shape) is rendered into a per-run @corbits/agent-runtime package, committed into the run's OWN definition asset on refs/heads/runs/<runId>, and deployed by pinning that commitSha. The config has to be inside the bytes: the approval probe and the run child evaluate the entry independently and the child refuses a definition whose recomputed wire hash differs, and every one of those fields is in the hashed preimage. The deploy goes through deployAdoptedWorkflowFromSource, the only front a folded run can use -- its anchor workflow_run row is minted before any deployment attaches to it, so the inserting front collides on the primary key and the prepared front needs an exclusive allocation it never has. The credential cipher is threaded instead of a pre-built delivery, since the front resolves the material itself from the deployed definition's own bindings; buildCredentialDelivery stays only for the credential: use grants the run's principal needs in its own grants.json. The step's input selector becomes the config's mode: `step` (with an optional literalInput, the workbench host's CL-6164 pin) or `section` with a per-turn timeout, so the Phase 1.3 swap changes a caller's argument rather than a branch here. Section mode authors onBodyFailure "continue" so one failed turn re-arms the section instead of retiring the run. hubPublicKey leaves FoldedRunsDeps: the adopting front does not take it, and nothing else in the folded-run path read it.
Records the conversion in the CL-6324 inventory: what deployAtHead now
does (render, commit into the run's own definition asset on a per-run
ref, deploy the pinned commit through the adopting front), why the asset
is reused rather than minted per deploy, and how the step/section shape
became config data.
Also records what still blocks EXECUTION -- CLOSURE_PACKAGE_DIR and the
sidecar's WorkflowProbeExecutor, the remaining typecheck failures -- and
a defect the conversion surfaced: apps/hub mints MCP credential handles
("mcp:<slug>") that the platform's ToolCredentialHandle grammar rejects,
which now fails closed at render time because the config is finally
parsed.
The deploy frame no longer carries a definition, so every sidecar test that built one now stages a source-ref pin and registers what the pinned closure evaluates to through an injected materializer. The lifecycle suite asserts the new contract directly: an onTrigger body's sources.json is staged and its definition is not, the child's env carries the materialized closure dir and the hub-approved wire hash, and the durable record carries the pin a restore re-materializes from. The step-coverage gate moves with it -- the frame's arktype can no longer narrow a table it cannot see, so coverage is checked against the closure-derived definition.
The sidecar no longer writes a definition into its deploy tree and reads it back. A deploy materializes the frame's frozen closure, evaluates the pinned code, and runs THAT; the boot-time restore re-materializes the same pin and re-derives the same definition, so both paths reach the runnable definition by one computation. The child gets the closure dir plus the hub-approved wire hash it re-verifies its own projection against, and the durable record carries the pin a restore replays. The pieces that make it executable: - Closure staging (`workflow-host-wiring/closure-staging.ts`) owns the durable per-deployment source stores, the mount resolution both paths derive from the pin alone, and the apply. Injectable so a test can stand in for fetch + SRI-verify + layout + evaluate. - A `WorkflowProbeExecutor` is wired at the boot edge, so a probe answers with a real inert projection and its wire hash instead of the hub-link's rejecting placeholder. Its airlocked child, the closure materializer, the closure apply, and the inline source-asset delivery come from upstream's own sidecar at `4ed8baf4` (see VENDORED.md). - Child spawns are in-memory: a rung lifts its inline children to refs and serves grandchildren from that map, so no rung reads a definition off disk at any depth. - `WORKFLOW_DEFINITION_REPO_ID`/`_REF` are gone. What survives is `WORKFLOW_DEFINITION_ID` — identity for the run-authenticated capabilities route a step tool calls, never a repo to read from.
Closes the three sidecar rows on the conversion table and records what the conversion did not prove: no run has executed end to end, the MCP credential-handle defect still gates a real launch, every test injects the closure materializer, and the pinned tool-package arm was left in place deliberately where upstream went all-source-tools.
…envelope A workflow-kind asset now accepts only a source codebase declaring an interchange.workflow entry; the envelope form is rejected at push time with a path-violation, which failed every seed run outright. The pusher renders the serialized definition into the two-file tree that form takes -- a package.json naming the entry, and the entry module default-exporting the definition -- so a code-sourced deploy evaluates the same definition it used to re-read off disk.
The deployments route now takes the code-sourced pair -- a `source`
naming the asset plus `package: { format: "source", commitSha }`, and
the `entry` the package.json declares -- so the bare `assetId` body the
seed sent is rejected outright. The pusher is the only place that knows
which commit the asset's main now sits at, so it reports the sha and the
deploy pins it.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebases the whole CL-6324 stack onto
main(so this PR bases main, notcl-6324-sidecar-closure) and then boots it for real — scratch database, real signup, real local Ollama, nothing mocked — to find out what the conversion actually costs.Rebase
18 commits replayed onto
main. Conflicts resolved as:mainwins for everything it owns, the stack wins at conversion sites.vendor/intx/workflow-host/src/index.ts— keptmain's workbench-owned adapter exports (action-invoker,effect-ledger) alongside the re-pin's renamedcreateInMemorySpawnChild.packages/chat/test/*— honoredmain's room-store cutover: the two suites it deleted stay deleted, and thelistMailblock it removed stays removed.packages/folded-runs/src/launch.ts— took the converted body; Fix MCP credential-handle grammar violation (mcp:<slug> -> mcp.<slug>) #98'smcp.<slug>rename is comment-only there and its real change (mcp-credential-bindings.ts,mcp-tools/src/tool.ts) merged clean.bun run typecheckexits 0.hub-client,onboarding,cli,chat,folded-runs,agent-runtimeandapps/sidecarsuites are green;check:killdatesis ok.What the real boot found
The proof ran the seed and launch path end to end against a scratch stack. It got to a fully seeded tenant with every default workflow deployed by source-ref and stopped at the folded launch. Three defects fixed forward, two conversion sites left open and named.
Fixed
@corbits/mcp-toolsshipped newsrc/under an unchanged0.0.4— Fix MCP credential-handle grammar violation (mcp:<slug> -> mcp.<slug>) #98's handle change edited the source and left the version, whichassertToolPackagesFreshrefuses. The seed's tool-registry publish failed before any workflow deployed. Bumped to0.0.5.workflow.jsonenvelope into workflow assets, which the asset gate now rejectspath-violation. The pusher renders the serialized definition into the source form the gate takes: apackage.jsonnaming aninterchange.workflowentry, plus that entry default-exporting the definition.assetId; the route now takessource+package: { format: "source", commitSha }+entry. The pusher reports the sha it left onmainand the deploy pins it.Still open (the milestone is behind these)
packages/folded-runs/src/definition.ts'sreadDefinitionJSONreads a folded launch body out ofworkflow.json, which a source-format asset does not carry — minting a chat 409snot_launchable. The same read appears four more times inpackages/agent-directory/src/routes.ts. Not mechanical: under the retirement a definition's body is whatever its closure evaluates to, and nothing hub-side persists that evaluated projection for a shared deploy (workflow_definitionholds the wire hash and manifests;workflow_run_launch_spec's frozen bundle covers exclusive placement only). Where a folded launch body comes from is the next design call.deploy/tool-packages-manifest.jsonis staged any more — the answer to the open question. The source-ref front never callsexecuteLaunchPhases, sowriteDeployTreenever runs and no deploy tree lands for the step;materializeStepToolsreads its manifest off that tree, so a folded run's pinned tool packages would materialize empty. The prompt moved into the rendered bytes, the tool manifest did not.stageWorkflowStepis the seam that still writes one.docs/revendor-inventory.mdcarries all five with the same detail.Not proven
RunStarted, the real reply, the timings, and the mid-turn sidecar restore all sit behind (4). No transcript is claimed for them.