chore(harness): orchestrate the 0.0.6 runtime and public-surface lane - #1525
Conversation
Bootstraps .llm/runs/release-0.0.6-features--orchestration/ for the topical runtime/public-surface lane that owns #1405 and #1398. Records identity/worktree proof, the live re-baseline of both issue bodies, the lane bindings and their two recorded deviations, the PLAN-EVAL/IMPL-EVAL decisions, and the #1405 research with line-cited call sites for both misreported reason strings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Records the early-terminated #1398 research with its unverified list intact, and plans the fix around two facts checked in-session: job.execute inherits the stored dispatch traceparent, and the stream publish span starts on the ambient context. Publishing execution mutations under the record's stored trace context therefore joins every record -- including the pre-span create() one -- to the job.execute trace, which is what the deferred TC-14 gate asserts. Makes the issue's definition of done mechanical: the two Flow-B OTEL gates deferred against #1398 come out of the deferral list and pass live. Also records the #1405 Tier-D dispatch identity and corrects an earlier worklog line that read runtime doctor "sessions: 0" as "nothing is running". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Re-verifies the slice independently (33/33) and demonstrates the new guards actually fire by reverting both fixes and observing 5 failures, per the gate-integrity rule that a guard enters only with its predicate proven. Records two advisory findings, and a defect in the orchestrator's own slice brief that the implementer surfaced rather than hid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
workers-combined does receive services__streams__http__0: background PluginReferences are reconciled from the plugin manifest dependency, asserted by install-plugin_test.ts, not declared in the Aspire contribution file. Records that the orchestrator's first reading of the contribution file led to the opposite conclusion and would have added an unnecessary Aspire slice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
MiniMax M3 separate-session verdict, verbatim. Folds in two verified findings: the trace-context join must explicitly wrap producer.upsert in context.with because StreamsTracerPort.startSpan takes no parent context, and D5 must update both tests that pin the deferral, not just the one the plan named. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Records the PLAN-EVAL PASS detail, the two findings verified and folded in, and the Tier-D dispatch identity for the #1398 implementation thread. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
DeepSeek V4 Flash 0731 verbatim verdict. Its per-fix revert isolation is what makes acceptance box 4 real: reverting each fix alone fails only that fix's tests, so the reasons are pinned individually rather than in aggregate. Records why the unreachable ?? fallback stays -- the suggested cleanup needs a non-null assertion the slice brief forbids -- and that the apparently-missing research file was an artifact of the evaluator worktree's base, not a gap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Captures merge 8ff1bcb from origin/main first-parent history after the fact, with the seven-check pre-merge gate record and the draft-CI skipping trap that check 4 caught before it could be read as green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
…y fixes Owner ruling: the #1405 class does not get a separate evaluator; focused negative tests, CI, close-gate, and independent diff review are sufficient. Records that I had read the waiver as a blocked-transport fallback rather than the class default, which cost one unnecessary dispatch, and that the ruling arrived after #1405 had already merged so it does not retract that record. Formal PLAN/IMPL eval remains mandatory for #1398. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
…t to CI Neither local scaffold.runtime run reached the restored OTEL gates -- run 1 died at fixture generation on a transient fetch, run 2 at a triggers-api health timeout after getting 17 steps further. Acceptance criterion 3 stays unproven and the PR cannot merge. States what the evidence does and does not support: the triggers-api timeout is consistent with an environmental failure but is not proven to be one, because the suite was not reproduced on a clean main checkout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
…spatcher Owner ruling: once #1524 lands, phase evaluations use the automatic status workflow unless the owner documents a local route or skip. #1536 stays on its current head and status; root re-enters status:impl-eval with the Qwen override after #1524 merges, and this lane only watches the verdict and finishes the merge gate. Records that the anticipated local #1536 evaluator was never launched -- prompt and worktree were prepared, the decision was raised instead, and no evaluator process or output exists -- so no duplicate spend occurred. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Both formerly-deferred OTEL gates ran by name with failed=0 skipped=0 on the postgres and sqlite tiers, verified from job logs. Acceptance criterion 3 is live-verified. Retires the open question about the two red local runs: CI runs the same suite with this change to a clean finish, which is the control I lacked, so the local triggers-api timeout is established as environmental rather than merely consistent with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Files the quality:gate root-coverage gap (hit independently by three sessions, and the reason the #1405 merge record rests on a target scan rather than the repo gate) and the undeclared plugin-streams-core imports, the latter filed as explicitly unverified with publish:dry-run evidence as its first acceptance box. Records the waiting state for #1536: #1524 is still open, so no automatic verdict exists and the merge gate cannot proceed. The staged body transform is dry-run only and deliberately unapplied. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
#1524 merged 7837ef4; the automatic phase dispatcher is live and D-4 routing is in force. Records #1536 verified unchanged at that moment and the exact baseline the verdict watch measures against. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Formal PLAN/IMPL eval triggers on labels, never a manual OpenHands dispatch. This lane already complies: no manual dispatch, and no local evaluator was launched for #1536. Records a measured timing finding: #1536's ready_for_review (08:53:43Z) and its status:impl-eval application (08:53:45Z) both precede the dispatcher merging (09:24:15Z), so the initial automatic IMPL-EVAL could not have fired and will not fire on its own -- the label re-entry is required, not merely planned. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Label re-entry executed exactly as instructed and confirmed clean, but the phase-eval workflow produced no run: openhands-phase-eval.yml is absent from head e4319c6, whose last main sync (08:45:46Z) predates #1524 merging (09:24:15Z). pull_request events resolve workflows from the PR merge ref, so no label cycling can trigger it. main has also moved to #1547, which fixes the dispatcher's own token step. Syncing the branch changes the head and forces a full CI re-run, so it is raised rather than taken unilaterally. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Owner-approved head change to f7d503f brings the phase-eval workflow and #1547's token fix into the merge ref. Same labels and sequence that produced no run on the old head now produce a successful dispatch, which is the control for the diagnosis. Replaces the comment-count watcher: OpenHands updates its summary comment in place, so a counting watcher would have polled to timeout while the verdict sat in an edited comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Both OTEL gates re-read by name from f7d503f's job logs (94073971396, 94073971501). The differing job ids are direct proof that reusing the pre-sync verification would have cited a head no longer on the PR; the body transform now requires head/job ids as arguments and asserts no stale reference survives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Both owned issues landed: #1405 at 8ff1bcb, #1398 at d7e2b67. Records the Qwen IMPL-EVAL PASS, the two problems the merge gate caught (a duplicate status label from automation, and a stale close-gate result whose job predated the ready-merge label), and the head-change discipline that kept pre-sync gate evidence out of the merge record. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Fills the failure-mode table with eight time-costing failures and their mitigations, and records a factual retrospective: what the lane produced, where layered review caught what the layer above missed, four mistakes this orchestrator made, and four candidate rules explicitly not promoted from a single occurrence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
|
[PHASE: REVIEW] — lane closeout Both owned issues are on
Both merged through the seven-check pre-merge gate ( #1398's acceptance was made mechanical rather than asserted. Its "live subscription observes the record within a bounded time" criterion was bound to two gates the repo had already deferred against this very issue. Done meant removing the deferral and having the gates pass — which they now do, by name, on both CI runtime tiers ( Filed from inside the run: #1542 ( Retrospective: Canary and stable cut remain root's; nothing here was published. |
Adds the mandatory context-pack and captures the #1398 Qwen IMPL-EVAL verdict as a run artifact rather than leaving it only as a PR comment. Untracks the two raw evaluator JSONL streams: they were 2.4MB of a 2.5MB run dir against a 96K largest-artifact precedent in the 0.0.5 run, which tracks no raw streams at all. They move to gitignored .llm/tmp scratch and remain on disk; their substance is already verbatim in plan-eval.md and evaluate-1405.md with run id, duration, event count and is_error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Seven checks at head bdc62b0. Check 6 is the load-bearing one for an evidence-lane PR: explicit grep confirms no packages/** or plugins/** source rode along, all 20 changed paths being under .llm/runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Summary
Control PR for the 0.0.6 runtime / public-surface lane. It carries orchestration evidence only —
the run dir under
.llm/runs/release-0.0.6-features--orchestration/and nothing else. It is notan implementation umbrella: both owned issues land through their own fresh leaf branches with draft
PRs opened directly against
main.Scope
.llm/runs/**)Two-PR dependency plan
fix/1405-durable-producer-rejection-taxonomyfix/1398-publish-job-executions-to-durable-streamWhy this order, and what the dependency actually is. #1398 publishes job executions through
DurableStreamProducerand must prove a live subscription observes the record within a boundedtime. When that proof fails, the settled write reason is the first diagnostic anyone reads — and
today two of those reasons misdescribe the state that produced them (#1405). Landing #1405 first
means #1398's failure diagnostics are trustworthy while its runtime evidence is being gathered. The
dependency is diagnostic, not structural: #1398 does not import anything #1405 adds, so if #1405
were blocked, #1398 could still proceed with the drift recorded.
#1398 builds on merged #1395 (versioned SSE/OTEL envelope, merged 2026-08-09T01:25:15Z) and #1402
(durable producer reconnect supervisor, merged 2026-08-09T05:11:32Z) — both ancestors of this
branch's baseline — but is not satisfied by them. It stays its own reviewable PR.
Slices
a4964f9afa4964f9af8ff1bcb8fd7e2b67b2cut-trace.mdcomplete (2 merges, 2 re-planning events, 8 time-costing failures),retrospective.mdwritten,context-pack.mdwrittenValidation
Bootstrap commit touches
.llm/runs/**only — no publishable source, so no package gate applies tothis PR. Gate evidence for the owned issues lands on their own PRs, not here.
Environment proof recorded in
worklog.md:git fetch origin main→origin/main@01aa12b67; lane opened at tip, clean tree.deno task agentic:runtime doctor→no_change (schema 1.0), components 18, sessions 0.Harness
.llm/runs/release-0.0.6-features--orchestration/.llm/harness/workflow/milestone-run.md(topical lane), role skillagent-milestone-orchestrator.lane-policy owner decision of 2026-08-08). Required for fix(triggers/streams): job executions are never published to the durable job stream, so no execution trace can span producer → stream → consumer #1398, separate session, before
implementation. No separate PLAN-EVAL for the two-PR sequencing plan itself; that decision is
recorded in
supervisor.mdrather than assumed.the
milestone-run.mdpre-merge gate per PR.Drift / Debt
canonical
deep_analysisFable 5 medium, on explicit owner instruction. Opposite-family review ofCodex work is unaffected: Fable 5 low remains the
review_codexreviewer for the fix(triggers/streams): job executions are never published to the durable job stream, so no execution trace can span producer → stream → consumer #1398 slice.Observed 2026-08-12: PR feat(agentic): automate formal evaluator phases #1524 is OPEN,
mergedAt: null, with its bounded-live-smoke andrepository-default-variable DoD boxes unticked. Per the brief's own condition this lane uses fresh
local evaluator sessions; re-checked before each dispatch.
Definition of Done
main(8ff1bcb8f) through the seven-check pre-merge gate; all 5 acceptance boxes ticked by the evidence mirror, not by hand —slices/pre-merge-gate-1528.mdmain(d7e2b67b2) through the seven-check pre-merge gate; both formerly-deferred OTEL gates pass by name on both CI tiers at the merging head (postgres94073971396passed=88 failed=0 skipped=0; sqlite94073971501passed=83 failed=0 skipped=0); all 4 acceptance boxes mirrored —slices/pre-merge-gate-1536.mdcut-trace.mdrecords both merges captured fromgit log origin/main --first-parentafter each merge, plus 2 re-planning events and 8 time-costing failures with mitigationsretrospective.mdwritten: what the lane produced, where each review layer caught what the layer above missed, four mistakes this orchestrator made, and four candidate rules explicitly not promoted from a single occurrence