Skip to content

chore(harness): orchestrate the 0.0.6 runtime and public-surface lane - #1525

Merged
rickylabs merged 22 commits into
mainfrom
chore/release-0.0.6-features-orchestration
Aug 12, 2026
Merged

chore(harness): orchestrate the 0.0.6 runtime and public-surface lane#1525
rickylabs merged 22 commits into
mainfrom
chore/release-0.0.6-features-orchestration

Conversation

@rickylabs

@rickylabs rickylabs commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Summary

Control PR for the 0.0.6 runtime / public-surface lane. It carries orchestration evidence only —
the run dir under .llm/runs/release-0.0.6-features--orchestration/ and nothing else. It is not
an implementation umbrella: both owned issues land through their own fresh leaf branches with draft
PRs opened directly against main.

Scope

Two-PR dependency plan

Order Issue PR Shape Depends on
1 #1405 fix/1405-durable-producer-rejection-taxonomy small, specified, deterministic; reasons-only nothing — lands first
2 #1398 fix/1398-publish-job-executions-to-durable-stream substantive public/runtime behaviour + live E2E soft-depends on #1405

Why this order, and what the dependency actually is. #1398 publishes job executions through
DurableStreamProducer and must prove a live subscription observes the record within a bounded
time. When that proof fails, the settled write reason is the first diagnostic anyone reads — and
today two of those reasons misdescribe the state that produced them (#1405). Landing #1405 first
means #1398's failure diagnostics are trustworthy while its runtime evidence is being gathered. The
dependency is diagnostic, not structural: #1398 does not import anything #1405 adds, so if #1405
were blocked, #1398 could still proceed with the drift recorded.

#1398 builds on merged #1395 (versioned SSE/OTEL envelope, merged 2026-08-09T01:25:15Z) and #1402
(durable producer reconnect supervisor, merged 2026-08-09T05:11:32Z) — both ancestors of this
branch's baseline — but is not satisfied by them. It stays its own reviewable PR.

Slices

Validation

Bootstrap commit touches .llm/runs/** only — no publishable source, so no package gate applies to
this PR. Gate evidence for the owned issues lands on their own PRs, not here.

Environment proof recorded in worklog.md:

  • git fetch origin mainorigin/main@01aa12b67; lane opened at tip, clean tree.
  • deno task agentic:runtime doctorno_change (schema 1.0), components 18, sessions 0.

Harness

Drift / Debt

Definition of Done

Bootstraps .llm/runs/release-0.0.6-features--orchestration/ for the topical
runtime/public-surface lane that owns #1405 and #1398.

Records identity/worktree proof, the live re-baseline of both issue bodies,
the lane bindings and their two recorded deviations, the PLAN-EVAL/IMPL-EVAL
decisions, and the #1405 research with line-cited call sites for both
misreported reason strings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
rickylabs and others added 5 commits August 12, 2026 09:49
Records the early-terminated #1398 research with its unverified list intact,
and plans the fix around two facts checked in-session: job.execute inherits the
stored dispatch traceparent, and the stream publish span starts on the ambient
context. Publishing execution mutations under the record's stored trace context
therefore joins every record -- including the pre-span create() one -- to the
job.execute trace, which is what the deferred TC-14 gate asserts.

Makes the issue's definition of done mechanical: the two Flow-B OTEL gates
deferred against #1398 come out of the deferral list and pass live.

Also records the #1405 Tier-D dispatch identity and corrects an earlier worklog
line that read runtime doctor "sessions: 0" as "nothing is running".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Re-verifies the slice independently (33/33) and demonstrates the new guards
actually fire by reverting both fixes and observing 5 failures, per the
gate-integrity rule that a guard enters only with its predicate proven.

Records two advisory findings, and a defect in the orchestrator's own slice
brief that the implementer surfaced rather than hid.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
workers-combined does receive services__streams__http__0: background
PluginReferences are reconciled from the plugin manifest dependency, asserted
by install-plugin_test.ts, not declared in the Aspire contribution file.

Records that the orchestrator's first reading of the contribution file led to
the opposite conclusion and would have added an unnecessary Aspire slice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
MiniMax M3 separate-session verdict, verbatim. Folds in two verified findings:
the trace-context join must explicitly wrap producer.upsert in context.with
because StreamsTracerPort.startSpan takes no parent context, and D5 must update
both tests that pin the deferral, not just the one the plan named.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Records the PLAN-EVAL PASS detail, the two findings verified and folded in, and
the Tier-D dispatch identity for the #1398 implementation thread.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
rickylabs and others added 14 commits August 12, 2026 10:11
DeepSeek V4 Flash 0731 verbatim verdict. Its per-fix revert isolation is what
makes acceptance box 4 real: reverting each fix alone fails only that fix's
tests, so the reasons are pinned individually rather than in aggregate.

Records why the unreachable ?? fallback stays -- the suggested cleanup needs a
non-null assertion the slice brief forbids -- and that the apparently-missing
research file was an artifact of the evaluator worktree's base, not a gap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Captures merge 8ff1bcb from origin/main first-parent history after the fact,
with the seven-check pre-merge gate record and the draft-CI skipping trap that
check 4 caught before it could be read as green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
…y fixes

Owner ruling: the #1405 class does not get a separate evaluator; focused
negative tests, CI, close-gate, and independent diff review are sufficient.
Records that I had read the waiver as a blocked-transport fallback rather than
the class default, which cost one unnecessary dispatch, and that the ruling
arrived after #1405 had already merged so it does not retract that record.

Formal PLAN/IMPL eval remains mandatory for #1398.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
…t to CI

Neither local scaffold.runtime run reached the restored OTEL gates -- run 1 died
at fixture generation on a transient fetch, run 2 at a triggers-api health
timeout after getting 17 steps further. Acceptance criterion 3 stays unproven
and the PR cannot merge.

States what the evidence does and does not support: the triggers-api timeout is
consistent with an environmental failure but is not proven to be one, because
the suite was not reproduced on a clean main checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
…spatcher

Owner ruling: once #1524 lands, phase evaluations use the automatic status
workflow unless the owner documents a local route or skip. #1536 stays on its
current head and status; root re-enters status:impl-eval with the Qwen override
after #1524 merges, and this lane only watches the verdict and finishes the
merge gate.

Records that the anticipated local #1536 evaluator was never launched -- prompt
and worktree were prepared, the decision was raised instead, and no evaluator
process or output exists -- so no duplicate spend occurred.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Both formerly-deferred OTEL gates ran by name with failed=0 skipped=0 on the
postgres and sqlite tiers, verified from job logs. Acceptance criterion 3 is
live-verified.

Retires the open question about the two red local runs: CI runs the same suite
with this change to a clean finish, which is the control I lacked, so the local
triggers-api timeout is established as environmental rather than merely
consistent with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Files the quality:gate root-coverage gap (hit independently by three sessions,
and the reason the #1405 merge record rests on a target scan rather than the
repo gate) and the undeclared plugin-streams-core imports, the latter filed as
explicitly unverified with publish:dry-run evidence as its first acceptance box.

Records the waiting state for #1536: #1524 is still open, so no automatic
verdict exists and the merge gate cannot proceed. The staged body transform is
dry-run only and deliberately unapplied.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
#1524 merged 7837ef4; the automatic phase dispatcher is live and D-4 routing
is in force. Records #1536 verified unchanged at that moment and the exact
baseline the verdict watch measures against.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Formal PLAN/IMPL eval triggers on labels, never a manual OpenHands dispatch.
This lane already complies: no manual dispatch, and no local evaluator was
launched for #1536.

Records a measured timing finding: #1536's ready_for_review (08:53:43Z) and its
status:impl-eval application (08:53:45Z) both precede the dispatcher merging
(09:24:15Z), so the initial automatic IMPL-EVAL could not have fired and will
not fire on its own -- the label re-entry is required, not merely planned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Label re-entry executed exactly as instructed and confirmed clean, but the
phase-eval workflow produced no run: openhands-phase-eval.yml is absent from
head e4319c6, whose last main sync (08:45:46Z) predates #1524 merging
(09:24:15Z). pull_request events resolve workflows from the PR merge ref, so no
label cycling can trigger it.

main has also moved to #1547, which fixes the dispatcher's own token step.
Syncing the branch changes the head and forces a full CI re-run, so it is
raised rather than taken unilaterally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Owner-approved head change to f7d503f brings the phase-eval workflow and
#1547's token fix into the merge ref. Same labels and sequence that produced no
run on the old head now produce a successful dispatch, which is the control for
the diagnosis.

Replaces the comment-count watcher: OpenHands updates its summary comment in
place, so a counting watcher would have polled to timeout while the verdict sat
in an edited comment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Both OTEL gates re-read by name from f7d503f's job logs (94073971396,
94073971501). The differing job ids are direct proof that reusing the pre-sync
verification would have cited a head no longer on the PR; the body transform now
requires head/job ids as arguments and asserts no stale reference survives.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Both owned issues landed: #1405 at 8ff1bcb, #1398 at d7e2b67. Records the
Qwen IMPL-EVAL PASS, the two problems the merge gate caught (a duplicate status
label from automation, and a stale close-gate result whose job predated the
ready-merge label), and the head-change discipline that kept pre-sync gate
evidence out of the merge record.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
Fills the failure-mode table with eight time-costing failures and their
mitigations, and records a factual retrospective: what the lane produced, where
layered review caught what the layer above missed, four mistakes this
orchestrator made, and four candidate rules explicitly not promoted from a
single occurrence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
@rickylabs

Copy link
Copy Markdown
Owner Author

[PHASE: REVIEW] — lane closeout

Both owned issues are on main. This control PR carries orchestration evidence only and closes nothing.

Issue PR Merge Closed
#1405 durable producer rejection taxonomy #1528 8ff1bcb8f auto, COMPLETED
#1398 job executions → durable job stream #1536 d7e2b67b2 auto, COMPLETED

Both merged through the seven-check pre-merge gate (slices/pre-merge-gate-1528.md, slices/pre-merge-gate-1536.md), each with review-threads clean.

#1398's acceptance was made mechanical rather than asserted. Its "live subscription observes the record within a bounded time" criterion was bound to two gates the repo had already deferred against this very issue. Done meant removing the deferral and having the gates pass — which they now do, by name, on both CI runtime tiers (failed=0 skipped=0).

Filed from inside the run: #1542 (quality:gate roots omit published packages — both merge records in this lane explicitly rest on explicit target scans rather than the repo gate) and #1543 (undeclared plugin-streams-core imports, filed as unverified with publish:dry-run evidence as its first acceptance box).

Retrospective: .llm/runs/release-0.0.6-features--orchestration/retrospective.md. It records eight time-costing failures with mitigations, four mistakes this orchestrator made, and four candidate rules explicitly not promoted — each appeared once and is left for a future run to confirm or falsify.

Canary and stable cut remain root's; nothing here was published.

@rickylabs rickylabs added status:impl and removed status:plan Harness plan phase labels Aug 12, 2026
Adds the mandatory context-pack and captures the #1398 Qwen IMPL-EVAL verdict
as a run artifact rather than leaving it only as a PR comment.

Untracks the two raw evaluator JSONL streams: they were 2.4MB of a 2.5MB run
dir against a 96K largest-artifact precedent in the 0.0.5 run, which tracks no
raw streams at all. They move to gitignored .llm/tmp scratch and remain on disk;
their substance is already verbatim in plan-eval.md and evaluate-1405.md with
run id, duration, event count and is_error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
@rickylabs
rickylabs marked this pull request as ready for review August 12, 2026 10:45
Seven checks at head bdc62b0. Check 6 is the load-bearing one for an
evidence-lane PR: explicit grep confirms no packages/** or plugins/** source
rode along, all 20 changed paths being under .llm/runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TKxrWGp5uxEHQ2NyZiZSMF
@rickylabs
rickylabs merged commit 0f0b6b6 into main Aug 12, 2026
15 checks passed
@github-actions github-actions Bot added the canary:0.0.6-canary.2 Published NetScript prerelease 0.0.6-canary.2 label Aug 12, 2026
@rickylabs
rickylabs deleted the chore/release-0.0.6-features-orchestration branch August 13, 2026 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:tooling canary:0.0.6-canary.2 Published NetScript prerelease 0.0.6-canary.2 ci:skip-e2e ci:skip-scaffold Skip the scaffold-static (deno-only) scaffold gate status:shipped type:chore Tooling, config, or housekeeping

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant