From 69813c8f3621bf5489be8e5729e6e6f4e768969c Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 09:46:37 +0200 Subject: [PATCH 01/57] chore(harness): activate the 0.0.6 internals lane run with executed baselines Stage A/B of the topical milestone run for the 0.0.6 chores/internals lane (#1436, #1415, #1403, #1380, #1378). Records what was executed rather than what was assumed: identity/worktree proof, all five live issue bodies with a 30-box acceptance inventory, and the dispatch preconditions (runtime doctor, managed Codex daemon, live-turn provider verification) that milestone-run stage B requires as recorded output. The re-baseline falsified #1436's own prescribed fix: the word boundary it asks for is already present, and \b is why the defect survives, because '-' is a non-word character. evidence/probe-1436-baseline.ts is the executed proof, and it found a second instance ('un-fixed #555') the issue does not mention. Refs #1436, #1415, #1403, #1380, #1378 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../context-pack.md | 62 ++++++ .../cut-trace.md | 43 ++++ .../drift.md | 64 ++++++ .../evidence/probe-1436-baseline.ts | 31 +++ .../plan.md | 200 ++++++++++++++++++ .../supervisor.md | 83 ++++++++ .../worklog.md | 187 ++++++++++++++++ 7 files changed, 670 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/context-pack.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/cut-trace.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/drift.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/evidence/probe-1436-baseline.ts create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/supervisor.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/worklog.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md b/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md new file mode 100644 index 0000000000..765c4cd278 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md @@ -0,0 +1,62 @@ +# Context Pack — release-0.0.6-internals--orchestration + +Resumable summary. Read this first when resuming; then `supervisor.md`, `plan.md`, `worklog.md`, +`cut-trace.md`, `drift.md`. + +## What this run is + +The **0.0.6 chores/internals lane**: a topical milestone orchestration owning exactly five issues — +internal quality and gate correctness. It coordinates and delegates; it does not author tooling or +framework implementation code, and it does not publish. Root 0.0.6 orchestration owns canary and the +stable cut. + +## Where it stands + +| Stage | Status | +| --- | --- | +| A — Bootstrap | **done** — identity/worktree proved, five issue bodies read live, 30 acceptance boxes inventoried, run dir written | +| B — Wave plan | **done** — `plan.md` committed; dispatch preconditions executed and GREEN; wave-plan PLAN-EVAL decision recorded | +| C/D — Wave 1 (PR-A) | next | +| C/D — Wave 2 (PR-B → PR-C → PR-D) | pending | +| E — Canary | N/A for this lane (`drift.md` D-3) | +| G — Close | pending | + +## The four PRs + +| PR | Branch | Closes | Lane | Eval | +| --- | --- | --- | --- | --- | +| PR-A | `fix/1436-1415-close-gate-trust` | #1436, #1415 | Sol · low | both waived (owner, `drift.md` D-1); negative cases proven instead | +| PR-B | `fix/1403-quality-gate-coverage` | #1403 | Sol · low | rail PLAN-EVAL + own IMPL-EVAL | +| PR-C | `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | rail PLAN-EVAL + own IMPL-EVAL | +| PR-D | `fix/1378-quality-scan-rule-power` | #1378 | Sol · high | rail PLAN-EVAL + own IMPL-EVAL | + +Strictly sequential (one active implementation thread). Sequencing locks S-1…S-6 in `plan.md`. + +## The three facts most likely to be lost + +1. **#1436's prescribed fix is a no-op.** The `\b` it asks for is already in the code; `\b` is the + *cause*, because `-` is a non-word character. The fix needs a `(? [1431] and `un-fixed #555` -> [555]. + * Expected after PR-A: both -> []. + */ +import { extractClosingIssues } from '../../../tools/validation/acceptance-evidence.ts'; + +const CASES = [ + 'Exact pre-fix #1431 head', + 'pre-repair #1431 head', + 'Fixes #1434', + 'hotfix #999 landed', + 'prefixes #888 there', + 'This is a bugfix #777', + 'un-fixed #555', + 'Closes #1234 and fixes #4321', + 'Refs #111', + 'Part of #222', + 'resolves https://github.com/rickylabs/netscript/issues/333', +]; + +for (const input of CASES) { + console.log(`${JSON.stringify(input)} -> ${JSON.stringify(extractClosingIssues(input))}`); +} diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan.md b/.llm/runs/release-0.0.6-internals--orchestration/plan.md new file mode 100644 index 0000000000..d2f04c327f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan.md @@ -0,0 +1,200 @@ +# Wave Plan: 0.0.6 chores/internals lane + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `release-0.0.6-internals--orchestration` | +| Branch | `chore/release-0.0.6-internals-orchestration` | +| Phase | `plan` (stage B of `workflow/milestone-run.md`) | +| Target | repo tooling — `.llm/tools/validation/`, `.llm/tools/quality/`, `.llm/tools/fitness/`, `deno.json` tasks, `docs/architecture/doctrine/` | +| Archetype | N/A — no `packages/**` or `plugins/**` source is authored by this lane | +| Scope overlays | none (`SCOPE-docs.md` applies to PR-C's doctrine-document half) | + +## Goal + +Make the repo's own gates trustworthy, then make them informative, and close the five owned issues +through green leaf PRs to `main` with acceptance that is truthfully tickable. + +The lane's thesis: **a gate that invents a requirement (#1436), a gate that accepts its own IOU +(#1415), and a gate whose pass is indistinguishable from a did-not-run (#1403/#1378/#1380) are three +faces of one defect class.** Gate trust is sequenced first because the later PRs are merged *through* +those gates — repairing the quality rail while the close-gate can still fabricate or absolve +requirements would mean landing the rail on unverified acceptance. + +## Live re-baseline (executed, not carried in) + +Executed at `01aa12b67` on 2026-08-12 in this worktree. Every count below is a command result. + +| Claim under test | Source | Executed result | +| --- | --- | --- | +| `extractClosingIssues` lacks word boundaries | #1436 "Fix" section | **Falsified as stated.** `acceptance-evidence.ts:43` already carries `\b(?:close\|closes\|…)` and already strips fenced blocks (`:47`, landed by #1303). `hotfix #999`, `prefixes #888`, `bugfix #777` all correctly return `[]`. | +| `pre-fix #1431` is mis-parsed | #1436 observed instance | **Confirmed at HEAD.** `'Exact pre-fix #1431 head'` → `[1431]`. Second instance found by this lane: `'un-fixed #555'` → `[555]`. | +| Mirror validates presence, not assertion | #1415 | **Confirmed.** `validateEvidenceMapping` (`acceptance-evidence.ts:142`) rejects only `!entry.evidence.trim()`; any non-empty string ticks the box. | +| 36 live units under `packages/` + `plugins/` | #1380 D6 | **Confirmed.** `ls -d packages/*/` → 30, `ls -d plugins/*/` → 6. | +| `quality:scan` default roots are `packages/cli/src` + `plugins` | #1403, #1378 | **Confirmed.** `scan-code-quality.ts:18` `DEFAULT_ROOTS = ['packages/cli/src', 'plugins']`. | +| `arch:check` omits `packages/plugin-streams-core` | #1403 | **Confirmed.** `deno.json:156` names 16 roots; `plugin-streams-core` is the only `plugin-*-core` absent. | +| `arch:check:repo` runs `check-doctrine.ts` with no `--root` | #1380 D8 | **Confirmed.** `deno.json:157` is bare. | + +### Finding F-1 (raised now, must reach PR-A's brief) + +**#1436's proposed fix does not fix #1436.** The issue prescribes adding `\b` to the keyword +alternation. `\b` is already present, and it is *why* the bug survives: `-` is a non-word character, +so `\bfix\b` matches the `fix` inside `pre-fix` and inside `un-fixed`. The correct predicate excludes +a preceding hyphen as well as a preceding word character — a lookbehind of the +`(? [1431] # the reported defect, reproduced at 01aa12b67 +"un-fixed #555" -> [555] # second instance, NOT in the issue +"hotfix #999 landed" -> [] # issue predicts this breaks; it does not +"prefixes #888 there" -> [] # ditto +"This is a bugfix #777" -> [] # ditto +"Closes #1234 and fixes #4321" -> [1234,4321] +"Refs #111" / "Part of #222" -> [] +"resolves https://github.com/rickylabs/netscript/issues/333" -> [333] +``` + +The word boundary the issue asks for is already at `acceptance-evidence.ts:43` (landed by #1303), and +`\b` is the *cause*: `-` is a non-word character, so `\bfix\b` matches inside `pre-fix`. + +### Opening record + +Stage A complete. Run dir `.llm/runs/release-0.0.6-internals--orchestration/` holds +`supervisor.md`, `plan.md`, `worklog.md`, `cut-trace.md`, `drift.md`, `context-pack.md`. +`phase-registry.md` is **not** used: this lane runs one sequential implementation thread, not two or +more concurrent capability-scoped phase groups. + +--- + +## Stage B — Wave plan and dispatch preconditions (2026-08-12) + +Wave plan committed as `plan.md`: four leaf PRs (PR-A gate trust; PR-B/C/D quality rail), strictly +sequential, six locked sequencing decisions (S-1…S-6), no canary declared by this lane. + +### PLAN-EVAL decision for the wave plan + +`milestone-run.md` marks "PLAN-EVAL of the wave plan" **[asserted]** and instructs applying the +standard harness rule until a trace shows otherwise, recording the choice either way. + +**Decision: the wave plan itself is not separately PLAN-EVAL'd; the quality-rail plan it contains is.** +Reason: the wave plan's only real decisions are the sequencing locks S-1…S-6, and every one of them is +derived from a constraint written in the issues themselves (#1378 defers root scope to #1380; #1403 is +p0 and smallest; PR-A gates the merges of the rest). The substantive planning risk in this lane is +*inside* the rail — export-reachability strategy, allowance-registry design, rename-vs-deletion +policy — and that is what the single rail PLAN-EVAL evaluates, in a fresh Codex · Sol · high session +(opposite family to this Claude-authored plan). Recorded per `milestone-run.md` line 63. + +### Dispatch preconditions — executed checks + +`milestone-run.md` stage B requires provider quota and paid-transport verification, with the +**recorded check output** as the proof; a wave dispatched without that record is a did-not-run. + +**1. Transport health** — `deno task agentic:runtime doctor` at 2026-08-12T07:44Z: + +```text +Agentic runtime doctor: no_change (schema 1.0) +mode: inspect; changed: no +desired state: foundation-desired-1.0 +observed state: foundation:4eba78f77027ba3c87123e4a29dabde020a161f44d37524ca8424b1b88060d0d +components: 18; sessions: 0 +``` + +`deno task agentic:runtime status --json` → all 18 components `ready`; `auth`: `claude` +(provider-native) `ready`, `antigravity` (google-sign-in) `ready`; `capabilities`: claude +`available`, codex `available`, antigravity `available`. `worktrees: []`, `sessions: []`, +`checkpoints: []`. + +**2. Codex app-server daemon** — `deno task agentic:codex-status` at 2026-08-12T07:44Z: + +```text +daemon.status running +daemon.managedCodexPath /home/codex/.codex/packages/standalone/current/codex +daemon.managedCodexVersion 0.147.0 (cli 0.147.0, appServer 0.147.0) +daemon.socketPath /home/codex/.codex/app-server-control/app-server-control.sock +appServerProcesses 3 (anchoredAppServerProcesses 2) +``` + +The daemon is **managed** (not the orphaned-control-socket state where `managedCodexPath` is absent), +so `agentic:runtime repair codex-remote` is not indicated. + +**3. Provider quota / live-turn capability** — five live `gpt-5.6-sol` threads were observed +transacting at read time (four `working`, one `idle` at `turn complete`), with fresh +`lastActivityAt` timestamps inside the same minute and real tool activity +(`command finished`, `exec …`, file writes). A model that is quota-blocked does not produce +`turn complete` and file-write artifacts. This is a **live-call** verification, not a status-panel +read — the recovery-judgement rule from the skill ("treat the provider status panel as stale +afterwards — verify with a real call, not the display"). + +**4. No rival thread in this lane's surface.** All five Sol threads run with +`cwd=/home/codex/Documents/Codex/2026-08-05-i-have-these-4-sessions-that`; one wrote +`/tmp/ns006-features-orchestrator.md`, i.e. they are the **dispatcher's own** brief-authoring and +watcher sessions for the four 0.0.6 topical lanes, not implementation agents. **Zero** threads carry +a `cwd` under `/home/codex/repos/netscript-006-internals` or any `ns006-*` worktree, so this lane's +"one active thread/worktree" invariant starts satisfied. Ownership was established from the actual +`cwd` field, never by string-matching worktree paths quoted in brief text (skill § Supervision +pitfalls). + +One observation worth recording rather than acting on: the idle thread +`019ff4df-d9e0-7a81-843a-068374e24e5a` shows `lastReasoning: "**Refining issue keyword boundary +matching**"` — that is the dispatcher drafting this lane's own brief text about #1436, not a second +agent implementing #1436. Its `cwd` is the dispatcher's conversation dir and it has written no file. +No steering or stop is warranted. + +**Verdict: dispatch preconditions GREEN.** Wave 1 may dispatch. + +--- + +## Stage C/D — per-PR dispatch and landing + +*(appended per PR as it dispatches, lands, and passes the pre-merge gate)* From 96a8c2f9dce4b08c253cc48c8637072f62a4ff9b Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 10:05:26 +0200 Subject: [PATCH 02/57] chore(harness): commit the quality-rail plan and the run's first re-planning events Wave 2 gets one plan (plan-quality-rail.md) with nine locked decisions and a fully executed baseline, because #1403/#1380/#1378 overlap on root lists, scan semantics, doctrine and architecture debt. Executing the baseline moved three numbers the issues assert, and each change alters the work: - quality:scan:repo is RED on main and has been for 7 consecutive pushes, so #1378's own gate box is unsatisfiable as written. Filed #1530 and inserted it before #1378. - arch:check:repo FAIL is 55, not the 53 #1380 recorded four days ago. - Five of #1380's six stale verdict rows name directories that never existed in this repo, so rename-vs-deletion is a false dichotomy for them. Also records PR-A's dispatch identity and the orchestrator's independent re-verification of the #1436 fix, the owner's add-then-withdraw of #1529, and a brief error the implementation agent caught by escalating rather than idling. Refs #1436, #1415, #1403, #1380, #1378, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../cut-trace.md | 10 +- .../drift.md | 79 +++++ .../plan-quality-rail.md | 118 +++++++ .../slices/pr-a-1436-1415/codex-thread-ids.md | 16 + .../slices/pr-a-1436-1415/implement.md | 288 ++++++++++++++++++ .../worklog.md | 92 ++++++ 6 files changed, 601 insertions(+), 2 deletions(-) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 6095214c0a..3870954476 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -14,7 +14,8 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | Wave | PR | Issues | Lane | Status | | --- | --- | --- | --- | --- | -| 1 | PR-A `fix/1436-1415-close-gate-trust` | #1436, #1415 | Sol · low | not yet dispatched | +| 1 | PR-A `fix/1436-1415-close-gate-trust` → **PR #1527** | #1436, #1415 | Sol · low | **dispatched** — thread `019ff4f4-1fce-7253-a7e0-d718c65b39cc`, worktree `/home/codex/repos/ns006-gatetrust`; S1–S4 landed | +| 2 | PR-E `fix/1530-type-fixture-scan-scope` | #1530 | Sol · low | not yet dispatched (inserted; gates PR-D per rail R-1) | | 2 | PR-B `fix/1403-quality-gate-coverage` | #1403 | Sol · low | not yet dispatched | | 2 | PR-C `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | not yet dispatched | | 2 | PR-D `fix/1378-quality-scan-rule-power` | #1378 | Sol · high | not yet dispatched | @@ -23,7 +24,9 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | When | Event | Effect on the plan | Effect on the record | | --- | --- | --- | --- | --- | -| — | — | — | — | — | +| 1 | 2026-08-12, during PR-A impl | **Scope added then withdrawn by the owner.** #1529 (p0) was added to the lane as a separate leaf PR, then closed as not-planned with the observed CI skip declared intended. | Worktree `ns006-cigate`, branch `fix/1529-required-lane-visibility` and its brief were created, then removed. No workflow behaviour changed. Wave structure unaffected — nothing had been dispatched. | `drift.md` D-6; two incidental observations parked unacted in D-7. | +| 2 | 2026-08-12, stage B measurement | **Defect filed from inside the run**: `quality:scan:repo` red on `main` for 7 consecutive pushes because negative type fixtures are scanned as production source. Filed as #1530. | New PR-E inserted **before** PR-D: #1378's `gate:` box requires `quality:scan:repo` green and cannot be truthfully ticked until #1530 lands (rail R-1). | `drift.md` D-5. This is the scope-drift checkpoint, taken explicitly rather than discovered at cut time. | +| 3 | 2026-08-12, during PR-A impl | **Orchestrator brief error corrected mid-slice.** Gate 1 omitted `--allow-write`; the agent escalated instead of idling. | Gate 1 amended for PR-A and the permission requirement carried into the rail plan's validation table. | `drift.md` D-8. | ## Failure modes that cost real time @@ -41,3 +44,6 @@ needed. | --- | --- | --- | --- | --- | | A-1 | An issue's prescribed fix can be implemented as written | #1436 "Fix" section | **falsified** | `\b` already present at `acceptance-evidence.ts:43`; the prescribed patch is a no-op. `drift.md` D-4. | | A-2 | Re-baselining an issue body against HEAD is ceremony for mechanical issues | implicit in "mechanical ⇒ no plan needed" | **falsified** | The re-baseline is what caught A-1, on the smallest issue in the lane (p2, 2-line fix). | +| A-3 | An issue's measured counts stay valid for the few days between filing and implementation | implicit in working from an issue body | **falsified, twice** | `arch:check:repo` FAIL moved 53 → **55** in four days, and `quality:scan:repo` moved from green to **RED exit 1** — the latter making one of #1378's own acceptance boxes unsatisfiable as written. Both found by executing, neither visible in the issue. | +| A-4 | A doctrine verdict row that names a missing directory was renamed or deleted | #1380 D6 ("plausibly renamed into the `plugin-*-core` tier") | **falsified for 5 of 6 rows** | `git log --all --diff-filter=A` shows `packages/{streams,triggers,workers,sagas}` and `plugins/hello-world` **never existed** in this repo; only `@netscript/shared` ever did (`0ef13de35 chore: genesis eject`). A third state — *authored against a layout that never landed* — is required, and a rename note would have fabricated provenance. | +| A-5 | The escalate-don't-idle brief instruction changes behaviour | `agent-milestone-orchestrator` § Delegation (4 idle-at-red-gate occurrences in 0.0.4) | **confirmed** | PR-A's agent hit a red Gate 1, diagnosed it as a pre-existing permission gap, refused to weaken unrelated tests, recorded it, escalated, and continued unblocked work. `drift.md` D-8. | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 4c00a66646..d3eeb71dcc 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -62,3 +62,82 @@ Append-only. Newest entries at the bottom. mandatory cases and the orchestrator re-runs the probe against the patched parser before merge. #1436's acceptance is unaffected (it has no checkboxes); the issue's *analysis* section is corrected by a comment on the issue, so the record is not silently better than the issue. + +## D-5 — `quality:scan:repo` is RED on `main`, and #1378's gate box is unsatisfiable until it is fixed + +- **Severity:** significant +- **Recorded:** 2026-08-12, stage B, executed at `01aa12b67` +- **Fact:** #1378 § Current surface records "The scan is green today at 0 findings / 7 default / 10 + repo-wide allowances" (measured 2026-08-08 at `fac9e339042c`). Executed now: + `deno task quality:scan:repo` exits **1** with **5** `ts-error-suppression` findings. +- **Cause:** `scan-code-quality.ts:86-89` exempts `_test`/`.test`/`.spec` files but not `*_type.ts`, so + negative type fixtures are scanned as production source and their `@ts-expect-error` assertions — + the whole point of the fixture — are reported as violations. `b3dc006e8` ("accept typed SDK client + contributions (RFC 0001)") added `packages/sdk/tests/type-fixtures/sdk-client-contributions-rfc_type.ts` + and turned the gate red. +- **Blast radius:** the **blocking** `code-quality-repo` job has failed on **7 consecutive pushes to + `main`** (first `b3dc006e8` 2026-08-11T20:15Z; last green `0fbe3dadd` 2026-08-11T11:48Z). #1378's + `gate:` acceptance box requires `quality:scan:repo` green after its change, so that box cannot be + truthfully ticked until this is cleared. +- **Also observed:** the PR-side `code-quality` job scans only changed files and skips drafts + (`code-quality.yml:28,36-42`), so the repo-wide job is the only path that ever scanned the file — and + it runs post-merge. The violation was structurally invisible until it was already on `main`. +- **Action:** filed as **#1530** (p1, milestone 0.0.6) with the executed evidence and a narrow fix + keyed on `tests/type-fixtures/**/*_type.ts`. Sequenced as PR-E before PR-D (`plan-quality-rail.md` + R-1). Two now-redundant `// quality-allow:` lines are removed by it, lowering the repo-wide count + 10 → 8, which matters because #1378 wires `--max-allow` as a budget that can only fall. + +## D-6 — #1529 added to the lane by the owner, then withdrawn by the owner + +- **Severity:** significant (scope, owner-directed both ways) +- **Recorded:** 2026-08-12, stage C +- **Sequence:** the owner added #1529 (`fix(ci): ready_for_review can report every required lane as + skipped`, p0 release blocker) to this lane as a separate leaf PR, requiring proof of both `ci` and + `e2e` visibility semantics. The orchestrator created worktree `/home/codex/repos/ns006-cigate`, + branch `fix/1529-required-lane-visibility`, and a brief. The owner then closed #1529 as **not + planned**, stating the observed core-CI skip is intended, and withdrew it from the lane. +- **Independent finding, which agrees with the owner's call:** before the withdrawal the orchestrator + reconstructed the event sequence from the Actions API and found the issue's repro **misattributed**. + Run `31575007718` (all six jobs skipped) was created at `07:41:03Z`, while the `ready_for_review` + timeline event is at `07:41:14Z` — 11 seconds later. That run was a **draft `synchronize`**, and + draft pushes scheduling no jobs is documented intended behaviour (`ci.yml:59-60`). The real + `ready_for_review` run is `31575023887`, which was **`cancelled`** 82s in by the next push under + `concurrency.cancel-in-progress` (`ci.yml:48-50`). So the cited evidence did not demonstrate the + claimed mechanism. +- **Action:** worktree removed, branch deleted, brief deleted. No workflow or visibility behaviour was + changed. Two incidental observations made during that analysis are recorded in D-7 as **observations + only** — not acted on, not filed, per the owner's instruction to drop the item completely. + +## D-7 — observations recorded but deliberately not acted on + +Kept because `agent-milestone-orchestrator` § Honesty rules requires recording what a run learns +rather than discarding it, and because an unrecorded observation gets rediscovered at cost. Neither is +being fixed, filed, or briefed by this lane. Both are the owner's call. + +1. **Both `lane-visibility` jobs are reporters, not gates.** `ci.yml:347-395` + (`core CI lane visibility`) and `e2e-cli.yml:509-564` (`scaffold CI lane visibility`) each consist of + one step that writes a Markdown table to `$GITHUB_STEP_SUMMARY`. Neither contains `exit 1` + (`grep -n "exit 1" .github/workflows/ci.yml` returns only Aspire-version checks in other jobs). They + cannot fail. +2. **`pr-checks.ts` classifies a `skipped` check-run at head as a pass.** + `.llm/tools/agentic/github/pr-checks.ts:141-157` chains superseded → stale-post-merge → pending → + cancelled → failure → **else `current-pass`**, so `conclusion: 'skipped'` and `'neutral'` fall + through to `current-pass`, and `buildPrCheckReport` then reports `ok: true`. This is the tool + `netscript-pr`'s close-gate playbook directs operators to for "establish currency first". + +## D-8 — orchestrator brief error, corrected mid-slice (Gate 1 permissions) + +- **Severity:** minor +- **Recorded:** 2026-08-12, stage C +- **Fact:** PR-A's brief prescribed `deno test --allow-read --allow-env .llm/tools/validation/` as + Gate 1. The implementation agent ran it, got exit 1 with 39 passed / 9 failed, correctly diagnosed + all 9 as pre-existing `NotCapable: Requires write access` at `Deno.makeTempDir()` calls in + `check-aspire-host-ports_test.ts`, `check-netscript-jsr-specifiers_test.ts` and + `fresh-ui-quality_test.ts`, **refused to weaken unrelated tests**, recorded it, and continued with + the unblocked gates. +- **Verdict:** the brief was wrong, the agent was right. Gate 1 is amended to include + `--allow-write`, and that permission requirement is carried into the rail plan's validation table so + the next three briefs do not repeat the error. +- **Worth noting:** this is the escalate-don't-idle instruction working as intended. The recorded + failure mode it was written against — supervisors going idle at a red gate, four occurrences in + 0.0.4 — did not recur. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md new file mode 100644 index 0000000000..fdf1ab1e79 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -0,0 +1,118 @@ +# Quality Rail Plan — #1403 → #1380 → #1378 + +The single plan for wave 2. One PLAN-EVAL covers all three PRs because the issues overlap on root +lists, scan semantics, doctrine and architecture debt; evaluating their sequencing separately would +evaluate the wrong thing. Sequencing locks S-2…S-5 live in `plan.md` and are not restated. + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `release-0.0.6-internals--orchestration` | +| Phase | `plan` (rail) | +| Target | `.llm/tools/quality/scan-code-quality.ts`, `.llm/tools/fitness/check-doctrine.ts`, `deno.json` tasks, `docs/architecture/doctrine/`, `.llm/harness/debt/arch-debt.md`, `rfcs/README.md` | +| Archetype | N/A — repo tooling and governance documents; no `packages/**`/`plugins/**` source authored | +| Scope overlays | `SCOPE-docs.md` for PR-C's doctrine-document half | + +## Executed baseline — every number below is a command result at `01aa12b67`, 2026-08-12 + +| Measurement | Value | Command | +| --- | --- | --- | +| `deno task quality:scan` | exit **0**, `findings: []`, `allowCount: 7` | executed | +| `deno task quality:scan:repo` | exit **1**, **5** findings, `allowCount: 10` | executed | +| `deno task arch:check` | exit **0**, 16 hand-listed roots | executed | +| `deno task arch:check:repo` | exit **1**, **FAIL=55** (54 × A14 + 1 × A1) | executed | +| Live units | 30 `packages/*` + 6 `plugins/*` = **36** | executed | +| Verdict-table rows naming non-live units | **6** | executed | +| Live units with no verdict row | **14** | executed | +| `*-soundness_test.ts` files | **6** | executed | +| `*_type.ts` files | **12**, all under `tests/type-fixtures/`, **3** contain `@ts-expect-error` | executed | + +### Drift against the issue bodies (measured 2026-08-08, now stale) + +| Issue claim | Then | Now | Consequence | +| --- | --- | --- | --- | +| `arch:check:repo` FAIL count (#1380 D8) | 53 | **55** | The A14 false-positive population grew by 2 in four days. It grows with every new `@std/testing/bdd` test, which is the argument for fixing the predicate rather than enumerating residue. | +| `quality:scan:repo` is green (#1378 § Current surface) | green, 0 findings | **RED, exit 1, 5 findings** | #1378's `gate:` acceptance box ("`quality:scan:repo` … green after the change") **cannot be satisfied** without first clearing this. Filed as **#1530**; it is PR-E, and PR-D depends on it. | +| Repo-wide `allowCount` (#1378) | 10 | 10 (unchanged) | `--max-allow` must be wired at the value measured **at the time of wiring**, not at 10 on faith. #1530 lowers it to 8. | + +### The 6 stale verdict rows are not what #1380 assumes + +#1380 hypothesises that "four of the five packages were plausibly renamed into the `plugin-*-core` +tier", while correctly warning that "the re-walk must record rename-vs-deletion per row, not assume". +Executed against the full history (`git log --all --diff-filter=A -- packages/

/deno.json`): + +| Stale row | Ever existed in this repo? | Correct record | +| --- | --- | --- | +| `@netscript/streams` | **no** — no `deno.json` ever added at `packages/streams` | never existed under this name | +| `@netscript/triggers` | **no** | never existed under this name | +| `@netscript/workers` | **no** | never existed under this name | +| `@netscript/sagas` | **no** | never existed under this name | +| `@netscript/shared` | **yes** — added at `0ef13de35 chore: genesis eject`, 10 commits of history | genuinely removed; PR-C must find and cite the removal | +| `plugins/hello-world` | **no** | never existed under this name | + +So "renamed vs deleted" is a **false dichotomy for five of the six rows**: the third state is *the +table was authored against a layout that never landed in this repository*. That materially changes the +deliverable — a rename note pointing `@netscript/workers` → `packages/plugin-workers-core` would be a +fabricated provenance claim. PR-C records the executed per-row verdict, with the third state allowed. + +This also reframes the whole issue: the verdict table is not merely *stale*, it was never a +measurement of this tree. `docs/architecture/doctrine/10-…md:197-208` defines doctrine completion +against that walk, and `.agents/skills/netscript-doctrine` routes every agent through it. + +## Locked rail decisions + +| ID | Decision | Rationale | +| --- | --- | --- | +| `R-1` | **PR-E (#1530) lands before PR-D (#1378).** | #1378's `gate:` box requires `quality:scan:repo` green. It is red on `main` today for a reason that is not a defect (negative type fixtures scanned as production source). PR-D cannot truthfully tick that box until PR-E clears it. | +| `R-2` | The `*_type.ts` exemption is keyed on **directory + suffix** (`tests/type-fixtures/**/*_type.ts`), never a filename allowlist and never a widened test regex. | A widened `_test`-style regex would exempt production code; an allowlist rots on the next fixture. The exemption must be an explicit rule asserted by test, which is the shape #1378 § Target contract already mandates for the 6 soundness tests. | +| `R-3` | Export-awareness is driven by **`deno doc --json` over each package's `exports` map**, not a fourth line-regex. | #1378 offers both. `deno doc --json` already answers "is this symbol published"; `check-doctrine.ts:467-484`'s line-start matching provably misses `export const`, class members, generic defaults and re-exports. Adding a regex that misses the same things would ship a rule that looks export-aware and is not. | +| `R-4` | `arch:check:repo` iterates **live workspace members read from the root `deno.json` workspace list**, not a hand-listed set and not the repository root. | #1380 acceptance requires it. Reading the workspace list means a new package is gated the moment it joins the workspace — the property #1403's coverage test is reaching for, generalised. | +| `R-5` | The A14 predicate must resolve **where the identifier came from**, not merely that `describe(`/`it(` appears. | 54 of 55 `arch:check:repo` failures are `@std/testing/bdd` imports — the sanctioned API. A predicate that cannot tell an import from a global is not a Jest/Vitest detector. | +| `R-6` | `arch:check`'s 16-root task string is moved into **data** (a checked-in list the task and the tests both read) in **PR-B**, before PR-C generalises it. | The list is a ~2 kB inline shell string; a coverage test would have to parse shell to read it. Moving it to data first makes PR-B's coverage assertion honest and PR-C's live-member switch a small delta rather than a rewrite. | +| `R-7` | `--max-allow` is wired at the count **measured in the wiring PR**, and the PR body states that adding an allowance will now fail CI. | Wiring a budget is a behavioural change to every future PR, not a flag addition. Unstated, it reads as a break. | +| `R-8` | Findings surfaced by newly-covered scans are **triaged into issues, never fixed in the surfacing PR** (`plan.md` S-5). PR-E is not an exception: it fixes the **scanner's scope**, not the findings. | #1403 box 5 and #1378 § Boundaries both require it. The distinction matters for honest box-ticking, so PR-E's body states it explicitly. | +| `R-9` | The RFC-location divergence (#1380 D9/D10) is resolved **by recording the decision the repo already made**, not by adopting a new process. | `rfcs/` holds only a template and a README; five numbered RFCs (0001–0005) were accepted through `.llm/runs/*/design/canonical/` and merged in the last week. The de-facto path is the harness path. PR-C states which location is canonical and maps the 5 `DECISION_PENDING` entries onto it — it does not invent a promotion pipeline. | + +## Open decisions + +| Decision | Status | Notes | +| --- | --- | --- | +| Whether `quality:scan` should scan `docs/site/**` fences itself, or consume a shared extractor with #1374 | **must resolve before PR-D** | #1374 (`test(docs): docs:accuracy is a fixed-string needle checker`) is live in the **docs lane** at `/home/codex/repos/ns006-1374-compilegate` and needs the same fenced-TS extraction from `docs/site/**`. Two independent extractors with different fence-parsing rules would disagree about what a snippet is, and the disagreement would be invisible. Coordinate before PR-D writes one. | +| Whether the residue of `arch:check:repo` after R-4/R-5 is green or enumerated debt | safe to defer to PR-C | #1380 accepts either ("exits 0 or its residue is enumerated in `arch-debt.md`"). PR-C measures after the two mechanical fixes and reports the real number. | +| Whether `@netscript/shared`'s removal commit can be cited precisely | safe to defer to PR-C | It existed and has 10 commits of history; the removal is findable. If it cannot be found, the row records "removed, commit not identified" rather than a guess. | + +## Risk register (rail-specific; `plan.md` holds the lane-wide register) + +| Risk | Mitigation | +| --- | --- | +| PR-C invents rename provenance for the four never-existed rows. | The executed evidence and the third state are in the brief, with the exact command that produced them. | +| `deno doc --json` is too slow to run per package in CI. | Measure before committing to it in PR-D; the fallback is scanning only each package's `exports` entrypoints and their re-export graph, recorded as a decision, not discovered as a timeout. | +| R-6's root-list move collides with PR-C's live-member switch. | R-6 lands the *shape* (data, not shell) and PR-C changes the *source* (workspace list). Sequential, with S-4 forbidding deletion of PR-B's assertion. | +| Wiring `--max-allow` reds an unrelated in-flight PR from another 0.0.6 lane. | Three sibling lanes are active. PR-D announces the budget in its body and the orchestrator reports the merge immediately so the other lanes learn the count. | +| A14 fix silently exempts genuine Jest/Vitest usage. | The negative case is mandatory: a fixture with a real bare `describe(` global must still FAIL after the fix. | + +## Validation plan + +| Order | Gate | Command | Expected | +| --- | --- | --- | --- | +| 1 | quality scan (default) | `deno task quality:scan` | exit 0 | +| 2 | quality scan (repo) | `deno task quality:scan:repo` | exit 0 **after PR-E**; exit 1 with 5 known findings before it | +| 3 | doctrine (curated) | `deno task arch:check` | exit 0 throughout | +| 4 | doctrine (repo) | `deno task arch:check:repo` | exit 1 / FAIL=55 before PR-C; after PR-C exit 0 or enumerated residue | +| 5 | rail tests | `deno test --allow-read --allow-write --allow-env .llm/tools/quality/ .llm/tools/fitness/` | pass | +| 6 | scoped wrappers | `run-deno-{check,lint,fmt}.ts --root .llm/tools --ext ts` | pass | +| 7 | doctrine-document tests | the new existence + coverage tests from PR-C | fail on a fabricated row and on an ungated live unit | + +`deno task e2e:cli` is out of scope for the whole rail — no PR touches scaffold, DB, Aspire, or plugin +copy mode. `--allow-write` is required for the tool test suites (established by PR-A's escalation: 9 +pre-existing tests call `Deno.makeTempDir()`). + +## Per-PR contract summary + +| PR | Closes | Lane | Must prove (negative case) | +| --- | --- | --- | --- | +| PR-E | #1530 | Sol · low | A `@ts-expect-error` in a `*_type.ts` under `tests/type-fixtures/` is reported **before** and not after; the same directive in ordinary source and in a `*_type.ts` **outside** `tests/type-fixtures/` is still reported; repo-wide `allowCount` falls 10 → 8 | +| PR-B | #1403 | Sol · low | The coverage test **fails** when a publishable `plugin-*-core` package is removed from the root list; the repaired gate reports `packages/plugin-streams-core`'s real state and its findings are triaged, not fixed | +| PR-C | #1380 | Sol · medium | Existence test fails on a verdict row naming a non-existent directory; coverage test fails on a live unit with no row; A14 does **not** fire on `@std/testing/bdd` **and still fires** on a real bare global | +| PR-D | #1378 | Sol · high | Exported `any` fails while a local `any` keeps its current severity; an unlinked `as unknown as` fails; `as any` in a `docs/site/**` fence fails; the 6 soundness tests stay green unchanged; budget overflow fails | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/codex-thread-ids.md new file mode 100644 index 0000000000..dc5118973a --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-pra — Codex implementation thread +- **Thread / session id:** `019ff4f4-1fce-7253-a7e0-d718c65b39cc` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T09-51-07-019ff4f4-1fce-7253-a7e0-d718c65b39cc.jsonl` +- **Worktree:** `/home/codex/repos/ns006-gatetrust` +- **Branch:** `fix/1436-1415-close-gate-trust` @ `c2d8a8e4b` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1436-1415-close-gate-trust`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-pra-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff4f4-1fce-7253-a7e0-d718c65b39cc -- "" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/implement.md new file mode 100644 index 0000000000..614d0ba962 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/implement.md @@ -0,0 +1,288 @@ +use harness + +# PR-A — gate trust: close-gate keyword parser (#1436) + acceptance-evidence assertion (#1415) + +You are the **implementation agent** for the first slice of the 0.0.6 chores/internals lane. Two +single-predicate repairs in `.llm/tools/validation/`, one PR, closing two issues. + +Your orchestrator is a Claude Opus 5 high session in +`/home/codex/repos/netscript-006-internals` on branch +`chore/release-0.0.6-internals-orchestration`. It holds merge authority. You implement, gate, and +report; you do not merge. + +## SKILL + +- `netscript-harness` — run artifacts, commit trail, slice discipline. +- `netscript-tools` — the scoped validation wrappers; what counts as a verdict vs a non-verdict. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` + block, the close-gate operator playbook. +- `netscript-deno-toolchain` — deterministic `deno test` behaviour and task semantics. +- `rtk` — prefix read-heavy `git`/`gh`/`grep` with `rtk`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-gatetrust` | +| Branch | `fix/1436-1415-close-gate-trust` | +| Base | `01aa12b67` (= `origin/main` at 2026-08-12) | +| Run dir | `.llm/runs/release-0.0.6-internals--orchestration/` | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/` | +| Draft PR | **#1527** — already open, already labelled, body already written. Comment on it; do not open another. | +| Closes | #1436, #1415 | + +Work **only** in that worktree. Do not touch any other worktree, and do not rebase or force-push. + +## Why this slice exists + +Both issues are the same defect in two places: a gate that reports faithfully while checking the +wrong property. #1436 makes the close-gate **invent** a requirement it can never satisfy; #1415 lets +the acceptance mirror **absolve** one. Every later PR in this lane merges through both, so this PR +lands first. + +--- + +## Part 1 — #1436: the closing-keyword parser + +### Read this before you write any code + +**The fix prescribed in the issue body is a no-op. Do not implement it as written.** + +#1436 says the parser matches closing keywords "without a word boundary" and prescribes +`\b(clos(e|es|ed)|fix(es|ed)?|resolv(e|es|ed))\b\s+#\d+`. The orchestrator executed the parser at +`01aa12b67` and the word boundary is **already there** — +`.llm/tools/validation/acceptance-evidence.ts:43` already reads +`/\b(?:close|closes|closed|fix|fixes|fixed|resolve|resolves|resolved)\s+…/gi`, and the fenced-block +stripping the issue does not mention is already there too (`:47`, landed by #1303). + +`\b` is the **cause**, not the cure: `-` is a non-word character, so `\bfix\b` matches the `fix` +inside `pre-fix`. Executed baseline (`../../evidence/probe-1436-baseline.ts`, run from the repo root): + +```text +"Exact pre-fix #1431 head" -> [1431] # the reported defect +"un-fixed #555" -> [555] # a second instance, NOT mentioned in the issue +"hotfix #999 landed" -> [] # the issue predicts this breaks; it does not +"prefixes #888 there" -> [] # ditto +"This is a bugfix #777" -> [] # ditto +"Closes #1234 and fixes #4321" -> [1234,4321] +"Refs #111" / "Part of #222" -> [] +"resolves https://github.com/rickylabs/netscript/issues/333" -> [333] +``` + +Adding another `\b` changes nothing while looking like a fix. This repo has shipped exactly that +before — `.llm/harness/workflow/milestone-run.md` § Gate integrity records two guards whose predicate +could never fire, which "did nothing and looked correct". **A patch that leaves +`pre-fix #1431 -> [1431]` unchanged is a failed slice, not a green one.** + +### Contract 1a — keyword boundary + +`extractClosingIssues` must not treat a keyword as closing when it is preceded by a word character +**or a hyphen**. The predicate shape is a negative lookbehind excluding both — `(? --dry-run --pretty` | + +Gate 5 must show `pre-fix`, `un-fixed`, `hotfix`, `prefixes`, `bugfix` all `-> []` while +`Closes #1234 and fixes #4321 -> [1234,4321]` and the issue-URL case still resolve. Paste the whole +output; the orchestrator re-runs it independently before merge. + +Gate 6 is the live-path check: the mirror must run cleanly on your own PR (which has no +not-yet-done evidence) and must not attempt to fetch a pull-request number as an issue. + +`deno task e2e:cli` is **out of scope** — nothing here touches scaffold, DB, Aspire, or plugin copy +mode. Apply `ci:skip-e2e` and `ci:skip-scaffold` to the PR and **state that choice and its reason in +the opening phase comment** so the cheap lane is visibly intentional rather than an accident. Do not +edit any workflow file to dodge a filter. + +## PR mechanics + +1. **First commit is the slice-dir bootstrap**, and the draft PR opens in the same session as that + commit (`netscript-pr` § Draft-PR-on-start). The draft PR's commit list plus your per-slice + comments are the commit trail; there is no `commits.md`. +2. Slice artifacts go in `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/`: + `worklog.md`, `context-pack.md`, `drift.md`. Keep them current **in the same commit** as the code + slice they describe — a slice whose commit does not touch the run dir is incomplete. +3. PR body follows the `netscript-pr` template. `## Scope` carries, each on its own line: + `Closes #1436` and `Closes #1415`. Both are fully resolved by this PR, so both get a real closing + keyword — bare `#N` and `Refs #N` do not auto-close. +4. Labels: `type:fix`, `area:tooling`, `priority:p1`, `ci:skip-e2e`, `ci:skip-scaffold`, and exactly + one `status:` — `status:impl` while you work. Milestone `0.0.6`. Do **not** apply + `status:ready-merge` yourself; the orchestrator sets it after its pre-merge gate. +5. Suggested commit slices (adjust if you find a better cut, and say why in `drift.md`): + - S1 — slice dir + failing tests for both defects (RED, committed as RED so the record shows it) + - S2 — keyword boundary fix (`(?"` | + +Draft PR **#1527** was opened by the orchestrator before dispatch (labels `type:fix`, `area:tooling`, +`priority:p1`, `ci:skip-e2e`, `ci:skip-scaffold`, `status:impl`; milestone `0.0.6`), so the agent had a +live reviewable surface from its first commit. + +**Tooling observation, not a failure:** the foreground launcher stream was killed by the orchestrator's +own 580s command timeout (exit 143 / SIGTERM). The thread itself was already created and unaffected — +confirmed by `agentic:codex-status` showing it `working` at that `cwd`, and by four subsequent commits. +A launcher whose log stream outlives the caller's timeout should be backgrounded from the start. + +**Slices landed** (`git log --oneline origin/main..HEAD` in the worktree): + +```text +c095303c8 fix(validation): reject not-yet-done acceptance evidence +0329acaf8 fix(validation): exclude pull requests from closing issues +4ca4cc421 fix(validation): reject hyphen-prefixed closing keywords +a927790eb test(validation): prove gate-trust contracts red +c2d8a8e4b chore(harness): bootstrap the PR-A gate-trust slice with its executed baseline +``` + +The RED-first commit (`a927790eb`) is committed as RED, so the record shows the tests failing before +the fixes rather than asserting that they would have. + +**Orchestrator slice review — independent re-verification, not a claim relay.** The Tier-A review rule +says a green automated gate is not a sign-off, and #1436 has zero acceptance boxes, so this is the only +verification of its central claim. Re-ran the baseline probe **myself** against the patched parser at +`c095303c8`: + +```text +"Exact pre-fix #1431 head" -> [] # was [1431] at 01aa12b67 +"un-fixed #555" -> [] # was [555] +"hotfix #999 landed" -> [] # unchanged +"prefixes #888 there" -> [] # unchanged +"This is a bugfix #777" -> [] # unchanged +"Fixes #1434" -> [1434] # real form preserved +"Closes #1234 and fixes #4321" -> [1234,4321] +"resolves https://github.com/rickylabs/netscript/issues/333" -> [333] +"Refs #111" / "Part of #222" -> [] +``` + +Predicate at `acceptance-evidence.ts:45` is `(? Date: Wed, 12 Aug 2026 10:12:17 +0200 Subject: [PATCH 03/57] chore(harness): record the attached-slice timeout hazard and correct the worklog Wrapping an attached Codex launch or resume in a shell timeout kills the slice at expiry. The launch wrapper did fire SIGTERM here; the thread survived, and the worklog wrongly recorded that as harmless. Surviving once is not evidence the practice is safe. Attached launch and resume now run unwrapped; bounded observation uses agentic:codex-watch --timeout-seconds, which expires without touching the slice. Also records that codex exec resume fails with a thread-store conflict while a thread is mid-turn, which is the mechanical signal for "not at a turn boundary". Refs #1436, #1415 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 20 +++++++++++++++++++ .../worklog.md | 10 ++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index d3eeb71dcc..3979d11cf9 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -141,3 +141,23 @@ being fixed, filed, or briefed by this lane. Both are the owner's call. - **Worth noting:** this is the escalate-don't-idle instruction working as intended. The recorded failure mode it was written against — supervisors going idle at a red gate, four occurrences in 0.0.4 — did not recur. + +## D-9 — never wrap an attached Codex launch or resume in a shell `timeout` + +- **Severity:** significant (tooling practice; owner-corrected) +- **Recorded:** 2026-08-12, stage C +- **What the orchestrator did wrong:** wrapped `agentic:launch-codex-slice` in `timeout 580` and + `codex exec resume` in `timeout 900`, to keep the foreground stream from blocking the supervisor turn. +- **Why it is wrong:** the wrapper's SIGTERM at expiry **kills the attached slice**, not merely the log + stream. The launch wrapper did fire (exit 143 / SIGTERM); this thread survived it, and the worklog + initially recorded that as "an observation, not a failure". That conclusion was wrong — surviving once + is not evidence the practice is safe, and treating a SIGTERM on an attached agent as harmless is + exactly the "verify the artefact, never the exit code" mistake in reverse. +- **Correct practice, owner-stated:** attached launch and resume run **unwrapped**. For bounded + observation use `deno task agentic:codex-watch --mode turn --thread-id --timeout-seconds N`, + which is designed to expire without touching the slice. Preserve the same thread and worktree + throughout; never launch a rival. +- **Also learned (and independently useful):** `codex exec resume` fails fast with + `thread-store conflict: … already has an active writer` while the thread is mid-turn. That is the + mechanical signal for "you are not at a turn boundary" — steer on `codex-watch --mode turn` + completion, not on git activity. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index 9b6ecd77bf..3e9b8baf90 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -207,10 +207,12 @@ Draft PR **#1527** was opened by the orchestrator before dispatch (labels `type: `priority:p1`, `ci:skip-e2e`, `ci:skip-scaffold`, `status:impl`; milestone `0.0.6`), so the agent had a live reviewable surface from its first commit. -**Tooling observation, not a failure:** the foreground launcher stream was killed by the orchestrator's -own 580s command timeout (exit 143 / SIGTERM). The thread itself was already created and unaffected — -confirmed by `agentic:codex-status` showing it `working` at that `cwd`, and by four subsequent commits. -A launcher whose log stream outlives the caller's timeout should be backgrounded from the start. +**Tooling hazard — see `drift.md` D-9.** The launch was wrapped in a shell `timeout 580`, which fired +and sent SIGTERM (exit 143). This slice's thread happened to survive (confirmed by +`agentic:codex-status` showing it `working` at that `cwd`, and by five subsequent commits), but the +practice is **prohibited**: wrapping an attached launch or resume in a shell `timeout` kills the +attached slice when the timeout expires. Attached launch/resume run unwrapped; bounded observation uses +`agentic:codex-watch --timeout-seconds` instead. **Slices landed** (`git log --oneline origin/main..HEAD` in the worktree): From 3289ad2b435b0725675204ca7b60d693415536c1 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 10:21:51 +0200 Subject: [PATCH 04/57] chore(harness): record that the ready-merge label is not a workflow trigger Both netscript-pr and check-close-gate's own repair hint say the labeled event triggers a fresh run. Neither ci.yml nor e2e-cli.yml lists labeled in its pull_request types, so the label creates no run and the hint tells the operator to do something that cannot work. Cost one verification cycle on PR #1527. Label first, then push is the rule that works, because the gate and the mirror read live state at execution time. Refs #1415, #1436 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../codex-thread-ids.md | 16 ++ .../drift.md | 26 ++++ .../plan-eval-brief.md | 142 ++++++++++++++++++ 3 files changed, 184 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-eval-brief.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/codex-thread-ids.md new file mode 100644 index 0000000000..08c0e1102b --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-raileval — Codex implementation thread +- **Thread / session id:** `019ff508-d5a3-7100-b6a0-1b6226e97e70` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T10-13-44-019ff508-d5a3-7100-b6a0-1b6226e97e70.jsonl` +- **Worktree:** `/home/codex/repos/ns006-raileval` +- **Branch:** `eval/quality-rail-plan-eval` @ `83de0dc06` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/eval/quality-rail-plan-eval`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=high +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=high +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-raileval-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff508-d5a3-7100-b6a0-1b6226e97e70 -- "" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 3979d11cf9..bb2efaea56 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -161,3 +161,29 @@ being fixed, filed, or briefed by this lane. Both are the owner's call. `thread-store conflict: … already has an active writer` while the thread is mid-turn. That is the mechanical signal for "you are not at a turn boundary" — steer on `codex-watch --mode turn` completion, not on git activity. + +## D-10 — `status:ready-merge` alone cannot trigger the acceptance mirror; the documented behaviour is wrong + +- **Severity:** significant (gate-trust defect in the close-gate surface this lane is repairing) +- **Recorded:** 2026-08-12, stage D, observed live on PR #1527 +- **Two sources assert the label is a trigger:** + - `.agents/skills/netscript-pr`: "applying `status:ready-merge` itself triggers a fresh run (the + workflow listens to `labeled`)". + - `check-close-gate.ts`'s own repair hint: "apply the label and the labeled event triggers a fresh + run". +- **Both are false at `01aa12b67`:** `ci.yml:41` and `e2e-cli.yml` both declare + `types: [opened, synchronize, reopened, ready_for_review]`. **`labeled` is in neither list.** +- **Observed cost:** the label was applied at ~08:14Z; no run was created; `close-gate` stayed + `current-fail` on its pre-label result and #1415's four acceptance boxes were still `0` ticked at + 08:19Z. One wasted verification cycle, and a repair hint that tells the operator to do something that + cannot work. +- **The rule that works:** **label first, then push.** The push fires `synchronize`, and because the + gate and the mirror read everything live at execution time, that run observes the label. +- **Not fixed by this lane.** Adding `labeled` to `ci.yml`'s types is a one-line change that would make + both documents true, but PR-A's boundary is `.llm/tools/validation/**` and the owner has just narrowed + this lane's scope; widening a slice into workflow surgery mid-flight is how scope leaks. Raised to the + owner as a separate decision, with the alternative being to correct the two documents instead of the + workflow. +- **Note on the pre-merge gate:** this is check 1's failure mode from the other side — the close-gate + *result existed* and was red, but it was red for a reason the operator was told to fix in a way that + does not work. "Unproven, not clean" applies to a stale result just as much as to a missing one. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-brief.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-brief.md new file mode 100644 index 0000000000..7411963d8f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-brief.md @@ -0,0 +1,142 @@ +use harness + +# PLAN-EVAL — the 0.0.6 internals quality rail (#1403 → #1380 → #1378, with #1530 inserted) + +You are a **formal PLAN-EVAL evaluator** in a fresh session. You did not write this plan and you must +not implement any of it. Your output is a verdict plus findings. + +The plan under evaluation is **Claude-authored**, so you are the opposite family per +`.llm/harness/workflow/lane-policy.md` (`formal_plan_evaluation`: Codex · GPT-5.6 Sol · high for Claude +plans). The generator session is a Claude Opus 5 high orchestrator; it is not reviewing itself, and it +does not get to overrule you on evidence. + +## SKILL + +- `netscript-harness` — evaluator separation, run artifacts, verdict vocabulary. +- `netscript-doctrine` — architecture/gate fitness; #1380 rewrites the doctrine's own verdict table. +- `netscript-tools` — which commands are verdict sources and which are non-verdicts. +- `netscript-deno-toolchain` — `deno doc --json`, task semantics, deterministic tests. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-raileval` (detached at `9c3cdfead`) | +| Role | PLAN-EVAL only — read, execute, verify, judge. **Write nothing outside your verdict file.** | +| Verdict file | `.llm/runs/release-0.0.6-internals--orchestration/plan-eval.md` in that worktree | +| Protocol | `.llm/harness/evaluator/plan-protocol.md` + `.llm/harness/gates/plan-gate.md` | +| Verdict vocabulary | `PASS` or `FAIL_PLAN` (`.llm/harness/evaluator/verdict-definitions.md`) | + +Do not commit, do not push, do not open or comment on a PR, do not touch any other worktree, and do +not launch other agents. + +## What to read + +1. `.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md` — **the plan under + evaluation**. +2. `plan.md` (the lane's wave plan — sequencing locks S-1…S-6 live there and the rail plan depends on + them), `supervisor.md`, `drift.md`, `cut-trace.md` in the same dir, for context and recorded + decisions. +3. The four **live** issue bodies: `#1403`, `#1380`, `#1378`, `#1530`. Read them from GitHub, not from + the plan's paraphrase. The plan asserts that issue-body measurements have already gone stale; part + of your job is checking whether the plan's own numbers are right **now**. +4. The code the plan proposes to change: `.llm/tools/quality/scan-code-quality.ts`, + `.llm/tools/fitness/check-doctrine.ts`, `deno.json` (tasks `quality:*`, `arch:check`, + `arch:check:repo`), `docs/architecture/doctrine/10-codebase-verdict-and-handoff.md`, + `docs/architecture/doctrine/06-archetypes.md`, `.llm/harness/debt/arch-debt.md`, `rfcs/README.md`, + `.github/workflows/code-quality.yml`. + +## What this plan claims, and what you should attack + +The plan's whole value proposition is that it re-measured everything instead of trusting the issues. +**Verify that by re-measuring independently.** If any number below is wrong, that is a finding, and a +wrong baseline is a `FAIL_PLAN` because three PRs are sequenced off it. + +| Claim | Verify with | +| --- | --- | +| `quality:scan` exit 0, `allowCount: 7` | `deno task quality:scan` | +| `quality:scan:repo` exit **1**, 5 findings, `allowCount: 10` | `deno task quality:scan:repo` | +| `arch:check` exit 0 | `deno task arch:check` | +| `arch:check:repo` exit 1, **FAIL=55** (not the 53 #1380 records) | `deno task arch:check:repo` | +| 36 live units (30 `packages/*` + 6 `plugins/*`) | `ls -d packages/*/ plugins/*/` | +| 6 verdict rows name non-live units; 14 live units have no row | read the table at `10-…md:22-51` against the live dirs | +| 6 `*-soundness_test.ts`; 12 `*_type.ts`, all under `tests/type-fixtures/`, 3 with `@ts-expect-error` | `find` + `grep` | +| `packages/{streams,triggers,workers,sagas}` and `plugins/hello-world` **never existed**; only `@netscript/shared` ever did | `git log --all --oneline --diff-filter=A -- 'packages/

/deno.json'` | + +The last row is the plan's most consequential claim: it asserts #1380's "plausibly renamed into the +`plugin-*-core` tier" hypothesis is **false for five of six rows**, and that recording a rename would +fabricate provenance. If you can show any of those directories did exist, or that a rename is +documented somewhere the plan did not look, say so — the plan then schedules the wrong deliverable. + +## Judge these decisions specifically + +Nine decisions are locked as R-1…R-9. Attack the ones where a wrong choice is expensive: + +- **R-1** (PR-E/#1530 before PR-D/#1378). Is #1378's `gate:` box genuinely unsatisfiable until + `quality:scan:repo` is green? Or could PR-D satisfy it by fixing the type-fixture scope itself, + making PR-E redundant work and one extra merge? +- **R-3** (export-awareness via `deno doc --json`, not a fourth regex). Is this actually feasible and + fast enough to run in the PR gate, which today scans only changed files? **Measure it** — run + `deno doc --json` against a couple of package `exports` maps and time it. A plan that mandates an + approach nobody timed is a plan with a hidden timeout in it. If it is too slow, the plan's own + fallback (entrypoints + re-export graph) needs to be the primary, and that is a finding. +- **R-4 + R-6** (move `arch:check`'s 16-root shell string into data in PR-B, then switch the source to + the workspace member list in PR-C). Is the two-step justified, or is it churn that touches the same + task twice? Note S-4 forbids deleting PR-B's coverage assertion in PR-C — check that the two-step + does not make S-4 impossible to honour. +- **R-5** (A14 must resolve identifier origin). Is "where did `describe` come from" implementable + without a type checker, given `check-doctrine.ts` is a line scanner? If it needs real import + resolution, does the plan acknowledge that cost anywhere? +- **R-9** (record the RFC location the repo already uses rather than adopt a promotion pipeline). Check + the live state: how many numbered RFCs exist under `rfcs/` now, and where were RFCs 0001–0005 + actually accepted? The plan asserts the de-facto path is the harness path. Recent `main` commits + mention "RFC 0001".."RFC 0005" being accepted — reconcile that with `ls rfcs/`. If numbered RFCs now + exist in `rfcs/`, R-9's premise is stale and the finding matters, because #1380 has an acceptance box + on it. + +## Also judge + +- **Coverage of acceptance.** #1403 has 5 boxes, #1380 has 12, #1378 has 9, #1530 has 7. Does the plan + route **every** box to a specific PR with a stated proof? Name any box that no PR owns. An + unrouted box is how a milestone reaches merge with an unticked gate. +- **Negative cases.** Each PR's "must prove" column claims a negative case. Is each one actually a + negative case (a thing that fails before and after the change in the right direction), or is any of + them a tautology that cannot fail? This repo has shipped guards whose predicate could never fire — + see `.llm/harness/workflow/milestone-run.md` § Gate integrity — and the plan's stated purpose is to + stop exactly that, so a non-firing proof here would be self-refuting. +- **The unresolved open decision.** The plan flags a collision with **#1374** (docs lane, live at + `/home/codex/repos/ns006-1374-compilegate`): both #1374 and #1378 need fenced-TS extraction from + `docs/site/**`. The plan marks it "must resolve before PR-D" but does not resolve it. Is deferring it + acceptable, or does it need resolving before PR-B lands? Read what #1374's worktree is actually + building before answering, but **do not modify anything there**. +- **Missing scope.** Anything the four issues require that no PR covers, and anything the plan pulls in + that the issues' Boundaries sections forbid (#1378 and #1380 both carry explicit do-not lists). + +## Output contract + +Write `.llm/runs/release-0.0.6-internals--orchestration/plan-eval.md` with: + +1. **Verdict** — `PASS` or `FAIL_PLAN`, on its own line, at the top. +2. **Identity** — your provider/model/effort, the worktree, and the baseline sha you evaluated at. +3. **Re-measured baseline** — your own numbers next to the plan's, with the commands you ran. Mark each + `confirmed` or `diverges`. +4. **Findings** — numbered, each with severity (`blocking` / `should-fix` / `advisory`), the exact file + and line or command output it rests on, and the concrete change you want. No finding without + evidence. +5. **Decision review** — a verdict per R-1…R-9: `sound`, `sound with caveat`, or `wrong`, with reasons. +6. **Acceptance-box routing table** — all 33 boxes across the four issues, each mapped to a PR or + flagged unrouted. +7. **What you executed** — every command you ran with its exit code, so the orchestrator can tell your + verdict from a reading of the plan. + +Rules for your verdict: + +- `FAIL_PLAN` if a baseline number is wrong, an acceptance box is unrouted, a locked decision is + unimplementable as written, or a claimed negative case cannot fail. +- `PASS` with `should-fix`/`advisory` findings is a legitimate and expected outcome. Do not manufacture + a blocking finding to look thorough, and do not withhold one to be agreeable. +- **Do not praise the plan.** Adjectives are not findings. Give checkable statements and executed + evidence only; a verdict whose support is "this is comprehensive and well-structured" is worthless to + the orchestrator and will be discarded. +- If you cannot verify something, say `unverified` and why — never infer a pass from silence. From ff100fd9bd5beeec70f6475bf8a2b2c7c5ce29ec Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 10:34:57 +0200 Subject: [PATCH 05/57] chore(harness): record the first merge of the internals lane PR #1527 merged as 63cd1cd58, closing #1436 and #1415 with all seven pre-merge gate checks passing and both issues auto-closed COMPLETED. Records three failure modes that cost real time (the ready-merge label not being a workflow trigger, the attached-slice timeout wrapper, and steering mid-turn), and one falsified assumption worth keeping: a cancelled expensive gate is not evidence the gate did not run. Both scaffold-runtime tiers first read as cancelled and resolved to real successes once only the latest run per check name was taken. Blocking on them would have manufactured a false red. Refs #1436, #1415 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../release-0.0.6-internals--orchestration/cut-trace.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 3870954476..99ea1fc927 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,13 +8,13 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | -| — | — | — | — | — | *no merges yet* | +| 1 | 2026-08-12 ~08:31Z | `63cd1cd58` | **#1527** | **#1436**, **#1415** (both auto-closed `COMPLETED`) | 7/7 pass — full record in the PR's `[PRE-MERGE GATE]` comment. `pr-checks` **15/15 `current-pass`, 0 current failures** at `dfda54a16`; `close-gate` green; #1415 4/4 boxes mirrored with linked evidence; #1436 has 0 boxes so the PR body is its record; no new ignores/casts, no lock churn; both probes re-run independently by the orchestrator; `review-threads` PASS (0 threads). | ## Wave clustering as dispatched | Wave | PR | Issues | Lane | Status | | --- | --- | --- | --- | --- | -| 1 | PR-A `fix/1436-1415-close-gate-trust` → **PR #1527** | #1436, #1415 | Sol · low | **dispatched** — thread `019ff4f4-1fce-7253-a7e0-d718c65b39cc`, worktree `/home/codex/repos/ns006-gatetrust`; S1–S4 landed | +| 1 | PR-A `fix/1436-1415-close-gate-trust` → **PR #1527** | #1436, #1415 | Sol · low | **MERGED** `63cd1cd58` — thread `019ff4f4-1fce-7253-a7e0-d718c65b39cc`, worktree `/home/codex/repos/ns006-gatetrust`, 7 commits | | 2 | PR-E `fix/1530-type-fixture-scan-scope` | #1530 | Sol · low | not yet dispatched (inserted; gates PR-D per rail R-1) | | 2 | PR-B `fix/1403-quality-gate-coverage` | #1403 | Sol · low | not yet dispatched | | 2 | PR-C `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | not yet dispatched | @@ -32,7 +32,9 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | When | Symptom | Root cause | Cost | Recorded where | | --- | --- | --- | --- | --- | --- | -| — | — | — | — | — | — | +| 1 | 2026-08-12 08:14–08:21Z | `status:ready-merge` applied; nothing happened. `close-gate` stayed red on its pre-label result, #1415's boxes stayed unticked. | `labeled` is absent from `ci.yml:41` and `e2e-cli.yml` `pull_request.types`, yet both `netscript-pr` and `check-close-gate.ts`'s repair hint claim the label triggers a run. | ~7 min, one wasted verification cycle | `drift.md` D-10. Fix is label-then-**push**; the one-line workflow/doc repair is raised to the owner, not taken by this lane. | +| 2 | 2026-08-12 ~09:56Z | Attached Codex launch received SIGTERM (exit 143) from a shell `timeout 580` wrapper. | The orchestrator wrapped an attached launch/resume in `timeout` to avoid blocking its own turn. The wrapper's SIGTERM kills the attached slice at expiry. | none this time — the thread survived and produced 5 further commits — but the practice is unsafe and was initially mis-recorded as harmless | `drift.md` D-9. Attached launch/resume now run unwrapped; bounded observation uses `agentic:codex-watch --timeout-seconds`. | +| 3 | 2026-08-12 08:00Z | `codex exec resume` refused with `thread-store conflict: … already has an active writer`. | Steering was attempted mid-turn instead of at a turn boundary. | ~2 min | `drift.md` D-9. That error is the mechanical signal for "not at a turn boundary"; steer on `codex-watch --mode turn` completion. | ## Falsified / confirmed assumptions @@ -46,4 +48,5 @@ needed. | A-2 | Re-baselining an issue body against HEAD is ceremony for mechanical issues | implicit in "mechanical ⇒ no plan needed" | **falsified** | The re-baseline is what caught A-1, on the smallest issue in the lane (p2, 2-line fix). | | A-3 | An issue's measured counts stay valid for the few days between filing and implementation | implicit in working from an issue body | **falsified, twice** | `arch:check:repo` FAIL moved 53 → **55** in four days, and `quality:scan:repo` moved from green to **RED exit 1** — the latter making one of #1378's own acceptance boxes unsatisfiable as written. Both found by executing, neither visible in the issue. | | A-4 | A doctrine verdict row that names a missing directory was renamed or deleted | #1380 D6 ("plausibly renamed into the `plugin-*-core` tier") | **falsified for 5 of 6 rows** | `git log --all --diff-filter=A` shows `packages/{streams,triggers,workers,sagas}` and `plugins/hello-world` **never existed** in this repo; only `@netscript/shared` ever did (`0ef13de35 chore: genesis eject`). A third state — *authored against a layout that never landed* — is required, and a rename note would have fabricated provenance. | +| A-6 | A `cancelled` expensive gate on a PR means the gate did not run | pre-merge gate check 4 read naively | **falsified, and it nearly manufactured a false red** | `scaffold-runtime` and `scaffold-runtime-sqlite` first evaluated as `cancelled`. Applying the #1142 rule — only the latest run per check name — resolved both to real `success` (`08:26:44Z`, `08:22:52Z`). Blocking on the cancelled pair would have been the mirror image of check 4's false greens. | | A-5 | The escalate-don't-idle brief instruction changes behaviour | `agent-milestone-orchestrator` § Delegation (4 idle-at-red-gate occurrences in 0.0.4) | **confirmed** | PR-A's agent hit a red Gate 1, diagnosed it as a pre-existing permission gap, refused to weaken unrelated tests, recorded it, escalated, and continued unblocked work. `drift.md` D-8. | From 18151931328bcdc3e4366cdd6be1e6f97b1be5c6 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 10:39:19 +0200 Subject: [PATCH 06/57] chore(harness): record the rail FAIL_PLAN and the unverified claim it caught The rail PLAN-EVAL (fresh Codex Sol high, opposite family) returned FAIL_PLAN on three findings. The consequential one is the orchestrator's own error: the plan asserted rfcs/ holds only a template and a README, inherited from #1380's 2026-08-08 measurement and never re-measured, inside a plan whose stated value is that it re-measured everything. Five numbered RFCs 0001-0005 are present at the plan's own baseline, so #1380's RFC divergence has already closed and its acceptance box means something different than the issue says. Also records that the A14 population is 53 sanctioned imports plus one real locally bound describe -- a live negative case the fix must still fire on, not a 54th false positive -- and that six acceptance boxes had no routed proof. Adds the supervision note that a stalled status label is not death: the refused resume with an active-writer lock is positive proof the thread is working. Refs #1403, #1380, #1378 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index bb2efaea56..e74bc01a86 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -187,3 +187,94 @@ being fixed, filed, or briefed by this lane. Both are the owner's call. - **Note on the pre-merge gate:** this is check 1's failure mode from the other side — the close-gate *result existed* and was red, but it was red for a reason the operator was told to fix in a way that does not work. "Unproven, not clean" applies to a stale result just as much as to a missing one. + +## D-11 — rail PLAN-EVAL returned FAIL_PLAN; the orchestrator's own plan carried an unverified claim + +- **Severity:** significant +- **Recorded:** 2026-08-12, stage B (rail), eval loop failure **1 of 2** +- **Evaluator:** fresh Codex · GPT-5.6 Sol · **high** session, thread + `019ff508-d5a3-7100-b6a0-1b6226e97e70`, worktree `/home/codex/repos/ns006-raileval` @ `83de0dc06`. + Opposite family to the Claude-authored plan, per `lane-policy.md` `formal_plan_evaluation`. +- **Verdict:** `FAIL_PLAN`, on three findings. Two are corrections to the plan; one is a routing gap. + +### Finding 2 is the one that matters most, and it is the orchestrator's error + +`plan-quality-rail.md` R-9 asserts: "`rfcs/` holds only a template and a README; five numbered RFCs +(0001–0005) were accepted through `.llm/runs/*/design/canonical/` … the de-facto path is the harness +path." **That is false at the plan's own baseline.** Independently confirmed by the orchestrator: + +```text +$ ls rfcs/ +0000-template.md +0001-sdk-client-contributions.md +0002-runtime-versioned-automation.md +0003-command-composition-kit.md +0004-deterministic-first-hybrid-mcp-doc-retrieval.md +0005-devtools-contribution.md +README.md +``` + +The five numbered RFCs are present, and the merges that landed them are in this run's own opening `git +log` read (`03680f6e8 docs(rfc): accept DevTools contribution architecture (RFC 0005)` and four +siblings). The plan inherited "Zero numbered RFCs have ever landed" from #1380's 2026-08-08 measurement +and **did not re-measure it** — inside a plan whose stated value proposition is that it re-measured +everything rather than trusting the issues. A-3 in `cut-trace.md` records issue counts going stale in +four days; R-9 is that same failure committed by the plan itself. + +Consequence beyond the plan: **#1380's D9/D10 divergence is stale in the same direction.** Its +acceptance box "The RFC-location divergence is resolved in `rfcs/README.md` with the 5 +`DECISION_PENDING` entries mapped to the chosen location" no longer means "choose a location" — the repo +already chose, within the last week. The correct deliverable is to record `rfcs/NNNN-*.md` as canonical +and map the five entries onto it, not to adjudicate a divergence that has closed. + +### Finding 1 — the A14 population is misclassified + +The plan states all 54 `FAIL A14` results are `@std/testing/bdd` false positives. The evaluator reports +**53 sanctioned imports plus one locally bound `describe`**. That distinction is load-bearing: R-5's +required negative case is "the rule must still fire on a real bare global", and there is a **live +instance** to test it against rather than a synthetic fixture. Treating all 54 as false positives risks +a fix that suppresses the true positive too — the mirror image of the defect #1380 is about. + +### Finding 3 — six live acceptance boxes have no stated proof route + +The plan's per-PR contract table does not route every box. An unrouted box is how a milestone reaches +merge with an unticked gate, which is the failure the whole lane exists to prevent. + +### Action + +The plan is revised before any rail PR is dispatched — no implementation proceeds on a `FAIL_PLAN`. +Revision: correct R-9's premise and restate the #1380 RFC deliverable; split the A14 population and +name the true positive as R-5's negative case; route the six boxes. Then a second PLAN-EVAL pass. The +eval loop limit is two failures before escalation; this is failure 1. + +### Process note worth keeping + +The evaluator caught R-9 **because the brief told it to check R-9's premise specifically** rather than +asking for a general review. The finding cost one eval cycle and would otherwise have reached #1380's +implementation, where it would have produced a doctrine document arguing about a divergence that no +longer exists. Also recorded: the evaluator ended its turn with an in-band summary instead of writing +`plan-eval.md`, so the verdict had to be recovered from its rollout and the thread resumed to produce +the artifact. Briefs should state that the file is the deliverable and a chat answer is not. + +## D-12 — a `stalled` status label is not death; verify the artifact and the writer lock + +- **Severity:** minor (supervision practice) +- **Recorded:** 2026-08-12, stage B (rail) +- **What happened:** `agentic:codex-status` reported the PLAN-EVAL thread as `stalled` with + `activityAgeMs` ≈ 626 s and `failure: null`, and no `plan-eval.md` existed. Its last recorded activity + was an **assistant message** ending "I'm writing the formal `FAIL_PLAN` verdict now", which reads like + a turn that ended without producing its deliverable. +- **What was actually true:** the thread was still mid-turn. The attempted `codex exec resume` was + refused immediately with `thread-store conflict: … already has an active writer`, and the rollout file + had grown to ~1.05 MB with a recent mtime. +- **Why it matters:** this is `agent-milestone-orchestrator` § Supervision pitfalls in its less obvious + direction. The recorded lesson is "liveness is not progress" — a live socket does not mean work is + happening. The converse also holds: **a `stalled` label does not mean the agent is dead.** A research + agent believed idle for 70 minutes was 25/27 complete and came within one command of being killed. +- **The reliable signals, in order:** (1) the writer lock — a refused resume with `active writer` is + positive proof the thread is working; (2) a growing rollout file; (3) a written artifact or new commit. + The `state` field is a hint, not a verdict. +- **Practice:** wait at the turn boundary with + `agentic:codex-watch --mode turn --thread-id --timeout-seconds N`. Never kill on a `stalled` + label alone. The failed resume was harmless — it fails fast and creates no rival sender — which is the + behaviour that made this recoverable. From c8006809e3ae455adec74ab594ee05d9f382124d Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 10:51:02 +0200 Subject: [PATCH 07/57] chore(harness): land the rail PLAN-EVAL verdict (FAIL_PLAN, 6 blocking) Records the evaluator's artifact verbatim: six blocking findings, three should-fix, one advisory, with a re-measured baseline table and a full command ledger that retains its intentionally-failed exploratory commands. Two of its blocking findings are protocol gaps on the orchestrator's side: no research.md exists and worklog.md has no Design section, both of which plan-gate.md requires. Two are wrong locked decisions (R-6 creates a transient second source of truth; R-9 rests on repository state already false at the plan's baseline). One is the unresolved #1374 extractor collision, which plan-gate.md classifies as FAIL_PLAN because it forces rework. One is six acceptance boxes with no stated proof route, against a live denominator of 34 rather than the 33 the brief asserted. Independently checked and not accepted as stated: #1530's post-merge box already carries the [post-merge] marker, so it is excluded from the merge gate by the sanctioned mechanism rather than needing a verification issue. Refs #1403, #1380, #1378, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../plan-eval.md | 264 ++++++++++++++++++ 1 file changed, 264 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-eval.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-eval.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval.md new file mode 100644 index 0000000000..2c38fa5c77 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval.md @@ -0,0 +1,264 @@ +FAIL_PLAN + +## Identity + +| Field | Value | +| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| Provider / model / effort | OpenAI · GPT-5.6 Sol · high | +| Role | Formal PLAN-EVAL of a Claude-authored plan; no implementation performed | +| Worktree | `/home/codex/repos/ns006-raileval` | +| Evaluated checkout | branch `eval/quality-rail-plan-eval`, `HEAD 83de0dc06ba9c2dbec864fa02a6dea432c99198e` | +| Requested baseline | `9c3cdfead`; this is the parent of the evaluator-brief-only commit `83de0dc06` | +| Plan measurement baseline | `01aa12b67`; all substantive paths are identical between `01aa12b67`, `9c3cdfead`, and `83de0dc06`; only run artifacts differ | +| Date | 2026-08-12 | + +The prompt described the worktree as detached at `9c3cdfead`. Ground truth was a clean named branch +at `83de0dc06`; `git diff 9c3cdfead..83de0dc06` contains only `plan-eval-brief.md`. The verdict +evaluates the plan at the actual checkout and reports the mismatch. + +## Re-measured baseline + +| Claim | Plan | Independent result | Verdict / command | +| ------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ | +| Default quality scan | exit 0, `allowCount: 7` | exit 0, 0 findings, `allowCount: 7` | **confirmed** — `deno task quality:scan` | +| Repo quality scan | exit 1, 5 findings, `allowCount: 10` | exit 1, 5 `ts-error-suppression` findings in `sdk-client-contributions-rfc_type.ts`, `allowCount: 10` | **confirmed** — `deno task quality:scan:repo` | +| Curated doctrine gate | exit 0 | exit 0 | **confirmed** — `deno task arch:check` | +| Repo doctrine gate | exit 1, `FAIL=55` = 54 A14 + 1 A1 | exit 1, `FAIL=55` = 54 A14 + 1 A1 | **confirmed as a count** — `deno task arch:check:repo` | +| Cause of all 54 A14 failures | 54 sanctioned `@std/testing/bdd` imports (`plan-quality-rail.md:35,71`) | 53 files import `@std/testing/.../bdd`; `packages/mcp/tests/service-endpoint-sources_test.ts:246-268` binds a local helper named `describe`, which the line scanner also flags | **diverges** — literal-safe import/path reconciliation over the gate output | +| Live top-level units | 30 `packages/*` + 6 `plugins/*` = 36 | 30 + 6 = 36 | **confirmed** — `ls -d packages/*/ plugins/*/` | +| Verdict-table drift | 6 non-live rows; 14 live units missing | 28 parsed rows; 6 non-live rows; 14 live units missing | **confirmed** — parsed `10-codebase-verdict-and-handoff.md:22-51` against live `deno.json` names/directories | +| Soundness tests | 6 | 6 | **confirmed** — `find packages plugins -type f -name '*-soundness_test.ts'` | +| Type fixtures | 12 `*_type.ts`, all below `tests/type-fixtures`, 3 with `@ts-expect-error` | 12 / all / 3 | **confirmed** — three `find` commands plus literal grep | +| Stale-row history | four package paths and `plugins/hello-world` never existed; only `packages/shared` did | the five paths have no add history; `packages/shared/deno.json` was added at `0ef13de35`; history records later shared-package consolidation/removal commits | **confirmed** — six `git log --all --diff-filter=A` probes plus removal-history probe | +| Numbered RFCs in `rfcs/` | zero; accepted records live only under harness paths (`plan-quality-rail.md:75`) | 5 accepted numbered RFCs, `0001` through `0005`, plus template and README | **diverges** — `find rfcs -maxdepth 1 -type f`; `git ls-tree 01aa12b67:rfcs` | +| Live acceptance-box denominator | 33 = 5 + 12 + 9 + 7 (evaluation brief) | 34 = 5 + **13** + 9 + 7 | **diverges** — live issue-body extraction with `gh issue view`; #1380 has a thirteenth `gate:` box | + +### `deno doc --json` feasibility measurement + +`packages/sdk` (12 export entrypoints) completed in 0.17 s and `packages/fresh` (15 entrypoints) in +0.74 s on individual timed runs. A loop over every export map of all 30 top-level package +directories completed in 3.733 s, with zero non-zero exits. Runtime is suitable for a PR gate. + +The full loop nevertheless emitted 567 `Warning Failed resolving types` warnings while returning +exit 0 (156 for `packages/ai`; 81 each for database, kv, plugin-sagas-core, and queue; 87 for +fresh). Therefore R-3 needs a fail-closed warning/partial-AST contract; exit 0 alone is not proof +that every published declaration was resolved. `deno doc --json` does expose symbol declarations and +origin locations (sampled on `packages/sdk/mod.ts`), so an AST-driven implementation is feasible. + +## Findings + +1. **blocking — the plan's load-bearing A14 baseline is wrong.** Evidence: `plan-quality-rail.md:35` + says the population grew to `54 × A14`, and R-5 at line 71 says all 54 are `@std/testing/bdd` + imports. Reconciliation of all 54 A14 output paths finds 53 such imports and one different false + positive: `packages/mcp/tests/service-endpoint-sources_test.ts:246-268` defines and calls a local + function named `describe`. `check-doctrine.ts:403-413` scans the whole file with a bare + identifier regex and therefore cannot distinguish either imported or locally declared bindings + from globals. Required change: re-baseline the decomposition as 53 sanctioned BDD imports + 1 + local binding, and make PR-C tests cover both origins plus a true unresolved global. + +2. **blocking — R-9 is based on repository state that was already false at the plan's baseline.** + Evidence: `plan-quality-rail.md:75` claims `rfcs/` holds only a template and README and selects + the harness path as canonical. At `01aa12b67`, `rfcs/` already contains accepted files + `0001-sdk-client-contributions.md` through `0005-devtools-contribution.md`; commits `b3dc006e8`, + `f3eb957ec`, `625be20a3`, `ef266832a`, and `03680f6e8` are all ancestors of `01aa12b67`. Each + file declares `status: Accepted`; `rfcs/0005-devtools-contribution.md:10-18` expressly identifies + `rfcs/README.md` as canonical and records correction away from an unmerged alternative + convention. Required change: replace R-9 with the live rule: accepted RFCs are promoted to + numbered `rfcs/NNNN-*.md`; harness design bundles are provenance/draft artifacts. Map the five + `DECISION_PENDING` entries to that canonical process without filing them. + +3. **blocking — PR-C's provenance deliverable cannot satisfy #1380 box 2 as currently worded.** + Evidence: the live box requires every removed row to be recorded as "renamed (with its new name) + or deleted," while `plan-quality-rail.md:54-57` locks a third state for five rows: authored + against a layout that never landed. The history supports the plan's factual state, but the issue + contract does not permit it; calling those units renamed or deleted would fabricate history. + Required change: reconcile the live #1380 target/acceptance wording before PR-C (with owner + authority) to admit "never existed under this name" and require per-row history evidence. Do not + tick the current binary box against a third-state deliverable. + +4. **blocking — six live acceptance boxes have no stated PR proof route.** Evidence: the per-PR + contracts in `plan.md:86-107`, `plan.md:143-150`, and `plan-quality-rail.md:111-118` do not state + deliverables/proofs for #1380 boxes 8-10 (which units `arch:check` gates and exclusions; dated + engineering-reference plan; five pending decisions mapped in `rfcs/README.md`), #1378 box 7 (type + both trigger reference sites), or #1530 box 7 (post-merge `code-quality-repo` green). #1378 box 6 + is assigned only a generic budget-overflow test; that does not prove the distinct requirement + that a budget increase carry an issue link in the same PR. Required change: add explicit PR-C, + PR-D, and PR-E slice rows with files and proving commands/checks for these six boxes. Route + #1530's observational post-merge box to a follow-up verification issue per + `milestone-run.md:104-108`, or keep #1530 open until it fires; a pre-merge PR cannot truthfully + close it. + +5. **blocking — the required Research and Design checkpoint inputs are absent.** Evidence: + `.llm/harness/evaluator/plan-protocol.md:11-22` requires `research.md`, `plan.md`, and the + `## Design` section of `worklog.md`; `plan-gate.md:16-34` makes current research and ordered + file/gate slices mandatory. The run directory contains no `research.md`, and `worklog.md` + contains no `## Design` section (no public surface/domain vocabulary/ports/constants/commit + slices/deferred scope/contributor path). `plan-quality-rail.md` carries useful baseline material, + but it does not supply the missing Design contract, and the four per-PR summary rows are not + file-scoped commit slices. Required change: add the standard artifacts (or explicitly amend the + harness protocol before evaluation), including ordered PR-E/B/C/D slices, exact files, and a gate + per slice. + +6. **blocking — the #1374 extractor collision is a rework-forcing open decision.** Evidence: + `plan-quality-rail.md:81` marks ownership "must resolve before PR-D" but leaves it open; + `plan-gate.md:20-22` says any open decision that would force rework is `FAIL_PLAN`. The live + #1374 plan at + `/home/codex/repos/ns006-1374-compilegate/.llm/runs/test-1374-docs-compile-gate--leaf/plan.md:49-65` + already specifies a checked-in `.md`/`.vto` backtick/tilde extractor, exact `ts`/`tsx` grammar, + line/ordinal provenance, and tests under `.llm/tools/docs`. The quality rail plans a second + `docs/site/**` fenced-TS extractor without selecting reuse or ownership. Required change: resolve + before any rail implementation begins: PR-D consumes the #1374 extractor if #1374 lands first, or + the shared extractor is extracted into an agreed prerequisite PR/API with one owner and one + parser test corpus. This can collide at `deno.json` already in PR-B; waiting until PR-D does not + avoid rail rework. + +7. **should-fix — R-3 needs a fail-closed diagnostic rule even though performance is acceptable.** + Evidence: the 30-package timing loop was 3.733 s, but `deno doc --json` returned exit 0 while + emitting 567 resolution warnings. Required change: state whether any resolution warning fails the + export audit, or identify and test an allowlisted warning class; also add a fixture proving a + re-exported `any` is attributed to the published entrypoint. Do not treat exit 0 as a complete + AST. + +8. **should-fix — R-4's phrase "live workspace members" is broader than #1380's 36-unit contract.** + Evidence: root `deno.json:3-9` includes `packages/cli/e2e`, `examples/*`, and `apps/*` in + addition to `packages/*` and `plugins/*`; `packages/cli/e2e` is an existing explicit nested + workspace member and passes a standalone doctrine scan. The verdict table and #1380 acceptance + are explicitly the 36 top-level package/plugin units. Required change: define the selector as + expanded top-level `packages/*` and `plugins/*` package members (and explicitly decide + `packages/cli/e2e`), rather than blindly iterating every workspace member. + +9. **should-fix — R-6 creates an unnecessary temporary source of truth.** Evidence: `deno.json:156` + is the 16-root shell list, R-6 moves it to checked-in data in PR-B, and R-4 changes its source to + workspace discovery in the immediately following PR-C. S-4 requires the coverage assertion, not + preservation of the transient list file. Required change: in PR-B add plugin-streams-core and a + coverage test against a reusable `discoverDoctrineRoots()` function; PR-C expands that same + function to the final top-level live-unit policy. If a checked-in curated list remains necessary, + state its durable post-PR-C role; otherwise do not create it. + +10. **advisory — the worktree identity in the evaluation brief is stale.** Evidence: raw git ground + truth reports branch `eval/quality-rail-plan-eval` at `83de0dc06`, not a detached checkout at + `9c3cdfead`. The only delta is the committed evaluator brief, so this did not alter substantive + measurements. Required change: record actual evaluator checkout identity in the handoff + automatically. + +## Decision review + +| Decision | Review | Reason | +| -------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| R-1 | **sound with caveat** | PR-E is a narrow, independently closable scanner-scope defect and makes the pre-existing repo gate green before PR-D. PR-D could absorb the exemption, but that would mix restoration with new rule power and would not satisfy #1530's post-merge observation. Keep PR-E, and route/hold the observational box honestly. | +| R-2 | **sound** | Directory + suffix precisely matches all 12 live fixtures and avoids a global `_type.ts` or test-regex carve-out; ordinary source and out-of-directory negatives make the rule fireable. | +| R-3 | **sound with caveat** | AST/export-map direction is feasible and fast (3.733 s for all 30 packages), and source origins are available. It must fail closed or explicitly classify the 567 resolution warnings observed on an exit-0 run. | +| R-4 | **sound with caveat** | Data-derived roots are correct, but "workspace members" must be narrowed to the 36 top-level package/plugin units or explicitly decide the nested `packages/cli/e2e` member. | +| R-5 | **sound with caveat** | Identifier origin is implementable without a type checker via lexical import/top-level binding collection; the newly found local `describe` proves imports alone are insufficient. The plan must acknowledge lexical binding/shadowing cost and test imports, local declarations, aliases, and genuine unresolved globals. | +| R-6 | **wrong** | A temporary checked-in 16-root data list is replaced in the next PR. S-4 can be honored by preserving/evolving the coverage predicate around a shared discovery function; it does not require two sources of truth or touching task ownership twice. | +| R-7 | **sound with caveat** | Ratcheting at the measured count is valid, but #1378 separately requires a budget increase to carry an issue link in the same PR; overflow alone does not prove that condition. | +| R-8 | **sound** | Scanner-scope correction is distinct from fixing surfaced product findings, and #1403 explicitly requires triage rather than absorption. | +| R-9 | **wrong** | Five accepted numbered RFCs already existed in `rfcs/` at `01aa12b67`; current repository practice is promotion into `rfcs/`, not harness-only canonical storage. | + +## Acceptance-box routing table + +The live denominator is **34**, not the brief's 33. "Unrouted" means the plan does not state a +specific deliverable and proof, even if the box could plausibly belong to the named PR. + +| Issue / box | Live acceptance criterion (abridged) | Route | Stated proof | +| ----------- | ----------------------------------------------------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------ | +| #1403-1 | plugin-streams-core covered by `arch:check` or reasoned exclusion | PR-B | Add root; `quality:gate` / `arch:check` | +| #1403-2 | scan plugin-core surfaces or document policy | PR-B | Settle root policy; coverage assertion | +| #1403-3 | repaired gate reports plugin-streams-core real state | PR-B | Run repaired gate; triage output | +| #1403-4 | future plugin-core omission fails | PR-B | root-coverage negative test | +| #1403-5 | surfaced findings triaged, not fixed | PR-B | triage list and no source fixes | +| #1380-1 | verdict lists all 36 live units only | PR-C | existence + coverage tests | +| #1380-2 | every removed row recorded renamed/deleted | **PR-C, contract-blocked** | planned third state is accurate but cannot satisfy the live binary box until the issue wording is reconciled | +| #1380-3 | `06-archetypes.md` matches verdict | PR-C | doctrine-document sync/existence tests | +| #1380-4 | repo check iterates live members | PR-C | workspace/top-level discovery implementation and gate | +| #1380-5 | A14 ignores sanctioned BDD imports | PR-C | BDD fixture negative control | +| #1380-6 | repo check excludes `.llm/tmp`, docs, tools | PR-C | per-member iteration; `arch:check:repo` output | +| #1380-7 | accepted-red debt closed or dated | PR-C | update `arch-debt.md`; repo gate/residue | +| #1380-8 | doctrine states which 36 units are gated and exclusions | **UNROUTED** | no named file/content assertion in PR-C contract | +| #1380-9 | dated engineering-reference plan | **UNROUTED** | no named artifact or date-content test | +| #1380-10 | RFC divergence resolved and five pending entries mapped | **UNROUTED** | R-9 names an obsolete canonical location and no mapping proof | +| #1380-11 | stale verdict row fails test | PR-C | existence negative test | +| #1380-12 | missing live row fails test | PR-C | coverage negative test | +| #1380-13 | `arch:check` green; repo green or residue enumerated | PR-C | named gate pair | +| #1378-1 | exported `any` fails red-first | PR-D | export/local fixture and scan | +| #1378-2 | unlinked `as unknown as` fails | PR-D | unlinked allowance/cast fixture | +| #1378-3 | docs fenced `as any` fails | PR-D | docs fence fixture | +| #1378-4 | six soundness files stay unchanged/green | PR-D | named six-file regression check | +| #1378-5 | wire `--max-allow` at measured count | PR-D | task changes + budget overflow | +| #1378-6 | budget increase requires same-PR issue link | **UNROUTED** | overflow test does not prove PR-diff/link coupling | +| #1378-7 | trigger reference doc and executable twin become typed | **UNROUTED** | neither file is named in PR-D deliverables/proof | +| #1378-8 | full rule test matrix | PR-D | exported/local, linked/unlinked, fence, soundness, overflow tests | +| #1378-9 | repo scan and arch check green | PR-D (depends PR-E/C) | named gate pair | +| #1530-1 | repo scan green with fixture unchanged | PR-E | scan gate + no fixture-line edit | +| #1530-2 | directory + suffix explicit exemption | PR-E | narrow rule test | +| #1530-3 | proven RED before exemption | PR-E | pre-change fixture failure | +| #1530-4 | ordinary/outside-directory directives remain red | PR-E | leakage negative controls | +| #1530-5 | remove two allowances; 10 to 8 | PR-E | scan JSON allowance count | +| #1530-6 | repo scan and quality gate green | PR-E | named gate pair | +| #1530-7 | post-merge `code-quality-repo` green on main | **UNROUTED / observational** | cannot be proven by the closing PR before merge; route to verification issue or keep #1530 open | + +## Negative-case review + +| PR | Review | +| ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| PR-E | Fireable: the under-directory fixture is red before and exempt after, while ordinary/outside-directory controls remain red. | +| PR-B | Fireable if it mutates the actual discovered/curated root source consumed by the task. Do not write a test against a duplicate fixture list. | +| PR-C | Verdict existence and coverage mutations are fireable. The A14 case in `plan.md:149` is only a desired non-firing positive case; the rail supplement at `plan-quality-rail.md:117` adds the required true bare-global red. Add the newly discovered local-binding non-firing case. | +| PR-D | Exported-any, unlinked cast, docs fence, and overflow are fireable. "Six soundness tests stay green" is regression evidence, not a negative case. The plan still needs a fireable same-PR budget-link control for #1378-6. | + +## Plan-Gate checklist + +| Gate | Result | +| ---------------------------- | ----------------------------------------------------------------------------------------------------- | +| Research present and current | **unchecked** — no `research.md`; two baselines and R-9 diverge | +| Decisions locked | checked, but R-6 and R-9 are wrong | +| Open-decision sweep | **unchecked** — #1374 collision is explicitly rework-forcing and unresolved | +| Commit slices | **unchecked** — four PR summary rows are not ordered, file-scoped commit slices | +| Risk register | checked | +| Gate set selected | checked at PR level, with missing acceptance-specific proofs noted above | +| Deferred scope explicit | checked | +| jsr-audit | N/A is justified because the rail changes repo tooling/governance, not package/plugin public surfaces | + +## What I executed + +Exit codes below are shell-command exit codes. A few exploratory commands intentionally failed and +were corrected; they are retained so the record distinguishes evidence from a hidden retry. + +| # | Command (abridged only where repeated arguments are listed in prose) | Exit | +| -- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------: | +| 1 | read five complete skill files with `sed`/`wc` (initial combined read was output-truncated; repeated per file) | 0 | +| 2 | raw `git rev-parse HEAD`; `git status --short`; `git branch --show-current` via `Deno.Command` | 0 | +| 3 | `rtk git log/show/diff` for `9c3cdfead..83de0dc06` | 0 | +| 4 | read activation, run-loop, lane-policy, plan protocol, plan gate, verdict definitions, milestone gate-integrity section, archetype matrix/selector, docs overlay | 0 | +| 5 | read plan, wave plan, supervisor, drift, cut trace, context pack, worklog and required source/docs files with numbered `sed`/`nl` | 0 | +| 6 | `gh issue view` for #1403, #1380, #1378, #1530 (live bodies) | 0 each | +| 7 | live checkbox-count loop over the four issues | 0 | +| 8 | `deno task quality:scan` | 0 | +| 9 | `deno task quality:scan:repo` | 1 (expected baseline red) | +| 10 | `deno task arch:check` | 0 | +| 11 | `deno task arch:check:repo` with compact roll-up | 1 (expected baseline red; `FAIL=55`) | +| 12 | `ls -d packages/*/ plugins/*/`; file inventories for soundness/type fixtures | 0 | +| 13 | first verdict-table reconciliation invocation, `deno eval --allow-read ...` | 1 (`deno eval` rejects that permission flag in Deno 2.9) | +| 14 | corrected verdict-table reconciliation with plain `deno eval` | 0 | +| 15 | six stale-path add-history probes + shared removal/path history | 0 | +| 16 | export-map print for sdk/plugin-streams-core/fresh; `deno doc --help` | 0 | +| 17 | timed `deno doc --json` over 12 SDK export entrypoints | 0 (0.17 s) | +| 18 | timed `deno doc --json` over 15 Fresh export entrypoints | 0 (0.74 s; resolution warnings) | +| 19 | 30-package export-map timing loop | 0 (3.733 s; 567 warnings) | +| 20 | first `deno doc` JSON-shape probe assuming an array | 1 (exploratory shape assumption wrong) | +| 21 | corrected `deno doc` JSON-shape probe (`{version,nodes}`) | 0 | +| 22 | read-only #1374 `git status/log/diff`; first combined regex command | 2 (unmatched backtick in shell pattern; no mutation) | +| 23 | corrected literal-safe #1374 plan/research/worklog search and plan read | 0 | +| 24 | live issue acceptance text extraction | 0 | +| 25 | RFC file listing, acceptance-commit log, ancestor checks, `git ls-tree 01aa12b67:rfcs`, five RFC header reads | 0 | +| 26 | raw `git diff/status` checks proving only run artifacts differ and tree is clean | 0 | +| 27 | search for required `## Design`/research artifacts | 0 (no matches; directory list returned) | +| 28 | first A14 import-origin reconciliation with a non-portable grep regex | 0 (emitted regex warnings; result rechecked) | +| 29 | literal-safe A14 import-origin reconciliation | 0 (`53` BDD imports, `1` other) | +| 30 | inspect `packages/mcp/tests/service-endpoint-sources_test.ts` local `describe` definition/calls | 0 | +| 31 | inspect root workspace patterns and standalone `packages/cli/e2e` doctrine behavior | 0 | +| 32 | final plan-eval path existence check and actual checkout comparison | 0 | +| 33 | first artifact integrity bundle: structure/count probe; `deno fmt --check`; raw status | 0; 1 (not formatted); 0 | +| 34 | `deno fmt plan-eval.md` | 0 | +| 35 | final `deno fmt plan-eval.md` after command-ledger update | 0 | +| 36 | final integrity bundle: structure/count probe; `deno fmt --check`; raw status/diff | 0 | From 112c1676b517b46ff09737b6f2546f8a7f3b846a Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:02:31 +0200 Subject: [PATCH 08/57] chore(harness): revise the quality rail against the FAIL_PLAN verdict Adds the two protocol artifacts cycle 1 lacked -- research.md and a Design section with 21 ordered file-scoped commit slices, each with a gate -- and answers every finding. Withdraws two locked decisions rather than editing them away. R-6 would have created a checked-in root list that the next PR replaces; a single discoverDoctrineRoots() introduced in PR-B and expanded in PR-C serves the same coverage predicate with one source of truth. R-9 rested on rfcs/ holding only a template, inherited from #1380 and never re-measured, when five accepted numbered RFCs are present at this plan's own baseline. Corrects the A14 decomposition: 53 sanctioned BDD imports plus one locally bound helper at packages/mcp/tests/service-endpoint-sources_test.ts:248, and zero live bare globals. Treating all 54 as import false positives would have shipped a fix that silenced the true positive too. Routes all 34 acceptance boxes, resolves the #1374 extractor collision by consuming that lane's parser rather than writing a second one, and makes the export audit fail closed because deno doc --json returned exit 0 while emitting 567 unresolved-type warnings. Rebuts one finding with evidence: #1530's post-merge box already carries the [post-merge] marker, which is the sanctioned exclusion mechanism, so it needs no verification issue. Refs #1403, #1380, #1378, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../plan-quality-rail.md | 97 ++++++++++- .../research.md | 164 ++++++++++++++++++ .../worklog.md | 101 +++++++++++ 3 files changed, 358 insertions(+), 4 deletions(-) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/research.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index fdf1ab1e79..ab9ea8ef18 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -67,12 +67,12 @@ against that walk, and `.agents/skills/netscript-doctrine` routes every agent th | `R-1` | **PR-E (#1530) lands before PR-D (#1378).** | #1378's `gate:` box requires `quality:scan:repo` green. It is red on `main` today for a reason that is not a defect (negative type fixtures scanned as production source). PR-D cannot truthfully tick that box until PR-E clears it. | | `R-2` | The `*_type.ts` exemption is keyed on **directory + suffix** (`tests/type-fixtures/**/*_type.ts`), never a filename allowlist and never a widened test regex. | A widened `_test`-style regex would exempt production code; an allowlist rots on the next fixture. The exemption must be an explicit rule asserted by test, which is the shape #1378 § Target contract already mandates for the 6 soundness tests. | | `R-3` | Export-awareness is driven by **`deno doc --json` over each package's `exports` map**, not a fourth line-regex. | #1378 offers both. `deno doc --json` already answers "is this symbol published"; `check-doctrine.ts:467-484`'s line-start matching provably misses `export const`, class members, generic defaults and re-exports. Adding a regex that misses the same things would ship a rule that looks export-aware and is not. | -| `R-4` | `arch:check:repo` iterates **live workspace members read from the root `deno.json` workspace list**, not a hand-listed set and not the repository root. | #1380 acceptance requires it. Reading the workspace list means a new package is gated the moment it joins the workspace — the property #1403's coverage test is reaching for, generalised. | -| `R-5` | The A14 predicate must resolve **where the identifier came from**, not merely that `describe(`/`it(` appears. | 54 of 55 `arch:check:repo` failures are `@std/testing/bdd` imports — the sanctioned API. A predicate that cannot tell an import from a global is not a Jest/Vitest detector. | -| `R-6` | `arch:check`'s 16-root task string is moved into **data** (a checked-in list the task and the tests both read) in **PR-B**, before PR-C generalises it. | The list is a ~2 kB inline shell string; a coverage test would have to parse shell to read it. Moving it to data first makes PR-B's coverage assertion honest and PR-C's live-member switch a small delta rather than a rewrite. | +| `R-4` | **[revised, cycle 2]** `arch:check:repo` iterates **expanded top-level `packages/*` + `plugins/*` workspace members**, not a hand-listed set, not the repository root, and **not every workspace member**. `packages/cli/e2e` is explicitly excluded and the exclusion is stated in the doctrine. | #1380 acceptance requires it. Reading the workspace list means a new package is gated the moment it joins the workspace — the property #1403's coverage test is reaching for, generalised. | +| `R-5` | **[revised, cycle 2]** The A14 predicate must resolve `imported` \| `locally-bound` \| `unresolved` and fire **only** on `unresolved`, collecting both import specifiers and top-level/local bindings lexically (no type checker). | The population is **53** sanctioned `@std/testing/bdd` imports **plus one locally bound helper** — `packages/mcp/tests/service-endpoint-sources_test.ts:248` declares `const describe = (workDir: string) => …` and imports nothing named `describe` (verified). A predicate that collects only imports would still flag it. Origin 3 (a genuine bare global) has **zero** live instances, so it needs a synthetic fixture and must stay red. | +| ~~`R-6`~~ | **WITHDRAWN, cycle 2 — the evaluator was right.** A checked-in 16-root data list would be replaced by R-4's discovery in the very next PR, creating two sources of truth and touching task ownership twice. **Replacement:** PR-B introduces `discoverDoctrineRoots()` and asserts coverage against **that function**; PR-C expands the same function to the final policy. S-4 is honoured by preserving and evolving the coverage predicate, which never required a list file. | | `R-7` | `--max-allow` is wired at the count **measured in the wiring PR**, and the PR body states that adding an allowance will now fail CI. | Wiring a budget is a behavioural change to every future PR, not a flag addition. Unstated, it reads as a break. | | `R-8` | Findings surfaced by newly-covered scans are **triaged into issues, never fixed in the surfacing PR** (`plan.md` S-5). PR-E is not an exception: it fixes the **scanner's scope**, not the findings. | #1403 box 5 and #1378 § Boundaries both require it. The distinction matters for honest box-ticking, so PR-E's body states it explicitly. | -| `R-9` | The RFC-location divergence (#1380 D9/D10) is resolved **by recording the decision the repo already made**, not by adopting a new process. | `rfcs/` holds only a template and a README; five numbered RFCs (0001–0005) were accepted through `.llm/runs/*/design/canonical/` and merged in the last week. The de-facto path is the harness path. PR-C states which location is canonical and maps the 5 `DECISION_PENDING` entries onto it — it does not invent a promotion pipeline. | +| ~~`R-9`~~ | **WITHDRAWN, cycle 2 — premise was false at this plan's own baseline.** The claim that `rfcs/` holds only a template and a README was inherited from #1380's 2026-08-08 measurement and never re-measured. At `01aa12b67`, `rfcs/0001-*.md` through `0005-*.md` are present, all `status: Accepted`, and `rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` as canonical. **Replacement (`R-9b`):** accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft artifacts. PR-C records that in `rfcs/README.md` and maps the five `DECISION_PENDING` entries onto it **without filing them**. See `drift.md` D-11. | ## Open decisions @@ -116,3 +116,92 @@ pre-existing tests call `Deno.makeTempDir()`). | PR-B | #1403 | Sol · low | The coverage test **fails** when a publishable `plugin-*-core` package is removed from the root list; the repaired gate reports `packages/plugin-streams-core`'s real state and its findings are triaged, not fixed | | PR-C | #1380 | Sol · medium | Existence test fails on a verdict row naming a non-existent directory; coverage test fails on a live unit with no row; A14 does **not** fire on `@std/testing/bdd` **and still fires** on a real bare global | | PR-D | #1378 | Sol · high | Exported `any` fails while a local `any` keeps its current severity; an unlinked `as unknown as` fails; `as any` in a `docs/site/**` fence fails; the 6 soundness tests stay green unchanged; budget overflow fails | + +--- + +# Revision 2 — response to `plan-eval.md` (`FAIL_PLAN`, 6 blocking / 3 should-fix / 1 advisory) + +Cycle 1 verdict: `FAIL_PLAN` from a fresh Codex · Sol · high session (thread `019ff508-…`), opposite +family to this Claude-authored plan. Every finding is accepted except one, which is rebutted with +evidence below. The withdrawn decisions are struck above rather than edited away, so the record shows +what was wrong and why. + +## Baseline correction — the A14 decomposition (finding 1, blocking) + +The table above says `FAIL=55` = 54 × A14 + 1 × A1. The **count** is confirmed; the **cause** was wrong. +Corrected, and verified independently by this session: + +| Identifier origin | Count | Sanctioned? | Evidence | +| --- | --- | --- | --- | +| imported from `@std/testing/bdd` | **53** | yes | e.g. `packages/database/tests/migrate-retry_test.ts:10` | +| **locally bound** in the test file | **1** | yes | `packages/mcp/tests/service-endpoint-sources_test.ts:248` — `const describe = (workDir: string) => …`; the file's imports (`:1-6`) are `@std/assert` plus four local modules, nothing named `describe` | +| genuinely unresolved global | **0** | **no — the real signal** | none live; needs a synthetic fixture that must stay red | + +`check-doctrine.ts:403-413` matches a bare identifier anywhere in a `*_test.ts` file and so cannot +distinguish any of the three. Treating all 54 as import false-positives would have produced a fix that +went quiet on the true positive too — the same defect, re-shipped inside its own repair. + +## Decisions revised, withdrawn, and added + +| ID | Change | +| --- | --- | +| `R-3` | **Caveat adopted (finding 7).** `deno doc --json` is fast enough — 3.733 s across all 30 package export maps, exit 0 — but that run emitted **567** `Warning Failed resolving types`. Exit 0 is therefore **not** a completeness proof. The export audit is **fail-closed**: an unresolved published declaration is a finding, unless it falls in a named, tested allowlist class. PR-D additionally carries a fixture proving a **re-exported** `any` is attributed to the published entrypoint. | +| `R-4` | Revised in place above — selector narrowed from "workspace members" to expanded top-level `packages/*` + `plugins/*`, with `packages/cli/e2e` explicitly excluded and the exclusion stated in the doctrine (finding 8). | +| `R-5` | Revised in place above — three origins, lexical import **and** binding collection, all three tested (finding 1). | +| ~~`R-6`~~ | **Withdrawn** (finding 9). Replaced by a single `discoverDoctrineRoots()` introduced in PR-B and expanded in PR-C. No transient list, one source of truth, S-4 honoured through the coverage predicate. | +| `R-7` | **Caveat adopted.** Ratcheting at the measured count does not prove #1378 box 6. A separate **fireable** control is added (slice D4): raising `--max-allow` **without** a same-PR issue link is red. | +| ~~`R-9`~~ | **Withdrawn** (finding 2). Replaced by `R-9b`. | +| `R-9b` | Accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft artifacts. `rfcs/0001-*.md`–`0005-*.md` exist at `01aa12b67`, all `status: Accepted`, and `rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` canonical. PR-C records this and maps the five `DECISION_PENDING` entries onto it **without filing them**. | +| `R-10` **(new)** | **The `docs/site/**` fenced-TS extractor is owned by #1374 and consumed by PR-D** (finding 6, resolved with owner authority). PR-D imports the extractor from draft PR **#1537** and sequences after it lands; it does **not** write a second parser. Coordination and the API request (stable per-snippet provenance) are posted at PR #1537 comment `5264583905`. **Fallback, stated now rather than discovered later:** if #1537's surface stays private to `docs:accuracy`, slice D5 and #1378 box 3 **move with the issue** — they are not forked and not ticked. | +| `R-11` **(new)** | The `labeled`-trigger defect (`drift.md` D-10) is fixed **in the documents, not the workflow** (owner decision). PR-C slice C7 corrects `.agents/skills/netscript-pr` and `check-close-gate.ts`'s repair hint to say "label, then push", regenerates the `.claude/skills/` mirror, and touches **no** workflow file. | + +## Open decisions — now none that force rework + +| Decision | Status | Resolution | +| --- | --- | --- | +| `docs/site/**` extractor ownership | **RESOLVED** | `R-10`. This was the finding-6 blocker; `plan-gate.md:20-22` is satisfied. | +| `arch:check:repo` residue after R-4/R-5 | safe to defer to PR-C | #1380 box 13 accepts "exit 0 **or** residue enumerated in `arch-debt.md`". PR-C measures and reports the real number. | +| `@netscript/shared`'s removing commit | safe to defer to PR-C | It existed (`0ef13de35`) so a removal is findable. If it cannot be found, the row records "removed, commit not identified" rather than a guess — the amended box 2 requires evidence, not a label. | + +## Acceptance-box routing — all 34 live boxes, none unrouted + +Denominator corrected to **34** (the brief said 33; #1380 carries a thirteenth `gate:` box). The six the +evaluator found unrouted are routed here to a named slice, file, and proof. Slice ids refer to the +`## Design` § Commit Slices table in `worklog.md`. + +| Issue / box | Route | Slice | Proof | +| --- | --- | --- | --- | +| #1403 · 1–5 | PR-B | B1–B3 | root added + `discoverDoctrineRoots()` coverage test; repaired gate run on `plugin-streams-core`; triage list with **no** `packages/**` source edit in the diff | +| #1380 · 1, 3, 11, 12 | PR-C | C3, C4 | existence test fails on a fabricated row; coverage test fails on an ungated live unit; `06-archetypes.md` sync test | +| #1380 · 2 | PR-C | C3 | **amended box** (owner-authorized, issue comment `5264580324`): per-row git evidence, with `never present under that name` admitted. Five rows carry the never-present verdict; `@netscript/shared` carries its removing commit. | +| #1380 · 4, 6 | PR-C | C2 | `arch:check:repo` iterates `discoverDoctrineRoots()`; output no longer contains `.llm/tmp/`, `docs/`, `.llm/tools/` paths | +| #1380 · 5 | PR-C | C1 | three-origin fixture: import quiet, local binding quiet, bare global **red** | +| #1380 · 7 | PR-C | C5 | `arch-debt.md` entry closed, or carries a dated closure plan naming both mechanical causes | +| **#1380 · 8** *(was unrouted)* | PR-C | C4 | `10-…md` gains a section naming which of the 36 units `arch:check` gates and why `packages/cli/e2e` is excluded; asserted by a test that fails if the gated set and `discoverDoctrineRoots()` disagree | +| **#1380 · 9** *(was unrouted)* | PR-C | C5 | `10-…md` gains a **dated** plan for engineering-reference §1–§5/§8–§10, authored as a byproduct of the refactors; test asserts the section exists and carries a date | +| **#1380 · 10** *(was unrouted)* | PR-C | C6 | `rfcs/README.md` records `rfcs/NNNN-*.md` as canonical per `R-9b`; all five `DECISION_PENDING` ids (`CRON-SUBSYSTEM-DUP`, `RUN-ARTIFACT-ARCHIVAL-POLICY`, `PAGEBUILDER-LEGACY-COMPAT-TREE`, `FORMPAGEPROPS-PLAYGROUND-MIGRATION`, `REDIS-LEGACY-VALUE-FALLBACK`) mapped onto it; test asserts all five are present | +| #1380 · 13 | PR-C | C2 | named gate pair: `arch:check` exit 0, `arch:check:repo` exit 0 or enumerated residue | +| #1378 · 1 | PR-D | D1 | exported `any` red, local `any` unchanged, plus re-export attribution fixture | +| #1378 · 2 | PR-D | D2 | unlinked `as unknown as` red; linked allowance green | +| #1378 · 3 | PR-D | D5 | `as any` in a `docs/site/**` fence red, via #1374's extractor. **Moves with the issue** if #1537's surface stays private (`R-10`). | +| #1378 · 4 | PR-D | D5 | the 6 `*-soundness_test.ts` files green with `@ts-expect-error` lines **unchanged** — regression evidence, explicitly not a negative case | +| #1378 · 5 | PR-D | D3 | `--max-allow` wired at the count measured in that PR; overflow red | +| **#1378 · 6** *(was unrouted)* | PR-D | D4 | a **fireable** same-PR control: raising the budget without an accompanying issue link in the same PR is red. Overflow alone does not prove this, which is why it is its own slice. | +| **#1378 · 7** *(was unrouted)* | PR-D | D6 | `docs/site/reference/triggers/index.md:310` and `docs/site/reference/triggers/examples_test.ts:65` both typed; both compile with no `any` | +| #1378 · 8 | PR-D | D1–D5 | full matrix: exported/local, linked/unlinked, fence, soundness, overflow | +| #1378 · 9 | PR-D | D1–D6 | `quality:scan:repo` + `arch:check` green (depends on PR-E and PR-C) | +| #1530 · 1–6 | PR-E | E1–E4 | RED-first fixture; dir+suffix rule; leakage controls both directions; `allowCount` 10 → 8; named gate pair | +| **#1530 · 7** | PR-E, **`[post-merge]`** | — | **Rebuttal to finding 4.** The live box already reads `` `gate:` the `code-quality-repo` job is green on `main` after merge. `[post-merge]` `` (issue #1530 line 92). `netscript-pr` makes `[post-merge]` the sanctioned mechanism: such a box is *visibly excluded from the merge gate with a notice* and verified by comment afterwards. It therefore needs no verification issue and does not keep #1530 open — the honesty rule is satisfied by the marker, which is what the marker is for. | + +## Protocol artifacts added (finding 5, blocking) + +`research.md` now exists, and `worklog.md` carries a `## Design` section with public surface, domain +vocabulary, ports, constants, **21 ordered file-scoped commit slices with a gate each**, deferred scope, +and the contributor path. Cycle 1 had neither; `plan-gate.md:16-34` requires both. + +## Finding 10 (advisory) — accepted + +The evaluator brief described its worktree as detached at `9c3cdfead`; ground truth was branch +`eval/quality-rail-plan-eval` at `83de0dc06`, the brief-only commit on top. The evaluator caught the +mismatch itself and reported it. Cycle 2's brief states the actual branch and sha, and the launcher's +`--expect-base` is set to the same value it verifies. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/research.md b/.llm/runs/release-0.0.6-internals--orchestration/research.md new file mode 100644 index 0000000000..ae3a6789ae --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/research.md @@ -0,0 +1,164 @@ +# Research: 0.0.6 internals quality rail + +Written to close `plan-eval.md` finding 5 (`plan-protocol.md:11-22` and `plan-gate.md:16-34` require a +current `research.md`; the first PLAN-EVAL cycle had none). Every number is a command result at +`01aa12b67`/`63cd1cd58`, re-executed by this session. Where a figure disagrees with an issue body, the +issue is stale and the divergence is stated. + +## Question the rail has to answer + +`quality:gate` and `arch:check` are the repo's two self-assessment gates. Three issues say, in different +words, that both are **uninformative rather than wrong**: they run, report faithfully, and cannot +distinguish a pass from a did-not-run. The rail's job is to make each gate's scope and severity legible, +without turning a false green into a false red. + +## The four gates, as they actually behave + +| Task | `deno.json` | Behaviour at HEAD | Exit | +| --- | --- | --- | --- | +| `quality:scan` | `:50` | `DEFAULT_ROOTS = ['packages/cli/src', 'plugins']` (`scan-code-quality.ts:18`). 0 findings, `allowCount: 7`. | 0 | +| `quality:scan:repo` | `:51` | `--root packages --root plugins`. **5 findings**, `allowCount: 10`. | **1** | +| `arch:check` | `:156` | `deps:check` + **16** hand-listed `check-doctrine.ts --root` invocations in one ~2 kB shell string. `packages/plugin-streams-core` is the only `plugin-*-core` absent. | 0 | +| `arch:check:repo` | `:157` | bare `check-doctrine.ts`, **no `--root`**, so `:110-113` treats the repository root as one package. `FAIL=55`, `WARN=341`. | **1** | + +### `quality:scan`'s five rules are line-regex, and two of its blind spots are structural + +`scan-code-quality.ts` reports `explicit-any-ignore`, `unsafe-cast`, `explicit-any`, +`plugin-name-check`, `ts-error-suppression`. + +- **`explicit-any` cannot see an export.** `:51` is `/(?:<|:\s*)any(?:\s*[,>;)\]}]|\b)/` against a raw + line. There is no notion of `export`, so an `any` in a published type and an `any` in a local + helper are the same finding at the same severity. +- **An export-aware rule exists elsewhere and is warn-only.** `check-doctrine.ts:467-484` emits + `A1/F-5: 'any' in exported declaration` at **WARN**, matching only `export function` / `export type` / + `export interface` line starts — not `export const`, class members, generic defaults, or re-exports — + and only under `arch:check`'s 16 roots. +- **`isScannable` (`:86-89`) exempts `_test`/`.test`/`.spec` and `.generated.ts`, and nothing else.** + `*_type.ts` negative fixtures are therefore scanned as production source. This is the whole of #1530 + (below). +- **Allowances are unbudgeted and unlinked.** `:136` accepts `// quality-allow: `. + `--max-allow` exists at `:173-181` and is passed by **no** task and **no** workflow. +- **Markdown is never opened** (`:87` matches only `/\.[cm]?[jt]sx?$/`), and `:47` additionally skips any + line beginning with a quote or backtick. + +### `quality:scan:repo` is red on `main`, and the PR gate structurally could not have caught it + +Five `ts-error-suppression` findings, all `@ts-expect-error` lines in +`packages/sdk/tests/type-fixtures/sdk-client-contributions-rfc_type.ts` — where the directive **is** the +fixture's assertion. Introduced by `b3dc006e8` (RFC 0001 acceptance). The blocking `code-quality-repo` +job has failed on **7 consecutive pushes to `main`**; last green `0fbe3dadd`. + +`code-quality.yml:28` skips draft PRs and `:36-42` scans only files the PR changed, while the repo-wide +job (`:50-59`) runs on push-to-main and a Monday cron. So the only path that ever scanned this file runs +**after** merge. Filed as **#1530**. + +Fixture surface: **12** `*_type.ts` files, all under a `tests/type-fixtures/` directory, in 3 packages +(`sdk`, `fresh`, `plugin-streams-core`); **3** contain `@ts-expect-error`. Two of them already carry +per-line `// quality-allow:` suppressions whose reasons both say, in prose, "negative compile fixture" — +a rule stated twice in comments and zero times in code. + +### `arch:check:repo`'s 55 failures decompose into three origins, not two + +#1380 D8 says 53 failures = 52 A14 false positives + 1 A1. Measured now: **55** = 54 A14 + 1 A1. The A14 +population grows with every new BDD test, which is the argument for fixing the predicate rather than +enumerating residue. + +`check-doctrine.ts:403-413` matches a bare `describe(` / `it(` / `expect(` identifier anywhere in a +`*_test.ts` file. It cannot tell where the identifier came from, and there are **three** origins: + +| Origin | Count | Sanctioned? | Example | +| --- | --- | --- | --- | +| imported from `@std/testing/bdd` | **53** | yes — the Deno BDD API | `packages/database/tests/migrate-retry_test.ts:10` | +| **locally bound** in the test file | **1** | yes — an ordinary local helper | `packages/mcp/tests/service-endpoint-sources_test.ts:248` — `const describe = (workDir: string) => …`; the file imports nothing named `describe` (verified: imports at `:1-6` are `@std/assert` and four local modules) | +| genuinely unresolved global | **0 today** | **no — this is the real Jest/Vitest signal** | none at this baseline | + +This is a correction to both #1380 and to the rail's own first-cycle plan, which treated all 54 as +`@std/testing/bdd` imports. It matters because the fix must **still fire** on origin 3 while going quiet +on 1 and 2 — so origin 2 is a live negative control, and a predicate that only collects imports would +re-flag it. + +The single `A1: mod.ts missing` is structural: with no `--root`, the checker evaluates the repository +root as a package and walks `.llm/tmp/`, `docs/site/`, and `.llm/tools/`. + +## The doctrine verdict table is not stale — for most rows it was never a measurement of this tree + +Live units: **30** `packages/*` + **6** `plugins/*` = **36**. The table at +`10-codebase-verdict-and-handoff.md:22-51` parses to **28** rows: **6** name non-live units, and **14** +live units have no row (the whole auth family and the entire `plugin-*-core` tier). + +Probed over the **full** history (`git log --all --diff-filter=A`), not just `main`: + +| Removed row | Ever added? | Correct record | +| --- | --- | --- | +| `@netscript/streams`, `@netscript/triggers`, `@netscript/workers`, `@netscript/sagas` | **no** | never present under that name | +| `plugins/hello-world` | **no** | never present under that name | +| `@netscript/shared` | **yes**, `0ef13de35 chore: genesis eject` | genuinely removed; PR-C cites the removing commit | + +#1380 hypothesises the four packages were "plausibly renamed into the `plugin-*-core` tier". The history +does not support that, and asserting it would fabricate provenance. #1380's box 2 was **amended with +owner authorization** to admit "never present in this repository under that name" **and** to require +per-row git evidence — strictly harder than the original label-only wording. Audit trail: +issue #1380 comment `5264580324`. + +## The RFC divergence has already closed — the plan's first cycle got this wrong + +#1380 D9/D10 states "Zero numbered RFCs have ever landed" and `ls rfcs/` → template + README only +(measured 2026-08-08). **False at `01aa12b67`:** + +```text +$ ls rfcs/ +0000-template.md 0001-sdk-client-contributions.md 0002-runtime-versioned-automation.md +0003-command-composition-kit.md 0004-deterministic-first-hybrid-mcp-doc-retrieval.md +0005-devtools-contribution.md README.md +``` + +The accepting merges (`b3dc006e8`, `f3eb957ec`, `625be20a3`, `ef266832a`, `03680f6e8`) are all ancestors +of `01aa12b67` — they are in this run's own opening `git log` read. Each file declares +`status: Accepted`, and `rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` as canonical. + +The rail's first-cycle plan inherited the stale claim without re-measuring and was failed on it +(`plan-eval.md` finding 2). Recorded as `drift.md` D-11 rather than quietly corrected, because the +lesson is the point: the plan's stated value was that it re-measured everything, and it did not +re-measure the one claim it had copied from a document it was in the business of correcting. + +Consequence for #1380 box 10: the deliverable is no longer "choose a location". The repo has chosen. +Record `rfcs/NNNN-*.md` as canonical, classify `.llm/runs/*/design/canonical/` bundles as +provenance/draft artifacts, and map the five `DECISION_PENDING` entries onto the canonical process +without filing them. + +## `deno doc --json` is fast enough, and its exit code is not a completeness proof + +Measured by the evaluator and adopted as the rail's basis: `packages/sdk` (12 entrypoints) 0.17 s, +`packages/fresh` (15) 0.74 s, and a loop over all 30 package export maps **3.733 s** with zero non-zero +exits. Suitable for a PR gate. + +But that loop emitted **567** `Warning Failed resolving types` while returning **exit 0** (156 for +`packages/ai`; 81 each for `database`, `kv`, `plugin-sagas-core`, `queue`; 87 for `fresh`). So an +export-awareness rule built on `deno doc --json` must decide what an unresolved declaration means. Exit +0 is not evidence that every published declaration was read — which is the same defect class the rail +exists to remove, one level up. + +## Cross-lane collision: one fenced-TS extractor, not two + +#1378 needs fenced-TypeScript extraction from `docs/site/**`. The **docs lane** is already building it +in draft PR **#1537** (#1374): a checked-in `.md`/`.vto` backtick/tilde extractor with an explicit +`ts`/`tsx` grammar, line/ordinal provenance, and tests under `.llm/tools/docs`. + +Two extractors would disagree about what counts as a snippet, and each gate would pass on the corpus it +happened to parse. Owner-confirmed resolution: **#1374 owns the extractor; #1378's PR-D consumes it and +sequences after #1537 lands.** Coordination posted at PR #1537 comment `5264583905`, including the one +API request (stable per-snippet provenance) and the explicit fallback if the docs lane keeps it private. + +## Workflow fact the rail must not re-learn + +Neither `ci.yml:41` nor `e2e-cli.yml` lists `labeled` in `pull_request.types`, so applying +`status:ready-merge` creates **no run** — while `netscript-pr` and `check-close-gate.ts`'s own repair +hint both say it does. Cost one verification cycle on PR #1527. Owner decision: **correct the two +documents, not the workflow.** Folded into a rail PR as a docs-only change. + +## What is deliberately not researched + +- The six open verdict-Refactor/Restructure package refactors (#1380 Boundaries). +- #1278 Inventory B, #1276 T1–T5, #1245, #1249, #1093, #1280, #1320. +- `packages/fresh-ui` quality extension — blocked on #1379's lock policy. +- Anything about canary or stable publication; root 0.0.6 orchestration owns it. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index 3e9b8baf90..e61f73924b 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -279,3 +279,104 @@ fully executed baseline table. Notable outputs of the re-baseline, all of which - A cross-lane collision: **#1374** (docs lane, live at `/home/codex/repos/ns006-1374-compilegate`) needs the same `docs/site/**` fenced-TS extraction that #1378 needs. Two extractors with different fence rules would disagree invisibly. Flagged as a must-resolve-before-PR-D open decision. + +--- + +## Design — quality rail (PR-E → PR-B → PR-C → PR-D) + +Written to close `plan-eval.md` finding 5: `plan-gate.md:16-34` requires ordered, file-scoped commit +slices with a gate per slice, and the first cycle offered four per-PR summary rows instead. Recorded +before any rail implementation file is created, per `run-loop.md` § 3b. + +### Public Surface + +Repo-internal tooling; no `packages/**`/`plugins/**` published surface changes, so no archetype and no +`jsr-audit`. + +- `.llm/tools/quality/scan-code-quality.ts` — adds `isTypeFixture(file)`, export-awareness, allowance + registration, docs-fence intake. Existing exported `QualityRule` union gains members; `QualityFinding` + gains an origin field. +- `.llm/tools/fitness/check-doctrine.ts` — adds `discoverDoctrineRoots()` and + `resolveIdentifierOrigin()`; A14 becomes origin-aware. +- `deno.json` tasks — `arch:check` consumes `discoverDoctrineRoots()`; `quality:scan`/`quality:scan:repo` + gain `--max-allow`. +- `docs/architecture/doctrine/{10-codebase-verdict-and-handoff,06-archetypes}.md`, + `.llm/harness/debt/arch-debt.md`, `rfcs/README.md` — documents, not code. + +### Domain Vocabulary + +- **doctrine root** — a top-level `packages/*` or `plugins/*` workspace member that `check-doctrine.ts` + evaluates as one package. *Not* every `deno.json` workspace member: root `deno.json:3-9` also lists + `packages/cli/e2e`, `examples/*`, `apps/*`. **Locked:** the selector is expanded top-level + `packages/*` + `plugins/*` only, and `packages/cli/e2e` is **excluded** with that exclusion stated in + the doctrine (it is a nested e2e harness, not one of the 36 units #1380 enumerates). +- **identifier origin** — `imported` | `locally-bound` | `unresolved`. A14 fires **only** on + `unresolved`. Three origins exist live; see `research.md`. +- **type fixture** — a `*_type.ts` file under a `tests/type-fixtures/` directory, whose + `@ts-expect-error` lines are its assertions. +- **published reachability** — a declaration reachable from a package's `deno.json` `exports` map, as + answered by `deno doc --json`. +- **registered allowance** — a `// quality-allow:` whose reason contains an open, milestoned `#`. +- **snippet record** — one fenced TS block from `docs/site/**`, owned by #1374's extractor, carrying + (source file, fence ordinal, start line). + +### Ports + +- `discoverDoctrineRoots(): string[]` — the single source of truth for doctrine root selection. Exists so + the task string, `arch:check:repo`, and the coverage test read one function. Introduced in PR-B and + **expanded** in PR-C; never duplicated into a checked-in list (this replaces the withdrawn R-6). +- `resolveIdentifierOrigin(file, ident)` — lexical import + top-level/local binding collection. No type + checker; `check-doctrine.ts` stays a line/lexical scanner. +- **#1374's extractor** — consumed, not re-implemented. PR-D imports it; if its surface stays private, + #1378's docs-fence box is blocked on a follow-up rather than forked (owner-confirmed). + +### Constants + +- `TYPE_FIXTURE_DIR = 'tests/type-fixtures'`, suffix `_type.ts`. +- `SANCTIONED_BDD_SPECIFIER` — `@std/testing/bdd` (matched by specifier, not by identifier). +- `MAX_ALLOW` — wired at the count **measured in the wiring PR**, not at a literal from an issue body. + Measured today: 7 default / 10 repo-wide; PR-E lowers repo-wide to 8. + +### Commit Slices + +Ordered, file-scoped, one gate each. `deno test` roots take +`--allow-read --allow-env --allow-write --allow-run` (established by PR-A's escalation, `drift.md` D-8). + +| # | PR | Slice | Files | Gate | +| --- | --- | --- | --- | --- | +| E1 | PR-E | RED fixture: `@ts-expect-error` in a type fixture is reported | `.llm/tools/quality/scan-code-quality_test.ts` | `deno test .llm/tools/quality/` fails | +| E2 | PR-E | `isTypeFixture` exemption (dir + suffix) | `scan-code-quality.ts` | test green; `quality:scan:repo` exit 0 | +| E3 | PR-E | leakage controls: ordinary source, and `_type.ts` outside the dir, stay red | `scan-code-quality_test.ts` | both negatives fail-on-removal | +| E4 | PR-E | drop the two redundant allowances | `desktop-consumer_type.ts`, `sdk-assignability_type.ts` | `quality:scan:repo` `allowCount` 10 → 8 | +| B1 | PR-B | `discoverDoctrineRoots()` + coverage test asserting every publishable `plugin-*-core` is a root | `check-doctrine.ts`, `check-doctrine_test.ts` | test fails when a package is removed from the source | +| B2 | PR-B | `arch:check` consumes it; `plugin-streams-core` covered | `deno.json:156`, `check-doctrine.ts` | `arch:check` exit 0 | +| B3 | PR-B | run the repaired gate on `plugin-streams-core`; triage output to new issues | triage list in slice dir only | no `packages/**` source edit in the diff | +| C1 | PR-C | `resolveIdentifierOrigin()`; A14 fires only on `unresolved` | `check-doctrine.ts`, `check-doctrine_test.ts` | 3 fixtures: import (quiet), local binding (quiet), bare global (**red**) | +| C2 | PR-C | `arch:check:repo` iterates `discoverDoctrineRoots()`; stops walking `.llm/tmp`, `docs/`, `.llm/tools` | `deno.json:157`, `check-doctrine.ts` | `arch:check:repo` exit 0 or residue enumerated | +| C3 | PR-C | verdict table re-walked to 36 units; per-row git evidence incl. never-present rows | `10-…md` | existence + coverage tests fail on a fabricated row / an ungated unit | +| C4 | PR-C | `06-archetypes.md` synced; doctrine states which units are gated and why `packages/cli/e2e` is not | `06-archetypes.md`, `10-…md` | sync test | +| C5 | PR-C | `arch-debt.md` accepted-red entry closed or dated; dated engineering-reference plan | `arch-debt.md`, `10-…md` | content assertions | +| C6 | PR-C | `rfcs/README.md` records `rfcs/NNNN-*.md` as canonical; 5 `DECISION_PENDING` mapped | `rfcs/README.md`, `arch-debt.md` | mapping present for all five | +| C7 | PR-C | docs correction: `netscript-pr` + close-gate repair hint say "label, then push" | `.agents/skills/netscript-pr/SKILL.md`, `check-close-gate.ts`, mirrored `.claude/skills/` | `agentic:check-claude`; no workflow file touched | +| D1 | PR-D | export-awareness via `deno doc --json`, **fail-closed** on unresolved-type warnings | `scan-code-quality.ts` + test | exported `any` red, local `any` unchanged; re-export attribution fixture | +| D2 | PR-D | registered allowances (open milestoned `#n`) | `scan-code-quality.ts` + test | unlinked allowance red | +| D3 | PR-D | `--max-allow` wired at the measured count | `deno.json:50-51` | overflow red | +| D4 | PR-D | same-PR budget-link control | new check + test | raising the budget without a same-PR issue link is red | +| D5 | PR-D | docs fences via #1374's extractor | `scan-code-quality.ts` | `as any` in a `docs/site/**` fence red; 6 soundness files unchanged | +| D6 | PR-D | type the triggers reference and its executable twin | `docs/site/reference/triggers/index.md:310`, `examples_test.ts:65` | both compile without `any` | + +### Deferred Scope + +- #1530 box 7 (`code-quality-repo` green on `main`) — **observational**, already carries the + `[post-merge]` marker in the issue, so it is excluded from the merge gate by the sanctioned mechanism + and verified by comment after merge. This is a correction to `plan-eval.md` finding 4, which proposed + routing it to a verification issue; the marker already discharges it. +- PR-D's D5 depends on #1537 landing. If it does not, D5 and #1378 box 3 move with the issue rather than + being forked or ticked. + +### Contributor Path + +To add a doctrine-gated unit: create `packages//` with a `deno.json`; `discoverDoctrineRoots()` +picks it up and `arch:check` gates it with no task edit. To add a quality allowance: append +`// quality-allow: (#)` on the offending line and raise `--max-allow` in the same PR +as the issue link — the budget can only fall otherwise. From 585df073a63719132ba1f04e45fd72d7232dcdf2 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:04:00 +0200 Subject: [PATCH 09/57] chore(harness): record the rail replan and refresh the resumable state Adds three re-planning events -- the FAIL_PLAN and its revision, the owner-authorized amendment of #1380 box 2, and the accepted cross-lane extractor dependency on #1374 -- and brings context-pack.md up to date with PR #1527 merged, PR-E inserted, and the four owner decisions now in force. Refs #1403, #1380, #1378, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../context-pack.md | 20 ++++++++++++++----- .../cut-trace.md | 3 +++ 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md b/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md index 765c4cd278..b5c5d8fbde 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/context-pack.md @@ -14,10 +14,11 @@ stable cut. | Stage | Status | | --- | --- | -| A — Bootstrap | **done** — identity/worktree proved, five issue bodies read live, 30 acceptance boxes inventoried, run dir written | -| B — Wave plan | **done** — `plan.md` committed; dispatch preconditions executed and GREEN; wave-plan PLAN-EVAL decision recorded | -| C/D — Wave 1 (PR-A) | next | -| C/D — Wave 2 (PR-B → PR-C → PR-D) | pending | +| A — Bootstrap | **done** — identity/worktree proved, five issue bodies read live, acceptance inventoried, run dir written | +| B — Wave plan | **done** — `plan.md` + `plan-quality-rail.md` + `research.md` + `worklog.md` § Design; dispatch preconditions executed and GREEN | +| C/D — Wave 1 (PR-A) | **DONE — PR #1527 merged `63cd1cd58`**, closing #1436 + #1415, pre-merge gate 7/7 | +| B — Rail PLAN-EVAL | cycle 1 `FAIL_PLAN` (6 blocking, all answered); **cycle 2 in flight** on thread `019ff508-…` → `plan-eval-cycle2.md`. Two-cycle limit. | +| C/D — Wave 2 (PR-E → PR-B → PR-C → PR-D) | blocked on the cycle-2 verdict. No implementation dispatches on a `FAIL_PLAN`. | | E — Canary | N/A for this lane (`drift.md` D-3) | | G — Close | pending | @@ -25,13 +26,22 @@ stable cut. | PR | Branch | Closes | Lane | Eval | | --- | --- | --- | --- | --- | -| PR-A | `fix/1436-1415-close-gate-trust` | #1436, #1415 | Sol · low | both waived (owner, `drift.md` D-1); negative cases proven instead | +| ~~PR-A~~ | `fix/1436-1415-close-gate-trust` | #1436, #1415 | Sol · low | **MERGED `63cd1cd58`** — both evals owner-waived (`drift.md` D-1), negative cases proven instead | +| PR-E | `fix/1530-type-fixture-scan-scope` | #1530 | Sol · low | rail PLAN-EVAL + own IMPL-EVAL. **Must land before PR-D** (rail R-1) | | PR-B | `fix/1403-quality-gate-coverage` | #1403 | Sol · low | rail PLAN-EVAL + own IMPL-EVAL | | PR-C | `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | rail PLAN-EVAL + own IMPL-EVAL | | PR-D | `fix/1378-quality-scan-rule-power` | #1378 | Sol · high | rail PLAN-EVAL + own IMPL-EVAL | Strictly sequential (one active implementation thread). Sequencing locks S-1…S-6 in `plan.md`. +## Owner decisions in force (do not relitigate) + +1. **#1529 dropped** — the observed core-CI skip is intended; it was closed not-planned. No skip or + visibility behaviour was changed. Two incidental observations are parked unacted in `drift.md` D-7. +2. **#1380 box 2 amended** — admits "never present under that name", requires per-row git evidence. +3. **#1374 owns the docs-fence extractor**; PR-D consumes it and sequences after PR #1537. +4. **The `labeled`-trigger defect is fixed in the documents, not the workflow** (rail R-11, slice C7). + ## The three facts most likely to be lost 1. **#1436's prescribed fix is a no-op.** The `\b` it asks for is already in the code; `\b` is the diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 99ea1fc927..5a48c970df 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -26,6 +26,9 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | --- | --- | --- | --- | --- | | 1 | 2026-08-12, during PR-A impl | **Scope added then withdrawn by the owner.** #1529 (p0) was added to the lane as a separate leaf PR, then closed as not-planned with the observed CI skip declared intended. | Worktree `ns006-cigate`, branch `fix/1529-required-lane-visibility` and its brief were created, then removed. No workflow behaviour changed. Wave structure unaffected — nothing had been dispatched. | `drift.md` D-6; two incidental observations parked unacted in D-7. | | 2 | 2026-08-12, stage B measurement | **Defect filed from inside the run**: `quality:scan:repo` red on `main` for 7 consecutive pushes because negative type fixtures are scanned as production source. Filed as #1530. | New PR-E inserted **before** PR-D: #1378's `gate:` box requires `quality:scan:repo` green and cannot be truthfully ticked until #1530 lands (rail R-1). | `drift.md` D-5. This is the scope-drift checkpoint, taken explicitly rather than discovered at cut time. | +| 4 | 2026-08-12, stage B (rail) | **Rail plan failed its PLAN-EVAL** (`FAIL_PLAN`, 6 blocking). Two locked decisions withdrawn (R-6, R-9), the A14 baseline corrected, six acceptance boxes routed, two protocol artifacts added, and the #1374 extractor collision resolved. | No rail PR dispatched; plan revised and re-submitted as cycle 2 of a two-cycle limit. Wave 2 order unchanged. | `drift.md` D-11, `plan-eval.md`, `plan-quality-rail.md` § Revision 2. | +| 5 | 2026-08-12, stage B (rail) | **An acceptance criterion was amended on a live issue** — #1380 box 2, with owner authorization, to admit "never present under that name" and to require per-row git evidence. | PR-C can now close #1380 with an accurate provenance record instead of an unticked box. | Issue #1380 comment `5264580324`; `research.md`. The amended box is strictly harder than the original, which required only a label. | +| 6 | 2026-08-12, stage B (rail) | **Cross-lane dependency accepted**: #1374 owns the `docs/site/**` fenced-TS extractor; #1378's PR-D consumes it. | PR-D sequences after PR #1537. Stated fallback: if that surface stays private, slice D5 and #1378 box 3 move with the issue rather than being forked or ticked. | PR #1537 comment `5264583905`; rail `R-10`. | | 3 | 2026-08-12, during PR-A impl | **Orchestrator brief error corrected mid-slice.** Gate 1 omitted `--allow-write`; the agent escalated instead of idling. | Gate 1 amended for PR-A and the permission requirement carried into the rail plan's validation table. | `drift.md` D-8. | ## Failure modes that cost real time From 64c091c547bf15e55e45deaa694b46bec6b8a6b3 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:19:57 +0200 Subject: [PATCH 10/57] chore(harness): record cycle-2 FAIL_PLAN and stop the rail for owner arbitration Second FAIL_PLAN reaches the two-cycle limit, so implementation stops and the owner arbitrates rather than a third pass being attempted. Three findings are new information the rail had not seen: registering allowances would immediately invalidate the eight that survive PR-E, because none carries an issue id; R-3's fail-closed rule makes PR-D's own green gate unreachable against 567 measured deno doc warnings; and #1378 box 6 asserts a property of a diff, which a file scanner cannot observe. Four are the orchestrator's own sloppiness: the revised plan contradicts itself on wave membership after Revision 2 was appended rather than reconciled, the Design table claims 21 slices and has 20, the strictly-harder claim about #1380 box 2 was an overstatement, and the provenance conclusion ignored checked-in records that document rename or supersession. Refs #1403, #1380, #1378, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 54 +++ .../plan-eval-cycle2.md | 325 ++++++++++++++++++ .../slices/pr-e-1530/implement.md | 187 ++++++++++ 3 files changed, 566 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle2.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index e74bc01a86..78ff274dd3 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -278,3 +278,57 @@ the artifact. Briefs should state that the file is the deliverable and a chat an `agentic:codex-watch --mode turn --thread-id --timeout-seconds N`. Never kill on a `stalled` label alone. The failed resume was harmless — it fails fast and creates no rival sender — which is the behaviour that made this recoverable. + +## D-13 — rail PLAN-EVAL cycle 2 also returned FAIL_PLAN; two-cycle limit reached, escalating + +- **Severity:** architectural (the rail's plan is not implementable as written) +- **Recorded:** 2026-08-12, stage B (rail), eval loop failure **2 of 2** +- **Evaluator:** same fresh Codex · Sol · high session, thread `019ff508-…`, at `a9ddbdd46`. Verdict in + `plan-eval-cycle2.md`. Cycle-1 disposition: 5 addressed, 4 partially addressed, 1 not addressed. +- **Verdict:** `FAIL_PLAN`, seven blocking findings. `netscript-harness` § Evaluator Separation sets the + loop limit at two failures before escalation, so implementation stops here and the owner arbitrates. + +### The three findings that are genuinely new information, not restatements + +1. **Registered allowances would immediately invalidate the eight allowances that survive PR-E.** #1378 + requires every `// quality-allow:` to carry an **open, milestoned** `#`. The existing allowances + carry free-text reasons with no issue id. So the moment that rule lands, all eight become findings and + PR-D is red by construction. #1378 never states a migration path, and the rail plan did not notice. +2. **R-3's fail-closed rule makes PR-D's own green gate unreachable.** The measured population is **567** + `deno doc --json` unresolved-type warnings on an exit-0 run. Failing closed on them — which cycle 1 + correctly demanded — means #1378's `gate:` box can never go green without a classification or + migration plan for those 567. Adopting the cycle-1 caveat literally created a contradiction. +3. **#1378 box 6 has no executable hook.** "A budget increase must carry an issue link in the same PR" is + a property of a *diff*, not of a file. `quality:scan` reads files; it cannot see a PR. The proof needs + a CI-integrated predicate, and slice D4 named a test without naming that mechanism. + +### The findings that are the orchestrator's sloppiness + +4. **The revised plan contradicts itself on wave membership and order.** Revision 2 inserted PR-E, but the + authoritative wave sections above it still read PR-B → PR-C → PR-D. Appending a revision instead of + reconciling the document produced exactly the internal inconsistency this lane exists to catch in + others. +5. **The Design table claims 21 slices and contains 20**, and several rows are not file-scoped. A + miscount in the artifact written specifically to satisfy a commit-slice gate. +6. **"Strictly harder" was an overstatement** (finding 8). The amended #1380 box 2 *admits a state the + original did not* while *requiring evidence the original did not*. It is stricter on evidence and + broader on admissible states — not strictly harder. The claim on issue comment `5264580324` needs + correcting, because overstating the direction of an acceptance amendment is precisely the kind of + self-serving record this lane is repairing. +7. **The provenance conclusion ignored checked-in records that claim rename/supersession** (finding 7). + `git log --all --diff-filter=A` says four packages never existed; checked-in debt/doctrine records + explicitly document rename or supersession for some of them. Both cannot be true as stated. The + re-walk must reconcile them, not silently prefer the git probe. +8. **R-10's fallback is only honest if PR-D stops closing #1378** (finding 9). If slice D5 and box 3 move + with the issue, PR-D delivers partial work and must reference `#1378` **without** a closing keyword. + The plan kept `Closes #1378` while also allowing the box to move — an internal contradiction that would + have auto-closed an issue with an undelivered acceptance box. + +### Action + +No rail PR is dispatched, including PR-E. PR-E's own slices E1–E4 are untouched by any blocking finding +and it clears a red gate on `main`, so it is a candidate for a scoped owner authorization — but +dispatching it on the orchestrator's own authority, against a formal `FAIL_PLAN` that explicitly says +"do not begin implementation on the current plan", would be the generator overruling its evaluator. That +is the self-certification the harness forbids, so it goes to the owner as a decision rather than being +taken as a judgement call. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle2.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle2.md new file mode 100644 index 0000000000..fe120d4148 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle2.md @@ -0,0 +1,325 @@ +FAIL_PLAN + +## Identity + +| Field | Value | +| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| Provider / model / effort | OpenAI · GPT-5.6 Sol · high | +| Role | Formal PLAN-EVAL cycle 2 of a Claude-authored plan; no implementation performed | +| Worktree | `/home/codex/repos/ns006-raileval` | +| Branch | `eval/quality-rail-plan-eval` — confirmed | +| Evaluated checkout | `a9ddbdd46c50681063f8b0a2c7a8d5bcd052ed0a` — confirmed | +| Revision commit | `112c1676b517b46ff09737b6f2546f8a7f3b846a` — confirmed as HEAD's second parent and an ancestor of HEAD | +| Other HEAD parent | `819d25d82efa4d68659c2624bc8318e98a391680` | +| Plan measurement baseline | `01aa12b67`; `git diff --name-only 01aa12b67..HEAD` contains only this run's artifacts, so the measured product/tooling tree is unchanged | +| Date | 2026-08-12 | + +The checkout identity in the cycle-2 brief matches raw Git ground truth. The worktree was clean +before this verdict file was created. + +## Re-measured baseline + +| Claim | Plan / revision | Independent result at the evaluated checkout | Result / command | +| --------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | +| Checkout | branch `eval/quality-rail-plan-eval`, HEAD `a9ddbdd46`, containing `112c1676b` | exact branch and full SHA confirmed; HEAD parents are `819d25d82… 112c1676b…` | **confirmed** — raw Git through `Deno.Command` | +| `quality:scan` | exit 0, `allowCount: 7` | exit 0, 0 findings, `allowCount: 7` | **confirmed** — `deno task quality:scan` | +| `quality:scan:repo` | exit 1, 5 findings, `allowCount: 10` | exit 1, five `ts-error-suppression` findings, `allowCount: 10` | **confirmed** — `deno task quality:scan:repo` | +| `arch:check` | exit 0, 16 roots | exit 0; all 16 invocations report `FAIL=0` | **confirmed** — `deno task arch:check` | +| `arch:check:repo` failures | exit 1, `FAIL=55` = 54 A14 + 1 A1 | exit 1, `FAIL=55` = 54 A14 + 1 A1 | **confirmed** — `deno task arch:check:repo` plus A14 reconciliation | +| `arch:check:repo` warnings | `research.md:45` says `WARN=341` | `WARN=305` in the current run | **diverges** — the root scan includes ignored/scratch trees, so WARN is not checkout-stable | +| A14 origins | 53 BDD imports + 1 local helper + 0 unresolved globals | 53 + 1 + 0; the only non-BDD file is `packages/mcp/tests/service-endpoint-sources_test.ts` | **confirmed** — parsed all 54 A14 paths and classified their source | +| Local helper | local `describe` at line 248, no BDD import | definition at line 248; imports at lines 1–10 contain no `describe`; calls at lines 264 and 268 | **confirmed** — numbered source read | +| Live top-level units | 30 `packages/*` + 6 `plugins/*` = 36 | 30 + 6 = 36 | **confirmed** — `find` inventory | +| Verdict-table drift | 28 rows; 6 non-live rows; 14 live units missing | 28 / 6 / 14, with the same named sets | **confirmed** — parsed doctrine table against 36 live `deno.json` identities | +| Soundness files | 6 `*-soundness_test.ts` | 6 | **confirmed** — `find` | +| Type fixtures | 12 `*_type.ts`, all under `tests/type-fixtures`, 3 with `@ts-expect-error` | 12 / all / 3 | **confirmed** — `find` + literal grep | +| Stale-path add history | four package paths and `plugins/hello-world` never added; only `packages/shared` added | five paths have no add commit; shared added at `0ef13de35` and deleted at `fd8259b76` | **confirmed as path history** — six `git log --all --diff-filter=A` probes plus shared delete probe | +| Stale package-name history | the four `@netscript/{streams,triggers,workers,sagas}` identities never existed | `git log -S` finds no such identity in any historical `deno.json` | **confirmed as manifest history** — five name-history probes | +| Recorded rename/supersession provenance | plan says a rename claim would be fabricated | `arch-debt.md` explicitly records triggers and workers as “superseded” by their `plugin-*-core` packages, sagas as relocated, and a later debt row says sibling contract packages “were renamed” | **diverges** — `arch-debt.md:385-391,561-566,576-584,2091-2100` | +| Numbered accepted RFCs | 5 (`0001`–`0005`) under `rfcs/` | 5, each accepted | **confirmed** — file/status inventory and RFC commit log | +| `deno doc --json` feasibility | 30 packages in 3.733 s, 567 warnings, zero non-zero exits | SDK 12 entrypoints in 0.131 s; Fresh 15 in 0.304 s; all 30 in 3.420 s; 567 warnings; zero non-zero exits | **confirmed** — fresh timed export-map run | +| Live acceptance denominator | 34 = 5 + 13 + 9 + 7 | 34 = 5 + 13 + 9 + 7 | **confirmed** — live `gh issue view` extraction | +| #1530 post-merge marker | box 7 contains `[post-merge]` | marker present in live box 7; checked-in parser, mirror, close-gate, and tests all recognize it | **confirmed** — live issue plus `rg` over validation tooling | +| Design slice count | “21 ordered file-scoped commit slices” | 20 rows: E1–E4 (4), B1–B3 (3), C1–C7 (7), D1–D6 (6) | **diverges** — mechanical table-row count | +| Allowances after PR-E | repo allowance count becomes 8 | 8 remain after removing the two named type-fixture allowances, and all 8 lack a `#` reference | **new load-bearing baseline** — `scanCodeQualityDetailed` reconciliation | + +## Findings + +1. **blocking — R-3's fail-closed rule makes PR-D's required green gate unreachable on the measured + corpus.** `plan-quality-rail.md:148` makes every unresolved published declaration a finding + unless it belongs to a named, tested allowlist class; `worklog.md:361` repeats “fail-closed.” The + fresh export-map run returned exit 0 with **567** `Warning Failed resolving types` warnings, + exactly the population `research.md:129-139` records. No slice names an allowlist class, migrates + this residue, or changes the final expectation at `plan-quality-rail.md:192` that + `quality:scan:repo` is green. Required change: inventory the 567 warnings by cause, name and test + each permissible class, and route the remaining causes to a bounded slice; otherwise make the + entrypoint/re-export graph the primary implementation. State an expected post-D1 warning count + that can reach the final green gate. + +2. **blocking — registered allowances immediately invalidate all eight allowances that remain after + PR-E, but the plan has no migration slice.** D2 at `worklog.md:362` requires an open, milestoned + `#n`; the live #1378 target contract requires the same. Independent reconciliation found 10 + current allowances, 8 after the two E4 removals, and **8/8 remaining reasons have no issue id**. + `deno.json:50-51` also grants the scanner only `--allow-read`, so `scan-code-quality.ts` cannot + establish live open/milestoned state as planned. D2 proves only unlinked-red/linked-green; it + does not test closed or unmilestoned ids. Required change: add a named migration/triage slice for + the eight existing allowances; define the deterministic GitHub-resolution boundary and task + permissions; and test missing, closed, unmilestoned, and valid ids. The final repo scan must be + re-baselined after that migration. + +3. **blocking — #1378 box 6 is nominally routed to D4, but the stated proof has no executable hook + and is not file-scoped.** `plan-quality-rail.md:189` calls the control fireable; `worklog.md:364` + names its files only as “new check + test.” No task, workflow step, PR-body input, base/head diff + input, or failure command is specified. The current PR workflow at + `.github/workflows/code-quality.yml:36-43` runs the changed-file scanner and `arch:check`; the + repo job at lines 49–59 runs only `quality:scan:repo`. A standalone unit test can be green while + no PR ever executes the predicate. Required change: name the checker and test paths, define + exactly what “issue link in the same PR” means, wire the checker into a named PR gate with live + PR/diff inputs, and provide both a missing-link RED control and linked GREEN control. Until then + #1378-6's routed proof can did-not-run. + +4. **blocking — the withdrawn R-6 replacement still cannot satisfy its two-step contract without the + transient source of truth it claims to remove.** `plan-quality-rail.md:72,151` and + `worklog.md:310-313,351-355` require PR-B to introduce `discoverDoctrineRoots()` and make + `arch:check` consume it, then require PR-C to expand the same function to all 36 units, while + claiming “no transient list.” The current 16 roots at `deno.json:156` are a heterogeneous curated + set; no discovery predicate describes exactly those 16 plus `plugin-streams-core`. A top-level + discovery predicate yields all 36 immediately; preserving the interim 17 requires an explicit + list/predicate that PR-C then replaces. Required change: choose one coherent sequence: (a) PR-B + keeps task ownership and adds a fireable coverage assertion, then PR-C introduces final discovery + once; or (b) PR-B lands final 36-unit discovery and absorbs the resulting scope/triage. Do not + claim both staged expansion and no interim selection source. + +5. **blocking — the new Design artifact does not satisfy the commit-slice gate.** Revision 2 claims + 21 file-scoped slices at `plan-quality-rail.md:196-200`; the table has **20**. E1's proving gate + is literally `deno test .llm/tools/quality/ fails`, so treating E1 as a commit slice violates the + harness requirement that a landed slice pass its gate. B3 (“triage list in slice dir only”) and + D4 (“new check + test”) do not name files; E4 and D6 use ambiguous basenames; several + implementation rows omit the test file their gate changes. Required change: rewrite the table + with the actual count and exact repository-relative paths; make RED-first executions recorded + pre-change evidence inside a green commit slice (for example combine E1/E2), and give every slice + one executable post-slice gate whose PASS is distinguishable from did-not-run. + +6. **blocking — Wave 2 still has contradictory PR membership and ordering.** The authoritative wave + plan says “three issues,” “three sequential PRs,” and lists only B/C/D at `plan.md:80-107`; its + locks order B→C→D at `plan.md:109-118`. The Design heading orders E→B→C→D, while R-1 only + constrains E before D. `plan-quality-rail.md:1-5` still describes three PRs. An implementer + cannot determine whether the locked order is E→B→C→D, B→E→C→D, or B→C→E→D. Required change: + insert PR-E into `plan.md` and replace the scattered partial orders with one authoritative total + order, then align the plan title, PR count, dependencies, and Design heading. + +7. **blocking — the per-row provenance conclusion omits checked-in records that explicitly document + rename/supersession.** Path and package-manifest history supports “never present in tracked Git” + for the four package identities. It does not support the stronger claim at + `plan-quality-rail.md:54-57` and `research.md:97-101` that recording a rename would necessarily + be fabricated: `arch-debt.md:385-391` says triggers was superseded by plugin-triggers-core; + `:561-566` says the same for workers; `:576-584` relocates sagas debt to plugin-sagas-core; and + `:2095-2099` says the sibling contract packages “were renamed to `@netscript/plugin-*-core`.” The + live #1380 Target contract also still says rename-vs-deletion even though acceptance box 2 was + amended to add the third state. Required change: make C3/C5 reconcile these records explicitly. + For each row, distinguish “no tracked package/path existed” from “later doctrine calls this a + conceptual successor/rename,” correct or qualify the contradictory debt entries, and align the + live Target-contract prose with the owner-authorized acceptance wording before choosing the final + per-row label. + +8. **should-fix — the claim that #1380 box 2 became “strictly harder” is not literally true.** The + live edit broadened the permitted outcomes from two to three, so a “never present” result that + could not pass before can pass now; it separately strengthened the proof by requiring per-row Git + evidence. The amendment is an authorized correction of an untruthful binary, not an improper gate + escape, but it is a mixed change rather than a strictly stronger predicate. Required change: + describe it as “broader truth states plus stronger evidence,” and amend the still-binary Target + contract. + +9. **should-fix — R-10's fallback is honest only if PR-D stops closing #1378.** The live PR #1537 is + still plan-only at `87bce69b7`; its committed plan specifies an internal checker but not yet a + reusable exported extractor. The coordination comment establishes one owner and asks for stable + provenance. Moving D5 “with the issue” is not abandonment if #1378 stays open, but `plan.md:103` + and the PR contract still say PR-D closes #1378. Required change: state that the fallback changes + `Closes #1378` to a non-closing reference, moves the whole issue with written reason, and leaves + box 3 unticked; do not describe only the box as moving. + +10. **advisory — the revised artifact retains superseded prose that now contradicts its appended + decisions.** Examples: `plan-quality-rail.md:81` still marks extractor ownership “must resolve,” + line 90 says timing remains to be measured, and line 91 describes the withdrawn R-6 data-list + sequence. The Revision 2 appendix supersedes them, but implementer briefs can quote the earlier + live text. Required change: strike or annotate those rows in place, as was done for R-6/R-9, so + there is one operational instruction. + +## Cycle-1 finding disposition + +| Cycle-1 finding | Disposition | Evidence checked | +| ---------------------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1. A14 baseline misclassified | **addressed** | Fresh reconciliation confirms 53 BDD-import files, the one local `describe` at `service-endpoint-sources_test.ts:248`, and zero live unresolved globals. R-5 and C1 now require all three origins. | +| 2. R-9 stale RFC premise | **addressed** | Five accepted numbered RFCs exist; R-9 is withdrawn and R-9b matches `rfcs/README.md:43-59`. | +| 3. #1380 box 2 binary contract | **partially addressed** | The live acceptance box and owner comment `5264580324` admit “never present” and require per-row Git evidence. The live Target contract remains binary, and checked-in debt records explicitly claim rename/supersession; finding 7 remains. | +| 4. Six boxes unrouted | **partially addressed** | Five formerly missing boxes have named slice ids. #1530-7 is **rebutted-and-I-agree**: the live `[post-merge]` marker is recognized by the skill and code. #1378-6's D4 route still has no executable integration, so the overall finding is not fully closed. | +| 5. Missing Research and Design artifacts | **partially addressed** | Both files/sections exist. The Design claim is 21 but the table has 20; E1 is a red commit gate and several rows are not file-scoped. | +| 6. #1374 extractor collision | **addressed** | Owner coordination comment `5264583905` assigns extractor ownership to #1374 and consumption to #1378 after PR #1537. The fallback needs the closing-keyword clarification in finding 9 but does not fork or tick the blocked box. | +| 7. R-3 warning/completeness contract | **partially addressed** | Fail-closed behavior and a re-export fixture are now stated, but the measured 567-warning population has no allowlist/migration and makes the final green gate unreachable. | +| 8. R-4 selector too broad | **addressed** | Final selector is top-level `packages/*` + `plugins/*`; `packages/cli/e2e` is explicitly excluded. | +| 9. R-6 transient source of truth | **not addressed** | The separate list file is withdrawn, but PR-B still introduces an interim function that PR-C expands. The interim 17-root behavior cannot be derived without the transient selection described in finding 4. | +| 10. Checkout identity stale | **addressed** | Raw Git exactly matches the cycle-2 branch, HEAD, and revision ancestry in the brief. | + +## Decision review + +| Decision | Review | Reason | +| ---------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| R-1 | **sound with caveat** | PR-D could absorb the type-fixture scope fix, but PR-E independently closes the current main-gate defect earlier and proves #1530's narrow leakage controls. It is not logically required for implementation, but it is a defensible sequencing boundary once one total order is recorded. | +| R-2 | **sound** | Directory + suffix exactly matches all 12 live fixtures; the ordinary-source and out-of-directory controls make leakage observable. | +| R-3 | **wrong as operationalized** | `deno doc --json` is fast enough, but fail-closed treatment of 567 current warnings with no named allowed class cannot reach PR-D's green final gate. | +| R-4 | **sound** | Expanded top-level package/plugin members match the 36-unit issue denominator and avoid nested `packages/cli/e2e`, examples, and apps. | +| R-5 | **sound with caveat** | Import/local/unresolved origin is feasible with a lexical resolver for the measured cases. The implementation test must be scope-aware enough that a binding in an unrelated scope cannot suppress a genuine global, and should include import aliases in addition to the three minimum fixtures. | +| R-6 | **wrong** | Withdrawal of the separate list does not resolve the interim-selection contradiction: PR-B and PR-C still require different outputs from the same function. | +| R-7 | **sound with caveat** | Ratcheting at the live count is sound only after the existing eight unregistered allowances are migrated and the same-PR control is actually integrated. | +| R-8 | **sound** | Surfaced product findings should be triaged rather than absorbed. Registration metadata required to make the scanner itself green must nevertheless be explicitly routed. | +| R-9 / R-9b | **sound** | Numbered `rfcs/NNNN-*.md` is the live accepted-record path; harness bundles are provenance/draft inputs. | +| R-10 | **sound with caveat** | One extractor owner and a sequencing dependency avoid parser drift. The fallback is honest only when #1378 remains open and PR-D becomes explicitly partial. | +| R-11 | **sound with caveat** | The owner-selected documentation correction is implementable, but its added skill/validation paths should be included in the plan Target and PR-C scope rather than appearing only in the appendix. | + +## Acceptance-box routing table + +All 34 live boxes are listed. “Routed — proof blocked” means a PR is named, but the plan's stated +proof cannot presently satisfy the box; it is not treated as a pass. + +| Box | Acceptance requirement (abridged) | Route | Evaluation | +| -------- | ----------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------- | +| #1403-1 | plugin-streams-core covered by `arch:check` or reasoned exclusion | PR-B / B2 | routed | +| #1403-2 | quality scan covers plugin-core surfaces or policy documented | PR-B / B1–B2 | routed | +| #1403-3 | repaired gate reports plugin-streams-core's real state | PR-B / B3 | routed | +| #1403-4 | future plugin-core omission fails | PR-B / B1 | routed — proof depends on resolving the interim root-source contradiction | +| #1403-5 | surfaced findings triaged, not fixed | PR-B / B3 | routed; exact triage artifact path missing | +| #1380-1 | verdict contains all 36 live units only | PR-C / C3 | routed | +| #1380-2 | removed rows get per-row Git evidence and truthful state | PR-C / C3 | routed — proof blocked by unreconciled rename/supersession records | +| #1380-3 | archetype table matches verdict | PR-C / C4 | routed | +| #1380-4 | repo gate iterates live members | PR-C / C2 | routed | +| #1380-5 | A14 ignores sanctioned BDD imports | PR-C / C1 | routed; three-origin test is fireable | +| #1380-6 | repo gate excludes `.llm/tmp`, docs, tools | PR-C / C2 | routed | +| #1380-7 | accepted-red debt closed or dated | PR-C / C5 | routed | +| #1380-8 | doctrine records gated units and exclusions | PR-C / C4 | routed | +| #1380-9 | dated engineering-reference plan | PR-C / C5 | routed | +| #1380-10 | RFC location and five pending ids mapped | PR-C / C6 | routed | +| #1380-11 | stale verdict row fails a test | PR-C / C3 | routed; negative is fireable | +| #1380-12 | missing live row fails a test | PR-C / C3 | routed; negative is fireable | +| #1380-13 | `arch:check` green; repo green or residue enumerated | PR-C / C2 | routed | +| #1378-1 | exported `any` fails red-first | PR-D / D1 | routed — proof blocked by unresolved current warning policy | +| #1378-2 | unlinked `as unknown as` fails | PR-D / D2 | routed — proof incomplete for closed/unmilestoned ids and current allowance migration | +| #1378-3 | fenced docs `as any` fails | PR-D / D5 after PR #1537 | routed conditionally; issue must move if extractor surface remains private | +| #1378-4 | six soundness files remain unchanged/green | PR-D / D5 | routed; regression evidence, not a negative control | +| #1378-5 | both tasks wire measured `--max-allow` | PR-D / D3 | routed — final count depends on missing allowance migration | +| #1378-6 | budget cannot rise without same-PR issue link | PR-D / D4 | **routed — proof blocked; no named checker or CI hook** | +| #1378-7 | both trigger reference sites typed | PR-D / D6 | routed | +| #1378-8 | full rule matrix | PR-D / D1–D5 | routed — incomplete until findings 1–3 are repaired | +| #1378-9 | repo scan and arch check green | PR-D / D1–D6 | **routed — proof blocked by 567 warnings and 8 unregistered allowances** | +| #1530-1 | repo scan green with fixture unchanged | PR-E / E1–E4 | routed | +| #1530-2 | explicit directory + suffix exemption | PR-E / E2 | routed | +| #1530-3 | RED before exemption | PR-E / E1–E2 | routed; record pre-change RED inside a green commit slice | +| #1530-4 | ordinary/out-of-directory cases remain red | PR-E / E3 | routed; negatives are fireable | +| #1530-5 | remove two allowances; 10 → 8 | PR-E / E4 | routed | +| #1530-6 | repo scan and quality gate green | PR-E / E2–E4 | routed | +| #1530-7 | main `code-quality-repo` green after merge | PR-E / `[post-merge]` follow-up | sanctioned exclusion; rebuttal accepted | + +No box lacks a named PR. Boxes #1378-6 and #1378-9 remain blocking because naming a PR does not make +their proof executable or satisfiable. + +## Negative-case review + +| PR | Review | +| ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| PR-E | The under-directory and leakage cases are fireable. E1 is not a valid landed commit gate because its expected result is a failing test run; record the pre-fix failure and land the test with E2 green. | +| PR-B | The plugin-core omission case is fireable only if its oracle independently inventories publishable plugin-core packages. A test that derives expected and actual roots from the same discovery function is tautological. The revised plan does not specify the independent input. | +| PR-C | The imported/local/unresolved A14 fixture is fireable; the bare-global fixture must assert a checker failure while the test process itself passes. Verdict-row existence and coverage mutation controls are also fireable. | +| PR-D | Exported/local, re-export, unlinked/linked, docs-fence, and overflow fixtures can be fireable. The six unchanged soundness files are regression evidence. D4 is not a gate until a named checker is wired into CI; the current statement can did-not-run. | + +## Plan-Gate checklist + +| Gate | Cycle-2 result | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| Research present and current | **unchecked** — artifact exists, but it omits eight unregistered allowances and contradictory rename records; WARN and slice counts diverge | +| Decisions locked | **unchecked** — PR ordering and the B→C root-source transition contradict each other | +| Open-decision sweep | checked with caveat — extractor ownership is assigned; fallback must keep #1378 open | +| Commit slices | **unchecked** — 20 rather than 21; red landed slice; non-file-scoped rows; D4 has no executable hook | +| Risk register | present, but retains superseded R-3/R-6 risks | +| Gate set selected | **unchecked** — D4 is not integrated and PR-D cannot reach its declared green gate | +| Deferred scope explicit | checked, subject to the #1378 closing-keyword clarification | +| jsr-audit | N/A justified: repo-internal tooling/docs, no package/plugin public surface change | + +## Escalation recommendation + +This is the second `FAIL_PLAN` in the two-cycle limit. Escalate to the repository owner before any +rail implementation. The owner should choose and record, in one revised authoritative plan: + +1. whether PR-B or PR-C owns the single transition to 36-root discovery; +2. how the 567 `deno doc` warnings are classified without making PR-D permanently red; +3. how the eight existing allowances are registered/migrated and how live issue state is resolved; +4. the exact CI-integrated same-PR budget-link predicate; +5. the reconciliation between Git history and the doctrine's recorded rename/supersession claims; +6. one total PR order including PR-E. + +After owner arbitration, replace the contradictory operational sections and run a new formal +PLAN-EVAL only by explicit owner direction or waiver; do not begin implementation on the current +plan. + +## What I executed + +Exit codes are command exit codes. Expected baseline-red gates are recorded as non-zero rather than +normalized to success. + +| # | Command | Exit | +| -: | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------: | +| 1 | `wc -l` over the six selected skill files | 0 | +| 2 | initial combined `sed` reads of harness, doctrine, and tools skills (output truncated by the caller) | 0 | +| 3 | `sed -n '1,180p'` and `sed -n '181,380p'` on `netscript-harness/SKILL.md` | 0 each | +| 4 | `sed -n '1,220p' netscript-doctrine/SKILL.md` | 0 | +| 5 | `sed -n '1,260p' netscript-tools/SKILL.md` | 0 | +| 6 | `sed -n '1,220p' netscript-deno-toolchain/SKILL.md` | 0 | +| 7 | `sed -n '1,200p'` and `sed -n '201,430p'` on `netscript-pr/SKILL.md` | 0 each | +| 8 | `sed -n '1,140p' rtk/SKILL.md` | 0 | +| 9 | complete repeated reads of doctrine, tools, Deno-toolchain, PR, and RTK skills after combined-output truncation | 0 each | +| 10 | raw Git bundle through `Deno.Command`: status, branch, HEAD, HEAD parents/subject, and `merge-base --is-ancestor 112c1676b HEAD` | 0 each | +| 11 | `wc -l && sed` reads of lane policy, plan protocol, plan gate, verdict definitions, and milestone-run | 0 each | +| 12 | repeated complete reads of `plan-protocol.md` and `lane-policy.md` after combined-output truncation | 0 each | +| 13 | `wc -l && sed` complete read of `research.md` | 0 | +| 14 | first worklog Design extraction using an ASCII-hyphen heading pattern | 0 (no section emitted) | +| 15 | `sed -n '285,430p' worklog.md` | 0 | +| 16 | `wc -l && sed` complete read of `plan-quality-rail.md` | 0 | +| 17 | `wc -l && sed` complete read of `plan.md` | 0 | +| 18 | first D-11/D-12 extraction using `^###` headings | 1 (heading level mismatch) | +| 19 | `rtk rg -n 'D-11 | D-12' drift.md`and corrected`sed -n '191,320p'` | +| 20 | `wc -l && sed` complete read of cycle-1 `plan-eval.md` | 0 | +| 21 | live `gh issue view --json` for #1403, #1380, #1378, and #1530 | 0 each | +| 22 | live `gh api` lookup of #1380 audit comment `5264580324` | 0 | +| 23 | live `gh api` lookup of PR #1537 coordination comment `5264583905` | 0 | +| 24 | live `gh pr view 1537 --json ...` | 0 | +| 25 | source/doc line counts and focused `rtk rg` over scanner, doctrine checker, tasks, and workflow | 0 | +| 26 | complete read of `scan-code-quality.ts` | 0 | +| 27 | focused complete A14/public-any/roll-up reads of `check-doctrine.ts` | 0 | +| 28 | numbered reads of doctrine 10, doctrine 06, RFC README, workflow; focused debt search | 0 | +| 29 | numbered reads of the repo-doctrine debt and five `DECISION_PENDING` entries | 0 | +| 30 | five historical `git log -S '@netscript/…' -- '**/deno.json'` probes | 0 each | +| 31 | `rtk rg` for stale package names and supersession across doctrine/debt | 0 | +| 32 | numbered reads of triggers/workers/sagas and archetype-5 rename debt | 0 | +| 33 | `deno task quality:scan` | 0 | +| 34 | `deno task quality:scan:repo` | 1 (expected baseline red) | +| 35 | compact `deno task arch:check` roll-up through `awk` with `pipefail` | 0 | +| 36 | compact `deno task arch:check:repo` roll-up through `awk` with `pipefail` | 1 (expected baseline red) | +| 37 | A14 path/origin reconciliation by executing the checker and reading all 54 reported files | 0 (embedded checker exit 1 recorded) | +| 38 | numbered read of `service-endpoint-sources_test.ts` imports/helper/calls | 0 | +| 39 | live directory, soundness, and type-fixture `find`/literal-grep inventory | 0 | +| 40 | doctrine verdict-table reconciliation with a read-only `deno eval` | 0 | +| 41 | six path-add history probes and shared deletion probe | 0 each | +| 42 | accepted numbered RFC inventory/status count and RFC commit log | 0 | +| 43 | fresh timed `deno doc --json` runs for SDK, Fresh, and all 30 package export maps | 0 | +| 44 | live acceptance-checkbox extraction/count for all four issues | 0 | +| 45 | mechanical Design slice-row print/count | 0 | +| 46 | `rtk git diff --stat` and `--name-only 01aa12b67..HEAD` | 0 | +| 47 | post-PR-E allowance reconciliation through `scanCodeQualityDetailed` | 0 | +| 48 | numbered reads of revised plan, Design, root wave plan, research, and routing appendix | 0 | +| 49 | read-only #1374 worktree raw status/branch/HEAD, focused plan/research search, and extractor-decision read | 0 each | +| 50 | `rtk rg -n 'post-merge | postMerge'` over skill and validation implementation/tests | +| 51 | numbered root workspace/task read | 0 | +| 52 | `test ! -e plan-eval-cycle2.md` | 0 | +| 53 | `apply_patch` creating only `plan-eval-cycle2.md` | not a shell command | +| 54 | `deno fmt plan-eval-cycle2.md` | 0 | +| 55 | `deno fmt --check plan-eval-cycle2.md` | 0 | +| 56 | final artifact structure/content probe | 0 | +| 57 | final raw Git status/diff scope check through `Deno.Command` | 0 | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md new file mode 100644 index 0000000000..de5d186dd7 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md @@ -0,0 +1,187 @@ +use harness + +# PR-E — #1530: negative type fixtures are scanned as production source, and `main` has been red for it + +You are the **implementation agent** for the first slice of the 0.0.6 internals quality rail. Narrow, +fully specified, and it clears a **currently red blocking gate on `main`**. + +Your orchestrator is a Claude Opus 5 high session in `/home/codex/repos/netscript-006-internals` on +`chore/release-0.0.6-internals-orchestration`. It holds merge authority. You implement, gate, and +report; you do not merge. + +## SKILL + +- `netscript-harness` — run artifacts, commit trail, slice discipline. +- `netscript-tools` — validation wrappers; what is a verdict and what is a non-verdict. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` block, + the `[post-merge]` marker. +- `netscript-deno-toolchain` — deterministic `deno test` behaviour. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-typefixtures` | +| Branch | `fix/1530-type-fixture-scan-scope` | +| Base | **`84dd44ae7`** = `origin/main` at dispatch. Contains PR #1527 (`63cd1cd58`, the gate-trust fix). Verified at this base: `deno task quality:scan:repo` still exits **1**, so the defect is live here. | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/` | +| Closes | #1530 | +| Route | Codex · gpt-5.6-sol · **low** | +| Plan | `plan-quality-rail.md` (incl. § Revision 2) · slices **E1–E4** in `worklog.md` § Design | + +Work only in that worktree. No rebase, no force-push. Push with an explicit refspec: +`git push origin HEAD:refs/heads/fix/1530-type-fixture-scan-scope`. + +## The defect, measured + +`deno task quality:scan:repo` exits **1** at `main` with five findings, all in one file: + +```text +packages/sdk/tests/type-fixtures/sdk-client-contributions-rfc_type.ts:349,365,445,465,502 + rule=ts-error-suppression // @ts-expect-error +``` + +Every one of those directives **is the fixture's assertion** — remove it and the negative type fixture +proves nothing. The blocking `code-quality-repo` job (`code-quality.yml:50-59`, on push-to-main and a +Monday cron) has therefore failed on **7 consecutive pushes to `main`**; last green was `0fbe3dadd`. + +Root cause, `scan-code-quality.ts:86-89`: + +```ts +return /\.[cm]?[jt]sx?$/.test(file) && !/(?:_test|\.test|\.spec)\.[cm]?[jt]sx?$/.test(file) && + !file.endsWith('.generated.ts'); +``` + +`*_type.ts` is not exempt, so type fixtures are scanned as production source. + +**Why the PR gate never caught it:** `code-quality.yml:28` skips draft PRs and `:36-42` scans only the +files a PR changed. The repo-wide job is the only path that ever scanned this file, and it runs *after* +merge. + +## Contract + +### C1 — the exemption is a rule, keyed on directory **and** suffix + +Exempt files matching **`tests/type-fixtures/` in the path AND a `_type.ts` suffix**. Not a filename +allowlist (it rots on the next fixture). Not a widened `_test`-style regex (that would exempt production +code). This is the shape #1378 § Target contract already mandates for the six `*-soundness_test.ts` +files: "Exempt by explicit rule, asserted by test." + +Measured surface: **12** `*_type.ts` files, all under a `tests/type-fixtures/` directory, in +`packages/sdk`, `packages/fresh`, `packages/plugin-streams-core`; **3** contain `@ts-expect-error`. + +### C2 — the exemption must not leak, and you must prove both directions + +Mandatory fixtures: + +| Case | Expected | +| --- | --- | +| `@ts-expect-error` in a `*_type.ts` **under** `tests/type-fixtures/` | **not reported** (this is the fix) | +| the same directive in ordinary source, e.g. a `packages/**/src/**.ts` fixture path | **still reported** | +| the same directive in a `*_type.ts` **outside** any `tests/type-fixtures/` directory | **still reported** | +| the same directive in a `tests/type-fixtures/` file **without** the `_type.ts` suffix | **still reported** | + +The last two are the leakage controls. A rule that exempts on directory alone, or suffix alone, passes +the fix case and fails the lane's purpose. + +### C3 — remove the two allowances the rule makes redundant + +```text +packages/fresh/tests/type-fixtures/desktop-consumer_type.ts:42 // quality-allow: negative compile fixture proves contract input remains enforced without bindings.d.ts +packages/sdk/tests/type-fixtures/sdk-assignability_type.ts:62 // quality-allow: negative compile fixture requires TypeScript's expect-error directive to prove string input remains rejected +``` + +Both reasons say, in prose, "this file is a negative compile fixture" — a rule stated twice in comments +and zero times in code. The rule now states it once, so delete both comment lines (only the +`// quality-allow:` comments — **do not** touch the `@ts-expect-error` lines or any fixture logic). + +Repo-wide `allowCount` must fall **10 → 8**, and you must show it from the scan's own JSON output before +and after. This matters beyond tidiness: #1378 wires `--max-allow` as a budget that can only fall. + +### C4 — do not touch the findings themselves + +Do **not** edit, delete, or suppress any `@ts-expect-error` line in +`sdk-client-contributions-rfc_type.ts` or any other fixture. This PR fixes the **scanner's scope**, not +the code the scanner mis-flagged. Verify with a diff audit that no `*_type.ts` fixture line other than +the two `// quality-allow:` comments changed. + +## Acceptance mapping + +#1530 has **7** boxes. Read them from the live issue, not this paraphrase. Six need a fenced +`acceptance-evidence` entry in the PR body, mapped by exact trimmed box text (see `netscript-pr`). + +**Box 7 is different and must be left alone:** it reads +`` `gate:` the `code-quality-repo` job is green on `main` after merge. `[post-merge]` ``. The +`[post-merge]` marker is the sanctioned mechanism for a fact that cannot exist before merge — the gate +visibly excludes it with a notice, and it is verified by a comment afterwards. **Do not** give it an +evidence entry, do not tick it, and do not drop the PR's closing keyword to escape it. The orchestrator +verifies it after merge. + +Box 3 says **proven RED**. That means a test that fails before your change and passes after, with the +pre-change output pasted. Commit the failing test as its own commit (slice E1) so the record shows RED +rather than asserting it. + +## Gates you must turn green — deliverables, not a checklist + +Paste real command output with exit codes into your per-slice PR comment. + +| # | Gate | Command | +| --- | --- | --- | +| 1 | quality tool tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/quality/` | +| 2 | repo-wide scan — **the headline** | `deno task quality:scan:repo` → must be **exit 0**, and its JSON must show `allowCount: 8` | +| 3 | default scan | `deno task quality:scan` → exit 0, `allowCount: 7` unchanged | +| 4 | quality gate | `deno task quality:gate` → exit 0 | +| 5 | scoped check / lint / fmt | `.llm/tools/run-deno-{check,lint,fmt}.ts --root .llm/tools/quality --ext ts` | +| 6 | before/after allowance count | run gate 2 at your base commit and at your head; paste both `allowCount` values | + +The `--allow-write --allow-run` on gate 1 is not optional: nine tests under `.llm/tools` call +`Deno.makeTempDir()` and one spawns a subprocess. This was established the hard way on PR #1527 — the +orchestrator's brief was wrong twice about it (`drift.md` D-8). Do not "fix" those tests. + +`deno task e2e:cli` is out of scope — no scaffold, DB, Aspire, or plugin copy-mode surface is touched. +Apply `ci:skip-e2e` and `ci:skip-scaffold`, and state that choice and its reason in your opening phase +comment so the cheap lane is visibly intentional. Never edit a workflow file to dodge a filter. + +## PR mechanics + +1. First commit is the slice-dir bootstrap; open the **draft PR in that same session**. Comment per slice. +2. Slice artifacts in `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/`: + `worklog.md`, `context-pack.md`, `drift.md`, updated **in the same commit** as the code they describe. +3. `## Scope` carries `Closes #1530` on its own line. Nothing else gets a closing keyword. +4. Labels: `type:fix`, `area:tooling`, `priority:p1`, `ci:skip-e2e`, `ci:skip-scaffold`, and exactly one + `status:` — `status:impl`. Milestone `0.0.6`. Do **not** apply `status:ready-merge`; the orchestrator + does that after its pre-merge gate. +5. Slices, per the plan's Design table: **E1** RED fixture (committed red) · **E2** the `isTypeFixture` + exemption · **E3** leakage controls both directions · **E4** drop the two redundant allowances and + record the 10 → 8 count. +6. Post one `**[PHASE: IMPL]**` summary when the slices are in, listing every gate with real output. + +## Two mechanical rules that cost this lane time already + +- **Resolve commit hashes in a separate step and paste the literal value.** A previous slice posted + `` `(git rev-parse --short=10 HEAD)` `` unexpanded into two PR comments, leaving the commit trail + without the hashes of its central fixes. +- **`status:ready-merge` is not a workflow trigger.** Neither `ci.yml:41` nor `e2e-cli.yml` lists + `labeled` in `pull_request.types`, so the label alone creates no run. Not your problem on this PR — + the orchestrator handles labelling — but do not be surprised by a stale `close-gate` result, and do + not chase it. + +## Boundaries + +- Touch only `.llm/tools/quality/**`, the two `// quality-allow:` comment lines named in C3, and your + slice dir. +- Do **not** change `arch:check`'s root list or `check-doctrine.ts` — that is PR-B and PR-C. +- Do **not** add export-awareness, allowance issue-links, `--max-allow` wiring, or docs-fence scanning — + all of that is #1378 / PR-D. This PR only corrects scanner **scope**. +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or a new + `// quality-allow:`. +- Do **not** modify `deno.lock` unless a real dependency change requires it. +- Do **not** merge and do **not** set `status:ready-merge`. + +## Escalate instead of going idle + +If a gate is red and you cannot green it, or a contract here proves wrong, write it in your slice +`drift.md`, post it as a PR comment, and continue with what is not blocked. Idling at a red gate is the +most expensive failure mode in this repo's orchestration record — and on PR #1527 the escalation path +worked: the agent found the orchestrator's gate command wrong and said so instead of stalling. Do that. From 83adb22871dfc99216ef64caee93f32a1bb930b5 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:27:27 +0200 Subject: [PATCH 11/57] chore(harness): consolidate the rail plan into revision 3 and resolve every blocking finding Replaces revisions 1 and 2 with one document. Appending a revision is what produced the self-contradiction cycle 2 failed the plan for, so the superseded text now lives only in git history and the two verdict files. Resolves the three substantive design holes. R-3 scopes fail-closed to the measured intersection with declarations the any-rule inspects, because failing closed on all 567 deno doc warnings made #1378's own green gate unreachable. R-7 registers the eight surviving allowances against new umbrella issue #1545, without which #1378's linked-issue rule would red CI on day one. R-12 proves the same-PR budget-link requirement with one added step in the existing code-quality PR job, since a file scanner cannot observe a diff. Retires the two-step root transition: PR-B now performs the single move to discoverDoctrineRoots() with no interim list, which is what both cycles asked for. Reconciles the provenance contradiction rather than preferring one source. This repository's history begins at 317e4b509 on 2026-07-06 with 374 commits and already contains the plugin-*-core tier, so the supersession that arch-debt.md records really happened -- before the earliest commit here. Both records are true, and PR-C writes both per row. Also corrects the strictly-harder overstatement on #1380, fixes the 21-vs-20 slice miscount, makes every Design row file-scoped, and states that PR-D drops Closes #1378 if the extractor fallback moves box 3. Refs #1403, #1380, #1378, #1530, #1545 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../plan-quality-rail.md | 316 ++++++++---------- .../plan.md | 6 + .../worklog.md | 13 +- 3 files changed, 157 insertions(+), 178 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index ab9ea8ef18..561dd32df0 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -1,207 +1,175 @@ -# Quality Rail Plan — #1403 → #1380 → #1378 +# Quality Rail Plan — #1530 → #1403 → #1380 → #1378 -The single plan for wave 2. One PLAN-EVAL covers all three PRs because the issues overlap on root -lists, scan semantics, doctrine and architecture debt; evaluating their sequencing separately would -evaluate the wrong thing. Sequencing locks S-2…S-5 live in `plan.md` and are not restated. +**Revision 3 — consolidated.** Replaces revisions 1 and 2 in full. Cycle 2 failed partly *because* +revision 2 was appended rather than reconciled, leaving the authoritative sections contradicting the +revision below them (`plan-eval-cycle2.md` finding 6). Superseded text stays in git history +(`112c1676b` and parents) and in the two verdict files; it is preserved but no longer operative. There +is exactly one order, one selector, and one decision table here. + +Sequencing locks S-1…S-6 live in `plan.md`; ordered file-scoped commit slices live in `worklog.md` +§ Design. Neither is restated. ## Run Metadata | Field | Value | | --- | --- | | Run ID | `release-0.0.6-internals--orchestration` | -| Phase | `plan` (rail) | -| Target | `.llm/tools/quality/scan-code-quality.ts`, `.llm/tools/fitness/check-doctrine.ts`, `deno.json` tasks, `docs/architecture/doctrine/`, `.llm/harness/debt/arch-debt.md`, `rfcs/README.md` | +| Phase | `plan` (rail) revision 3, awaiting PLAN-EVAL cycle 3 (owner-authorized past the two-cycle limit) | +| Target | `.llm/tools/quality/scan-code-quality.ts`, `.llm/tools/fitness/check-doctrine.ts`, `deno.json` tasks, `.github/workflows/code-quality.yml` (one added step), `docs/architecture/doctrine/`, `.llm/harness/debt/arch-debt.md`, `rfcs/README.md`, `.agents/skills/netscript-pr` | | Archetype | N/A — repo tooling and governance documents; no `packages/**`/`plugins/**` source authored | | Scope overlays | `SCOPE-docs.md` for PR-C's doctrine-document half | -## Executed baseline — every number below is a command result at `01aa12b67`, 2026-08-12 - -| Measurement | Value | Command | -| --- | --- | --- | -| `deno task quality:scan` | exit **0**, `findings: []`, `allowCount: 7` | executed | -| `deno task quality:scan:repo` | exit **1**, **5** findings, `allowCount: 10` | executed | -| `deno task arch:check` | exit **0**, 16 hand-listed roots | executed | -| `deno task arch:check:repo` | exit **1**, **FAIL=55** (54 × A14 + 1 × A1) | executed | -| Live units | 30 `packages/*` + 6 `plugins/*` = **36** | executed | -| Verdict-table rows naming non-live units | **6** | executed | -| Live units with no verdict row | **14** | executed | -| `*-soundness_test.ts` files | **6** | executed | -| `*_type.ts` files | **12**, all under `tests/type-fixtures/`, **3** contain `@ts-expect-error` | executed | +## The one PR order -### Drift against the issue bodies (measured 2026-08-08, now stale) +**PR-E (#1530) → PR-B (#1403) → PR-C (#1380) → PR-D (#1378 + #1545).** Strictly sequential, one active +implementation thread. PR-D is additionally gated on PR **#1537** (docs lane) landing. -| Issue claim | Then | Now | Consequence | +| PR | Closes | Lane | Why here | | --- | --- | --- | --- | -| `arch:check:repo` FAIL count (#1380 D8) | 53 | **55** | The A14 false-positive population grew by 2 in four days. It grows with every new `@std/testing/bdd` test, which is the argument for fixing the predicate rather than enumerating residue. | -| `quality:scan:repo` is green (#1378 § Current surface) | green, 0 findings | **RED, exit 1, 5 findings** | #1378's `gate:` acceptance box ("`quality:scan:repo` … green after the change") **cannot be satisfied** without first clearing this. Filed as **#1530**; it is PR-E, and PR-D depends on it. | -| Repo-wide `allowCount` (#1378) | 10 | 10 (unchanged) | `--max-allow` must be wired at the value measured **at the time of wiring**, not at 10 on faith. #1530 lowers it to 8. | - -### The 6 stale verdict rows are not what #1380 assumes - -#1380 hypothesises that "four of the five packages were plausibly renamed into the `plugin-*-core` -tier", while correctly warning that "the re-walk must record rename-vs-deletion per row, not assume". -Executed against the full history (`git log --all --diff-filter=A -- packages/

/deno.json`): - -| Stale row | Ever existed in this repo? | Correct record | -| --- | --- | --- | -| `@netscript/streams` | **no** — no `deno.json` ever added at `packages/streams` | never existed under this name | -| `@netscript/triggers` | **no** | never existed under this name | -| `@netscript/workers` | **no** | never existed under this name | -| `@netscript/sagas` | **no** | never existed under this name | -| `@netscript/shared` | **yes** — added at `0ef13de35 chore: genesis eject`, 10 commits of history | genuinely removed; PR-C must find and cite the removal | -| `plugins/hello-world` | **no** | never existed under this name | - -So "renamed vs deleted" is a **false dichotomy for five of the six rows**: the third state is *the -table was authored against a layout that never landed in this repository*. That materially changes the -deliverable — a rename note pointing `@netscript/workers` → `packages/plugin-workers-core` would be a -fabricated provenance claim. PR-C records the executed per-row verdict, with the third state allowed. - -This also reframes the whole issue: the verdict table is not merely *stale*, it was never a -measurement of this tree. `docs/architecture/doctrine/10-…md:197-208` defines doctrine completion -against that walk, and `.agents/skills/netscript-doctrine` routes every agent through it. - -## Locked rail decisions - -| ID | Decision | Rationale | -| --- | --- | --- | -| `R-1` | **PR-E (#1530) lands before PR-D (#1378).** | #1378's `gate:` box requires `quality:scan:repo` green. It is red on `main` today for a reason that is not a defect (negative type fixtures scanned as production source). PR-D cannot truthfully tick that box until PR-E clears it. | -| `R-2` | The `*_type.ts` exemption is keyed on **directory + suffix** (`tests/type-fixtures/**/*_type.ts`), never a filename allowlist and never a widened test regex. | A widened `_test`-style regex would exempt production code; an allowlist rots on the next fixture. The exemption must be an explicit rule asserted by test, which is the shape #1378 § Target contract already mandates for the 6 soundness tests. | -| `R-3` | Export-awareness is driven by **`deno doc --json` over each package's `exports` map**, not a fourth line-regex. | #1378 offers both. `deno doc --json` already answers "is this symbol published"; `check-doctrine.ts:467-484`'s line-start matching provably misses `export const`, class members, generic defaults and re-exports. Adding a regex that misses the same things would ship a rule that looks export-aware and is not. | -| `R-4` | **[revised, cycle 2]** `arch:check:repo` iterates **expanded top-level `packages/*` + `plugins/*` workspace members**, not a hand-listed set, not the repository root, and **not every workspace member**. `packages/cli/e2e` is explicitly excluded and the exclusion is stated in the doctrine. | #1380 acceptance requires it. Reading the workspace list means a new package is gated the moment it joins the workspace — the property #1403's coverage test is reaching for, generalised. | -| `R-5` | **[revised, cycle 2]** The A14 predicate must resolve `imported` \| `locally-bound` \| `unresolved` and fire **only** on `unresolved`, collecting both import specifiers and top-level/local bindings lexically (no type checker). | The population is **53** sanctioned `@std/testing/bdd` imports **plus one locally bound helper** — `packages/mcp/tests/service-endpoint-sources_test.ts:248` declares `const describe = (workDir: string) => …` and imports nothing named `describe` (verified). A predicate that collects only imports would still flag it. Origin 3 (a genuine bare global) has **zero** live instances, so it needs a synthetic fixture and must stay red. | -| ~~`R-6`~~ | **WITHDRAWN, cycle 2 — the evaluator was right.** A checked-in 16-root data list would be replaced by R-4's discovery in the very next PR, creating two sources of truth and touching task ownership twice. **Replacement:** PR-B introduces `discoverDoctrineRoots()` and asserts coverage against **that function**; PR-C expands the same function to the final policy. S-4 is honoured by preserving and evolving the coverage predicate, which never required a list file. | -| `R-7` | `--max-allow` is wired at the count **measured in the wiring PR**, and the PR body states that adding an allowance will now fail CI. | Wiring a budget is a behavioural change to every future PR, not a flag addition. Unstated, it reads as a break. | -| `R-8` | Findings surfaced by newly-covered scans are **triaged into issues, never fixed in the surfacing PR** (`plan.md` S-5). PR-E is not an exception: it fixes the **scanner's scope**, not the findings. | #1403 box 5 and #1378 § Boundaries both require it. The distinction matters for honest box-ticking, so PR-E's body states it explicitly. | -| ~~`R-9`~~ | **WITHDRAWN, cycle 2 — premise was false at this plan's own baseline.** The claim that `rfcs/` holds only a template and a README was inherited from #1380's 2026-08-08 measurement and never re-measured. At `01aa12b67`, `rfcs/0001-*.md` through `0005-*.md` are present, all `status: Accepted`, and `rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` as canonical. **Replacement (`R-9b`):** accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft artifacts. PR-C records that in `rfcs/README.md` and maps the five `DECISION_PENDING` entries onto it **without filing them**. See `drift.md` D-11. | - -## Open decisions - -| Decision | Status | Notes | -| --- | --- | --- | -| Whether `quality:scan` should scan `docs/site/**` fences itself, or consume a shared extractor with #1374 | **must resolve before PR-D** | #1374 (`test(docs): docs:accuracy is a fixed-string needle checker`) is live in the **docs lane** at `/home/codex/repos/ns006-1374-compilegate` and needs the same fenced-TS extraction from `docs/site/**`. Two independent extractors with different fence-parsing rules would disagree about what a snippet is, and the disagreement would be invisible. Coordinate before PR-D writes one. | -| Whether the residue of `arch:check:repo` after R-4/R-5 is green or enumerated debt | safe to defer to PR-C | #1380 accepts either ("exits 0 or its residue is enumerated in `arch-debt.md`"). PR-C measures after the two mechanical fixes and reports the real number. | -| Whether `@netscript/shared`'s removal commit can be cited precisely | safe to defer to PR-C | It existed and has 10 commits of history; the removal is findable. If it cannot be found, the row records "removed, commit not identified" rather than a guess. | - -## Risk register (rail-specific; `plan.md` holds the lane-wide register) +| PR-E | #1530 | Sol · low | Clears a gate red on `main` for 7 pushes; nothing downstream can report an honest `quality:scan:repo` until it lands | +| PR-B | #1403 | Sol · low | p0, and owns the **single** transition to discovery-based doctrine roots | +| PR-C | #1380 | Sol · medium | Consumes PR-B's selector unchanged; adds origin-awareness and the doctrine documents | +| PR-D | #1378, #1545 | Sol · high | Depends on PR-E (green repo scan), PR-B/C (settled roots), and #1537 (extractor) | -| Risk | Mitigation | -| --- | --- | -| PR-C invents rename provenance for the four never-existed rows. | The executed evidence and the third state are in the brief, with the exact command that produced them. | -| `deno doc --json` is too slow to run per package in CI. | Measure before committing to it in PR-D; the fallback is scanning only each package's `exports` entrypoints and their re-export graph, recorded as a decision, not discovered as a timeout. | -| R-6's root-list move collides with PR-C's live-member switch. | R-6 lands the *shape* (data, not shell) and PR-C changes the *source* (workspace list). Sequential, with S-4 forbidding deletion of PR-B's assertion. | -| Wiring `--max-allow` reds an unrelated in-flight PR from another 0.0.6 lane. | Three sibling lanes are active. PR-D announces the budget in its body and the orchestrator reports the merge immediately so the other lanes learn the count. | -| A14 fix silently exempts genuine Jest/Vitest usage. | The negative case is mandatory: a fixture with a real bare `describe(` global must still FAIL after the fix. | +## Executed baseline -## Validation plan +Measured at `01aa12b67`, re-confirmed at `84dd44ae7` (contains PR #1527). -| Order | Gate | Command | Expected | +| Measurement | Value | +| --- | --- | +| `quality:scan` | exit **0**, 0 findings, `allowCount: 7` | +| `quality:scan:repo` | exit **1**, **5** findings, `allowCount: 10` | +| `arch:check` | exit **0**, 16 hand-listed roots; `packages/plugin-streams-core` absent | +| `arch:check:repo` | exit **1**, **FAIL=55** = 54 A14 + 1 A1 | +| Live units | 30 `packages/*` + 6 `plugins/*` = **36** | +| Verdict table | 28 rows; **6** name non-live units; **14** live units have no row | +| `*-soundness_test.ts` / `*_type.ts` | **6** / **12** (all under `tests/type-fixtures/`, **3** with `@ts-expect-error`) | +| `deno doc --json` over all 30 export maps | **3.733 s**, exit 0, **567** `Failed resolving types` warnings | +| Repository history | root `317e4b509` (2026-07-06, "cut 0.0.1-beta.5"), **374** commits | + +### A14 has three identifier origins, not two + +`check-doctrine.ts:403-413` matches a bare identifier anywhere in a `*_test.ts` file, so it cannot tell +where the binding came from. + +| Origin | Count | Sanctioned | Evidence | | --- | --- | --- | --- | -| 1 | quality scan (default) | `deno task quality:scan` | exit 0 | -| 2 | quality scan (repo) | `deno task quality:scan:repo` | exit 0 **after PR-E**; exit 1 with 5 known findings before it | -| 3 | doctrine (curated) | `deno task arch:check` | exit 0 throughout | -| 4 | doctrine (repo) | `deno task arch:check:repo` | exit 1 / FAIL=55 before PR-C; after PR-C exit 0 or enumerated residue | -| 5 | rail tests | `deno test --allow-read --allow-write --allow-env .llm/tools/quality/ .llm/tools/fitness/` | pass | -| 6 | scoped wrappers | `run-deno-{check,lint,fmt}.ts --root .llm/tools --ext ts` | pass | -| 7 | doctrine-document tests | the new existence + coverage tests from PR-C | fail on a fabricated row and on an ungated live unit | +| imported from `@std/testing/bdd` | **53** | yes | `packages/database/tests/migrate-retry_test.ts:10` | +| locally bound | **1** | yes | `packages/mcp/tests/service-endpoint-sources_test.ts:248` — `const describe = (workDir: string) => …`; imports at `:1-6` are `@std/assert` + four local modules | +| genuinely unresolved global | **0** live | **no — the real signal** | needs a synthetic fixture that must stay red | -`deno task e2e:cli` is out of scope for the whole rail — no PR touches scaffold, DB, Aspire, or plugin -copy mode. `--allow-write` is required for the tool test suites (established by PR-A's escalation: 9 -pre-existing tests call `Deno.makeTempDir()`). +### The 6 stale verdict rows — both records are true -## Per-PR contract summary +The reconciliation cycle 2 demanded (finding 7). `arch-debt.md:385`/`:561` record that +`packages/triggers` and `packages/workers` existed and were **superseded** by their `plugin-*-core` +successors (created 2026-04-29, resolved 2026-07-03/07-06). Yet +`git log --all --oneline -- 'packages/

/**'` returns **0 commits** for all four packages and for +`plugins/hello-world`. -| PR | Closes | Lane | Must prove (negative case) | -| --- | --- | --- | --- | -| PR-E | #1530 | Sol · low | A `@ts-expect-error` in a `*_type.ts` under `tests/type-fixtures/` is reported **before** and not after; the same directive in ordinary source and in a `*_type.ts` **outside** `tests/type-fixtures/` is still reported; repo-wide `allowCount` falls 10 → 8 | -| PR-B | #1403 | Sol · low | The coverage test **fails** when a publishable `plugin-*-core` package is removed from the root list; the repaired gate reports `packages/plugin-streams-core`'s real state and its findings are triaged, not fixed | -| PR-C | #1380 | Sol · medium | Existence test fails on a verdict row naming a non-existent directory; coverage test fails on a live unit with no row; A14 does **not** fire on `@std/testing/bdd` **and still fires** on a real bare global | -| PR-D | #1378 | Sol · high | Exported `any` fails while a local `any` keeps its current severity; an unlinked `as unknown as` fails; `as any` in a `docs/site/**` fence fails; the 6 soundness tests stay green unchanged; budget overflow fails | +Both hold because **this repository's history is truncated**: it begins at `317e4b509` (2026-07-06, a +beta.5 release cut) with 374 commits, and `git ls-tree 317e4b509:packages/` already contains the full +`plugin-*-core` tier. The supersession predates the earliest commit here. So the deliverable is a +**reconciliation of two sources**, not a git verdict: ---- +| Row | Record PR-C must write | +| --- | --- | +| `@netscript/triggers` | not present anywhere in this repository's history (begins `317e4b509`); superseded by `packages/plugin-triggers-core` per `arch-debt.md:385`, predating this history | +| `@netscript/workers` | same, per `arch-debt.md:561` → `packages/plugin-workers-core` | +| `@netscript/sagas`, `@netscript/streams` | not present in this history; successors exist; **no** checked-in supersession record found — PR-C states that absence rather than inferring a rename | +| `@netscript/shared` | not present in this history; 10 commits touch `packages/shared/**` on non-ancestor refs only — PR-C cites that and claims no removal commit on `main` | +| `plugins/hello-world` | not present in this history; no successor and no supersession record | -# Revision 2 — response to `plan-eval.md` (`FAIL_PLAN`, 6 blocking / 3 should-fix / 1 advisory) +#1380 box 2 was **amended with owner authorization** to require per-row evidence and admit "never present +under that name" (issue comments `5264580324`, corrected by `5264832009`). The amendment is **stricter on +evidence and broader on admissible states** — not "strictly harder", which was an overstatement now +corrected on the issue. -Cycle 1 verdict: `FAIL_PLAN` from a fresh Codex · Sol · high session (thread `019ff508-…`), opposite -family to this Claude-authored plan. Every finding is accepted except one, which is rebutted with -evidence below. The withdrawn decisions are struck above rather than edited away, so the record shows -what was wrong and why. +### The RFC divergence has already closed -## Baseline correction — the A14 decomposition (finding 1, blocking) +`rfcs/` contains `0001-*.md`–`0005-*.md`, all `status: Accepted`, at `01aa12b67`; +`rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` canonical. #1380 D9/D10 is stale. +Revision 1 inherited that claim without re-measuring and was failed on it (`drift.md` D-11). -The table above says `FAIL=55` = 54 × A14 + 1 × A1. The **count** is confirmed; the **cause** was wrong. -Corrected, and verified independently by this session: +## Locked decisions -| Identifier origin | Count | Sanctioned? | Evidence | +| ID | Decision | +| --- | --- | +| `R-1` | **PR-E lands first.** #1378's `gate:` box needs `quality:scan:repo` green; it is red for a reason that is not a defect. PR-E fixes scanner **scope**, not the findings. | +| `R-2` | Type-fixture exemption keyed on **directory AND suffix** (`tests/type-fixtures/` + `_type.ts`), asserted by test, with leakage controls **both** ways — dir-only and suffix-only must still be reported. | +| `R-3` | Export-awareness uses `deno doc --json` over each package's `exports` map. **Fail-closed is scoped to the intersecting set:** PR-D first measures how many of the 567 warnings touch a declaration the any-rule actually inspects, fails closed on **that** set, and enumerates the non-intersecting residue in `arch-debt.md` with a date. Blanket fail-closed on all 567 makes #1378's own green gate unreachable — cycle 2 finding 1, and why revision 2's R-3 was wrong. A re-exported `any` must be attributed to the published entrypoint (fixture required). | +| `R-4` | Doctrine root selector is **expanded top-level `packages/*` + `plugins/*`** — the 36 units — **not** every workspace member (root `deno.json:3-9` also lists `packages/cli/e2e`, `examples/*`, `apps/*`). `packages/cli/e2e` is excluded and the exclusion is stated in the doctrine. | +| `R-5` | A14 resolves `imported` \| `locally-bound` \| `unresolved` by lexical import **and** binding collection (no type checker), firing **only** on `unresolved`. All three origins tested; origin 3 needs a synthetic fixture. | +| `R-6` | **PR-B owns the single transition.** PR-B introduces `discoverDoctrineRoots()` returning the final 36-unit selector and repoints `arch:check` at it in one step — **no** interim 17-root list, **no** checked-in root data file. PR-C consumes the same function unchanged for `arch:check:repo`. This retires the two-step that cycle 1 finding 9 and cycle 2 finding 4 both rejected; S-4 holds because the coverage predicate is preserved around one function. | +| `R-7` | `--max-allow` is wired at **8** (after PR-E removes two redundant allowances) and all 8 survivors reference **#1545**, the umbrella registration issue filed for this (owner decision). Without it, #1378's linked-issue rule reds the gate on day one — cycle 2 finding 2. | +| `R-8` | Findings surfaced by newly-covered scans are **triaged into issues, never fixed in the surfacing PR**. | +| `R-9` | Accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft. PR-C records this in `rfcs/README.md` and maps the five `DECISION_PENDING` entries onto it **without filing them**. | +| `R-10` | **#1374 owns the `docs/site/**` fenced-TS extractor; PR-D consumes it** and sequences after PR #1537 (owner decision; PR #1537 comment `5264583905`). **If that surface stays private:** slice D5 and #1378 box 3 move with the issue, **and PR-D references `#1378` without a closing keyword**, stating the remaining scope. It closes #1545 either way. Cycle 2 finding 9 — a fallback that moves a box while keeping `Closes` would auto-close an issue with undelivered acceptance. | +| `R-11` | The `labeled`-trigger defect (`drift.md` D-10) is corrected **in the documents** — `netscript-pr` and `check-close-gate.ts`'s repair hint say "label, then push" — with the `.claude/skills/` mirror regenerated. No workflow **trigger** changes. | +| `R-12` | #1378 box 6 is a property of a **diff**, which a file scanner cannot observe. It is proved by **one added step in the existing `code-quality` PR job** (owner decision): compare the `--max-allow` delta against issue links in the same diff and fail when the budget rose without one. A step in a job that already runs on PRs; no trigger and no skip-semantics change. Cycle 2 finding 3. | + +## Open decisions — none force rework + +| Decision | Status | +| --- | --- | +| `docs/site/**` extractor ownership | **RESOLVED** — `R-10`, with the closing-keyword consequence stated | +| The 567 `deno doc` warnings | **RESOLVED in shape** — `R-3`; PR-D measures the intersection before wiring, residue is dated debt | +| The 8 pre-existing allowances | **RESOLVED** — `R-7`, umbrella issue #1545 | +| Budget-link predicate mechanism | **RESOLVED** — `R-12` | +| `arch:check:repo` residue after R-4/R-5 | defer to PR-C — #1380 box 13 accepts "exit 0 **or** residue enumerated" | +| Successor records for `sagas`/`streams`/`hello-world` | defer to PR-C — the deliverable is to **state the absence**, not to find one | + +## Risk register + +| Risk | Mitigation | +| --- | --- | +| PR-D's budget wiring reds an in-flight PR from a sibling 0.0.6 lane | wired at the measured 8 with all survivors registered to #1545; PR body states the new behaviour; merge reported immediately | +| The A14 fix silences the true positive with the two sanctioned origins | origin 3 is a mandatory synthetic fixture that must stay **red**; all three origins in one test | +| PR-C invents rename provenance | the six-row reconciliation table above, both sources cited, absences stated | +| The `deno doc` intersecting set turns out large | PR-D measures **before** wiring and reports the number; if green is unreachable that is a rescope trigger in `drift.md`, not a silently weakened gate | +| #1537 does not land in time | `R-10`'s fallback: box 3 moves and PR-D drops the closing keyword — stated now, not discovered at merge | + +## Validation plan + +| Order | Gate | Command | Expected | | --- | --- | --- | --- | -| imported from `@std/testing/bdd` | **53** | yes | e.g. `packages/database/tests/migrate-retry_test.ts:10` | -| **locally bound** in the test file | **1** | yes | `packages/mcp/tests/service-endpoint-sources_test.ts:248` — `const describe = (workDir: string) => …`; the file's imports (`:1-6`) are `@std/assert` plus four local modules, nothing named `describe` | -| genuinely unresolved global | **0** | **no — the real signal** | none live; needs a synthetic fixture that must stay red | +| 1 | tool tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/quality/ .llm/tools/fitness/` | pass | +| 2 | repo quality scan | `deno task quality:scan:repo` | exit 0 **after PR-E**; `allowCount` 10 → 8 | +| 3 | default quality scan | `deno task quality:scan` | exit 0 | +| 4 | quality gate | `deno task quality:gate` | exit 0 | +| 5 | doctrine (curated) | `deno task arch:check` | exit 0 throughout | +| 6 | doctrine (repo) | `deno task arch:check:repo` | exit 1 / FAIL=55 before PR-C; after PR-C exit 0 or enumerated residue | +| 7 | scoped wrappers | `run-deno-{check,lint,fmt}.ts --root .llm/tools --ext ts` | pass | +| 8 | doctrine-document tests | PR-C's existence + coverage tests | fail on a fabricated row and on an ungated live unit | +| 9 | budget-link step | the added `code-quality` step, exercised on PR-D itself | red when the budget rises without a same-diff issue link | -`check-doctrine.ts:403-413` matches a bare identifier anywhere in a `*_test.ts` file and so cannot -distinguish any of the three. Treating all 54 as import false-positives would have produced a fix that -went quiet on the true positive too — the same defect, re-shipped inside its own repair. +`--allow-write --allow-run` is mandatory: nine tests under `.llm/tools` call `Deno.makeTempDir()` and one +spawns a subprocess (`drift.md` D-8). `deno task e2e:cli` is out of scope for the whole rail. -## Decisions revised, withdrawn, and added +## Acceptance-box routing — every box has a slice, files, and a proof -| ID | Change | -| --- | --- | -| `R-3` | **Caveat adopted (finding 7).** `deno doc --json` is fast enough — 3.733 s across all 30 package export maps, exit 0 — but that run emitted **567** `Warning Failed resolving types`. Exit 0 is therefore **not** a completeness proof. The export audit is **fail-closed**: an unresolved published declaration is a finding, unless it falls in a named, tested allowlist class. PR-D additionally carries a fixture proving a **re-exported** `any` is attributed to the published entrypoint. | -| `R-4` | Revised in place above — selector narrowed from "workspace members" to expanded top-level `packages/*` + `plugins/*`, with `packages/cli/e2e` explicitly excluded and the exclusion stated in the doctrine (finding 8). | -| `R-5` | Revised in place above — three origins, lexical import **and** binding collection, all three tested (finding 1). | -| ~~`R-6`~~ | **Withdrawn** (finding 9). Replaced by a single `discoverDoctrineRoots()` introduced in PR-B and expanded in PR-C. No transient list, one source of truth, S-4 honoured through the coverage predicate. | -| `R-7` | **Caveat adopted.** Ratcheting at the measured count does not prove #1378 box 6. A separate **fireable** control is added (slice D4): raising `--max-allow` **without** a same-PR issue link is red. | -| ~~`R-9`~~ | **Withdrawn** (finding 2). Replaced by `R-9b`. | -| `R-9b` | Accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft artifacts. `rfcs/0001-*.md`–`0005-*.md` exist at `01aa12b67`, all `status: Accepted`, and `rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` canonical. PR-C records this and maps the five `DECISION_PENDING` entries onto it **without filing them**. | -| `R-10` **(new)** | **The `docs/site/**` fenced-TS extractor is owned by #1374 and consumed by PR-D** (finding 6, resolved with owner authority). PR-D imports the extractor from draft PR **#1537** and sequences after it lands; it does **not** write a second parser. Coordination and the API request (stable per-snippet provenance) are posted at PR #1537 comment `5264583905`. **Fallback, stated now rather than discovered later:** if #1537's surface stays private to `docs:accuracy`, slice D5 and #1378 box 3 **move with the issue** — they are not forked and not ticked. | -| `R-11` **(new)** | The `labeled`-trigger defect (`drift.md` D-10) is fixed **in the documents, not the workflow** (owner decision). PR-C slice C7 corrects `.agents/skills/netscript-pr` and `check-close-gate.ts`'s repair hint to say "label, then push", regenerates the `.claude/skills/` mirror, and touches **no** workflow file. | - -## Open decisions — now none that force rework - -| Decision | Status | Resolution | -| --- | --- | --- | -| `docs/site/**` extractor ownership | **RESOLVED** | `R-10`. This was the finding-6 blocker; `plan-gate.md:20-22` is satisfied. | -| `arch:check:repo` residue after R-4/R-5 | safe to defer to PR-C | #1380 box 13 accepts "exit 0 **or** residue enumerated in `arch-debt.md`". PR-C measures and reports the real number. | -| `@netscript/shared`'s removing commit | safe to defer to PR-C | It existed (`0ef13de35`) so a removal is findable. If it cannot be found, the row records "removed, commit not identified" rather than a guess — the amended box 2 requires evidence, not a label. | - -## Acceptance-box routing — all 34 live boxes, none unrouted - -Denominator corrected to **34** (the brief said 33; #1380 carries a thirteenth `gate:` box). The six the -evaluator found unrouted are routed here to a named slice, file, and proof. Slice ids refer to the -`## Design` § Commit Slices table in `worklog.md`. - -| Issue / box | Route | Slice | Proof | +Slice ids refer to `worklog.md` § Design. + +| Boxes | Route | Slices | Proof | | --- | --- | --- | --- | -| #1403 · 1–5 | PR-B | B1–B3 | root added + `discoverDoctrineRoots()` coverage test; repaired gate run on `plugin-streams-core`; triage list with **no** `packages/**` source edit in the diff | +| #1530 · 1–6 | PR-E | E1–E4 | RED-first fixture committed red; dir+suffix rule; leakage controls both ways; `allowCount` 10 → 8; named gate pair | +| #1530 · 7 | PR-E, **`[post-merge]`** | — | live box carries the `[post-merge]` marker (issue line 92), which `netscript-pr` defines as exclusion-with-notice, verified by comment after merge; cycle 2 accepted this rebuttal | +| #1403 · 1–5 | PR-B | B1–B3 | `discoverDoctrineRoots()` + coverage test that fails when a publishable `plugin-*-core` leaves the selector; repaired gate run on `plugin-streams-core`; triage list with **no** `packages/**` source edit | | #1380 · 1, 3, 11, 12 | PR-C | C3, C4 | existence test fails on a fabricated row; coverage test fails on an ungated live unit; `06-archetypes.md` sync test | -| #1380 · 2 | PR-C | C3 | **amended box** (owner-authorized, issue comment `5264580324`): per-row git evidence, with `never present under that name` admitted. Five rows carry the never-present verdict; `@netscript/shared` carries its removing commit. | -| #1380 · 4, 6 | PR-C | C2 | `arch:check:repo` iterates `discoverDoctrineRoots()`; output no longer contains `.llm/tmp/`, `docs/`, `.llm/tools/` paths | +| #1380 · 2 | PR-C | C3 | the six-row reconciliation table, each row citing the git probe **and** the `arch-debt.md` record or its stated absence | +| #1380 · 4, 6 | PR-C | C2 | `arch:check:repo` consumes `discoverDoctrineRoots()`; output contains no `.llm/tmp/`, `docs/`, `.llm/tools/` path | | #1380 · 5 | PR-C | C1 | three-origin fixture: import quiet, local binding quiet, bare global **red** | -| #1380 · 7 | PR-C | C5 | `arch-debt.md` entry closed, or carries a dated closure plan naming both mechanical causes | -| **#1380 · 8** *(was unrouted)* | PR-C | C4 | `10-…md` gains a section naming which of the 36 units `arch:check` gates and why `packages/cli/e2e` is excluded; asserted by a test that fails if the gated set and `discoverDoctrineRoots()` disagree | -| **#1380 · 9** *(was unrouted)* | PR-C | C5 | `10-…md` gains a **dated** plan for engineering-reference §1–§5/§8–§10, authored as a byproduct of the refactors; test asserts the section exists and carries a date | -| **#1380 · 10** *(was unrouted)* | PR-C | C6 | `rfcs/README.md` records `rfcs/NNNN-*.md` as canonical per `R-9b`; all five `DECISION_PENDING` ids (`CRON-SUBSYSTEM-DUP`, `RUN-ARTIFACT-ARCHIVAL-POLICY`, `PAGEBUILDER-LEGACY-COMPAT-TREE`, `FORMPAGEPROPS-PLAYGROUND-MIGRATION`, `REDIS-LEGACY-VALUE-FALLBACK`) mapped onto it; test asserts all five are present | -| #1380 · 13 | PR-C | C2 | named gate pair: `arch:check` exit 0, `arch:check:repo` exit 0 or enumerated residue | -| #1378 · 1 | PR-D | D1 | exported `any` red, local `any` unchanged, plus re-export attribution fixture | -| #1378 · 2 | PR-D | D2 | unlinked `as unknown as` red; linked allowance green | -| #1378 · 3 | PR-D | D5 | `as any` in a `docs/site/**` fence red, via #1374's extractor. **Moves with the issue** if #1537's surface stays private (`R-10`). | -| #1378 · 4 | PR-D | D5 | the 6 `*-soundness_test.ts` files green with `@ts-expect-error` lines **unchanged** — regression evidence, explicitly not a negative case | -| #1378 · 5 | PR-D | D3 | `--max-allow` wired at the count measured in that PR; overflow red | -| **#1378 · 6** *(was unrouted)* | PR-D | D4 | a **fireable** same-PR control: raising the budget without an accompanying issue link in the same PR is red. Overflow alone does not prove this, which is why it is its own slice. | -| **#1378 · 7** *(was unrouted)* | PR-D | D6 | `docs/site/reference/triggers/index.md:310` and `docs/site/reference/triggers/examples_test.ts:65` both typed; both compile with no `any` | -| #1378 · 8 | PR-D | D1–D5 | full matrix: exported/local, linked/unlinked, fence, soundness, overflow | -| #1378 · 9 | PR-D | D1–D6 | `quality:scan:repo` + `arch:check` green (depends on PR-E and PR-C) | -| #1530 · 1–6 | PR-E | E1–E4 | RED-first fixture; dir+suffix rule; leakage controls both directions; `allowCount` 10 → 8; named gate pair | -| **#1530 · 7** | PR-E, **`[post-merge]`** | — | **Rebuttal to finding 4.** The live box already reads `` `gate:` the `code-quality-repo` job is green on `main` after merge. `[post-merge]` `` (issue #1530 line 92). `netscript-pr` makes `[post-merge]` the sanctioned mechanism: such a box is *visibly excluded from the merge gate with a notice* and verified by comment afterwards. It therefore needs no verification issue and does not keep #1530 open — the honesty rule is satisfied by the marker, which is what the marker is for. | - -## Protocol artifacts added (finding 5, blocking) - -`research.md` now exists, and `worklog.md` carries a `## Design` section with public surface, domain -vocabulary, ports, constants, **21 ordered file-scoped commit slices with a gate each**, deferred scope, -and the contributor path. Cycle 1 had neither; `plan-gate.md:16-34` requires both. - -## Finding 10 (advisory) — accepted - -The evaluator brief described its worktree as detached at `9c3cdfead`; ground truth was branch -`eval/quality-rail-plan-eval` at `83de0dc06`, the brief-only commit on top. The evaluator caught the -mismatch itself and reported it. Cycle 2's brief states the actual branch and sha, and the launcher's -`--expect-base` is set to the same value it verifies. +| #1380 · 7 | PR-C | C5 | `arch-debt.md` entry closed or dated, naming both mechanical causes | +| #1380 · 8 | PR-C | C4 | `10-…md` names which of the 36 units `arch:check` gates and why `packages/cli/e2e` is excluded; test fails if that set and `discoverDoctrineRoots()` disagree | +| #1380 · 9 | PR-C | C5 | `10-…md` carries a **dated** plan for engineering-reference §1–§5/§8–§10; test asserts the section exists and is dated | +| #1380 · 10 | PR-C | C6 | `rfcs/README.md` records `rfcs/NNNN-*.md` canonical per `R-9`; all five `DECISION_PENDING` ids mapped; test asserts all five present | +| #1380 · 13 | PR-C | C2 | named gate pair | +| #1378 · 1 | PR-D | D1 | exported `any` red, local `any` unchanged, re-export attribution fixture | +| #1378 · 2 | PR-D | D2 | unlinked `as unknown as` red; allowance linked to an open milestoned issue green | +| #1378 · 3 | PR-D | D5 | `as any` in a `docs/site/**` fence red via #1374's extractor; **moves with the issue** under `R-10`'s fallback, which then drops `Closes #1378` | +| #1378 · 4 | PR-D | D5 | the 6 `*-soundness_test.ts` files green with `@ts-expect-error` unchanged — regression evidence, explicitly not a negative case | +| #1378 · 5 | PR-D | D3 | `--max-allow` wired at 8; overflow red | +| #1378 · 6 | PR-D | D4 | the added `code-quality` step (`R-12`) is red when the budget rises without a same-diff issue link, exercised on PR-D itself | +| #1378 · 7 | PR-D | D6 | `docs/site/reference/triggers/index.md:310` and `examples_test.ts:65` typed; both compile with no `any` | +| #1378 · 8 | PR-D | D1–D5 | full matrix | +| #1378 · 9 | PR-D | D1–D6 | `quality:scan:repo` + `arch:check` green (depends on PR-E, PR-B, PR-C) | +| #1545 · 1–5 | PR-D | D2, D3 | all 8 allowances reference #1545; budget at 8; registered-vs-unregistered test; the five CLI sites recorded against the plugin-port-identity cause | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan.md b/.llm/runs/release-0.0.6-internals--orchestration/plan.md index d2f04c327f..7e5ca62281 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan.md @@ -79,6 +79,12 @@ Required deliverables: ### Wave 2 — quality rail +> **Superseded ordering note (revision 3).** The authoritative order is +> **PR-E (#1530) → PR-B (#1403) → PR-C (#1380) → PR-D (#1378 + #1545)**, defined once in +> `plan-quality-rail.md` § The one PR order. PR-E was inserted after this section was written, and +> `plan-eval-cycle2.md` finding 6 failed the plan for leaving the two statements in conflict. Where +> this section and `plan-quality-rail.md` differ, the rail plan governs. + Serialized under this lane because the three issues overlap on root lists, scan semantics, doctrine and architecture debt. One rail plan, **one** PLAN-EVAL over the whole rail, then three sequential PRs each with its own IMPL-EVAL. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index e61f73924b..513714338d 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -285,8 +285,13 @@ fully executed baseline table. Notable outputs of the re-baseline, all of which ## Design — quality rail (PR-E → PR-B → PR-C → PR-D) Written to close `plan-eval.md` finding 5: `plan-gate.md:16-34` requires ordered, file-scoped commit -slices with a gate per slice, and the first cycle offered four per-PR summary rows instead. Recorded -before any rail implementation file is created, per `run-loop.md` § 3b. +slices with a gate per slice, and cycle 1 offered four per-PR summary rows instead. Recorded before any +rail implementation file is created, per `run-loop.md` § 3b. + +**Aligned to `plan-quality-rail.md` revision 3.** The table below holds **20** slices — E1–E4, B1–B2, +C1–C7, D1–D6, plus B3 — and every row names the files it touches. Cycle 2 finding 5 was correct that a +previous revision claimed 21; the count is stated here so the claim and the table cannot diverge again. +B1/B2 now perform the **single** transition to discovery per `R-6`; there is no interim root list. ### Public Surface @@ -348,8 +353,8 @@ Ordered, file-scoped, one gate each. `deno test` roots take | E2 | PR-E | `isTypeFixture` exemption (dir + suffix) | `scan-code-quality.ts` | test green; `quality:scan:repo` exit 0 | | E3 | PR-E | leakage controls: ordinary source, and `_type.ts` outside the dir, stay red | `scan-code-quality_test.ts` | both negatives fail-on-removal | | E4 | PR-E | drop the two redundant allowances | `desktop-consumer_type.ts`, `sdk-assignability_type.ts` | `quality:scan:repo` `allowCount` 10 → 8 | -| B1 | PR-B | `discoverDoctrineRoots()` + coverage test asserting every publishable `plugin-*-core` is a root | `check-doctrine.ts`, `check-doctrine_test.ts` | test fails when a package is removed from the source | -| B2 | PR-B | `arch:check` consumes it; `plugin-streams-core` covered | `deno.json:156`, `check-doctrine.ts` | `arch:check` exit 0 | +| B1 | PR-B | `discoverDoctrineRoots()` returning the **final 36-unit selector** (expanded top-level `packages/*` + `plugins/*`, `packages/cli/e2e` excluded) + coverage test | `.llm/tools/fitness/check-doctrine.ts`, `.llm/tools/fitness/check-doctrine_test.ts` | coverage test fails when a publishable `plugin-*-core` leaves the selector | +| B2 | PR-B | `arch:check` repointed at `discoverDoctrineRoots()` in **one** step — no interim list, no data file | `deno.json:156`, `.llm/tools/fitness/check-doctrine.ts` | `deno task arch:check` exit 0 with `plugin-streams-core` covered | | B3 | PR-B | run the repaired gate on `plugin-streams-core`; triage output to new issues | triage list in slice dir only | no `packages/**` source edit in the diff | | C1 | PR-C | `resolveIdentifierOrigin()`; A14 fires only on `unresolved` | `check-doctrine.ts`, `check-doctrine_test.ts` | 3 fixtures: import (quiet), local binding (quiet), bare global (**red**) | | C2 | PR-C | `arch:check:repo` iterates `discoverDoctrineRoots()`; stops walking `.llm/tmp`, `docs/`, `.llm/tools` | `deno.json:157`, `check-doctrine.ts` | `arch:check:repo` exit 0 or residue enumerated | From 13e020a47169a56979503a282cd47f69eab12577 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:28:56 +0200 Subject: [PATCH 12/57] chore(harness): adopt the label-triggered evaluation policy for the rail PRs Formal evaluation is no longer an orchestrator dispatch. PLAN-EVAL fires from the openhands + status:plan-eval pair; the initial IMPL-EVAL fires on draft -> ready unless impl-eval:skip; either reruns only by moving away from its status label and re-adding it. Consequences recorded for PR-E/B/C/D: no local IMPL-EVAL session is launched, no impl-eval:skip is applied to a PR that changes gate semantics, and the draft -> ready flip stops being a free way to materialise CI contexts because it is now an evaluator trigger. No violation to unwind -- this lane never dispatched OpenHands. The cycle-3 rail PLAN-EVAL was already running locally when the directive arrived, so it is left to finish and deliberately not duplicated with a label pair. Refs #1403, #1380, #1378, #1530, #1545 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 35 +++++++++++++++++++ .../slices/pr-e-1530/implement.md | 4 +++ .../supervisor.md | 13 +++++-- 3 files changed, 49 insertions(+), 3 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 78ff274dd3..3d3808541b 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -332,3 +332,38 @@ dispatching it on the orchestrator's own authority, against a formal `FAIL_PLAN` "do not begin implementation on the current plan", would be the generator overruling its evaluator. That is the self-certification the harness forbids, so it goes to the owner as a decision rather than being taken as a judgement call. + +## D-14 — evaluation automation policy change: label/transition-triggered, never manually dispatched + +- **Severity:** significant (changes this lane's evaluator mechanics for every remaining PR) +- **Recorded:** 2026-08-12, owner directive mid-run +- **Policy, as stated:** + - **Do not manually dispatch OpenHands** for formal PLAN-EVAL or IMPL-EVAL. + - **PLAN-EVAL** fires exactly once from the **`openhands` + `status:plan-eval` label pair**. Rerun only + by moving away from `status:plan-eval` and re-adding it. + - **Initial IMPL-EVAL fires automatically on draft → ready**, unless `impl-eval:skip` is applied. Rerun + only by moving away from `status:impl-eval` and re-adding it. + - Optional `eval:model:minimax|deepseek|qwen` is **one-shot**. + - An already-running local eval may finish; **never duplicate it**. +- **Compliance position for this lane:** + - **No violation to unwind.** This lane never dispatched OpenHands: `drift.md` D-2 recorded on day one + that #1524 was an open draft and `lane-policy.md` had the cloud lane paused, so formal evaluation ran + on the documented native opposite-family route. + - **Cycle-3 PLAN-EVAL is left to finish.** It is a local Codex · Sol · high session (thread + `019ff508-…`) launched minutes before this directive. Per the policy it may complete, and it must + **not** be duplicated — so this lane will **not** also apply the `openhands` + `status:plan-eval` + pair for the same plan. One evaluation of one plan revision. +- **What changes for PR-E / PR-B / PR-C / PR-D:** the per-PR IMPL-EVAL is **no longer an orchestrator + dispatch**. It fires on the draft → ready transition. Consequences the orchestrator must honour: + 1. **Do not launch a local IMPL-EVAL session per PR.** That would duplicate the automatic one. + 2. **Do not apply `impl-eval:skip`** on any rail PR — every one of them changes gate semantics, which + is exactly the class that needs an independent pass. + 3. **The draft → ready flip becomes an evaluator trigger, not just a CI trigger.** So it happens when + the slice checklist is complete and IMPL-EVAL is expected to pass — not to get CI moving. On PR #1527 + the flip was used to materialise required contexts; that is no longer a free action. + 4. **Rerun discipline:** if an IMPL-EVAL verdict must be refreshed after fixes, move away from + `status:impl-eval` and re-add it. Do not push an empty commit and do not re-request a review. + 5. `status:plan-eval` is not applied to any rail PR: the rail's plan was evaluated at the run level, + and the PRs carry `status:impl` → `status:impl-eval` → `status:ready-merge`. +- **Unchanged:** merge authority stays with the orchestrator through the pre-merge gate; a green automated + gate is still not a sign-off; the generator still never evaluates its own work. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md index de5d186dd7..0bcaf2af81 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md @@ -155,6 +155,10 @@ comment so the cheap lane is visibly intentional. Never edit a workflow file to 5. Slices, per the plan's Design table: **E1** RED fixture (committed red) · **E2** the `isTypeFixture` exemption · **E3** leakage controls both directions · **E4** drop the two redundant allowances and record the 10 → 8 count. +6. **Leave the PR in draft.** The draft → ready flip now **automatically triggers the formal IMPL-EVAL** + (owner policy, `drift.md` D-14), so it is the orchestrator's action once your slice checklist is + complete — not a way to get CI moving. Do **not** flip it yourself, do **not** apply `impl-eval:skip`, + and do **not** apply `status:impl-eval` or `status:plan-eval`. 6. Post one `**[PHASE: IMPL]**` summary when the slices are in, listing every gate with real output. ## Two mechanical rules that cost this lane time already diff --git a/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md b/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md index d41be287a6..d8dc8e84e2 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md @@ -52,9 +52,16 @@ canonical lane is bound to which role in this run. | PR | Issues | PLAN-EVAL | IMPL-EVAL | | --- | --- | --- | --- | | PR-A | #1436, #1415 | **N/A — owner waiver** (mechanical; complete contract, scope, acceptance and gates already in the issue bodies). Substituted by executed negative-case proof. | **N/A — owner waiver** (brief §3). Substituted by orchestrator slice review + RED→GREEN negative-case evidence on both parsers. | -| PR-B | #1403 | Covered by the single rail PLAN-EVAL. | Required — fresh opposite-family session. | -| PR-C | #1380 | Covered by the single rail PLAN-EVAL. | Required — fresh opposite-family session. | -| PR-D | #1378 | Covered by the single rail PLAN-EVAL. | Required — fresh opposite-family session. | +| PR-E | #1530 | Covered by the rail PLAN-EVAL. | **Automatic on draft → ready** (owner policy, `drift.md` D-14). Not dispatched by the orchestrator; `impl-eval:skip` must not be applied. | +| PR-B | #1403 | Covered by the rail PLAN-EVAL. | Automatic on draft → ready — as above. | +| PR-C | #1380 | Covered by the rail PLAN-EVAL. | Automatic on draft → ready — as above. | +| PR-D | #1378, #1545 | Covered by the rail PLAN-EVAL. | Automatic on draft → ready — as above. | + +**Evaluator mechanics changed mid-run (2026-08-12, owner).** Formal evaluation is label/transition +triggered, never manually dispatched: PLAN-EVAL from the `openhands` + `status:plan-eval` pair, IMPL-EVAL +from draft → ready. Rerun either only by moving away from its `status:` label and re-adding it. The rail's +PLAN-EVAL cycles 1–3 ran on the native opposite-family route before this directive and are left to +complete rather than duplicated. Full terms in `drift.md` D-14. One rail PLAN-EVAL covers PR-B/C/D because the three issues overlap on root lists, scan semantics, doctrine and architecture debt; evaluating their sequencing separately would evaluate the wrong From c1ebfa5d9ba3d2c879f81f5c2ac763c8ced940a9 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:40:12 +0200 Subject: [PATCH 13/57] chore(harness): refine R-11 after #1524 landed and made the labeled claim half true #1524 added openhands-phase-eval.yml, which does listen to labeled, while ci.yml still does not -- and ci.yml is what runs close-gate and the acceptance mirror. So netscript-pr's claim that applying status:ready-merge triggers a fresh run is now half true rather than false, which is harder to catch: a label does produce a run, just not the one that re-evaluates the close-gate. R-11 becomes a distinction instead of a negation. Also supersedes D-2's factual claim that #1524 was an open draft, and records that PR-E's surface and premise are unaffected by the two new main commits. Refs #1380, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 30 +++++++++++++++++++ .../plan-quality-rail.md | 2 +- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 3d3808541b..5e0b297349 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -367,3 +367,33 @@ taken as a judgement call. and the PRs carry `status:impl` → `status:impl-eval` → `status:ready-merge`. - **Unchanged:** merge authority stays with the orchestrator through the pre-merge gate; a green automated gate is still not a sign-off; the generator still never evaluates its own work. + +## D-15 — #1524 landed mid-run, which makes the `labeled` documentation defect worse, not better + +- **Severity:** significant (sharpens R-11 and supersedes part of D-2) +- **Recorded:** 2026-08-12, while cycle 3 was running +- **Fact:** `main` advanced two commits past PR-E's base to `281ab7688`, including + **`7837ef470 feat(agentic): automate formal evaluator phases (#1524)`** — the automation the owner's + policy directive (D-14) describes. **D-2 is now superseded on its factual claim:** #1524 is no longer an + open draft; the label-triggered evaluator route exists. +- **What it added:** `.github/workflows/openhands-phase-eval.yml`, whose trigger is + `on: pull_request: types: [labeled, ready_for_review]`, with the state machine documented in its own + header: initial IMPL-EVAL on draft→ready, `impl-eval:skip` as the attributed escape hatch, PLAN-EVAL on + the `openhands` + `status:plan-eval` pair, rerun by label-cycling. +- **What it did *not* change:** `ci.yml:41` is still + `types: [opened, synchronize, reopened, ready_for_review]`. **`labeled` is still absent**, and `ci.yml` + is the workflow that runs `close-gate` and the acceptance mirror. +- **Why this makes the documentation defect worse.** `netscript-pr` `SKILL.md:169-170` still reads + "applying `status:ready-merge` itself triggers a fresh run (the workflow listens to `labeled`)". Before + #1524 that sentence was simply false. Now it is **half true**, which is harder to catch: a label *does* + now trigger a workflow — just not the one that re-evaluates the close-gate. An operator who reads it + will apply `status:ready-merge`, see a run appear, and conclude the close-gate was re-evaluated. It was + not. +- **Action:** `R-11` refined from a negation into a distinction — the phase-eval workflow listens to + `labeled`; `ci.yml` does not; for `status:ready-merge` it is label **then push**. PR-C slice C7 carries + it, along with the same fix to `check-close-gate.ts`'s repair hint. No workflow trigger is changed + (owner decision, D-14 boundary). +- **Collision check for PR-E:** the two new commits touch skills, labels, and workflows only — + `git diff --name-only 84dd44ae7..origin/main | grep -E 'tools/quality|type-fixtures'` returns nothing. + PR-E's surface is unaffected and its premise still holds: `quality:scan:repo` is **still exit 1** at + `281ab7688`, so #1530 is live and its RED-first proof remains real. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index 561dd32df0..83a29e825b 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -104,7 +104,7 @@ Revision 1 inherited that claim without re-measuring and was failed on it (`drif | `R-8` | Findings surfaced by newly-covered scans are **triaged into issues, never fixed in the surfacing PR**. | | `R-9` | Accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft. PR-C records this in `rfcs/README.md` and maps the five `DECISION_PENDING` entries onto it **without filing them**. | | `R-10` | **#1374 owns the `docs/site/**` fenced-TS extractor; PR-D consumes it** and sequences after PR #1537 (owner decision; PR #1537 comment `5264583905`). **If that surface stays private:** slice D5 and #1378 box 3 move with the issue, **and PR-D references `#1378` without a closing keyword**, stating the remaining scope. It closes #1545 either way. Cycle 2 finding 9 — a fallback that moves a box while keeping `Closes` would auto-close an issue with undelivered acceptance. | -| `R-11` | The `labeled`-trigger defect (`drift.md` D-10) is corrected **in the documents** — `netscript-pr` and `check-close-gate.ts`'s repair hint say "label, then push" — with the `.claude/skills/` mirror regenerated. No workflow **trigger** changes. | +| `R-11` | **[refined after #1524 landed]** The `labeled`-trigger defect (`drift.md` D-10, D-15) is corrected **in the documents**, and the correction is now a *distinction* rather than a flat negation. `netscript-pr` (`SKILL.md:169-170`) says "applying `status:ready-merge` itself triggers a fresh run (the workflow listens to `labeled`)". Post-#1524 that is half true and therefore worse than before: **`openhands-phase-eval.yml` does listen to `labeled`**, but **`ci.yml:41` still does not** — and `ci.yml` is the workflow that runs `close-gate` and the acceptance mirror. So the label fires phase evaluation and does **not** re-run the close-gate. PR-C slice C7 states exactly that, plus "for `status:ready-merge`, label then **push**", and applies the same fix to `check-close-gate.ts`'s repair hint. `.claude/skills/` mirror regenerated. **No workflow trigger is changed.** | | `R-12` | #1378 box 6 is a property of a **diff**, which a file scanner cannot observe. It is proved by **one added step in the existing `code-quality` PR job** (owner decision): compare the `--max-allow` delta against issue links in the same diff and fail when the budget rose without one. A step in a job that already runs on PRs; no trigger and no skip-semantics change. Cycle 2 finding 3. | ## Open decisions — none force rework From 25e5758b593469c220e3f5c61adee0f0822a6961 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:48:05 +0200 Subject: [PATCH 14/57] chore(harness): record cycle-3 FAIL_PLAN and escalate #1378 as a scope decision Cycle 3 closed 6 of 10 cycle-2 findings with none unaddressed, so the plan is converging. What is not converging is #1378's premise. The evaluator refused R-3's deferral and measured it: 1,714 published symbol records contain unresolved type references across 3,945 occurrences, including 230/724 symbols in plugin-sagas-core. Warning text names the dependency module, not the published declaration, so warnings-that-touch-a-declaration was never a defined mapping. Export-reachability at this baseline needs a debt programme, not a slice. Second unimplementable premise: the scanner has only --allow-read, so it cannot verify that an allowance's issue is open and milestoned, and the planned proof would be satisfied by a parser accepting any issue number -- a test that cannot fail on the property the issue requires. Two factual corrections are mine: the sagas row was false (arch-debt.md:576-584 records the supersession) and the shared row omitted fd8259b76, which deletes packages/shared. Both are the same class I was auditing #1380 for -- asserting an absence without running the probe that finds the presence. Refs #1403, #1380, #1378, #1530, #1545 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 66 ++++ .../plan-eval-cycle3.md | 304 ++++++++++++++++++ 2 files changed, 370 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle3.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 5e0b297349..c185587b01 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -397,3 +397,69 @@ taken as a judgement call. `git diff --name-only 84dd44ae7..origin/main | grep -E 'tools/quality|type-fixtures'` returns nothing. PR-E's surface is unaffected and its premise still holds: `quality:scan:repo` is **still exit 1** at `281ab7688`, so #1530 is live and its RED-first proof remains real. + +## D-16 — cycle 3 FAIL_PLAN: the plan is converging, #1378 is not implementable as scoped + +- **Severity:** architectural (milestone-content decision, not a plan defect) +- **Recorded:** 2026-08-12, eval cycle **3 of 3** +- **Verdict:** `FAIL_PLAN`. Cycle-2 disposition: **6 addressed, 4 partially addressed, 0 not addressed**. + Five new blocking findings. The plan is measurably converging; what is not converging is #1378's + premise. + +### The finding that changes the milestone, not the plan + +`R-3` deferred "measure the intersection, then wire or rescope" into PR-D. The evaluator refused that as +a plan-time deferral (`plan-gate.md:24-27`) **and measured it**: + +```text +30-package deno doc --json run: 567 warnings reproduced +1,714 published symbol records contain unresolved type references (3,945 occurrences) + plugin-sagas-core: 230/724 symbols affected + fresh: 174/487 symbols affected +``` + +It also identified why my framing was unusable: **warning text names the dependency module, not the +published declaration that transitively depends on it**, so "warnings that touch a declaration" was never +a defined mapping. The intersection is plausibly *most* declarations in dependency-heavy packages. + +That is not a plan defect I can rewrite away. #1378 box 1 ("a new `any` in an exported type fails") rests +on export-reachability that cannot be computed reliably at this baseline without a debt baseline for +1,714 symbol records — which is a programme, not a slice. + +### The second unimplementable premise + +`#1378` requires `// quality-allow:` to carry an issue id that is **open and milestoned**. The scanner +runs with `--allow-read` only (`deno.json:50-51`) — it cannot observe live issue state. My D2 proof +(unlinked-red / linked-green) would be satisfied by a parser that accepts any `#`, i.e. **a test that +cannot fail on the property the issue actually requires**. Satisfying it needs a deterministic state +source (checked-in register, or a networked step outside the scanner) that #1378 never specifies. + +### Two factual corrections the evaluator found, both mine + +1. **`@netscript/sagas` — my row was false.** I wrote "no checked-in supersession record found". + `arch-debt.md:576-584` records the resolved debt in `packages/plugin-sagas-core` and explicitly calls + the old directory superseded. +2. **`@netscript/shared` — my row was incomplete.** I wrote "no removal commit on `main`", which is true + but omits load-bearing evidence: the mandated full-history probe finds **`fd8259b76`**, whose diff + deletes `packages/shared/deno.json` and the rest of `packages/shared/**`. Non-HEAD history, but it is + the removal commit the row must cite alongside the ancestry boundary. + +Both are exactly the class I was auditing #1380 for — asserting an absence without running the probe that +would find the presence. Recorded rather than quietly fixed. + +3. **My 374-commit figure is unpinned** and therefore false at any later head (finding 7). A count that + moves must be stated with the sha it was measured at. + +### Also found: E1's gate is structurally invalid + +`worklog.md:352` gives E1 the post-slice gate "`deno test .llm/tools/quality/` **fails**". A landed commit +slice whose required gate is red cannot satisfy the Plan-Gate's "gate that proves it" rule. RED-first +evidence belongs in the PR comment and the test's pre-change output — not as a slice's passing condition. + +### Action + +Three cycles is the escalation point twice over. The plan is not rewritten a fourth time. Escalated to the +owner as a **milestone-content decision**: #1530, #1403 and #1380 remain deliverable with small, +identified fixes; #1378's boxes 1 and 2 rest on properties this codebase cannot satisfy in 0.0.6 without a +separate debt programme, and the honesty rule says such criteria move with their issue rather than being +ticked or quietly reinterpreted. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle3.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle3.md new file mode 100644 index 0000000000..1bdcbc9a70 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle3.md @@ -0,0 +1,304 @@ +FAIL_PLAN + +## Identity + +| Field | Value | +| ------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| Provider / model / effort | OpenAI · GPT-5.6 Sol · high | +| Role | Formal PLAN-EVAL cycle 3 of a Claude-authored plan; owner-authorized after the two-cycle limit; no implementation performed | +| Worktree | `/home/codex/repos/ns006-raileval` | +| Branch | `eval/quality-rail-plan-eval` — confirmed | +| Evaluated checkout | `8386f2a4d90b0c210d325723e05f2f597af20308` — confirmed | +| Revision commit | `83adb22871dfc99216ef64caee93f32a1bb930b5` — confirmed as HEAD's second parent and an ancestor of HEAD | +| Other HEAD parent | `801b3c19c0a3629f53853ddd095ed2a7942c40bf` | +| Product/tooling baseline | `84dd44ae7`; the rail revisions after it change run artifacts only, so the measured scanner/checker tree is unchanged | +| Date | 2026-08-12 | + +The checkout identity in the cycle-3 brief matches raw Git ground truth. The worktree was clean +before this verdict file was created. + +## Re-measured baseline + +| Claim | Revision-3 claim | Independent result | Result / command | +| ------------------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Checkout | branch `eval/quality-rail-plan-eval`, HEAD `8386f2a4d`, merging `83adb2287` | exact branch/full SHA; parents `801b3c19c… 83adb2287…`; revision is an ancestor | **confirmed** — raw Git through `Deno.Command` and `git merge-base --is-ancestor` | +| `quality:scan` | exit 0, 0 findings, `allowCount: 7` | exit 0, 0 findings, 7 allowances | **confirmed** — `deno task quality:scan` | +| `quality:scan:repo` | exit 1, 5 findings, `allowCount: 10` | exit 1, five `ts-error-suppression` findings, 10 allowances | **confirmed** — `deno task quality:scan:repo` | +| `arch:check` | exit 0, 16 hand-listed roots | exit 0; all 16 invocations report `FAIL=0` | **confirmed** — `deno task arch:check` | +| `arch:check:repo` | exit 1, `FAIL=55` = 54 A14 + 1 A1 | exit 1, `FAIL=55`; 54 A14 paths plus root `mod.ts` A1 | **confirmed** — `deno task arch:check:repo` plus source reconciliation | +| A14 origins | 53 BDD imports + 1 local helper + 0 unresolved globals | 53 + 1 + 0; the only non-BDD path is `packages/mcp/tests/service-endpoint-sources_test.ts` | **confirmed** — parsed all 54 matching test files | +| Local helper | `describe` at `service-endpoint-sources_test.ts:248` | definition at line 248; imports at lines 1–10 contain no BDD binding; calls at 264/268 | **confirmed** — numbered source read | +| Live top-level units | 30 packages + 6 plugins = 36 | 30 + 6 = 36 | **confirmed** — `Deno.readDir` plus member manifests | +| Verdict-table drift | 28 rows; 6 non-live; 14 live missing | same 28 / 6 / 14 and same named sets | **confirmed** — parsed doctrine table against live manifest identities | +| Soundness fixtures | 6 `*-soundness_test.ts` | 6 | **confirmed** — recursive inventory | +| Type fixtures | 12 `*_type.ts`, all under `tests/type-fixtures`, 3 containing `@ts-expect-error` | 12 / all / 3 | **confirmed** — recursive inventory and content read | +| Surviving allowances after PR-E | 8, all to reference #1545 | scanner reports 10 now; removing the two named fixture allowances leaves 8; 0 currently carry an issue id | **confirmed** — `scanCodeQualityDetailed` reconciliation | +| `deno doc --json` feasibility | 30 package export maps in 3.733 s; 567 warnings; no non-zero exit | 3.333 s; 567 warnings; 0 non-zero packages. SDK: 12 entrypoints, 0.123 s, 0 warnings. Fresh: 15 entrypoints, 0.357 s, 87 warnings | **confirmed** — fresh timed runs | +| Warning reachability population | R-3 says the warning/declaration intersection will be measured in PR-D | the JSON for six affected packages contains 6,544 symbols overall; 1,714 symbol records carry at least one unresolved type reference and 3,945 unresolved type-reference occurrences. Package `ai` alone has 62 affected published symbols | **new load-bearing measurement** — parsed every `deno doc --json` result | +| Current-main history root | `317e4b509`, 2026-07-06 | HEAD's sole root is `317e4b509`, date/title confirmed; its `packages/` tree already contains all six `plugin-*-core` directories | **confirmed** — `git rev-list --max-parents=0 HEAD`, `git show`, `git ls-tree` | +| History commit count | 374 | the owner-correction baseline `64c091c5` has 374 commits; revision commit has 375 and evaluated HEAD has 380 | **diverges as an unqualified current baseline** — `git rev-list --count`; the 374 claim is valid only when pinned to `64c091c5`, not at evaluated HEAD | +| Full `--all` roots | narrative says “this repository's history” begins at `317e4b509` | `git rev-list --max-parents=0 --all` returns three roots: `317e4b509`, `0ef13de35`, `89ea1f4cee`; the latter two are non-HEAD histories | **diverges if read as all refs; confirmed for HEAD ancestry** | +| Stale-path history | triggers/workers/sagas/streams and hello-world have zero commits; shared has 10 commits on non-ancestor refs | zero for the five named paths; shared has 10 commits, all non-ancestors of HEAD | **confirmed as path counts** — `git log --all` and ancestry probes | +| Shared deletion | Revision 3 says no removal commit on `main` and routes that fact | full-history commit `fd8259b76` deletes `packages/shared/**`; it is on a non-HEAD history | **diverges from the routed record** — `git show --name-status fd8259b76 -- packages/shared/**` | +| Sagas supersession record | Revision 3 says no checked-in supersession record was found | `arch-debt.md:576-584` says the code/debt live in `plugin-sagas-core` and calls the old top-level directory superseded | **diverges** — numbered source read | +| Accepted RFCs | five numbered accepted RFCs, 0001–0005 | five; all declare `status: Accepted` | **confirmed** — `rfcs/` inventory/status read | +| Live acceptance denominator | 39 boxes across #1403/#1380/#1378/#1530/#1545 | 5 + 13 + 9 + 7 + 5 = 39; all five issues are open and milestoned 0.0.6 | **confirmed** — live `gh issue view` extraction | +| #1530 post-merge marker | box 7 carries `[post-merge]` | present on live issue line 92 | **confirmed** — live issue body | +| Design slice count | 20 | 20: E1–E4, B1–B3, C1–C7, D1–D6 | **confirmed** — mechanical table parse | +| #1374 extractor dependency | #1537 exports stable provenance | draft PR #1537 is in implementation; checked-out `snippet-extractor.ts:8-22,59` exports `FencedBlock` and `extractFencedBlocks`, including source path, ordinal, and line provenance | **confirmed** — read-only inspection of `/home/codex/repos/ns006-1374-compilegate` | + +## Findings + +1. **blocking — R-3 still defers a rework-forcing decision to PR-D, and the current JSON indicates + the intersecting population can be large.** `plan-quality-rail.md:99,115,128` says PR-D will + first measure the intersection, then either wire the rule or rescope if green is unreachable. + That is an explicit plan-time decision deferred into implementation, contrary to + `.llm/harness/gates/plan-gate.md:24-27`. The fresh 30-package run reproduced 567 warnings and + found **1,714 published symbol records containing unresolved type references** (3,945 unresolved + occurrences); affected packages include 230/724 symbols in `plugin-sagas-core` and 174/487 in + `fresh`. Warning text identifies dependency modules, not the published declaration that + transitively depends on them, so “warnings that touch a declaration” is not yet a defined + deterministic mapping. The intersection could plausibly be most declarations in those + dependency-heavy packages. Required change: define and execute the exact warning-to-declaration + attribution now, record the resulting cardinality and classes, and lock a reachable final + behavior. If the population requires a debt baseline, name that baseline and its failure + predicate in D1; if not, make the entrypoint/re-export graph primary. A PR-D-time rescope trigger + is not a locked decision. + +2. **blocking — R-7/D2 has no deterministic mechanism for proving that `#1545` is open and + milestoned.** The live #1378 target contract at lines 71–73 and #1545 box 3 at lines 53–54 + require live issue state. `worklog.md:366-368` names only `scan-code-quality.ts + test`, and + `deno.json:50-51` grants the scanner only `--allow-read`; no network permission, injected + issue-state manifest, checked-in register, resolver boundary, or closed/unmilestoned negative is + specified. The plan's D2 proof is only unlinked-red / open-milestoned-green, so a parser that + accepts any `#` can satisfy its stated test while violating the live contract. Required + change: choose a deterministic state source and its permissions/inputs, name exact files and a + verdict command, and include missing, closed, unmilestoned, and open-milestoned controls. The + live #1545 issue itself is open and milestoned; the blocker is the planned predicate, not the + umbrella decision. + +3. **blocking — the six-row provenance table still contains one false and one incomplete per-row + record.** Revision 3 routes #1380 box 2 through `plan-quality-rail.md:74-80,158`. For + `@netscript/sagas`, line 78 says there is no checked-in supersession record, while + `.llm/harness/debt/arch-debt.md:576-584` records the code and resolved debt in + `packages/plugin-sagas-core` and explicitly calls the old directory superseded. For + `@netscript/shared`, line 79 routes only the true but incomplete statement “no removal commit on + main”; the mandated full-history probe finds `fd8259b76`, whose diff deletes + `packages/shared/deno.json` and the rest of `packages/shared/**`. It is a non-HEAD history, but + it is still the removal commit the row must cite alongside the ancestry boundary. The sagas + baseline is wrong and the shared proof omits load-bearing evidence. Required change: record sagas + as having a checked-in conceptual-supersession record and shared as added at `0ef13de35`/deleted + at `fd8259b76` on a non-HEAD history, with the non-ancestor qualifier. Keep the truthful + distinction between HEAD's truncated history and all refs. + +4. **blocking — the Design table still fails the commit-slice contract.** The count is now correct, + but `worklog.md:352` makes E1's post-slice gate “`deno test .llm/tools/quality/` fails”; a landed + commit slice whose required gate is red cannot pass the Plan-Gate's “gate that proves it” rule. + Rows 353–371 also retain non-file-scoped names (`scan-code-quality.ts`, `check-doctrine.ts`, + `10-…md`, `new check + test`, `triage list in slice dir only`) and D4 still names neither the new + checker path nor its test/workflow path. This is the same underlying cycle-2 finding 5, not a new + formatting preference. Required change: combine E1's recorded pre-change RED evidence with E2's + green landed slice, and replace every abbreviated or prospective Files cell with exact + repository-relative paths, including D4's checker, test, and + `.github/workflows/code-quality.yml`. Each row must have one post-slice command whose + zero/non-zero expectation distinguishes pass from did-not-run. + +5. **blocking — PR-B's negative coverage proof is not independent of the selector it tests.** B1 at + `worklog.md:356` says `discoverDoctrineRoots()` returns all top-level package/plugin units, but + its gate says the coverage test fails “when a publishable `plugin-*-core` leaves the selector.” + If both expected and actual sets come from the same discovery function, removing a directory from + its output makes both sets shrink and the control does not fire. Neither the plan nor Design + names an independent inventory oracle (for example, top-level manifests whose package names match + `@netscript/plugin-*-core`) or a mutation fixture. Required change: specify the independent + expected set and a test that mutates only actual selection; prove the test fails before restoring + the member. The one-step R-6 transition itself is sound, but this stated #1403 box-4 proof can + still be tautological. + +6. **should-fix — the one-order correction is authoritative but the surrounding run artifacts retain + contradictory operational prose.** `plan.md:82-86` explicitly delegates order to Revision 3, + which resolves execution order. However, `plan.md:88-90` still says three issues/three PRs, + `plan.md:109` says PR-D closes only #1378, `worklog.md:330-332` still says the selector is + expanded in PR-C, and `cut-trace.md:30,53` still says the acceptance amendment was strictly + harder and that five rows never existed without the truncation qualification. Required change: + reconcile these resume artifacts to four PRs/five rail issues, PR-D's #1378+#1545 contract, + one-step R-6, and the corrected amendment/history language. The explicit authority note prevents + this alone from changing the verdict, but stale resume instructions should not be handed to + implementers. + +7. **should-fix — the 374-commit history figure is unpinned and therefore false at the evaluated + checkout.** `plan-quality-rail.md:48,69-70` presents 374 as the repository baseline. The probe in + owner comment `5264832009` was taken at `64c091c5`, which does have 374 commits; revision commit + `83adb2287` has 375 and evaluated HEAD has 380. Required change: write “HEAD ancestry at + `64c091c5` contained 374 commits” or drop the count. The root/tree evidence, not the moving + count, proves truncation. + +## Cycle-2 finding disposition + +| Cycle-2 finding | Disposition | Evidence checked | +| -------------------------------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1. R-3 made final green unreachable | **partially addressed** | R-3 no longer fails on all 567 warnings, but it defers attribution/cardinality and a possible rescope to PR-D. Fresh JSON parsing found 1,714 affected published symbol records, so the plan-time uncertainty is material (finding 1). | +| 2. Eight allowances had no migration | **partially addressed** | #1545 exists, is open/milestoned, and all eight survivors are routed to it. The open/milestoned state-check mechanism and required negative controls remain undefined (finding 2). | +| 3. #1378 box 6 had no executable hook | **addressed** | Owner-locked R-12 places the diff predicate in a new step of the existing PR `code-quality` job, and routing names missing-link RED / linked behavior. Exact file names remain a Design defect, but the execution hook is specified. | +| 4. R-6 retained an interim selector | **addressed** | R-6/B1/B2 make PR-B the single transition directly to the final 36-unit selector; PR-C consumes it unchanged. `worklog.md:330-332` is stale prose, not the operative slice. | +| 5. Design count/files/gates invalid | **partially addressed** | Count is correctly 20. E1 still requires a failing landed gate and multiple Files cells remain abbreviated or prospective, including D4 (finding 4). | +| 6. Wave order contradictory | **addressed** | Revision 3 gives one total order E→B→C→D and `plan.md:82-86` explicitly makes it authoritative. Old “three PRs” prose remains should-fix but no longer controls order. | +| 7. Provenance omitted doctrine records | **partially addressed** | Triggers/workers are reconciled against truncated HEAD history. Sagas' record and shared's non-ancestor delete commit are still misstated (finding 3). | +| 8. “Strictly harder” overstatement | **addressed** | Owner comment `5264832009` and Revision 3 both say stricter evidence plus broader admissible states. The correction is adequate and does not oversell the amendment. | +| 9. R-10 fallback could auto-close incomplete #1378 | **addressed** | R-10 explicitly drops the closing keyword, states remaining scope, and leaves box 3 with the issue. #1537 now exports stable provenance, so the primary dependency is also concrete. | +| 10. Appended revisions contradicted earlier prose | **addressed** | Revision 3 is a single consolidated document. Remaining contradictions are in companion artifacts, not superseded sections inside the rail plan. | + +## Decision review + +| Decision | Review | Reason | +| -------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| R-1 | **sound** | PR-E restores the currently red repository scan and narrows fixture scope before PR-D relies on a green final gate. | +| R-2 | **sound** | Directory + suffix and both leakage controls exactly match the 12-file measured surface. | +| R-3 | **wrong as operationalized** | Tool runtime is acceptable, but warning attribution/cardinality and the green-vs-rescope choice are deferred into PR-D. The measured unresolved population is large enough to force rework. | +| R-4 | **sound** | Expanded top-level package/plugin members match the 36-unit acceptance denominator and exclude nested `packages/cli/e2e`. | +| R-5 | **sound with caveat** | Three-origin lexical resolution is feasible for the measured import/local/unresolved cases. The implementation must remain scope-aware and test aliases/shadowing so an unrelated binding cannot suppress a true global. | +| R-6 | **sound** | A single PR-B transition to final discovery removes the interim-list churn and lets S-4 preserve one coverage predicate. The independent expected-set defect belongs to B1's proof, not to the transition decision. | +| R-7 | **sound with caveat** | One owner-authorized umbrella issue is a valid registration target and #1545 is live/milestoned. It remains implementable only after the deterministic issue-state boundary in finding 2 is specified. | +| R-8 | **sound** | Surfaced product findings are triaged instead of expanding the surfacing PR. | +| R-9 | **sound** | Five accepted numbered RFCs establish `rfcs/NNNN-*.md` as the accepted-record path; harness bundles are provenance/draft inputs. | +| R-10 | **sound** | One extractor owner is now concrete, and the fallback keeps #1378 open by removing its closing keyword rather than abandoning or falsely ticking box 3. | +| R-11 | **sound with caveat** | The owner-selected document correction is implementable. Companion run artifacts still contain the superseded behavior and should be updated. | +| R-12 | **sound with caveat** | A PR-diff property belongs in the existing PR job. The exact checker/test files and deterministic issue-link syntax must be named in Design. | + +## Acceptance-box routing table + +All 39 live boxes are listed. “Routed — proof blocked” means a PR is named but the stated proof +cannot currently satisfy the contract. + +| Box | Acceptance requirement (abridged) | Route | Evaluation | +| -------- | ----------------------------------------------------------------- | -------------------------- | ----------------------------------------------------------------------- | +| #1403-1 | plugin-streams-core covered by `arch:check` or reasoned exclusion | PR-B / B1–B2 | routed | +| #1403-2 | quality scan covers plugin-core surfaces or policy documented | PR-B / B1–B2 | routed | +| #1403-3 | repaired gate reports plugin-streams-core's real state | PR-B / B3 | routed | +| #1403-4 | future plugin-core omission fails | PR-B / B1 | **routed — proof blocked; independent expected-set oracle absent** | +| #1403-5 | surfaced findings triaged, not fixed | PR-B / B3 | routed; exact triage artifact path should be named | +| #1380-1 | verdict contains all 36 live units only | PR-C / C3 | routed | +| #1380-2 | removed rows get per-row Git evidence and truthful state | PR-C / C3 | **routed — proof blocked by false sagas/shared records** | +| #1380-3 | archetype table matches verdict | PR-C / C4 | routed | +| #1380-4 | repo gate iterates live members | PR-C / C2 | routed | +| #1380-5 | A14 ignores sanctioned BDD imports | PR-C / C1 | routed; imported/local/unresolved controls fire | +| #1380-6 | repo gate excludes `.llm/tmp`, docs, tools | PR-C / C2 | routed | +| #1380-7 | accepted-red debt closed or dated | PR-C / C5 | routed | +| #1380-8 | doctrine records gated units and exclusions | PR-C / C4 | routed | +| #1380-9 | dated engineering-reference plan | PR-C / C5 | routed | +| #1380-10 | RFC location and five pending ids mapped | PR-C / C6 | routed | +| #1380-11 | stale verdict row fails a test | PR-C / C3 | routed; fabricated-row negative fires | +| #1380-12 | missing live row fails a test | PR-C / C3 | routed; missing-row negative fires | +| #1380-13 | `arch:check` green; repo green or residue enumerated | PR-C / C2 | routed | +| #1378-1 | exported `any` fails red-first | PR-D / D1 | **routed — proof blocked by unresolved R-3 attribution contract** | +| #1378-2 | unlinked `as unknown as` fails | PR-D / D2 | **routed — proof blocked for closed/unmilestoned references** | +| #1378-3 | fenced docs `as any` fails | PR-D / D5 after #1537 | routed; fallback leaves issue open and moves box honestly | +| #1378-4 | six soundness files remain unchanged/green | PR-D / D5 | routed; regression evidence, not a negative control | +| #1378-5 | both tasks wire measured `--max-allow` | PR-D / D3 | routed | +| #1378-6 | budget cannot rise without same-PR issue link | PR-D / D4, existing PR job | routed; executable hook exists; exact files still missing | +| #1378-7 | both trigger reference sites typed | PR-D / D6 | routed | +| #1378-8 | full rule matrix | PR-D / D1–D5 | **routed — incomplete until findings 1–2 are repaired** | +| #1378-9 | repo scan and arch check green | PR-D / D1–D6 | **routed — proof blocked by R-3 and issue-state contracts** | +| #1530-1 | repo scan green with fixture unchanged | PR-E / E1–E4 | routed | +| #1530-2 | explicit directory + suffix exemption | PR-E / E2 | routed | +| #1530-3 | RED before exemption | PR-E / E1–E2 | routed, but RED must be pre-change evidence inside a green landed slice | +| #1530-4 | ordinary/out-of-directory cases remain red | PR-E / E3 | routed; controls fire | +| #1530-5 | remove two allowances; 10 → 8 | PR-E / E4 | routed | +| #1530-6 | repo scan and quality gate green | PR-E / E2–E4 | routed | +| #1530-7 | main `code-quality-repo` green after merge | PR-E / `[post-merge]` | sanctioned exclusion-with-notice; live marker confirmed | +| #1545-1 | all eight comments reference #1545 and register | PR-D / D2 | routed | +| #1545-2 | both budgets wired at 8 | PR-D / D3 | routed | +| #1545-3 | unlinked red; open-milestoned green | PR-D / D2 | **routed — proof blocked; state-resolution mechanism absent** | +| #1545-4 | five CLI sites mapped to cause and owner/none | PR-D / D2 | routed | +| #1545-5 | repo scan green with `allowCount: 8` | PR-D / D2–D3 | **routed — proof blocked until issue-state mechanism is defined** | + +No box lacks a named PR. Five boxes are routed to proofs that are not yet executable or truthful. + +## Negative-case review + +| PR | Review | +| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| PR-E | Directory/suffix leakage controls are fireable. The RED-first assertion is valid evidence but cannot itself be a landed slice gate; E1 must be folded into a subsequent green slice. | +| PR-B | The repaired-gate run is observable. The root-coverage negative can be tautological unless expected publishable plugin-core members come from an independent oracle; the plan does not name one. | +| PR-C | Imported/local/unresolved A14 controls, fabricated verdict row, and missing live row can all fail in the intended direction. The provenance content assertion would encode false sagas/shared records as currently written. | +| PR-D | Exported/local, re-export, unlinked, docs-fence, and overflow controls can fire. D2 lacks closed/unmilestoned controls and a state resolver; D4 has a CI hook but no exact checker/test files; D1's warning-intersection failure condition is undefined. | + +## Plan-Gate checklist + +| Gate | Cycle-3 result | +| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Research present and current | **unchecked** — the research artifact remains stale on the “strictly harder” claim and does not contain Revision 3's truncation reconciliation; the consolidated plan then misstates sagas/shared history | +| Decisions locked | **unchecked** — R-3 deliberately defers warning attribution/cardinality and possible rescope; D2 leaves live issue-state resolution undefined | +| Open-decision sweep | **unchecked** — Revision 3 labels R-3 “resolved in shape,” but its own risk says implementation may discover that green is unreachable | +| Commit slices | **unchecked** — E1's gate is red by design; multiple rows are not repository-path scoped; D4 has no named files | +| Risk register | checked with caveat — risks are named, but R-3's mitigation is a deferred rescope rather than a plan-time resolution | +| Gate set selected | **unchecked** — PR-B's coverage negative can derive expected/actual from the same function; D2 cannot test the live-state contract as specified | +| Deferred scope explicit | checked — #1530 post-merge observation and R-10 fallback are explicit and honest | +| jsr-audit | checked N/A — repo-internal tooling/docs; no published package/plugin surface change planned | + +## What I executed + +Commands are listed in execution order. An exit of 1 is expected where the command is the measured +red verdict or an intentionally missing-path/read probe; it is not normalized to zero. + +| # | Command | Exit / result | +| ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | +| 1 | Read all six selected skill files with `wc`/`sed` (initial combined read) | 0; combined output truncated, so each selected skill was reread individually | +| 2–7 | Full `sed` reads of `netscript-harness`, `netscript-doctrine`, `netscript-tools`, `netscript-deno-toolchain`, `netscript-pr`, `rtk` SKILL.md | 0 each | +| 8 | `rtk ls .llm/harness/{workflow,evaluator,gates,archetypes,scopes}` | 0 | +| 9 | Raw Git identity via `deno eval` / `Deno.Command` (`status`, branch, HEAD, parents, revision ancestry) | 0; exact identity confirmed | +| 10 | Read harness activation/run-loop/lane-policy/plan protocol/plan gate/verdict definitions/matrix/milestone docs; attempted `.llm/harness/scopes/SCOPE-docs.md` | 1 for missing old path; all other reads 0 | +| 11 | `rtk ls .llm/harness` and `find` for `SCOPE-docs.md` | 0; located `.llm/harness/archetypes/SCOPE-docs.md` | +| 12 | Full read of lane policy and correct docs scope overlay | 0 | +| 13 | Full numbered read of consolidated `plan-quality-rail.md` | 0 | +| 14 | Combined reads of research/plan/worklog/drift/cycle-2 verdict plus attempted `.llm/harness/context-pack.md` | context-pack old path 1; other reads 0; output truncated and followed by focused reads | +| 15 | `rtk grep` for context-pack/design/drift/finding headings | 0 | +| 16–18 | Focused full reads of run `context-pack.md`, Design section, D-11–D-13, research, plan, and complete cycle-2 verdict | 0 | +| 19 | Live `gh issue view` for #1403/#1380/#1378/#1530/#1545; comment 5264832009; PR #1537 | 0; large combined output truncated and followed by focused calls | +| 20 | Focused live reads of #1378/#1530/#1545 | 0 | +| 21 | `gh api` comment 5264832009 | 0; owner correction read in full | +| 22–23 | Numbered reads of rail artifacts, doctrine, debt, scanner, checker, tasks, and workflow | 0 | +| 24 | `deno doc --json` package-ai structure/timing probe | 0; command misuse `deno eval --allow-read` first exited 1, corrected invocation exited 0 | +| 25–30 | Parse `deno doc --json` object/node/type-ref structure for sdk/ai | 0 each | +| 31 | `rtk grep --glob` attempt | 2; RTK grep does not accept ripgrep's `--glob` form | +| 32–33 | `rtk proxy rg` exported/all `any` probes | 1 for no exported-line matches; 0 for broad matches | +| 34 | Fresh 30-package export-map timing/warning/type-reference census | 0; 3.333 s, 567 warnings, 1,714 affected symbol records | +| 35 | Warning-class samples for six affected packages | 0 | +| 36 | `rtk proxy deno task quality:scan` | 0 | +| 37 | `rtk proxy deno task quality:scan:repo` | 1; expected current red, 5 findings/10 allowances | +| 38 | `rtk proxy deno task arch:check` | 0 | +| 39 | `rtk proxy deno task arch:check:repo` | 1; expected current red, `FAIL=55 WARN=305 INFO=1` | +| 40 | Recursive Deno inventory for live units, verdict rows, soundness/type fixtures, A14 origins, allowance comments | 0 | +| 41 | `scanCodeQualityDetailed` current/surviving allowance reconciliation | 0; 10 current, 8 after E4, 0 issue-linked | +| 42 | HEAD/all-ref roots, commit counts, root tree, stale-path logs, shared history | 0; HEAD root confirmed; three all-ref roots found | +| 43–45 | Commit-count pinning and non-ancestor/shared delete probes | 0; 374 belongs to `64c091c5`; `fd8259b76` deletes shared on a non-HEAD history | +| 46 | `rtk proxy rg` debt records for streams/sagas | 0; sagas supersession record found | +| 47 | Numbered local-helper read and BDD import count | 0; 53 BDD files | +| 48 | Mechanical Design-table parse | 0; 20 rows | +| 49 | Live acceptance extraction for five issues | 0; total 39 | +| 50 | `rtk ls rfcs` plus status extraction | 0; five accepted numbered RFCs | +| 51 | Fresh sdk/fresh `deno doc --json` timings | 0; 0.123 s / 0.357 s | +| 52 | Full reads of `supervisor.md` and `cut-trace.md` | 0 | +| 53 | Read-only #1374 worktree status/HEAD/extractor search | 0; worktree has in-progress implementation, HEAD `b1129dd7b` | +| 54 | Read-only numbered #1374 extractor/API read | 0; stable exported provenance confirmed | +| 55–57 | Numbered order/cut-trace/research/live-issue acceptance reads | 0 | +| 58 | Final `rtk git status --short`, diff, branch, HEAD before verdict write | 0; clean, correct checkout | +| 59 | Verdict head/tail, status, `git diff --check`, and single-verdict count | 0; one `FAIL_PLAN`, only the cycle-3 file untracked | +| 60 | First `deno fmt --check` plus whitespace/verdict checks | format check 1; whitespace and verdict checks 0 | +| 61 | `deno fmt plan-eval-cycle3.md` | 0; formatting limited to the verdict file | +| 62 | Final format/whitespace/verdict/path-scope validation | 0; expected all checks green | + +## Escalation recommendation + +This third failure is caused by the **plan and companion plan artifacts**, not by an impossible +issue contract. The issues should not be weakened or rescoped to remove their acceptance boxes. The +owner should rescope the planning work into two bounded pre-implementation decisions: + +1. a measured export-reachability design that defines warning-to-declaration attribution and locks a + reachable residue policy before PR-D; and +2. a deterministic allowance-registration design that resolves open/milestoned issue state and has + closed/unmilestoned controls. + +Separately correct the sagas/shared provenance rows and replace the Design table with exact paths +and green post-slice gates. After those corrections, use owner authority to waive another full +formal cycle only if an independent reviewer checks these four exact repairs from executed evidence; +do not start implementation on the current plan. From 2a9c674fcd655cb1574cfac915bba384f8656a44 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:51:10 +0200 Subject: [PATCH 15/57] chore(harness): rescope the rail to #1549 and fix the three carried defects Revision 4. #1378 and #1545 moved to 0.0.7 with written reasons; #1549 carries the provable half in 0.0.6. R-3 is withdrawn by rescope and R-7 drops the issue-id requirement, because the scanner has --allow-read only. Fixes the three defects cycle 3 left standing. E1 no longer asks a landed slice to have a red gate -- the RED evidence is the pre-change output in the PR comment. B1's coverage test now derives its expected set independently of the function under test, because a test that asks the implementation what to expect cannot fail. And the provenance rows are corrected: sagas DOES have a checked-in supersession record at arch-debt.md:583, and shared was deleted at fd8259b76, both verified rather than taken from the verdict. The 374-commit figure is now pinned to the sha it was measured at. Refs #1403, #1380, #1530, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../plan-quality-rail.md | 20 ++++++++++++------- .../worklog.md | 4 ++-- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index 83a29e825b..e3037fa99e 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -21,15 +21,20 @@ Sequencing locks S-1…S-6 live in `plan.md`; ordered file-scoped commit slices ## The one PR order -**PR-E (#1530) → PR-B (#1403) → PR-C (#1380) → PR-D (#1378 + #1545).** Strictly sequential, one active +**PR-E (#1530) → PR-B (#1403) → PR-C (#1380) → PR-D (#1549).** Strictly sequential, one active implementation thread. PR-D is additionally gated on PR **#1537** (docs lane) landing. +**Revision 4 — rescope, not another rewrite.** After a third `FAIL_PLAN` the owner authorized rescoping +the *issues* rather than the plan. **#1378 and #1545 moved to 0.0.7** because export-reachability severity +and allowance issue-state verification were measured unimplementable at this baseline; **#1549** carries +the provable half and stays in 0.0.6. Written reasons are on all three issues. `drift.md` D-16, D-17. + | PR | Closes | Lane | Why here | | --- | --- | --- | --- | | PR-E | #1530 | Sol · low | Clears a gate red on `main` for 7 pushes; nothing downstream can report an honest `quality:scan:repo` until it lands | | PR-B | #1403 | Sol · low | p0, and owns the **single** transition to discovery-based doctrine roots | | PR-C | #1380 | Sol · medium | Consumes PR-B's selector unchanged; adds origin-awareness and the doctrine documents | -| PR-D | #1378, #1545 | Sol · high | Depends on PR-E (green repo scan), PR-B/C (settled roots), and #1537 (extractor) | +| PR-D | **#1549** | Sol · **medium** (was high; export-reachability was the complex half and it moved) | Depends on PR-E (green repo scan), PR-B/C (settled roots), and #1537 (extractor) | ## Executed baseline @@ -45,7 +50,7 @@ Measured at `01aa12b67`, re-confirmed at `84dd44ae7` (contains PR #1527). | Verdict table | 28 rows; **6** name non-live units; **14** live units have no row | | `*-soundness_test.ts` / `*_type.ts` | **6** / **12** (all under `tests/type-fixtures/`, **3** with `@ts-expect-error`) | | `deno doc --json` over all 30 export maps | **3.733 s**, exit 0, **567** `Failed resolving types` warnings | -| Repository history | root `317e4b509` (2026-07-06, "cut 0.0.1-beta.5"), **374** commits | +| Repository history | root `317e4b509` (2026-07-06, "cut 0.0.1-beta.5"); **374** commits **measured at `84dd44ae7`** — the count moves with `main`, so it is only meaningful pinned to a sha (cycle 3 finding 7) | ### A14 has three identifier origins, not two @@ -75,8 +80,9 @@ beta.5 release cut) with 374 commits, and `git ls-tree 317e4b509:packages/` alre | --- | --- | | `@netscript/triggers` | not present anywhere in this repository's history (begins `317e4b509`); superseded by `packages/plugin-triggers-core` per `arch-debt.md:385`, predating this history | | `@netscript/workers` | same, per `arch-debt.md:561` → `packages/plugin-workers-core` | -| `@netscript/sagas`, `@netscript/streams` | not present in this history; successors exist; **no** checked-in supersession record found — PR-C states that absence rather than inferring a rename | -| `@netscript/shared` | not present in this history; 10 commits touch `packages/shared/**` on non-ancestor refs only — PR-C cites that and claims no removal commit on `main` | +| `@netscript/sagas` | not present in this history; **a checked-in supersession record exists** — `arch-debt.md:583-584` reads "the top-level `packages/sagas` directory named in this heading no longer exists — the code and this resolved debt live entirely in `packages/plugin-sagas-core`". Cycle 3 finding 3: revision 3 asserted no record existed, which was **false**. Verified by the orchestrator. | +| `@netscript/streams` | not present in this history; successor `packages/plugin-streams-core` exists; no supersession record found — PR-C states that absence **after** running the same `arch-debt.md` probe that found the sagas one | +| `@netscript/shared` | not present in HEAD's history; **added at `0ef13de35`, deleted at `fd8259b76`** (`feat(contracts): consolidate shared foundation package`, 2026-06-05, which deletes `packages/shared/deno.json` and 25 further `packages/shared/**` paths) — both on **non-ancestor** history. Cycle 3 finding 3: revision 3 said only "no removal commit on `main`", true but omitting the load-bearing commit. PR-C cites the commit **and** the ancestry qualifier. | | `plugins/hello-world` | not present in this history; no successor and no supersession record | #1380 box 2 was **amended with owner authorization** to require per-row evidence and admit "never present @@ -96,11 +102,11 @@ Revision 1 inherited that claim without re-measuring and was failed on it (`drif | --- | --- | | `R-1` | **PR-E lands first.** #1378's `gate:` box needs `quality:scan:repo` green; it is red for a reason that is not a defect. PR-E fixes scanner **scope**, not the findings. | | `R-2` | Type-fixture exemption keyed on **directory AND suffix** (`tests/type-fixtures/` + `_type.ts`), asserted by test, with leakage controls **both** ways — dir-only and suffix-only must still be reported. | -| `R-3` | Export-awareness uses `deno doc --json` over each package's `exports` map. **Fail-closed is scoped to the intersecting set:** PR-D first measures how many of the 567 warnings touch a declaration the any-rule actually inspects, fails closed on **that** set, and enumerates the non-intersecting residue in `arch-debt.md` with a date. Blanket fail-closed on all 567 makes #1378's own green gate unreachable — cycle 2 finding 1, and why revision 2's R-3 was wrong. A re-exported `any` must be attributed to the published entrypoint (fixture required). | +| ~~`R-3`~~ | **WITHDRAWN by rescope (revision 4).** Export-awareness moved to 0.0.7 with #1378. Cycle 3 measured the premise: 567 warnings on an exit-0 run and **1,714 published symbol records with unresolved type references** (3,945 occurrences), with warning text naming the dependency module rather than the dependent declaration — so there is no deterministic attribution to build the rule on. PR-D does **not** attempt it. Superseded rationale: Export-awareness uses `deno doc --json` over each package's `exports` map. **Fail-closed is scoped to the intersecting set:** PR-D first measures how many of the 567 warnings touch a declaration the any-rule actually inspects, fails closed on **that** set, and enumerates the non-intersecting residue in `arch-debt.md` with a date. Blanket fail-closed on all 567 makes #1378's own green gate unreachable — cycle 2 finding 1, and why revision 2's R-3 was wrong. A re-exported `any` must be attributed to the published entrypoint (fixture required). | | `R-4` | Doctrine root selector is **expanded top-level `packages/*` + `plugins/*`** — the 36 units — **not** every workspace member (root `deno.json:3-9` also lists `packages/cli/e2e`, `examples/*`, `apps/*`). `packages/cli/e2e` is excluded and the exclusion is stated in the doctrine. | | `R-5` | A14 resolves `imported` \| `locally-bound` \| `unresolved` by lexical import **and** binding collection (no type checker), firing **only** on `unresolved`. All three origins tested; origin 3 needs a synthetic fixture. | | `R-6` | **PR-B owns the single transition.** PR-B introduces `discoverDoctrineRoots()` returning the final 36-unit selector and repoints `arch:check` at it in one step — **no** interim 17-root list, **no** checked-in root data file. PR-C consumes the same function unchanged for `arch:check:repo`. This retires the two-step that cycle 1 finding 9 and cycle 2 finding 4 both rejected; S-4 holds because the coverage predicate is preserved around one function. | -| `R-7` | `--max-allow` is wired at **8** (after PR-E removes two redundant allowances) and all 8 survivors reference **#1545**, the umbrella registration issue filed for this (owner decision). Without it, #1378's linked-issue rule reds the gate on day one — cycle 2 finding 2. | +| `R-7` | **[rescoped, revision 4]** `--max-allow` is wired at the count measured in the implementing PR (**8** after PR-E), and in 0.0.6 it carries **no issue-id requirement** — the scanner has `--allow-read` only and cannot verify an issue is open and milestoned (cycle 3 finding 2). The registration rule and #1545 moved to 0.0.7. The budget still only falls. Superseded text: wired at **8** (after PR-E removes two redundant allowances) and all 8 survivors reference **#1545**, the umbrella registration issue filed for this (owner decision). Without it, #1378's linked-issue rule reds the gate on day one — cycle 2 finding 2. | | `R-8` | Findings surfaced by newly-covered scans are **triaged into issues, never fixed in the surfacing PR**. | | `R-9` | Accepted RFCs are promoted to numbered `rfcs/NNNN-*.md`; `.llm/runs/*/design/canonical/` bundles are provenance/draft. PR-C records this in `rfcs/README.md` and maps the five `DECISION_PENDING` entries onto it **without filing them**. | | `R-10` | **#1374 owns the `docs/site/**` fenced-TS extractor; PR-D consumes it** and sequences after PR #1537 (owner decision; PR #1537 comment `5264583905`). **If that surface stays private:** slice D5 and #1378 box 3 move with the issue, **and PR-D references `#1378` without a closing keyword**, stating the remaining scope. It closes #1545 either way. Cycle 2 finding 9 — a fallback that moves a box while keeping `Closes` would auto-close an issue with undelivered acceptance. | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index 513714338d..47d168062e 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -349,11 +349,11 @@ Ordered, file-scoped, one gate each. `deno test` roots take | # | PR | Slice | Files | Gate | | --- | --- | --- | --- | --- | -| E1 | PR-E | RED fixture: `@ts-expect-error` in a type fixture is reported | `.llm/tools/quality/scan-code-quality_test.ts` | `deno test .llm/tools/quality/` fails | +| E1 | PR-E | RED fixture proving `@ts-expect-error` in a type fixture is reported, **plus** the exemption that makes it pass — one landed slice | `.llm/tools/quality/scan-code-quality_test.ts`, `.llm/tools/quality/scan-code-quality.ts` | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/quality/` **passes**. The RED evidence is the test's pre-change output pasted in the PR comment, not a red gate on a landed commit — cycle 3 finding 4: a slice whose required gate is red cannot satisfy the Plan-Gate. | | E2 | PR-E | `isTypeFixture` exemption (dir + suffix) | `scan-code-quality.ts` | test green; `quality:scan:repo` exit 0 | | E3 | PR-E | leakage controls: ordinary source, and `_type.ts` outside the dir, stay red | `scan-code-quality_test.ts` | both negatives fail-on-removal | | E4 | PR-E | drop the two redundant allowances | `desktop-consumer_type.ts`, `sdk-assignability_type.ts` | `quality:scan:repo` `allowCount` 10 → 8 | -| B1 | PR-B | `discoverDoctrineRoots()` returning the **final 36-unit selector** (expanded top-level `packages/*` + `plugins/*`, `packages/cli/e2e` excluded) + coverage test | `.llm/tools/fitness/check-doctrine.ts`, `.llm/tools/fitness/check-doctrine_test.ts` | coverage test fails when a publishable `plugin-*-core` leaves the selector | +| B1 | PR-B | `discoverDoctrineRoots()` returning the **final 36-unit selector** (expanded top-level `packages/*` + `plugins/*`, `packages/cli/e2e` excluded) + coverage test asserted against an **independently derived** expected set | `.llm/tools/fitness/check-doctrine.ts`, `.llm/tools/fitness/check-doctrine_test.ts` | the coverage test enumerates publishable units from the filesystem/workspace list **without** calling `discoverDoctrineRoots()`, then asserts the function's output equals it — cycle 3 finding 5: a test that derives its expectation from the function under test cannot fail | | B2 | PR-B | `arch:check` repointed at `discoverDoctrineRoots()` in **one** step — no interim list, no data file | `deno.json:156`, `.llm/tools/fitness/check-doctrine.ts` | `deno task arch:check` exit 0 with `plugin-streams-core` covered | | B3 | PR-B | run the repaired gate on `plugin-streams-core`; triage output to new issues | triage list in slice dir only | no `packages/**` source edit in the diff | | C1 | PR-C | `resolveIdentifierOrigin()`; A14 fires only on `unresolved` | `check-doctrine.ts`, `check-doctrine_test.ts` | 3 fixtures: import (quiet), local binding (quiet), bare global (**red**) | From ce011b5f157a3f90bfbbf2c6a9e4f25ff3a060bc Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 11:51:38 +0200 Subject: [PATCH 16/57] chore(harness): record the rescope decision and its per-criterion reasons Refs #1378, #1545, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 39 +++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index c185587b01..d7ae0d3e2d 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -463,3 +463,42 @@ owner as a **milestone-content decision**: #1530, #1403 and #1380 remain deliver identified fixes; #1378's boxes 1 and 2 rest on properties this codebase cannot satisfy in 0.0.6 without a separate debt programme, and the honesty rule says such criteria move with their issue rather than being ticked or quietly reinterpreted. + +## D-17 — rescope executed: which acceptance criteria moved, and why + +- **Severity:** architectural (milestone content changed) +- **Recorded:** 2026-08-12, owner-authorized after cycle 3 `FAIL_PLAN` +- **Authorization:** owner directive — "If PLAN-EVAL cycle 3 fails, rescope the issues and report exactly + which acceptance criteria move and why." + +### Moved to 0.0.7 + +| Issue | Criteria moved | Why it cannot be truthfully ticked in 0.0.6 | +| --- | --- | --- | +| **#1378** (whole issue) | box 1 (exported `any` fails), box 2 (unlinked `as unknown as` fails), and the exported/local + linked/unlinked halves of box 8 | **Measured, not judged.** `deno doc --json` over 30 export maps: 3.733 s, exit 0, **567** warnings, **1,714** published symbol records with unresolved type references (3,945 occurrences; 230/724 in `plugin-sagas-core`, 174/487 in `fresh`). Warning text names the *dependency module*, not the dependent declaration, so export-reachability has **no deterministic attribution** at this baseline. Separately, box 2 requires the allowance's issue to be **open and milestoned**, and `scan-code-quality.ts` runs `--allow-read` only (`deno.json:50-51`) — it cannot observe live issue state, so the obvious proof cannot fail on the property the box requires. | +| **#1545** | all five boxes | Its acceptance depends on the registration rule that moved with #1378. Boxes 1 and 3 require the mechanism to exist. | + +### Stayed in 0.0.6 + +| Issue | Status | +| --- | --- | +| **#1530** | unchanged — PR-E. Only defect was E1's gate framing, fixed in the Design table. | +| **#1403** | unchanged — PR-B. Only defect was the coverage test deriving its expectation from the function under test; now independently derived. | +| **#1380** | unchanged — PR-C. Only defects were two provenance rows, corrected against verified evidence. | +| **#1549** *(new)* | the provable half of #1378: docs-fence scanning via #1374's extractor, soundness/type-fixture exemption asserted by rule, `--max-allow` at the measured count, the same-PR budget-link predicate in the existing `code-quality` job, and typing `docs/site/reference/triggers/index.md:310` + its twin. | + +### What was deliberately **not** done + +- **No id-presence check dressed up as registration.** Accepting any `#` without verifying open/milestoned + would satisfy the written test while violating the contract — the unearned-green pattern this lane exists + to remove. #1549 explicitly declines it. +- **No fourth plan rewrite.** Three cycles closed 6 of 10 findings with none unaddressed; the residue was in + the issues' premises, not the plan's structure. +- **No criterion reinterpreted to fit.** Each moved box moved with a written reason on its issue + (#1378 and #1545 comments), per the honesty rule. + +### Net effect on the lane + +Owned issues: **#1436, #1415 closed** (PR #1527, merged `63cd1cd58`). **#1530, #1403, #1380, #1549** remain +in 0.0.6 across PR-E → PR-B → PR-C → PR-D. **#1378, #1545** carry to 0.0.7. The lane's deliverable count is +unchanged at four remaining PRs; what changed is that PR-D's scope is now provable. From 7675842dfa37ecbded19673602d6d69bb8ead398 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 12:45:15 +0200 Subject: [PATCH 17/57] chore(harness): record cycle-4 dispatch through the automated phase-eval path Opened the run-record PR #1553 as a draft surface and applied the openhands + status:plan-eval pair, after verifying the workflow's PLAN-EVAL branch does not test draft state. Dispatch verified from the posted trigger and run 31588750658 rather than assumed from a green workflow. Records the route shift: cycles 1-3 were native Codex Sol high, cycle 4 is MiniMax M3 over OpenRouter chosen by the phase dispatcher, not by orchestrator escalation -- which also makes it a genuine third opinion on a plan three Codex cycles have failed. Refs #1403, #1380, #1530, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index d7ae0d3e2d..bdfbb8a7b0 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -502,3 +502,35 @@ ticked or quietly reinterpreted. Owned issues: **#1436, #1415 closed** (PR #1527, merged `63cd1cd58`). **#1530, #1403, #1380, #1549** remain in 0.0.6 across PR-E → PR-B → PR-C → PR-D. **#1378, #1545** carry to 0.0.7. The lane's deliverable count is unchanged at four remaining PRs; what changed is that PR-D's scope is now provable. + +## D-18 — PLAN-EVAL cycle 4 runs on the automated path; evaluator route changed by automation, not escalation + +- **Severity:** minor (route record) +- **Recorded:** 2026-08-12 +- **Owner directive:** run a separate PLAN-EVAL cycle 4 on the rescoped plan before dispatching PR-E; do + not waive it, because the change remains a complex cross-cutting quality-gate change. +- **Mechanism used, per D-14:** the rescoped plan needed a PR surface for the label pair, so the control + branch's run-record PR **#1553** was opened **as a draft** and given `openhands` + `status:plan-eval`. + Verified from `openhands-phase-eval.yml:24-34` that the PLAN-EVAL branch does **not** test draft state + (only `status:impl-eval` requires non-draft), so a draft record PR is the correct surface. +- **Dispatch verified, not assumed:** run `29334443743` succeeded on the `labeled` event and posted the + trigger; run **31588750658** is the evaluator. + +```text +@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan +head=ce011b5f157a3f90bfbbf2c6a9e4f25ff3a060bc +Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a +``` + +- **Route shift worth recording.** Cycles 1–3 ran on the native opposite-family route (Codex · Sol · high), + which `lane-policy.md` makes the local default, with OpenRouter reserved for a third opinion or a + quota-blocked native family. Cycle 4 is **MiniMax M3 over OpenRouter** — the phase-bound PLAN-EVAL model + in `lane-policy.md` § Native-first formal evaluation. The change is **chosen by the automation**, not an + orchestrator escalation: the phase dispatcher resolves the model. Recorded so the run's route history is + not mistaken for an unrecorded escalation. It also means cycle 4 is a genuine **third opinion** on a plan + three Codex cycles have already failed, which is the more useful pairing here. +- **No duplication.** The local Codex evaluator thread finished cycle 3 and was not resumed. One evaluation + of one plan revision, per D-14. +- **#1553 must never leave draft.** Draft → ready fires the initial IMPL-EVAL, which is meaningless for a + record PR that ships no code. `netscript-pr` also forbids merging a PR at `status:plan-eval`, which is + correct for this surface. From 69ef5f15dea8736528d36bbcd071a45682b86be2 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 12:56:10 +0200 Subject: [PATCH 18/57] chore(harness): reconcile companion prose to the rescoped surface (cycle 4 F-4) Cycle 4 passed the plan itself -- rescope bounded and reasoned, the three cycle-3 defects repaired, and the Plan-Gate checklist cleared -- and failed on companion artifacts still describing the pre-rescope surface. That is the same sin as cycle 2 finding 6: the rail plan was reconciled and the documents around it were not. Six stale statements fixed: plan.md's three-PRs prose and its PR-D-closes-#1378 heading, worklog's expanded-in-PR-C port note, the rail routing table's #1378 and #1545 rows, the routing denominator, and two open decisions that read RESOLVED where the truth is WITHDRAWN BY RESCOPE. Calling a withdrawn decision resolved is the unearned-green wording this lane exists to remove. F-5 (no implementation committed) is expected at this phase and the evaluator says so; it discharges when PR-E dispatches. Refs #1403, #1380, #1530, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 38 +++++++++++++++++++ .../plan-quality-rail.md | 29 +++++++------- .../plan.md | 26 +++++++++---- .../worklog.md | 6 ++- 4 files changed, 75 insertions(+), 24 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index bdfbb8a7b0..55b32efc2a 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -534,3 +534,41 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a - **#1553 must never leave draft.** Draft → ready fires the initial IMPL-EVAL, which is meaningless for a record PR that ships no code. `netscript-pr` also forbids merging a PR at `status:plan-eval`, which is correct for this surface. + +## D-19 — cycle 4 verdict `FAIL_RESCOPE`: the plan passed, its companion prose did not + +- **Severity:** minor (documentation consistency; no scope or decision changed) +- **Recorded:** 2026-08-12, PLAN-EVAL cycle 4, MiniMax M3 over OpenRouter via the automated phase-eval path +- **Verdict:** `FAIL_RESCOPE` on PR #1553 at `ce011b5f1`, run `31588750658`. **Five findings PASS**, two + FAIL — and the two are not about the plan's decisions: + - **F-1 PASS** — the rescope is bounded, per-criterion reasoned, and honest; criteria moved with their + issues and none was reinterpreted. + - **F-2 PASS** — the three defects cycle 3 left standing are repaired (E1's red-gate structure, B1's + independent oracle, the two provenance rows). + - **F-3 PASS** — **the Plan-Gate checklist is cleared**: research current, decisions locked, 20 + file-scoped slices each with a green post-slice gate, gate set fireable, deferred scope explicit. + - **F-6 / F-7 PASS** — lane discipline and verdict vocabulary. + - **F-4 FAIL** — companion artifacts still describe the pre-rescope surface. + - **F-5 FAIL** — no implementation slice is committed. The evaluator states explicitly that this is + **not a Plan-Gate failure of the plan**; it discharges the moment PR-E is dispatched. +- **So the blocking defect is F-4 alone**, and it is the same sin as cycle 2 finding 6: I reconciled the + rail plan and left the companion documents behind. Six stale statements, all now fixed in one + run-artifact-only commit: + 1. `plan.md` — "the three issues … three sequential PRs" → four, with the order named in its own prose + rather than delegated to another file. + 2. `plan.md` — "PR-D … closes #1378 … export-aware `any` severity" → closes **#1549**, effort dropped + high → medium because export-reachability was the complex half and it moved. + 3. `worklog.md` § Ports — `discoverDoctrineRoots()` "expanded in PR-C" → PR-B performs the **single** + transition and PR-C consumes it unchanged. + 4. `plan-quality-rail.md` routing — the nine #1378 rows and five #1545 rows replaced by seven #1549 rows, + with the moved slices marked out-of-milestone rather than deleted. + 5. `plan-quality-rail.md` routing header — denominator corrected to the in-0.0.6 boxes, stating that the + moved boxes are *out of milestone*, not unrouted. + 6. `plan-quality-rail.md` open decisions — R-3 and R-7 marked **withdrawn by rescope**, not "resolved". + Calling a withdrawn decision resolved is precisely the unearned-green wording this lane removes. +- **One factual note back to the evaluator:** its Remaining-risks section says "#1549 must be filed with + provable-half boxes before PR-D dispatch … the lane must reconcile before filing #1549". #1549 was + **already filed** with exactly those seven boxes before this evaluation ran — it is referenced in D-17 at + the evaluated head. The risk is discharged, not outstanding. +- **Action:** F-4 fixed; PLAN-EVAL re-triggered by label-cycling `status:plan-eval` per D-14's rerun path + (the only sanctioned rerun mechanism — not a push, not a re-request). PR-E dispatches on `PASS`. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index e3037fa99e..50cda17e72 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -118,9 +118,9 @@ Revision 1 inherited that claim without re-measuring and was failed on it (`drif | Decision | Status | | --- | --- | | `docs/site/**` extractor ownership | **RESOLVED** — `R-10`, with the closing-keyword consequence stated | -| The 567 `deno doc` warnings | **RESOLVED in shape** — `R-3`; PR-D measures the intersection before wiring, residue is dated debt | -| The 8 pre-existing allowances | **RESOLVED** — `R-7`, umbrella issue #1545 | -| Budget-link predicate mechanism | **RESOLVED** — `R-12` | +| The 567 `deno doc` warnings | **WITHDRAWN BY RESCOPE — not resolved.** Export-reachability moved to 0.0.7 with #1378 (`drift.md` D-17). Nothing in 0.0.6 depends on an answer. Cycle 4 finding 4 flagged this row as still claiming a 0.0.6 resolution. | +| The 8 pre-existing allowances | **WITHDRAWN BY RESCOPE — not resolved.** The registration rule and #1545 moved to 0.0.7. In 0.0.6, `--max-allow` is wired at the measured count with **no** issue-id requirement, so the population needs no register yet. | +| Budget-link predicate mechanism | **RESOLVED** — `R-12`, one added step in the existing `code-quality` PR job. Stays in 0.0.6 as #1549 box 5. | | `arch:check:repo` residue after R-4/R-5 | defer to PR-C — #1380 box 13 accepts "exit 0 **or** residue enumerated" | | Successor records for `sagas`/`streams`/`hello-world` | defer to PR-C — the deliverable is to **state the absence**, not to find one | @@ -151,7 +151,11 @@ Revision 1 inherited that claim without re-measuring and was failed on it (`drif `--allow-write --allow-run` is mandatory: nine tests under `.llm/tools` call `Deno.makeTempDir()` and one spawns a subprocess (`drift.md` D-8). `deno task e2e:cli` is out of scope for the whole rail. -## Acceptance-box routing — every box has a slice, files, and a proof +## Acceptance-box routing — every in-milestone box has a slice, files, and a proof + +**Denominator after the rescope: 28 in-0.0.6 boxes** — #1530 (7, one `[post-merge]`), #1403 (5), +#1380 (13), #1549 (7) minus the 4 `gate:`/duplicate overlaps counted once. The 14 boxes of #1378 and +#1545 are **out of milestone**, not unrouted (cycle 4 finding 4). Slice ids refer to `worklog.md` § Design. @@ -169,13 +173,10 @@ Slice ids refer to `worklog.md` § Design. | #1380 · 9 | PR-C | C5 | `10-…md` carries a **dated** plan for engineering-reference §1–§5/§8–§10; test asserts the section exists and is dated | | #1380 · 10 | PR-C | C6 | `rfcs/README.md` records `rfcs/NNNN-*.md` canonical per `R-9`; all five `DECISION_PENDING` ids mapped; test asserts all five present | | #1380 · 13 | PR-C | C2 | named gate pair | -| #1378 · 1 | PR-D | D1 | exported `any` red, local `any` unchanged, re-export attribution fixture | -| #1378 · 2 | PR-D | D2 | unlinked `as unknown as` red; allowance linked to an open milestoned issue green | -| #1378 · 3 | PR-D | D5 | `as any` in a `docs/site/**` fence red via #1374's extractor; **moves with the issue** under `R-10`'s fallback, which then drops `Closes #1378` | -| #1378 · 4 | PR-D | D5 | the 6 `*-soundness_test.ts` files green with `@ts-expect-error` unchanged — regression evidence, explicitly not a negative case | -| #1378 · 5 | PR-D | D3 | `--max-allow` wired at 8; overflow red | -| #1378 · 6 | PR-D | D4 | the added `code-quality` step (`R-12`) is red when the budget rises without a same-diff issue link, exercised on PR-D itself | -| #1378 · 7 | PR-D | D6 | `docs/site/reference/triggers/index.md:310` and `examples_test.ts:65` typed; both compile with no `any` | -| #1378 · 8 | PR-D | D1–D5 | full matrix | -| #1378 · 9 | PR-D | D1–D6 | `quality:scan:repo` + `arch:check` green (depends on PR-E, PR-B, PR-C) | -| #1545 · 1–5 | PR-D | D2, D3 | all 8 allowances reference #1545; budget at 8; registered-vs-unregistered test; the five CLI sites recorded against the plugin-port-identity cause | +| **#1549 · 1–2** | PR-D | D5 | `as any` in a `docs/site/**` fence red, proven red-first, **consuming** #1374's extractor — a test asserts no second fence parser exists in the tree | +| **#1549 · 3** | PR-D | D5 | the 6 `*-soundness_test.ts` files green with `@ts-expect-error` unchanged, and a test asserts the exemption rather than relying on filename luck | +| **#1549 · 4** | PR-D | D3 | `--max-allow` wired into both tasks at the count measured in that PR; overflow fixture red. **No issue-id requirement in 0.0.6.** | +| **#1549 · 5** | PR-D | D4 | the added `code-quality` step (`R-12`) is red when the budget rises without a same-diff issue link, with a linked-GREEN control; exercised on PR-D itself | +| **#1549 · 6** | PR-D | D6 | `docs/site/reference/triggers/index.md:310` and `examples_test.ts:65` typed; both compile with no `any` | +| **#1549 · 7** | PR-D | D3–D6 | `gate:` `quality:scan:repo` + `arch:check` green (depends on PR-E, PR-B, PR-C) | +| ~~#1378 · 1–9~~ · ~~#1545 · 1–5~~ | **moved to 0.0.7** | ~~D1, D2~~ | Slices **D1** (export-awareness) and **D2** (allowance registration) move with their issues. Written reasons on #1378 and #1545; measurement in `drift.md` D-17. Not routed here and not ticked here. | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan.md b/.llm/runs/release-0.0.6-internals--orchestration/plan.md index 7e5ca62281..23dc29281d 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan.md @@ -84,10 +84,14 @@ Required deliverables: > `plan-quality-rail.md` § The one PR order. PR-E was inserted after this section was written, and > `plan-eval-cycle2.md` finding 6 failed the plan for leaving the two statements in conflict. Where > this section and `plan-quality-rail.md` differ, the rail plan governs. +> +> **The order, stated here so this section is not a hazard on its own:** +> **PR-E (#1530) → PR-B (#1403) → PR-C (#1380) → PR-D (#1549)** — four sequential PRs, not three. -Serialized under this lane because the three issues overlap on root lists, scan semantics, doctrine -and architecture debt. One rail plan, **one** PLAN-EVAL over the whole rail, then three sequential -PRs each with its own IMPL-EVAL. +Serialized under this lane because the issues overlap on root lists, scan semantics, doctrine and +architecture debt. One rail plan, one PLAN-EVAL over the whole rail, then **four** sequential PRs. +Each PR's IMPL-EVAL now fires automatically on draft → ready rather than being dispatched +(`drift.md` D-14). **PR-B — `fix/1403-quality-gate-coverage`** · closes #1403 · lane `light_implementation` (Sol · low) · review `review_codex_light`. @@ -106,11 +110,17 @@ Re-walks the verdict table to the 36 live units, records rename-vs-deletion per root, stops A14 firing on `@std/testing/bdd` imports, and closes-or-dates the accepted-red `arch-debt.md` entry. -**PR-D — `fix/1378-quality-scan-rule-power`** · closes #1378 · lane `complex_implementation` -(Sol · high) · review `review_codex_complex` (Fable 5 · medium). - -The largest slice: export-aware `any` severity, allowances that require a linked open issue and are -budgeted via `--max-allow`, and `docs/site/**` fenced-TS extraction. +**PR-D — `fix/1549-quality-scan-provable-half`** · closes **#1549** · lane `normal_implementation` +(Sol · medium) · review `review_codex` (Fable 5 · low). + +**Rescoped (revision 4, owner-authorized).** #1378 and #1545 moved to **0.0.7**: export-reachability +severity and allowance issue-state verification were measured unimplementable at this baseline (567 +`deno doc` warnings, 1,714 published symbol records with unresolved types, and a scanner that has +`--allow-read` only). #1549 carries the provable half: `docs/site/**` fenced-TS scanning consuming +#1374's extractor, the soundness/type-fixture exemption asserted by rule, `--max-allow` at the measured +count **with no issue-id requirement**, the same-PR budget-link step in the existing `code-quality` job, +and typing `docs/site/reference/triggers/index.md:310` with its executable twin. Effort drops from high +to medium because export-reachability was the complex half and it moved. Full reasons: `drift.md` D-17. ### Locked sequencing decisions diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index 47d168062e..1924ccf761 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -328,8 +328,10 @@ Repo-internal tooling; no `packages/**`/`plugins/**` published surface changes, ### Ports - `discoverDoctrineRoots(): string[]` — the single source of truth for doctrine root selection. Exists so - the task string, `arch:check:repo`, and the coverage test read one function. Introduced in PR-B and - **expanded** in PR-C; never duplicated into a checked-in list (this replaces the withdrawn R-6). + the task string, `arch:check:repo`, and the coverage test read one function. **PR-B performs the single + transition** to the final 36-unit selector; PR-C **consumes it unchanged** for `arch:check:repo` and does + not expand it. There is no interim root set and no checked-in list — that two-step was the withdrawn R-6, + rejected by cycle 1 finding 9 and cycle 2 finding 4. - `resolveIdentifierOrigin(file, ident)` — lexical import + top-level/local binding collection. No type checker; `check-doctrine.ts` stays a line/lexical scanner. - **#1374's extractor** — consumed, not re-implemented. PR-D imports it; if its surface stays private, From 4215f819da28bb1e50659ba8a12df4651f9cefe9 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 13:17:48 +0200 Subject: [PATCH 19/57] chore(harness): PLAN-EVAL PASS at cycle 5 -- Plan-Gate cleared Verdict PASS on PR #1553 run 31589648809 at head 69ef5f15d, reached by label-cycling status:plan-eval, which is the only sanctioned rerun path and the only one that works, since the phase-eval workflow ignores pushes. Records the five-cycle history and why it was not ceremony: cycles 1-3 each found a defect that would otherwise have shipped, and cycle 3's 1,714-symbol measurement is what forced the rescope rather than any amount of re-reading. Implementation authorized. PR-E dispatches next. Refs #1403, #1380, #1530, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 30 +++ .../plan-eval-cycle4.md | 150 ++++++++++++++ .../plan-eval-cycle5.md | 193 ++++++++++++++++++ 3 files changed, 373 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle4.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle5.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 55b32efc2a..fe8a43ecbd 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -572,3 +572,33 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a the evaluated head. The risk is discharged, not outstanding. - **Action:** F-4 fixed; PLAN-EVAL re-triggered by label-cycling `status:plan-eval` per D-14's rerun path (the only sanctioned rerun mechanism — not a push, not a re-request). PR-E dispatches on `PASS`. + +## D-20 — PLAN-EVAL PASS at cycle 5; Plan-Gate cleared, implementation authorized + +- **Severity:** minor (milestone in the run's own process) +- **Recorded:** 2026-08-12 +- **Verdict:** **`PASS`** — PR #1553, run `31589648809`, evaluated head `69ef5f15d`, MiniMax M3 over + OpenRouter via the automated phase-eval path. Verdict body saved as `plan-eval-cycle5.md`; cycle 4's + `FAIL_RESCOPE` saved as `plan-eval-cycle4.md`. +- **Rerun mechanism used:** label-cycled `status:plan-eval` → `status:plan` → `status:plan-eval`, the only + sanctioned rerun path (D-14). Exactly one `status:` label at every point. Not a push, not a + re-requested review — and a push would not have worked, since `openhands-phase-eval.yml` triggers only on + `labeled` and `ready_for_review`. +- **Cycle history, for the retrospective:** + +| Cycle | Route | Verdict | What it cost, and what it bought | +| --- | --- | --- | --- | +| 1 | Codex · Sol · high (local) | `FAIL_PLAN` | Caught R-9's premise — a claim inherited from #1380 and never re-measured, inside a plan whose whole value was re-measurement | +| 2 | Codex · Sol · high (local) | `FAIL_PLAN` | Caught the allowance rule reddening CI on day one, the unreachable green gate, and the diff-vs-file impossibility of box 6 | +| 3 | Codex · Sol · high (local) | `FAIL_PLAN` | **Measured** the premise instead of arguing it: 1,714 published symbol records with unresolved types. That number is what forced the rescope | +| 4 | MiniMax M3 (automated) | `FAIL_RESCOPE` | Passed the plan and the Plan-Gate; failed the companion prose still describing the pre-rescope surface | +| 5 | MiniMax M3 (automated) | **`PASS`** | Plan-Gate cleared | + +- **Judgement worth recording:** five cycles is far more than this repo's two-cycle norm, and it was not + ceremony. Cycles 1–3 each found a defect that would have shipped: a no-op premise, a gate red by + construction, and a rule whose green state was unreachable. Cycle 3's measurement in particular is the + kind of finding no amount of plan re-reading produces — it required executing `deno doc --json` and + parsing the result. The cost was borne almost entirely by the *plan*, not by implementation, which is + where it is cheapest to pay. +- **Implementation is now authorized.** PR-E dispatches first. F-5 of cycle 4 (no committed slice) + discharges on that dispatch, as the evaluator stated. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle4.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle4.md new file mode 100644 index 0000000000..bb2859f015 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle4.md @@ -0,0 +1,150 @@ +@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan head=ce011b5f157a3f90bfbbf2c6a9e4f25ff3a060bc + + +Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a +Evaluated head SHA: ce011b5f157a3f90bfbbf2c6a9e4f25ff3a060bc + +use harness + +## SKILL + +- `netscript-harness` — apply the Plan-Gate and preserve evaluator independence. +- `openhands-handoff` — publish one machine-readable OpenHands verdict. +- `netscript-tools` — use repository-native, read-only evidence commands. + +Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push, +or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head +SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then +evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture, +tests, consumer proof, release risk, and false-done states. Verify claims directly where useful. + +Return concise findings with exact evidence and required amendments. End with exactly one supported +verdict line using `OPENHANDS_VERDICT: PASS`, `OPENHANDS_VERDICT: FAIL_PLAN`, or +`OPENHANDS_VERDICT: FAIL_RESCOPE`. Write the same verdict to `OPENHANDS_SUMMARY_PATH`. This is a +planning gate: never emit `FAIL_FIX` for implementation that has not started. + + +## OpenHands Agent — Completed + +OPENHANDS_VERDICT: FAIL_RESCOPE + +Model: `openrouter/minimax/minimax-m3` +Provider: `OPENROUTER` +Job status: success +Verdict source: summary-file +Summary source: agent +Housekeeping: ack=success trace=success commit-artifacts=skipped replies=skipped draft-pr=skipped +Trace: .llm/tmp/run/openhands/pr-1553/run-31588750658-1 (in the Actions artifact) +Artifact: openhands-agent-31588750658-1 + +# PLAN-EVAL · 0.0.6 chores/internals lane · cycle 4 (post-rescope) + +## Identity + +| Field | Value | +| --- | --- | +| Run | `release-0.0.6-internals--orchestration` | +| Branch | `chore/release-0.0.6-internals-orchestration` | +| Phase | `plan` (Plan-Gate, before any rail implementation commit) | +| Trigger model | `openrouter/minimax/minimax-m3` | +| Evaluated HEAD | `ce011b5f1` — `chore(harness): record the rescope decision and its per-criterion reasons` | +| Trusted base | `d7e2b67b2` (origin/main) | +| Evaluator session | this cloud session — Plan-Gate pass, not an implementation lane | +| Verdict | FAIL_RESCOPE | + +## Scope summary + +The lane owns the 0.0.6 chores/internals work. PR-A (#1436+#1415, close-gate trust) merged at `63cd1cd58`. Four rail PRs remain, in the order Revision 3 locks: PR-E (#1530), PR-B (#1403), PR-C (#1380), PR-D (now #1549, the provable half of #1378). After cycle-3 owner-authorized rescope, #1378 and #1545 move to 0.0.7; #1549 carries the provable half in 0.0.6. PR-D's bootstrap branch `origin/fix/1530-type-fixture-scan-scope` carries only an `implement.md` slice brief — no rail code yet. No implementation slice is committed against this branch; HEAD differs from `origin/main` only in orchestration/run artifacts. + +## Findings (evaluated against ce011b5f1) + +### F-1 — PASS — Rescope decision is recorded with per-criterion evidence +`drift.md:467-504` (D-17, committed at HEAD) names exactly which acceptance boxes moved to 0.0.7, what stays in 0.0.6, and why. The moved set is bounded (#1378 boxes 1, 2, exported/local and linked/unlinked halves of 8; all of #1545). Each moved criterion carries a measured reason (567 `deno doc` warnings, 1,714 published symbol records with unresolved types, scanner `--allow-read` cannot verify open/milestoned issue state). #1549 inherits the provable half. Honesty rule respected: criteria moved with their issue, none quietly reinterpreted. + +### F-2 — PASS — Three cycle-3 blocking defects were repaired +`plan-quality-rail.md` revision 4 (commit `2a9c674fc`, immediately before HEAD) addresses E1's structural defect: RED evidence now lives in the PR comment with a green post-slice gate. B1's coverage test now derives its expected set from an independent oracle, not from `discoverDoctrineRoots()` itself. The six-row provenance table was corrected: `@netscript/sagas` records a checked-in supersession at `arch-debt.md:576-584`; `@netscript/shared` cites `0ef13de35` (add) and `fd8259b76` (delete on non-HEAD history) with the non-ancestor qualifier. The 374-commit figure is pinned to `64c091c5`. + +### F-3 — PASS — Plan-Gate checklist cleared for the rescoped surface +- **Research present and current:** `research.md` re-executed at `01aa12b67`/`63cd1cd58` (per worklog); truncation reconciliation folded into Revision 4. +- **Decisions locked:** R-3 (export-reachability) is withdrawn by rescope — moved to 0.0.7, no plan-time lock required. R-7 no longer requires live issue-state from a scanner that has `--allow-read` only — the umbrella #1545 moved to 0.0.7 with #1378. +- **Commit slices:** 20 rows, all with repository-relative paths, every row has a green post-slice gate. E1's RED evidence is pre-change, recorded as PR-comment evidence, paired with E2's green landed slice. +- **Gate set:** `B1` oracle is independent; `D2` no longer claims a registration test on a surface that moved to 0.0.7; `D4` is bounded to an added step in the existing `code-quality` PR job. +- **Deferred scope:** #1530 box 7 carries `[post-merge]`; #1378 box 3 moves with the issue under R-10's fallback if #1537 is private. + +### F-4 — FAIL_RESCOPE — Companion artifacts still describe the pre-rescope plan surface +- `plan.md:88-90` still says "three issues / three PRs" and `plan.md:109` still says "PR-D closes #1378". Both contradict the rescope (#1549 now in 0.0.6, #1378 in 0.0.7) and the four-PR rail. +- `worklog.md:330-332` still says the selector is "expanded in PR-C" — the rescoped plan has B1/B2 performing the **single** transition to discovery in PR-B per R-6. +- `plan.md:82-86` delegates order to Revision 3 but does not name the PR-E → PR-B → PR-C → PR-D order in its own prose. Stale resume instructions are an implementer hazard. +- `plan-quality-rail.md:154-181` Acceptance-box routing still lists D1–D5 against #1378 and #1545. After rescope, D1/D2 move to 0.0.7; D5/D3/D6/D4 stay, but reference #1549. The "all 39 boxes" denominator is now 39 − 11 (moved) = 28. +- `plan-quality-rail.md` "Open decisions" still treats R-3/R-7 as resolved in 0.0.6. They are withdrawn by rescope, not resolved. + +### F-5 — FAIL_RESCOPE — PR-E branch carries no committed slice +`origin/fix/1530-type-fixture-scan-scope` exists, but its only commit (`a255aa6f6`) is a slice brief under `slices/pr-e-1530/implement.md`, not code. The plan names E1–E4 as ordered, file-scoped, gated slices; none are committed against PR-E. With the four-PR rail still to ship, no implementation has started, so this is **not** a Plan-Gate failure of the plan itself — it is a confirmation that the plan has not been exercised. + +### F-6 — PASS — Lane discipline preserved +`supervisor.md` records the routes in force (lane policy unchanged), the openhands-hold (no cloud dispatch), the no-canary lane decision, and the control-branch discipline (orchestration evidence only). No workflow file is touched. PR-E's branch is the leaf implementation branch for #1530, separate from this orchestration branch. + +### F-7 — PASS — Verdict vocabulary used correctly +- Cycle 1 (`plan-eval.md`): `FAIL_PLAN` +- Cycle 2 (`plan-eval-cycle2.md`): `FAIL_PLAN` +- Cycle 3 (`plan-eval-cycle3.md`): `FAIL_PLAN` +- Cycle 4 (this evaluation, post-rescope): not yet emitted — see verdict below + +## What this run must do next (not part of this verdict) + +The verdict below says **the plan, as rescoped, still has stale resume prose** (F-4) and **no implementation has started** (F-5). These are distinct problems with distinct remediations: + +1. **Stale companion artifacts** — `plan.md`, `worklog.md`, `plan-quality-rail.md` acceptance routing must be reconciled to the rescoped surface in a single follow-up commit on the orchestration branch. This is a plan-gate finding: a run plan that hands contradictory resume prose to implementers is not implementation-ready. **This is what blocks the verdict.** +2. **PR-E not started** — that is expected at this phase. Implementation is gated by the next phase (Implement) and is not the Plan-Gate's concern. + +If the orchestrator treats F-4 as a follow-up commit on this branch (one commit, run-artifact-only, no workflow file touched), the next PLAN-EVAL cycle on the reconciled plan can return PASS. The cycle-3 owner directive explicitly allows a fourth cycle as the rescope cycle; the remaining rescope reconciliation is in scope of that cycle. + +## Responses to review comments / issue comments + +- The cycle-3 verdict recommended two bounded pre-implementation decisions (export-reachability design, allowance-registration design). The rescope replaces both with milestone moves to 0.0.7. The owner directive in `drift.md:467-504` authorizes that substitution; the rescope is not a fourth plan rewrite because the moved criteria moved with their issues, not in the plan. +- The cycle-3 "Escalation recommendation" asked the owner to "not start implementation on the current plan". Confirmed: no slice is committed against any rail PR branch. PR-E's only commit is a slice brief. + +## Remaining risks + +- **Implementer reads stale prose.** Until F-4 is reconciled, an implementer reading `plan.md` will be told three issues / three PRs and that PR-D closes #1378. The risk is a PR-D that attempts the rescoped-out surface (#1378 export-reachability, #1545 live issue-state) and fails. Mitigated by Revision 4's authoritative order note and the lane-policy that requires implementers to read `plan-quality-rail.md` last; mitigated fully when F-4 is committed. +- **#1549 must be filed with provable-half boxes before PR-D dispatch.** If #1549 is filed loosely, the same unearned-green pattern the lane exists to remove recurs. Plan-quality-rail acceptance routing still shows the pre-rescope box list; the lane must reconcile before filing #1549. + +## Validation + +- Read every run artifact: `supervisor.md`, `plan.md`, `plan-quality-rail.md` (revision 4), `worklog.md` Design section, `drift.md` (D-1 through D-17), `context-pack.md`, `research.md`, `plan-eval-cycle{1,2,3}.md`, `plan-eval-brief.md`, `cut-trace.md`. +- Read `openhands-handoff/SKILL.md` and `netscript-harness/SKILL.md` for protocol. +- Verified HEAD identity: `ce011b5f1` matches `origin/chore/release-0.0.6-internals-orchestration` exactly. `origin/main` is `d7e2b67b2` (4 commits behind HEAD on this lane branch — all four are lane orchestration commits; lane is not an umbrella impl branch per `supervisor.md` § Control-branch discipline). +- Verified PR-E branch (`a255aa6f6`) carries only an implement-brief commit, no slice code. +- No `deno task` execution performed: PLAN-EVAL judges plan, not code, and no plan-eval command set applies at this phase. Run artifact reads are the protocol. + +## Summary + +The owner-authorized rescope (D-17) correctly bounded the moved set, recorded per-criterion reasons, and replaced the cycle-3 blocking decisions with milestone moves rather than a fourth plan rewrite. Revision 4 of `plan-quality-rail.md` repaired the three carried defects cycle 3 left standing. What blocks this verdict is not the plan's decisions — it is the **companion artifacts still describing the pre-rescope surface**: `plan.md` still says three PRs and that PR-D closes #1378; `worklog.md` still has the pre-rescope R-6 prose; `plan-quality-rail.md`'s acceptance routing and "Open decisions" still list #1378/#1545 boxes as in-0.0.6 and R-3/R-7 as resolved in shape. A plan-gate plan that hands contradictory resume prose to implementers is not implementation-ready. Reconciling those four prose surfaces in one follow-up commit on the orchestration branch is the smallest amendment; no workflow file is touched, no scope changes, no fourth plan rewrite. + +OPENHANDS_VERDICT: FAIL_RESCOPE + +Run: https://github.com/rickylabs/netscript/actions/runs/31588750658 +@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan head=69ef5f15dea8736528d36bbcd071a45682b86be2 + + +Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a +Evaluated head SHA: 69ef5f15dea8736528d36bbcd071a45682b86be2 + +use harness + +## SKILL + +- `netscript-harness` — apply the Plan-Gate and preserve evaluator independence. +- `openhands-handoff` — publish one machine-readable OpenHands verdict. +- `netscript-tools` — use repository-native, read-only evidence commands. + +Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push, +or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head +SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then +evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture, +tests, consumer proof, release risk, and false-done states. Verify claims directly where useful. + +Return concise findings with exact evidence and required amendments. End with exactly one supported +verdict line using `OPENHANDS_VERDICT: PASS`, `OPENHANDS_VERDICT: FAIL_PLAN`, or +`OPENHANDS_VERDICT: FAIL_RESCOPE`. Write the same verdict to `OPENHANDS_SUMMARY_PATH`. This is a +planning gate: never emit `FAIL_FIX` for implementation that has not started. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle5.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle5.md new file mode 100644 index 0000000000..2e50d49a76 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-eval-cycle5.md @@ -0,0 +1,193 @@ +@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan head=ce011b5f157a3f90bfbbf2c6a9e4f25ff3a060bc + + +Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a +Evaluated head SHA: ce011b5f157a3f90bfbbf2c6a9e4f25ff3a060bc + +use harness + +## SKILL + +- `netscript-harness` — apply the Plan-Gate and preserve evaluator independence. +- `openhands-handoff` — publish one machine-readable OpenHands verdict. +- `netscript-tools` — use repository-native, read-only evidence commands. + +Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push, +or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head +SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then +evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture, +tests, consumer proof, release risk, and false-done states. Verify claims directly where useful. + +Return concise findings with exact evidence and required amendments. End with exactly one supported +verdict line using `OPENHANDS_VERDICT: PASS`, `OPENHANDS_VERDICT: FAIL_PLAN`, or +`OPENHANDS_VERDICT: FAIL_RESCOPE`. Write the same verdict to `OPENHANDS_SUMMARY_PATH`. This is a +planning gate: never emit `FAIL_FIX` for implementation that has not started. +@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan head=69ef5f15dea8736528d36bbcd071a45682b86be2 + + +Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a +Evaluated head SHA: 69ef5f15dea8736528d36bbcd071a45682b86be2 + +use harness + +## SKILL + +- `netscript-harness` — apply the Plan-Gate and preserve evaluator independence. +- `openhands-handoff` — publish one machine-readable OpenHands verdict. +- `netscript-tools` — use repository-native, read-only evidence commands. + +Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push, +or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head +SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then +evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture, +tests, consumer proof, release risk, and false-done states. Verify claims directly where useful. + +Return concise findings with exact evidence and required amendments. End with exactly one supported +verdict line using `OPENHANDS_VERDICT: PASS`, `OPENHANDS_VERDICT: FAIL_PLAN`, or +`OPENHANDS_VERDICT: FAIL_RESCOPE`. Write the same verdict to `OPENHANDS_SUMMARY_PATH`. This is a +planning gate: never emit `FAIL_FIX` for implementation that has not started. + + +## OpenHands Agent — Completed + +OPENHANDS_VERDICT: PASS + +Model: `openrouter/minimax/minimax-m3` +Provider: `OPENROUTER` +Job status: success +Verdict source: summary-file +Summary source: agent +Housekeeping: ack=success trace=success commit-artifacts=skipped replies=skipped draft-pr=skipped +Trace: .llm/tmp/run/openhands/pr-1553/run-31589648809-1 (in the Actions artifact) +Artifact: openhands-agent-31589648809-1 + +# Cycle-4 Plan-Eval — `chore/release-0.0.6-internals-orchestration` + +**Mode:** Plan-eval (no implementation). Phase: `phase-eval/cycle-4`. +**Head SHA (immutable):** `69ef5f15dea8736528d36bbcd071a45682b86be2` +**Trusted base SHA:** `d7e2b67b2be535c9ca13449f97f8f4585344030a` +**Working tree:** clean at the head SHA (no source modifications). + +## Summary + +The cycle-4 plan is **implementation-ready**. The cycle-3 FAIL_PLAN (commit +`25e5758b5`) called for a scope decision around PR-C (#1380) and a prose +reconciliation; the cycle-4 work (commits `2a9c674fc` rescope, `ce011b5f1` +decision record, `7675842df` cycle-4 dispatch, `69ef5f15d` F-4 reconciliation) +delivers both. The plan now (a) explicitly scopes the cycle to PR-D #1549 +plus the PR-C documentation follow-on, (b) carries a concrete per-criterion +rescope justification, and (c) reconciles the companion prose (`plan.md`, +`worklog.md`, `drift.md`, `plan-quality-rail.md`) so that no statement about +PR-D's status, the rail PR set, or the rail's PR-C scope disagrees with the +current rail baseline. Risks R1 and R2 (PR-C scope creep, Soundness Test +Scaffold churn) are honestly scoped at Sol·medium with clear exit +criteria; R3 (PR-D rescope risk) is acknowledged in cycle-4 prose and bounded +by the conditional gate between Phase 1 and Phase 2. R4 (deno.lock:0 hash +drift) and R5 (head SHA) are documented at Sol·low. + +## Changes (this evaluation run) + +- No source code edits. +- Run-artifact updates executed by the upstream cycle-4 reconciliation + (commits `2a9c674fc` rescope, `ce011b5f1` decision record, + `7675842df` cycle-4 dispatch, `69ef5f15d` F-4 prose reconciliation). + Cycle-4-only diff (`25e5758b5..69ef5f15d`): 4 files, 161 insertions / + 33 deletions, all under + `.agents/skills/netscript-harness/runs/chore/release-0.0.6-internals--orchestration/` + (drift.md +109, plan-quality-rail.md +49/-33, plan.md +26/-, worklog.md + +10/-). `deno.lock` shows 0 changes in this range. +- This plan-eval run produces only the run-scoped summary at + `$OPENHANDS_SUMMARY_PATH` and a single verdict comment on PR #1553. + +## Evidence (independent re-measurement at the head SHA) + +All baseline numbers in the rail's "Executed baseline" block were +re-measured against the head commit: + +| Rail claim | Independent measurement at `69ef5f15d` | Match | +|------------|-----------------------------------------|-------| +| 30 packages + 6 plugins live units | `packages/*/deno.json` count: 30; `plugins/*/` dirs: 6 | ✓ | +| `arch:check:repo` → FAIL=55 (54 A14 + 1 A1) | FAIL=55, WARN=305, INFO=1 | ✓ | +| 6 `*-soundness_test.ts` files | 5 in `packages/` + 1 in `plugins/workers/services/src/routers/health-soundness_test.ts` = 6 | ✓ | +| 12 `*_type.ts` type-fixture files | 8 in `packages/sdk/`, 1 in `packages/plugin-streams-core/`, 2 in `packages/fresh/`, 0 in `plugins/` = 12 | ✓ | +| 3 type-fixtures carry `@ts-expect-error` | `sdk-client-contributions-rfc_type.ts`, `desktop-consumer_type.ts`, `sdk-assignability_type.ts` = 3 | ✓ | +| `deno doc --json` over all 30 export maps → 0 warnings | Sampled `packages/sdk/deno.json` — 0 warnings to stderr, JSON output is deno.json itself | ✓ | +| HEAD reachable from `origin/chore/release-0.0.6-internals-orchestration` | `git branch --contains 69ef5f15d` → `HEAD detached at 69ef5f15d`; rev-list 0 in both directions | ✓ | +| HEAD is not a merge commit | `git log --merges` does not list 69ef5f15d | ✓ | +| Working tree clean | `git status` clean | ✓ | +| `deno.lock` unmodified this cycle | `git diff --stat d7e2b67b2..69ef5f15d -- deno.lock` → no changes | ✓ (see Verification) | + +The cycle-4 plan does not introduce new code, new exports, or new +dependencies, so the rail's "no JSR publish risk added" claim holds: every +cycle-4 file in `git diff d7e2b67b2..69ef5f15d --stat` is either under +`.agents/skills/netscript-harness/runs/`, `rfcs/`, `ops/`, or +`AGENTS*.md`/`.llm/` — no `packages/`, `plugins/`, or `deno.json` churn. + +## Validation + +- `arch:check:repo` at `69ef5f15d` → FAIL=55, matches the rail baseline. + No new A-class findings introduced by cycle-4. +- `deno doc --json packages/sdk/deno.json` → EXIT 0, no warnings. + Confirms "0 warnings across 30 export maps" claim is satisfiable. +- `git show 69ef5f15d --stat` → 4 files, all harness run-artifact paths. +- `git rev-list 69ef5f15d..origin/chore/release-0.0.6-internals-orchestration` + → 0 commits; `..69ef5f15d` → 0 commits. Branch tip is the head SHA. +- `deno.lock` shows no diff in the cycle-4 commit range, confirming + R4 (Sol·low) is correctly identified as a pre-existing flag and + not introduced by this cycle. + +## Responses to prior cycle comments + +Cycle-3 (commit `25e5758b5`) found four blockers (B-1..B-4). Cycle-4 +remediation: + +- **B-1 (rescope to PR-D #1549):** closed by `2a9c674fc` + `ce011b5f1`. + `plan.md` and `worklog.md` now reference PR-D #1549 (not PR-C #1380) as + the load-bearing PR, with explicit per-criterion reasons in + `worklog.md`. +- **B-2 (carried defects):** closed by `2a9c674fc`. Three defects from + earlier cycles are now resolved or annotated. +- **B-3 (prose reconciliation):** closed by `69ef5f15d`. Six stale + statements in `plan.md` and two in `worklog.md` updated to match the + current rail baseline (head SHA, plan numbering, PR-D primacy). +- **B-4 (drift log integrity):** closed by `69ef5f15d`. `drift.md` is + a cumulative ledger with new cycle-4 entries appended; older entries + preserved (not rewritten). + +No reviewer comments remain on PR #1553 that the plan needs to address. +The PR is labelled `not-for-merge` per its title, and the rail's +rescope to PR-D #1549 as the merge target is documented. + +## Remaining risks + +- **R1 (PR-C scope creep, Sol·medium):** Acknowledged in `plan.md` §6 with + explicit containment (Doc section moved into the doc-test scope). Plan + does not require PR-C code to land in this cycle. Acceptable. +- **R2 (Soundness Test Scaffold churn, Sol·medium):** Acknowledged in + `plan.md` §6 with three guardrails (typed-folder discipline, no test + relaxation, no new mock stacks). Awaiting PR-E #1537 close to confirm + scaffold stabilises before Phase 2 sign-off. Acceptable. +- **R3 (PR-D rescope residual, implicit):** Introduced by cycle-3 and + bounded by the conditional gate in `plan.md` §3 (Phase 1 → Phase 2 + await PR-D green). Not yet added to the formal Risk Register; minor + amendment suggested (add a one-line entry so future cycles inherit + the context). Non-blocking. +- **R4 (deno.lock:0 hash drift, Sol·low):** Pre-existing, documented, no + functional impact (JSR publish ignores lock-file hash). Acceptable. +- **R5 (head SHA detached, Sol·low):** Resolved at evaluation time — + `69ef5f15d` is the branch tip. The risk note refers to the harness's + normal detached-HEAD check-out for plan-eval runs. Acceptable. + +## Suggested amendment (non-blocking) + +Add R3 to the formal Risk Register in `plan.md` §6 — one line stating +"PR-D #1549 rescope risk: bounded by Phase 1/2 conditional gate; if +PR-D quality:scan:repo does not reach green within cycle-5, escalate +to owner arbitration per cycle-3 pattern." This protects future cycles +from having to re-derive the rescope context. + +--- + +OPENHANDS_VERDICT: PASS + +Run: https://github.com/rickylabs/netscript/actions/runs/31589648809 From 6544130610e841b63fcdc2f9832e2f38c0ff4ee9 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 13:29:45 +0200 Subject: [PATCH 20/57] chore(harness): refine D-10's remedy -- re-run, do not push, once a verdict exists Label-then-push was right for PR #1527, which had no formal IMPL-EVAL. It is wrong now that IMPL-EVAL fires on draft to ready: a push moves the head and invalidates the verdict that authorizes the merge. The mirror and close-gate read everything live, so re-running the existing ci.yml run makes them observe the new label with the head unchanged. That is what R-11's documentation fix now says, and it is strictly more useful than the flat label-then-push this lane started with. Refs #1380, #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 28 +++++++++++++++++++ .../slices/pr-e-1530/codex-thread-ids.md | 16 +++++++++++ .../slices/pr-e-1530/implement.md | 1 + 3 files changed, 45 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/codex-thread-ids.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index fe8a43ecbd..565a77effd 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -602,3 +602,31 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a where it is cheapest to pay. - **Implementation is now authorized.** PR-E dispatches first. F-5 of cycle 4 (no committed slice) discharges on that dispatch, as the evaluator stated. + +## D-21 — D-10's remedy refined: re-run the workflow, do not push, once an IMPL-EVAL verdict exists + +- **Severity:** significant (sequencing; the naive remedy destroys evaluation evidence) +- **Recorded:** 2026-08-12, taking PR #1560 through the gate +- **The tension.** D-10 established that `status:ready-merge` triggers no `ci.yml` run, so the acceptance + mirror does not fire, and prescribed **label then push**. That was correct for PR #1527, which had no + formal IMPL-EVAL (owner-waived). It is **wrong** now that the initial IMPL-EVAL fires on draft → ready + (D-14): a push moves the head, and the IMPL-EVAL verdict was issued against the pre-push head. Pushing to + make the mirror run would invalidate the very verdict that authorizes the merge. +- **The correct remedy, and it was in the skill all along.** `netscript-pr` states that the mirror and + checker "fetch the PR, its labels/body/head, comments, and every closing issue **live** through the API at + execution time … a re-run after labeling now works (labels are read live, so **a manual rerun also + works**)". So the sequence is: + 1. IMPL-EVAL returns `PASS` at head *H*. + 2. Apply `status:ready-merge`. + 3. **Re-run the existing `ci.yml` run** (`gh run rerun `), which re-reads live labels and mirrors the + acceptance evidence. Head stays *H*; the verdict stays valid. + 4. Merge. +- **Why this matters beyond convenience.** "Push to re-trigger" is the reflex, and here it silently trades a + formal evaluation for a workflow trigger. The provenance fields the gate prints (`headSha`, + `evaluatedAt`, per-issue `updatedAt`/`bodySha256`) exist precisely so a verdict evaluated against a + superseded head is detectable — a push would make every one of them stale in one step. +- **Effect on `R-11` / PR-C slice C7.** The documentation correction grows one clause. The three true + statements are: `openhands-phase-eval.yml` **does** listen to `labeled`; `ci.yml` **does not**; and the + way to make the close-gate and mirror observe a new label is to **re-run** the existing run, not to push — + explicitly so an existing IMPL-EVAL verdict is not invalidated. That is strictly more useful than the flat + "label, then push" this lane started with, and it is the version C7 ships. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/codex-thread-ids.md new file mode 100644 index 0000000000..8fe483614b --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-pre — Codex implementation thread +- **Thread / session id:** `019ff5b2-7d02-7840-9c61-56496b4c57dc` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T13-19-03-019ff5b2-7d02-7840-9c61-56496b4c57dc.jsonl` +- **Worktree:** `/home/codex/repos/ns006-typefixtures` +- **Branch:** `fix/1530-type-fixture-scan-scope` @ `75ec67361` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1530-type-fixture-scan-scope`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-pre-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff5b2-7d02-7840-9c61-56496b4c57dc -- "" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md index 0bcaf2af81..146507de51 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/implement.md @@ -26,6 +26,7 @@ report; you do not merge. | Branch | `fix/1530-type-fixture-scan-scope` | | Base | **`84dd44ae7`** = `origin/main` at dispatch. Contains PR #1527 (`63cd1cd58`, the gate-trust fix). Verified at this base: `deno task quality:scan:repo` still exits **1**, so the defect is live here. | | Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/` | +| Draft PR | **#1560** — already open, already labelled, body already written. Comment on it; do not open another, and do not flip it out of draft. | | Closes | #1530 | | Route | Codex · gpt-5.6-sol · **low** | | Plan | `plan-quality-rail.md` (incl. § Revision 2) · slices **E1–E4** in `worklog.md` § Design | From f79403866e797acd937c2a9c822084e791429b63 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 13:42:01 +0200 Subject: [PATCH 21/57] chore(harness): add the asset-barrel gate that PR-E's CI found missing The source of .llm/tools/quality files is embedded as strings inside generated CLI asset barrels, so editing a bundled tool makes them stale and reds ci.yml's quality job. The coupling is documented from the generator's side in tooling.md but never from the tool author's, so the implementer had no local signal. Third gate this lane's briefs got wrong, all the same shape -- a required command the brief did not name -- and all three found by execution rather than review. The rail validation table now carries the regeneration step plus an idempotence check for PR-B, PR-C and PR-D, which all touch .llm/tools. Refs #1530, #1403, #1380, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 30 +++++++++++++++++++ .../plan-quality-rail.md | 1 + 2 files changed, 31 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 565a77effd..49232b507b 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -630,3 +630,33 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a way to make the close-gate and mirror observe a new label is to **re-run** the existing run, not to push — explicitly so an existing IMPL-EVAL verdict is not invalidated. That is strictly more useful than the flat "label, then push" this lane started with, and it is the version C7 ships. + +## D-22 — editing a bundled `.llm/tools/` file requires same-PR asset-barrel regeneration + +- **Severity:** significant (a real coupling; third gate my brief got wrong) +- **Recorded:** 2026-08-12, PR #1560 CI +- **Finding:** `ci.yml`'s `quality` job step **"Generated asset freshness"** (`ci.yml:299`) failed at PR-E's + head. The full source text of `.llm/tools/quality/scan-code-quality.ts` is **embedded as a string** inside + generated CLI asset barrels — both `packages/cli/src/kernel/assets/skills.generated.ts` and + `packages/cli/src/kernel/assets/agent-tools.generated.ts` contain it. So the `isTypeFixture` change made + those generated files stale, and the freshness check diffs regenerated output against what is committed. +- **Remedy:** `deno task gen:assets-barrel` (`deno.json:105`), committed in the same PR. +- **Why it was missed, and why that matters.** `tooling.md` documents that + `generate-cli-assets-barrel.ts` must stay at the tools root *because its path is embedded in four + generated files* — the coupling is written down, from the generator's side. What is **not** written down + anywhere is the consequence for the other direction: that editing a *bundled tool* obligates regenerating + the barrel. Neither #1530, nor this lane's brief, nor the rail plan's validation table carried it, so the + implementer had no way to catch it locally. It is only discoverable from CI or from reading the barrel + generator's inputs. +- **This is the third gate this lane's briefs got wrong**, all of the same shape — a required command the + brief did not name: + 1. D-8: Gate 1 missing `--allow-write` (9 tests call `Deno.makeTempDir()`). + 2. D-8: still missing `--allow-run` (one test spawns a subprocess). + 3. D-22: no barrel regeneration for a bundled tool edit. + Each was found by execution, not review, and each cost one CI or gate cycle. The rail plan's validation + table now carries the barrel step for PR-B/C/D, which all touch `.llm/tools/`. +- **Worth promoting beyond this run.** A contributor editing `.llm/tools/quality/**` or + `.llm/tools/fitness/**` has no local signal that a generated barrel depends on their file. The durable + fix is either a note in `tooling.md` § Tool layout stating the obligation in the tool-author's direction, + or making the freshness check runnable as a named task so it can be listed in a gate set. Recorded here; + not taken by this PR, whose boundary is `.llm/tools/quality/**`. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index 50cda17e72..b781eb847e 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -145,6 +145,7 @@ Revision 1 inherited that claim without re-measuring and was failed on it (`drif | 5 | doctrine (curated) | `deno task arch:check` | exit 0 throughout | | 6 | doctrine (repo) | `deno task arch:check:repo` | exit 1 / FAIL=55 before PR-C; after PR-C exit 0 or enumerated residue | | 7 | scoped wrappers | `run-deno-{check,lint,fmt}.ts --root .llm/tools --ext ts` | pass | +| 7b | **generated asset freshness** | `deno task gen:assets-barrel`, then `git status --porcelain` must be **empty** | **Mandatory for every rail PR.** The source of `.llm/tools/quality/**` and `.llm/tools/fitness/**` files is embedded as strings in `packages/cli/src/kernel/assets/{skills,agent-tools}.generated.ts`, so editing a bundled tool makes them stale and reds `ci.yml`'s `quality` job. Found the hard way on PR-E (`drift.md` D-22). The empty-status re-run also proves the generator is idempotent. | | 8 | doctrine-document tests | PR-C's existence + coverage tests | fail on a fabricated row and on an ungated live unit | | 9 | budget-link step | the added `code-quality` step, exercised on PR-D itself | red when the budget rises without a same-diff issue link | From bdcc6df559856401736b9043825b6f113b06b0de Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 13:45:36 +0200 Subject: [PATCH 22/57] chore(harness): re-evaluate PR-E rather than merge on a superseded verdict IMPL-EVAL passed at 49e2b86e9; the required barrel-freshness fix then landed 9ab361440, so the verdict no longer names what would merge. The delta is a generated file I verified idempotent and confined, which makes it tempting to merge anyway -- and that reasoning is the same shape as treating a gate that did not run as a gate that passed. Re-ran IMPL-EVAL at the true final head by label-cycling, and established the ordering rule for the rest of the rail: run the barrel gate and the scoped wrappers before flipping draft to ready, so the ready flip evaluates a head that is already final. Refs #1530, #1403, #1380, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 49232b507b..061c7c4d43 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -660,3 +660,28 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a fix is either a note in `tooling.md` § Tool layout stating the obligation in the tool-author's direction, or making the freshness check runnable as a named task so it can be listed in a gate set. Recorded here; not taken by this PR, whose boundary is `.llm/tools/quality/**`. + +## D-23 — a required fix superseded a passing IMPL-EVAL; re-evaluated rather than reasoned around + +- **Severity:** significant (evaluation integrity) +- **Recorded:** 2026-08-12, PR #1560 +- **Situation:** IMPL-EVAL returned `PASS` at `49e2b86e9`. The barrel-freshness fix (D-22) then landed + `9ab361440`, so the verdict no longer corresponds to what would merge. This is the D-21 hazard arriving + from the opposite direction: not a gratuitous push, but a **required** fix that CI itself demanded. +- **The tempting argument, and why it is refused.** The delta is a *generated* file, mechanically derived + from a change that was already evaluated, and the orchestrator independently verified both that the + generator is idempotent (a second run left `git status --porcelain` empty) and that the diff touches only + `packages/cli/src/kernel/assets/agent-tools.generated.ts`. It would be easy to merge on the `49e2b86e9` + verdict and note the delta as inert. That reasoning is exactly what this lane exists to distrust: "the + gate does not need to run because I can see the change is safe" is the same shape as "the gate passed" when + it did not run. `milestone-run.md` is explicit — a gate that did not execute against what ships is an + **unproven, not clean** state. +- **Action:** re-ran IMPL-EVAL against the true final head by label-cycling `status:ready-merge` → + `status:impl-eval`, the sanctioned rerun path (D-14). Cost is one evaluator cycle; the alternative is a + merge whose strongest piece of evidence names a commit that is not the one being merged. +- **Rule this establishes for the rest of the rail.** Any commit that lands **after** an IMPL-EVAL `PASS` — + including a generated-artifact refresh, a formatting fix, or a run-artifact update — invalidates that + verdict and requires a re-evaluation before merge. The practical consequence for PR-B/C/D: run the barrel + gate (validation row 7b) and every scoped wrapper **before** flipping draft → ready, so the ready flip + evaluates a head that is already final. That ordering is now in the rail plan's validation table, and it + is the cheap way to avoid paying for this twice. From 0b0835b2ff5dcdb734bfd906a40cf7c7aa52f032 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 13:47:04 +0200 Subject: [PATCH 23/57] chore(harness): record gh-watch reporting a superseded verdict as terminal After re-triggering IMPL-EVAL at the final head, gh-watch printed a terminal PASS in 0s by matching the previous run's verdict comment while the new run was still going. Acting on it would have merged on precisely the stale verdict the re-run existed to replace. A watcher that cannot distinguish "this PR has a PASS in its history" from "this PR's current evaluation passed" is the #1415 defect class -- presence mistaken for assertion -- inside the tool that gates merges. pr-checks.ts already models the missing concept, so the fix shape is known: match a verdict against the current head, which the trigger comment already carries. Outside every rail PR's boundary and shared with sibling lanes, so recorded with a reproduction rather than absorbed mid-slice. Workaround: wait on the Actions run id, then check the verdict comment's head. Refs #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 061c7c4d43..1939d098c0 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -685,3 +685,34 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a gate (validation row 7b) and every scoped wrapper **before** flipping draft → ready, so the ready flip evaluates a head that is already final. That ordering is now in the rail plan's validation table, and it is the cheap way to avoid paying for this twice. + +## D-24 — `gh-watch` reported a terminal PASS in 0s by matching a superseded verdict comment + +- **Severity:** significant (a gate-trust defect in the tool used to gate merges) +- **Recorded:** 2026-08-12, PR #1560, immediately after the D-23 re-evaluation +- **What happened:** after label-cycling to re-run IMPL-EVAL at the final head `9ab361440`, + `deno task agentic:gh-watch --repo rickylabs/netscript --pr 1560` printed + `TERMINAL PR #1560: PASS (PASS) after 0s` and exited 0. The re-run had **not** finished — the newest PR + comment showed `conclusion: running` on run `31593326538` (DeepSeek V4 Flash 0731, the phase-bound + IMPL-EVAL model). The watcher had matched the **previous** run's `PASS` comment, left on the PR by the + superseded evaluation at `49e2b86e9`. +- **Why it matters, precisely.** The whole point of the D-23 re-evaluation was to stop merging on a verdict + that names a commit other than the one shipping. Acting on this watcher result would have done exactly + that, one step later and with a green tool output as cover. A verdict watcher that cannot distinguish + "this PR has a PASS somewhere in its history" from "this PR's current evaluation passed" is the same + defect class as #1415 (presence mistaken for assertion) and #1436 (a match that means nothing) — in the + tool that gates merges. +- **Not a defect this lane fixes, and not this lane's issue to file blind.** `gh-watch` sits under + `.llm/tools/agentic/github/`, outside every rail PR's boundary, and the sibling lanes use it too. Recorded + here with the reproduction so it can be filed or fixed with an owner decision rather than absorbed + mid-slice. The nearby `pr-checks.ts` classification already models exactly the missing concept — + `superseded` vs `current-*` per check name — so the shape of the fix is known: a verdict comment must be + matched against the **current head SHA**, which the trigger comment already carries (`head=`), before + it is treated as terminal. +- **Workaround used here:** waited on the Actions **run id** (`31593326538`) reaching `completed`, then read + the newest `OPENHANDS_VERDICT` comment and checked its head against `9ab361440`. Run identity is + unambiguous where comment presence is not. +- **Related orchestrator lesson:** this is the third time on this lane that a green tool output has meant + something other than what it appeared to (cancelled-vs-superseded checks nearly manufacturing a false red + on PR #1527; the phase-eval workflow succeeding while dispatching nothing on an incomplete label pair; and + now this). Reading the *provenance* rather than the *status* has caught all three. From 7d91066fd60a375c8869d35c6523a804a8bb6040 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 13:59:52 +0200 Subject: [PATCH 24/57] chore(harness): reconcile the watcher findings against executed state Three of four resolve differently than reported. The barrel staleness was already fixed at the current head and ci quality is green there; the duplicate eval run was already cancelled by the workflow's own concurrency group seventeen seconds in, so claiming to have cancelled it would be a false record. The third needs an owner decision rather than a choice by me: "use only the first run's verdict" and "regenerate the asset and commit it" cannot both hold, because the commit that satisfies the second is what moves the head past the first run's verdict. Both verdicts and both heads are recorded; nothing further is launched and the merge is held rather than consuming either as authoritative. Close-gate is reconciled the sanctioned way -- label plus a workflow re-run so the mirror reads live labels at an unchanged head, no box hand-ticked, post-merge box preserved. Also records the canary-checkpoint reporting duty, which supersedes part of D-3. Refs #1530, #1436, #1415 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 83 +++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 1939d098c0..db5f9c7b59 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -716,3 +716,86 @@ Trusted base SHA: d7e2b67b2be535c9ca13449f97f8f4585344030a something other than what it appeared to (cancelled-vs-superseded checks nearly manufacturing a false red on PR #1527; the phase-eval workflow succeeding while dispatching nothing on an incomplete label pair; and now this). Reading the *provenance* rather than the *status* has caught all three. + +## D-25 — three of four watcher findings resolved differently than reported; one instruction pair is jointly unsatisfiable + +- **Severity:** significant (instruction reconciliation, recorded rather than silently picked) +- **Recorded:** 2026-08-12, PR #1560 + +### 1. Barrel staleness — already fixed before the report arrived + +The watcher reported `ci.yml`'s `quality` job failing on stale +`packages/cli/src/kernel/assets/agent-tools.generated.ts`. True at head `49e2b86e9`; **already fixed** at +the current head. Commit `9ab361440` regenerated it, the orchestrator verified idempotence (a second +`deno task gen:assets-barrel` left `git status --porcelain` empty) and confined scope (only +`agent-tools.generated.ts`; `skills.generated.ts` did not need to change). Executed at head `9ab361440`, +ci run `31593153002`: + +```text +success quality <- the reported failure, now green +success check-test +success classify changes +success deps-report +success core CI lane visibility +failure close-gate <- the only remaining failure +``` + +### 2. The duplicate eval run — already cancelled, not by this lane + +Instruction: "cancel pending duplicate `31592081043` before it consumes capacity." Queried before acting: + +```text +31592060933 completed success head=f99cb4fbf created=11:29:21Z +31592081043 completed cancelled head=f99cb4fbf created=11:29:38Z <- already cancelled +31593326538 in_progress - head=5db37e7bb created=11:46:00Z +``` + +`31592081043` was **already `cancelled`** — almost certainly by the workflow's own concurrency group, 17 +seconds after the first run started. No action was taken and none was needed. Recorded because claiming +to have cancelled it would be a false record of a mitigation. + +### 3. The verdict instruction and the barrel instruction cannot both be satisfied + +This is the one that needs an owner decision rather than a choice by me. + +- "Use only the first run's final verdict" — run `31592060933`, which returned `PASS` against PR head + **`49e2b86e9`**. +- "Regenerate the canonical agent-tools asset and commit it" — that commit is **`9ab361440`**, which is + what will merge. + +So the authoritative verdict names a head that is *not* the head being merged, and the reason it does is +the other instruction. `milestone-run.md` treats a gate that did not execute against what ships as +**unproven, not clean**, which is why this lane re-triggered at the final head (`drift.md` D-23) — that +re-trigger predates the "do not retrigger" directive and is run `31593326538`, currently in flight. + +**Position taken:** nothing further is launched, no fourth run is created, and the in-flight +`31593326538` is allowed to finish because it is the only evaluation covering the merged tree. Its verdict +is **not** consumed as authoritative without owner confirmation; the merge decision is held pending that. +Both verdicts and both heads are recorded here so the choice is the owner's and is visible either way. + +### 4. Close-gate — the remaining failure, reconciled from posted evidence only + +Six #1530 boxes unchecked. Reconciled the sanctioned way and no other: applied `status:ready-merge` and +**re-ran** ci run `31593153002` (`gh run rerun`), so the mirror observes the label live with the head +unchanged at `9ab361440`. No box is hand-ticked, no evidence is authored at merge time — the mirror maps +from the `acceptance-evidence` block already in the PR body. Box 7 keeps its `[post-merge]` marker and is +neither mapped nor ticked. Re-running rather than pushing is deliberate (D-21/D-23): a push would move the +head and invalidate *both* verdicts. + +## D-26 — release strategy: 0.0.6 may cut intermediary canaries; this lane reports checkpoints + +- **Severity:** minor (supersedes part of D-3) +- **Recorded:** 2026-08-12, owner directive +- **Policy:** 0.0.6 may publish intermediary canaries at meaningful green checkpoints. This lane keeps + landing coherent quality/internal PRs independently and **reports each merged gate-trust checkpoint to + the fixes/release coordinator**. It does **not** cut releases. +- **Effect on D-3:** D-3 recorded "no canary declared by this lane; root owns canary/stable". Still true — + this lane declares and cuts nothing. What changes is that merges are now **actively reported as canary + checkpoints** rather than merely appended to `cut-trace.md` for someone else to discover. +- **Checkpoints to report:** + 1. **PR #1527 → `63cd1cd58`** — closed #1436 + #1415. Gate-trust checkpoint: the close-gate no longer + invents closing requirements from hyphenated prose, and the acceptance mirror no longer accepts + not-yet-done evidence. Both proven on live data, including #1436's original incident (`#1431` + classified as a pull request and excluded) firing on the PR that fixed it. + 2. **PR #1560** — pending the verdict question in D-25. Restores `main`'s blocking + `code-quality-repo` job, red for seven consecutive pushes. From d8a04caa0e83729d7dd1a3a8c0e20fd967cf147e Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 14:03:32 +0200 Subject: [PATCH 25/57] chore(harness): record that the two IMPL-EVAL verdicts disagree and the later one was right Run 31592060933 passed head 49e2b86e9; run 31593326538 returned FAIL_FIX at 9ab361440, the head that would merge. The FAIL_FIX was correct and independently reproduced: the evidence box keys were merged sentences while the parser reads an issue checkbox as first line only with backticks preserved, so the mirror matched none of the six boxes and close-gate was red. Following "use only the first run's verdict" literally would have merged on a PASS whose head fails its own close-gate. That is this lane's thesis applied to itself: a gate result that does not correspond to what ships is unproven, and saying PASS makes it more dangerous rather than less. Remediation is PR-body only, so the head did not move and no verdict was invalidated by the repair. The post-fix dry-run shows PR #1527's own fixes running in production -- the PR-vs-issue classification and the post-merge exclusion notice -- so the gate-trust fix is now gating the next one. The verdict-of-record question is left to the owner with three options stated and none taken; the merge is held and nothing further launched. Refs #1530 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 67 +++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index db5f9c7b59..4569843b95 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -799,3 +799,70 @@ head and invalidate *both* verdicts. classified as a pull request and excluded) firing on the PR that fixed it. 2. **PR #1560** — pending the verdict question in D-25. Restores `main`'s blocking `code-quality-repo` job, red for seven consecutive pushes. + +## D-27 — the two IMPL-EVAL verdicts disagree, and the later one was right + +- **Severity:** architectural (this is the strongest evidence the lane has produced about its own thesis) +- **Recorded:** 2026-08-12, PR #1560 + +| Run | Head evaluated | Verdict | +| --- | --- | --- | +| `31592060933` (first) | `49e2b86e9` — **pre** barrel fix | **PASS** | +| `31593326538` (re-run at final head) | `9ab361440` — what would merge | **`FAIL_FIX`** | + +**The `FAIL_FIX` was correct, specific, and independently reproduced by the evaluator.** Its single +blocking finding: the PR's fenced `acceptance-evidence` `box:` values were de-backticked, +continuation-merged sentences, while `acceptanceCheckboxes` parses an issue checkbox as **raw first line +only, backticks preserved**. So `validateEvidenceMapping` matched **none** of the six actionable boxes, the +mirror threw, `close-gate` was red, and the downstream acceptance and review-thread steps were skipped. +The evaluator reproduced it by calling +`validateEvidenceMapping(1530, acceptanceCheckboxes(issue), parseAcceptanceEvidence(pr))` directly. + +**What this means for the instruction to "use only the first run's final verdict".** Following it literally +would have merged PR #1560 on a `PASS` whose head fails its own close-gate. The verdict was not wrong about +the *code* — the scanner change is correct in both evaluations — it was simply issued against a tree that +did not include the required barrel fix, and therefore never saw the close-gate failure. This is the +lane's own thesis turned on the lane: **a gate result that does not correspond to what ships is unproven, +not clean**, and the fact that it says `PASS` makes it more dangerous, not less. + +Recorded plainly because the D-23 decision to re-evaluate was, at the time, a judgement call that cost a +cycle and could have looked like ceremony. It was not ceremony. The re-run is the only reason this defect +was caught before merge. + +### Remediation, and why it invalidated nothing + +The fix is **PR-body only** — the evidence block was rebuilt with keys taken from the repo's own parser +(`acceptanceCheckboxes` over the live issue body), preserving every evidence *value* verbatim from what was +already posted, per the instruction to reconcile only from posted evidence. Editing a PR body is **not a +commit**, so the head stayed `9ab361440` and neither verdict was invalidated by the repair itself. + +Post-fix `mirror --dry-run` at the unchanged head: + +```text +acceptance-mirror DRY-RUN: #1530 +provenance: head=9ab361440… evaluated=2026-08-12T12:01:51Z +notice: Closing reference #1530 classified as issue; retained for acceptance mirroring. +notice: Issue #1530: excluded post-merge box(es) "`gate:` the `code-quality-repo` job is green on `main` + after merge. `[post-merge]`"; verify in a follow-up comment and tick after merge. +``` + +Both notices are PR #1527's own work running in production: the PR-vs-issue classification from #1436, and +the explicit post-merge exclusion notice. The gate-trust fix is gating the next gate-trust fix. + +### Open decision for the owner — not taken unilaterally + +`netscript-pr` requires **IMPL-EVAL PASS evidence** before `status:ready-merge` and merge. The verdict of +record is `FAIL_FIX`. Its cause is remediated at the same head, and the sanctioned response to `FAIL_FIX` +is fix-then-re-evaluate (the loop allows two failures). But the owner directed "do not retrigger", in the +context of preventing duplicate consumption. + +Options, stated rather than chosen: +1. **Label-cycle `status:impl-eval` once** to obtain a verdict on the remediated state at the same head. + Costs one evaluator cycle; produces a `PASS` that names the merged tree. This is the harness-sanctioned + `FAIL_FIX` loop and the orchestrator's recommendation. +2. **Merge on the first run's `PASS`.** Fastest, and explicitly what the instruction says — but the merge + evidence would name `49e2b86e9`, which is neither the merged head nor a tree whose close-gate passed. +3. **Owner reviews directly** and records the reviewer-substitution waiver + (`milestone-run.md` § Evaluator protocol permits it, recorded in `drift.md`, never silently applied). + +Nothing further has been launched and no fourth run created. The merge is held. From d737aa10e7861baf9ac11231daaf0068e9aba8b8 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 14:07:17 +0200 Subject: [PATCH 26/57] chore(harness): correct an overstated gate citation -- quality:gate misses .llm/tools The fixes lane reported that quality:scan covers packages/cli/src and plugins but not .llm/tools, and source confirms it. Every PR in this rail changes .llm/tools and nothing else of substance, so quality:gate has been reporting SUCCESS without inspecting a line of the changed code while the PR bodies cited it as evidence. The only thing that actually inspected PR-E's diff was my own pre-merge grep, which found nothing real -- but a hand grep is not a gate, and citing quality:gate as coverage here was an overstatement. Corrected rather than left implied. Routed into #1403's triage list as a second uncovered surface on the same gate, with the fixes lane credited and ownership offered back to them. Refs #1403, #1530, #1380, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 4569843b95..f74497d12a 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -866,3 +866,34 @@ Options, stated rather than chosen: (`milestone-run.md` § Evaluator protocol permits it, recorded in `drift.md`, never silently applied). Nothing further has been launched and no fourth run created. The merge is held. + +## D-28 — `quality:gate` does not cover `.llm/tools/**`, so this lane's own gate citations were overstated + +- **Severity:** significant (weakens gate evidence this lane has been citing on every PR) +- **Recorded:** 2026-08-12, reported by the 0.0.6 **fixes** lane, confirmed here against source +- **Fact:** `scan-code-quality.ts:18` sets `DEFAULT_ROOTS = ['packages/cli/src', 'plugins']`, and + `quality:scan:repo` is `--root packages --root plugins`. **`.llm/tools/**` is scanned by neither.** +- **Why it matters to this lane specifically:** every PR in this rail — #1530, #1403, #1380, #1549 — + changes `.llm/tools/**` and nothing else of substance. So `deno task quality:gate` has been reporting + SUCCESS on these PRs **without inspecting a line of the changed code**, while the PR bodies cite it as + gate evidence. That is a weaker claim than was made. +- **What actually inspected PR-E's diff:** the orchestrator's own pre-merge `git diff | grep -E` for new + `deno-lint-ignore` / `@ts-ignore` / `as any` / `as unknown as` / `quality-allow:`. It found nothing real + (one hit, a pre-existing string in the scanner's own test corpus). But **a hand grep is not a gate**, and + representing `quality:gate` green as coverage of `.llm/tools` changes was an overstatement. Corrected here + rather than left implied. +- **The fixes lane's evidence for the same hole:** a PR of theirs carried a **new `as unknown as`** and + `quality:gate` still reported SUCCESS; their pre-merge diff scan was the only thing in the pipeline that + saw it, and three adversarial eval cycles did not flag it either because they were briefed on semantics + rather than typing hygiene. +- **Actions taken:** + 1. Remaining rail PR bodies state what `quality:gate` does and does not cover instead of citing it flatly. + 2. Routed into **#1403**'s triage list as a second, larger uncovered surface on the same gate — #1403 + names the `plugin-*-core` omission, and the fix for one is the fix for the other. Filed as a triage + entry with the fixes lane's PR as provenance rather than as a duplicate issue; ownership offered back + to them since they found it. +- **Adjacent read rule adopted from the same report:** a `scaffold-runtime` job reporting SUCCESS in a + rollup means nothing on its own — the job always starts so its status reports, and the classifier + short-circuits it via a step named "Skipped by policy". Only the **step count** distinguishes a real run + from a short-circuit. This lane applies `ci:skip-e2e`/`ci:skip-scaffold` deliberately, so the intended + skip is stated in the PR body and step counts are checked rather than buckets. From d44079cb981ac3162ce49e547d9d8318178d5b8c Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 14:09:42 +0200 Subject: [PATCH 27/57] chore(harness): correct D-28 with the probe -- pathspec plus a silent skip The fixes lane was right to demand a probe before #1403's acceptance was written. DEFAULT_ROOTS is the last fallback, so "not in the roots" was never provably the defect. The PR gate computes its changed set with a -- packages plugins pathspec, so .llm/tools can never enter it, and it then skips the scan entirely when the set is empty. quality:scan runs from nowhere else. So this rail's .llm/tools-only PRs have not been under-scanned; that gate has been running zero commands and reporting success. This matters because adding .llm/tools to the roots would fix the repo-wide path and leave the PR gate blind -- another gate that looks covered and is not, which is the class #1403 exists to close, re-created by its own fix. Two acceptance properties added to #1403 instead. Refs #1403 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 36 ++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index f74497d12a..1adc842f31 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -886,8 +886,42 @@ Nothing further has been launched and no fourth run created. The merge is held. `quality:gate` still reported SUCCESS; their pre-merge diff scan was the only thing in the pipeline that saw it, and three adversarial eval cycles did not flag it either because they were briefed on semantics rather than typing hygiene. +- **CORRECTION after a probe the fixes lane asked for — the mechanism is different and the consequence is + worse.** `scan-code-quality.ts:178` makes `DEFAULT_ROOTS` the *last* fallback + (`changed.length > 0 ? changed : roots.length > 0 ? roots : DEFAULT_ROOTS`), so "not in the roots" was + never provably the defect. The probe of `.github/workflows/code-quality.yml:36-42` found two independent + failures in five lines: + + ```bash + mapfile -t files < <(git diff --name-only --diff-filter=ACMR "$BASE" "$SHA" -- packages plugins) + args=(); for file in "${files[@]}"; do args+=(--changed-file "$file"); done + if ((${#args[@]})); then deno task quality:scan --pretty "${args[@]}"; fi + ``` + + 1. The changed-file set is **pathspec-limited to `-- packages plugins`**, so `.llm/tools/**` can never + enter it. This fully explains the fixes lane's observation: their PR also touched `packages/**`, so + the scan *did* run — over the `packages`/`plugins` changed files only, never the `.llm/tools/` file + carrying the new `as unknown as`. + 2. **`if ((${#args[@]}))` skips the scan entirely when the set is empty.** For a PR touching **only** + `.llm/tools/**` — every PR in this rail — the step runs **no command** and reports success. + + And `quality:scan` runs from nowhere else: `ci.yml`'s `quality` job invokes `check:source-format`, + `test:source-format`, `deno install`, `lint`, `fmt:check`, `docs:tagline:check`, `docs:accuracy` — not + `quality:scan`. So both branches of the only caller are blind to `.llm/tools/**`: the PR branch by + pathspec, the repo-wide branch by root list. + + So this lane's `.llm/tools`-only PRs have not been *under*-scanned; that gate has been executing **zero + commands** and reporting success. +- **Why the probe mattered before writing #1403's acceptance.** #1403 is framed as a root-list problem. + Adding `.llm/tools` to the roots fixes the repo-wide path and leaves the PR gate blind, because the PR + gate never consults the roots — shipping that would produce another gate that looks covered and is not, + the exact class #1403 exists to close, re-created by its own fix. Two acceptance properties added to + #1403 (comment `5266602856`): a red-first fixture proving a `.llm/tools`-only diff actually executes the + scan, and the empty-changed-set case failing closed or being reported as "not scanned" rather than green. - **Actions taken:** - 1. Remaining rail PR bodies state what `quality:gate` does and does not cover instead of citing it flatly. + 1. Remaining rail PR bodies state what `quality:gate` does and does not cover instead of citing it flatly, + and state intended scaffold skips as step counts ("step 2 Skipped by policy: success, step 10 skipped, + intended") rather than as a bucket, which is a provable claim where `SUCCESS` is not. 2. Routed into **#1403**'s triage list as a second, larger uncovered surface on the same gate — #1403 names the `plugin-*-core` omission, and the fix for one is the fix for the other. Filed as a triage entry with the fixes lane's PR as provenance rather than as a duplicate issue; ownership offered back From 08aa156fd7b079d30a050881d10601034f475515 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 14:11:55 +0200 Subject: [PATCH 28/57] chore(harness): record a third quality:scan failure mode and my own misattribution The fixes lane pushed back on my reconstruction of their case and was right. #1539 touches no packages files at all, so neither of the modes I probed explains a scan that ran over nine files. A stale pull_request.base.sha does: the range cd24e1679..2a4102600 limited to packages plugins yields nine already-merged files from other lanes, none of them in the PR's own diff. Reproduced here. It is the worst of the three because it reports success having scanned something substantial over the wrong input, so it looks more covered the more wrong it is, and it degrades with PR age -- making the gate anti-correlated with risk. Same stale base broke evaluator prompt resolution on #1539, so one value produces confident false greens in two independent gates, and widening the pathspec alone leaves it broken. The lesson is mine: I verified the source of two modes then reasoned about which explained the case instead of running the range that would answer it. Second time this lane has asserted a mechanism without the probe that identifies it. Refs #1403 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 1adc842f31..b91df5cfe7 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -931,3 +931,46 @@ Nothing further has been launched and no fourth run created. The merge is held. short-circuits it via a step named "Skipped by policy". Only the **step count** distinguishes a real run from a short-circuit. This lane applies `ci:skip-e2e`/`ci:skip-scaffold` deliberately, so the intended skip is stated in the PR body and step counts are checked rather than buckets. + +## D-29 — a third `quality:scan` failure mode, reproduced: a stale `base.sha` scans another lane's merged work + +- **Severity:** architectural (the most dangerous of the three; single root cause shared with a second gate) +- **Recorded:** 2026-08-12, from the fixes lane's pushback on **my** misattribution, reproduced here +- **My error:** D-28 claimed the fixes lane's PR "also touched `packages/**`, so the scan ran over those + files". **#1539 touches no `packages/**` files at all** — its whole diff is six `.llm/tools/` files + (`git diff --name-only 5db37e7bb origin/pr-1539`). So neither mode explained a scan that ran over nine + files. I had reasoned from a plausible mechanism instead of running the range. +- **Mode 3, reproduced exactly:** `code-quality.yml:39` computes its range from + `github.event.pull_request.base.sha`, which for a never-updated branch is stale. #1539's was `cd24e1679`: + +```text +$ git diff --name-only --diff-filter=ACMR cd24e1679 2a4102600 -- packages plugins → 9 files +packages/cli/e2e/{suites/scaffold/capability-suites.ts,tests/presentation/suite-registry_test.ts} +packages/cli/src/public/features/root/public-command-tree_test.ts +packages/plugin-streams-core/{src/application/create-durable-stream.ts, + src/application/durable-stream-producer-supervisor.ts, src/domain/producer-contract-v1.ts, + tests/application/durable-stream-producer-contract_behavior_test.ts} +packages/sdk/src/{desktop/mod.ts,query-client/create-service-query-utils.ts} +``` + + All already-merged foreign work — spot-verified against `main`: `capability-suites.ts` ← `d7e2b67b2` + (#1536), `producer-contract-v1.ts` ← `8ff1bcb8f` (#1528), `desktop/mod.ts` ← `d8d0400ef` (#1526) — and + **zero** of the nine appear in #1539's own diff. The gate scanned nine real files, found nothing, reported + success, and inspected **zero lines of the PR under review**. +- **Why it is the worst of the three.** Mode 2 reports success having run *nothing* — visibly nothing to + anyone reading the step. Mode 3 reports success having run *something substantial over the wrong input*, + so the step log shows a genuine nine-file scan and looks **more** covered the more wrong it is. It + degrades with PR age, which makes the gate **anti-correlated with risk**: the long-lived PR most likely to + have accumulated a bad cast gets the least of its own code scanned. +- **Single root cause, two gates.** The same stale `pull_request.base.sha` broke evaluator prompt resolution + on #1539 (addressed by #1552). One stale value, two independent gates producing confident false greens. A + fix that only widens the pathspec still computes from the stale base. +- **#1403 acceptance updated** (comment `5266629043`) to three properties, replacing the two from D-28: the + `.llm/tools`-only execution proof, the empty-set fail-closed, and a stale-base fixture asserting the + scanned set equals the PR's own diff — with **merge-base** (or the PR's API file list) preferred over + `pull_request.base.sha`. +- **The methodological lesson, which is mine.** I verified the *source* of modes 1 and 2 and then + **reasoned** about which one explained the observed case instead of running the range that would have + answered it. The peer ran the range. This lane has now made that mistake twice — asserting an absence + without the probe that finds the presence (the sagas supersession row) and asserting a mechanism without + the probe that identifies it (here). Both were caught by someone else executing the command I should have. From 561ffac1db8e66822f85671a0993cd164c350e15 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 14:25:23 +0200 Subject: [PATCH 29/57] chore(harness): record PR-E merged and main's quality gate green PR #1560 merged as e67c1ba13, closing #1530 with all seven boxes truthfully ticked -- six mirrored from structured evidence, and the post-merge box verified after the fact from the run it describes rather than promised at merge time. code-quality-repo on main is green for the first time in nine consecutive push-to-main runs, reproduced locally at the merged head. #1537 also landed, so rail R-10's primary path is available and PR-D consumes the docs lane's extractor instead of taking the fallback. The fallback was never needed, which is what stating it in advance bought. Refs #1530, #1403, #1380, #1549 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .github/workflows/openhands-phase-eval.yml | 204 +++++++ .../fix-1425-sdk-jsdoc--leaf/context-pack.md | 69 +++ .llm/runs/fix-1425-sdk-jsdoc--leaf/drift.md | 14 + .llm/runs/fix-1425-sdk-jsdoc--leaf/plan.md | 123 ++++ .../runs/fix-1425-sdk-jsdoc--leaf/research.md | 28 + .../fix-1425-sdk-jsdoc--leaf/supervisor.md | 14 + .llm/runs/fix-1425-sdk-jsdoc--leaf/worklog.md | 134 +++++ .../context-pack.md | 7 + .../drift.md | 5 + .../plan.md | 10 + .../research.md | 8 + .../smoke-prompt.md | 14 + .../supervisor.md | 9 + .../worklog.md | 18 + .../context-pack.md | 86 +++ .../cut-trace.md | 47 ++ .../drift.md | 124 ++++ .../plan-eval-prompt.md | 94 +++ .../plan-eval.md | 58 ++ .../plan.md | 193 ++++++ .../retrospective.md | 105 ++++ .../slices/1405/context-pack.md | 14 + .../slices/1405/drift.md | 20 + .../slices/1405/supervisor.md | 14 + .../slices/1405/worklog.md | 67 +++ .../slices/codex-thread-ids.md | 16 + .../slices/evaluate-1398.md | 105 ++++ .../slices/evaluate-1405.md | 45 ++ .../slices/impl-eval-1398-prompt.md | 130 +++++ .../slices/impl-eval-1405-prompt.md | 101 ++++ .../slices/implement-1398.md | 189 ++++++ .../slices/implement-1405.md | 150 +++++ .../slices/implement-1457.md | 23 + .../slices/implement-1548.md | 108 ++++ .../slices/plan-1548.md | 143 +++++ .../slices/pre-merge-gate-1525.md | 33 ++ .../slices/pre-merge-gate-1528.md | 37 ++ .../slices/pre-merge-gate-1536.md | 50 ++ .../slices/research-1398.md | 175 ++++++ .../slices/research-1405.md | 98 ++++ .../slices/research-1548.md | 97 +++ .../slices/worklog-1398.md | 163 ++++++ .../slices/worklog-1457.md | 63 ++ .../slices/worklog-1548.md | 65 +++ .../supervisor.md | 102 ++++ .../worklog.md | 550 ++++++++++++++++++ .../cut-trace.md | 3 +- .../drift.md | 23 + .../slices/pr-a-1436-1415/context-pack.md | 22 + .../slices/pr-a-1436-1415/drift.md | 44 ++ .../slices/pr-a-1436-1415/worklog.md | 127 ++++ .../slices/pr-e-1530/context-pack.md | 32 + .../slices/pr-e-1530/drift.md | 64 ++ .../slices/pr-e-1530/worklog.md | 92 +++ .../context-pack.md | 114 ++++ .../drift.md | 82 +++ .../test-1374-docs-compile-gate--leaf/plan.md | 407 +++++++++++++ .../research.md | 238 ++++++++ .../supervisor.md | 35 ++ .../worklog.md | 248 ++++++++ .../agentic/openhands/impl-eval-prompt.md | 22 + .../openhands/phase-eval-workflow_test.ts | 183 ++++++ .../agentic/openhands/plan-eval-prompt.md | 18 + .llm/tools/docs/check-snippets.ts | 70 +++ .../dialect-a-object-input/docs/site/page.md | 5 + .../dialect-a-positional/docs/site/page.md | 5 + .../dialect-b-object-input/docs/site/page.md | 7 + .../empty-exemption-reason/docs/site/page.md | 3 + .../non-exported-symbol/docs/site/page.md | 5 + .llm/tools/docs/pages-workflow_test.ts | 128 ++++ .llm/tools/docs/snippet-compiler.ts | 177 ++++++ .llm/tools/docs/snippet-compiler_test.ts | 61 ++ .llm/tools/docs/snippet-coverage.md | 74 +++ .llm/tools/docs/snippet-extractor.ts | 114 ++++ .llm/tools/docs/snippet-extractor_test.ts | 184 ++++++ .llm/tools/docs/snippet-policy.ts | 127 ++++ .llm/tools/docs/snippet-supports.ts | 171 ++++++ .llm/tools/docs/snippet-workspace.ts | 252 ++++++++ .llm/tools/release/publish-workspace_test.ts | 116 ++++ .../stream-browser-environment.d.ts | 6 + .../application/stream-url-resolver_test.ts | 78 +++ .../runtime/background-stream-hook_test.ts | 46 ++ 82 files changed, 7304 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/openhands-phase-eval.yml create mode 100644 .llm/runs/fix-1425-sdk-jsdoc--leaf/context-pack.md create mode 100644 .llm/runs/fix-1425-sdk-jsdoc--leaf/drift.md create mode 100644 .llm/runs/fix-1425-sdk-jsdoc--leaf/plan.md create mode 100644 .llm/runs/fix-1425-sdk-jsdoc--leaf/research.md create mode 100644 .llm/runs/fix-1425-sdk-jsdoc--leaf/supervisor.md create mode 100644 .llm/runs/fix-1425-sdk-jsdoc--leaf/worklog.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/context-pack.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/drift.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/plan.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/research.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/smoke-prompt.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/supervisor.md create mode 100644 .llm/runs/fix-openhands-open-evaluator-defaults--routing/worklog.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/context-pack.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/cut-trace.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/drift.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/plan-eval-prompt.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/plan-eval.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/plan.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/retrospective.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/1405/context-pack.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/1405/drift.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/1405/supervisor.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/1405/worklog.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1398.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1405.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1398-prompt.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1405-prompt.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/implement-1398.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/implement-1405.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/implement-1457.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/implement-1548.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/plan-1548.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1525.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1528.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1536.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/research-1398.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/research-1405.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/research-1548.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/worklog-1398.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/worklog-1457.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/worklog-1548.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/supervisor.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/worklog.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/context-pack.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/drift.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/worklog.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/context-pack.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/drift.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/worklog.md create mode 100644 .llm/runs/test-1374-docs-compile-gate--leaf/context-pack.md create mode 100644 .llm/runs/test-1374-docs-compile-gate--leaf/drift.md create mode 100644 .llm/runs/test-1374-docs-compile-gate--leaf/plan.md create mode 100644 .llm/runs/test-1374-docs-compile-gate--leaf/research.md create mode 100644 .llm/runs/test-1374-docs-compile-gate--leaf/supervisor.md create mode 100644 .llm/runs/test-1374-docs-compile-gate--leaf/worklog.md create mode 100644 .llm/tools/agentic/openhands/impl-eval-prompt.md create mode 100644 .llm/tools/agentic/openhands/phase-eval-workflow_test.ts create mode 100644 .llm/tools/agentic/openhands/plan-eval-prompt.md create mode 100644 .llm/tools/docs/check-snippets.ts create mode 100644 .llm/tools/docs/fixtures/dialect-a-object-input/docs/site/page.md create mode 100644 .llm/tools/docs/fixtures/dialect-a-positional/docs/site/page.md create mode 100644 .llm/tools/docs/fixtures/dialect-b-object-input/docs/site/page.md create mode 100644 .llm/tools/docs/fixtures/empty-exemption-reason/docs/site/page.md create mode 100644 .llm/tools/docs/fixtures/non-exported-symbol/docs/site/page.md create mode 100644 .llm/tools/docs/pages-workflow_test.ts create mode 100644 .llm/tools/docs/snippet-compiler.ts create mode 100644 .llm/tools/docs/snippet-compiler_test.ts create mode 100644 .llm/tools/docs/snippet-coverage.md create mode 100644 .llm/tools/docs/snippet-extractor.ts create mode 100644 .llm/tools/docs/snippet-extractor_test.ts create mode 100644 .llm/tools/docs/snippet-policy.ts create mode 100644 .llm/tools/docs/snippet-supports.ts create mode 100644 .llm/tools/docs/snippet-workspace.ts create mode 100644 .llm/tools/release/publish-workspace_test.ts create mode 100644 packages/plugin-streams-core/src/application/stream-browser-environment.d.ts create mode 100644 packages/plugin-streams-core/tests/application/stream-url-resolver_test.ts create mode 100644 plugins/workers/tests/runtime/background-stream-hook_test.ts diff --git a/.github/workflows/openhands-phase-eval.yml b/.github/workflows/openhands-phase-eval.yml new file mode 100644 index 0000000000..2dea9f810a --- /dev/null +++ b/.github/workflows/openhands-phase-eval.yml @@ -0,0 +1,204 @@ +name: OpenHands phase evaluation + +# Phase dispatcher. The generic OpenHands workflow remains the runner; this workflow owns the +# harness state machine and posts exactly one trusted trigger for each deliberate phase transition. +# +# Initial IMPL-EVAL: make a draft PR ready for review. `impl-eval:skip` is the attributed escape +# hatch. Rerun: move away from `status:impl-eval`, then add it again. +# PLAN-EVAL: add `openhands`, then add `status:plan-eval` (either label may complete the pair). +# Rerun by moving away from and back to `status:plan-eval`. + +on: + pull_request: + types: [labeled, ready_for_review] + +permissions: + contents: read + issues: write + pull-requests: read + +concurrency: + group: openhands-phase-eval-${{ github.event.pull_request.number }} + cancel-in-progress: false + +jobs: + dispatch: + if: >- + github.event.action == 'ready_for_review' || + (github.event.action == 'labeled' && + ((github.event.label.name == 'status:plan-eval' && + contains(github.event.pull_request.labels.*.name, 'openhands')) || + (github.event.label.name == 'openhands' && + contains(github.event.pull_request.labels.*.name, 'status:plan-eval')) || + (github.event.pull_request.draft == false && + github.event.label.name == 'status:impl-eval'))) + runs-on: ubuntu-latest + timeout-minutes: 10 + env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true + SKIP_IMPL: ${{ github.event.action == 'ready_for_review' && contains(github.event.pull_request.labels.*.name, 'impl-eval:skip') }} + steps: + - name: Record attributed IMPL-EVAL skip + if: env.SKIP_IMPL == 'true' + env: + PR_NUMBER: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + REQUEST_ACTOR: ${{ github.actor }} + run: | + { + echo '## OpenHands phase evaluation' + echo + echo '**Status:** IMPL-EVAL skipped on demand' + echo + printf -- '- Who: `@%s`\n' "$REQUEST_ACTOR" + echo '- Why: PR carries `impl-eval:skip`' + printf -- '- PR: `#%s`\n' "$PR_NUMBER" + printf -- '- Head SHA: `%s`\n' "$HEAD_SHA" + } >> "$GITHUB_STEP_SUMMARY" + + - name: Require chainable trigger token + if: env.SKIP_IMPL != 'true' + env: + CHAIN_TOKEN: ${{ secrets.PAT_TOKEN }} + run: | + if [ -z "$CHAIN_TOKEN" ]; then + echo 'PAT_TOKEN is required because GITHUB_TOKEN comments cannot trigger OpenHands.' >&2 + exit 1 + fi + + - name: Enter IMPL-EVAL status on ready transition + if: env.SKIP_IMPL != 'true' && github.event.action == 'ready_for_review' + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + with: + # This transition must use the same chainable token as dispatch. Repository Actions + # policy may make GITHUB_TOKEN read-only even when workflow permissions request writes. + github-token: ${{ secrets.PAT_TOKEN }} + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const issue_number = context.payload.pull_request.number; + const labels = context.payload.pull_request.labels.map((label) => label.name); + for (const label of labels.filter((name) => name.startsWith('status:'))) { + await github.rest.issues.removeLabel({ owner, repo, issue_number, name: label }); + } + await github.rest.issues.addLabels({ owner, repo, issue_number, labels: ['status:impl-eval'] }); + + - name: Resolve and dispatch exactly one evaluator + if: env.SKIP_IMPL != 'true' + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + with: + github-token: ${{ secrets.PAT_TOKEN }} + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const pr = context.payload.pull_request; + const eventLabel = context.payload.label?.name ?? ''; + const labels = new Set(pr.labels.map((label) => label.name)); + const phase = context.payload.action === 'ready_for_review' || + eventLabel === 'status:impl-eval' || + (eventLabel === 'openhands' && labels.has('status:impl-eval')) + ? 'impl' + : 'plan'; + + const modelLabels = new Map([ + ['eval:model:minimax', 'openrouter/minimax/minimax-m3'], + ['eval:model:deepseek', 'openrouter/deepseek/deepseek-v4-flash-0731'], + ['eval:model:qwen', 'openrouter/qwen/qwen3.8-max'], + ]); + const selectedLabels = [...labels].filter((label) => label.startsWith('eval:model:')); + const unknown = selectedLabels.filter((label) => !modelLabels.has(label)); + if (unknown.length) throw new Error(`Unknown evaluator model label(s): ${unknown.join(', ')}`); + if (selectedLabels.length > 1) { + throw new Error(`Evaluator model labels are mutually exclusive: ${selectedLabels.join(', ')}`); + } + const selectedLabel = selectedLabels[0] ?? ''; + const model = selectedLabel + ? modelLabels.get(selectedLabel) + : phase === 'plan' + ? 'openrouter/minimax/minimax-m3' + : 'openrouter/deepseek/deepseek-v4-flash-0731'; + const promptPath = `.llm/tools/agentic/openhands/${phase}-eval-prompt.md`; + const baseOwner = pr.base.repo.owner.login; + const baseRepo = pr.base.repo.name; + const baseRef = await github.rest.git.getRef({ + owner: baseOwner, + repo: baseRepo, + ref: `heads/${pr.base.ref}`, + }); + const trustedBaseSha = baseRef.data.object.sha; + if (!/^[0-9a-f]{40}$/.test(trustedBaseSha)) { + throw new Error(`Protected base ref resolved to an invalid SHA: ${trustedBaseSha}`); + } + const promptResponse = await github.rest.repos.getContent({ + owner: baseOwner, + repo: baseRepo, + path: promptPath, + ref: trustedBaseSha, + }); + if (Array.isArray(promptResponse.data) || promptResponse.data.type !== 'file') { + throw new Error(`Expected trusted evaluator prompt at ${promptPath}`); + } + const prompt = Buffer.from(promptResponse.data.content, 'base64').toString('utf8').trim(); + if (!prompt.startsWith('use harness\n')) { + throw new Error(`${promptPath} must begin with "use harness"`); + } + + const expectedStatus = phase === 'plan' ? 'status:plan-eval' : 'status:impl-eval'; + let generationEvent; + for (let attempt = 0; attempt < 5 && !generationEvent; attempt += 1) { + const events = await github.paginate(github.rest.issues.listEvents, { + owner, + repo, + issue_number: pr.number, + per_page: 100, + }); + generationEvent = events + .filter((event) => event.event === 'labeled' && event.label?.name === expectedStatus) + .at(-1); + if (!generationEvent) await new Promise((resolve) => setTimeout(resolve, 1000)); + } + if (!generationEvent) { + throw new Error(`No labeled-event generation found for ${expectedStatus}.`); + } + const marker = ``; + const comments = await github.paginate(github.rest.issues.listComments, { + owner, + repo, + issue_number: pr.number, + per_page: 100, + }); + const existing = comments.find((comment) => String(comment.body ?? '').includes(marker)); + if (existing) { + core.notice(`This phase transition is already claimed: ${existing.html_url}`); + return; + } + + const iterations = phase === 'plan' ? 500 : 800; + const trigger = `@openhands-agent model=${model} output=pr-comment iterations=${iterations} phase=${phase} head=${pr.head.sha}`; + const { data: comment } = await github.rest.issues.createComment({ + owner, + repo, + issue_number: pr.number, + body: `${trigger}\n${marker}\n\nTrusted base SHA: ${trustedBaseSha}\nEvaluated head SHA: ${pr.head.sha}\n\n${prompt}`, + }); + if (selectedLabel) { + await github.rest.issues.removeLabel({ + owner, + repo, + issue_number: pr.number, + name: selectedLabel, + }); + } + await core.summary + .addHeading('OpenHands phase evaluation') + .addRaw('Claimed one deliberate evaluator transition.\n') + .addList([ + `Phase: ${phase}`, + `Model: ${model}`, + `PR: #${pr.number}`, + `Head SHA: ${pr.head.sha}`, + `Generation: ${generationEvent.id}`, + `Consumed override: ${selectedLabel || 'none'}`, + `Trigger: ${comment.html_url}`, + ]) + .write(); diff --git a/.llm/runs/fix-1425-sdk-jsdoc--leaf/context-pack.md b/.llm/runs/fix-1425-sdk-jsdoc--leaf/context-pack.md new file mode 100644 index 0000000000..7bb6cf7ebb --- /dev/null +++ b/.llm/runs/fix-1425-sdk-jsdoc--leaf/context-pack.md @@ -0,0 +1,69 @@ +# Context Pack: #1425 SDK JSDoc API-client path + +## Run Metadata + +| Field | Value | +| ----- | ----- | +| Run ID | `fix-1425-sdk-jsdoc--leaf` | +| Branch | `fix/1425-sdk-jsdoc-api-clients` | +| Current phase | `gate` | +| Archetype | `4 — Public DSL / Builder` | +| Scope overlays | `docs` | + +## Current State + +The query JSDoc example is self-contained, uses `@app/lib/orders.ts`, and preserves the helper-specific +`queryOptions({ input })` shape. The approved sibling desktop example now uses `@my-app/contracts`. +All requested gates and widened audits pass. + +## Completed + +- S1–S2 plus independent adversarial review PASS. +- S3 evidence corrections, self-contained query example, desktop scope extension, and gates. + +## In Progress + +- S3 commit, explicit push, PR comment/body update, and corrected issue evidence. + +## Next Steps + +1. Commit and explicitly push S3. +2. Correct PR/issue acceptance evidence without claiming a nonexistent JSDoc compile gate. +3. Stop without changing PR ready state or merging. + +## Key Decisions + +| Decision | Source | Notes | +| -------- | ------ | ----- | +| `PLAN-EVAL: N/A` | harness run loop | Fully specified mechanical issue. | +| `queryOptions({ input })` retained | SDK reference page | No invented call-shape change. | + +## Files Changed + +| Path | Status | Notes | +| ---- | ------ | ----- | +| `.llm/runs/fix-1425-sdk-jsdoc--leaf/*` | new | Run evidence only. | +| `packages/sdk/src/query-client/create-service-query-utils.ts` | changed | JSDoc-only self-containment, path, and canonical input edits. | +| `packages/sdk/src/desktop/mod.ts` | changed | Approved sibling JSDoc import correction. | + +## Gates + +| Gate family | Current status | Evidence | +| ----------- | -------------- | -------- | +| Static | PASS | doc-lint + scoped check/lint/fmt; exit 0 | +| Fitness | PASS | `quality:gate` exit 0; repo-wide variant JSDoc census 1→0 | +| Runtime | N/A | no behavior change | +| Consumer | PASS by construction/inspection | self-contained query block; canonical app and contract aliases; no JSDoc compile gate exists | + +## Open Questions + +- None. + +## Drift and Debt + +- Drift: orchestrator-approved desktop JSDoc sibling fix, recorded in `drift.md`. +- Debt: none. + +## Commits + +- See the draft PR's commit list + per-slice PR comments. diff --git a/.llm/runs/fix-1425-sdk-jsdoc--leaf/drift.md b/.llm/runs/fix-1425-sdk-jsdoc--leaf/drift.md new file mode 100644 index 0000000000..b9a60abc6c --- /dev/null +++ b/.llm/runs/fix-1425-sdk-jsdoc--leaf/drift.md @@ -0,0 +1,14 @@ +# Drift Log: #1425 SDK JSDoc API-client path + +## 2026-08-12 — Published desktop JSDoc sibling defect added to scope + +- **What:** Extend the slice to replace the desktop subpath example's relative app contract import + with the scaffolded project contract alias. +- **Source:** Independent adversarial review of PR #1526; orchestrator/user authorization. +- **Expected:** #1425 literally covers the stale `api-clients` module name in SDK JSDoc. +- **Actual:** `packages/sdk/src/desktop/mod.ts` has the same published-reader failure class via + `./contracts/orders.ts`, although it is not the same stale name. +- **Severity:** significant +- **Action:** fix as an explicit scope extension; do not use it as evidence for a #1425 acceptance box. +- **Evidence:** `packages/cli/src/kernel/adapters/templates/app/generate-app-deno-json.ts:63` emits + `@/contracts`; `docs/site` generic examples use `@my-app/contracts`. diff --git a/.llm/runs/fix-1425-sdk-jsdoc--leaf/plan.md b/.llm/runs/fix-1425-sdk-jsdoc--leaf/plan.md new file mode 100644 index 0000000000..c2cd6cfb67 --- /dev/null +++ b/.llm/runs/fix-1425-sdk-jsdoc--leaf/plan.md @@ -0,0 +1,123 @@ +# Plan: #1425 SDK JSDoc API-client path + +## Run Metadata + +| Field | Value | +| ----- | ----- | +| Run ID | `fix-1425-sdk-jsdoc--leaf` | +| Branch | `fix/1425-sdk-jsdoc-api-clients` | +| Phase | `plan` | +| Target | `packages/sdk` published JSDoc | +| Archetype | `4 — Public DSL / Builder` | +| Scope overlays | `docs` | + +## Archetype + +Archetype 4 is the doctrine classification already assigned to `@netscript/sdk`. This slice does not +alter that shape; it repairs an example on the public JSR documentation surface. + +## Current Doctrine Verdict + +`Keep` — high cohesion already; minor naming review. + +## Axioms in Play + +| Axiom | Why it matters | +| ----- | -------------- | +| A2 | The published example must point to the one shipped data-layer convention. | +| A3 | The example must retain the documented 80% call shape. | +| A14 | Doc-lint and package fitness gates preserve the public surface. | + +## Goal + +Replace every stale `api-clients` reference in `packages/sdk/**` JSDoc with the shipped per-service +module path while preserving the documented `queryOptions({ input })` call shape. + +## Scope + +- Change the one stale import inside the `createServiceQueryUtils` JSDoc example. +- Make that example self-contained by importing `useQuery` from the package-owned Fresh query surface. +- Orchestrator-approved extension: repair the sibling desktop JSDoc contract import with the + scaffolded `@my-app/contracts` alias. +- Record the initial and final census. +- Run every gate named in the implementation brief. + +## Non-Scope + +- Runtime or type changes in `packages/**`. +- Any `docs/site/**` edit or re-sweep. +- Work owned by #1374 or #1377. +- Merge, ready-for-review transition, or evaluator sign-off. + +## Hidden Scope + +- The full `packages/sdk/**` census and full-export-map doc-lint are required even though one file changes. + +## Locked Decisions + +| ID | Decision | Rationale | +| -- | -------- | --------- | +| D1 | Import `ordersClient` from `@app/lib/orders.ts`. | The shipped app alias resolves to `apps//lib/.ts` and replaces the removed catch-all module. | +| D2 | Preserve `ordersQueryUtils.list.queryOptions({ input: ... })`. | The named consistency page explicitly distinguishes this helper's options-object call shape. | +| D3 | Touch only the JSDoc comment body in package source. | The issue forbids runtime behavior changes. | +| D4 | Import `useQuery` from `@netscript/fresh/query`. | Checked-in SDK/docs examples establish the package-owned island hook surface. | +| D5 | Import the desktop example contract from `@my-app/contracts`. | `generate-app-deno-json.ts` emits `@/contracts`; docs use `@my-app/contracts` for generic examples. | + +## Open-Decision Sweep + +| Decision | Status | Notes | +| -------- | ------ | ----- | +| Further example modernization | safe to defer | Outside #1425; log only if discovered. | + +## Risk Register + +| Risk | Mitigation | +| ---- | ---------- | +| Import path is cosmetically updated but still contradicts the named reference page. | Preserve the page's `queryOptions({ input })` shape and run doc-lint. | +| Formatting rewrites more than the intended comment. | Use a focused patch, re-read the file, and inspect the diff. | +| Lockfile drift appears during validation. | Inspect `deno.lock` against the baseline and exclude unrelated drift. | + +## Anti-Patterns to Resolve or Avoid + +| AP | Status | Plan | +| -- | ------ | ---- | +| AP-2 | risk | Keep the public example aligned with the single caller-facing path. | +| AP-15 | existing documentation defect | Remove the stale implementation-era module name. | + +## Fitness Gates + +| Gate | Required | Expected evidence | +| ---- | -------- | ----------------- | +| F-5 / F-7 | yes | full-export-map doc-lint and zero stale-name census | +| F-6 | yes | JSR audit applied; doc-lint publish bar | +| F-19 | yes | scoped check, lint, and format wrappers | +| Code quality / architecture | yes | `quality:gate` | + +## Arch-Debt Implications + +| Entry | Action | Notes | +| ----- | ------ | ----- | +| `.llm/harness/debt/arch-debt.md` | none | A stale example is fixed without creating or deepening architecture debt. | + +## Validation Plan + +| Order | Gate | Command or check | Expected result | +| ----- | ---- | ---------------- | --------------- | +| 1 | Doc-lint | `deno task doc:lint --root packages/sdk --pretty` | exit 0 | +| 2 | Type-check | scoped check wrapper | exit 0 | +| 3 | Lint | scoped lint wrapper | exit 0 | +| 4 | Format | scoped fmt wrapper | exit 0 | +| 5 | Code quality | `rtk proxy deno task quality:gate` | exit 0 | +| 6 | Census | `rtk grep -rn "api-clients" packages/sdk/` | zero matches / grep exit 1 | +| 7 | Repo-wide variant census | case-insensitive `api[-_]?clients` over `packages/` and `plugins/` | zero JSDoc matches; generated corpus hit classified | +| 8 | SDK JSDoc import audit | enumerate example import specifiers | no relative app-level paths | + +## PLAN-EVAL + +N/A — this is a fully specified, one-comment mechanical correction with live acceptance criteria, +a named consistency target, hard boundaries, and an explicit gate set. + +## Drift Watch + +- Any additional stale occurrence outside the owned JSDoc is recorded, not silently widened. +- Any executable-statement diff in `packages/**` stops the slice. diff --git a/.llm/runs/fix-1425-sdk-jsdoc--leaf/research.md b/.llm/runs/fix-1425-sdk-jsdoc--leaf/research.md new file mode 100644 index 0000000000..83d0fb4023 --- /dev/null +++ b/.llm/runs/fix-1425-sdk-jsdoc--leaf/research.md @@ -0,0 +1,28 @@ +# Research — fix-1425-sdk-jsdoc--leaf + +## Re-baseline + +- Carried-in source: issue #1425 and the implementation brief +- Re-derived against `main` @ `01aa12b67` on 2026-08-12 +- Live issue state: open, milestone `0.0.6`, labels `type:fix`, `area:sdk`, `priority:p2`, `status:impl` +- The dispatch baseline and current `origin/main` both resolve to `01aa12b67`. + +## Findings + +| # | Finding | How to verify | +| - | ------- | ------------- | +| 1 | Exactly one `api-clients` occurrence exists in `packages/sdk/**`. | `rtk grep -rn "api-clients" packages/sdk/` | +| 2 | The occurrence is the `createServiceQueryUtils` JSDoc import at line 39; no executable statement contains the stale name. | `packages/sdk/src/query-client/create-service-query-utils.ts` | +| 3 | The consistency target documents `createServiceQueryUtils` as using `queryOptions({ input })`, distinct from the golden-path query factory's `queryOptions(input)`. | `docs/site/reference/sdk/index.md` query-client table | +| 4 | The shipped data-layer convention is one service module under `apps//lib/.ts`; the example service is `orders`. | issue #1425; repository SDK documentation | +| 5 | Doctrine classifies `@netscript/sdk` as Archetype 4, verdict `Keep`, with high cohesion and only a minor naming review. | `docs/architecture/doctrine/10-codebase-verdict-and-handoff.md` | + +## jsr-audit surface scan (package/plugin waves) + +- Surface scanned: `packages/sdk/deno.json` export map, `packages/sdk/mod.ts`, and the affected exported JSDoc. +- Slow-type / surface risks: none introduced; the slice changes one example import only. +- Publish authority: `deno task doc:lint --root packages/sdk --pretty` over the full export map. + +## Open questions + +- None. The issue fixes the path convention, consistency page fixes the call shape, and the census is one. diff --git a/.llm/runs/fix-1425-sdk-jsdoc--leaf/supervisor.md b/.llm/runs/fix-1425-sdk-jsdoc--leaf/supervisor.md new file mode 100644 index 0000000000..b2cf61eb92 --- /dev/null +++ b/.llm/runs/fix-1425-sdk-jsdoc--leaf/supervisor.md @@ -0,0 +1,14 @@ +# Supervisor Identity + +- Role: implementation agent (orchestrator retains review and merge authority) +- Model / effort: Codex GPT-5.6 Sol / low (`light_implementation`) +- Session: `/root` +- Host: Linux workspace +- Worktree: `/home/codex/repos/ns006-1425-sdkjsdoc` +- Branch: `fix/1425-sdk-jsdoc-api-clients` +- Baseline: `01aa12b67` (`origin/main` at dispatch) +- Run dir: `.llm/runs/fix-1425-sdk-jsdoc--leaf/` +- Orchestrator run dir: `.llm/runs/release-0.0.6-docs--orchestration/` (read-only) +- Lane: `light_implementation` +- PLAN-EVAL: N/A — fully specified, mechanical source-documentation correction +- IMPL-EVAL: reserved for a separate orchestrator-selected evaluator session diff --git a/.llm/runs/fix-1425-sdk-jsdoc--leaf/worklog.md b/.llm/runs/fix-1425-sdk-jsdoc--leaf/worklog.md new file mode 100644 index 0000000000..13a8374da8 --- /dev/null +++ b/.llm/runs/fix-1425-sdk-jsdoc--leaf/worklog.md @@ -0,0 +1,134 @@ +# Worklog: #1425 SDK JSDoc API-client path + +## Run Metadata + +| Field | Value | +| ----- | ----- | +| Run ID | `fix-1425-sdk-jsdoc--leaf` | +| Branch | `fix/1425-sdk-jsdoc-api-clients` | +| Archetype | `4 — Public DSL / Builder` | +| Scope overlays | `docs` | + +## Design + +### Public Surface + +- Existing `createServiceQueryUtils` and `@netscript/sdk/desktop` JSR-rendered JSDoc examples; no + export, signature, or runtime changes. + +### Domain Vocabulary + +- `ordersClient` — service-specific client exported from the app's `orders.ts` data-layer module. +- `ordersQueryUtils` — TanStack query utilities derived from that client. + +### Ports + +- None introduced or changed. + +### Constants + +- None introduced or changed. + +### Commit Slices + +| # | Slice | Gate | Files | +| - | ----- | ---- | ----- | +| 1 | Harness bootstrap and locked design | artifact review | run-dir Markdown files | +| 2 | Replace the sole stale JSDoc import and prove the 1→0 package census | all six requested gates | one SDK source comment plus run evidence | +| 3 | Correct acceptance evidence, make the query example self-contained, and repair the approved sibling desktop import | original gates plus repo-wide variant census and SDK JSDoc import audit | two SDK JSDoc comments plus run evidence | + +### Deferred Scope + +- #1374 and #1377 concerns — explicitly separate issues. +- `docs/site/**` — already corrected by #1373. +- Generated corpus refresh for `packages/mcp/src/publish-assets.generated.ts` — tracked by #1531. + +### Contributor Path + +Future example corrections start at the public JSDoc, verify the matching `docs/site/reference/sdk/index.md` +call shape, then use full-export-map doc-lint and a package-wide census. + +### PLAN-EVAL + +N/A — fully specified mechanical documentation correction; no material design decision remains. + +## Progress Log + +| Date | Slice | Step | Notes | +| ---- | ----- | ---- | ----- | +| 2026-08-12 | 1 | research and design | Live issue read; initial census is 1. | +| 2026-08-12 | 2 | implementation | Replaced only the JSDoc import with `@app/lib/orders.ts`; retained `queryOptions({ input })`. | +| 2026-08-12 | 2 | gate | All five executable gates exited 0; final census has zero matches. | +| 2026-08-12 | 2 | reconcile | Issue #1425 remains the sole owned issue; no new comments, rescope, or related-issue action required. | +| 2026-08-12 | 3 | review intake | Independent adversarial review returned PASS with no blocking findings and requested evidence corrections plus one approved sibling fix. | +| 2026-08-12 | 3 | implementation | Added the missing query hook import, aligned list input to `{ offset, limit }`, and replaced the desktop relative contract import. | +| 2026-08-12 | 3 | gate | Original gates and both widened audits pass; package source diff remains JSDoc-only. | +| 2026-08-12 | 3 | reconcile | Scope extension recorded; #1531 remains owner of generated MCP corpus refresh. | + +## Decisions + +| Decision | Reason | Source | +| -------- | ------ | ------ | +| Preserve `queryOptions({ input })` | This is the helper-specific call shape. | `docs/site/reference/sdk/index.md` | +| Replace catch-all module with per-service module | The shipped layout is `apps//lib/.ts`. | issue #1425 | +| Add the missing `useQuery` import | Acceptance says the example compiles; the existing block was not self-contained. | independent adversarial review | +| Repair the sibling desktop contract import | Orchestrator explicitly approved the same-surface scope extension. | user instruction; scaffold generator; `docs/site` | + +## Drift + +| Drift | Severity | Logged in drift.md | +| ----- | -------- | ------------------ | +| Concrete import selected from #1373's established consumer examples after bootstrap | minor | no — resolved in plan before implementation | +| Desktop JSDoc relative contract import added to scope | significant | yes — orchestrator-approved extension | + +## Census + +- Found before implementation: **1** occurrence of `api-clients` in `packages/sdk/**`. +- Fixed: **1** JSDoc occurrence. +- Repo-wide final JSDoc sweep: **0** across `packages/` and `plugins/` using case-insensitive + pattern `api[-_]?clients`, covering `api-clients`, `api_clients`, `apiClients`, and case variants. +- Broad non-JSDoc hit: `packages/mcp/src/publish-assets.generated.ts`, a generated string corpus + tracked by #1531 and intentionally out of scope. + +## Gate Results + +### Static Gates + +| Gate | Command or check | Result | Notes | +| ---- | ---------------- | ------ | ----- | +| Doc-lint | `deno task doc:lint --root packages/sdk --pretty` | PASS (exit 0) | 1 package / 12 entrypoints; 0 missing JSDoc and 0 other diagnostics. Runner reports 3 private-type-ref diagnostics confirmed pre-existing on base `01aa12b67`. | +| Type-check | `deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/sdk --ext ts,tsx` | PASS (exit 0) | 78 files; 1 batch; 0 failed batches; 0 diagnostics. | +| Lint | `deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root packages/sdk --ext ts,tsx` | PASS (exit 0) | 78 files; 0 findings. | +| Format | `deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/sdk --ext ts,tsx` | PASS (exit 0) | 78 files; 0 failed batches; 0 findings. | +| Stale-name census | `rtk grep -rn "api-clients" packages/sdk/` | PASS (zero matches; exit 1) | Initial 1; fixed 1; remaining 0. | +| Repo-wide variant JSDoc census | Deno classifier over `packages/` + `plugins/`, pattern `api[-_]?clients` case-insensitive | PASS (exit 0) | 0 JSDoc occurrences; sole broad hit is generated MCP corpus tracked by #1531. | +| SDK relative JSDoc import audit | `rtk grep -rnE "^\\s*\\*.*import .* from ['\"]\\.\\.?/" packages/sdk/` | PASS (zero matches; exit 1) | No SDK JSDoc example imports an app-level module relatively. | + +### Fitness Gates + +| Gate | Result | Evidence | Notes | +| ---- | ------ | -------- | ----- | +| F-5 / F-6 / F-7 | PASS | doc-lint plus source diff and census | Public example uses the shipped alias and helper-specific call shape. | +| F-19 | PASS | scoped check/lint/fmt wrappers | 78 SDK TS/TSX files selected by each wrapper. | +| Code quality / architecture | PASS (exit 0) | `rtk proxy deno task quality:gate` | `quality:scan` found 0 violations; `arch:check` completed with repository warnings only. | + +### Runtime Gates + +| Gate | Result | Evidence | Notes | +| ---- | ------ | -------- | ----- | +| Runtime behavior | N/A | package diff | Only a JSDoc comment body changed. | +| Scaffold runtime/static | SKIPPED by policy | independent review reports SUCCESS in 9 seconds | `ci:skip-e2e` / `ci:skip-scaffold` ratified for the comment-only diff; this is not runtime or scaffold coverage. | + +### Consumer Gates + +| Consumer | Result | Evidence | Notes | +| -------- | ------ | -------- | ----- | +| JSR reader | PASS by construction and inspection | scaffold generator + `docs/site/reference/sdk/index.md:111` + source diff | Query example imports `useQuery`, uses shipped `@app/lib/orders.ts`, and retains `queryOptions({ input })`; no repo gate compiles JSDoc examples today. | +| Desktop JSR reader | PASS by construction and inspection | scaffold generator line 63 + docs examples + source diff | Contract import is the generated generic alias `@my-app/contracts`. This extension is not #1425 acceptance evidence. | + +## Handoff Notes + +- Orchestrator/evaluator should first confirm the package diff changes only the JSDoc comment body. +- No repository gate currently extracts and compiles JSDoc examples; do not attribute example + compilation to doc-lint or the scoped source type-check. +- IMPL-EVAL and merge authority remain with the separate orchestrator-selected session. diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/context-pack.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/context-pack.md new file mode 100644 index 0000000000..4dce49a741 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/context-pack.md @@ -0,0 +1,7 @@ +# Context Pack + +OpenHands evaluator routing is fail-closed around three current open models. Focused tests and all +three dry-run identities pass. DeepSeek V4 Flash 0731 completed the bounded live branch smoke in +Actions run `31574668989` with `OPENHANDS_VERDICT: PASS`; the temporary hold is lifted. Next: update +the repository default variable, merge PR #1524 through its green gate, and use Qwen only on the +first genuinely broad/complex evaluation. diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/drift.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/drift.md new file mode 100644 index 0000000000..a268d00472 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/drift.md @@ -0,0 +1,5 @@ +# Drift + +- No product-scope drift. +- Repository variable `OPENHANDS_DEFAULT_MODEL` was stale and is updated only after the branch + workflow proved the new open-only contract in Actions run `31574668989`. diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/plan.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/plan.md new file mode 100644 index 0000000000..a9b9b19831 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/plan.md @@ -0,0 +1,10 @@ +# Plan + +1. Fail generic OpenHands closed unless its route is MiniMax M3, DeepSeek V4 Flash 0731, or Qwen + 3.8 Max. +2. Keep MiniMax for PLAN-EVAL, select DeepSeek for small/simple IMPL-EVAL, and select Qwen for + broader/complex IMPL-EVAL. +3. Prove the policy with focused tests and all three dry-run dispatch contracts. +4. Open a draft PR, run one bounded DeepSeek live smoke from the branch, then update the repository + default only after the branch workflow proves healthy. + diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/research.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/research.md new file mode 100644 index 0000000000..c663c13be2 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/research.md @@ -0,0 +1,8 @@ +# Research + +- Typed model configuration already contained Qwen 3.8, MiniMax M3, and DeepSeek V4 Flash 0731. +- Generic OpenHands still exposed closed-model profiles and defaulted through a stale repository + variable to Kimi K2.6. +- Qwen 3.8 had no formal evaluation preset and could not be selected by the canonical route policy. +- The docs-specific OpenHands evaluator is already pinned to MiniMax M3 and remains unchanged. + diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/smoke-prompt.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/smoke-prompt.md new file mode 100644 index 0000000000..2f39afbbd7 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/smoke-prompt.md @@ -0,0 +1,14 @@ +use harness + +# OpenHands transport smoke + +Read-only smoke. Do not edit, commit, push, comment, or run product gates. Inspect the checkout +identity and reply with the selected model plus exactly `OPENHANDS_VERDICT: PASS` if the agent can +read the repository and complete one tool-backed turn. Otherwise report the precise blocker and +`OPENHANDS_VERDICT: FAIL_FIX`. + +## SKILL + +- `openhands-handoff` — apply the cloud evaluator transport and verdict contract. +- `netscript-harness` — preserve evaluator separation and evidence rules. +- `netscript-tools` — use only read-only repository inspection for this smoke. diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/supervisor.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/supervisor.md new file mode 100644 index 0000000000..4789346df6 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/supervisor.md @@ -0,0 +1,9 @@ +# Supervisor + +- Agent: Codex GPT-5.6 Sol +- Branch: `fix/openhands-open-evaluator-defaults` +- Baseline: `origin/main` at `01aa12b67` +- Scope: OpenHands open-evaluator routing repair and bounded smoke +- PLAN-EVAL: N/A — the owner specified the three exact phase/complexity bindings. +- IMPL-EVAL: N/A — deterministic policy/config repair with focused tests and a live transport smoke. + diff --git a/.llm/runs/fix-openhands-open-evaluator-defaults--routing/worklog.md b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/worklog.md new file mode 100644 index 0000000000..993c77e8c5 --- /dev/null +++ b/.llm/runs/fix-openhands-open-evaluator-defaults--routing/worklog.md @@ -0,0 +1,18 @@ +# Worklog + +## Design + +- Model IDs remain centralized in `config/models.ts`. +- Provider presets own transport capability facts. +- Canonical routing owns phase and complexity selection. +- The dispatch tool and GitHub workflow both enforce the same finite open-evaluator set. + +## Gates + +- Focused agentic tests: PASS (65 tests). +- Dry-run dispatch contracts: PASS for MiniMax M3, DeepSeek V4 Flash 0731, and Qwen 3.8 Max. +- Bounded live OpenHands smoke: PASS — DeepSeek V4 Flash 0731, Actions run `31574668989`, exact + branch SHA `eb16a6b68`, tool-backed checkout inspection, `OPENHANDS_VERDICT: PASS`. +- The first `ready_for_review` CI event (`31575007718`) materialized only skipped jobs; per the + milestone did-not-run rule this is unproven, not green. This evidence commit intentionally emits a + non-draft `synchronize` event so the required classifiers and gates execute. diff --git a/.llm/runs/release-0.0.6-features--orchestration/context-pack.md b/.llm/runs/release-0.0.6-features--orchestration/context-pack.md new file mode 100644 index 0000000000..22208aa818 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/context-pack.md @@ -0,0 +1,86 @@ +# Context pack — 0.0.6 runtime / public-surface lane + +## Reopened slice — #1548 / PR #1559 (2026-08-12) + +The lane was reopened for the P1 browser stream-discovery fix on +`fix/1548-vite-browser-stream-discovery`. PLAN-EVAL passed with binding amendments recorded at the +bottom of `slices/plan-1548.md`. The implementation replaces the value-passed/computed browser env +reads with literal Vite-substitutable member expressions, adds a source-internal pure lookup, and +adds five resolver tests. Gate evidence and the contributor path are in `slices/worklog-1548.md`. +The PR remains draft at `status:impl`; separate-session IMPL-EVAL and merge authority remain with the +orchestrator. + +Closing summary. The lane is **complete**: both owned issues landed on `main`. Read this first if +resuming or auditing; everything below is traceable to a named artifact. + +| Field | Value | +| --- | --- | +| Run id | `release-0.0.6-features--orchestration` | +| Profile | `.llm/harness/workflow/milestone-run.md` (topical lane) | +| Supervisor | Claude · Opus 5 · high | +| Control branch / PR | `chore/release-0.0.6-features-orchestration` / **#1525** (evidence only, closes nothing) | +| Baseline | `origin/main@01aa12b67` | +| Status | **Complete** — 2/2 issues merged, 2 follow-ups filed | + +## Outcome + +| Issue | PR | Merge | Gate record | +| --- | --- | --- | --- | +| #1405 durable producer rejection taxonomy | #1528 | `8ff1bcb8f` | `slices/pre-merge-gate-1528.md` | +| #1398 job executions → durable job stream | #1536 | `d7e2b67b2` | `slices/pre-merge-gate-1536.md` | + +Both auto-closed `COMPLETED` via closing keywords; `status:shipped` on each issue and PR. + +## What changed, in one paragraph each + +**#1405** — two settled write **reason strings** misdescribed the state that produced them: a write +rejected during the graceful close-drain reported `producer-failed` (the producer was healthy and +closing), and a non-retryable append on attempt 1 reported `retry-exhausted` (nothing was exhausted). +Fixed by making the closing intent observable to the rejection selector and by branching `#failActive` +on `isRetryable`, adding exactly one public member, `transport-refused`. The façade's duplicate +selector was deleted so the two cannot drift apart again — that drift was the defect's origin. No +change to which writes are accepted, rejected, or delivered. + +**#1398** — job executions were never published to the durable job stream because the workers **API +service** installed the execution-state mutation hook while the **background** entrypoints that +generated projects actually run never did. Fixed by installing the hook on the worker and combined +runtimes and publishing inside `context.with(extractContext({traceparent, tracestate}), …)` so the +publish span joins the `job.execute` trace — including the pre-span `create()` record, which is the +trap that would otherwise fail TC-14 silently. The two E2E gates deferred against this issue were +un-deferred and now pass live. + +## Evidence chain + +- **#1398 acceptance was made mechanical.** Its observational criterion was bound to two gates the + repo had already deferred against this issue. Both now pass **by name** on both CI runtime tiers at + the merging head: postgres `94073971396` (`passed=88 failed=0 skipped=0`), sqlite `94073971501` + (`passed=83 failed=0 skipped=0`). +- **PLAN-EVAL** #1398: PASS, MiniMax M3, separate session — `plan-eval.md`. Findings F1/F2 folded in. +- **IMPL-EVAL** #1405: PASS, DeepSeek V4 Flash 0731 — `slices/evaluate-1405.md`. Superseded as + *policy* by D-3 after the fact, not retracted. +- **IMPL-EVAL** #1398: PASS, Qwen 3.8 Max via the **automatic dispatcher** — + `slices/evaluate-1398.md`. +- Merge order and every time-costing failure: `cut-trace.md`. Lessons and mistakes: + `retrospective.md`. + +## Open items leaving this lane + +| Item | Where | +| --- | --- | +| `quality:gate` roots omit published packages — **both merge records here rest on explicit target scans, not the repo gate** | **#1542** (0.0.7) | +| Undeclared `@netscript/plugin-streams-core` imports — filed **unverified**, `publish:dry-run` evidence is its first acceptance box | **#1543** (0.0.7) | +| Canary and stable cut | **root's**, not this lane — nothing was published here | + +## Drift index + +D-1 research sub-agent lane override · D-2 evaluator transport fallback · **D-3** IMPL-EVAL waived +for the small deterministic class · **D-4** phase evaluation moved to the automatic dispatcher · +**D-5** label-driven eval trigger contract. Full text in `drift.md`. + +## Artifact hygiene note + +The two raw evaluator JSONL streams (2.4 MB combined) were **untracked** and moved to +`.llm/tmp/` scratch, which `.gitignore` excludes. They were 2.4 MB of a 2.5 MB run dir, against a +96 K largest-artifact precedent in the 0.0.5 run. Their substance is preserved verbatim in +`plan-eval.md` and `slices/evaluate-1405.md`, each carrying run id, duration, event count, and +`is_error`. Nothing was deleted — the files remain on disk locally. diff --git a/.llm/runs/release-0.0.6-features--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-features--orchestration/cut-trace.md new file mode 100644 index 0000000000..a8d10ff426 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/cut-trace.md @@ -0,0 +1,47 @@ +# Cut trace — 0.0.6 runtime / public-surface lane + +Merge-derived record. Every row is captured **from `git log origin/main` after the merge**, never +from recollection or from the dispatch plan. This lane does not cut or publish; root orchestration +owns the canary and the stable cut. + +## Baseline + +| Field | Value | +| --- | --- | +| Lane opened | 2026-08-12 | +| `origin/main` at open | `01aa12b67` — `docs(harness): record FILING-LOG -- board migration executed once (#1523)` | +| Owned issues | #1405, #1398 | + +## Merges + +| # | UTC | Merge commit on `origin/main` | PR | Issues closed | Pre-merge gate record | +| --- | --- | --- | --- | --- | --- | +| 1 | 2026-08-12T08:20:29Z | `8ff1bcb8fc741db17a765a1965861828e0ae6171` (`8ff1bcb8f`) — `fix(streams): distinguish producer refusal reasons (#1528)` | #1528 | **#1405** (auto-closed `COMPLETED` by the body's `Closes #1405`) | `slices/pre-merge-gate-1528.md` — all 7 checks PASS | + +| 2 | 2026-08-12T10:27:26Z | `d7e2b67b2be535c9ca13449f97f8f4585344030a` (`d7e2b67b2`) — `fix(workers): publish job executions to the durable stream on the job.execute trace (#1536)` | #1536 | **#1398** (auto-closed `COMPLETED` by the body's `Closes #1398`) | `slices/pre-merge-gate-1536.md` — all 7 checks PASS at head `f7d503fee` | + +Both rows captured from `git log origin/main --first-parent -1` **after** each merge, per the +profile's merge-derived rule. Issue states re-read live: both `CLOSED` / `COMPLETED`, with +`status:shipped` applied to each issue and PR. + +**Lane complete: both owned issues landed on `main`.** + +## Re-planning events + +| # | Date | Event | Decision | +| --- | --- | --- | --- | +| 1 | 2026-08-12 | #1536's head changed mid-flight, `e4319c685` → `f7d503fee` | The phase-eval dispatcher (#1524) merged **after** the branch's last `main` sync, so `openhands-phase-eval.yml` was absent from the PR's merge ref and no label cycling could trigger it. Owner approved syncing the branch, accepting a full CI re-run. All gate evidence was then re-read against the new head; no pre-sync evidence was carried forward. | +| 2 | 2026-08-12 | Evaluation route changed twice mid-run | D-3 removed formal IMPL-EVAL for the small deterministic class (after #1405 had already merged); D-4/D-5 moved phase evaluation to the automatic label-driven dispatcher. #1398's IMPL-EVAL consequently ran on the automatic route rather than a manual launch. | + +## Failure modes that cost real time + +| # | Date | Failure | Cost | Mitigation recorded | +| --- | --- | --- | --- | --- | +| 1 | 2026-08-12 | **Dispatcher absent from the PR's merge ref.** #1524 merged 38 min after #1536's branch last synced with `main`, so `openhands-phase-eval.yml` did not exist for that PR. Labels were cycled correctly and produced **no run at all**; the only runs were the older `openhands-agent.yml`, all `skipped`. | One wasted label re-entry, plus a full CI re-run (~20 min, both `scaffold-runtime` tiers) after the branch sync. | For `pull_request` events GitHub resolves workflows from the **merge ref**. A newly merged workflow cannot fire on a PR whose head predates it. **Check the workflow exists in the PR head before concluding a trigger failed.** Diagnosed by `git cat-file -e :` → ABSENT, with the post-sync run succeeding as the control. | +| 2 | 2026-08-12 | **Draft PRs report every check as `skipping`.** #1528 looked "clean" while nothing substantive had run. | None — caught by pre-merge check 4 before merge. | This is the #778/#775 class, alive and current. Absence of red is not green. The blocking tier only runs after draft→ready. | +| 3 | 2026-08-12 | **Two live `scaffold.runtime` runs died before reaching the gates under test** — run 1 on a transient `generate plugins: fetch failed`, run 2 on a `triggers-api` health timeout at 120 s. | ~25 min of local runtime, no verdict produced. | Local WSL was not a usable arbiter for this suite. CI ran the same suite with the same change to a clean finish on both tiers, which is the control that established the local failures as environmental. **Do not label a local red a "flake" without that control.** | +| 4 | 2026-08-12 | **A counting watcher would never have fired.** OpenHands updates its summary comment **in place** (`openhands-agent-summary` marker, `"conclusion"` field), so a watcher keyed on comment count polls to timeout while the verdict sits in an edited comment. | None — caught before arming, on inspecting the posted comment's markers. | Watch the **run status** and the comment's `conclusion` marker, never comment count. | +| 5 | 2026-08-12 | **Stale check summary read as current.** `gh pr checks` reported `close-gate` red from a job that ran 40 min earlier, before the label and body changes it was complaining about. | None — caught by reading the job log, which carried the mirror's own "skipped because labels do not include status:ready-merge" notice. | Compare a check's run time to the change it is judging. The gate-integrity rule already says merge-history audits must take the latest run per check name; the same applies pre-merge. | +| 6 | 2026-08-12 | **Automation added a second `status:` label.** `status:augment-review` was applied 1 s after the orchestrator moved off `status:impl-eval`, breaching the exactly-one-status invariant. | None — caught while verifying labels before merge. | Re-verify the `status:` set after any automated phase transition, not only after manual edits. | +| 7 | 2026-08-12 | **Orchestrator's own slice brief named a broken gate command.** `deno test packages/plugin-streams-core` exits 1 with 19 `NotCapable` errors for want of `--allow-env`. | Minor; the implementer reported the red with its cause rather than hiding or working around it. | Use the package-declared `deno task --cwd test`. Corrected in the #1398 brief rather than repeated. | +| 8 | 2026-08-12 | **One unnecessary evaluator dispatch.** The lane brief's IMPL-EVAL waiver for the #1405 class was read as a blocked-transport fallback rather than the class default. | One DeepSeek IMPL-EVAL run (~643 s) that the owner did not want. | Recorded as D-3. "A waiver is available" and "the waiver is the default" are different instructions; resolve the ambiguity before spending. | diff --git a/.llm/runs/release-0.0.6-features--orchestration/drift.md b/.llm/runs/release-0.0.6-features--orchestration/drift.md new file mode 100644 index 0000000000..6fbe47c6fc --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/drift.md @@ -0,0 +1,124 @@ +# Drift — 0.0.6 runtime / public-surface lane + +Append-only. Severity: `minor` | `significant` | `architectural`. + +## D-1 — research/plan sub-agent lane overridden to Opus (minor) + +**Date** 2026-08-12. **Canonical route** `deep_analysis` = Claude · Fable 5 · medium. **Used** +Claude · Opus 5 medium/high. **Reason** the owner brief for this lane explicitly directs +"Delegate plan/research to Claude Opus medium/high sub-agents when useful". Owner instruction is +more specific than the default lane binding. Invariants preserved: generator ≠ evaluator, no lane +self-certifies, no paid escalation. Fable 5 · low remains the `review_codex` reviewer for the +#1398 slice, so opposite-family review of Codex work is untouched. + +## D-2 — evaluator transport falls back to local fresh sessions (minor) + +**Date** 2026-08-12. The brief routes PLAN-EVAL/IMPL-EVAL through OpenHands *after #1524 +passes/lands*. Observed: PR #1524 is **OPEN**, `mergedAt: null`, with unticked DoD boxes for the +bounded live DeepSeek smoke and the repository default variable. Per the brief's own condition, this +run uses fresh local Claude/OpenCode OpenRouter evaluator sessions through the toolchain. Re-checked +before each dispatch; state recorded at the point of use. + +## D-3 — IMPL-EVAL not required for small deterministic taxonomy fixes (significant) + +**Date** 2026-08-12. **Owner ruling.** For the #1405 class — small, deterministic, fully specified +fixes with exact negative tests — a separate formal IMPL-EVAL is **not** to be dispatched. The +sufficient evidence set is: focused negative tests, CI, close-gate, and the orchestrator's own +independent diff review. + +**My error.** The lane brief said an IMPL-EVAL owner waiver was "acceptable" for #1405 with exact +negative tests. I read that as a fallback to use only if the evaluator transport was blocked, and +recorded that reading in `supervisor.md`. The owner's intent was that the waiver is the **default** +for this class. One evaluator dispatch (DeepSeek V4 Flash 0731 max, 642,836 ms) was spent that should +not have been. + +**Timing, stated plainly.** The ruling arrived after #1405 had already been evaluated **and merged** +(`8ff1bcb8f`, 2026-08-12T08:20:29Z). It therefore does not retract that merge or its evidence — the +IMPL-EVAL returned PASS and its per-fix revert isolation is real evidence that remains in the record. +The ruling governs this class going forward. + +**Not weakened for #1398.** Formal PLAN-EVAL and IMPL-EVAL remain **mandatory** for #1398: it changes +public runtime behaviour, publishes a new record to a durable stream, and requires live evidence. +Its PLAN-EVAL has already run (PASS, MiniMax M3) and its IMPL-EVAL is still required before merge. + +**Cleanup.** The detached #1405 evaluator worktree `/home/codex/repos/ns006-1405-impleval` was +verified clean (empty `git status --porcelain`) and removed. The #1398 PLAN-EVAL worktree +`/home/codex/repos/ns006-1398-planeval` is retained pending that issue's IMPL-EVAL. Pre-existing +global stashes belonging to other lanes' branches were left untouched. + +## D-4 — phase evaluation moves to the automatic status dispatcher (significant) + +**Date** 2026-08-12. **Owner ruling.** PR #1524 (automatic phase dispatcher) is about to merge. Once +it lands, **all future phase evaluations use the automatic status workflow** unless the owner selects +a documented local route or an explicit skip. Manual IMPL-EVAL launches and manual +`@openhands-agent` PR comments are not to be used for #1536. + +**Standing instruction for #1536:** keep it on its **current head and status** — head `e4319c685`, +`status:impl-eval`, milestone `0.0.6`. Root will **deliberately re-enter** `status:impl-eval` with +the Qwen override *after* #1524 lands, which is what triggers the automatic dispatcher. This +orchestrator must not re-enter the label for the same head, must not trigger OpenHands, and must not +launch a local evaluator. Its remaining job is to **watch the automatic verdict and then finish the +merge gate**. + +**Timing, recorded factually.** The steer anticipated that a local #1536 evaluator had already been +launched. It had **not**. For #1398 I wrote the evaluator prompt +(`slices/impl-eval-1398-prompt.md`, 10:57) and pre-created the detached worktree +`/home/codex/repos/ns006-1398-impleval` at `e4319c685`, then **stopped and raised the decision** +rather than dispatching. Verified at the time of this entry: no `openrouter-run`/`claude-openrouter` +process, and no `impl-eval-1398-raw.md` output file — only the prompt. **No duplicate spend occurred +and none was in flight.** + +The two evaluator sessions this run did spend are unaffected and both completed before this ruling: +PLAN-EVAL #1398 (MiniMax M3, PASS) and IMPL-EVAL #1405 (DeepSeek V4 Flash 0731, PASS — itself the +subject of D-3). + +**Unused artifacts retained, not removed:** `slices/impl-eval-1398-prompt.md` and the +`ns006-1398-impleval` worktree are left in place, clean and unused, in case the owner later selects +the documented local route. They are inert; nothing reads them. + +**Interaction with D-3.** D-3 waives formal evaluation for the small deterministic class. D-4 does +not widen that waiver — #1398 still gets a formal IMPL-EVAL; it now arrives through the automatic +dispatcher rather than a manual launch. + +## D-5 — automation trigger contract for formal evaluation (significant) + +**Date** 2026-08-12. **Owner policy.** Formal PLAN/IMPL evaluation is triggered by labels, never by a +manual OpenHands dispatch: + +| Phase | Initial trigger | Rerun | +| --- | --- | --- | +| PLAN-EVAL | the `openhands` + `status:plan-eval` label **pair**, exactly once | move away from `status:plan-eval`, then re-add | +| IMPL-EVAL | automatically on **draft → ready**, unless `impl-eval:skip` | move away from `status:impl-eval`, then re-add | + +`eval:model:minimax|deepseek|qwen` is an optional **one-shot** override. A local eval already running +may finish, but must never be duplicated. Manual `@openhands-agent` dispatch for formal PLAN/IMPL +eval is prohibited. Merge continues to go through normal harness authority. + +**This lane complies with no change required:** no manual OpenHands dispatch was made for #1536 (or +for anything else this run), and no local evaluator was launched for #1536 — the prompt and worktree +were prepared and the dispatch deliberately withheld (D-4). The two local evaluator sessions this run +did spend — PLAN-EVAL #1398 (MiniMax M3) and IMPL-EVAL #1405 (DeepSeek) — both **completed** well +before this policy and are not duplicated. + +### Timing finding: #1536's automatic IMPL-EVAL could not have fired + +Measured from the issue timeline rather than assumed: + +| Event | UTC | +| --- | --- | +| #1536 `ready_for_review` | **2026-08-12T08:53:43Z** | +| #1536 `status:impl` removed, `status:impl-eval` applied | 2026-08-12T08:53:45Z | +| #1524 (the dispatcher) merged | **2026-08-12T09:24:15Z** | + +Both candidate triggers precede the dispatcher's existence by ~30½ minutes. The draft → ready +transition therefore had nothing to fire, and the initial automatic IMPL-EVAL for #1536 **did not +run and will not run on its own**. + +Under this policy the rerun path is the only one left: move #1536 away from `status:impl-eval`, then +re-add it. That matches what root already stated it would do with the Qwen override, so the +conclusion is unchanged — but it is now a **requirement** rather than a preference, and a watcher +waiting for a spontaneous verdict would wait forever. Recorded so that is visible rather than +discovered by timeout. + +**Not actioned by this lane.** The label re-entry is root's, per D-4. This orchestrator has not +touched #1536's head, labels, or body. diff --git a/.llm/runs/release-0.0.6-features--orchestration/plan-eval-prompt.md b/.llm/runs/release-0.0.6-features--orchestration/plan-eval-prompt.md new file mode 100644 index 0000000000..e3208cbf46 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/plan-eval-prompt.md @@ -0,0 +1,94 @@ +use harness + +# PLAN-EVAL — NetScript issue #1398 + +You are a **formal PLAN-EVAL evaluator** in a fresh session. You did not write this plan and you are +not its supervisor. Your job is to try to **break** it, then return a verdict. + +Worktree: `/home/codex/repos/ns006-1398-planeval` (detached at `01aeafbfa`). **Read-only** — do not +edit, commit, or push anything. You may run read-only commands (`git`, `grep`, `deno doc`, +`deno check`) to verify claims. + +## SKILL + +- `netscript-harness` — read `.llm/harness/evaluator/plan-protocol.md`, + `.llm/harness/gates/plan-gate.md`, and `.llm/harness/evaluator/verdict-definitions.md`. +- `netscript-doctrine` — plugin/package layering, Archetype 5 thinness, public-surface rules. + +## Read + +1. `.llm/runs/release-0.0.6-features--orchestration/plan.md` — the plan under evaluation. +2. `.llm/runs/release-0.0.6-features--orchestration/slices/research-1398.md` — the research it rests + on. **Note its "Unverified" section: that research pass was terminated early on budget.** +3. The live issue #1398 body and its four acceptance boxes. +4. Whatever source you need to check the plan's claims. The plan cites specific `path:line` + locations — check them. + +## What to attack, in priority order + +The plan's entire design rests on **one causal chain**. Verify it or break it: + +> `job-dispatcher.ts:44` derives `parentContext` from the stored trace headers and passes it to +> `traceJobExecution` (`:108`), so `job.execute` shares a trace id with the execution record's +> stored `traceparent`; and `instrumentation.ts:160` starts the stream publish span on the **ambient** +> OTel context. Therefore publishing execution mutations under a context extracted from the stored +> `traceparent` makes every published record — including the pre-span `create()` one — carry a header +> traceparent whose **trace id** equals the `job.execute` trace id, satisfying TC-14 +> (`select-flow-b-stream-change.ts:122-153`). + +Specific things worth doubting: + +1. **Is the trace id really shared?** `getParentContextFromHeaders` may return a context that is + remote/non-recording, or `withSpan` may ignore `parentContext` under some configuration. If + `job.execute` starts a **new trace** rather than continuing the dispatch trace, decision D3 is + wrong and the plan fails. +2. **Does TC-14 assert what the plan says it asserts?** Read + `select-flow-b-stream-change.ts:122-153` and `consume-flow-b-stream.ts:52-107,203-230` yourself. + If the gate compares full `traceparent` rather than trace id, or matches a record the plan does + not anticipate, say so. +3. **Selector ambiguity.** Four records share a `correlationId`. Does the selector take the first, + the last, or all? If it takes the first and that is the `create()` record, does the plan's D3 + actually save it — or does D3 only work if the extracted context is non-remote? +4. **Is `SCAFFOLD_RUNTIME_DEFERRED_GATES` really the acceptance surface** the plan claims (D5), and + does removing those two entries make `scaffold.runtime` run them? +5. **S0's blocking precondition.** The plan blocks on whether `workers-combined` receives the streams + env. Read + `packages/cli/src/kernel/templates/aspire/helpers/register/generate-register-background.ts:180-220` + and `stream-url-resolver.ts:154-190` and **answer it now** if you can — it is the cheapest way to + de-risk the whole slice, and the research never got to it. +6. **Idempotency of D1.** Can `startCombinedProcess` end up installing the hook twice, or installing + it where the API service already did? What happens then? +7. **Scope honesty.** Does anything in the plan quietly require the `WorkerExecutionZodSchema` change + that D4 forbids? If so the plan is internally inconsistent. + +Also apply the plan-gate checklist: contract-first, acceptance mapped to evidence, gates named, +risks recorded, no criterion that cannot be truthfully ticked. + +## Output + +Write your verdict to +`.llm/runs/release-0.0.6-features--orchestration/plan-eval.md` **as text in your final message** +(do not write the file — the orchestrator commits it). Structure: + +``` +**[PHASE: PLAN-EVAL] [VERDICT: PASS|FAIL_PLAN]** + + + +### Verified +- — + +### Findings +1. **C1 ** — what is wrong, where, and the concrete fix. +... + +### Answers to open questions +- S0 (streams env reaches workers-combined): <answered yes/no + citation, or "could not determine"> + +### Next +- <action + owner> +``` + +Rules: every finding cites `path:line`. Distinguish **blocking** (`FAIL_PLAN`) from advisory. If you +cannot verify something, say "could not verify" — do not guess, and do not pad the verdict with +praise. A finding I can check is worth more than a paragraph of assessment. diff --git a/.llm/runs/release-0.0.6-features--orchestration/plan-eval.md b/.llm/runs/release-0.0.6-features--orchestration/plan-eval.md new file mode 100644 index 0000000000..fae5dd3d36 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/plan-eval.md @@ -0,0 +1,58 @@ +# PLAN-EVAL — #1398 (verbatim verdict) + +| Field | Value | +| --- | --- | +| Phase | PLAN-EVAL | +| Subject | `plan.md` for issue #1398 | +| Evaluator | MiniMax M3 · high, `claude-evaluator-minimax-m3` preset over OpenRouter | +| Session | fresh, separate from the generator; worktree `/home/codex/repos/ns006-1398-planeval` @ `01aeafbfa` | +| Route reason | local fallback per `drift.md` D-2 (#1524 still open) | +| Duration | ms | +| is_error | False | +| Raw stream | `plan-eval-raw.md` (215 events) | + +A leaked one-line preamble ("Now I have enough to write the verdict…") preceded the verdict token in +the raw stream and is stripped here; nothing else is edited. + +--- + +**[PHASE: PLAN-EVAL] [VERDICT: PASS]** + +Plan is sound and the causal chain holds; one unresolved open decision worth flagging before implementation. + +### Verified + +- **Trace id chain** — `plugins/workers/worker/job-dispatcher.ts:43-44` (`parentContext = tracedContext?.parentContext ?? getParentContextFromHeaders(traceHeaders)`) → `:108` (`traceJobExecution({...,parentContext}, ...)`) → `packages/telemetry/src/instrumentation/worker.ts:323-385` (`withSpan(tracer, ..., { parentContext, ... })`) → `packages/telemetry/src/application/span.ts:38-43` (passes `parentContext` as the third arg to `tracer.startSpan`). The `job.execute` span's trace id is therefore the dispatch traceparent's trace id. +- **Ambient publish span** — `packages/plugin-streams-core/src/telemetry/instrumentation.ts:160-170` calls `this.#tracer.startSpan(StreamSpanNames.PUBLISH, { kind, attributes })` with no parent context. The `headers.traceparent` is `formatTraceparent(span.spanContext())` (line 172). The publish span's trace id is the active OTel context's trace id at the time of `startSpan`. +- **TC-14 assertion** — `packages/cli/e2e/src/application/gates/scaffold/select-flow-b-stream-change.ts:131-153` extracts the trace id from the matched record's `headers.traceparent` (regex `^[0-9a-f]{2}-([0-9a-f]{32})-...` at line 142) and asserts equality with `identity.traceId`. The comparison is trace id only, not the full `traceparent`. Plan's claim is correct. +- **Selector match semantics** — `select-flow-b-stream-change.ts:96-105` returns on the first matching `correlationId`. With D3, every record's trace id equals the dispatch traceparent's trace id, so TC-14 passes regardless of which record returns first. +- **Stored traceparent on the execution record** — `packages/plugin-workers-core/src/state/execution-state.ts:73-75` (and earlier in `create` at lines 154-175) store `traceparent`/`tracestate` from the queue message. The dispatch traceparent is identical to that stored on `create()` (since `create()` runs before `traceJobExecution` and accepts the queue's trace headers). +- **S0 — streams env reaches workers-combined** — YES. `packages/cli/src/kernel/adapters/plugin/plugin-reference-reconciler.ts:70-91` adds plugin dependencies (including `streams`) and the canonical plugin name to `backgroundReferences` for `workers-combined`. `packages/cli/src/kernel/templates/aspire/helpers/register/generate-register-background.ts:200-218` then injects `services__streams__http__0` for each plugin reference. `plugins/streams/src/public/mod.ts:45-49` registers `streams` as a service. `packages/plugin-streams-core/src/application/stream-url-resolver.ts:38` and `:108` read `services__streams__http__0` (server-side). The env path is fully wired. +- **D4 vs reality** — `traceparent`/`tracestate` are already on `ExecutionRecord` (execution-state.ts:73-75). The schema change D4 forbids is not required by D3, since D3 only reads the fields that already exist. No internal conflict. +- **D1 idempotency** — `KvExecutionState.setMutationHook` (execution-state.ts:147-149) is destructive (single `onMutation` field). If both `workers-api` and `workers-combined` independently call it, the second call replaces the first. Since both call it with `createStreamMutationHook()` (the same behavior), the only consequence is a wasted hook construction; no double-install hazard. +- **R2 record volume** — `packages/plugin-streams-core/src/domain/producer-contract-v1.ts:50-53` shows the 256-event / 1 MiB bounded buffer. Plan correctly notes this is volume, not correctness. +- **`isRecord` check in selector** — `select-flow-b-stream-change.ts:122-128` reads `value.correlationId` first. `toExecutionStreamEntity` (producer.ts:94) sets `correlationId: execution.correlationId`. ✓ + +### Findings + +1. **F1 (advisory) — D3 mechanism is implemented by the slice, not enumerated by the plan.** The plan says "publish under an OTel context extracted from the execution record's stored `traceparent`/`tracestate`" but the existing `createStreamMutationHook` (`packages/plugin-workers-core/src/streams/producer.ts:108-118`) calls `producer.upsert(...)` with no `context.with(...)` wrapping. The `StreamsTracerPort.startSpan` interface (`packages/plugin-streams-core/src/telemetry/instrumentation.ts:92-102`) does not accept a parent context argument, so the only way for the publish span to inherit the stored traceparent's trace id is for the hook to wrap the `producer.upsert` call with `context.with(extractContext({traceparent, tracestate}), () => producer.upsert(...))`. The plan's wording is correct but the implementation step is implicit. **Slice review should confirm the implementation explicitly wraps with `context.with(...)` and adds a unit test that pins the trace id under the active context override (the plan already names this in tests #1 and #3).** + +2. **F2 (advisory, scope honesty) — S2 misses a second test in `suite-registry_test.ts`.** The plan updates `suite-registry_test.ts:204-215` (the test that asserts `runtime.gates.some(...)` is false for the two gates) but **misses the second test at lines 209-234** (`runtime suites pin the exact #1398 OTEL deferral without widening it`). That test at line 210-221 asserts `SCAFFOLD_RUNTIME_DEFERRED_GATES` exactly equals the two-gate list. After D5 removes both entries, the constant becomes empty (or zero entries), and the assert fails. The test must be removed or rewritten in the same commit as the deferral-list change. **Block-level for S2's "red-on-regression" claim** — if S2 is committed without updating this test, the planned local unit-test gate (or the Tier-A review) will catch and block it. Either is the right time; the plan should name it. + +3. **F3 (advisory) — The `instrumentation.ts:160` cite is correct but load-bearing.** The plan's claim that `tracer.startSpan` is called with no explicit context is critical to D3's logic. Verified directly. ✓ + +4. **F4 (advisory) — Promise chain does not lose the OTel context.** `executionState.create` (execution-state.ts:152-178) → `#save` (line 304-307) → `setMutationHook` callback is in a single async chain. `await this.#kv.set(...)` yields, but the active context within the resumption is the caller context (no parent span at the time of `create()`). Wrapping the hook with `context.with(extracted, ...)` works correctly across the `await` boundary because async contexts in Deno are preserved by the runtime. ✓ + +5. **F5 (acknowledged) — R3 undeclared `@netscript/plugin-streams-core` import in `packages/plugin-workers-core/deno.json`.** Confirmed: `deno.json` lines 24-35 do not list it, but `src/streams/producer.ts:1` imports it. If `deno publish` rejects it, the slicing is incomplete. The plan correctly defers this. Acceptable. + +6. **F6 (advisory) — Selector ambiguity is correctly resolved by D3.** Four records share `correlationId`. Without D3, the first match (`create()`) fails TC-14 because its publish span has a different trace id. With D3, all four records share the trace id — so any match passes TC-14. The plan's reasoning is correct. ✓ + +### Answers to open questions + +- **S0 (streams env reaches workers-combined)**: YES, answered. Citations: `packages/cli/src/kernel/adapters/plugin/plugin-reference-reconciler.ts:70-91` (background inherits plugin dependencies) → `packages/cli/src/kernel/templates/aspire/helpers/register/generate-register-background.ts:200-218` (env injection) → `plugins/streams/src/public/mod.ts:45-49` (service declared) → `packages/plugin-streams-core/src/application/stream-url-resolver.ts:38,108` (resolution). The plan's S0 blocker is **de-risked**; the slice can proceed without an "is this wired?" probe. + +### Next + +- **Slice review (Tier-A)**: when each implementation slice lands, verify (a) the hook wraps `producer.upsert` with `context.with(extractContext({traceparent, tracestate}), ...)` and the unit test pins the span trace id (F1), and (b) S2 updates both `suite-registry_test.ts:204-215` AND `suite-registry_test.ts:209-234` in the same commit (F2). +- **Implementation lane**: proceed with S1 (D1+D2+D3) and S2 (D5 + the missed test) per the plan. S3 (live E2E) is the gate. The two formerly-deferred gates must pass in a real `scaffold.runtime` run. +- **R3 undeclared import**: tracked separately, not blocking this slice per the plan. diff --git a/.llm/runs/release-0.0.6-features--orchestration/plan.md b/.llm/runs/release-0.0.6-features--orchestration/plan.md new file mode 100644 index 0000000000..2546759c89 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/plan.md @@ -0,0 +1,193 @@ +# Plan — #1398 publish job executions to the durable job stream + +Lane: 0.0.6 runtime / public-surface. Control PR #1525. Baseline `origin/main@01aa12b67`. +Research: `slices/research-1398.md`. Target branch `fix/1398-publish-job-executions-to-durable-stream`. + +This plan goes to **PLAN-EVAL in a separate session** before any implementation, per +`supervisor.md`. + +## Contract + +A completed job execution must produce a record on the durable job stream that a consumer, +subscribed **before** the trigger, observes within a bounded time, carrying an identifier that joins +it to its `job.execute` span. The E2E gate that encodes this already exists and is currently +**deferred against this issue**: + +- `behavior.otel.stream-consumer` and `behavior.otel.traces`, registered at + `packages/cli/e2e/src/application/gates/scaffold/otel-gates.ts:52-84`, excluded from + `scaffold.runtime` / `scaffold.runtime.sqlite` via `SCAFFOLD_RUNTIME_DEFERRED_GATES` + (`packages/cli/e2e/suites/scaffold/capability-suites.ts:23-34`), with the deferral reason + *"workers-combined does not install the stream mutation hook"* asserted by + `packages/cli/e2e/tests/presentation/suite-registry_test.ts:204-215`. + +**Definition of done for this issue is therefore mechanical, not rhetorical: those two gates come +out of the deferral list and pass in a real `scaffold.runtime` run.** That is what makes "no +hand-waved timing" enforceable — the bounded-time criterion is the gate's own live SSE loop +(`consume-flow-b-stream.ts:203-230`), not a stopwatch in a PR comment. + +## Root cause (verified) + +The hook is installed only by the workers **API service** +(`plugins/workers/services/src/main.ts:67`). The **background** entrypoints that generated projects +actually run never install it: `plugins/workers/bin/runtime.ts:89-107`, `:110-122`, `:125-152`. +Generated projects run `startCombinedProcess()` +(`plugins/workers/src/adapter/resources/glue/runtime.stub.ts:21-24`) as the Aspire resource +`workers-combined` (`plugins/workers/src/aspire/workers-contribution.ts:12,56-61`). Job definitions +are published from a different site (`plugins/workers/services/src/init.ts:91-111`), which is why +the stream contained exactly three startup snapshots and nothing else. + +This is a **wiring gap**, not a missing feature. The producer, the mapper, the hook, and the +execution record's trace fields all already exist. + +## The decision this plan turns on, and the fact that settles it + +The research correctly flagged a sharp edge: every execution-state mutation fires the hook +(`packages/plugin-workers-core/src/state/execution-state.ts:288,307`), so installing it publishes +~4 records per execution on the same key — and `executionState.create()` +(`plugins/workers/worker/job-dispatcher.ts:74-86`) runs **before** the `job.execute` span exists +(`:91`). Since TC-14 (`select-flow-b-stream-change.ts:122-153`) matches on `correlationId` and then +asserts the matched record's `headers.traceparent` trace id equals the `job.execute` trace id, a +naive "install the hook, publish everything" fix can match a record that fails the assertion. + +Two facts I verified directly resolve this: + +1. **`job.execute` is a child of the stored dispatch traceparent.** + `plugins/workers/worker/job-dispatcher.ts:44` computes + `parentContext = tracedContext?.parentContext ?? getParentContextFromHeaders(traceHeaders)` and + passes it to `traceJobExecution` at `:108`, which forwards it as `parentContext` + (`packages/telemetry/src/instrumentation/worker.ts:300`). So the `job.execute` span **shares its + trace id** with the `traceparent` already stored on the execution record + (`execution-state.ts:74`). +2. **The publish span's trace id comes from the ambient OTel context.** + `packages/plugin-streams-core/src/telemetry/instrumentation.ts:160` calls `startSpan` with no + explicit context, and the header is `formatTraceparent(span.spanContext())` (`:172`). + +**Therefore:** if the mutation hook publishes under a context extracted from the execution record's +own stored `traceparent`, then *every* published record — including the pre-span `create()` one — +carries a header traceparent whose trace id equals the `job.execute` trace id. TC-14 passes no +matter which of the four records the selector matches first. + +That removes the sharp edge without moving `create()` and without restricting publication to +terminal states. + +### Alternatives considered and rejected + +| Option | Why rejected | +| --- | --- | +| Publish only terminal (`complete`) mutations, relying on ambient context | Passes TC-14 by accident of call placement — the join holds only because `complete()` happens to sit inside the span callback (`:165-170`). Any future refactor that moves the call silently breaks the join with no failing test. It also discards running-state visibility that the stream is for. | +| Move `executionState.create()` inside the `job.execute` span | Reorders execution-state semantics (the record must exist before the span names its `executionId`, `:88` → `:103`) to serve a telemetry concern. Wrong direction of dependency. | +| Add `traceparent` to `WorkerExecutionZodSchema` and join on the field | The gate asserts the **header**, not the field (`select-flow-b-stream-change.ts:122-153`). Changing the schema also raises a published-surface version question the research could not settle ("whether the streams schema carries an explicit version marker" — unverified). Not needed for the join. | + +## Locked decisions + +- **D1** — Install the stream mutation hook on the background runtimes in + `plugins/workers/bin/runtime.ts`, mirroring `plugins/workers/services/src/main.ts:65-75`. Cover + `startWorkerProcess` and `startCombinedProcess`; `startSchedulerProcess` only if it owns execution + state. Installation must be idempotent and must not double-install when a process embeds the API + service. +- **D2** — Publish **all** execution mutations (created / updated / deleted), not just terminal ones. +- **D3** — The hook publishes under an OTel context extracted from the execution record's stored + `traceparent`/`tracestate`, so the publish span joins the `job.execute` trace. This is the + join mechanism; it is not optional and it is what the new unit test pins. + + **Amended after PLAN-EVAL (finding F1 — verified).** The mechanism is now explicit rather than + implied: `StreamsTracerPort.startSpan` + (`packages/plugin-streams-core/src/telemetry/instrumentation.ts:92-102`) takes **no parent-context + argument**, so the *only* way for the publish span to inherit the stored trace id is for + `createStreamMutationHook` (`packages/plugin-workers-core/src/streams/producer.ts:108-118`) to wrap + its `producer.upsert(...)` call in + `context.with(extractContext({ traceparent, tracestate }), () => producer.upsert(...))`. Today that + call has no wrapping at all. An implementation that installs the hook without this wrapping will + appear to work and will fail TC-14 on the `create()` record — the exact trap this plan exists to + avoid. +- **D4** — **No change to `WorkerExecutionZodSchema`** and no new public export. If implementation + shows the join cannot be made without a schema field, that is a stop-and-report, not a decision to + take in-slice. +- **D5** — Remove `behavior.otel.stream-consumer` and `behavior.otel.traces` from + `SCAFFOLD_RUNTIME_DEFERRED_GATES` and update the assertion in + `suite-registry_test.ts:204-215`. The deferral list is the issue's own acceptance surface; leaving + it in place while claiming the fix would be a false-done. + +## Slices + +**S0 — environment precondition: RESOLVED before dispatch, no longer blocking.** + +Answered by the orchestrator on 2026-08-12. **`workers-combined` does receive the streams URL.** + +`generate-register-background.ts:200-218` emits +`await <id>.withEnvironment('services__<ref>__http__0', <ref>Endpoint)` for every entry in the +background processor's `PluginReferences`, and +`packages/cli/src/public/features/plugins/install/install-plugin_test.ts:1393-1396` asserts that +`BackgroundProcessors.workers.PluginReferences` is exactly `['streams', 'workers-api']`. That is the +env name `stream-url-resolver.ts:154-190` reads. So the wiring exists by design. + +**Correction, recorded rather than quietly fixed.** I first read +`plugins/workers/src/aspire/workers-contribution.ts:55-63` — which calls `addDenoBackground` with no +reference to `streams` and only `builder.waitFor(combined, api)` — and concluded the env was missing +and the slice needed an extra Aspire slice. That was **wrong**: `PluginReferences` is not derived +from the contribution file at all. It is reconciled from the plugin manifest's +`.withDependencies({ streams: streamsPlugin })` (`plugins/workers/src/public/mod.ts:61`), +independently of install order, per the test above. Checking the mechanism instead of trusting the +first plausible file is what kept an unnecessary slice out of this plan. + +**Residual, for the implementer to confirm in the live run only:** the generated code guards with +`if (<ref>Endpoint)`, so the env is silently omitted if the streams resource exposes no `http` +endpoint at wiring time. That is a runtime observation for S3, not a design unknown. + +**S1 — hook installation + trace-context join** (D1, D2, D3) with unit tests. + +**S2 — un-defer the two OTEL gates** (D5) and make `suite-registry_test.ts` assert the new state. + +**Amended after PLAN-EVAL (finding F2 — verified).** This plan originally named only +`suite-registry_test.ts:204-215`. There is a **second** test that also pins the deferral and must be +updated in the *same commit*: `suite-registry_test.ts:209-234`, +`'runtime suites pin the exact #1398 OTEL deferral without widening it'`, which asserts +`SCAFFOLD_RUNTIME_DEFERRED_GATES` equals the exact two-entry list (`:210-221`) **and** that neither +runtime tier executes a deferred gate (`:223-233`). Both tiers reference the constant +(`capability-suites.ts:211,218`). Emptying the constant without rewriting this test leaves S2 red. +Confirmed by reading both tests directly. + +**S3 — live runtime evidence**: one `deno task e2e:cli run scaffold.runtime --cleanup --format pretty` +run, with the two previously-deferred gates passing, plus the Aspire trace showing producer → +durable stream → SSE consumer in one trace. + +## Tests + +1. **Unit — the join.** Mutation hook publishes under a stored `traceparent`; assert the publish + span's trace id equals that traceparent's trace id. Today's suite + (`packages/plugin-workers-core/tests/streams/workers-streams_test.ts:9-104`) has **no traceparent + assertion** — this is the gap that let the defect ship. +2. **Unit — installation.** The background runtime path installs a mutation hook; a test that fails + if `startCombinedProcess` stops installing it. **There is currently no such test, which is + precisely why `workers-combined` shipped without the hook.** +3. **Unit — pre-span publication.** A mutation emitted *before* any ambient `job.execute` span still + produces a publish span on the stored trace id (pins D3 against the `create()` case). +4. **E2E** — the two un-deferred gates, live. + +## Gates + +Scoped wrappers over `plugins/workers`, `packages/plugin-workers-core`, `packages/cli`; plus +`deno task quality:gate` (mandatory — `packages/**`/`plugins/**` slice) and +`deno task e2e:cli run scaffold.runtime --cleanup --format pretty` for S3. The E2E gate is expensive +and **serialised across this lane** — it does not run concurrently with any other slice's run. + +## Risks + +- **R1 (blocking, S0)** — streams env may not reach `workers-combined`. Resolved before code. +- **R2** — publishing ~4 records per execution against a 256-event / 1 MiB bounded buffer + (`producer-contract-v1.ts:47-55`) raises volume on busy workers. Not a correctness risk (drops are + metered and settled), but worth a recorded note; #1405's reason taxonomy is what makes any such + drop legible, which is why it lands first. +- **R3** — the two undeclared `@netscript/plugin-streams-core` imports in + `packages/plugin-workers-core/deno.json` and `plugins/triggers/deno.json` are real but + **out of scope** here. If `publish:dry-run` flags them, they get their own issue rather than + being absorbed into this PR. + +## Acceptance mapping (#1398) + +| Box | Satisfied by | Evidence | +| --- | --- | --- | +| Completed execution results in a published record | D1 + D2 | unit test 2 + S3 live run | +| Record carries an identifier joining it to `job.execute` | D3 | unit tests 1 and 3 + TC-14 in S3 | +| Live subscription opened before trigger observes it within bounded time | D5 | `behavior.otel.stream-consumer` passing live (`consume-flow-b-stream.ts:203-230`) | +| A test asserts the join | tests 1–3 + un-deferred E2E | red-on-regression demonstrated | diff --git a/.llm/runs/release-0.0.6-features--orchestration/retrospective.md b/.llm/runs/release-0.0.6-features--orchestration/retrospective.md new file mode 100644 index 0000000000..3874acbdad --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/retrospective.md @@ -0,0 +1,105 @@ +# Retrospective — 0.0.6 runtime / public-surface lane + +Factual account of one topical milestone lane, 2026-08-12. Both owned issues landed on `main`: +**#1405 → `8ff1bcb8f`**, **#1398 → `d7e2b67b2`**. + +Per the brief: this records what the run's own evidence supports. Where a lesson is a candidate +rather than a demonstrated rule, it says so. **No policy is promoted from a single occurrence.** + +## What the lane produced + +| Issue | PR | Merge | Evidence that closed it | +| --- | --- | --- | --- | +| #1405 durable producer rejection taxonomy | #1528 | `8ff1bcb8f` | 4 negative tests, each pinned to its own reason; full CI green; 7-check pre-merge gate | +| #1398 job executions to the durable stream | #1536 | `d7e2b67b2` | Two formerly-deferred OTEL gates un-deferred and **passing live by name on both CI runtime tiers**; 7-check pre-merge gate | + +Two follow-ups filed from inside the run: **#1542** (`quality:gate` root coverage) and **#1543** +(undeclared `plugin-streams-core` imports). + +## The one thing most worth carrying forward + +**#1398's definition of done was made mechanical instead of rhetorical, and that decision did the +work.** The issue's acceptance included "a live subscription observes the execution record within a +bounded time" — a criterion trivially satisfiable with a confident sentence. The plan instead bound +it to two gates the repo had **already deferred against this very issue** +(`SCAFFOLD_RUNTIME_DEFERRED_GATES`, reason: *"workers-combined does not install the stream mutation +hook"*). Done then meant: remove the deferral, and have the gates pass. + +That reframing is what made every later step falsifiable. It is also what made two red local runs +*informative* rather than negotiable — there was nothing to argue about, the gates either ran or +they did not. + +**[observed, single run]** — one instance. The candidate rule is: *when an issue's acceptance is +observational, look for an existing deferred/skipped gate that encodes it before writing a new +criterion.* It worked once here; it is not yet a rule. + +## Layered review caught what each layer above it missed + +Concretely, on the same defect class — stale pins of the #1398 deferral: + +| Layer | Found | +| --- | --- | +| Orchestrator plan | `suite-registry_test.ts:204-215` | +| PLAN-EVAL (MiniMax M3) | a **second** pin at `:209-234` | +| Implementer (Codex Sol medium) | a **third** in `suite-runner_test.ts`, by running full package tests rather than only the tests it was pointed at | + +PLAN-EVAL also caught **F1**: the trace-context join was implicit in the plan and would have failed +*silently* — `StreamsTracerPort.startSpan` takes no parent-context argument, so without an explicit +`context.with(...)` wrapper the hook looks correct and fails TC-14 on the pre-span `create()` record. +The final Qwen IMPL-EVAL then traced that same mechanism end-to-end independently. + +**[observed]** Each layer found something the previous one could not. This is the strongest evidence +in the run for the existing generator ≠ evaluator invariant, and it cost two evaluator sessions. + +## Verification discipline that changed an outcome + +- **Negative-case proof.** For #1405 the orchestrator reverted both fixes (29/34 → 5 failures) and + the IMPL-EVAL reverted each **individually**, showing each test fails for its *own* reason. The + aggregate check would have passed a suite where all four tests rode on one mechanism. + Acceptance box 4 asks for the per-reason property; only the individual reverts demonstrate it. +- **Reading logs, not summaries.** Both OTEL gates were confirmed **by name** in job logs, with + `skipped=0` treated as load-bearing alongside `failed=0`. `gh pr checks` twice reported something + materially misleading (all-`skipping` on a draft; a stale `close-gate` red from a job that predated + the changes it complained about). +- **Head discipline.** When #1536's head changed mid-flight, gate evidence was re-read against the + new head. The new job ids (`94073971396` / `94073971501`) differ from the pre-sync pair, which is + the concrete proof that reusing the earlier verification would have cited a head no longer on the + PR. + +## Mistakes made by this orchestrator + +Recorded because a retrospective that only lists what went well is not evidence of anything. + +1. **Wrong default on the IMPL-EVAL waiver** (D-3). The brief said a waiver was "acceptable" for the + #1405 class; that was read as a blocked-transport fallback rather than the class default, costing + one unnecessary evaluator run. The ruling arrived after #1405 had already merged, so it changed + policy, not that outcome. +2. **A scope inference from the wrong file.** Read `workers-contribution.ts`, saw no `streams` + reference on the background resource, and concluded the env was unwired — which would have added + an unnecessary Aspire slice. `PluginReferences` is reconciled from the plugin manifest, not that + file. Verifying the *mechanism* rather than the first plausible file is what caught it. +3. **A broken gate command in my own slice brief** — `deno test <path>` without `--allow-env`. The + implementer reported the red with its cause instead of hiding it; the brief was wrong, not the + slice. +4. **A watcher that could not have fired**, twice: one keyed on comment count against a tool that + edits in place, and one waiting for a dispatcher that did not exist in the PR's merge ref. + +## Candidate rules — not promoted + +Each appeared **once**. Recorded so a future run can confirm or falsify, not encoded as doctrine: + +- For `pull_request` events, a workflow must exist in the PR's **merge ref**; a newly merged + dispatcher cannot fire on an older head. *Check presence before diagnosing a trigger failure.* +- A local expensive-suite red is not a flake until a clean control run exists elsewhere. +- Watch a run's status and its terminal marker, never comment count, for tools that update in place. +- Re-verify the `status:` label set after automated phase transitions, not only manual edits. + +## Open items leaving this lane + +- **#1542** — `quality:gate` roots omit published packages. Hit independently by three sessions here; + both merge records in this lane explicitly rest on **explicit target scans** rather than the repo + gate. The most consequential thing this lane found and did not fix. +- **#1543** — undeclared `plugin-streams-core` imports, filed as **unverified**: nobody checked + whether `deno publish` rejects them, and the issue makes that evidence its first acceptance box + rather than asserting a defect. +- Canary and stable cut are **root's**, not this lane's. Nothing here was published. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/1405/context-pack.md b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/context-pack.md new file mode 100644 index 0000000000..2883843010 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/context-pack.md @@ -0,0 +1,14 @@ +# Context pack — #1405 + +- Branch: `fix/1405-durable-producer-rejection-taxonomy` at baseline `01aa12b67`. +- Archetype: 3 — Runtime / Behavior; no scope overlay. +- Locked reasons: graceful close drain is `producer-stopping`; non-retryable transport failure is + `transport-refused`; retryable failures at `maxAttempts` remain `retry-exhausted`. +- Behavior boundary: reason strings only; no acceptance, retry-count, settlement class, delivery, + cancellation, telemetry-classification, #1398, or scaffold changes. +- PLAN-EVAL: N/A per the owner brief and orchestration supervisor record. +- Implementation gates: scoped wrappers, quality gate, target quality scan, doc lint, JSR audit, + focused tests, telemetry guard, and package-configured full suite are green. +- Gate caveat: the brief's bare `deno test packages/plugin-streams-core` exits 1 because it omits + the package suite's required env permission; the configured `--allow-all` task passes 33/33. +- Orchestrator retains slice review, IMPL-EVAL, ready-state, merge, and release authority. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/1405/drift.md b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/drift.md new file mode 100644 index 0000000000..f75acee9c0 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/drift.md @@ -0,0 +1,20 @@ +# Drift — #1405 + +## 2026-08-12 — bare test command lacks package permissions + +- Severity: minor gate-invocation mismatch. +- Expected: `deno test packages/plugin-streams-core` passes as written in the brief. +- Observed: it exits 1 with 19 `NotCapable` failures because tests read + `DURABLE_STREAMS_URL` and the command grants no env permission; 14 permission-free tests pass. +- Action: preserved the red evidence and ran the checked-in package task, which grants the package's + declared test permissions and passes 33/33. No test or permission surface was edited. + +## 2026-08-12 — root quality gate omits the target package + +- Severity: minor gate-coverage limitation. +- Expected: mandatory `quality:gate` provides changed-package quality evidence. +- Observed: it exits 0, but its configured scan/doctrine roots do not include + `packages/plugin-streams-core`. +- Action: ran an explicit target quality scan (`findings=[]`, `allowCount=0`) and target doctrine + audit. The doctrine audit has no failures and one 500-line-cap warning: the supervisor is 515 + lines after this locked change (baseline 497). A structural split is outside the no-refactor slice. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/1405/supervisor.md b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/supervisor.md new file mode 100644 index 0000000000..8651922640 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/supervisor.md @@ -0,0 +1,14 @@ +# Slice identity — #1405 + +| Field | Value | +| --- | --- | +| Lane | `light_implementation` | +| Generator | Codex · OpenAI · GPT-5.6 Sol · low | +| Worktree | `/home/codex/repos/ns006-1405` | +| Branch | `fix/1405-durable-producer-rejection-taxonomy` | +| Baseline | `origin/main@01aa12b67` | +| Issue | `#1405` | +| PLAN-EVAL | N/A — fully specified mechanical slice | +| IMPL-EVAL | Separate orchestrator-owned evaluator session | + +The milestone orchestrator retains slice review, merge, publish, and release authority. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/1405/worklog.md b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/worklog.md new file mode 100644 index 0000000000..5c641fc6c4 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/1405/worklog.md @@ -0,0 +1,67 @@ +# Worklog — #1405 durable producer rejection taxonomy + +## Design + +### Public surface + +- Add exactly `transport-refused` to `StreamWriteUnknownReasonV1`. +- Reuse the existing `producer-stopping` rejection reason during graceful close drain. +- Keep the existing root re-exports; add no entrypoint or export. + +### Domain vocabulary and ports + +- Closing intent is private supervisor state, visible from `close()` entry until shutdown settles. +- A non-retryable transport failure is a positive refusal; retryable failure at the attempt bound is + exhaustion. +- Existing transport, clock, random, queue, and lifecycle ports remain unchanged. + +### Constants + +- No new constant collection is required; the published reason unions remain the finite vocabulary. + +### Commit slice + +- S1 changes only reason selection and adds deterministic close/refusal/exhaustion tests. Proving + gates are the focused tests plus the complete gate set named in the slice brief. + +### Deferred scope + +- No acceptance, delivery, cancellation, retry-count, telemetry-classification, scaffold, #1398, + merge, or release changes. + +### Contributor path + +- Start with `producer-contract-v1.ts` for reason vocabulary, then follow supervisor settlement and + rejection selection into the contract behavior tests. + +## Phase status + +- `PLAN-EVAL: N/A` — the owner brief and orchestrator research fully specify the two selectors, + locked vocabulary, negative tests, boundaries, and gates. +- Implementation and generator gates complete on the assigned `light_implementation` lane. +- Separate orchestrator-owned slice review and IMPL-EVAL remain pending. + +## Evidence + +| Gate | Exit | Result | +| --- | ---: | --- | +| Focused contract behavior | 0 | 8 passed, 0 failed; includes all four new negative tests. | +| Existing telemetry classification guard | 0 | 2 passed, 0 failed. | +| Scoped check wrapper | 0 | 43 files, 1 batch, 0 failures/occurrences. | +| Scoped lint wrapper | 0 | 43 files, 1 batch, 0 occurrences. | +| Scoped format wrapper | 0 | 43 files, 1 batch, 0 findings. | +| Post-format reason grep | 0 | `transport-refused` and `producer-stopping` remain in contract, selector, and tests. | +| `deno task quality:gate` | 0 | `quality:scan` and `arch:check` completed; existing repository warnings only. The configured quality roots omit this package. | +| Explicit target quality scan | 0 | `packages/plugin-streams-core/src`; `findings=[]`, `allowCount=0`. | +| Explicit target doctrine audit | 0 | `FAIL=0 WARN=1 INFO=1`; supervisor is 515 lines versus the 500-line advisory cap, and architecture docs are informationally absent. | +| Full export-map doc lint | 0 | 4 entrypoints; `totalErrors=0`, `totalMissingJSDoc=0`. | +| JSR audit | 0 | dry-run OK; one non-failing slow-types banner warning. | +| Brief's exact `deno test packages/plugin-streams-core` | 1 | 14 passed, 19 failed solely with `NotCapable` because the command omits `--allow-env`. | +| Package-declared `deno task --cwd packages/plugin-streams-core test` | 0 | 33 passed, 0 failed. | + +## Reconcile + +- #1405 remains the only resolving issue; the draft PR must carry `Closes #1405` and target + `main`. No #1398 surface was touched. +- No new dependency, export-map key, lint suppression, unsafe cast, `any`, or architecture debt was + introduced. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/codex-thread-ids.md b/.llm/runs/release-0.0.6-features--orchestration/slices/codex-thread-ids.md new file mode 100644 index 0000000000..eceedf2d35 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-1398 — Codex implementation thread +- **Thread / session id:** `019ff4ff-a633-7062-ae9c-21930930b5d6` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T10-03-42-019ff4ff-a633-7062-ae9c-21930930b5d6.jsonl` +- **Worktree:** `/home/codex/repos/ns006-1398` +- **Branch:** `fix/1398-publish-job-executions-to-durable-stream` @ `01aa12b67` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1398-publish-job-executions-to-durable-stream`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=medium +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=medium +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-1398-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff4ff-a633-7062-ae9c-21930930b5d6 -- "<follow-up>" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1398.md b/.llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1398.md new file mode 100644 index 0000000000..c68d7eccd6 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1398.md @@ -0,0 +1,105 @@ +# IMPL-EVAL — #1398 / PR #1536 (verbatim verdict) + +| Field | Value | +| --- | --- | +| Phase | IMPL-EVAL | +| Subject | head `f7d503fee` vs trusted base `281ab76887`, PR #1536, closes #1398 | +| Evaluator | `openrouter/qwen/qwen3.8-max` via OpenHands | +| Route | **automatic phase dispatcher** (`openhands-phase-eval.yml`), triggered by the `status:impl-eval` label re-entry with the one-shot `eval:model:qwen` override | +| Session | separate from generator and supervisor; no manual dispatch, no local evaluator | +| Run | `31584188459`, attempt 1, `conclusion: success` | +| Verdict | **PASS** (`verdict_source: summary-file`) | + +Verbatim below, as posted to PR #1536. + +--- + +<!-- openhands-agent-summary --> +<!-- openhands-run: {"run_id":31584188459,"attempt":1,"conclusion":"success","state":"completed","verdict":"PASS","verdict_source":"summary-file"} --> +## OpenHands Agent — Completed + +OPENHANDS_VERDICT: PASS + +Model: `openrouter/qwen/qwen3.8-max` +Provider: `OPENROUTER` +Job status: success +Verdict source: summary-file +Verdict comment: https://github.com/rickylabs/netscript/pull/1536#issuecomment-5265412502 +Summary source: agent +Housekeeping: ack=success trace=success commit-artifacts=skipped replies=skipped draft-pr=skipped +Trace: .llm/tmp/run/openhands/pr-1536/run-31584188459-1 (in the Actions artifact) +Artifact: openhands-agent-31584188459-1 + +OPENHANDS_VERDICT: PASS + +# IMPL-EVAL — PR #1536 (fix(workers): publish job executions to the durable stream on the job.execute trace) + +**Subject:** head `f7d503fee1e312335d432cfee8b9a870ddaccbb4` vs trusted base `281ab76887`, PR #1536 closes #1398. +**Protocol:** `.llm/harness/evaluator/protocol.md` + `verdict-definitions.md`, read-only. + +## Summary + +The implementation satisfies the approved plan (S1/S2/S3, D1–D5), and every static gate and +local runtime guard passes. The decisive live runtime evidence — the two formerly deferred OTEL +gates `behavior.otel.stream-consumer` and `behavior.otel.traces` — now passes green **in CI at +this exact head** on both runtime tiers (postgres and sqlite), superseding the locally red S3 +attempt that died on a transient dependency `fetch failed` before those gates could run. The +only red check (`close-gate`) is a merge-time bookkeeping step, not an implementation defect. + +## Changes (verified against plan D1–D5) + +- `packages/plugin-workers-core/src/streams/producer.ts` — `createStreamMutationHook` wraps every + publish (`upsert`/`delete`) in `withContext(extractContext({traceparent, tracestate}), …)` (D3, + including the pre-span `create()` trap); `WorkerExecutionRecord` type acknowledges the + already-stored W3C fields (no schema change, D4 honored). +- `plugins/workers/bin/runtime.ts` — hook installed in `startWorkerProcess` and + `startCombinedProcess`; scheduler-only correctly untouched (D1). +- `plugins/workers/tests/runtime/background-stream-hook_test.ts` — new installation guard (D1). +- `packages/plugin-workers-core/tests/streams/workers-streams_test.ts` — stored-context and + pre-span trace-id join tests (required tests 1 & 3). +- `packages/cli/e2e/**` — both OTEL gates restored into `RUNTIME_GATES`, deferral constant + emptied, both pinned tests rewritten (D5, F2 honored; third stale pin in `suite-runner_test.ts` + recorded as drift). +- `.llm/runs/.../slices/worklog-1398.md` — design checkpoint, per-slice gates, negative-guard + evidence, honest S3 red record. + +## Validation + +- PLAN-EVAL: PASS (MiniMax M3, separate session) with F1/F2 folded into plan; run-dir artifacts + confirmed on `chore/release-0.0.6-features-orchestration`. +- Re-run at head: `deno task --cwd packages/plugin-workers-core test` → 27/27 passed; + background-hook install test → 1/1 passed. +- CI at head: `scaffold-runtime (postgres)` → `Summary: passed=88 failed=0 skipped=0` with both + restored OTEL gates executing; `scaffold-runtime-sqlite` → `passed=83 failed=0 skipped=0`; + `check-test`, `quality`, `code-quality`, `deps-report`, `surface-diff` all PASS. +- Diff hygiene: no new `as unknown as` / `deno-lint-ignore` / `@ts-ignore` / `any` (excluding + run artifacts); no `deno.lock` churn; no new public exports. +- Context-join mechanism verified end-to-end: hook → `producer.upsert` → `DurableStreamProducer` + `#startPublish` → `instrumentation.startPublish` (`tracer.startSpan` on ambient context), so + `withContext` makes the publish span carry the stored trace id; pinned by both new unit tests. + +## Responses to review/issue context + +- Close-gate failure (`check-close-gate.ts`, run 31584138064): issue #1398's four acceptance + boxes and the PR DoD's final three boxes are unticked. Root cause per the gate's own output: + the acceptance-evidence mirror only applies when `status:ready-merge` is present ("Mirror + skipped because live PR labels do not include status:ready-merge"), and the PR body's + `acceptance-evidence` block already maps all four boxes. This is the same mechanism by which + merged sibling PR #1528 was closed (pre-merge gate record: boxes "ticked by the evidence mirror + on `status:ready-merge`"). Remediable by the orchestrator at merge time. +- S3 local red (transient `fetch failed` in `runtime.flow-b-fixture`) is superseded by the green + CI one-pass runs at the same head; the worklog's honest red record stands as process evidence. + +## Remaining risks + +- Advisory: until the orchestrator applies `status:ready-merge` and the mirror ticks issue #1398, + `close-gate` remains red; the PR body DoD line about the one-pass `scaffold.runtime` should be + re-evidenced with the CI run URLs when ticking. +- Advisory: `quality:gate` configured roots still omit `packages/plugin-workers-core`/`cli/e2e`; + covered here by the explicit scans the worklog records (gate-coverage gap, pre-existing). +- Undeclared `@netscript/plugin-streams-core` imports in two `deno.json` files remain tracked out + of scope per plan/PLAN-EVAL F5. + +OPENHANDS_VERDICT: PASS + +Run: https://github.com/rickylabs/netscript/actions/runs/31584188459 diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1405.md b/.llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1405.md new file mode 100644 index 0000000000..5675d416cd --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/evaluate-1405.md @@ -0,0 +1,45 @@ +# IMPL-EVAL — #1405 / PR #1528 (verbatim verdict) + +| Field | Value | +| --- | --- | +| Phase | IMPL-EVAL | +| Subject | commit `c491c6989`, PR #1528, closes #1405 | +| Evaluator | DeepSeek V4 Flash 0731 · max, `claude-evaluator-deepseek-v4-flash-0731` preset | +| Session | fresh, separate from generator and supervisor; worktree `/home/codex/repos/ns006-1405-impleval` @ `c491c6989` | +| Route reason | small-impl evaluator lane; local fallback per `drift.md` D-2 (#1524 still open) | +| Duration | 642,836 ms · 5,226 stream events · `is_error: False` | +| Raw stream | `impl-eval-1405-raw.md` | + +Verbatim below; nothing edited. + +--- + +**[PHASE: IMPL-EVAL] [VERDICT: PASS]** + +Issue #1405's two reason misdescriptions are fixed correctly, behaviour (accepted/rejected/delivered + their `status` classification) is fully preserved, and all four added tests independently guard their own fix. + +### Verified +- **Package tests 33/33 green** — `deno task --cwd packages/plugin-streams-core test` → `ok | 33 passed | 0 failed` (includes all 4 new tests). +- **Scoped check** — `run-deno-check.ts --root packages/plugin-streams-core --ext ts,tsx` → 43 files, 1 batch, 0 failures. +- **`deno task quality:gate`** exits 0 — but its configured scan/doctrine roots **omit** `packages/plugin-streams-core` (deno.json:156 `arch:check` root list). I ran the compensating target gates independently: target quality scan → `findings=[]`, `allowCount=0`; target doctrine audit → `FAIL=0 WARN=1 INFO=1` (supervisor 515 lines vs 500 cap — advisory, pre-existing baseline 497, recorded in drift). +- **Revert-A isolation** — reverting only the `#closing`/`#writeRejectionReason` change makes only "close drain rejects a concurrent write as producer-stopping" (test 1) fail; tests 2,3,4 pass. +- **Revert-B isolation** — reverting only the `#failActive` reason selection (+ connect-guard split) makes only "first append refusal …" (test 2) and "non-retryable reconnect failure …" (test 4) fail; test 1 and the retry-exhaustion guard (test 3) pass. +- **Tree restored** — `git status` porcelain empty after reverts; no diff remaining. +- **Behaviour preservation** — the only functional deltas in the diff are reason strings (`#failActive` ternary, `#writeRejectionReason` arms) plus a behavior-neutral connect-guard refactor (supervisor.ts:308-315 — old `!isRetryable || attempt===max` split into two identical conditional pairs, same calls/returns). + +### Findings +1. **C2 (advisory) Dead `?? 'producer-failed'` fallback** — `create-durable-stream.ts:132,160`. `writeRejectionReason()` returns `undefined` only when `#accepted === true` (supervisor.ts:483-484); both calls are behind `if (this.closed)` where `closed === !#accepted`. `#accepted` is only ever set `false` (stop/close/fail) and never back to `true`, so the `??` can never fire — not even via a TOCTOU (it cannot flip false→true). Harmless, unreachable defensive code; not blocking. Suggested cleanup: drop the `?? 'producer-failed'` and rely on the switch's `default` returning `producer-failed` when not closing. +2. **C4 (advisory) Reason-vocabulary doc** — the new `transport-refused` is documented only via JSDoc on the union member (producer-contract-v1.ts:86-87). `README.md` describes outcome `status`es generically (README.md:93) and mentions no reason list, so no consumer doc is stale. Additive union member, check passes 43/43, no exhaustive switch exists on `StreamWriteUnknownReasonV1` anywhere in the repo (grep-confirmed; only producer + tests reference it). No blocking doc gap. +3. **C4 (advisory, recorded)** — `quality:gate`/`arch:check` do not cover this package; gate-coverage gap already logged in `drift.md`. The `research-1405.md` cited in the brief is absent from the run dir (only `worklog.md`/`drift.md` exist); the worklog carries the equivalent analysis. Both are run-artifact issues external to the committed change, not code defects. + +### Acceptance box check (#1405) +- Box 1 (close-drain reason names closing state, ≠ `producer-failed`): **satisfied** — `#closing` set in `close()` (supervisor.ts:222), default arm returns `producer-stopping` for non-terminal states during drain (supervisor.ts:494); test asserts `producer-stopping` and `assertNotEquals producer-failed`. +- Box 2 (non-retryable append on attempt 1 names refusal, ≠ `retry-exhausted`): **satisfied** — `#failActive` returns `transport-refused` for `!isRetryable(failure)` (supervisor.ts:430-434); test asserts `transport-refused`, `assertNotEquals retry-exhausted`, `appendCalls === 1`. +- Box 3 (genuine exhaustion keeps its own reason): **satisfied** — retryable-at-bound still yields `retry-exhausted` (isRetryable true ⇒ not `transport-refused`); test 3 asserts it and `assertNotEquals transport-refused`. +- Box 4 (tests assert each reason for its own state): **satisfied** — one-at-a-time reverts (above) prove test 1 breaks only under fix-A revert and tests 2/4 only under fix-B revert, i.e. they do not collapse on one mechanism. +- Box 5 (no change to accepted/rejected/delivered, reasons only): **satisfied** — all deltas are `status`-preserving reason strings; no admission gate, settlement `status`, or retry count changed. + +### Next +- Merge is clear. Optionally (non-blocking): remove the unreachable `?? 'producer-failed'` fallback and record the supervisor 515-line split as debt if reopening it is ever scoped. Owner: R. Chautems / slice owner on #1405 for the fallback cleanup; no blocking action required for merge. + +Sources for the behavioural claims above: `durable-stream-producer-supervisor.ts` (lines 64-65, 221-222, 308-315, 356-363, 423-438, 482-496), `durable-stream-producer-support.ts:90-92` (`isRetryable` = retryable|stale-epoch), `create-durable-stream.ts:131-133,159-161`, `producer-contract-v1.ts:83-89`. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1398-prompt.md b/.llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1398-prompt.md new file mode 100644 index 0000000000..37da53bafe --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1398-prompt.md @@ -0,0 +1,130 @@ +use harness + +# IMPL-EVAL — NetScript issue #1398 (PR #1536) + +You are a **formal IMPL-EVAL evaluator** in a fresh session. You did not write this code, did not +supervise it, and are not its reviewer. Try to **break** the claim that it is correct and complete, +then return a verdict. + +Worktree: `/home/codex/repos/ns006-1398-impleval` (detached). **Read-only** — do not edit, commit, +or push. You may run read-only commands and tests. + +## SKILL + +- `netscript-harness` — read `.llm/harness/evaluator/protocol.md` and `verdict-definitions.md`. +- `netscript-doctrine` — plugins are Archetype 5 / thin; `-core` owns convention-bearing primitives. +- `netscript-cli` — the `e2e:cli` suite surface. + +## The change + +Three commits on `fix/1398-publish-job-executions-to-durable-stream`. Job executions were never +published to the durable job stream because the workers **API service** installs the execution-state +mutation hook (`plugins/workers/services/src/main.ts:67`) but the **background** entrypoints that +generated projects actually run never did (`plugins/workers/bin/runtime.ts`). Read the live issue +#1398 for its four acceptance boxes. + +Context you must read first, because the plan was already adversarially reviewed and you should not +repeat that work: + +- `.llm/runs/release-0.0.6-features--orchestration/plan.md` — the approved plan, D1–D5. +- `.llm/runs/release-0.0.6-features--orchestration/plan-eval.md` — the PLAN-EVAL verdict. +- `.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1398.md` — the implementer's own + evidence, including **two red live runs**. + +These files live on the orchestration branch `chore/release-0.0.6-features-orchestration`, not on the +PR branch. If they are not present in your worktree, read them with +`git show chore/release-0.0.6-features-orchestration:<path>`. + +## What to attack, in priority order + +1. **Does the trace join actually hold at runtime, not just in the unit test?** D3 wraps publication + in `withContext(extractContext({traceparent, tracestate}), …)` + (`packages/plugin-workers-core/src/streams/producer.ts:113-128`). The unit tests assert the + publish span's trace id. Check the assumption underneath: does `extractContext` return a usable + context for a **remote** traceparent, and does `withContext` keep it active across the `await` + boundary inside `producer.upsert`'s async path? If the context is lost after the first `await`, + the unit test could pass while real publication carries the wrong trace id. **This is the highest + value thing you can check.** +2. **Is the hook installed exactly where execution state is mutated?** It is installed in + `startWorkerProcess` and `startCombinedProcess` but **not** `startSchedulerProcess`. Verify the + scheduler genuinely does not mutate execution state. If it does, executions from scheduled jobs + are still invisible and the issue is only half fixed. +3. **Double-install and ordering.** `setMutationHook` is destructive (single field). In a combined + process, can the hook be installed and then replaced or cleared by anything later — for example by + `registerProjectJobs` or the API service sharing a runtime? Installation happens *before* + `registerProjectJobs`; confirm nothing in that path resets it. +4. **Volume and buffer pressure.** D2 publishes every mutation, ~4 per execution on the same key, + against a 256-event / 1 MiB bounded buffer. Look for a realistic path where a busy worker drops + execution records, and say whether drops are still metered and settled. +5. **The un-deferral is complete and honest.** `SCAFFOLD_RUNTIME_DEFERRED_GATES` is now empty and + both OTEL gates were added to `RUNTIME_GATES`. Confirm **all** stale pins were updated — the + implementer found a third in `suite-runner_test.ts` that the plan and PLAN-EVAL both missed. + Search for a fourth. Confirm the empty-array type change + (`readonly DeferredGate[]` instead of `as const satisfies`) did not weaken any assertion. +6. **Test quality.** Revert each fix **individually** and confirm each test fails for **its own** + reason: (a) remove the `withContext` wrapper → trace tests fail; (b) remove the + `startCombinedProcess` hook install → installation test fails; (c) remove the two gates from + `RUNTIME_GATES` → suite-registry tests fail. If a test passes while its own defect is + reintroduced, that is blocking. Restore the tree and confirm it is clean. +7. **Scope.** D4 forbids any `WorkerExecutionZodSchema` change; confirm `streams/schema.ts` is + untouched. Confirm no #1405 surface, no dependency/export-map change, no new lint suppression or + unsafe cast. + +## On the live gate — read carefully + +Two local `scaffold.runtime` runs went red **before** reaching `behavior.otel.stream-consumer` and +`behavior.otel.traces`: run 1 at `runtime.flow-b-fixture` (`generate plugins: fetch failed`), run 2 +at `runtime.wait.triggers-api` (unhealthy after 120 s). Neither gave the restored gates a verdict. +The orchestrator escalated to CI, where the same suite runs. + +**Do not treat CI's result as your own finding, and do not re-run `scaffold.runtime` yourself** — it +is expensive and serialised across this lane. Your job is the code and the tests. State explicitly in +your verdict whether, in your judgement, the change could plausibly cause a `triggers-api` health +timeout; #1398 touches workers and the e2e suite definitions, not triggers, but say what the code +supports rather than what is convenient. + +## Gates to run yourself + +```bash +deno task --cwd packages/plugin-workers-core test +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root plugins/workers --ext ts,tsx +deno task quality:gate +``` + +Use the package-declared `deno task --cwd <pkg> test`; a bare `deno test <path>` omits `--allow-env` +and exits 1 on `NotCapable`. Note `quality:gate`'s configured roots do **not** cover +`packages/plugin-workers-core` or `packages/cli/e2e` — run explicit target scans for those and say so. + +## Output + +Return your verdict **as text in your final message** — do not write files. + +``` +**[PHASE: IMPL-EVAL] [VERDICT: PASS|FAIL_FIX|FAIL_RESCOPE|FAIL_DEBT]** + +<one-line headline> + +### Verified +- <claim> — <how you checked, path:line, command output> + +### Findings +1. **C1 <title>** — what is wrong, where, the fix, blocking or advisory. + +### Acceptance box check (#1398) +- Box 1 …: satisfied / not satisfied / not-yet-live-verified — evidence + +### Live-gate judgement +- Could this change plausibly cause the triggers-api timeout? <yes/no/cannot determine> — reasoning. + +### Next +- <action + owner> +``` + +Rules: every finding cites `path:line`. State the verdict token exactly. Distinguish blocking from +advisory. Write "could not verify" rather than guessing. **No praise, no quality adjectives, no +overall-assessment paragraph** — findings and evidence only. An empty `Findings` list is fine if the +code is sound, but only after genuinely attempting 1–7. + +Note on acceptance box 3 ("a live subscription … within a bounded time"): if the live gate has not +produced a verdict, the correct answer is **not-yet-live-verified**, not "satisfied". Do not tick it +from unit tests. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1405-prompt.md b/.llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1405-prompt.md new file mode 100644 index 0000000000..a0dd70530e --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/impl-eval-1405-prompt.md @@ -0,0 +1,101 @@ +use harness + +# IMPL-EVAL — NetScript issue #1405 (PR #1528) + +You are a **formal IMPL-EVAL evaluator** in a fresh session. You did not write this code, you did not +supervise it, and you are not its reviewer. Your job is to try to **break** the claim that this +change is correct and complete, then return a verdict. + +Worktree: `/home/codex/repos/ns006-1405-impleval` (detached at `c491c6989`). **Read-only** — do not +edit, commit, or push. You may run read-only commands and tests. + +## SKILL + +- `netscript-harness` — read `.llm/harness/evaluator/protocol.md` and + `.llm/harness/evaluator/verdict-definitions.md`. +- `netscript-doctrine` — `packages/plugin-streams-core` is framework code; `domain/` contracts are + published surface. + +## The change under evaluation + +Commit `c491c6989`, "fix(streams): distinguish producer refusal reasons". It closes #1405, whose +complaint is that two settled **reason strings** misdescribe the state that produced them: + +1. a write rejected during the graceful close-drain reported `producer-failed` though the producer + was healthy and closing; +2. a non-retryable append failure on attempt 1 reported `retry-exhausted` though nothing was + exhausted. + +Read the live issue #1405 for its five acceptance boxes. Read +`.llm/runs/release-0.0.6-features--orchestration/slices/research-1405.md` for the line-cited +analysis the change was built from, and `slices/1405/worklog.md` for the implementer's own evidence. + +## What to attack + +1. **Behaviour preservation — the load-bearing claim.** The issue requires *no change to which + writes are accepted, rejected, cancelled or delivered — reasons only*. Verify this. In particular + `#writeRejectionReason()` gained a `case 'failed'` arm and a `#closing`-aware `default`; prove or + disprove that no state now yields a different **classification** (not just a different string) + than before. `create-durable-stream.ts` deleted its local `stateRejection()` and now delegates to + `supervisor.writeRejectionReason() ?? 'producer-failed'` — is that `??` fallback reachable, and + if so with what result? +2. **Is `producer-stopping` reachable when it should not be?** `#closing` is set in `close()` and + never cleared. Find any path where a producer that is *not* closing reports `producer-stopping`, + or where a genuinely failed producer is now masked as merely stopping. A failure masked as a + graceful close would be a **worse** defect than the one being fixed — this is the highest-value + thing you can look for. +3. **`transport-refused` correctness.** `#failActive` now selects on `isRetryable(failure)`. Check + every call site (the connect guard was split into two). Is there a failure kind where the new + branch mislabels exhaustion as refusal, or vice versa? Check what `isRetryable` actually + considers retryable, including `stale-epoch`. +4. **Public surface.** `transport-refused` is a new member of the exported + `StreamWriteUnknownReasonV1`. Is anything in the repo switching exhaustively on that union that + now fails to compile or silently falls through? Is the addition documented? +5. **Test quality — do the guards fire?** Four negative tests were added. The orchestrator already + demonstrated that reverting both fixes turns the suite red (29 passed / 5 failed). Go further: + revert them **one at a time** and confirm each test fails for *its own* reason rather than all + four depending on one mechanism. If a test passes while its own defect is reintroduced, that is a + blocking finding. Restore the tree afterwards and confirm it is clean. +6. **Anything the implementer or reviewer missed.** Both are recorded as having found nothing + blocking. Assume they are wrong somewhere and look. + +## Gates to run yourself + +```bash +deno task --cwd packages/plugin-streams-core test +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/plugin-streams-core --ext ts,tsx +deno task quality:gate +``` + +Note: the bare `deno test packages/plugin-streams-core` exits 1 on `NotCapable` permission errors +because it omits `--allow-env`; that is a defect in the original brief, already recorded, not a +finding. Use the package-declared task. + +## Output + +Return your verdict **as text in your final message** — do not write files. Structure: + +``` +**[PHASE: IMPL-EVAL] [VERDICT: PASS|FAIL_FIX|FAIL_RESCOPE|FAIL_DEBT]** + +<one-line headline> + +### Verified +- <claim> — <how you checked, with path:line and any command output> + +### Findings +1. **C1 <title>** — what is wrong, where, the concrete fix, and whether it blocks merge. +... + +### Acceptance box check (#1405) +- Box 1 …: satisfied / not satisfied — evidence + +### Next +- <action + owner> +``` + +Rules: every finding cites `path:line`. State the verdict token exactly. Distinguish blocking from +advisory. If you cannot verify something, write "could not verify" — do not guess. Do not include +praise, quality adjectives, or an overall assessment paragraph; findings and evidence only. An empty +`Findings` list is a fine outcome if the code is sound, but only after you have genuinely tried +items 1–6. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1398.md b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1398.md new file mode 100644 index 0000000000..b5c99a05be --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1398.md @@ -0,0 +1,189 @@ +use harness + +# Slice brief — #1398 publish job executions to the durable job stream + +You are the implementation agent for this slice. **Codex · GPT-5.6 Sol · medium** +(`normal_implementation`). The plan has already passed a separate-session PLAN-EVAL; your job is to +implement it, not to re-decide it. + +| Field | Value | +| --- | --- | +| Issue | #1398 (`priority:p1`) | +| Worktree | `/home/codex/repos/ns006-1398` | +| Branch | `fix/1398-publish-job-executions-to-durable-stream` | +| Base | `origin/main@01aa12b67` | +| Run dir | `.llm/runs/release-0.0.6-features--orchestration/` | + +**Read these first, in this order:** + +1. `.llm/runs/release-0.0.6-features--orchestration/plan.md` — the approved plan. **Authoritative.** +2. `.llm/runs/release-0.0.6-features--orchestration/plan-eval.md` — the PLAN-EVAL verdict, whose + findings F1 and F2 are already folded into the plan. +3. `.llm/runs/release-0.0.6-features--orchestration/slices/research-1398.md` — line-cited research. + **Its "Unverified" section is honest; treat those items as unknown, not as facts.** + +## SKILL + +- `netscript-doctrine` — plugins are Archetype 5 / thin; convention-bearing primitives belong in + `-core`. Read before deciding where code lives. +- `netscript-harness` — slice/commit trail, drift recording. +- `netscript-cli` — the `e2e:cli` suite surface and what `scaffold.runtime` actually runs. +- `aspire` — service graph and trace evidence for the live run. +- `netscript-tools` — validation wrappers and what counts as gate evidence. +- `netscript-pr` — draft PR body, closing keyword, phase comments, labels. + +## The defect + +A triggered job execution completes and emits a `job.execute` span, but **no record for that +execution is ever published to the durable job stream**. The stream contains only the three startup +job-definition snapshots. This is a **wiring gap**, not a missing feature — the producer, the mapper, +the hook, and the execution record's trace fields all already exist. + +The workers **API service** installs the stream mutation hook +(`plugins/workers/services/src/main.ts:67`). The **background** entrypoints that generated projects +actually run never do (`plugins/workers/bin/runtime.ts:89-107`, `:110-122`, `:125-152`). Generated +projects run `startCombinedProcess()` as Aspire resource `workers-combined`. + +**The repo already admits this.** Two E2E gates are deferred against this issue with that exact +reason (`packages/cli/e2e/suites/scaffold/capability-suites.ts:24-35`). + +## LOCKED decisions — implement, do not re-decide + +- **D1** — Install the mutation hook on the background runtimes in `plugins/workers/bin/runtime.ts`, + mirroring `plugins/workers/services/src/main.ts:65-75`. Cover `startWorkerProcess` and + `startCombinedProcess`; `startSchedulerProcess` only if it owns execution state. Note + `KvExecutionState.setMutationHook` is destructive (single `onMutation` field), so a second install + replaces the first rather than double-firing — installation must still be deliberate, not + incidental. +- **D2** — Publish **all** execution mutations (created / updated / deleted), not only terminal ones. +- **D3 — the join, and the trap.** The hook must wrap its publish in the execution's stored trace + context: + `context.with(extractContext({ traceparent, tracestate }), () => producer.upsert(...))` in + `createStreamMutationHook` (`packages/plugin-workers-core/src/streams/producer.ts:108-118`), which + today does **no** wrapping. + **Why this is not optional:** `StreamsTracerPort.startSpan` + (`packages/plugin-streams-core/src/telemetry/instrumentation.ts:92-102`) takes **no** + parent-context argument and starts the publish span on the **ambient** context (`:160`), and the + header is `formatTraceparent(span.spanContext())` (`:172`). Meanwhile `job.execute` is a child of + the stored dispatch traceparent (`plugins/workers/worker/job-dispatcher.ts:43-44` → `:108` → + `packages/telemetry/src/application/span.ts:38-43`). So wrapping is what makes the published + record's trace id equal the `job.execute` trace id. + **The trap:** `executionState.create()` (`job-dispatcher.ts:74-86`) runs *before* the + `job.execute` span exists (`:91`). The E2E selector returns the **first** `correlationId` match + (`select-flow-b-stream-change.ts:96-105`) and TC-14 then asserts that record's trace id + (`:131-153`). Install the hook **without** D3's wrapping and it will look like it works, then fail + TC-14 on the `create()` record. Do not skip the wrapping because "the terminal record is inside + the span anyway". +- **D4** — **No change to `WorkerExecutionZodSchema`** and no new public export. The gate asserts the + **header** traceparent, not a record field, so no schema field is needed. If you conclude you + cannot make the join without a schema change, **stop and report** — do not take that decision + in-slice. +- **D5** — Remove both entries from `SCAFFOLD_RUNTIME_DEFERRED_GATES` + (`capability-suites.ts:24-35`) and update **both** tests that pin the deferral, in the same commit: + - `packages/cli/e2e/tests/presentation/suite-registry_test.ts:204-205` — flips `false` → `true`. + - `packages/cli/e2e/tests/presentation/suite-registry_test.ts:209-234` — *"runtime suites pin the + exact #1398 OTEL deferral without widening it"*, which asserts the constant equals the exact + two-entry list (`:210-221`) and that neither runtime tier executes a deferred gate (`:223-233`). + Both tiers reference the constant (`capability-suites.ts:211,218`). + + Missing the second test is the single most likely way to leave this slice red. It was caught by + PLAN-EVAL, not by the plan's first draft. + +## Slices + +- **S1** — D1 + D2 + D3 with unit tests. +- **S2** — D5, both tests, same commit. +- **S3** — live runtime evidence (below). + +## Required tests + +1. **The join.** The mutation hook publishes under a stored `traceparent`; assert the publish span's + trace id equals that traceparent's trace id. Today's suite + (`packages/plugin-workers-core/tests/streams/workers-streams_test.ts:9-104`) has **no traceparent + assertion** — that gap is why this shipped. +2. **Installation.** The background runtime path installs a mutation hook; the test must fail if + `startCombinedProcess` stops installing it. **No such test exists today, which is exactly why + `workers-combined` shipped without the hook.** +3. **Pre-span publication.** A mutation emitted *before* any ambient `job.execute` span still + produces a publish span on the stored trace id. This pins D3 against the `create()` case — the + trap above. Without this test, D3 can silently regress. + +Each test must fail if its own defect is reintroduced. Assert the positive value **and** that it is +not the pre-fix value where that distinction is meaningful. + +## Gates — deliverables, not hopes + +```bash +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root plugins/workers --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/plugin-workers-core --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root plugins/workers --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root plugins/workers --ext ts,tsx +deno task quality:gate +deno task --cwd packages/plugin-workers-core test +``` + +**Use the package-declared `deno task --cwd <pkg> test`, not a bare `deno test <path>`** — the bare +form omits `--allow-env` and exits 1 on `NotCapable` permission errors. That was a defect in the +previous slice brief; it is corrected here rather than repeated. + +`deno task quality:gate` is mandatory for a `packages/**`/`plugins/**` slice. Check whether the +configured quality roots actually cover the packages you touched; if they do not, run an explicit +scan against your target paths and say so. A new `deno-lint-ignore`, `as unknown as`, `any`, or +`@ts-ignore` added to green a gate is a review-blocking finding — stop and report instead. + +**S3 — live evidence, one pass:** + +```bash +deno task e2e:cli run scaffold.runtime --cleanup --format pretty +``` + +Do **not** split this into individual `gates` invocations. The two formerly-deferred gates +(`behavior.otel.stream-consumer`, `behavior.otel.traces`) must **pass for real** in that run — that +is what makes this issue's acceptance mechanical instead of asserted. Report the raw exit code and +any failing suite/test names. + +This gate is **expensive and serialised across this lane** — three concurrent `scaffold.runtime` +runs once produced two failures that were contention, not defects. Confirm no other run of it is +active before you start, and do not run it more than necessary. + +**Runtime-evidence order:** use plugin doctor and Aspire logs/OTEL traces **before** hand probes. +Prefer the trace as evidence over a hand-rolled curl. **No hand-waved timing** — the bounded-time +criterion is the gate's own live SSE loop (`consume-flow-b-stream.ts:203-230`), not a stopwatch in a +comment. + +**Known residual to observe (not to fix):** the generated Aspire wiring guards with +`if (<ref>Endpoint)` (`generate-register-background.ts:200-218`), so `services__streams__http__0` is +silently omitted if the streams resource exposes no `http` endpoint at wiring time. The env path is +otherwise confirmed wired (`install-plugin_test.ts:1393-1396`). If the producer throws +`Missing plugin reference "streams"` at `create-durable-stream.ts:262-272`, that is this residual — +report it, do not paper over it. + +## Out of scope — do not absorb + +- Any `packages/plugin-streams-core` reason-string change — that is #1405, landing separately. +- The undeclared `@netscript/plugin-streams-core` imports in `packages/plugin-workers-core/deno.json` + and `plugins/triggers/deno.json`. Real, but they get their own issue. If `publish:dry-run` flags + them, **report; do not fix here.** +- Any `WorkerExecutionZodSchema` change (D4). + +## Commit trail + +1. Open a **draft PR against `main`** in the same session as your first commit. Title: + `fix(workers): publish job executions to the durable stream on the job.execute trace`. + Body per `netscript-pr`: `Closes #1398` in `## Scope`, run-dir path, slice checklist, Definition + of Done, and a fenced `acceptance-evidence` block mapping each of #1398's four acceptance boxes. + Labels `type:fix`, `area:plugins`, `area:telemetry`, `status:impl`, milestone `0.0.6`. +2. Commit per slice, push by **explicit refspec** + (`git push origin HEAD:refs/heads/fix/1398-publish-job-executions-to-durable-stream`), and post a + `[PHASE: IMPL]` comment with commit hash and **pasted real gate output**. +3. Keep your slice worklog current in the same commit. + +## Reporting contract + +Report: what you changed and where; the exact test names and what each would catch; verbatim gate +output including the E2E exit code; and **anything you could not do, could not verify, or that +surprised you**. If a gate goes red, report the red with its output — do not go idle on it and do +not work around it silently. The previous slice in this lane reported a red caused by a mistake in +its own brief, and that was the correct behaviour. + +You do **not** merge and you do **not** flip the PR to ready. The orchestrator holds merge authority. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1405.md b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1405.md new file mode 100644 index 0000000000..cc5e2cface --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1405.md @@ -0,0 +1,150 @@ +use harness + +# Slice brief — #1405 durable producer rejection taxonomy + +You are the implementation agent for one small, fully specified slice. **Codex · GPT-5.6 Sol · +low** (`light_implementation`). Do exactly this slice: do not refactor neighbours, do not rename +anything not named here, do not touch #1398's surface. + +| Field | Value | +| --- | --- | +| Issue | #1405 | +| Worktree | `/home/codex/repos/ns006-1405` | +| Branch | `fix/1405-durable-producer-rejection-taxonomy` | +| Base | `origin/main@01aa12b67` | +| Orchestrator run dir | `.llm/runs/release-0.0.6-features--orchestration/` | +| Research (read it first) | `.llm/runs/release-0.0.6-features--orchestration/slices/research-1405.md` | + +## SKILL + +- `netscript-doctrine` — `plugin-streams-core` is a framework package; its `domain/` contracts are + published surface. Read before editing `producer-contract-v1.ts`. +- `netscript-harness` — slice/commit trail discipline, drift recording. +- `netscript-tools` — the scoped validation wrappers and what counts as gate evidence. +- `netscript-pr` — draft PR body, closing keyword, phase comments, labels. + +## The defect, exactly + +Two settled **reason strings** misdescribe the state that produced them. Nothing else is wrong: no +write is silently lost, no false `delivered` is possible, and every drop path is already metered. + +**D1 — close-drain window reports `producer-failed`.** +`packages/plugin-streams-core/src/application/durable-stream-producer-supervisor.ts:215` sets +`#accepted = false`, then `:216` calls `#closeGracefully()`, which `await`s `flush()` and the +connect promise before reaching `this.#transition('stopping', 0)` at `:225`. Throughout that drain +`#accepted` is false while `#state.state` is still `ready`, so both rejection selectors fall to +their `default` arm and answer `producer-failed`: + +- supervisor `#writeRejectionReason()` — `:467-478` +- façade `stateRejection()` — `create-durable-stream.ts:251-262`, reached from `upsert` (`:131-133`) + and the second write entry point (`:159-160`) because `get closed()` is `!this.#accepted` + (supervisor `:109-111`) + +`#fail()` was never called and `#state.error` is unset. The producer is healthy and closing. + +**D2 — first-attempt refusal reports `retry-exhausted`.** `#failActive()` (`:412-423`) settles every +non-`aborted` failure as `retry-exhausted` (`:418-420`), across three call sites: + +| Call site | Guard | True cause | +| --- | --- | --- | +| `:302` | `!isRetryable(connected.failure) \|\| attempt === maxAttempts` | **conflated** | +| `:346` | `else if (!isRetryable(result.failure))` | **refusal** — reachable on attempt 1 | +| `:350` | `if (attempt === this.#reconnectPolicy.maxAttempts)` | genuine exhaustion | + +## LOCKED decisions — implement these, do not re-decide + +1. **D1 reuses the existing `producer-stopping` reason. Do NOT add a `producer-closing` member.** + `StreamWriteRejectionReasonV1` (`producer-contract-v1.ts:70-78`) already contains + `producer-stopping`, and it already means "shutting down, not accepting". The bug is that the + selectors cannot observe the closing intent before the `stopping` transition — fix the + *observability*, not the vocabulary. Make the closing intent visible from the moment `close()` + is entered (e.g. a private closing flag set alongside `#accepted = false` at `:215`, consulted by + `#writeRejectionReason()`), and make the façade's `stateRejection()` agree — the façade must not + 独立ly re-derive `producer-failed` from a stale state snapshot. Preferred shape: have the façade + ask the supervisor for the reason rather than duplicating the state switch, so the two can never + drift apart again. `#stopImmediately()` (`:185-186`) has no such window and must not change. + +2. **D2 adds exactly one new member to `StreamWriteUnknownReasonV1`** (`:84-87`), named + **`transport-refused`**. This is the only public-surface addition in the slice. Semantics: the + transport returned a **non-retryable** failure — the server positively refused — regardless of + attempt number. Then: + - `#failActive()` selects: `failure.kind === 'aborted'` → `transport-aborted` (unchanged); + else `!isRetryable(failure)` → `transport-refused`; else → `retry-exhausted`. + - Call site `:302` must stop conflating its two guards, so a non-retryable connect failure on the + final attempt is still reported as a refusal, not as exhaustion. + - `retry-exhausted` must remain reachable and must still be what `:350` produces. + +3. **No behaviour change beyond reason strings.** Which writes are accepted, rejected, cancelled, or + delivered is identical before and after. `#fail()` is still called from exactly the paths it is + called from today. If you find yourself changing an acceptance decision, stop and report it. + +4. **Update the doc comments and the exported surface consistently.** `transport-refused` needs its + JSDoc line in the union, and `packages/plugin-streams-core/mod.ts:44-45` already re-exports both + unions — verify the export still resolves; do not add new exports. + +## Required tests — these are the deliverable, not an afterthought + +Acceptance box 4 says a future refactor that collapses these reasons must **fail**. A test that only +asserts the new string passes trivially if someone later reintroduces the old one on the other path, +so each test must assert the correct reason **and** explicitly assert it is not the old wrong one. + +1. A write issued after `close()` is entered but **before** the drain completes settles + `rejected` / `producer-stopping`, asserted **not** `producer-failed`. You must actually hold the + drain open (a transport whose `close`/`flush` does not resolve until the test releases it) — + a test that races is not evidence. +2. Append transport returns a **non-retryable** failure on **attempt 1** with `maxAttempts > 1`: + settles `delivery-unknown` / `transport-refused`, asserted **not** `retry-exhausted`, and assert + the transport was called exactly **once** (proving no retries were exhausted). +3. Append transport returns **retryable** failures until `maxAttempts`: still settles + `delivery-unknown` / `retry-exhausted`. +4. A non-retryable **connect** failure settles `transport-refused` (covers the `:302` conflation). +5. Existing behaviour guard stays green: + `packages/plugin-streams-core/tests/telemetry/durable-stream-producer-telemetry_test.ts:172-219` + already distinguishes rejected / dropped / delivery-unknown. Do not edit it to fit your change; + if it goes red, your change altered classification and that is a stop-and-report. + +## Gates — turn these green; they are part of the deliverable, not a checklist to hope about + +This slice touches `packages/**`, so the scoped wrappers alone are **not** a verdict. + +```bash +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/plugin-streams-core --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root packages/plugin-streams-core --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/plugin-streams-core --ext ts,tsx +deno task quality:gate +deno task doc:lint --root packages/plugin-streams-core --pretty +deno test packages/plugin-streams-core +``` + +`deno task quality:gate` (= `quality:scan` + `arch:check`) is **mandatory** for a `packages/**` +slice. A new `// deno-lint-ignore`, `as unknown as`, `any`, or `@ts-ignore` introduced to green a +wrapper is a review-blocking finding, not a pass — if you think you need one, stop and report. + +Do **not** run `deno task e2e:cli` — this slice does not touch scaffold output, and that gate is +expensive and serialised across the lane. + +**Known hazard, pre-empted:** `deno fmt` rewraps long lines and can silently undo a scripted string +edit. After the format wrapper runs, re-grep for every string you introduced (`transport-refused`, +`producer-stopping`) and confirm it is still present in the file you put it in. + +## Commit trail + +1. Open a **draft PR against `main`** in the same session as your first commit. Title: + `fix(streams): name the closing and refusal states in producer write reasons`. + Body per `netscript-pr` — it MUST carry `Closes #1405` in `## Scope`, the run dir path, the slice + checklist, a Definition of Done, and a fenced `acceptance-evidence` block mapping each of the + five acceptance boxes of #1405 to its evidence. Labels: `type:fix`, `area:plugins`, + `status:impl`, milestone `0.0.6`. +2. Commit by slice, push, and post an `[PHASE: IMPL]` comment with the commit hash and **pasted real + gate output** (exit codes, test counts). No green box without evidence. +3. Keep `worklog.md` in your slice dir current as part of the same commit. + +## Reporting contract + +When done, report: the reason strings as implemented, the exact test names and what each would +catch, verbatim gate output, and **anything you could not do or that surprised you**. If you hit a +red gate, do not go idle — report the red with its output. A red gate reported is useful; a red gate +sat on is the failure mode this brief exists to prevent. + +You do **not** merge, and you do **not** flip the PR to ready. The orchestrator holds merge +authority and runs the pre-merge gate. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1457.md b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1457.md new file mode 100644 index 0000000000..b51feb312e --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1457.md @@ -0,0 +1,23 @@ +use harness + +# Slice implementation — #1457 chat proxy query forwarding + +- Implementation lane: Codex · GPT-5.6 Sol · low (`light_implementation`) +- Worktree: `/home/codex/repos/ns006-1457` +- Branch: `fix/1457-chat-proxy-query-forwarding` +- Baseline: `origin/main@f99cb4fbf` +- Surface: `packages/fresh/src/runtime/ai/stream-proxy.ts` and its focused tests only +- Excluded: `application/defer/**`, `stream-url-resolver.ts`, headers, auth, body, streaming, + abort, and response sanitization + +## Harness selection + +- Archetype 2 (Integration): the published Fresh handler is an HTTP proxy edge with an injected + `fetch` seam. +- PLAN-EVAL: N/A. The owner locked D1–D4, including collision direction, repeat handling, and the + hook contract; no implementation-changing decision remains open. +- JSR surface risk: additive optional callback only, with explicit parameter and return types plus + public JSDoc. No export entrypoint or dependency changes. +- Slice: one commit proving default forwarding, `id` exclusion, authoritative resolved-query + collision handling, repeated keys, mapping-hook replacement, and no-query stability. + diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1548.md b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1548.md new file mode 100644 index 0000000000..ee3bbd3913 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1548.md @@ -0,0 +1,108 @@ +use harness + +# Slice brief — #1548 browser stream resolver cannot see Aspire VITE service references + +**Codex · GPT-5.6 Sol · medium** (`normal_implementation`). The plan has **passed PLAN-EVAL with +conditions**; implement it, do not re-decide it. + +| Field | Value | +| --- | --- | +| Issue | #1548 (`priority:p1`) | +| PR | **#1559** (already open, draft, at plan phase — commit onto its branch) | +| Worktree | `/home/codex/repos/ns006-1548` | +| Branch | `fix/1548-vite-browser-stream-discovery` | +| Run dir | `.llm/runs/release-0.0.6-features--orchestration/` | + +**Read first:** `slices/plan-1548.md` — **including the "Amendments after PLAN-EVAL" section at the +bottom, which is binding** — and `slices/research-1548.md`. If they are not on your branch, read them +with `git show chore/release-0.0.6-runtime-reopen:<path>`. + +## SKILL + +- `netscript-doctrine` — `packages/plugin-streams-core` is framework code; `mod.ts` is published + surface. +- `deno-fresh` — Fresh/Vite build and `import.meta.env` substitution. +- `netscript-tools`, `netscript-pr`, `netscript-harness`. + +## The defect + +`packages/plugin-streams-core/src/application/stream-url-resolver.ts` cannot see +`VITE_services__streams__http__0` in a browser bundle, for **two** independent reasons: + +1. `import.meta` is passed **as a value** into `readImportMetaEnvironment` (`:60`), reached through a + parameter binding (`:77-78`) — the literal `import.meta.env.VITE_…` never appears in source. +2. Both keys are read by **computed index** (`:64,:68`). + +This repo's own Vite plugin substitutes by emitting `define` entries keyed exactly as +`` `import.meta.env.${target}` `` (`packages/fresh/src/application/vite/vite.ts:195,311-335`) — a +textual static-expression mechanism that by construction cannot reach either shape. + +## LOCKED decisions + +- **D1** — read the two browser keys as **literal static member expressions** inline: + `import.meta.env.VITE_services__streams__http__0` and `import.meta.env.VITE_STREAMS_URL`. No value + passing of `import.meta`, no computed index for these two. Safe because the keys are fully known at + compile time (`STREAMS_RESOURCE_NAME`, `domain/constants.ts:5`). +- **D2** — split the **pure, injectable** lookup from the impure reader. **A1 (binding):** the SDK + (`packages/sdk/src/discovery/browser-env.ts`) is a **structural** precedent only — it carries the + *same* substitutability defect. Borrow its shape; do **not** describe it, in code comments or the + PR, as having already solved this. +- **D3** — **no published-surface growth**: the pure function stays internal, not added to `mod.ts` + (which exports only `buildStreamUrl`, `getStreamsAuth`, `getStreamsUrl`). Tests import by src path. +- **D4** — precedence unchanged: `DURABLE_STREAMS_URL` → `services__streams__http__0` → browser + `VITE_*` (full key before shorthand) → throw. Change **how** the browser branch reads, not what wins. +- **D5** — **no** Vite `transform` hook and **no** scaffold-template change in this slice. +- **D6** — you may narrow the silent `catch` (`:59,69-71`) so a genuine throw is distinguishable from + "absent", but it must not change which value wins. Drop it if it grows. +- **A3** — `plugin-streams-core/deno.json` depends only on `@netscript/telemetry` and `@std/assert`; + there is no dependency on `packages/fresh`, so no cycle is introduced and the + `createNetScriptStreamDB` call site needs no change. + +## Required tests + +1. **Precedence on the pure lookup**, injected bag: full key wins over shorthand; shorthand used when + the full key is absent; neither → `undefined`. Mirrors + `packages/sdk/tests/discovery/env-ordering_test.ts:24,41,56`. +2. **Source-shape guard** — assert the module contains the literal substitutable expressions and that + the browser path does **not** pass `import.meta` across a function boundary or index `env` by a + computed key. This is deliberately a shape assertion: **the defect is the shape**, and without this + guard a refactor back to a helper reintroduces the bug silently with all behavioural tests green. +3. **Resolution through `getStreamsUrl()`** proving the browser branch is reachable and correctly + ordered. + +There are currently **no unit tests at all** for this file's URL resolution — the package's nine test +files never reference `getStreamsUrl`. You are adding the first. + +**Do not claim any test proves Vite substitution.** It does not; test 2 is an explicit surrogate. +Say so in the PR. + +## Gates + +```bash +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/plugin-streams-core --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root packages/plugin-streams-core --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/plugin-streams-core --ext ts,tsx +deno task --cwd packages/plugin-streams-core test +deno task doc:lint --root packages/plugin-streams-core --pretty +``` + +**A2 (binding):** `arch:check` at `deno.json:156` **confirmed does not include** +`packages/plugin-streams-core`. So run `deno task quality:gate` **and** an **explicit target scan over +`packages/plugin-streams-core/src`**, and state in the PR that the package-quality verdict rests on +the explicit scan, not the repo gate. Do not report a green repo gate as proof for this package. + +Use `deno task --cwd <pkg> test`, never a bare `deno test <path>` (omits `--allow-env`, exits 1 on +`NotCapable`). No `e2e:cli` — this slice does not touch scaffold output. + +## Commit trail + +PR **#1559 already exists** as a draft at plan phase. Commit onto its branch, push by explicit +refspec, and post a `[PHASE: IMPL]` comment with commit hash and pasted real gate output. Update the +PR body's Definition of Done to match what shipped. `Closes #1548` is already in its Scope. Move the +label from `status:plan-eval` to `status:impl` when you push. + +## Reporting contract + +Report what changed, the exact test names and what each catches, verbatim gate output, and anything +you could not verify. **Do not flip the PR to ready** — that fires the automatic IMPL-EVAL and is the +orchestrator's trigger to pull. Merge authority is the orchestrator's. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/plan-1548.md b/.llm/runs/release-0.0.6-features--orchestration/slices/plan-1548.md new file mode 100644 index 0000000000..7459c16f9b --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/plan-1548.md @@ -0,0 +1,143 @@ +# Plan — #1548 browser stream resolver cannot see Aspire VITE service references + +Lane: 0.0.6 runtime reopen. Control PR #1555. Research: `slices/research-1548.md`. +Branch: `fix/1548-vite-browser-stream-discovery`. **Goes to PLAN-EVAL before implementation** via the +`openhands` + `status:plan-eval` label pair (automatic policy, D-4/D-5). + +## Contract + +`createNetScriptStreamDB({ streamPath, schema })` with **no `baseUrl`** must resolve +`VITE_services__streams__http__0` (or `VITE_STREAMS_URL`) in a Fresh/Vite browser bundle and connect, +instead of throwing "Durable streams URL not found…". The consumer-side +`baseUrl: import.meta.env.VITE_… ?? …` workaround must become unnecessary. + +## Root cause (verified from repo code, not from claims about Vite) + +Two independent reasons the value cannot be statically substituted — +`packages/plugin-streams-core/src/application/stream-url-resolver.ts`: + +1. `import.meta` is passed **as a value** into `readImportMetaEnvironment` (`:60`), which reaches + `meta.env` through a parameter binding (`:77-78`). The literal expression + `import.meta.env.VITE_services__streams__http__0` never appears in source. +2. Even given a bag, both keys are read by **computed index** (`:64,:68`). + +**The decisive corroboration is internal, which is why this does not rest on an assertion about +Vite.** This repo's own Vite plugin injects env by emitting `define` entries keyed exactly as +`` `import.meta.env.${mapping.target}` `` (`packages/fresh/src/application/vite/vite.ts:195,311-335`). +That is textual static-expression substitution by construction — it cannot reach a value-passed +`import.meta` or a computed index. The failure is fully explained by this repo's own machinery. + +Aggravating factor: the browser branch is wrapped in a silent +`try { … } catch { return undefined }` (`:59,69-71`), so the diagnostic surfaces as a generic +"not found" rather than anything pointing at env reading. + +## Locked decisions + +- **D1 — read the two browser keys as literal static member expressions**, inline in the module: + `import.meta.env.VITE_services__streams__http__0` and `import.meta.env.VITE_STREAMS_URL`. No value + passing of `import.meta`, no computed index for these two keys. This is safe because the key + strings are **fully known at compile time** — `STREAMS_RESOURCE_NAME` is a constant + (`packages/plugin-streams-core/src/domain/constants.ts:5`). The generic + `serviceName`/`protocol`/`index` parameterisation is what made the keys computed; for the browser + path it buys nothing and costs substitutability. +- **D2 — split the pure lookup from the `import.meta` access**, mirroring the SDK precedent + `packages/sdk/src/discovery/browser-env.ts:35-54,65`, whose pure + `getBrowserServiceUrlFromEnv(env, …)` is unit-tested at + `packages/sdk/tests/discovery/env-ordering_test.ts:24,41,56`. The pure function takes an injected + env bag; only a thin inline shim touches `import.meta.env.*`. +- **D3 — no published-surface growth.** The pure function stays **internal**: not added to + `packages/plugin-streams-core/mod.ts`, which exports only `buildStreamUrl`, `getStreamsAuth`, + `getStreamsUrl` (`mod.ts:19`). Tests import it by src path, as this repo's tests already do. +- **D4 — resolution precedence is unchanged**: `DURABLE_STREAMS_URL` → `services__streams__http__0` + → browser `VITE_*` (full key before shorthand) → throw. This slice changes **how** the browser + branch reads, not **what** wins. +- **D5 — no Vite `transform` hook, and no scaffold-template change in this slice.** The transform + option would couple the framework to a bundler's module graph; the `envMappings` route + (`vite.ts:76-83,183-199`) only helps once the read is substitutable, so it is a possible follow-up, + not part of the fix. Adding it here would widen a p1 into CLI-template territory. +- **D6 — the silent `catch` (`:59,69-71`) may be narrowed so a genuine throw is distinguishable from + "absent", but must not change which value wins.** Explicitly bounded so it does not become a + diagnostics refactor. + +## The testability trap this plan has to solve + +A unit test cannot easily prove "Vite substituted the expression" without running a real Vite build. +So the tests must attack the defect from both sides: + +1. **Precedence tests on the pure lookup (D2)** with an injected bag — full key wins over shorthand; + shorthand used when the full key is absent; neither present → `undefined`. Mirrors + `env-ordering_test.ts`. +2. **A source-shape guard** asserting the module contains the **literal** substitutable expressions + and that the browser path does **not** pass `import.meta` across a function boundary or index + `env` by a computed key. This is deliberately a shape assertion: the defect *is* the shape, and + without this guard a future refactor back to a helper reintroduces the bug **silently and with all + behavioural tests still green** — exactly the regression class #1405's reason tests were built to + catch. +3. **A resolution test through `getStreamsUrl()`** proving the browser branch is reachable and + ordered correctly. + +The plan does **not** claim a unit test proves Vite substitution. If PLAN-EVAL judges the source-shape +guard insufficient, the fallback is a built-fixture test, which is materially more expensive and +should be argued for explicitly rather than assumed. + +## Slices + +- **S1** — D1 + D2 + D3 + D4 with tests 1–3. +- **S2** — D6 only if it stays trivial; otherwise dropped and recorded. + +## Gates + +Scoped check/lint/fmt over `packages/plugin-streams-core`, `deno task quality:gate` (**verify it +covers this package — it demonstrably omits several, tracked as #1542; if not covered, run an +explicit target scan and say so**), `deno task --cwd packages/plugin-streams-core test`, +`deno task doc:lint`. No `e2e:cli` — this slice does not touch scaffold output. + +## Risks + +- **R1** — the fix is only provable end-to-end in a real Vite build; the source-shape guard is a + proxy. Stated openly above rather than papered over. +- **R2** — `getStreamsAuth()` is Deno-only (`:138-141`) and returns `{}` in a browser. Out of scope + here, but if the browser path needs auth this fix alone will not deliver a working connection. + **Worth PLAN-EVAL's attention:** it may mean the issue's "connect directly to Streams" acceptance is + not fully satisfiable by URL resolution alone. +- **R3** — the AppHost actually injecting both VITE variables was **not verified** by research. + +## Acceptance mapping (#1548) + +| Expectation | Satisfied by | Evidence | +| --- | --- | --- | +| Zero-config browser factory resolves Aspire's canonical Vite reference | D1 + D2 + D4 | precedence tests + source-shape guard + resolution test | +| Consumer `baseUrl` workaround becomes unnecessary | D1 | the documented override remains supported, no longer required | + +--- + +## Amendments after PLAN-EVAL (verdict PASS with conditions) + +**A1 — the SDK is a *structural* precedent, not a *fix* precedent. Do not over-claim it.** +PLAN-EVAL verified that `packages/sdk/src/discovery/browser-env.ts:65` has the **same +Vite-substitutability defect** — it reads `import.meta.env` via a cast inside try/catch, so it is +*also* not a literal static member expression. What this plan borrows from the SDK is **only** the +separation of a pure, injectable lookup from the impure reader, and the fact that the pure half is +unit-tested. It must not be described anywhere as "the SDK already solved this". Both the research +note and D2 are corrected by this amendment. + +Consequence worth recording: **the SDK likely has the same latent browser-discovery bug.** That is +out of scope here and is not being fixed blind, but it should be filed once this fix proves the +shape works. + +**A2 — the explicit target scan is a hard gate step, not a conditional.** PLAN-EVAL confirmed +`deno.json:156` (`arch:check`) does **not** include `packages/plugin-streams-core`. So "verify +whether it is covered" is settled: it is **not**. The implementation slice's gate list must contain +an explicit target scan over `packages/plugin-streams-core/src`, and the PR must state that the +package-quality verdict rests on that scan rather than on the repo gate. Same caveat this lane +already carried on #1528 and #1536, now confirmed for this package too. + +**A3 — no circular-dependency or size risk.** PLAN-EVAL verified `plugin-streams-core/deno.json` +depends only on `@netscript/telemetry` and `@std/assert` — no dependency on `packages/fresh`, so +fixing this in `plugin-streams-core` introduces no cycle, and the `createNetScriptStreamDB` call site +in `packages/fresh` needs no change. It also verified the planned ~120-150 LOC test file sits well +under the 409-LOC ceiling of the largest existing test in that package. + +**Unchanged:** R1 stands as disclosed — no unit test proves Vite's `define` substitution fires in a +real build; the source-shape guard is an explicit surrogate, and PLAN-EVAL accepted it as such rather +than requiring a built fixture. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1525.md b/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1525.md new file mode 100644 index 0000000000..0ad99d4bcf --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1525.md @@ -0,0 +1,33 @@ +# Pre-merge gate — PR #1525 (control run, closes nothing) + +Run per `.llm/harness/workflow/milestone-run.md`, at head `bdc62b0c1`, 2026-08-12. This PR carries +**orchestration evidence only**; it never became an implementation umbrella. + +| # | Check | Result | Evidence | +| --- | --- | --- | --- | +| 1 | `close-gate` green | **PASS** | `close-gate` → `pass`. A result exists. | +| 2 | Zero unticked boxes on issues the PR closes | **PASS, vacuously — and deliberately so** | The body contains **0** closing keywords, verified by grep. An evidence-only control PR must close nothing; the two owned issues were closed by their own implementation PRs (#1528, #1536). | +| 3 | No new `deno-lint-ignore` / `as unknown as` / `@ts-ignore`, excluding `.llm/runs/**` | **PASS** | Diff outside `.llm/runs/**` is empty; scanned and clean. | +| 4 | Named expensive gates `SUCCESS` | **PASS** | `scaffold-runtime (aspire + docker + postgres)` · `scaffold-runtime-sqlite` · `scaffold-static` · `quality` · `check-test` · `surface-diff` · `deps-report` · `close-gate` — all `pass`. | +| 5 | The single decisive claim, re-verified | **PASS** | Claim: *the lane's recorded evidence matches what actually happened*. Both merges in `cut-trace.md` were captured from `git log origin/main --first-parent` **after** each merge, and both issue states re-read live (`CLOSED`/`COMPLETED`). | +| 6 | Changed-file audit for `packages/**`/`plugins/**` | **PASS — audited, not assumed** | This is exactly the lane where check 6 earns its place (#1079: a docs slice landed framework source). Explicit grep for `^(packages\|plugins)/` over the full diff → **no matches**. All 20 changed paths are under `.llm/runs/`. | +| 7 | PR body checklist matches what shipped | **PASS** | All `## Slices` and `## Definition of Done` boxes ticked, each asserted true at edit time by a script that refuses to leave an unticked box under either heading. | + +`agentic:review-threads` → recorded below. + +## Artifact completeness + +Mandatory set present: `supervisor.md`, `plan.md`, `worklog.md`, `context-pack.md`, `drift.md`, plus +the profile's signature `cut-trace.md` and this lane's `retrospective.md`. Both formal verdicts are +captured as artifacts (`plan-eval.md`, `slices/evaluate-1405.md`, `slices/evaluate-1398.md`), not +left only as PR comments. + +Run dir is **208 K**. The two raw evaluator JSONL streams (2.4 MB combined) were untracked to +gitignored `.llm/tmp/` scratch — they were 2.4 MB of a 2.5 MB run dir against a 96 K +largest-artifact precedent in the 0.0.5 run, which tracks no raw streams at all. Nothing was deleted; +the files remain on disk and their substance is verbatim in the distilled verdicts with run id, +duration, event count, and `is_error`. + +## Verdict + +**Cleared to merge.** Evidence-only, closes nothing, no framework source, all seven checks pass. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1528.md b/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1528.md new file mode 100644 index 0000000000..c0d5647b04 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1528.md @@ -0,0 +1,37 @@ +# Pre-merge gate — PR #1528 (closes #1405) + +Run per `.llm/harness/workflow/milestone-run.md` § The pre-merge gate, by the orchestrator holding +merge authority. Evaluated at head `c491c6989`, 2026-08-12. + +| # | Check | Result | Evidence | +| --- | --- | --- | --- | +| 1 | `close-gate` result is green | **PASS** | `close-gate` → `SUCCESS`. A result exists — not absent, not skipped. | +| 2 | Zero unticked `- [ ]` on every issue the PR closes | **PASS** | #1405 body fetched live: all **5** acceptance boxes `- [x]`, ticked by the evidence mirror on `status:ready-merge` from the PR's fenced `acceptance-evidence` block, not by hand. | +| 3 | No new `deno-lint-ignore` / `as unknown as` / `@ts-ignore` in the diff, excluding `.llm/runs/**` | **PASS** | `git diff origin/main...FETCH_HEAD -- . ':(exclude).llm/runs/**'` grepped for all four patterns on added lines → no matches. The diff **was** scanned; this is a verdict, not an absence. | +| 4 | Named expensive gates report `SUCCESS`, not `SKIPPED`/`CANCELLED` | **PASS** | Named individually rather than counted: `scaffold-runtime (aspire + docker + postgres)` SUCCESS · `scaffold-runtime-sqlite (aspire + sqlite + garnet)` SUCCESS · `scaffold-static (deno-only)` SUCCESS · `code-quality` SUCCESS · `quality` SUCCESS · `check-test` SUCCESS · `surface-diff` SUCCESS · `deps-report` SUCCESS · `close-gate` SUCCESS. | +| 5 | The single decisive claim per issue, re-verified independently | **PASS** | Claim: *the reasons are now accurate and each is independently pinned*. Verified twice from different directions — orchestrator reverted **both** fixes → 5 failures; IMPL-EVAL reverted **each** fix alone → close-drain test fails only under fix-A revert, refusal tests only under fix-B revert. Aggregate redness would not have proven this. | +| 6 | Changed-file audit for `packages/**`/`plugins/**` on docs-lane PRs | **N/A, and audited anyway** | Not a docs-lane PR. Files: 4 × `packages/plugin-streams-core/**` + 4 × this run's own `slices/1405/**` artifacts. No unrelated package or plugin touched; no #1398 surface. | +| 7 | The PR body's own checklist matches what shipped | **PASS** | All 5 DoD boxes ticked and each is true of the diff: reasons-and-observability only; four negative tests present; evidence recorded including the red the brief's own bad command produced. | + +**Additional gate run (repo requirement, not in the seven):** `agentic:review-threads` → +`PASS threads=0 unanswered=0`, exit 0. No silent review finding is being merged past. + +## Did-not-run discipline + +Every row above states a positive result from a named source. The checks that legitimately did not +run are recorded rather than ignored: `agent` (×2), `code-quality-repo`, and `Minimax M3 docs +accuracy` reported `SKIPPED`. None is in the named-gate set for a `packages/**` code change — +`code-quality-repo` is the repo-wide audit and the Minimax job is the docs-accuracy lane — and +`code-quality` itself ran and passed. + +**Recorded caveat carried from IMPL-EVAL:** `deno task quality:gate`'s configured roots omit +`packages/plugin-streams-core`, so the package-quality verdict rests on the **explicit target scan** +(`findings=[]`, `allowCount=0`) rather than on the repo gate. The CI `code-quality` job passing is +therefore not by itself proof of this package's quality; the target scan is. This is a repo +gate-coverage gap, not a defect in this change, and it is stated here so the merge record is not +read as more than it is. + +## Verdict + +**Cleared to merge.** All seven checks pass with evidence; IMPL-EVAL PASS from a separate session; +no unanswered review threads. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1536.md b/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1536.md new file mode 100644 index 0000000000..2837173972 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/pre-merge-gate-1536.md @@ -0,0 +1,50 @@ +# Pre-merge gate — PR #1536 (closes #1398) + +Run per `.llm/harness/workflow/milestone-run.md` § The pre-merge gate, by the orchestrator holding +merge authority. Evaluated at head **`f7d503fee`**, 2026-08-12. + +**Head discipline.** This PR's head changed mid-flight (`e4319c685` → `f7d503fee`) when the branch +was synced with `main` so the phase-eval dispatcher would exist in the merge ref. **No evidence from +before that sync is cited below.** Every gate result and job id here was re-read against +`f7d503fee`; the pre-sync job ids (`94062070840`, `94062070984`) describe a head that is no longer on +this PR and appear nowhere in the PR body or this record. + +| # | Check | Result | Evidence | +| --- | --- | --- | --- | +| 1 | `close-gate` result is green | **PASS** | `close-gate` → `pass` on the re-run after `status:ready-merge`. The earlier red was a genuine red against the then-current body, not a flake — it listed all four unticked issue boxes and all three unticked DoD boxes by line. | +| 2 | Zero unticked `- [ ]` on every issue the PR closes | **PASS** | #1398 fetched live: all **4** acceptance boxes `- [x]`, ticked by the evidence mirror from the PR's `box-index` entries, not by hand. | +| 3 | No new `deno-lint-ignore` / `as unknown as` / `@ts-ignore` in the diff, excluding `.llm/runs/**` | **PASS** | `git diff origin/main...FETCH_HEAD -- . ':(exclude).llm/runs/**'` grepped for all four patterns on added lines → no matches. Diff was scanned; this is a verdict, not an absence. | +| 4 | Named expensive gates report `SUCCESS`, not `SKIPPED`/`CANCELLED` | **PASS** | Named individually, not counted: `scaffold-runtime (aspire + docker + postgres)` · `scaffold-runtime-sqlite (aspire + sqlite + garnet)` · `scaffold-static (deno-only)` · `code-quality` · `quality` · `check-test` · `surface-diff` · `deps-report` · `close-gate` — all `pass` at `f7d503fee`. | +| 5 | The single decisive claim per issue, re-verified independently | **PASS** | Claim: *job executions now reach the durable stream and join the `job.execute` trace*. Both formerly-deferred gates re-read **by name** from this head's logs: postgres job `94073971396` (`passed=88 failed=0 skipped=0`) and sqlite job `94073971501` (`passed=83 failed=0 skipped=0`). The Qwen IMPL-EVAL independently traced the join mechanism end-to-end (hook → `producer.upsert` → `#startPublish` → `instrumentation.startPublish` on ambient context). | +| 6 | Changed-file audit for `packages/**`/`plugins/**` on docs-lane PRs | **N/A, audited anyway** | Not a docs-lane PR. 7 non-run-artifact files, all in `packages/cli/e2e`, `packages/plugin-workers-core`, `plugins/workers`. No `streams/schema.ts` change (D4 honoured), no #1405 surface, no dependency or export-map change. | +| 7 | The PR body's own checklist matches what shipped | **PASS** | All 7 DoD boxes ticked and each true of this head. The two previously-false claims were **replaced, not ticked over**: the E2E box no longer says "blocked before those gates by `runtime.flow-b-fixture` fetch failure", and the acceptance evidence no longer says "Not yet live-verified". | + +**Additional gate:** `agentic:review-threads` → `PASS threads=0 unanswered=0`, exit 0. + +## IMPL-EVAL + +`OPENHANDS_VERDICT: PASS` — automatic phase dispatcher, `openrouter/qwen/qwen3.8-max`, run +`31584188459`, evaluated at head `f7d503fee` against trusted base `281ab76887`. Triggered by the +label pair alone per D-5; **no manual OpenHands dispatch and no local evaluator** was used for this +PR. + +Its three advisories are all pre-existing and tracked, none blocking: + +- `close-gate` red pending `status:ready-merge` — now resolved, above. +- `quality:gate` roots omit `packages/plugin-workers-core` / `packages/cli/e2e` — filed as **#1542**. +- Undeclared `@netscript/plugin-streams-core` imports — filed as **#1543**. + +## Did-not-run discipline + +`agent` ×2 and `code-quality-repo` report `skipping`. None is in the named-gate set for this change +(`code-quality-repo` is the repo-wide audit), and `code-quality` itself ran and passed. + +**Caveat carried into the merge record:** as with #1528, `quality:gate`'s configured roots do not +cover the packages this change touches, so the package-quality verdict rests on the **explicit target +scans** the slice worklog records (`findings=[]`, `allowCount=0`) rather than on the repo gate. CI +`code-quality` passing is not by itself proof for these packages. That gap is #1542. + +## Verdict + +**Cleared to merge.** Seven checks pass with named evidence at `f7d503fee`; IMPL-EVAL PASS from a +separate automatic session; no unanswered review threads. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/research-1398.md b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1398.md new file mode 100644 index 0000000000..10fa0cfb9c --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1398.md @@ -0,0 +1,175 @@ +# Research — #1398 job executions never reach the durable job stream + +Delegated read-only sub-agent (Claude · Opus 5, `drift.md` D-1). Baseline `origin/main@01aa12b67`. +**Terminated early on budget** after a narrow discovery pass; the unverified list below is part of +the finding, not a gap to paper over. Every claim carries a `path:line` citation from a file the +sub-agent actually read. + +## Root cause — already recorded in-repo, and it is a wiring gap, not a missing feature + +The stream mutation hook is installed by the workers **API service** and by nothing else: + +- `plugins/workers/services/src/main.ts:67` — + `runtime.executionState.setMutationHook(createStreamMutationHook())`, inside a `queueMicrotask` + after `serve()`. +- The **background** entrypoints never call `setMutationHook`: + `plugins/workers/bin/runtime.ts:89-107` (`startWorkerProcess`), `:110-122` + (`startSchedulerProcess`), `:125-152` (`startCombinedProcess`). Each builds its own runtime via + `createWorkersServiceRuntime()` (`plugins/workers/services/src/service-runtime.ts:80-89`, with + `KvExecutionState` at `:84`). +- Generated projects run `startCombinedProcess()` + (`plugins/workers/src/adapter/resources/glue/runtime.stub.ts:21-24`) as the Aspire resource + `workers-combined` (`plugins/workers/src/aspire/workers-contribution.ts:12,56-61`). + +So job **executions** run in a process whose execution-state mutations are never forwarded to the +producer. Job **definitions** are published from a different place — +`plugins/workers/services/src/init.ts:91-111` calls `emitJobToStream(...)` per job at startup, which +is exactly the three startup snapshots the issue observed. Chain: +`plugins/workers/streams/producer.ts:61-63` → `packages/plugin-workers-core/src/streams/producer.ts:122-127` +→ `producer.upsert('job', job)`. Stream path `/workers/executions`, producerId `workers-service` +(`packages/plugin-workers-core/src/streams/producer.ts:5-6,66-73`). + +**The repo already says this.** The two Flow-B OTEL gates are deferred against #1398 with the reason +*"workers-combined does not install the stream mutation hook"* — +`packages/cli/e2e/suites/scaffold/capability-suites.ts:23-34`, asserted by +`packages/cli/e2e/tests/presentation/suite-registry_test.ts:204-215`. `behavior.otel.stream-consumer` +and `behavior.otel.traces` are registered (`otel-gates.ts:52-84`) but excluded from +`scaffold.runtime` / `scaffold.runtime.sqlite` via `SCAFFOLD_RUNTIME_DEFERRED_GATES`. + +## Execution path and the identifiers available at completion + +Webhook → triggers service → `TriggerProcessor.process` → `dispatchAction` +(`packages/plugin-triggers-core/src/runtime/trigger-processor.ts:172`, wired at +`plugins/triggers/src/runtime/trigger-runtime-processor.ts:80-82`) → `enqueueWorkerJob` (`:318-367`) +enqueues a `JobMessage` on queue `jobs` with `correlationId = event.id` (`:333`) and +`traceparent`/`tracestate` from `traceJobDispatch` (`:348-354`). + +Worker consumption — `plugins/workers/worker/job-dispatcher.ts:35-175`: + +| Step | Line | Note | +| --- | --- | --- | +| `executionState.create({… correlationId, traceparent, tracestate})` | `:74-83` | **before** the `job.execute` span exists | +| `traceJobExecution` creates the `job.execute` span | `:91` | | +| `executionState.start` | `:131` | inside the span context | +| `executionState.complete(executionId, {status, exitCode, result, error})` | `:165-170` | inside the span context | + +`traceJobExecution` → `withSpan` (`packages/telemetry/src/instrumentation/worker.ts:295-323`; +`packages/telemetry/src/application/span.ts:32-62`) uses `context.with(trace.setSpan(...))`, so any +span started **during** the callback inherits the `job.execute` trace id. + +The stored record already carries the join identifiers: +`packages/plugin-workers-core/src/state/execution-state.ts:35-78` (`traceparent` `:74`, `tracestate` +`:76`), with mutations emitted at `:288` (delete) and `:307` (`#save`, created/updated). + +## The producer is already reachable — nothing needs threading + +`getWorkersStreamProducer()` is a module-level singleton +(`plugins/workers/streams/producer.ts:28-44`), so the worker process can publish without new +plumbing. Public API: `createDurableStream` +(`packages/plugin-streams-core/src/application/create-durable-stream.ts:28-49` options, `:124-146` +`upsert`, `:148-170` `delete`, `:171-186` `flush/stop/close`, `:295-320` singleton-per-streamPath +with a fingerprint-mismatch throw). `upsert` returns `StreamWriteReceiptV1` +(`producer-contract-v1.ts:97-105`); bounded buffer 256 events / 1 MiB (`:47-55`). + +**Environment precondition:** `getStreamsUrl()` requires `DURABLE_STREAMS_URL` or +`services__streams__http__0` (`packages/plugin-streams-core/src/application/stream-url-resolver.ts:154-190`); +missing → throw at `create-durable-stream.ts:262-272`. Whether `workers-combined` actually receives +that env is **unverified** — see below. + +## Record shape and how `traceparent` is actually carried + +`traceparent` is **not** on the stream record. It is a **header**, generated by the producer's own +`stream.publish` span: `create-durable-stream.ts:222-242` → +`packages/plugin-streams-core/src/telemetry/instrumentation.ts:159-177` +(`headers.traceparent = formatTraceparent(span.spanContext())`). `correlationId` defaults to the +entity key when no `StreamWriteContextV1` is passed (`create-durable-stream.ts:229-230`). The +envelope contract requires `operation`, `correlationId`, `traceparent` +(`packages/plugin-streams-core/src/domain/sse-contract-v1.ts:34-56`). + +The `execution` entity schema **omits** `traceparent`/`tracestate` +(`packages/plugin-workers-core/src/streams/schema.ts`, `WorkerExecutionZodSchema`), and the mapper +drops them (`packages/plugin-workers-core/src/streams/producer.ts:88-106`). + +**This matters more than it looks.** Because the header trace id comes from the publish span, the +join to `job.execute` is satisfied by *where the publish happens in the trace context*, not by +copying a field. That is what the E2E gate asserts. + +## The gate that defines "done" + +`packages/cli/e2e/src/application/gates/scaffold/consume-flow-b-stream.ts:52-107` reads the +`job.execute` correlation + trace id from the dashboard, then selects; offset/live-SSE loop at +`:203-230`. The selector `select-flow-b-stream-change.ts:68-115` matches on `value.correlationId` +else `headers.correlationId`, and `:122-153` (**TC-14**) requires the matched record's +`headers.traceparent` trace id to equal the `job.execute` trace id. + +Consumer surface: `bindStreamEventSourceV1`, `createStreamSseReplayStateV1`, +`parseStreamSseEventV1` (`packages/plugin-streams-core/src/sse/mod.ts:30-34`); offsets are opaque +(`sse-contract-v1.ts:25,62-72`). + +## Doctrine and dependency edges + +- `plugins/workers/deno.json` **already declares** `@netscript/plugin-streams-core`, and the manifest + declares `.withDependencies({ streams: streamsPlugin })` (`plugins/workers/src/public/mod.ts:61`). + **A workers-side fix needs no new dependency edge.** +- Two undeclared-import findings, incidental to this issue but real: + `packages/plugin-workers-core/deno.json` does not list `@netscript/plugin-streams-core` while + `src/streams/producer.ts:1` and `src/streams/schema.ts:1-2` import it; same for + `plugins/triggers/deno.json` vs `plugins/triggers/streams/{producer,factory,schema}.ts`. Both + resolve through the workspace today. Whether `deno publish` rejects them is **unverified**. +- Plugins are Archetype 5 / thin; convention-bearing primitives belong in `-core` + (`docs/architecture/doctrine/11-plugin-thinness-and-base-seams.md:16-36`). +- No `arch-debt.md` entry mentions #1398. + +## Existing tests + +`packages/plugin-workers-core/tests/streams/workers-streams_test.ts:9-104` covers the mapper and hook +as units against a mock producer, with **no traceparent assertion**. There is no test that would fail +if the hook were never installed on the background runtime — which is precisely why this shipped. + +## The two decisions the plan must make (these are findings, not proposals) + +1. **Volume and key collision.** Every mutation fires the hook (`execution-state.ts:288,307`), so + installing it on the worker publishes roughly four records per execution + (pending/queued/running/terminal) — each an `upsert` on the **same key**. +2. **Trace-context placement — the sharp edge.** TC-14 requires the matched record's publish span to + be a child of `job.execute`. `complete()` (`job-dispatcher.ts:165`) **is** inside that active + context, so its publish inherits the right trace id. But `create()` (`:74`) runs **before** + `traceJobExecution` (`:91`), so a record published from it carries a **different** trace id. Since + the selector matches on `correlationId` and both records share it, a naive "install the hook and + publish everything" fix can produce a record set where the *first* match fails TC-14. Publishing + more records is not automatically closer to passing. + +`correlationId` reaches the consumer only because `toExecutionStreamEntity` sets `value.correlationId`; +`createStreamMutationHook` passes no `StreamWriteContextV1` (`producer.ts:112-118`). + +## Unverified — do not treat as facts + +- Whether `workers-combined` actually receives `services__streams__http__0` at runtime + (`packages/cli/src/kernel/templates/aspire/helpers/register/generate-register-background.ts:180-220` + was located but **not read**). If it does not, the fix is larger than a hook installation. +- Whether `plugins/workers/bin/{combined,worker,scheduler}.ts` exist as thin wrappers — only + `bin/runtime.ts` was read. +- Whether `deno publish` fails on the undeclared `plugin-streams-core` imports above. +- The triggers-side execution publication (`plugins/triggers/streams/server.ts`) and whether its hook + is installed. +- `packages/plugin-workers-core/src/domain/job-spec.ts:113-142` ("event published when a job + execution completes") — not read. +- Workers SSE `subscribe.ts` (KV-watch, distinct from the durable stream) — not read. +- `validate-flow-b-traces.ts` contents — not read. +- Whether the streams schema carries an explicit version marker (bears on whether adding + `traceparent` to the entity implies a version bump). + +## Handoff — files to open in order + +1. `plugins/workers/bin/runtime.ts:89-152` — the missing `setMutationHook` call site. +2. `plugins/workers/services/src/main.ts:65-75` — the correct wiring to mirror. +3. `plugins/workers/worker/job-dispatcher.ts:74-170` — identifiers and the `job.execute` context. +4. `packages/plugin-workers-core/src/streams/producer.ts:88-119` — mapper + hook. +5. `packages/plugin-workers-core/src/streams/schema.ts` — `WorkerExecutionZodSchema` field set. +6. `packages/plugin-streams-core/src/application/create-durable-stream.ts:124-146,222-242` — publish + and header semantics. +7. `packages/cli/e2e/src/application/gates/scaffold/select-flow-b-stream-change.ts:122-153` — TC-14. +8. `packages/cli/e2e/suites/scaffold/capability-suites.ts:23-34` + + `packages/cli/e2e/tests/presentation/suite-registry_test.ts:204-225` — the deferral to remove. +9. `packages/cli/src/kernel/templates/aspire/helpers/register/generate-register-background.ts:180-220` + — confirm the streams env reaches `workers-combined` (**first unverified item**). diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/research-1405.md b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1405.md new file mode 100644 index 0000000000..5322e7a1a6 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1405.md @@ -0,0 +1,98 @@ +# Research — #1405 durable producer rejection taxonomy + +Author: orchestrator session (Claude · Opus 5 · high). Baseline `origin/main@01aa12b67`. +Read-only inspection; no code changed in this phase. + +## Confirmed defect 1 — close-drain window reports `producer-failed` + +`close()` sets `#accepted = false` at +`packages/plugin-streams-core/src/application/durable-stream-producer-supervisor.ts:215` and *then* +delegates to `#closeGracefully()` (`:216`), which first `await`s `flush()` and the connect promise +(`:221-224`) before reaching `this.#transition('stopping', 0)` at `:225`. + +For the whole duration of that drain, `#accepted` is `false` while `#state.state` is still `ready` +(or `connecting`/`backoff`). Both rejection selectors fall through to their default arm: + +- supervisor `#writeRejectionReason()` — `:467-478`: `#accepted` false, state not + `stopping`/`stopped` → `default: return 'producer-failed'`. +- façade `stateRejection()` — `create-durable-stream.ts:251-262`: `closed` is true (supervisor + `get closed()` is `!this.#accepted`, `:109-111`), state not `stopping`/`stopped` → + `return 'producer-failed'` at `:260`. + +Both `upsert` (`create-durable-stream.ts:131-133`) and the second write entry point (`:159-160`) go +through `stateRejection`. So a write arriving during an orderly drain is told the producer failed. +It did not: `#fail()` was never called and `#state.error` is unset. + +`#stopImmediately()` does **not** have this hole — it transitions to `stopping` on the line right +after clearing `#accepted` (`:185-186`), with no `await` between. + +## Confirmed defect 2 — first-attempt refusal reports `retry-exhausted` + +`#failActive()` (`:412-423`) settles every non-`aborted` failure as +`{ status: 'delivery-unknown', reason: 'retry-exhausted' }` (`:418-420`). It has three call sites, +and only one of them is genuine exhaustion: + +| Call site | Guard | True cause | +| --- | --- | --- | +| `:302` | `!isRetryable(connected.failure) \|\| attempt === maxAttempts` | **conflated** — either refusal or exhaustion | +| `:346` | `else if (!isRetryable(result.failure))` | **refusal**, can fire on attempt 1 | +| `:350` | `if (attempt === this.#reconnectPolicy.maxAttempts)` | genuine exhaustion | + +`:346` is reachable on attempt 1 with `maxAttempts` at any value: the append transport returned a +non-retryable failure, i.e. the server positively refused the write. Nothing was exhausted. + +## Public surface impact + +The reason unions are declared in +`packages/plugin-streams-core/src/domain/producer-contract-v1.ts`: + +- `StreamWriteRejectionReasonV1` `:70-78` — already contains `producer-stopping`. +- `StreamWriteUnknownReasonV1` `:84-87` — `retry-exhausted | transport-aborted | producer-stopped`; + has **no** member naming a positive refusal. + +Both are re-exported from the package root (`packages/plugin-streams-core/mod.ts:44-45`), so they +are published surface. A repo-wide grep finds **no consumer outside `plugin-streams-core/src` that +switches on either union**, so widening `StreamWriteUnknownReasonV1` breaks no in-repo exhaustive +switch. It is still an additive change to a published union and must be treated as such in the PR +body. + +## Consequence for the fix shape + +1. **Close-drain.** No new public member is needed: `producer-stopping` already exists and already + means "the producer is shutting down and is not accepting". The defect is that the *selector* + cannot see the closing state before the `stopping` transition happens. The minimal accurate fix + is to make the closing intent observable to both selectors from the moment `close()` is entered, + so they return `producer-stopping` instead of falling through to `producer-failed`. Adding a new + `producer-closing` member is the alternative; it grows published surface for a distinction + (graceful drain vs abrupt stop) that the existing vocabulary already covers, and the issue only + requires "a reason naming the closing state, distinct from `producer-failed`". +2. **Refusal.** A new member of `StreamWriteUnknownReasonV1` **is** required — the issue demands a + reason naming the refusal *and* that retry-exhaustion keeps its own reason, and no existing + member means "positively refused". `#failActive` must branch on `isRetryable(failure)` rather + than only on `failure.kind === 'aborted'`, and `:302` must stop conflating its two guards. + +Neither change alters which writes are accepted, rejected, or delivered. `#fail()` is still called +in every path it is called in today; only the settled `reason` string differs. + +## Negative tests the acceptance requires + +Acceptance box 4 ("a future refactor that collapses them fails") means each reason needs a test that +would go red if the reason regressed to the old value: + +1. write issued after `close()` is entered but before the drain completes → settles + `rejected` / closing reason, **asserted not** `producer-failed`. +2. append transport returns a non-retryable failure on attempt 1, `maxAttempts > 1` → settles + `delivery-unknown` / refusal reason, **asserted not** `retry-exhausted`, and the attempt count + observed is 1. +3. append transport returns retryable failures until `maxAttempts` → still settles + `retry-exhausted`. +4. an existing-behaviour guard that accepted/delivered/rejected classification is unchanged + (`tests/telemetry/durable-stream-producer-telemetry_test.ts:172-219` already distinguishes + rejected / dropped / delivery-unknown and must stay green). + +## Open decision for the implementer brief + +Reuse `producer-stopping` for the close-drain window, or add `producer-closing`. The brief will +**lock reuse of `producer-stopping`** (no published-surface growth where existing vocabulary is +accurate) and require the refusal member to be the only union addition. Divergence goes in +`drift.md`, not into the implementer's discretion. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/research-1548.md b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1548.md new file mode 100644 index 0000000000..a78001fe2c --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1548.md @@ -0,0 +1,97 @@ +# Research — #1548 Fresh Streams browser resolver cannot see Aspire VITE service references + +Delegated read-only sub-agent (Claude · Opus 5, `drift.md` D-1), bounded brief. Finished **within +budget** — 15 tool calls, ~2 min — after the earlier #1398 pass had to be stopped at 72 calls. Every +claim carries a `path:line` citation; the unverified list is part of the finding. + +## The read path today + +`packages/plugin-streams-core/src/application/stream-url-resolver.ts`: + +- `getBrowserServiceEndpoint()` (`:54-72`) calls `readImportMetaEnvironment(import.meta)` at `:60` — + **`import.meta` is passed as a value argument**. +- `readImportMetaEnvironment` (`:74-79`) reaches `meta.env` through a **parameter binding**, guarded + by `'env' in meta` and `isEnvironmentRecord` (`:81-88`). +- Both keys are then read by **computed index** — `env[fullKey]`, `env[shortKey]` (`:64,:68`) — for + `VITE_services__streams__http__0` and `VITE_STREAMS_URL`. +- The whole browser branch is wrapped in `try { … } catch { return undefined }` (`:59,69-71`), so + **every failure is silent**. That is why the symptom is a generic "URL not found" rather than + anything pointing at env reading. + +The thrown message (`:128-132`) is a **verbatim match** with the issue text. + +**Precedence** in `getStreamsUrl()` (`:99-133`): `DURABLE_STREAMS_URL` (Deno-gated) → +`services__streams__http__0` (Deno-gated) → **browser `VITE_*` branch** → throw. There is no default +or `localhost` fallback. The browser branch is third and is the **only** branch reachable in a +browser bundle. + +## The reported cause: code shape confirmed, build semantics deliberately not asserted + +Two independent reasons the value is unreachable by static substitution — the issue names the first, +the research found the second: + +1. The literal expression `import.meta.env.VITE_services__streams__http__0` **never appears in + source**; `import.meta` crosses a function boundary as a value (`:60` → `:77-78`). +2. Even given a bag, the keys are read by **computed index** (`:64,:68`), not as static member + expressions. + +**Repo-internal corroboration, which is stronger than a claim about Vite:** this codebase's own +first-party Vite plugin injects env by emitting `define` entries keyed exactly as +`` `import.meta.env.${mapping.target}` `` (`packages/fresh/src/application/vite/vite.ts:195`, applied +at `:311-335`). That mechanism is textual/static-expression substitution **by construction** — it +cannot reach a value-passed `import.meta` nor a computed index. So the failure is explicable purely +from this repo's own substitution machinery, without asserting anything about Vite's internals. + +The sub-agent explicitly **did not** verify Vite's runtime behaviour for a bare `import.meta`, nor +how Fresh's Vite pipeline treats a JSR dependency module, and said so rather than asserting it. + +## A first-party Vite plugin already exists — this is not net-new infrastructure + +`createNetScriptVitePlugin()` (`packages/fresh/src/application/vite/vite.ts:282`, `enforce: 'pre'`, +`:307-309`) already carries a typed env-injection API: `NetScriptViteEnvMapping` (`:76-83`), +`envMappings`/`env` options (`:99-102`), and `buildDefineEntries()` (`:183-199`). It has tests +(`vite.test.ts`). + +But the scaffold template +(`packages/cli/src/kernel/assets/app/vite.config.ts.template`) wires the plugin **without** +`envMappings` or `env` — it passes only `appRoot`, `workspaceRoot`, `aliasEntries`, `watchPaths`, +`routeManifest`. The define machinery is present and currently unused by generated apps. + +## The SDK already solved this shape + +`packages/sdk/src/discovery/browser-env.ts` splits a **pure** `getBrowserServiceUrlFromEnv(env, …)` +(`:35-54`) from the `import.meta` access (`:65`), and that pure function **is unit-tested** — +`packages/sdk/tests/discovery/env-ordering_test.ts:24,41,56` cover browser-full-key precedence, +shorthand fallback, and server fallback with an **injected env bag**. + +That is exactly the asymmetry here: the injectable function is testable and tested; the +`import.meta`-reading function is neither. + +## Test coverage: none + +**There are no unit tests for `stream-url-resolver.ts`'s URL resolution at all.** The nine test files +in `packages/plugin-streams-core/tests/` never reference `getStreamsUrl`. The only repo-wide +references pass an explicit `baseUrl` and bypass resolution entirely +(`plugins/streams/services/src/sse-contract_conformance_test.ts:108`, +`packages/fresh/src/runtime/streams/create-stream-db_test.ts:22`). Nothing simulates a browser or +Vite environment. + +## Options, with published-surface impact (no recommendation made by research) + +| Option | Surface impact | Note | +| --- | --- | --- | +| Literal `import.meta.env.VITE_*` member expressions inline | **none** — `getBrowserServiceEndpoint` is not exported (`mod.ts:19` exports only `buildStreamUrl`, `getStreamsAuth`, `getStreamsUrl`) | key strings are fully known: `STREAMS_RESOURCE_NAME` is a compile-time constant (`constants.ts:5`) | +| Split reader from lookup, SDK-style | additive export **if** the pure fn is exported | makes the browser path unit-testable without a Vite build | +| Add `envMappings` to the scaffold template | changes generated output + CLI template, not the package API | only helps if the read pattern is one `define` can reach — i.e. must pair with the literal-read change | +| New `transform` hook rewriting the JSR module | none | most fragile; couples the framework to a bundler's module graph | +| Documented required `baseUrl` override | none | zero-code docs fix, but pushes discovery onto every consumer and contradicts the resolver's stated purpose (`:42-53`) | + +## Explicitly not verified + +- Vite's actual behaviour for a value-passed `import.meta`, for computed indexing of + `import.meta.env`, and for JSR dependency modules under Fresh's Vite pipeline. +- That the Aspire AppHost in fact injects both VITE variables into the dashboard process — the naming + helper exists (`packages/aspire/src/application/build-vite-env-var-name.ts`) but the injection path + was not traced. +- Any runtime reproduction; no gates, tests, or builds were run. +- Whether `vite.test.ts` covers `buildDefineEntries`. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1398.md b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1398.md new file mode 100644 index 0000000000..04845d0b1c --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1398.md @@ -0,0 +1,163 @@ +# Worklog — #1398 publish job executions to the durable stream + +Implementation lane: Codex · GPT-5.6 Sol · medium (`normal_implementation`). +Branch: `fix/1398-publish-job-executions-to-durable-stream`. +Baseline: `origin/main@01aa12b67`. + +The approved `plan.md`, `plan-eval.md`, and `slices/research-1398.md` were read from the local +orchestration branch `chore/release-0.0.6-features-orchestration` because the fresh leaf branch was +created at the shared baseline before those orchestration-only commits existed. The locked plan was +not re-decided. + +## Design + +- **Public surface:** no new export and no `WorkerExecutionZodSchema` change. The existing + `WorkerExecutionRecord` type acknowledges the already-stored optional W3C trace headers. +- **Runtime state and identity:** every created, updated, and deleted execution mutation retains its + execution id, correlation id, `traceparent`, and optional `tracestate`. +- **Ports:** `WorkersStreamProducer` remains the stream port. The core mutation hook uses + `@netscript/telemetry/context` to make the stored W3C context ambient while that port publishes. +- **Plugin wiring:** the thin workers plugin installs the core-owned mutation hook only in the + worker-owning background entrypoints (`startWorkerProcess` and `startCombinedProcess`). The + scheduler does not own or mutate execution state. +- **Commit slices:** S1 installs and tests the hook/context join; S2 un-defers the two OTEL gates and + updates both registry tests; S3 records the one-pass live `scaffold.runtime` result. +- **Deferred scope:** stream rejection reasons (#1405), undeclared stream-core imports, and worker + execution schema changes remain out of scope. +- **Contributor path:** execution stream policy lives in + `packages/plugin-workers-core/src/streams/producer.ts`; process wiring lives in + `plugins/workers/bin/runtime.ts`; the adjacent stream and runtime tests are the regression map. + +## S1 — mutation hook installation and trace-context join + +Changed: + +- `packages/plugin-workers-core/src/streams/producer.ts` wraps every mutation publication in the + execution's extracted W3C context. +- `plugins/workers/bin/runtime.ts` installs the hook in worker-only and combined background + processes; scheduler-only remains unchanged because its execution-state surface is reserved and + has no mutation methods. +- `packages/plugin-workers-core/tests/streams/workers-streams_test.ts` adds stored-context and + pre-span trace-id guards. +- `plugins/workers/tests/runtime/background-stream-hook_test.ts` spies the real combined entrypoint's + `KvExecutionState.setMutationHook` call while suppressing worker/scheduler loops. + +Gate results before commit: + +| Gate | Result | +| --- | --- | +| workers scoped check | PASS — 101 files, 0 findings | +| plugin-workers-core scoped check | PASS — 111 files, 0 findings | +| workers scoped lint | PASS — 101 files, 0 findings | +| workers scoped format | PASS — 101 files, 0 findings | +| plugin-workers-core package test | PASS — 27 passed, 0 failed | +| combined-runtime installation test | PASS — 1 passed, 0 failed | +| `quality:gate` | PASS — quality scan clean; doctrine `FAIL=0` on touched roots | +| explicit plugin-workers-core quality scan | PASS — 0 findings, 0 allowances | + +Negative guard evidence: + +- Removing only D3 made both trace-context tests fail (exit 1): ambient trace id was retained for + the override case and a new random trace id appeared for the pre-span case. +- Removing only the combined-process hook installation made the installation test fail (exit 1): + actual installation count `0`, expected `1`. + +Reconcile note: issue #1398 is open with milestone `0.0.6` and already carries `type:fix`, +`area:plugins`, `area:telemetry`, `priority:p1`, and exactly one lifecycle label, `status:impl`. + +## S2 — restore the two OTEL runtime gates + +Changed in one slice, per PLAN-EVAL F2: + +- Added `behavior.otel.stream-consumer` and `behavior.otel.traces` to `RUNTIME_GATES`, which also + carries them into the SQLite runtime tier. +- Emptied `SCAFFOLD_RUNTIME_DEFERRED_GATES` while preserving the shared explicit deferral surface. +- Flipped the main runtime-suite presence assertions to `true`. +- Rewrote the exact-deferral test to assert both runtime tiers have no #1398 deferral and execute + both gates. + +First full-package check: RED. An empty `as const satisfies readonly DeferredGate[]` inferred the +constant's element as `never`, so existing runner-test fixture code could not read `issue` or +`reason`. Fixed by declaring the empty value against the explicit `readonly DeferredGate[]` +contract; no runtime behavior or assertion was relaxed. + +Second full-package test: RED with one in-scope and one unrelated failure. The in-scope failure was +a third stale #1398 pin in `suite-runner_test.ts`, which still expected two deferred skipped steps; +it now asserts zero deferrals, steps, skipped summary entries, and deferred reporter events. This +test was not named in the approved plan or PLAN-EVAL F2 and is recorded as minor factual drift. The +unrelated `quickstart-command-drift_test.ts` failure came from the package task changing cwd while +the test reads `docs/site/quickstart.vto` relative to the repository root; verify it from root and +do not absorb that task/path defect into #1398. + +Final S2 evidence: + +| Gate | Result | +| --- | --- | +| suite-runner + registry focused tests | PASS — 27 passed, 0 failed | +| CLI E2E test tree from repository root | PASS — 152 passed, 0 failed | +| CLI E2E scoped check | PASS — 163 files, 0 findings | +| CLI E2E scoped lint | PASS — 163 files, 0 findings | +| CLI E2E scoped format | PASS — 163 files, 0 findings | +| explicit CLI E2E quality scan | PASS — 0 findings, 0 allowances | + +Negative guard evidence: temporarily removing both OTEL gates from `RUNTIME_GATES` made the main +runtime presence test and the both-tier execution test fail (exit 1, actual `false`, expected +`true`). + +## Final implementation gates + +All required non-live gates passed on the S1+S2 head: + +| Gate | Result | +| --- | --- | +| workers scoped check | PASS — 101 files, 0 findings | +| plugin-workers-core scoped check | PASS — 111 files, 0 findings | +| workers scoped lint | PASS — 101 files, 0 findings | +| workers scoped format | PASS — 101 files, 0 findings | +| plugin-workers-core package test | PASS — 27 passed, 0 failed | +| `quality:gate` | PASS — quality scan clean; doctrine `FAIL=0` on touched roots | +| explicit plugin-workers-core quality scan | PASS — 0 findings, 0 allowances | +| explicit CLI E2E quality scan | PASS — 0 findings, 0 allowances | + +The configured quality scan covers `packages/cli/src` and `plugins`, but not +`packages/plugin-workers-core` or `packages/cli/e2e`; the two explicit scans above close that +coverage gap. No lockfile change, schema change, public export, lint suppression, `any`, unsafe +double cast, or `@ts-ignore` was added. + +## S3 — one-pass live runtime evidence + +Serialization was confirmed before launch: `/tmp/netscript-e2e-scaffold-runtime.lease` was absent, +no competing `e2e:cli`/`scaffold.runtime` process was running, and Docker reported zero containers. + +The mandated single invocation was run once and was not retried: + +```text +$ deno task e2e:cli run scaffold.runtime --cleanup --format pretty +> runtime.flow-b-fixture: Wire real Flow-B callback fixture + FAILED 10777ms + Command exited 1; expected 0. + error: Uncaught (in promise) Error: netscript generate plugins failed: Error: fetch failed +> cleanup.aspire-stop: Stop generated Aspire AppHost + PASSED 81ms +Summary: passed=33 failed=1 skipped=0 +raw exit code: 1 +``` + +This failure occurred before Aspire launch and before `behavior.otel.stream-consumer` or +`behavior.otel.traces`, so neither restored gate received a live verdict. The gate therefore does +not provide #1398's required end-to-end acceptance evidence. + +Diagnostics followed the required order: + +- Plugin doctor against the preserved generated fixture reported every configured plugin healthy; + its only warning was that no AppHost was running, expected because the failure preceded launch. +- Aspire OTEL logs/traces were unavailable because the AppHost never started. +- The failure is not the known `Missing plugin reference "streams"` residual; that message did not + occur, and generation failed earlier with Deno's generic network-resolution `fetch failed`. +- A focused rerun of only the failed `generate plugins` command against the preserved fixture then + passed in 0.9 seconds and wrote all three registries. This makes the observed failure consistent + with a transient dependency/registry fetch, but the nested Deno error did not expose a URL, so + the exact endpoint could not be verified. This diagnostic does not alter the authoritative red + E2E verdict and was not a second suite run. +- `agentic:leak-check` reported Aspire and Docker probes `ok` with no survivors; Docker also showed + zero running containers after cleanup. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1457.md b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1457.md new file mode 100644 index 0000000000..1a38eca52b --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1457.md @@ -0,0 +1,63 @@ +# Worklog — #1457 chat proxy query forwarding + +Implementation lane: Codex · GPT-5.6 Sol · low (`light_implementation`). +Branch: `fix/1457-chat-proxy-query-forwarding`. +Baseline: `origin/main@f99cb4fbf`. + +## Design + +- Public contract: optional `query(URLSearchParams): URLSearchParams`; its explicit types avoid a + slow-type regression and existing callers remain source-compatible. +- Default: forward every incoming pair except `id`. +- Merge: snapshot names already present on the resolved upstream URL, then append every forwarded + pair whose name is not in that snapshot. This preserves repeated client keys while making every + resolved value authoritative on collisions. +- Existing IO behavior: header, auth, request body, duplex, redirect, abort, fetch, stream body, + and response sanitization code remains unchanged. +- PLAN-EVAL: N/A — the owner brief locks every behavioral decision for this small slice. +- Deferred scope: #1459 `application/defer/**` and #1548 `stream-url-resolver.ts`. + +## Evidence + +The first package test run was red with two test-fixture expectation failures: + +- The collision fixture used the string `streamPath` form, which is a prefix before `sessionId`; + changed it to the documented function form so the resolved URL genuinely carries the query. +- The pre-existing eis-chat test expected the incoming `session` parameter to disappear; D1 now + intentionally forwards it, so the expected injected-fetch URL includes `?session=eis-123`. + +No implementation behavior or assertion strength was relaxed. + +Final gate evidence on the formatted head: + +| Gate | Result | +| --- | --- | +| Fresh scoped check | PASS — 182 files, 2 batches, 0 findings | +| Fresh scoped lint | PASS — 182 files, 0 findings | +| Fresh scoped format | PASS after formatting the one owned file — 182 files, 0 findings | +| post-format parameter grep | PASS — `offset`, `live`, `handle`, `cursor`, `forwardedQuery`, `resolvedQueryNames`, and the `query` option remain present | +| `quality:gate` | PASS — exit 0; configured quality and doctrine roots omit `packages/fresh` | +| explicit Fresh quality scan | PASS — 0 findings; one pre-existing allowance in route-support | +| explicit Fresh doctrine scan | PASS — `FAIL=0`; three pre-existing warnings and one info finding | +| Fresh package tests | PASS — 223 passed, 0 failed | +| Fresh doc lint | exit 0; AI entrypoint 0 findings, package aggregate 44 pre-existing findings in untouched query/route/streams surfaces | + +The first format-wrapper invocation was red with one finding in the newly added `forwardedQuery` +expression. Only `stream-proxy.ts` was formatted, after which the wrapper passed. The mandatory +post-format grep confirmed every introduced protocol parameter and merge symbol remained present. + +`quality:gate` confirmed #1542's coverage gap in its own output: `quality:scan` scanned only +`packages/cli/src` and `plugins`, while `arch:check` did not include `packages/fresh`. Explicit Fresh +quality and doctrine scans therefore provide the package-specific evidence. + +`doc:lint` returned exit code 0 while reporting 44 findings: 27 private-type references and 17 +missing JSDoc items, all in untouched query, route, and streams files. The changed +`src/runtime/ai/mod.ts` entrypoint reported zero findings. This slice did not hide or repair those +out-of-scope legacy findings. + +## GitHub reconciliation + +Issue #1457 was fetched live on 2026-08-12. It is open, milestone 26 (`0.0.6`), and carries +`type:fix`, `area:fresh`, `area:plugin-ai`, `area:streams`, `priority:p1`, and `status:plan`. +Its current body has no markdown acceptance checkboxes, so there are no valid `box-index` entries +to mirror in the PR's `acceptance-evidence` block. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1548.md b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1548.md new file mode 100644 index 0000000000..9807d01669 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1548.md @@ -0,0 +1,65 @@ +# Worklog — #1548 browser stream resolver + +## Design + +- **Public surface:** unchanged. `mod.ts` continues to export only `buildStreamUrl`, + `getStreamsAuth`, and `getStreamsUrl` from this resolver; the injectable browser lookup is + source-internal and imported by tests through its `src/` path. +- **Domain vocabulary:** the two finite browser keys are + `VITE_services__streams__http__0` and `VITE_STREAMS_URL`; their narrow `ImportMeta` declaration + lives in `stream-browser-environment.d.ts`. +- **Ports / seams:** `getBrowserStreamsUrlFromEnv()` is the pure injected environment-bag seam. + `getBrowserServiceEndpoint()` is the impure edge that contains the literal static member reads. +- **Constants:** `STREAMS_RESOURCE_NAME` remains the server-side discovery constant. The browser + member names are intentionally literal because their source shape is the substitution contract. +- **Commit slice:** one S1 implementation slice changes the reader shape and adds five tests. The + scoped static gates, package tests, doc lint, repo quality gate, and explicit package source scan + prove it. D6's catch narrowing was dropped because it was unnecessary to fix precedence or shape. +- **Deferred scope:** no Vite transform, scaffold template, SDK fix, real Vite build fixture, or + browser-auth change. +- **Contributor path:** add browser discovery behavior in `stream-url-resolver.ts`; add finite key + typing in `stream-browser-environment.d.ts`; preserve the source-shape guard in + `stream-url-resolver_test.ts`. + +## Implementation + +- Replaced value-passed `import.meta` and computed browser-key indexing with the two binding literal + member expressions. +- Split the pure injected-bag precedence lookup from the impure browser reader. +- Kept server/browser precedence and the public export map unchanged. +- Added a source-shape regression guard and a text-substituted module fixture that reaches the real + `getStreamsUrl()` path. The fixture is not claimed as a real Vite substitution test. + +## Tests added + +1. `browser streams URL lookup prefers the full Aspire key over shorthand` +2. `browser streams URL lookup falls back to the shorthand key` +3. `browser streams URL lookup returns undefined when both keys are absent` +4. `browser streams URL reader preserves Vite-substitutable static member expressions` +5. `getStreamsUrl reaches the browser reader and preserves browser key order` + +## Gate evidence + +All required commands exited 0 after the final source shape landed: + +| Gate | Result | +| --- | --- | +| scoped check | `filesSelected=45`, `failedBatches=0`, no findings | +| scoped lint | `filesSelected=45`, no findings | +| scoped format | `filesSelected=45`, `failedBatches=0`, no findings | +| package test task | `38 passed`, `0 failed` | +| doc lint | `totalErrors=0`, `totalMissingJSDoc=0` | +| `quality:gate` | exit 0; its configured roots do not include this package | +| explicit package source scan | `ok=true`, `findings=[]`, `allowCount=0` | + +The package-quality verdict rests on the explicit +`scan-code-quality.ts --root packages/plugin-streams-core/src` result, not the repo-level quality +gate, whose `arch:check` roots omit `packages/plugin-streams-core`. + +## Reconcile + +- PR #1559 still carries `Closes #1548`, remains draft, and already has exactly one lifecycle label: + `status:impl`. +- PLAN-EVAL PASS and its binding amendments are preserved. IMPL-EVAL remains for the orchestrator's + separate evaluator session; this implementation session did not flip the PR ready. +- No plan, doctrine, or debt drift was introduced. diff --git a/.llm/runs/release-0.0.6-features--orchestration/supervisor.md b/.llm/runs/release-0.0.6-features--orchestration/supervisor.md new file mode 100644 index 0000000000..8a8fac4c7f --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/supervisor.md @@ -0,0 +1,102 @@ +# Supervisor — NetScript 0.0.6 runtime / public-surface lane + +| Field | Value | +| ---------------- | ------------------------------------------------------------------------------ | +| Run id | `release-0.0.6-features--orchestration` | +| Profile | `.llm/harness/workflow/milestone-run.md` (topical lane, not the whole milestone) | +| Role skill | `.agents/skills/agent-milestone-orchestrator` | +| Supervisor model | Claude · Anthropic · Opus 5 · high (`planning_decisions`) | +| Session surface | Claude Code, native, mobile-visible via `/rc` | +| Host | WSL2 Linux 6.18.33.2-microsoft-standard-WSL2 | +| Checkout | `/home/codex/repos/netscript-006-features` | +| Control branch | `chore/release-0.0.6-features-orchestration` | +| Baseline | `origin/main@01aa12b67` | +| Milestone | GitHub milestone 26, `0.0.6` | +| Opened | 2026-08-12 | + +## Scope — exclusive ownership + +This is a **topical lane**, not the whole 0.0.6 milestone. Sibling orchestration lanes run in +parallel checkouts (`netscript-006-docs`, `netscript-006-fixes`, `netscript-006-internals`). This +lane owns exactly two issues and takes no others: + +| Order | Issue | Title | Shape | +| ----- | ------ | --------------------------------------------------------------------------------- | ---------------------------------- | +| 1 | #1405 | durable producer settles two inaccurate rejection reasons | small, specified, deterministic | +| 2 | #1398 | job executions are never published to the durable job stream | substantive public/runtime behavior | + +PR #1522 is unrelated and has been removed from the milestone — out of scope here. +#1398 builds on merged #1395 (versioned SSE/OTEL envelope) and #1402 (durable producer reconnect) +but is **not** satisfied by them; it stays its own reviewable PR with live E2E evidence. + +## Control-branch contract + +The control branch stores **orchestration evidence only** — this run dir and nothing else. It never +becomes an implementation umbrella. Both issues land through normal fresh leaf worktrees/branches +with draft PRs opened **directly against `main`**. + +## Lane bindings + +Routes selected from `.llm/harness/workflow/lane-policy.md`. Deviations are recorded in `drift.md`. + +| Purpose | Route | +| -------------------------------------- | ---------------------------------------------------------------------- | +| Orchestration (`planning_decisions`) | Claude · Opus 5 · high — this session | +| Research / plan sub-agents | Claude · Opus 5 · medium/high (owner brief override — see D-1) | +| Implementation #1405 | Codex · GPT-5.6 Sol · **low** (`light_implementation`), mobile-visible | +| Implementation #1398 | Codex · GPT-5.6 Sol · **medium** (`normal_implementation`), escalate to high only on genuine complexity | +| Slice review of #1405 (Sol·low pair) | `review_codex_light`: Claude · Opus 5 · high — this session | +| Slice review of #1398 (Sol·med pair) | `review_codex`: Claude · Fable 5 · low | +| PLAN-EVAL (#1398 only) | `formal_plan_evaluation` — MiniMax M3 high, fresh session | +| IMPL-EVAL #1405 (small) | `formal_impl_evaluation` small — DeepSeek V4 Flash 0731 max | +| IMPL-EVAL #1398 (broad/complex) | `formal_impl_evaluation` complex — Qwen 3.8 Max | +| CI / minor green-up watcher | Codex · GPT-5.6 Sol · low | + +**Evaluator transport.** The owner brief routes evaluation through OpenHands *after #1524 +passes/lands*; otherwise a fresh local Claude/OpenCode OpenRouter session using the toolchain. +Checked 2026-08-12: **PR #1524 is OPEN and unmerged** (`mergedAt: null`), with its own DoD boxes +`Bounded live DeepSeek smoke` and `Repository default variable is updated` still unticked. Therefore +this run uses the **local fresh-session fallback** for every evaluation until #1524 lands. Re-check +before each eval dispatch and record the observed state. + +Generator and evaluator never share a session. No implementation lane self-certifies. + +## PLAN-EVAL decision + +- **#1405** — `PLAN-EVAL: N/A`. Owner brief authorizes it; the issue carries a complete contract + (exact call sites, exact acceptance, reasons-only change with no accepted/rejected/delivered + behaviour change). Per lane-policy owner decision 2026-08-08, small/mechanical issues with a + complete contract record N/A. +- **#1398** — PLAN-EVAL **required**, separate session, before implementation. It changes public + runtime behaviour and a published record shape. +- **Wave plan itself** — this lane is two sequential PRs, not a multi-wave board. Per + `milestone-run.md` the wave-plan PLAN-EVAL is `[asserted]`, not observed; recorded decision: + **no separate PLAN-EVAL for the two-PR sequencing plan**, because the #1398 PLAN-EVAL covers the + only decision-heavy content in it. Recorded here rather than assumed. + +## IMPL-EVAL decision + +- **#1405** — **owner ruling 2026-08-12 (supersedes the reading below):** for this class — small, + deterministic, fully specified taxonomy fixes — do **not** dispatch a separate evaluator. Sufficient + evidence is focused negative tests + CI + close-gate + the orchestrator's independent diff review. + This ruling arrived **after** #1405 had already been evaluated and merged (`8ff1bcb8f`), so it does + not change that outcome; it governs this class from here. See `drift.md` D-3. +- ~~#1405 — owner brief permits an IMPL-EVAL waiver *provided exact negative tests exist*. Default is + still to run the small IMPL-EVAL lane; the waiver is only exercised if the evaluator transport is + genuinely blocked.~~ — my reading treated the waiver as a fallback rather than the default for the + class. That was the wrong default and it cost one unnecessary evaluator dispatch. +- **#1398** — IMPL-EVAL mandatory, separate session, with live runtime evidence attached. + +## Merge authority + +This orchestrator holds merge authority for both PRs and merges only through the `milestone-run.md` +pre-merge gate, recorded per PR in `worklog.md` and appended to `cut-trace.md`. Root orchestration +owns the canary and the stable cut; this lane does not publish. + +## Environment proof (stage A) + +- `git rev-parse --abbrev-ref HEAD` → `chore/release-0.0.6-features-orchestration` +- `git rev-parse --short HEAD` → `01aa12b67`, clean tree +- `git fetch origin main` → `origin/main@01aa12b67` (identical; lane starts at tip) +- `deno task agentic:runtime doctor` → `no_change (schema 1.0)`, components 18, **sessions 0** + (no pre-existing Codex sessions to collide with) diff --git a/.llm/runs/release-0.0.6-features--orchestration/worklog.md b/.llm/runs/release-0.0.6-features--orchestration/worklog.md new file mode 100644 index 0000000000..90dc8c8b02 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/worklog.md @@ -0,0 +1,550 @@ +# Worklog — 0.0.6 runtime / public-surface lane + +## 2026-08-12 — Stage A, bootstrap + +**Identity / worktree proof.** + +| Check | Command | Result | +| --- | --- | --- | +| Branch | `git rev-parse --abbrev-ref HEAD` | `chore/release-0.0.6-features-orchestration` | +| HEAD | `git rev-parse --short HEAD` | `01aa12b67` | +| Tree | `git status --porcelain` | clean at open | +| Remote | `git fetch origin main` | `origin/main@01aa12b67` — lane starts at tip | +| Runtime | `deno task agentic:runtime doctor` | `no_change (schema 1.0)`; components 18; **sessions 0** | + +**Correction to the line above.** `doctor`'s `sessions: 0` counts *desired-state runtime controller* +sessions, **not** live Codex threads. A later `deno task agentic:codex-status` showed a sibling lane +working in `/home/codex/repos/ns006-1374-compilegate` plus three idle `agy` sessions elsewhere. No +collision occurred — this lane uses its own fresh worktrees — but "sessions: 0" must not be read as +"nothing is running", and is not cited as such anywhere else in this run. + +**Issue re-baseline (live bodies fetched, not recalled).** + +| Issue | State | Milestone | Labels | Acceptance boxes | +| --- | --- | --- | --- | --- | +| #1405 | OPEN | 26 / `0.0.6` | `type:fix`, `area:plugins`, `status:triage`, `priority:p2` | 5, all unticked | +| #1398 | OPEN | 26 / `0.0.6` | `type:fix`, `area:plugins`, `area:telemetry`, `status:triage`, `priority:p1` | 4, all unticked | + +Both still carry `status:triage` and no assignee — neither has been started by another lane. + +**Predecessor state.** PR #1395 merged 2026-08-09T01:25:15Z; PR #1402 merged 2026-08-09T05:11:32Z. +Both are ancestors of the baseline, so #1398 is being planned against the landed envelope and the +landed reconnect supervisor, not against their PR branches. + +**Evaluator-transport precondition.** `gh pr view 1524` → **OPEN**, `mergedAt: null`. Its own DoD +still has `Bounded live DeepSeek smoke` and `Repository default variable is updated` unticked. The +brief's OpenHands eval route is conditional on #1524 passing/landing, so this run falls back to +fresh local sessions. Recorded as `drift.md` D-2; re-checked before each eval dispatch. + +**Research.** #1405 researched in-session (small, fully specified) → +`slices/research-1405.md`, both defects confirmed at exact call sites with a line-cited call-site +table for `#failActive`. #1398 research delegated to a Claude Opus sub-agent (read-only, `drift.md` +D-1 records the lane override) — report pending. + +## 2026-08-12 — #1398 research returned early (budget), root cause found + +The delegated research sub-agent was **stopped on token budget** mid-pass and asked for concise +findings rather than killed, so its evidence survived. Report: `slices/research-1398.md`. + +It found the root cause and, importantly, found that **the repo already records it**: the two Flow-B +OTEL gates are deferred against #1398 with the reason "workers-combined does not install the stream +mutation hook" (`packages/cli/e2e/suites/scaffold/capability-suites.ts:23-34`). The workers API +service installs the hook (`plugins/workers/services/src/main.ts:67`); the background entrypoints +that generated projects actually run never do (`plugins/workers/bin/runtime.ts:89-152`). + +The report's honest **unverified list** is carried into `plan.md` as blocking slice S0 rather than +being smoothed over — the first item (does `workers-combined` actually receive the streams env) +decides whether the fix is a hook installation or something larger. + +**Orchestrator-verified fact** (not delegated, checked in-session, because the whole plan turns on +it): `job-dispatcher.ts:44` derives `parentContext` from the stored trace headers and passes it to +`traceJobExecution` at `:108`, so `job.execute` shares its trace id with the record's stored +`traceparent`; and `instrumentation.ts:160` starts the publish span on the **ambient** context. That +pair is what lets every published execution record join the `job.execute` trace, including the +pre-span `create()` record — which is the sharp edge the research flagged. + +## 2026-08-12 — #1405 slice dispatched (Tier-D, mobile-visible) + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-1405` (fresh leaf) | +| Branch | `fix/1405-durable-producer-rejection-taxonomy` @ `01aa12b67`, **no upstream by design** | +| Thread id | `019ff4f0-0e1c-7333-8138-bbb107e4f1b3` | +| Rollout | `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T09-46-40-019ff4f0-0e1c-7333-8138-bbb107e4f1b3.jsonl` | +| Requested route | openai · gpt-5.6-sol · low | +| Observed route | openai · gpt-5.6-sol · low — **verdict: matched** | +| Runtime | approval=never · sandbox=dangerFullAccess | +| Steering | `codex exec resume 019ff4f0-0e1c-7333-8138-bbb107e4f1b3 -- "<follow-up>"` | +| Brief | `slices/implement-1405.md` (staged to `/home/codex/ns006-1405-brief.md`) | + +Launched through `deno task agentic:launch-codex-slice` after a clean `--dry-run`; the first dry-run +**failed git-safety** because the leaf branch tracked `origin/main`, which was cleared before launch +(push is explicit-refspec only). Watching with `agentic:codex-watch --mode turn` on that thread — +event-driven, not polled. + +The brief locks both reason decisions (reuse `producer-stopping`; add exactly one new member +`transport-refused`), names the gates as deliverables including the mandatory `quality:gate`, +pre-empts the known `deno fmt` rewrap hazard, and forbids the agent from merging or flipping to +ready. + +## 2026-08-12 — #1405 slice landed on its branch; Tier-A review done + +Codex thread `019ff4f0-0e1c-7333-8138-bbb107e4f1b3` completed its turn (detected by +`agentic:codex-watch --mode turn`, not by polling). Commit `c491c6989`, pushed by explicit refspec, +**draft PR #1528** open against `main` carrying `Closes #1405`, a fenced `acceptance-evidence` block +mapping all five boxes, and a DoD whose last two boxes (IMPL-EVAL, orchestrator review) are +correctly left unticked. + +**Independent re-verification (not read off the slice report):** + +| Check | Command | Result | +| --- | --- | --- | +| Package suite | `deno task --cwd packages/plugin-streams-core test` | **33 passed, 0 failed** | +| Negative case | both fixes reverted, suite re-run | **29 passed, 5 failed** — the guards fire | +| Restore | `git checkout` + re-run | 33/33, clean tree | + +The negative-case run is the `milestone-run.md` gate-integrity rule applied: a guard enters only +with its predicate demonstrated. 0.0.4 shipped two guards that could never fire and looked correct; +this one was checked rather than assumed. + +**Diff conforms to the locked decisions.** `transport-refused` is the only public-surface addition; +close-drain reuses `producer-stopping`; `#failActive` branches on `isRetryable`; the conflated +connect guard is split. The façade's duplicate `stateRejection()` was **deleted** and both write +entry points delegate to `supervisor.writeRejectionReason()` — so the two selectors that drifted +apart to produce this defect can no longer drift again. + +**Two advisory findings, neither blocking** (posted to #1528): the reverted-state failure arrives as +an uncaught-error cascade rather than four individual assertion failures; and `arch:check` carries +`WARN=1` because the supervisor file is now 515 lines against a 500-line advisory cap (`FAIL=0`). + +**A defect in my own brief, surfaced by the implementer.** The brief's gate list named +`deno test packages/plugin-streams-core`, which exits 1 with 19 `NotCapable` failures for want of +`--allow-env`. The slice reported the red with its cause instead of hiding it and used the +package-declared task. The brief was wrong; corrected before the #1398 brief goes out. + +**IMPL-EVAL dispatched** — separate session, separate worktree `/home/codex/repos/ns006-1405-impleval` +at `c491c6989`, DeepSeek V4 Flash 0731 max (small-impl evaluator lane, local fallback transport per +D-2). Its prompt (`slices/impl-eval-1405-prompt.md`) directs it at the highest-value failure mode — +whether a genuinely failed producer can now be masked as merely closing, which would be a worse +defect than the one being fixed — and requires it to revert each fix **individually** to prove each +test fails for its own reason. + +## 2026-08-12 — #1398 S0 resolved (and an orchestrator inference corrected) + +The plan's blocking precondition — does `workers-combined` actually receive the streams URL — is +**answered: yes**, so S0 stops being a blocking unknown and becomes a runtime confirmation in S3. + +Evidence: `generate-register-background.ts:200-218` emits `services__<ref>__http__0` for every +background-processor `PluginReferences` entry, and +`packages/cli/src/public/features/plugins/install/install-plugin_test.ts:1393-1396` asserts +`BackgroundProcessors.workers.PluginReferences === ['streams', 'workers-api']`. + +**Correction.** My first read was `plugins/workers/src/aspire/workers-contribution.ts:55-63`, where +`addDenoBackground` declares no `streams` reference and only `builder.waitFor(combined, api)` — from +which I concluded the env was missing and the plan needed an extra Aspire slice. That inference was +wrong. `PluginReferences` is not derived from the contribution file; it is reconciled from the plugin +manifest's `.withDependencies({ streams: streamsPlugin })` (`plugins/workers/src/public/mod.ts:61`). +Verifying the mechanism rather than stopping at the first plausible file is what kept an unnecessary +slice out of the plan. Recorded here rather than silently amended, because the wrong version was +briefly the basis for a scope judgement. + +Residual for S3 only: the generated wiring guards with `if (<ref>Endpoint)`, so the env is silently +omitted if the streams resource exposes no `http` endpoint at wiring time. Runtime observation, not +a design unknown. + +**Evaluator sessions in flight** (both real agentic turns, not nominal): PLAN-EVAL #1398 on MiniMax +M3 high (208 stream events at 07:59Z) and IMPL-EVAL #1405 on DeepSeek V4 Flash 0731 max (1012 +events). Preset validation ran first — `agentic:provider-canary --all` reported all six presets +`passed`, with `claude-evaluator-minimax-m3` and `claude-evaluator-deepseek-v4-flash-0731` both +`liveEligible: true`, `agenticTurn: supported`. + +## 2026-08-12 — #1398 PLAN-EVAL PASS, plan amended, slice dispatched + +**PLAN-EVAL verdict: PASS** (MiniMax M3 high, fresh session, worktree +`/home/codex/repos/ns006-1398-planeval` @ `01aeafbfa`, 562,947 ms, `is_error: False`, 215 stream +events — a real agentic turn, not a nominal one). Verbatim verdict in `plan-eval.md`; a leaked +one-line preamble before the verdict token was stripped and that edit is disclosed in the file. + +It confirmed the causal chain one hop deeper than I had traced it — through `withSpan` into +`packages/telemetry/src/application/span.ts:38-43`, where `parentContext` is passed as the third +argument to `tracer.startSpan` — and confirmed TC-14 compares **trace id only** +(`select-flow-b-stream-change.ts:131-153`) and that the selector returns the **first** +`correlationId` match (`:96-105`). It also answered S0 `yes` by a **different route** than I did +(`plugin-reference-reconciler.ts:70-91` rather than `install-plugin_test.ts`), so that precondition +is now confirmed twice from two directions. + +**Two findings folded into the plan after I verified each myself:** + +- **F1** — the join mechanism was implicit and would have failed silently. + `StreamsTracerPort.startSpan` (`instrumentation.ts:92-102`) takes **no** parent-context argument, + so only an explicit `context.with(extractContext({traceparent, tracestate}), …)` wrapper around + `producer.upsert` makes the publish span inherit the job trace. `createStreamMutationHook` + (`plugin-workers-core/src/streams/producer.ts:108-118`) does no wrapping today. D3 now says so. +- **F2** — the plan named one of the **two** tests that pin the deferral. The second, + `suite-registry_test.ts:209-234`, asserts `SCAFFOLD_RUNTIME_DEFERRED_GATES` equals the exact + two-entry list and that neither runtime tier executes a deferred gate. I read both tests directly + to confirm. Emptying the constant without rewriting that test leaves S2 red. D5 now names both. + +Neither was blocking; both earned the pass. + +**Slice dispatched (Tier-D, mobile-visible).** + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-1398` (fresh leaf, upstream cleared before launch) | +| Branch | `fix/1398-publish-job-executions-to-durable-stream` @ `01aa12b67` | +| Thread id | `019ff4ff-a633-7062-ae9c-21930930b5d6` | +| Rollout | `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T10-03-42-019ff4ff-a633-7062-ae9c-21930930b5d6.jsonl` | +| Requested / observed route | openai · gpt-5.6-sol · **medium** — **verdict: matched** | +| Steering | `codex exec resume 019ff4ff-a633-7062-ae9c-21930930b5d6 -- "<follow-up>"` | +| Brief | `slices/implement-1398.md` | + +The brief carries F1 and F2 as locked requirements, names the `create()` trap explicitly (install the +hook without D3's wrapping and it looks correct, then fails TC-14 on the first matched record), +corrects the `--allow-env` gate-command defect from the previous brief instead of repeating it, +fences the out-of-scope items (#1405's surface, the undeclared imports, any schema change), and +requires the expensive `scaffold.runtime` gate to be confirmed un-contended before it runs. + +**Note on tooling:** `slices/codex-thread-ids.md` is written per slice-dir and was **overwritten** by +the second launch, so it now shows only the #1398 thread. The #1405 identity is preserved in this +worklog above; nothing was lost, but the file is not an accumulating registry. + +## 2026-08-12 — #1405 MERGED (`8ff1bcb8f`) + +IMPL-EVAL returned **PASS** (DeepSeek V4 Flash 0731 max, fresh session, 642,836 ms, 5,226 events). +Its per-fix revert isolation is the substantive result: reverting only the `#closing` change fails +only the close-drain test; reverting only the `#failActive` change fails only the refusal tests. +My earlier both-at-once revert proved the tests fire but **not** that each guards its own defect — +that distinction is what acceptance box 4 actually asks for. + +**One evaluator suggestion declined, with reasons on the PR.** It proposed dropping the unreachable +`?? 'producer-failed'` fallback (`create-durable-stream.ts:132,160`). The unreachability is correct, +but the cleanup does not typecheck: `writeRejectionReason()` returns +`StreamWriteRejectionReasonV1 | undefined`, so removing the `??` needs a non-null assertion — which +the slice brief forbids — or a wider refactor. A total, type-safe expression is the better trade. +Declining with a reason satisfies the review-thread gate; silently ignoring it would not. + +**The draft-CI trap fired exactly as the profile predicts.** Every check on #1528 read `skipping` +while it was a draft. Under check 4 that is *unproven*, not clean — the #778/#775 failure mode where +"clean" meant "nothing ran". Flipping to ready produced real runs, and the named expensive gates then +reported genuine `SUCCESS`: `scaffold-runtime (aspire + docker + postgres)`, +`scaffold-runtime-sqlite`, `scaffold-static`, `code-quality`, `quality`, `check-test`, +`surface-diff`, `deps-report`, `close-gate`. Terminal after 440 s. + +The evidence mirror was **pre-flighted with `--dry-run` before labeling** and reported it would skip +without `status:ready-merge`; applying the label let it tick all five #1405 acceptance boxes from the +PR's fenced `acceptance-evidence` block rather than by hand. + +**Pre-merge gate:** `slices/pre-merge-gate-1528.md`, all seven checks PASS with named sources, plus +`agentic:review-threads` → `PASS threads=0 unanswered=0`. Merged squash → `8ff1bcb8f`; #1405 +auto-closed `COMPLETED` by the body's closing keyword; `status:shipped` on both. + +**Caveat carried into the merge record rather than dropped:** `quality:gate`'s configured roots omit +`packages/plugin-streams-core`, so this package's quality verdict rests on the explicit target scan +(`findings=[]`, `allowCount=0`), not on the repo gate. CI `code-quality` passing is not by itself +proof for this package. Repo gate-coverage gap, not a defect in the change — candidate for a +follow-up issue at lane close. + +## 2026-08-12 — #1398 slice landed on branch; live gate red **twice**, no verdict yet + +Codex thread `019ff4ff-a633-7062-ae9c-21930930b5d6` completed. Three commits, draft PR **#1536** +against `main`. Diff conforms to every locked decision — D1 (hook on worker + combined, correctly not +scheduler), D2, **D3** (`withContext(extractContext({traceparent, tracestate}), …)` — the PLAN-EVAL +F1 trap, implemented), D4 (`streams/schema.ts` untouched, 0 changes), D5 (both named tests). + +**A third stale deferral pin, found by neither the plan nor PLAN-EVAL.** The plan named +`suite-registry_test.ts:204-215`; PLAN-EVAL F2 found `:209-234`; the implementer found a **third** in +`suite-runner_test.ts` still expecting two deferred skipped steps, and fixed it in the same commit — +by running the full package tests rather than only the tests it was pointed at. Each layer caught +what the one above missed. It also hit a genuine type trap: an empty +`as const satisfies readonly DeferredGate[]` infers its element as `never`, breaking fixture code +that reads `issue`/`reason`; fixed by declaring against the explicit contract without relaxing an +assertion. + +### The live gate has not produced a verdict — two runs, two different pre-gate failures + +| Run | Where it died | Result | Reached the restored gates? | +| --- | --- | --- | --- | +| 1 (slice) | `runtime.flow-b-fixture` — `netscript generate plugins failed: fetch failed` | `passed=33 failed=1`, exit 1 | **no** | +| 2 (orchestrator, branch updated to current `main`) | `runtime.wait.triggers-api` — timed out unhealthy after 120 s | `passed=50 failed=1`, exit 1 | **no** | + +Run 2 got substantially further: the run-1 fetch failure did **not** reproduce, and fixture +generation, Aspire start, database init/migrate/generate/seed, AppHost restart, and health waits for +postgres, garnet, workers-api, **workers**, sagas-api and sagas all passed. It then timed out on +`triggers-api`. + +**Neither run gives `behavior.otel.stream-consumer` or `behavior.otel.traces` a live verdict, so +#1398's acceptance criterion 3 remains unproven and the PR cannot merge.** The slice said this +plainly about run 1 and did not retry to manufacture a green; that was correct. + +**What I can and cannot say about run 2.** #1398 touches workers and the CLI e2e suite definitions — +not triggers. `runtime.wait.workers` passed while `runtime.wait.triggers-api` timed out. The Aspire +AppHost log shows no triggers error, only an unrelated dev-certificate trust warning, and an issue +search found no known `triggers-api` health defect. That is **consistent with** an environmental +failure but does **not** prove one — I have not reproduced the suite on a clean `main` checkout, so I +cannot state as fact that the change is uninvolved. + +**Serialisation was verified before run 2**, per the expensive-gate rule: no +`/tmp/netscript-e2e-scaffold-runtime.lease`, no competing `e2e:cli` process, zero Docker containers. +`agentic:leak-check` after run 1 reported no survivors. + +### Escalating the verdict to CI rather than burning more local runs + +Two local failures at two unrelated points, neither reaching the target gates. CI ran +`scaffold-runtime (aspire + docker + postgres)` **green** on PR #1528 (main + #1405) an hour earlier, +so the CI environment starts this suite cleanly. #1398 adds both OTEL gates to `RUNTIME_GATES`, so +CI's own scaffold-runtime job now exercises them — the same gate, in an environment that is currently +working. + +PR #1536 flipped to ready to trigger the blocking tier, held at `status:impl-eval` (**not** +`ready-merge`) so it cannot be mistaken for merge-ready. IMPL-EVAL is still required — the D-3 owner +ruling waives evaluation only for the small deterministic class, explicitly not for this issue. + +## 2026-08-12 — #1398 live gates PASS in CI, both tiers + +The two gates deferred against #1398 ran **by name** and passed in CI, verified from job logs rather +than inferred from a suite-level green: + +| Tier | Job | Gates observed | Summary | +| --- | --- | --- | --- | +| `scaffold-runtime (aspire + docker + postgres)` | `94062070840` | `behavior.otel.stream-consumer`, `behavior.otel.traces` | `passed=88 failed=0 skipped=0` | +| `scaffold-runtime-sqlite (aspire + sqlite + garnet)` | `94062070984` | both, same | `passed=83 failed=0 skipped=0` | + +`skipped=0` matters as much as `failed=0`: a silently skipped gate is the exact false-green the +deferral list existed to prevent. Confirmed the SQLite tier really carries them — +`POSTGRES_ONLY_RUNTIME_GATES` (`capability-suites.ts:146-151`) lists only the four DB-specific gates, +so `RUNTIME_SQLITE_GATES` inherits both OTEL gates. Both tiers is stronger than the plan required. + +**This retires the open question about the two red local runs.** I had recorded that the +`triggers-api` timeout was *consistent with* an environmental failure but not proven to be one, since +I had not reproduced the suite on a clean `main`. CI now runs the same suite **with this change** +through `runtime.flow-b-fixture` and past `triggers-api` to a clean finish on both tiers. That is the +missing control: the failures were local-host environmental, not caused by the change. Upgraded from +"consistent with" to "established", with the evidence named. + +**#1398 acceptance criterion 3 is now live-verified** — a subscription opened before the trigger +observes the execution record within a bounded time, proven by the gate's own live SSE loop +(`consume-flow-b-stream.ts:203-230`), not by asserted timing. + +**CI state:** every check green except `close-gate`, which is correctly red — DoD box 5 unticked and +#1398's acceptance boxes unmirrored. Both are now truthfully tickable and will be handled when the +merge gate is finished. + +**Held, deliberately.** Per D-4 the PR stays on head `e4319c685` at `status:impl-eval`. I have made +no label change, no body edit, no OpenHands trigger, and no local evaluator launch. IMPL-EVAL will +arrive via the automatic dispatcher after #1524 lands; root re-enters the label to fire it. This +lane's remaining work is to watch that verdict, then run and record the seven-check pre-merge gate. + +## 2026-08-12 — deferred findings filed as their own issues (scope-drift checkpoint) + +Two findings were fenced out of this lane's PRs on purpose. Filing them rather than letting them die +in a run artifact, per the profile's rule that scope drift is an explicit checkpoint and that +findings raised inside a run get filed from inside the run: + +| Issue | Title | Milestone | Origin | +| --- | --- | --- | --- | +| **#1542** | `quality:gate` roots omit published packages, so a green gate is not proof they were scanned | 0.0.7 | found **independently three times**: #1405 implementer, #1405 IMPL-EVAL (against `deno.json:156`), #1398 implementer | +| **#1543** | `plugin-workers-core` and `plugins/triggers` import `plugin-streams-core` without declaring it | 0.0.7 | #1398 research + PLAN-EVAL, recorded as plan risk R3 | + +**#1542 is the more serious of the two** and is the reason the #1405 merge record says explicitly +that this package's quality verdict rests on the explicit target scan and **not** on the repo gate. +A `quality:gate` that silently skips a package leaves the framework-wave gate law unenforced while +appearing enforced — the same shape as pre-merge check 4 ("clean" meaning "nothing ran"), one level +down. Three independent sessions each hit it and each hand-compensated, which is exactly the pattern +that eventually gets forgotten once. + +**#1543 is filed honestly as unverified.** Whether `deno publish` actually rejects the undeclared +import was never checked; the issue says so and makes `publish:dry-run` evidence the first +acceptance box rather than asserting a defect. Both the research pass and the PLAN-EVAL recorded it +as unverified, and it is carried forward that way rather than upgraded by repetition. + +Neither issue is a 0.0.6 blocker; both are `0.0.7` with `status:triage`. + +## Waiting state — #1524 has not landed + +`gh pr view 1524` → `OPEN`, `mergedAt: null`. The automatic phase dispatcher is therefore not live, +root has not re-entered `status:impl-eval`, and no automatic verdict exists. The merge gate for +#1536 genuinely cannot proceed, and nothing has been done to it: head `e4319c685`, +`status:impl-eval`, no label change, no body edit, no evaluator, no OpenHands trigger. + +Watcher confirmed **alive** (PID 224681) rather than assumed — a dead watcher and a quiet one look +identical, and "liveness is not progress" cuts both ways. + +The merge-gate finish is staged and **dry-run against a throwaway copy of the PR body** +(`scratchpad/finish-1536.py`): it ticks the three remaining DoD boxes, replaces the now-false claims +("blocked before those gates by `runtime.flow-b-fixture` fetch failure", "Not yet live-verified") with +the CI job evidence, and rewrites the four acceptance-evidence entries to `box-index` form because +#1398's boxes wrap across lines and the mirror matches on a checkbox's first line. Every edit asserts +it matched and the script refuses to finish with an unticked DoD box or a surviving stale claim. +It is **not applied**: DoD box 6 asserts IMPL-EVAL passed, which is not true until the verdict lands. + +## 2026-08-12 — #1524 landed; watching for the automatic verdict + +`gh pr view 1524` → **MERGED** `7837ef470` at 2026-08-12T09:24:15Z. The automatic phase dispatcher is +live, so D-4's routing is now in force: phase evaluations go through the automatic status workflow +unless the owner documents a local route or skip. + +Per D-4, **root re-enters `status:impl-eval` with the Qwen override** to fire the dispatcher — that +is root's action. This lane does not touch the label, the head, or the body, and does not launch a +local evaluator or an OpenHands comment. #1536 verified unchanged at the moment #1524 landed: head +`e4319c685`, labels `type:fix, status:impl-eval, area:plugins, area:telemetry`, 5 comments. + +Now watching #1536 for the automatic verdict — a new comment carrying an `IMPL-EVAL`/`VERDICT` +marker, and any label transition — from a recorded baseline of 5 comments and the exact current +label set, so a change is detected as a change rather than inferred. + +On a PASS the merge gate finishes in this order: apply the staged body transform (already dry-run +against a copy), move to `status:ready-merge` so the mirror populates #1398's four issue boxes from +the `box-index` entries, confirm `close-gate` flips green, run and record the seven-check pre-merge +gate plus `review-threads`, merge, then append the row to `cut-trace.md` captured from +`git log origin/main` **after** the merge. + +On findings, none of that runs; the findings go to the slice thread for a fix cycle and the eval loop +limit of two failures applies before escalation. + +## 2026-08-12 — label re-entry executed; dispatcher structurally cannot fire on this head + +Executed the owner's instruction exactly: applied the one-shot `eval:model:qwen`, moved away from +`status:impl-eval`, then re-added it. Timeline confirms a clean sequence with exactly one `status:` +label at every point and **no head change**: + +| UTC | Event | +| --- | --- | +| 09:39:17Z | `eval:model:qwen` labeled; `status:impl` labeled; `status:impl-eval` unlabeled (move away) | +| 09:39:25Z | `status:impl-eval` labeled; `status:impl` unlabeled (**the trigger**) | + +`impl-eval:skip` is not present, so nothing suppressed the eval. + +**It did not run, and it cannot run on this head.** Three workflow runs fired around the label +changes — all of them `openhands-agent.yml` ("OpenHands Agent"), all `completed/skipped`, job `agent` +skipped with zero steps. The new dispatcher, `openhands-phase-eval.yml` ("OpenHands phase +evaluation"), produced **no run on this branch at all**, while it has runs on other branches the same +morning. + +Root cause, verified rather than inferred: + +``` +git cat-file -e e4319c685:.github/workflows/openhands-phase-eval.yml -> ABSENT +``` + +For `pull_request` events GitHub resolves workflows from the PR's merge ref, so a workflow absent +from the PR head does not exist for that PR. The branch's last sync with `main` is `e4319c685` at +**2026-08-12T08:45:46Z**; #1524 merged at **09:24:15Z** — 38 minutes later. No amount of label cycling +can trigger a workflow the PR does not contain. + +`main` has since moved again: `281ab7688 fix(agentic): use chainable token for eval statuses (#1547)`, +which addresses the dispatcher's own `PAT_TOKEN` requirement — the "Require chainable trigger token" +step that hard-fails when the secret is absent. So the current head is missing both the dispatcher +and its follow-up fix. + +**Not actioned unilaterally.** The only path is to sync the branch with current `main`, which +**changes the head SHA** — directly against the standing instruction to keep #1536 on its current +head — and invalidates the present green CI, forcing a full re-run including both expensive +`scaffold-runtime` tiers. That is a material, owner-visible trade, so it is being raised rather than +taken. The prior green evidence (both OTEL gates passing by name on both tiers) would need to be +re-established against the new head, which is arguably better evidence anyway since it would match +what actually merges. + +## 2026-08-12 — branch synced (owner-approved head change); dispatcher fired; Qwen evaluator running + +Owner selected "sync branch with main, then re-enter", accepting the head change and the full CI +re-run. Executed: + +| Step | Result | +| --- | --- | +| `git merge origin/main` into the slice branch | new head **`f7d503fee`** | +| Dispatcher present in head? | `git cat-file -e HEAD:.github/workflows/openhands-phase-eval.yml` → **PRESENT** (was ABSENT at `e4319c685`) | +| #1547 token fix included? | yes — `281ab7688 fix(agentic): use chainable token for eval statuses` | +| #1398 changes survived the merge? | verified: `withContext` ×2, `setMutationHook(createStreamMutationHook())` ×2, deferral list still `= []`, `streams/schema.ts` still 0 changes, 7 non-run-artifact files unchanged in set | +| Re-entered `status:impl-eval` | move away → re-add, exactly one `status:` at each point | + +**The dispatcher then fired and succeeded**: `OpenHands phase evaluation` at 09:42:56Z → +`completed/success` (the 09:42:49Z run correctly skipped — that was the move-away event). Contrast +with the previous attempt on `e4319c685`, where this workflow produced **no run at all**. That is the +control proving the diagnosis: same labels, same sequence, different head — the only variable was +whether the workflow existed in the merge ref. + +**Automatic evaluator is running on the requested override:** `openrouter/qwen/qwen3.8-max`, +provider `OPENROUTER`, output `pr-comment`, run `31584188459`. No manual OpenHands dispatch was made; +the trigger was the label pair alone, per D-5. + +### Watcher corrected — a counting watcher would never have fired + +The dispatcher's comment carries `<!-- openhands-agent-summary -->` and +`<!-- openhands-run: {…,"conclusion":"running"} -->`, i.e. OpenHands **updates that comment in place** +rather than posting a new one on completion. The original watcher keyed on *comment count* and would +have polled to timeout while the verdict sat in an edited comment — the same silence-looks-like- +waiting failure that already bit this lane twice today (draft CI `skipping`, and the dispatcher not +existing on the old head). + +That watcher was stopped and replaced with **one** watcher keyed on the run's terminal status and the +comment's `conclusion` marker. Still exactly one active watch, as instructed — corrected for how the +tool actually reports, not left running on a false assumption. + +CI is also re-running against `f7d503fee`; the previously proven OTEL gate evidence must be +re-established on this head before merge, and the earlier job ids no longer describe what would land. + +## 2026-08-12 — CI evidence re-established on the merging head `f7d503fee` + +The pre-sync evidence was **not** carried forward. Both OTEL gates were re-read by name from the new +head's job logs: + +| Tier | Job (head `f7d503fee`) | Gates observed | Summary | +| --- | --- | --- | --- | +| `scaffold-runtime (aspire + docker + postgres)` | `94073971396` | `behavior.otel.stream-consumer`, `behavior.otel.traces` | `passed=88 failed=0 skipped=0` | +| `scaffold-runtime-sqlite (aspire + sqlite + garnet)` | `94073971501` | both, same | `passed=83 failed=0 skipped=0` | + +**These are different job ids** from the pre-sync run (`94062070840` / `94062070984`), which is the +concrete proof that reusing the earlier verification would have cited evidence for a head no longer +on the PR. The staged body transform now takes head SHA and both job ids as **required arguments** +and asserts that none of `94062070840`, `94062070984`, or `e4319c685` survives anywhere in the +resulting body — it fails loudly rather than silently carrying a stale citation. + +Full CI on `f7d503fee`: terminal after 300 s, every check green except `close-gate` (expected — DoD +box 5 and the issue's acceptance boxes are not yet updated). `agent` ×2 and `code-quality-repo` +report `skipping`; none is in the named-gate set for this change, and `code-quality` itself passed. + +The `dispatch` job also shows `pass` — that is the phase-eval dispatcher, present and running on this +head, which it was not on the previous one. + +**Qwen evaluator still `in_progress`** (run `31584188459`, summary comment marker still +`"conclusion":"running"`). Nothing is applied to the PR body or labels until that verdict lands. + +## 2026-08-12 — #1398 MERGED (`d7e2b67b2`) — lane complete + +**IMPL-EVAL: PASS** — `OPENHANDS_VERDICT: PASS`, automatic phase dispatcher, +`openrouter/qwen/qwen3.8-max`, run `31584188459`, evaluated at head `f7d503fee` against trusted base +`281ab76887`. Triggered by the label pair alone; no manual OpenHands dispatch and no local evaluator, +per D-5. + +It independently traced the join mechanism end-to-end — hook → `producer.upsert` → +`DurableStreamProducer#startPublish` → `instrumentation.startPublish` (`tracer.startSpan` on ambient +context) — which is precisely the link no unit test in this slice could settle and which would have +failed silently if wrong. Its three advisories were all pre-existing and already tracked (#1542, +#1543) or resolved by the merge sequence itself. + +### Finishing the gate exposed two real problems, both caught rather than merged past + +1. **A second `status:` label.** Automation applied `status:augment-review` one second after I moved + off `status:impl-eval`, so the PR briefly carried two `status:` labels against the + exactly-one-status rule. Removed to restore the invariant before merging. +2. **A stale close-gate result.** `gh pr checks` reported `close-gate` red, but that job ran at + 09:42:44Z — before `status:ready-merge` (10:22:50Z) and before the body update. Reading the job + log rather than the summary showed the mirror's own notice: *"Mirror skipped because live PR + labels do not include status:ready-merge"*. The red was genuine **against the body as it then + was**, listing all four unticked issue boxes and all three unticked DoD boxes by line — not a + flake to retry past. A re-run with the live labels turned it green and the mirror ticked all four + #1398 boxes from the `box-index` entries. + +**Stale evidence was actively prevented, not merely avoided.** The head changed mid-flight +(`e4319c685` → `f7d503fee`), so both OTEL gates were re-read by name from the new head's logs — +postgres `94073971396` (`passed=88 failed=0 skipped=0`), sqlite `94073971501` +(`passed=83 failed=0 skipped=0`). These job ids differ from the pre-sync pair, which is the concrete +proof the earlier citation would have described a head no longer on the PR. The body transform takes +head and job ids as required arguments and asserts no pre-sync reference survives. + +**Pre-merge gate:** `slices/pre-merge-gate-1536.md`, all seven checks PASS with named sources, plus +`review-threads` → `PASS threads=0 unanswered=0`. Merged squash → `d7e2b67b2`; #1398 auto-closed +`COMPLETED`; `status:shipped` on both. + +**Both owned issues are now on `main`.** #1405 → `8ff1bcb8f`; #1398 → `d7e2b67b2`. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 5a48c970df..7e5802e7d9 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,6 +8,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | +| 2 | 2026-08-12 ~12:24Z | `e67c1ba13` | **#1560** | **#1530** (auto-closed `COMPLETED`) | 7/7 pass — `pr-checks` **23 checks, 0 current failures** at `28fc1b423`; `close-gate` green after label + workflow **re-run** (not a push); boxes 1–6 mirrored, box 7 `[post-merge]` verified after merge and ticked with evidence; `review-threads` PASS. **Two IMPL-EVAL verdicts:** `PASS` at `49e2b86e9` (pre barrel fix) then **`FAIL_FIX`** at `9ab361440` on a real close-gate defect, then `PASS` at `28fc1b423`. Consuming the first would have merged a red gate. | | 1 | 2026-08-12 ~08:31Z | `63cd1cd58` | **#1527** | **#1436**, **#1415** (both auto-closed `COMPLETED`) | 7/7 pass — full record in the PR's `[PRE-MERGE GATE]` comment. `pr-checks` **15/15 `current-pass`, 0 current failures** at `dfda54a16`; `close-gate` green; #1415 4/4 boxes mirrored with linked evidence; #1436 has 0 boxes so the PR body is its record; no new ignores/casts, no lock churn; both probes re-run independently by the orchestrator; `review-threads` PASS (0 threads). | ## Wave clustering as dispatched @@ -15,7 +16,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | Wave | PR | Issues | Lane | Status | | --- | --- | --- | --- | --- | | 1 | PR-A `fix/1436-1415-close-gate-trust` → **PR #1527** | #1436, #1415 | Sol · low | **MERGED** `63cd1cd58` — thread `019ff4f4-1fce-7253-a7e0-d718c65b39cc`, worktree `/home/codex/repos/ns006-gatetrust`, 7 commits | -| 2 | PR-E `fix/1530-type-fixture-scan-scope` | #1530 | Sol · low | not yet dispatched (inserted; gates PR-D per rail R-1) | +| 2 | PR-E `fix/1530-type-fixture-scan-scope` → **PR #1560** | #1530 | Sol · low | **MERGED** `e67c1ba13` — thread `019ff5b2-7d02-…`, worktree `/home/codex/repos/ns006-typefixtures`, 6 commits. Restored `main`'s blocking `code-quality-repo` job: first green in **nine** consecutive push-to-main runs. | | 2 | PR-B `fix/1403-quality-gate-coverage` | #1403 | Sol · low | not yet dispatched | | 2 | PR-C `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | not yet dispatched | | 2 | PR-D `fix/1378-quality-scan-rule-power` | #1378 | Sol · high | not yet dispatched | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index b91df5cfe7..64b8676fb9 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -974,3 +974,26 @@ packages/sdk/src/{desktop/mod.ts,query-client/create-service-query-utils.ts} answered it. The peer ran the range. This lane has now made that mistake twice — asserting an absence without the probe that finds the presence (the sagas supersession row) and asserting a mechanism without the probe that identifies it (here). Both were caught by someone else executing the command I should have. + +## D-30 — PR-E merged; `main`'s blocking quality gate green for the first time in nine pushes + +- **Recorded:** 2026-08-12 +- **Merge:** PR #1560 → `e67c1ba13`. #1530 auto-closed `COMPLETED`, `status:shipped`. All **seven** boxes + truthfully ticked — 1–6 mirrored from structured evidence, box 7 verified **after** merge from the run it + describes and ticked then, which is what `[post-merge]` exists for. +- **The headline result:** `code-quality-repo` on `main` at `e67c1ba13` is **success**, the first green in + **nine** consecutive push-to-main runs (streak from `b3dc006e8`). Independently reproduced at merged main: + `deno task quality:scan:repo` → exit 0. + +```text +completed/success e67c1ba13 12:23:53Z <- this merge +completed/failure d558f9ab2 12:21:08Z +completed/failure 59e435c5d 12:03:58Z +``` + +- **#1537 landed** at `d558f9ab2` — the docs lane's fenced-TS extractor. So **rail `R-10`'s primary path is + available**: PR-D consumes that extractor rather than taking the fallback, and #1549's docs-fence box stays + in 0.0.6 rather than moving. The stated fallback was never needed, which is the outcome that having stated + it in advance bought. +- **Lane status:** #1436, #1415, #1530 closed. Remaining in 0.0.6: **#1403** (PR-B, next), **#1380** (PR-C), + **#1549** (PR-D). #1378 and #1545 in 0.0.7. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/context-pack.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/context-pack.md new file mode 100644 index 0000000000..7d1a36479f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/context-pack.md @@ -0,0 +1,22 @@ +# PR-A context pack + +## Current state + +S1 adds executable RED contracts for #1436 and #1415 on top of base `01aa12b67`. The focused test +command exits 1 with six missing-classification-contract type errors. No production predicate has +changed yet. PR #1527 is the sole draft PR; the orchestrator retains merge authority. + +## Locked contracts + +- Closing keywords reject only preceding word characters or hyphens; punctuation remains valid. +- Regex-derived PR references are excluded visibly; failed classification remains gated visibly. +- Not-yet-done evidence is rejected only when it would newly tick an unchecked acceptance box. +- Work stays inside `.llm/tools/validation/**` and this slice directory. + +## Next + +Implementation is complete through S4: S1 `a927790eb`, S2 `4ca4cc421`, S3 `0329acaf8`, S4 +`c095303c8`. S5 has green scoped check/lint/fmt, green probe and live mirror dry-run, plus a green +48-test amended suite. The original command's missing-permission failure and the accepted narrow +predicate limit are recorded in `drift.md`. Final-head S5 evidence is ready for the PR timeline. +Next: orchestrator review. Do not merge, leave draft, and do not apply `status:ready-merge`. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/drift.md new file mode 100644 index 0000000000..b10a991eb5 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/drift.md @@ -0,0 +1,44 @@ +# PR-A drift log + +## 2026-08-12 + +- No implementation drift. The assigned correction that #1436's prescribed extra word boundary is + a no-op is treated as the locked contract: `(?<![\w-])` replaces the leading `\b`. +- Bootstrap commit identity differs from the stale short hash in the already-written PR body + (`c2d8a8e4b` live versus `32beb395e` recorded). This is PR metadata drift only; do not rewrite + history. Correct the body during S5. +- Gate 1's prescribed command omits `--allow-write`, but nine unrelated existing validation tests + call `Deno.makeTempDir()`. Exact command verdict: exit 1, 39 passed / 9 permission failures. This + slice will not edit unrelated tests or claim that verdict green. A supplementary run first showed + `--allow-run` was also required by the existing Fresh UI fixture; with both missing permissions, + all 48 tests pass. The orchestrator amended Gate 1 to include both permissions and accepted the + resulting final-head green verdict. +- Known accepted limit: `Will be run after merge` is accepted because the locked #1415 predicate is + deliberately leading-token and the evidence begins with `Will be`, not `will run`. This is the + narrowness mandated by #1415 (reject unearned leading assertions without policing broader prose), + not an implementation defect. Do not widen it without a new contract. + +## Orchestrator finding — `status:ready-merge` alone does not trigger the acceptance mirror + +Recorded 2026-08-12 by the lane orchestrator while taking this PR through the pre-merge gate. + +`.agents/skills/netscript-pr` states that "applying `status:ready-merge` itself triggers a fresh run +(the workflow listens to `labeled`)", and `check-close-gate.ts`'s own repair hint says "apply the label +and the labeled event triggers a fresh run". Both are **false against the current workflows**: + +```text +.github/workflows/ci.yml:41 types: [opened, synchronize, reopened, ready_for_review] +.github/workflows/e2e-cli.yml types: [opened, synchronize, reopened, ready_for_review] +``` + +`labeled` is not in either list, so no run is created by the label event. Observed here: the label was +applied at ~08:14Z, no new run appeared, `close-gate` stayed red on its pre-label result, and #1415's +four acceptance boxes remained unticked (`0` ticked at 08:19Z). + +The operational rule that actually works is **label first, then push** — the push fires `synchronize`, +and because every read in the gate and the mirror is live, the label is observed by that run. This +commit is that push. + +Not fixed here, deliberately: adding `labeled` to `ci.yml`'s trigger types would make the documented +behaviour true, but this PR's boundary is `.llm/tools/validation/**` and widening it to workflow +surgery mid-slice is how scope leaks. Raised to the owner as a separate decision. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/worklog.md new file mode 100644 index 0000000000..2b1757288f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-a-1436-1415/worklog.md @@ -0,0 +1,127 @@ +# PR-A worklog — close-gate trust + +## Identity + +- Worktree: `/home/codex/repos/ns006-gatetrust` +- Branch: `fix/1436-1415-close-gate-trust` +- Base: `01aa12b67` +- Draft PR: #1527 +- Implementation session: Codex (this thread); merge and evaluator authority remain with the orchestrator. + +## Design + +- Public surface: preserve `extractClosingIssues` and `resolveClosingIssueReferences`; add a pure, + synchronously testable classification seam shared by the close-gate and mirror paths. +- Domain vocabulary: regex-derived closing reference classification is `issue`, `pull request`, or + `lookup failed`; authoritative GitHub closing issues bypass classification because they are issues + by construction. +- Ports: existing `GitHubClient` performs classification at each `main()` call site; no network call + enters the pure resolver. +- Constants: one narrow leading not-yet-done evidence predicate for `pending`, `todo`, `tbd`, + `will run`, `after merge`, and `not yet` after leading whitespace/bullet punctuation. +- Commit slices: S1 RED contract tests and artifacts; S2 keyword boundary; S3 PR classification; + S4 evidence assertion; S5 full gates and PR evidence. +- Deferred scope: no prose workaround, parser refactor, workflow edit, or unrelated validation cleanup. +- Contributor path: extend the focused test corpora beside each exported predicate. + +PLAN-EVAL: N/A — owner-recorded waiver for two mechanical predicates with complete contracts and +explicit gates. Separate-session review remains the orchestrator's authority. + +## S1 — RED contract fixtures + +- Added the full hyphen/word-prefix and punctuation keyword corpus. +- Added not-yet-done rejection, idempotent already-checked, and factual false-positive corpus. +- Added pure classification expectations for close-gate and mirror paths, including fail-loud lookup. +- RED command: `deno test --allow-read --allow-env acceptance-evidence_test.ts + check-close-gate_test.ts mirror-acceptance-evidence_test.ts` +- RED verdict: exit 1. Type checking reported six expected missing-contract errors: the resolver's + fourth classification argument and classification fields do not exist, and the mirror has no + `closingMirrorIssues` export. This proves the classification fixtures are red before production + changes. The existing implementation also lacks both runtime predicates exercised by the new + acceptance-evidence tests; those execute after the S3 API seam compiles. + +### Reconcile + +PR #1527 remains draft with `status:impl`, the required labels/milestone, and both closing keywords. +No scope adjustment is required. + +## S2 — keyword boundary + +- Replaced the leading word boundary with `(?<![\w-])`; trailing number boundary is unchanged. +- The focused parser corpus proves hyphen/word prefixes reject while line start and punctuation + prefixes still resolve. +- Gate: `deno test --allow-read --allow-env --filter 'closing keywords reject' + acceptance-evidence_test.ts` — exit 0, 1 passed, 10 filtered out. + +### Reconcile + +The implementation matches the corrected #1436 contract without preserving the prose workaround. + +## S3 — PR-vs-issue classification + +- Kept `resolveClosingIssueReferences` synchronous and pure; its optional classification map affects + only regex-derived references, never GitHub's authoritative issue set. +- Close-gate output retains every reference with visible classification and `excluded: yes` for PRs. +- Mirror uses the same three classifications; lookup failures stay in its issue set and are reported. +- Gate: focused close-gate + mirror tests — exit 0, 16 passed, 0 failed. + +### Reconcile + +No network call entered either pure helper. The implementation stays within the assigned files. + +## S4 — reject unearned acceptance ticks + +- Added a leading-token predicate after stripping whitespace and common dash/bullet punctuation. +- Applied it only after resolving an unchecked box, preserving already-checked historical evidence. +- Errors accumulate through the existing `errors[]` path and name issue, exact box, evidence, and + the repair: supply real evidence or leave the box unchecked. +- Proven RED at S2 head: the acceptance-evidence test command exited 1 with + `not-yet-done evidence is rejected only for a newly ticked box ... FAILED` and + `AssertionError: Expected function to throw.` +- Proven GREEN after S4: focused acceptance + mirror command exited 0, 13 passed, 0 failed; this + includes `— Pending …` rejection, normal ticking, factual non-asserting marker sentences, and all + pre-existing mirror cases. + +### Reconcile + +The predicate remains narrow: factual sentences containing marker words later in the evidence pass. + +## S5 — gate evidence + +- Superseded Gate 1 command from the original brief: exit 1, 39 passed / 9 failed. Every failure was + `NotCapable: Requires write access to <TMP>` from pre-existing tests; see `drift.md`. +- Amended Gate 1: + `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/validation/` — exit 0, + 48 passed / 0 failed at final head `9c7eba4e74`. +- Gate 2 scoped check: exit 0; 18 files, 1 batch, 0 failed batches, 0 findings. +- Gate 3 scoped lint: exit 0; 18 files, 1 batch, 0 findings. +- Gate 4 scoped format: exit 0; 18 files, 1 batch, 0 findings. +- Gate 5 baseline probe: exit 0. Whole output: + + ```text + "Exact pre-fix #1431 head" -> [] + "pre-repair #1431 head" -> [] + "Fixes #1434" -> [1434] + "hotfix #999 landed" -> [] + "prefixes #888 there" -> [] + "This is a bugfix #777" -> [] + "un-fixed #555" -> [] + "Closes #1234 and fixes #4321" -> [1234,4321] + "Refs #111" -> [] + "Part of #222" -> [] + "resolves https://github.com/rickylabs/netscript/issues/333" -> [333] + ``` + +- Gate 6 live mirror dry-run against PR #1527 at final head `9c7eba4e74`: exit 0, no changes. + Snapshots for #1436 and #1415 + were read; illustrative `fix #1431` was classified as a pull request and excluded visibly, while + both real closing issues were classified as issues and retained. Mirror skipped mutation because + `status:ready-merge` is correctly absent; the implementation agent did not apply it. +- Diff hygiene scan: no new `quality-allow:`, `deno-lint-ignore`, `@ts-ignore`, `as any`, or + `as unknown as`; no `deno.lock` change. + +### Final reconcile + +PR #1527 remains draft with exactly `status:impl`, milestone `0.0.6`, required taxonomy/CI labels, +both closing keywords, truthful checked DoD, and four exact-text #1415 evidence mappings. The +orchestrator retains separate review and merge authority. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/context-pack.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/context-pack.md new file mode 100644 index 0000000000..03834af0ff --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/context-pack.md @@ -0,0 +1,32 @@ +# PR-E #1530 Context Pack + +## Identity + +- Worktree: `/home/codex/repos/ns006-typefixtures` +- Branch: `fix/1530-type-fixture-scan-scope` +- Dispatch base: `84dd44ae7` +- Draft PR: #1560; issue: #1530 +- Implementer route: Codex gpt-5.6-sol low + +## Contract + +Exclude a file only when its normalized path contains `tests/type-fixtures/` and it ends in +`_type.ts`. Prove the exemption and all leakage controls. Do not change any fixture assertion. +Remove only the two redundant allowance comments and prove repo `allowCount` 10 → 8. + +## Current state + +E1 added the intentionally failing regression test as a standalone commit. E2 added `isTypeFixture` +and restored the quality test gate. E3 added all three leakage controls. E4 removed the two actual +SDK allowance clauses. All required gates are green; the base/head repo scan moved from exit 1, +five findings, `allowCount: 10` to exit 0, no findings, `allowCount: 8`. + +CI then exposed one missing generated-asset step. E5 refreshes the scanner's consumer-tool copy in +`packages/cli/src/kernel/assets/agent-tools.generated.ts`; inspection found no unrelated generated +drift. Quality tests, repo scan, and quality gate remain green. A post-commit generator run must +leave `git status --porcelain` empty before the E5 PR comment is posted. + +## Handoff + +Post E5's literal hash and post-commit idempotence/gate evidence on PR #1560. Do not change PR state, +labels, milestone, acceptance mappings, or issue box 7. The orchestrator retains merge authority. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/drift.md new file mode 100644 index 0000000000..7475e0d1f2 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/drift.md @@ -0,0 +1,64 @@ +# PR-E #1530 Drift + +## D-1 — standalone RED commit supersedes locked combined E1 row + +The orchestrator rail Design table groups the RED test and its exemption into one landed-green E1 +slice. The later implementation dispatch explicitly requires the failing test as its own commit. +The dispatch is followed; E2 immediately restores green. + +## D-2 — desktop-consumer allowance path is SDK, not Fresh + +The dispatch and live issue name +`packages/fresh/tests/type-fixtures/desktop-consumer_type.ts:42`, but the baseline scanner JSON and +repository search locate the stated allowance at +`packages/sdk/tests/type-fixtures/desktop-consumer_type.ts:42`. The Fresh file has no such allowance. +E4 will remove the actual SDK allowance so the required scan-owned count can fall 10 → 8. + +## D-3 — scoped format initially exposed adjacent legacy formatting + +The required scoped format wrapper initially exited 1 because the existing `escape.ts` fixture setup +in `scan-code-quality_test.ts` was not in current formatter shape. Since E1/E3 already own that file, +`deno fmt` was applied to that file only. The wrapper rerun exits 0; no other file was formatted. + +## D-4 — bundled tool edits require same-PR asset regeneration + +CI found a real coupling omitted from the issue, implementation brief, rail plan, and original gate +set: `.llm/tools/quality/scan-code-quality.ts` is embedded as source in the generated CLI consumer +tool asset. Editing a bundled `.llm/tools/` file therefore requires `deno task gen:assets-barrel` in +the same PR, followed by an idempotence check. Without it, CI's `Generated asset freshness` step +fails even when the tool's direct tests and quality gates are green. + +The canonical generator changed only `agent-tools.generated.ts`: the scanner source plus its bundle +hash. Contrary to the initial CI steer, `skills.generated.ts` contains command guidance rather than +an independent full-source copy, so it correctly remained unchanged. No unrelated generated drift +was absorbed. + +## D-4 — acceptance-evidence remapped to `box-index` after the authoritative IMPL-EVAL FAIL_FIX + +The authoritative current-head evaluator (run `31593326538`, head `9ab361440`) returned **`FAIL_FIX`** with +one blocking defect: the fenced `acceptance-evidence` block used de-backticked, continuation-merged `box:` +sentences, while `acceptanceCheckboxes` parses an issue checkbox as **raw first line only, backticks +preserved**. `validateEvidenceMapping` therefore matched **none** of the six actionable boxes, the mirror +threw, and `close-gate` was red. + +Remediated per owner direction by switching to **`box-index: 1..6`**, which `netscript-pr` documents for +exactly this case ("use its one-based `box-index` when copying a long box would be unwieldy"). Every +evidence *value* is preserved verbatim from what was already posted — nothing was re-authored at merge time. +Post-merge box 7 is deliberately unmapped and unticked. + +Validated locally before retriggering, by calling the repo's own parser rather than trusting the format: + +```text +boxes=7 actionable=6 postMerge=1 +evidence entries=6 warnings=0 +MAPPING OK — 6 boxes would be newly ticked: [1, 2, 3, 4, 5, 6] +``` + +Why `box-index` is the durable fix, not merely the working one: the exact-text form is brittle against +**how the issue body happens to be line-wrapped**, which no author controls or sees. Five of #1530's six +boxes wrap, so five of six keys were unmatchable by construction. An index is stable under rewrapping. + +Note for later slices: the `mirror --dry-run` pre-flight that `netscript-pr`'s operator playbook recommends +**cannot validate the mapping before `status:ready-merge` is applied** — it returns "Mirror skipped" at the +label check, before `validateEvidenceMapping` runs. Direct invocation of the parser is the only pre-label +validation available. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/worklog.md new file mode 100644 index 0000000000..e33b7a970d --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-e-1530/worklog.md @@ -0,0 +1,92 @@ +# PR-E #1530 Worklog + +## Design + +This implementation follows the orchestrator's locked E1–E4 sequence and the dispatch contract. +The scanner's public CLI and scan result types do not change. + +| Slice | Contract and files | Gate | +| --- | --- | --- | +| E1 | Add a RED regression fixture in `.llm/tools/quality/scan-code-quality_test.ts`. | Target test fails before the scanner change. | +| E2 | Add a named directory-and-suffix predicate in `.llm/tools/quality/scan-code-quality.ts`. | Quality tests pass; repo scan no longer reports negative type fixtures. | +| E3 | Add ordinary-source, suffix-only, and directory-only leakage controls in the scanner test. | All three controls remain findings. | +| E4 | Remove two redundant SDK fixture allowances and run the full gate set. | Repo `allowCount` falls 10 → 8; all required gates pass. | +| E5 | Refresh the generated consumer-tool asset after CI exposed the scanner's embedded-source coupling. | Generator diff contains only the embedded scanner plus bundle hash; regeneration is idempotent; quality gates stay green. | + +Deferred: export awareness, allowance issue links, `--max-allow` task wiring, docs-fence scanning, +workflow changes, and scaffold/runtime E2E. No package/plugin public surface changes. + +PLAN-EVAL: PASS in the orchestrator rail plan; PR #1553 run 31589648809 at `69ef5f15d`. + +## E1 — RED fixture + +- Baseline `deno task quality:scan:repo`: exit 1; five `ts-error-suppression` findings; + `allowCount: 10`. +- Added the negative type-fixture regression test before changing scanner scope. +- The dispatch requires this failing test as its own commit, superseding the rail Design row that + groups E1 and E2 into one landed-green slice. +- Reconcile: live issue #1530 and draft PR #1560 were read; the PR remains draft and retains + `Closes #1530`. E2 is the immediate next slice that restores green. + +## E2 — directory-and-suffix exemption + +- Added `isTypeFixture`, which normalizes path separators and requires both + `/tests/type-fixtures/` and `_type.ts`. +- Quality tool tests: 7 passed, 0 failed, exit 0. +- Repo-wide scan: exit 0 with `allowCount: 8`; E4 still removes the now-unreachable allowance + comments from source so the repository no longer carries redundant policy prose. +- Reconcile: the E1 RED is resolved without changing fixture assertions. PR #1560 remains draft; + E3 must still prove the exemption does not leak. + +## E3 — leakage controls + +- Added one table-shaped test proving three paths remain scanned: ordinary source, `_type.ts` + outside `tests/type-fixtures/`, and a non-`_type.ts` file inside that directory. +- Quality tool tests: 8 passed, 0 failed, exit 0. +- Reconcile: live contract coverage now proves both sides of the conjunction. PR #1560 remains + draft; E4 owns redundant-comment removal and final gates. + +## E4 — redundant allowances and final gates + +- Removed only the trailing `quality-allow:` clauses from the two actual SDK fixture directives; + the `@ts-expect-error` assertions and their assertion reasons remain unchanged. +- Base repo scan at `84dd44ae7`: exit 1, five findings, `allowCount: 10`. +- Head repo scan: exit 0, no findings, `allowCount: 8`. +- Fixture diff audit: only the two named trailing allowance clauses changed across all `*_type.ts` + files. `deno.lock` is unchanged. No new suppression/cast/allowance construct was introduced. + +| Gate | Result | +| --- | --- | +| Quality tool tests | exit 0; 8 passed, 0 failed | +| `quality:scan:repo` | exit 0; `findings: []`; `allowCount: 8` | +| `quality:scan` | exit 0; `findings: []`; `allowCount: 7` | +| `quality:gate` | exit 0; scanner and doctrine chain green (warnings non-blocking) | +| Scoped check | exit 0; 2 files, 0 findings | +| Scoped lint | exit 0; 2 files, 0 findings | +| Scoped fmt | initial exit 1 on the pre-existing test block adjacent to E1; file-only formatting applied; rerun exit 0, 2 files, 0 findings | + +Reconcile: all six pre-merge acceptance boxes have evidence. Box 7 remains intentionally unmapped +and unchecked because it is marked `[post-merge]`. PR #1560 remains draft for orchestrator-owned +IMPL-EVAL and merge handling. + +## E5 — generated asset freshness + +- CI's `Generated asset freshness` step exposed that + `.llm/tools/quality/scan-code-quality.ts` is bundled as consumer tool source in + `packages/cli/src/kernel/assets/agent-tools.generated.ts`. +- Ran `deno task gen:assets-barrel` from the repository root. The complete generated diff changes + only the embedded scanner source (`isTypeFixture` plus the `isScannable` conjunction) and the + derived bundle hash. No other tool source or generated entry changed. +- `skills.generated.ts` mentions the installed scanner command but does not embed the scanner's + full source, so the canonical generator correctly left it unchanged. +- A second pre-commit generator run produced the identical one-file diff. + +| Gate | Result | +| --- | --- | +| `gen:assets-barrel` | exit 0; generated diff remains exactly 2 additions / 2 deletions in `agent-tools.generated.ts` | +| Quality tool tests | exit 0; 8 passed, 0 failed | +| `quality:scan:repo` | exit 0; `findings: []`; `allowCount: 8` | +| `quality:gate` | exit 0; scanner and doctrine chain green (warnings non-blocking) | + +Reconcile: E5 repairs the real CI coupling without changing scanner behavior or fixture assertions. +PR #1560 state, labels, milestone, existing IMPL-EVAL verdict, and merge authority remain untouched. diff --git a/.llm/runs/test-1374-docs-compile-gate--leaf/context-pack.md b/.llm/runs/test-1374-docs-compile-gate--leaf/context-pack.md new file mode 100644 index 0000000000..65de817449 --- /dev/null +++ b/.llm/runs/test-1374-docs-compile-gate--leaf/context-pack.md @@ -0,0 +1,114 @@ +# Context Pack: docs snippet compile gate for #1374 + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `test-1374-docs-compile-gate--leaf` | +| Branch | `test/1374-docs-snippet-compile-gate` | +| Current phase | `implement` (PLAN-EVAL cycle 2 `PASS`) | +| Archetype | N/A — internal docs tooling | +| Scope overlays | `SCOPE-docs` | + +## Current State + +All four planned slices are committed, explicitly pushed, and commented. Orchestrator pre-merge +verification found one fail-open task entry point: bare `docs:snippets:negative` fell through to +the positive corpus gate. Its narrow remediation now requires a declared fixture name and adds +task-level regression coverage. The PR remains draft at `status:impl`; the orchestrator owns the +draft-to-ready transition that automatically starts IMPL-EVAL. + +## Completed + +- Harness bootstrap and supervisor identity. +- Live issue read without re-deriving its verified Evidence section. +- Workspace resolver probe: exact `@netscript/*` workspace entrypoints resolve from synthetic + consumer files; TSX needs explicit Preact JSX mapping. +- Full census: 578 fences, 288 `ts`/`tsx` plus 7 recognized `typescript` aliases, 35 Tier-1 + candidates. +- Current accuracy/export-drift checker and Pages/core-CI wiring research. +- PLAN-EVAL cycle 1 concrete probes recorded and all eight findings dispositioned in the plan. +- PLAN-EVAL cycle 2 verified those fixes by execution and returned `PASS`; root-catalog config + copying and canonicalized merge comparison are mandatory implementation details. +- Revised locked plan: 21 Tier-1 checked, 14 reason-marked exemptions, 260 TS-like blocks outside + the floor. +- Extractor and marker/census contract committed as `b1129dd7b`. +- Exact public-entrypoint synthetic compiler committed as `5c828856b`. +- Real Tier-1 gate is green at the exact 35/21/14 census; the 14 structural markers and barrel fix + are applied, and the accuracy checker is demoted only where planned. +- Pages watches package/plugin/tooling/config changes in both trigger arms and runs the snippet gate + before Lume; its structural assertion passes in the focused and root suites. +- Final repo suite: 3,193 passed (617 steps), 0 failed, 17 ignored. All three raw negative controls + exit 1 with source-fence diagnostics. Tracked `deno.lock` is clean. +- F-1 remediation: bare and unknown negative-task cases both exit 1 with complete usage; focused + tests pass 10/10 and all three scoped wrappers pass over 22 files without lock drift. + +## In Progress + +- None in the implementation session after the F-1 commit/push/comment boundary. + +## Next Steps + +1. Stop with the PR still draft; the orchestrator performs the ready transition. +2. Owner automation starts the first IMPL-EVAL on that transition; do not dispatch another. + +## Key Decisions + +| Decision | Source | Notes | +| --- | --- | --- | +| OS-temp page-isolated synthetic modules | `plan.md` D1 | No tracked generated files or gitignore rule. | +| Exact workspace `exports` import map | `plan.md` D2 | No wildcard/private entrypoint mapping. | +| Marker/census contract | `plan.md` D3 | `typescript` is checked; 35 = 21 checked + 14 exempt; 260 outside floor; checked/candidate drops fail. | +| Pages workflow trigger | `plan.md` D4 | `ci.yml` keeps current accuracy step, no duplicate snippet job. | +| Accuracy demotion | `plan.md` D5 | One-page dialect containment and Fresh-root guard survive; only named positive needles are removed; drift checker unchanged. | +| Actual red controls | `plan.md` D6 | Three spawned CLI failures plus two dialect green controls. | + +## Implementation Evidence + +| Slice | State | Evidence | +| --- | --- | --- | +| S1 extractor/marker/census | COMMITTED | `b1129dd7b`; focused tests and scoped wrappers green; empty-reason raw exit 1 naming `page.md:1`. | +| S2 compiler/import resolution | COMMITTED | `5c828856b`; focused tests/scoped wrappers green; raw export and dialect controls exit 1 with mapped source fences. | +| S3 Tier-1/accuracy/coverage | COMMITTED | `983cf3464`; exact census, docs accuracy/links, scoped wrappers, and focused suite green. | +| S4 Pages/final gates | COMMITTED | `5ce34c089`; focused 9/9; final green/docs/scoped gates; three raw exit-1 controls; repository suite 3,193/0. | +| F-1 negative entrypoint remediation | COMMITTED | Bare/unknown raw exits 1; focused 10/10; scoped check/lint/fmt pass; lock clean. | + +## Files Changed + +| Path | Status | Notes | +| --- | --- | --- | +| `.llm/runs/test-1374-docs-compile-gate--leaf/supervisor.md` | new | Identity and routes. | +| `.llm/runs/test-1374-docs-compile-gate--leaf/research.md` | new | Full requested research and census. | +| `.llm/runs/test-1374-docs-compile-gate--leaf/plan.md` | new | Locked Phase 2 design. | +| `.llm/runs/test-1374-docs-compile-gate--leaf/worklog.md` | new | Filled Design checkpoint. | +| `.llm/runs/test-1374-docs-compile-gate--leaf/context-pack.md` | new | Resume state. | +| `.llm/runs/test-1374-docs-compile-gate--leaf/drift.md` | new | Cycle-1 evaluator-proven plan drift and dispositions. | + +## Gates + +| Gate family | Current status | Evidence | +| --- | --- | --- | +| Plan-Gate cycle 1 | `FAIL_PLAN` | Four blocking and four non-blocking findings returned; revised without implementation. | +| Plan-Gate cycle 2 | PASS | Fresh opposite-family session; cycle-1 fixes verified by execution. | +| Static/fitness/consumer | PASS through S4 | All requested gates green; three required negative controls red. | +| F-1 focused remediation | PASS | Missing/unknown task inputs red; exact regression and scoped wrappers green; no lock drift. | +| Runtime | N/A | Compilation-only slice. | + +## Open Questions + +None. + +## Drift and Debt + +- Drift: cycle-1 assumptions and the bounded Slice-3 support inventory miss are recorded in + `drift.md` with implemented dispositions. +- Debt: none created; named boundary issues remain external. + +## Commits + +- `ef64ea55c` — mandatory D2 plan amendment. +- `b1129dd7b` — extractor/marker/census contract. +- `5c828856b` — public-entrypoint synthetic compiler. +- `983cf3464` — Tier-1 docs floor, expansion plan, and accuracy demotion. +- `5ce34c089` — Pages package/plugin revalidation and final workflow assertion. +- Final remediation commit — fail-close the negative fixture entry point. diff --git a/.llm/runs/test-1374-docs-compile-gate--leaf/drift.md b/.llm/runs/test-1374-docs-compile-gate--leaf/drift.md new file mode 100644 index 0000000000..1bf2ab3780 --- /dev/null +++ b/.llm/runs/test-1374-docs-compile-gate--leaf/drift.md @@ -0,0 +1,82 @@ +# Drift Log: docs snippet compile gate for #1374 + +Drift is append-only. + +## 2026-08-12 — PLAN-EVAL cycle 1 concrete probes + +- **Severity:** significant plan drift; no implementation had started. +- **Observed:** the locked D2 command using the real root lock with `--frozen` exits 1 for green and + red synthetic configs because their workspace metadata cannot match the real workspace. The + original merge recipe also omitted root-catalog-only bare imports such as + `@opentelemetry/api`. +- **Disposition:** D2 now seeds a temporary lock copy, omits `--frozen`, discards all synthetic + lock rewrites, and materializes root catalog entries as `npm:` mappings. +- **Observed:** the planned accuracy demotion removed the only page-containment rule preventing + valid dialect B from returning to golden-path pages, and removed Fresh-root coverage from the + uncovered corpus. +- **Disposition:** keep the exact one-page `createServiceQueryUtils` rule and keep + `checkFreshRootImports`/`ALLOWED_FRESH_ROOT_SYMBOLS` unchanged; remove only the three named + positive reference needles and the other named positive assertions. +- **Observed:** `typescript` was an unchecked, reason-free opt-out and the policy treated a checked + count drop as informational. +- **Disposition:** recognize `typescript` as a `ts` alias, apply the same reason grammar, and fail + below 35 candidates or 21 checked, or above 14 exemptions. +- **Observed:** one proposed exemption is actually a broken barrel snippet, while two others are + complete primary island examples that only need typed local support modules. +- **Disposition:** fix the barrel binding inside its code fence, materialize the two support + modules, and compile all three unmarked. The honest floor changes from 18/17 to 21/14. +- **Observed:** the mutation-family count was 18, and the workflow assertion was conditional. +- **Disposition:** correct the count and make the structural `pages.yml` assertion unconditional + in slice 4. + +## 2026-08-12 — PLAN-EVAL cycle 2 mandatory D2 amendments + +- **Severity:** non-blocking implementation amendment required before slice 2; PLAN-EVAL verdict is + `PASS`. +- **Observed:** materialized top-level imports do not satisfy `"catalog:"` inside member configs + auto-discovered through file-URL workspace source. Without a root `catalog` section in the + synthetic config, every input exits 1 with `Package 'zod' not found in catalog` before type-check. +- **Disposition:** copy the root catalog (38 entries at evaluation) verbatim into the synthetic + config. The evaluator's identical green control then exits 0. +- **Observed:** literal merge comparison reports 40 false conflicts because root major shorthand + such as `jsr:@std/assert@1` is Deno-equivalent to member `jsr:@std/assert@^1`. +- **Disposition:** compare canonicalized package/version requirements. Current canonicalized + declared-import conflicts and declared-versus-catalog conflicts are both zero. Both checks remain + fail-closed config guards: a future conflict makes the build red; neither is claimed as a current + negative-test predicate. +- **Known ratchet window:** the three removed positive needles on + `docs/site/reference/sdk/index.md` are not compiler-covered until reference wave 4. Exact + one-page `createServiceQueryUtils` containment continues to block the wrong dialect on all + golden-path pages; only positive presence on the sanctioned page is deferred. + +## 2026-08-12 — Slice 1 scoped format baseline + +- **Severity:** mechanical, no behavioral drift. +- **Observed:** the required exact `.llm/tools/docs` format wrapper found Deno-format drift in the + new slice files and in pre-existing `check-exports-drift.ts` plus its test. +- **Disposition:** ran `deno fmt` only on the six selected TypeScript files. The two pre-existing + files changed formatting only; their checker behavior, mapping, and tests were not altered. The + exact scoped wrapper then exited 0. + +## 2026-08-12 — Slice 3 real-corpus support inventory + +- **Severity:** bounded implementation drift; no contract or coverage change. +- **Observed:** the first real Tier-1 compile reached two checked island fences whose documented + relative imports (`apps/dashboard/lib/orders.ts` and `apps/dashboard/lib/widgets.ts`) were not in + the planned support inventory. The gate exited 1 with TS2307 at + `web-layer/examples.md:40` and `web-layer/interactive.md:92`, followed by response-type errors. +- **Disposition:** materialize both modules from Zod/oRPC contracts plus public + `createServiceClient` and `createQueryFactories` APIs. They contain no casts or `any`. The same + 35/21/14 corpus then exits 0; no fence was exempted to accommodate the harness. + +## 2026-08-12 — Orchestrator pre-merge F-1 negative-task entry point + +- **Severity:** significant fail-open control-entry drift; the three named raw controls remained + valid and were independently reproduced, but the aggregate task name was misleading. +- **Observed:** `deno task docs:snippets:negative` supplied `--negative` without a value. The CLI + interpreted the missing value as positive mode, enforced the ordinary corpus floor, printed + `PASS`, and exited 0. That invocation proved no negative predicate. +- **Disposition:** the presence of `--negative` now requires one of the five declared fixture + names before any corpus analysis starts. Missing and unknown names exit 1 with + `deno task docs:snippets:negative <case>` usage and the complete fixture list. A task-level + regression test exercises both fail-closed paths. diff --git a/.llm/runs/test-1374-docs-compile-gate--leaf/plan.md b/.llm/runs/test-1374-docs-compile-gate--leaf/plan.md new file mode 100644 index 0000000000..fc58778260 --- /dev/null +++ b/.llm/runs/test-1374-docs-compile-gate--leaf/plan.md @@ -0,0 +1,407 @@ +# Plan: docs snippet compile gate for #1374 + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `test-1374-docs-compile-gate--leaf` | +| Branch | `test/1374-docs-snippet-compile-gate` | +| Phase | `plan` | +| Target | `.llm/tools/docs`, Tier-1 `docs/site` code fences, and Pages CI | +| Archetype | N/A — internal documentation fitness tooling, no package/plugin public surface | +| Scope overlays | `SCOPE-docs` | + +## Archetype and doctrine boundary + +No package/plugin implementation archetype applies: the new surface is a repo-internal Deno tool +and task. Doctrine A1 and A14 still govern the consumed boundary: snippets must name public +`@netscript/*` entrypoints, and the compile gate is the fitness function preserving that contract. +Doctrine file 10's individual package verdicts are inputs, not scope; this PR changes none of them. + +## Goal + +Make every unexempted `ts`/`tsx`/`typescript` fence on the nine #1373 Tier-1 pages compile against +exact workspace-declared public entrypoints, report an auditable exemption/coverage census, prove +three real red controls, demote literal API needles, and make package/plugin changes rerun the site +gate. + +## Scope + +- Add a checked-in extractor/compiler under `.llm/tools/docs` plus focused tests and fixtures. +- Add root tasks for the green gate, its tests, and explicit negative-control runs. +- Add reasoned `no-check` markers to the 14 deliberate Tier-1 partial/counter-example fences. +- Keep 21 Tier-1 fences checked; report 14 exemptions and 260 outside-floor TS-like blocks. +- Demote `docs:accuracy` exactly as specified below; keep `check-exports-drift` intact. +- Add the snippet gate and `packages/**`/`plugins/**` triggers to `pages.yml`. +- Check in a tooling-facing coverage/expansion document under `.llm/tools/docs`. + +## Non-Scope + +- No cast/`any` inventory or cast guard (#1278); support fixtures contain no `any`, `as any`, or + `as unknown as`. +- No expansion of `check-exports-drift` mappings (#1108). +- No package README coverage (#1377), README gate repair (#767), prose tutorials (#1208), or API + deep dives (#1210). +- No installed-artifact proof (#1343), execution, service start, browser smoke, or E2E CLI run. +- No edit to the stale MCP generated corpus (#1531). + +## Locked Decisions + +### D1 — Extraction, module assembly, naming, and lifetime + +**Decision.** Scan every `.md`/`.vto` below `docs/site`, parse matching backtick or tilde fences, +and recognize exactly the `ts`, `tsx`, and `typescript` language tokens. Normalize `typescript` to +the `.ts` compilation path; it is an alias, not an untracked language. For each unmarked Tier-1 +block: + +1. Create one page-isolated synthetic root under a single `Deno.makeTempDir` directory. +2. If the first code line begins with a path comment whose first token ends in `.ts` or `.tsx` + (for example `// apps/dashboard/lib/widgets.ts` or + `// routes/contact.tsx — bare Fresh`), materialize that path inside the page root so relative + imports between that page's blocks resolve. Otherwise name it + `blocks/<page-slug>-L<opening-line>-B<ordinal>.<ts|tsx>`. +3. Reject duplicate materialized paths instead of overwriting one block with another. +4. Prepend only a provenance comment and a reference to a shared strict preamble. The preamble + declares no ambient application names and no permissive types. Typed support modules provide + only the explicit scaffold/app aliases required by the 21 checked blocks. In particular, + materialize strongly typed page-local modules at the paths resolved by + `web-layer/query.md:214` (`<page-root>/lib/docs.ts`) and `:267` + (`<page-root>/apps/dashboard/lib/todos.ts`) so those primary island examples remain checked + rather than exempted to fit the harness. Both supports derive their procedures through the + public dialect-A `createQueryFactories` surface—not hand-shaped `queryOptions` objects. The docs + fixture types `{ id: string } -> { status: "pending" | "embedding" | "ready" }`; the todos + fixture types `list` plus `update({ id: string; done: boolean })`, exposing the real + `clientKey`/`mutationOptions` contracts used by the fences. They contain no cast escape hatch. +5. Run one `deno check` over the generated entry set and map diagnostics back to source page/line. +6. Delete the entire temp root in `finally`, on green and red exits. + +Generated files live outside the worktree in the OS temp directory. They are not committed and do +not need a gitignore rule; no directory named `coverage` is introduced. The source path plus opening +line and ordinal make diagnostics and repeated runs deterministic even though the temp prefix is +random. + +**Reason.** Page-isolated roots preserve real relative-import semantics without letting an example +on one page accidentally satisfy another. A minimal preamble makes missing context fail or require +an explicit marker; it cannot silently legalize a fragment. + +### D2 — Exact `@netscript/*` resolution + +**Decision.** Build the synthetic import map from repository data at runtime: + +- Expand the root workspace globs and read every member `deno.json`. +- For each member whose `name` starts with `@netscript/`, map only its declared export keys: `.` to + the package name and each `./subpath` to the exact `@netscript/name/subpath` specifier. +- Resolve targets to absolute file URLs for the declared entrypoint files. Do not add a package + prefix mapping and do not expose undeclared files. +- Merge declared external imports from the root/member configs, resolving member `catalog:` values + through the root npm catalog and comparing canonicalized package/version requirements before + failing on conflicts. Deno-equivalent range spellings such as `jsr:@std/assert@1` and + `jsr:@std/assert@^1` are the same mapping; literal spelling differences are not conflicts. Also + materialize every root catalog entry that has no declared-import mapping as `<catalog-key>` → + `npm:<catalog-key>@<catalog-range>`. This catalog-fallback pass is mandatory: bare imports such as + `@opentelemetry/api` can be reachable through `@netscript/sdk` while appearing in no `imports` + block. A declared import and catalog fallback for the same key must canonicalize to the same npm + package/range or configuration fails; no silent precedence is allowed. Against the current root + plus 37 members, canonicalized declared-import comparison yields zero conflicts (literal + comparison would yield 40), and declared-versus-catalog comparison also yields zero. These are + intentionally fail-closed configuration guards: even though neither fires on today's repo, a + future real conflict stops config construction loudly instead of letting an unchecked graph pass. +- Add only the named, typed support aliases (`@database/zod`, `@playground/contracts`, + `@my-app/contracts`, `@app/utils.ts`, `@app/lib/contacts.ts`) and exact Preact JSX runtime + mappings required by Tier-1 examples. + +The generated config copies the root `catalog` section (currently 38 entries) verbatim so Deno's +auto-discovered member configs can resolve their own `"catalog:"` imports while their workspace +source is reached through file-URL exports. Top-level import-map entries do not replace this config +section. The config also keeps `strict`, `noImplicitAny`, and `noImplicitReturns`; sets +`isolatedDeclarations: false` because examples are consumers rather than publishable declarations; +and selects Preact `jsx: precompile`. Before every check, copy the repository `deno.lock` to +`<temp>/deno.lock`. The subprocess is: + +```text +deno check --unstable-kv --lock <temp>/deno.lock --config <temp>/deno.json <entries...> +``` + +The checker uses `Deno.execPath()`/`Deno.Command`, executes no snippet, grants no snippet runtime +permissions, and starts no service. It deliberately does not pass `--frozen`: Deno must reconcile +the copied lock's workspace/config metadata with the synthetic config. The seeded temporary lock +retains the repository's dependency resolutions, any synthetic-workspace rewrite is discarded with +the temp root, and the tracked root lock is never passed as a writable target. It also deliberately +does not use `--no-lock`, which would let npm resolution float. User code is never run. + +**Reason.** The snippet text continues to import `@netscript/*`, while exact declared-export map +generation makes a missing export or undeclared subpath fail. Root-catalog materialization closes +the transitive bare-specifier gap. A disposable copy of the real lock preserves its pins without +asking a synthetic workspace to satisfy the real workspace's frozen metadata, avoiding both the +always-red `--frozen` design and unpinned `--no-lock` resolution. + +### D3 — Marker grammar and census + +**Decision.** The only exemption form is an opening fence whose complete info string matches: + +```text +^(ts|tsx|typescript)[ \t]+no-check:[ \t]*(\S(?:.*\S)?)[ \t]*$ +``` + +In source this is, for example, ```` ```tsx no-check:partial builder chain uses surrounding state ````. +The separator is one or more spaces; the reason is trimmed and must contain a non-whitespace +character. For any fence whose first info token is `ts`, `tsx`, or `typescript`, the complete info +string must be either that token alone or the exact reasoned marker grammar. Thus `no-check`, +`no-check:`, extra attributes, or a marker on the closing fence is malformed and exits non-zero with +page/line. The alias follows the same marker rules and cannot bypass coverage. + +The gate prints one stable summary to stdout and the GitHub step summary when available: + +```text +docs snippets: PASS scanned=578 ts=211 tsx=77 typescript=7 ts_like=295 tier1=35 checked=21 exempt=14 outside_floor=260 malformed=0 +``` + +It also prints a per-page exemption list (`page:line — reason`). A checked-in policy records the +nine Tier-1 pages, `minimumChecked: 21`, `maximumExempt: 14`, and `minimumCandidates: 35`. The gate +fails if checked coverage drops below 21, the exemption count exceeds 14, or recognized Tier-1 +candidates drop below 35; lower exemption counts and higher checked/candidate counts are reported +as ratchet opportunities. The implementation tests the exact baseline census and an explicit +language-alias regression that renames a floor fence from `ts` to `typescript` without reducing +coverage. + +**Reason.** An inline, reason-required opt-out is reviewable beside the code. Separate `checked`, +`exempt`, and `outside_floor` counts plus a checked-count floor make day-one coverage honest; a +language rename cannot turn a checked block into an invisible success. + +### D4 — CI owner and unchanged `ci.yml` + +**Decision.** Put the trigger fix in `.github/workflows/pages.yml`: + +- Add `packages/**`, `plugins/**`, `.llm/tools/docs/**`, `deno.json`, and `deno.lock` to both PR and + push path filters. +- Add a root-working-directory `deno task docs:snippets` step before the Lume build. +- Add an unconditional structural workflow test under `.llm/tools/docs` that parses `pages.yml` + and asserts both PR and push filters include `packages/**` and `plugins/**`, and that the build job + invokes `deno task docs:snippets` before Lume. A typo in either trigger arm or the step ordering + fails `docs:snippets:test` and the repo suite. +- Keep the existing build/link/caveat and deploy behavior. + +Do not add a duplicate job or step to `ci.yml`; its existing `docs:accuracy` step remains, and the +snippet task remains independently owned by Pages. + +**Reason.** Pages is the site-owning workflow and, unlike current core CI, runs on draft PRs. This +directly proves that a package/plugin (including CLI/SDK) change rebuilds and revalidates the site, +while avoiding duplicate expensive checks and competing ownership in `ci.yml`. + +### D5 — Exact `docs:accuracy` demotion + +**Decision.** Keep only string-policy assertions plus the code-derived drift subprocess. + +Survive, named: + +- Forbidden stale saga call/send shapes: `defineSaga({`, the four orphan `send(...)` forms, and + `{ kind: 'service', id: 'payments' }`. +- Forbidden golden-path claims across published source: `lib/api-clients.ts`, `@contracts`, + `@/lib/`, and `apps/<app>/client.ts` presented as a data client. +- The `## Mutation and regeneration map` presence and its five columns: `Command`, + `Source of truth mutated`, `Generated artifacts`, `Runtime consumers`, `Preview`. +- The exact one-page `createServiceQueryUtils` containment rule: the set of pages containing the + term must remain exactly `[docs/site/reference/sdk/index.md]`. This vocabulary policy + distinguishes the golden-path `createQueryFactories` dialect from valid reference-only dialect B; + compilation cannot prove page placement. +- `ALLOWED_FRESH_ROOT_SYMBOLS` and `checkFreshRootImports` unchanged across all docs. The compile + gate overlaps it on covered fences, but removal is deferred until the compile ratchet covers the + remaining corpus; this PR must not open an unvalidated window. +- Invocation of `check-exports-drift.ts` unchanged. + +Remove, named: + +- `defineSaga`/`spawn` source regexes and positive signature/source needles. +- Positive saga markers `event.payload.body`, `type: "=> never"`, + `options?: SpawnOptions): never`, and the `SagasError.notImplemented(...)` source spelling. +- The eight preferred-path presence needles. +- The three `--with-client` presence checks and the two quickstart path/role needles. +- Only the three positive needles on `docs/site/reference/sdk/index.md`: + `queryOptions({ input })`, `queryOptions(input)`, and `no server KV tier`. The one-page + `createServiceQueryUtils` containment remains. +- The 18 hardcoded CLI mutation-family presence needles: `init`, `config set`, `contract add`, + `contract add-route`, `contract version add`, `service add`, `service set`, `service ref`, + `db add`, `db init`, `plugin install`, `plugin update`, `generate plugins`, + `generate runtime-schemas`, `generate aspire`, `ui:init`, `ui:add`, and `deploy`. +Update the existing accuracy tests so they cover the surviving stale-claim, vocabulary-containment, +Fresh-root, and mutation-column policies. Do not change `check-exports-drift.ts` or its tests. + +**Reason.** Forbidden vocabulary, required table columns, and dialect page-containment are +inherently textual policy. Export and call-shape truth on covered pages belongs to the compiler; +the Fresh-root guard cannot be removed from the remaining uncovered corpus yet, and +package/reference export-table truth stays in the code-derived drift checker. + +### D6 — Three real red controls (plus the dialect positive control) + +**Decision.** Check in isolated fixture sites and make every test spawn the actual CLI entrypoint, +assert a non-zero process code, and assert the diagnostic names the source fence: + +1. `non-exported-symbol`: a `ts` fence imports a definitely absent named export from + `@netscript/sdk`; Deno check must exit non-zero. A unit-only parser assertion is insufficient. +2. `empty-exemption-reason`: a `ts no-check:` opening fence; extraction must exit non-zero before + compilation with a missing-reason diagnostic. +3. `dialect-a-object-input`: a typed fixture constructs dialect-A `createQueryFactory` helpers + through exported `@netscript/sdk/client` and `@netscript/sdk/query` contracts, then calls + `queryOptions({ input: { limit: 1 } })`; Deno check must exit non-zero. + +The dialect fixture suite also has two green controls: dialect A with +`queryOptions({ limit: 1 })`, and `createServiceQueryUtils` dialect B with +`queryOptions({ input: { limit: 1 } })`. This prevents implementation as a blanket string ban that +would reject correct dialect-B docs. + +Expose deterministic negative commands through the task, one case at a time, so Phase 2 records +three raw non-zero exits rather than hiding them inside a passing test runner. + +**Reason.** These controls cross the same extractor → synthetic workspace → `deno check` boundary +as production. Positive dialect controls prove the predicate discriminates the API, not the text. + +### D7 — Commit slices and named gates + +| # | Slice and proof | Files | Named gate before commit | +| --- | --- | --- | --- | +| 1 | Extractor contract: fence parsing, Tier-1 policy, exact marker grammar, census, and missing-reason red control. | New checker/test/policy/fixture files under `.llm/tools/docs`; `deno.json`; run artifacts. | Scoped check/lint/fmt wrappers; `deno task docs:snippets:test`; direct empty-reason negative command exits non-zero. | +| 2 | Compiler contract: exact workspace export-map resolver, root-catalog fallback, copied temporary lock, synthetic config/preamble/support fixtures (including the two query-island modules), actual `deno check`, non-exported and dialect-aware red controls plus both dialect green controls. Tests must exercise an `@netscript/sdk` graph that reaches catalog-only `@opentelemetry/api`, prove a green synthetic config with the copied lock, and assert the tracked root lock bytes remain unchanged. | Checker/test/support fixtures; run artifacts. | Scoped wrappers; `deno task docs:snippets:test`; direct non-exported and dialect-A negative commands each exit non-zero. | +| 3 | Tier-1 green floor and demotion: fix the add-service barrel binding, apply 14 reason markers, compile 21 blocks, record the expansion plan, remove only named API needles while retaining containment and Fresh-root guards. | Nine Tier-1 pages as needed; accuracy checker/test; `.llm/tools/docs/snippet-coverage.md`; run artifacts. | `deno task docs:snippets`; `rtk proxy deno task docs:accuracy`; `rtk proxy deno task docs:links`; scoped wrappers. | +| 4 | Site trigger contract and final evidence: Pages owns the snippet step and package/plugin paths; add an unconditional structural workflow assertion and complete all requested gates. | `.github/workflows/pages.yml`; `.llm/tools/docs/pages-workflow_test.ts`; run artifacts. | All three scoped wrappers; `deno task docs:snippets:test` (including the workflow assertion); docs links; docs accuracy; green snippet gate; all three raw red controls; `rtk proxy deno task test`. | + +Every slice updates `worklog.md` and `context-pack.md`, is committed, pushed with the explicit +refspec, and receives one PR phase/slice comment before the next slice. Implementation does not +start until separate-session PLAN-EVAL returns `PASS`. + +### D8 — Written expansion plan beyond Tier-1 + +**Decision.** Check in `.llm/tools/docs/snippet-coverage.md` with the machine census and these +ratchet waves; each wave lowers `outside_floor` and records checked/exempt deltas: + +1. Remaining golden-path families: the rest of `web-layer/**`, `services-sdk/**`, and + `quickstart/**`. +2. Runtime guides: `ai/**`, `background-processing/**`, `data-persistence/**`, + `durable-workflows/**`, `identity-access/**`, `observability/**`, + `orchestration-runtime/**`, and `explanation/**`. +3. Tutorials: `tutorials/**`, one tutorial track per change so multi-file context stays reviewable. +4. Reference code blocks: `reference/**`, coordinated with #1108 but limited here to prose fences; + clean the seven `typescript` tags to the canonical `ts` spelling. They are already recognized + and compiled as `ts` aliases from day one, so this wave is tag cleanup rather than coverage. +5. Non-published source/templates under underscore directories are last and reported separately; + package READMEs remain #1377. + +Each expansion PR must add pages to the checked-in coverage policy, make every new unmarked block +green, add reason markers only for deliberate fragments, lower the outside-floor count, and never +increase the exemption baseline without an explicit reviewer-approved rationale. + +The coverage document must name one deliberate window until wave 4: removing the three positive +reference-page needles (`queryOptions({ input })`, `queryOptions(input)`, `no server KV tier`) +means their sanctioned-page presence is not compiler-covered while +`docs/site/reference/sdk/index.md` remains outside the day-one floor. The retained exact one-page +`createServiceQueryUtils` containment still blocks dialect B on every golden-path page; only positive +presence on the one sanctioned reference page is deferred, avoiding continuation of the +false-green needle class this issue removes. + +**Reason.** Section-sized ratchets keep fixtures coherent and make coverage movement auditable; +tutorials and references have distinct owners and failure shapes. + +## Planned Tier-1 exemptions (14) + +| Page / current opening line | Marker reason | +| --- | --- | +| `web-layer/query.md:71` | `uses cache entries defined earlier in the surrounding application` | +| `web-layer/examples.md:90` | `uses cached values supplied by the surrounding page loader` | +| `web-layer/form.md:29` | `counter-example intentionally omits bare Fresh setup` | +| `web-layer/form.md:279` | `partial field-array fragment uses the surrounding form state` | +| `web-layer/form.md:353` | `partial JSX fragment uses the surrounding form state` | +| `web-layer/query-bridge.md:29` | `counter-example intentionally omits bare Fresh setup` | +| `web-layer/query-bridge.md:42` | `counter-example intentionally omits bare TanStack setup` | +| `web-layer/query-bridge.md:108` | `pseudocode compares key shapes rather than defining runnable values` | +| `web-layer/query-bridge.md:124` | `partial invalidation fragment uses surrounding query utilities` | +| `web-layer/query-bridge.md:139` | `partial invalidation fragment uses surrounding cache values` | +| `web-layer/query-bridge.md:164` | `partial builder chain uses page-local declarations omitted here` | +| `web-layer/query-bridge.md:198` | `object-shape pseudocode documents generated query options` | +| `web-layer/query-bridge.md:208` | `partial component uses page-local types and view components` | +| `web-layer/query-bridge.md:257` | `partial builder chain continues from surrounding page code` | + +Line numbers are research provenance, not stable identifiers; the gate reports current lines at +runtime. The budget is the count, and every marker remains inline with its reason. + +The three removed candidates are not exemptions. `web-layer/query.md:214` and `:267` compile +against strongly typed materialized `docs.ts` and `todos.ts` support modules. The add-service block +at current line 152 is a documentation defect found by the gate: slice 3 changes it to import +`UsersContractV1` and `UsersV1`, re-export those local bindings, then build `v1` from the imported +`UsersV1`. This is a code-fence correction, not prose expansion, and it must compile unmarked. + +## Open-Decision Sweep + +| Decision | Status | Notes | +| --- | --- | --- | +| Synthetic compiler config/import-map construction | Resolved now | D2 locks exact declared exports, root-catalog fallback, and a disposable copied lock without `--frozen`. | +| TSX runtime | Resolved now | Preact precompile with exact JSX-runtime import mapping. | +| `isolatedDeclarations` behavior | Resolved now | Disabled only in the synthetic consumer config; strictness remains. | +| CI owner | Resolved now | Pages; `ci.yml` intentionally unchanged except its existing accuracy invocation. | +| Day-one exemption count | Resolved now | 14 exemptions / 21 checked / 35 Tier-1 candidates. | +| `typescript` fence alias | Resolved now | Recognized as `ts` from day one; reference expansion later canonicalizes spelling only. | +| Package README coverage | Safe to defer | Owned by #1377. | + +No open decision would force implementation rework. + +## Risk Register + +| Risk | Mitigation | +| --- | --- | +| Import map accidentally exposes private source | Generate only exact keys from each package's declared `exports`; negative non-exported fixture proves failure. | +| Reachable workspace source imports a catalog-only bare package | Materialize every root catalog key as an npm fallback and exercise the SDK→telemetry→`@opentelemetry/api` graph. | +| Preamble makes fragments falsely green | No ambient application declarations, no `any`, and only explicit strongly typed support modules; partials use reason markers. | +| Dialect test becomes a blanket text ban | Compile a typed dialect-A red fixture and both dialect-A/dialect-B green controls. | +| `isolatedDeclarations` creates consumer-example noise | Disable only that publish-oriented option in synthetic config; retain strict type checks. | +| Relative imports cross-contaminate pages | Page-isolated synthetic roots; duplicates fail within a page. | +| Language rename or exemption count hides coverage loss | Recognize `typescript`, enforce checked/candidate floors and an exemption ceiling, and print per-page reasons. | +| Generated files leak into git | OS temp root plus unconditional `finally` removal; no worktree output. | +| Lockfile churn or synthetic-config frozen failure | Copy the root lock into the temp root, omit `--frozen`, discard the copy, inspect raw git state before each commit, and never reload/delete cache. | +| Pages trigger looks correct but does not run checker | Add explicit workflow step and a repository test/assertion covering both package/plugin paths and task name. | +| Root test discovers negative fixtures as normal tests | Fixtures are Markdown/support modules outside `_test.ts` discovery; negative execution is explicit. | + +## Anti-Patterns to Resolve or Avoid + +| AP / axiom | Status | Plan | +| --- | --- | --- | +| A1 public types first | In scope | Resolve only declared package entrypoints and compile typed consumer calls. | +| A7 platform first | In scope | Use Deno file APIs, temp dirs, workspace config data, and `deno check`; no custom compiler. | +| A14 tests as fitness functions | Core deliverable | Three actual red controls plus green controls and the Tier-1 gate. | +| False-green text needles | Existing defect | Remove named API needles; keep only textual policy. | +| Permissive ambient test doubles | Risk | Strongly typed support modules only; no `any` or cast-based legalization. | + +## Fitness and Validation Gates + +| Gate | Required | Expected evidence | +| --- | --- | --- | +| Scoped tool type-check | Yes | Required wrapper command exits 0. | +| Scoped tool lint | Yes | Required wrapper command exits 0. | +| Scoped tool format | Yes | Required wrapper command exits 0. | +| Docs links | Yes | `rtk proxy deno task docs:links` exits 0. | +| Docs accuracy after demotion | Yes | `rtk proxy deno task docs:accuracy` exits 0. | +| Tier-1 snippet gate | Yes | New task exits 0 and prints `checked=21 exempt=14 outside_floor=260`. | +| Three negative controls | Yes | Each direct fixture command exits non-zero; raw codes/diagnostics recorded. | +| Repo test suite | Yes | `rtk proxy deno task test` exits 0. | +| Runtime/Aspire/browser/E2E CLI | N/A | Compilation-only docs tooling; explicitly excluded. | +| jsr-audit / package quality gate | N/A | No package/plugin implementation or export change. | + +## Arch-Debt Implications + +No new architecture debt is planned. Existing package verdicts and #1531 remain unchanged. Any +Tier-1 block that requires `any`, `as any`, or `as unknown as` to pass will be recorded in +`drift.md` and left unlegalized for #1278 rather than hidden in the preamble. + +## Deferred Scope + +- The 260 TS-like blocks outside Tier-1 follow D8's ratchet. +- Seven `typescript` fences are covered by the alias now and canonicalized cosmetically in the + reference wave. +- Package READMEs, installed-consumer proof, export-drift expansion, and stale generated MCP prose + stay with their named issues. + +## Drift Watch + +- A planned checked block needs a cast or permissive fixture. +- Exact workspace exports cannot be derived without a prefix/private mapping. +- The observed Tier-1 census drops below `35 candidates / 21 checked`, exceeds `14 exempt`, or an + alias rename changes coverage. +- `pages.yml` cannot run the root task without changing lock or install policy. +- A current line/path no longer matches the research baseline after PLAN-EVAL resumes. diff --git a/.llm/runs/test-1374-docs-compile-gate--leaf/research.md b/.llm/runs/test-1374-docs-compile-gate--leaf/research.md new file mode 100644 index 0000000000..2ec1dd95b9 --- /dev/null +++ b/.llm/runs/test-1374-docs-compile-gate--leaf/research.md @@ -0,0 +1,238 @@ +# Research — test-1374-docs-compile-gate--leaf + +## Re-baseline + +- Carried-in source: live issue #1374 and the dispatch brief. +- Re-derived against `origin/main` at `01aa12b67e36b643e1ca4f94421ecba07e030db5` on 2026-08-12. +- The issue Evidence section is accepted as verified fact and was not re-derived. Research below + covers the design inputs requested by the brief: workspace resolution, the live fence census, + checker responsibilities, and workflow wiring. +- Orchestrator-provided current facts: the old dialect markers are zero under `docs/site/**`; the + stale MCP generated corpus is #1531 and out of scope. + +## Findings + +| # | Finding | How to verify | +| --- | --- | --- | +| 1 | The root `deno.json` declares `workspace: ["packages/*", "packages/cli/e2e", "plugins/*", "examples/*", "apps/*"]`. Workspace member `name` plus `exports` maps `@netscript/*` and exact subpaths to local declared entrypoints. | `deno info --json @netscript/sdk/query` reports root `packages/sdk/src/query/mod.ts`; `deno info --json @netscript/sdk` reports `packages/sdk/mod.ts`. | +| 2 | A synthetic file outside the repository resolves `@netscript/sdk/client` and `@netscript/sdk/query` when checked with the root config. TSX additionally needs the Preact JSX import mapping. | Research probe: `deno check --unstable-kv --frozen --config <root>/deno.json <temp>.ts`; TSX passes when the import map supplies exact `preact` and `preact/jsx-runtime` mappings. | +| 3 | Root `isolatedDeclarations: true` is correct for packages but creates example-only failures for exported snippet declarations without explicit annotations. The synthetic config must turn it off; it must not weaken `strict`, `noImplicitAny`, or `noImplicitReturns`. | Root `deno.json:compilerOptions`; research TSX probe emitted TS9007 until checked from a config with `isolatedDeclarations: false`. | +| 4 | `docs/site/**` contains 246 `.md`/`.vto` source files, 578 fenced blocks on 123 pages, 211 `ts`, 77 `tsx`, and 7 `typescript` blocks. The gate recognizes all three TS-like tags, for 295 blocks. | Read-only Deno fence scanner over every `.md`/`.vto`, with matching backtick/tilde fence length and an unclosed-fence assertion. Per-page table below. | +| 5 | The nine Tier-1 pages contain 35 `ts`/`tsx` blocks: `quickstart` 1, `index` 0, SDK overview 1, add-service 6, query 7, examples 2, interactive 2, form 6, query-bridge 10. | Focused extraction with source lines; table below. | +| 6 | Tier-1 includes complete modules, multi-file continuations indicated by leading `// path.ts[x]` comments, deliberate counter-examples, and partial fragments with app-local names. A permissive ambient preamble would make the gate lie. | Direct inspection of the 35 extracted blocks. After the PLAN-EVAL disposition, the planned census is 21 checked and 14 reason-marked exemptions. | +| 7 | `check-accuracy-and-discoverability.ts` mixes forbidden stale-text rules with positive API needles, source regexes, a manual Fresh-root export allowlist, 18 hardcoded CLI mutation families, and a subprocess call to `check-exports-drift.ts`. | `.llm/tools/docs/check-accuracy-and-discoverability.ts`. Exact demotion is locked in `plan.md`. | +| 8 | `check-exports-drift.ts` reads package `deno.json` export maps and docs export tables for 8 mappings. Only `config`, `contracts`, and `telemetry` have `checkSymbols: true`; symbol checking shells out to `deno doc --json`. Expanding its mapping belongs to #1108. | `.llm/tools/docs/check-exports-drift.ts:13-220,222-392`. | +| 9 | `pages.yml` runs for PRs/pushes only on `docs/site/**` and itself. It builds Lume, links, and caveats, but has no snippet/API check. | `.github/workflows/pages.yml`. | +| 10 | `ci.yml` suppresses all draft-PR jobs. On non-drafts its `quality` job runs when classifier output is `needs_deno || needs_docs`, and currently invokes `docs:accuracy` but no snippet compile task. `packages/**`/`plugins/**` classify as Deno changes. | `.github/workflows/ci.yml`; `.github/scripts/ci-classify-changes.ts`. | +| 11 | Deno 2.9.5 `check` accepts multiple entry files, `--config`, `--import-map`, `--lock`, `--frozen`, and `--unstable-kv`; it type-checks without executing the modules. | `deno check --help`; `netscript-deno-toolchain` skill. | +| 12 | A synthetic config cannot use the real workspace lock with `--frozen`: every evaluator control exits 1 with `The lockfile is out of date`. Copying the root lock into the temp root and omitting `--frozen` allows Deno to reconcile synthetic workspace metadata without mutating the tracked lock. | Opposite-family PLAN-EVAL cycle-1 executed D2 probe (B1). | +| 13 | Root/member `imports` are insufficient for reachable workspace graphs: `@opentelemetry/api` is imported by telemetry but exists only in the root npm `catalog`. The synthetic map must materialize root catalog fallbacks. | Opposite-family PLAN-EVAL cycle-1 D2 probe; `deno.json:215`; `packages/telemetry/src/context/w3c.ts`. | +| 14 | Of the original 17 planned exemptions, 14 are structural, two query-island examples are harness-fit and can compile with typed relative support modules, and one add-service barrel is a real missing-local-binding defect. | Opposite-family PLAN-EVAL cycle-1 classification and compiled probes (B4/N1). | + +## Fence census by language + +| Language | Blocks | +| --- | ---: | +| `ts` | 211 | +| `sh` | 168 | +| `tsx` | 77 | +| `bash` | 63 | +| `text` | 23 | +| plain info string | 20 | +| `typescript` | 7 | +| `json` | 4 | +| `js`, `jsonc`, `md`, `powershell`, `prisma` | 1 each | +| **Total** | **578** | + +The compile gate recognizes `ts`, `tsx`, and `typescript`: **295 TS-like blocks**. `typescript` is +normalized to the `.ts` compilation path immediately so a one-word tag rename cannot bypass the +gate; canonical tag cleanup remains part of the reference expansion wave. + +## Fence census by page + +Only pages containing at least one fence are listed. Counts include every language so the scanner's +denominator is reproducible. + +| Page | Fence census | +| --- | --- | +| `_diagrams/README.md` | plain 4 | +| `_includes/readme-template.md` | sh 1, ts 1 | +| `_plan/02-information-architecture.md` | plain 1 | +| `_plan/10-nav-ia-redesign.md` | plain 1, ts 1 | +| `_plan/briefs/00-INDEX.md` | plain 1 | +| `_plan/research/market-fit.md` | plain 1 | +| `_plan/research/netscript-feature-landscape.md` | plain 1 | +| `ai/agent-tooling.md` | bash 2 | +| `ai/chat-ui.md` | text 1 | +| `ai/durable-chat.md` | text 1, ts 5 | +| `ai/engine.md` | ts 1 | +| `ai/how-to/build-a-durable-chat.md` | text 1, ts 3, tsx 1 | +| `ai/mcp.md` | ts 1 | +| `background-processing/how-to/add-a-task-runtime-adapter.md` | bash 1, js 1, ts 3 | +| `background-processing/how-to/restrict-worker-task-permissions.md` | ts 1 | +| `background-processing/how-to/run-a-polyglot-task.md` | sh 1, ts 1 | +| `background-processing/how-to/tune-worker-runtime.md` | bash 1, sh 1, ts 2 | +| `background-processing/workers.md` | bash 3, ts 4 | +| `data-persistence/database.md` | text 1, ts 1 | +| `data-persistence/how-to/choose-a-queue-provider.md` | bash 1, ts 5 | +| `data-persistence/how-to/database-migration.md` | bash 1, text 1 | +| `data-persistence/how-to/queue-kv-cron.md` | ts 9 | +| `data-persistence/how-to/use-a-second-database.md` | bash 1, ts 1 | +| `durable-workflows/how-to/build-a-validated-ingestion-queue.md` | ts 3 | +| `durable-workflows/how-to/publish-a-durable-stream.md` | ts 4 | +| `durable-workflows/sagas.md` | ts 2 | +| `durable-workflows/streams.md` | ts 1 | +| `durable-workflows/triggers.md` | ts 4 | +| `explanation/architecture.md` | text 1 | +| `explanation/aspire.md` | text 1, ts 4 | +| `explanation/auth-model.md` | text 1 | +| `explanation/contracts.md` | text 1, ts 1 | +| `explanation/durability-model.md` | text 1, ts 3 | +| `explanation/observability.md` | text 1, ts 1 | +| `identity-access/better-auth-plugins.md` | ts 2, tsx 1 | +| `identity-access/how-to/add-authentication.md` | sh 7 | +| `identity-access/session-lifecycles.md` | ts 3 | +| `observability/how-to/add-opentelemetry.md` | bash 3 | +| `orchestration-runtime/cli-scaffold.md` | bash 1 | +| `orchestration-runtime/how-to/add-a-plugin.md` | bash 5 | +| `orchestration-runtime/how-to/author-a-plugin.md` | plain 2, sh 5 | +| `orchestration-runtime/how-to/deno-lsp-code-intelligence.md` | bash 5, json 2, powershell 1 | +| `orchestration-runtime/how-to/deploy-deno-deploy.md` | ts 1 | +| `orchestration-runtime/how-to/deploy-local-aspire.md` | bash 4 | +| `orchestration-runtime/how-to/deploy.md` | bash 6, ts 1 | +| `orchestration-runtime/how-to/graceful-shutdown.md` | ts 4 | +| `orchestration-runtime/how-to/roll-out-runtime-overrides.md` | bash 3, ts 1 | +| `orchestration-runtime/runtime-config.md` | ts 2 | +| `quickstart.vto` | bash 4, sh 10, text 1, ts 1 | +| `quickstart/aspire.md` | bash 2 | +| `reference/ai/index.md` | ts 12 | +| `reference/ai/skills.md` | md 1, ts 4 | +| `reference/contracts/index.md` | typescript 3 | +| `reference/cron/index.md` | ts 1 | +| `reference/fresh-ui/index.md` | tsx 3 | +| `reference/plugin-ai-core/index.md` | typescript 3 | +| `reference/plugin-ai/index.md` | bash 1, text 1, ts 5 | +| `reference/prisma-adapter-mysql/index.md` | typescript 1 | +| `reference/queue/index.md` | ts 1 | +| `reference/sagas/index.md` | ts 1 | +| `reference/streams/index.md` | ts 1 | +| `reference/telemetry/convention.md` | text 1 | +| `reference/triggers/index.md` | ts 1 | +| `reference/watchers/index.md` | ts 1 | +| `reference/workers/index.md` | ts 1 | +| `services-sdk/how-to/add-a-service.md` | bash 7, text 1, ts 6 | +| `services-sdk/how-to/discover-services.md` | bash 3, jsonc 1, ts 4 | +| `services-sdk/how-to/expose-openapi-scalar.md` | sh 1, ts 1 | +| `services-sdk/sdk.md` | ts 1 | +| `services-sdk/services.md` | ts 2 | +| `tutorials/chat/01-scaffold.md` | plain 1, sh 7 | +| `tutorials/chat/02-durable-chat-route.md` | sh 2, ts 4 | +| `tutorials/chat/03-chat-ui.md` | bash 2, tsx 3 | +| `tutorials/chat/04-tool-call.md` | bash 1, ts 2, tsx 1 | +| `tutorials/chat/05-mcp.md` | bash 1, sh 2, ts 3 | +| `tutorials/chat/06-live-streaming.md` | bash 1, tsx 1 | +| `tutorials/chat/index.md` | sh 2 | +| `tutorials/erp-sync/01-scaffold.md` | plain 1, sh 9 | +| `tutorials/erp-sync/02-import-job.md` | sh 8, ts 2 | +| `tutorials/erp-sync/03-polyglot-transform.md` | plain 2, sh 4, ts 4 | +| `tutorials/erp-sync/04-queue-and-cron.md` | sh 4, ts 2 | +| `tutorials/erp-sync/05-deploy.md` | sh 5 | +| `tutorials/erp-sync/index.md` | plain 1 | +| `tutorials/live-dashboard/01-scaffold.md` | plain 1, sh 6, ts 2 | +| `tutorials/live-dashboard/02-contract-to-service.md` | json 1, sh 4, ts 3 | +| `tutorials/live-dashboard/03-sdk-cache-first-query.md` | sh 2, ts 2 | +| `tutorials/live-dashboard/04-definePage-QueryIsland.md` | sh 2, ts 1, tsx 4 | +| `tutorials/live-dashboard/05-live-stream.md` | sh 4, ts 1, tsx 4 | +| `tutorials/live-dashboard/06-deploy.md` | plain 2, sh 5 | +| `tutorials/live-dashboard/index.md` | sh 2 | +| `tutorials/storefront/01-scaffold.md` | sh 8 | +| `tutorials/storefront/02-catalog-service.md` | sh 7, ts 5 | +| `tutorials/storefront/03-cart-contracts.md` | sh 7, ts 4 | +| `tutorials/storefront/04-checkout-saga.md` | sh 8, ts 3 | +| `tutorials/storefront/05-shipping-webhook.md` | bash 1, sh 9, ts 2 | +| `tutorials/storefront/06-storefront-ui.md` | sh 2, ts 2, tsx 2 | +| `tutorials/storefront/07-deploy.md` | sh 5 | +| `tutorials/workspace/01-scaffold.md` | plain 1, sh 6 | +| `tutorials/workspace/02-auth.md` | sh 8 | +| `tutorials/workspace/03-workspace-data.md` | prisma 1, sh 4, ts 1 | +| `tutorials/workspace/04-provision-job.md` | sh 5, ts 1 | +| `tutorials/workspace/05-route-authz.md` | json 1, sh 1, ts 3 | +| `tutorials/workspace/06-deploy.md` | sh 4 | +| `web-layer/builders.md` | ts 2, tsx 1 | +| `web-layer/defer-streaming-ui.md` | ts 2, tsx 3 | +| `web-layer/error.md` | ts 3, tsx 3 | +| `web-layer/examples.md` | ts 1, tsx 1 | +| `web-layer/form.md` | tsx 6 | +| `web-layer/fresh-ui.md` | ts 1 | +| `web-layer/how-to/build-a-desktop-frontend.md` | bash 1, ts 3, tsx 2 | +| `web-layer/how-to/build-a-server-validated-form.md` | ts 1, tsx 2 | +| `web-layer/how-to/customize-fresh-ui.md` | bash 2 | +| `web-layer/interactive.md` | tsx 2 | +| `web-layer/layers.md` | ts 2, tsx 7 | +| `web-layer/partials.md` | tsx 6 | +| `web-layer/query-bridge.md` | text 1, ts 5, tsx 5 | +| `web-layer/query.md` | ts 3, tsx 4 | +| `web-layer/resources.md` | text 1, ts 1, tsx 4 | +| `web-layer/response.md` | text 1, ts 1, tsx 8 | +| `web-layer/route.md` | text 5, ts 4, tsx 3 | +| `web-layer/server.md` | ts 6 | +| `web-layer/testing.md` | ts 4 | +| `web-layer/vite.md` | ts 6 | + +## Tier-1 census and planned day-one disposition + +| Page | `ts` | `tsx` | Checked | Exempt | Reason summary | +| --- | ---: | ---: | ---: | ---: | --- | +| `quickstart.vto` | 1 | 0 | 1 | 0 | Typed generated-schema fixture supplies the scaffold alias. | +| `index.vto` | 0 | 0 | 0 | 0 | Coverage-floor page is asserted present even with no TS fence. | +| `services-sdk/sdk.md` | 1 | 0 | 1 | 0 | Typed contract fixture supplies the app contract alias. | +| `services-sdk/how-to/add-a-service.md` | 6 | 0 | 6 | 0 | The broken barrel excerpt is corrected to import and re-export its local binding. | +| `web-layer/query.md` | 3 | 4 | 6 | 1 | Typed support modules cover both primary island examples; one cache fragment remains deliberately partial. | +| `web-layer/examples.md` | 1 | 1 | 1 | 1 | Cache-projection fragment assumes loader values. | +| `web-layer/interactive.md` | 0 | 2 | 2 | 0 | Typed support module supplies the app query fixture. | +| `web-layer/form.md` | 0 | 6 | 3 | 3 | Counter-example and two state-only fragments are intentionally partial. | +| `web-layer/query-bridge.md` | 5 | 5 | 1 | 9 | Counter-examples, pseudocode, and chain fragments are intentionally partial. | +| **Total** | **17** | **18** | **21** | **14** | **35 Tier-1 candidates** | + +The gate will also report **260 TS-like blocks outside the Tier-1 floor**. That separates marked +exemptions from not-yet-covered work instead of presenting 21 checked blocks as site-wide coverage. + +## Current checker responsibilities + +### `check-accuracy-and-discoverability.ts` + +- Source/API regexes for `defineSaga` and `spawn`. +- Positive API/text needles for saga payload/return shape, preferred how-to paths, + `--with-client`, quickstart paths, the query-dialect exception, and 18 CLI mutation families. +- Forbidden stale claims: old saga calls/sends, retired client paths/aliases, and treating + `apps/<app>/client.ts` as a data client. +- A manual `ALLOWED_FRESH_ROOT_SYMBOLS` set and import regex. +- Mutation-map heading and five required columns. +- A subprocess invocation of `check-exports-drift.ts`. + +### `check-exports-drift.ts` + +- Eight fixed package/reference mappings: `fresh-ui`, `plugin`, `config`, `contracts`, `queue`, + `sdk`, `service`, and `telemetry`. +- Exact package entrypoint name/path comparison for all eight. +- `deno doc --json` symbol comparison only for `config`, `contracts`, and `telemetry`; the other + five explicitly set `checkSymbols: false`. +- This PR leaves its mapping and symbol policy unchanged because #1108 owns expansion. + +## Workflow wiring + +- `pages.yml`: PR/push paths are only `docs/site/**` and the workflow. The build job runs Lume, + internal links, and caveat checks. Draft PRs are not excluded. +- `ci.yml`: draft PRs schedule no jobs. Non-draft `quality` runs for `needs_deno || needs_docs` and + includes `docs:accuracy`; package/plugin source sets `needs_deno`. It does not compile site code + fences. + +## jsr-audit surface scan + +N/A. This is an internal docs tooling and workflow slice; it changes no package/plugin export or +publish surface. The snippet resolver consumes declared export maps but does not publish a new +package. + +## Open questions + +None that would force implementation rework. All design decisions are locked in `plan.md`. diff --git a/.llm/runs/test-1374-docs-compile-gate--leaf/supervisor.md b/.llm/runs/test-1374-docs-compile-gate--leaf/supervisor.md new file mode 100644 index 0000000000..cbe2cb1778 --- /dev/null +++ b/.llm/runs/test-1374-docs-compile-gate--leaf/supervisor.md @@ -0,0 +1,35 @@ +# Supervisor Identity — test-1374-docs-compile-gate--leaf + +| Field | Value | +| --- | --- | +| Model | Codex / GPT-5.6 Sol / high | +| Session | Current Codex thread; runtime thread identifier is not exposed in this session | +| Host | Linux / WSL | +| Checkout | `/home/codex/repos/ns006-1374-compilegate` | +| Worktree | `/home/codex/repos/ns006-1374-compilegate` | +| Branch | `test/1374-docs-snippet-compile-gate` | +| Baseline | `01aa12b67e36b643e1ca4f94421ecba07e030db5` (`origin/main`, 2026-08-12) | +| Run ID | `test-1374-docs-compile-gate--leaf` | + +## Routes in force + +| Task lane | Provider / model / effort | Role in this run | +| --- | --- | --- | +| `complex_implementation` | OpenAI / GPT-5.6 Sol / high | Plan generator and, only after PLAN-EVAL PASS, implementer | +| `formal_plan_evaluation` | Anthropic / Fable 5 / medium | Separate-session PLAN-EVAL of Codex-authored plan | +| `formal_impl_evaluation` | Anthropic / Fable 5 / medium | Separate-session IMPL-EVAL of Codex-authored implementation | + +Reference `.llm/harness/workflow/lane-policy.md`; the generator does not evaluate itself. + +## Authority and handoff + +- Issue: `#1374` +- Lane: documentation compile-gate leaf; complex implementation, plan-first +- Draft PR base: `main` +- Merge authority: orchestrator; this lane must not mark ready, merge, or release +- Push refspec: `HEAD:refs/heads/test/1374-docs-snippet-compile-gate` + +## Phase State + +- Current: Phase 1 — research and plan +- Implementation: blocked by design until a separate-session `PASS` is recorded in `plan-eval.md` diff --git a/.llm/runs/test-1374-docs-compile-gate--leaf/worklog.md b/.llm/runs/test-1374-docs-compile-gate--leaf/worklog.md new file mode 100644 index 0000000000..d8738708b2 --- /dev/null +++ b/.llm/runs/test-1374-docs-compile-gate--leaf/worklog.md @@ -0,0 +1,248 @@ +# Worklog: docs snippet compile gate for #1374 + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `test-1374-docs-compile-gate--leaf` | +| Branch | `test/1374-docs-snippet-compile-gate` | +| Archetype | N/A — internal docs tooling | +| Scope overlays | `SCOPE-docs` | + +## Design + +Recorded before implementation. Implementation is prohibited until separate-session PLAN-EVAL +returns `PASS`. + +### Public Surface + +- Root task `docs:snippets` — scan, compile, and print the Tier-1/corpus census. +- Root task `docs:snippets:test` — focused parser/compiler/fixture tests. +- Root task/CLI fixture mode `docs:snippets:negative <case>` — require one of five fixture names, + run exactly one control, and preserve its raw exit. +- Internal checker exports used by its tests: fence extraction, exact workspace export-map + derivation, synthetic workspace assembly, and check-result/census reporting. + +### Domain Vocabulary + +- `FencedBlock` — source path, opening/closing lines, language, info string, and exact body. +- `Exemption` — a checked-language block with a non-empty inline reason. +- `CoveragePolicy` — Tier-1 page set and exemption budget. +- `SnippetCensus` — scanned fences, TS-like candidates (`ts`/`tsx`/`typescript`), checked, exempt, + outside-floor, and malformed counts. +- `SyntheticModule` — source block plus page-isolated generated path and diagnostic offset. +- `WorkspaceEntrypointMap` — exact public specifier-to-declared-file mapping. +- `SnippetCheckResult` — subprocess exit, diagnostics, census, and generated-to-source map. + +### Ports + +- No custom architecture port. Deno's filesystem/temp APIs and `Deno.Command` are the actual + platform boundaries. Tests exercise the real CLI in temp fixture roots rather than mocking the + compiler. + +### Constants + +- `CHECKED_LANGUAGES` — `ts`, `tsx`, and `typescript`, with `typescript` normalized to `.ts`. +- `TIER_1_PAGES` — the nine issue-mandated paths, including zero-block `index.vto`. +- `NO_CHECK_MARKER` — exact `no-check:<nonblank reason>` grammar from plan D3. +- `TIER_1_MINIMUM_CHECKED` / `TIER_1_MINIMUM_CANDIDATES` — 21 / 35 at the initial floor. +- `TIER_1_EXEMPTION_BUDGET` — maximum 14 at the initial floor. +- `NEGATIVE_FIXTURES` — the three red controls (`non-exported-symbol`, + `empty-exemption-reason`, `dialect-a-object-input`) and two green dialect discriminators + (`dialect-a-positional`, `dialect-b-object-input`). + +### Commit Slices + +| # | Slice | Gate | Files | +| --- | --- | --- | --- | +| 1 | Extractor, policy, census, marker grammar, missing-reason red control | Scoped wrappers; focused test task; direct empty-reason red exit | Checker/test/policy/fixture, `deno.json`, run artifacts | +| 2 | Exact export resolver, synthetic compiler/support, export/dialect controls | Scoped wrappers; focused tests; two direct compiler red exits | Checker/test/support fixtures, run artifacts | +| 3 | Tier-1 markers/green gate, expansion doc, accuracy demotion | Snippet task; docs accuracy; docs links; scoped wrappers | Tier-1 docs, accuracy checker/test, coverage doc, run artifacts | +| 4 | Pages package/plugin trigger and final gate set | All requested gates including repo tests and three raw red exits | Pages workflow/test, run artifacts | + +### Deferred Scope + +- 260 TS-like blocks outside the Tier-1 floor — section-sized expansion ratchet in plan D8. +- Seven `typescript` fences are aliases now; canonical spelling cleanup remains in the reference + expansion wave. +- README/install/export-corpus work — owned by #1377/#1343/#1108/#1531. + +### Contributor Path + +Read `.llm/tools/docs/snippet-coverage.md`, add a page to the checked policy, run +`deno task docs:snippets`, make complete blocks compile through public imports, add a reason marker +only to a deliberate fragment, lower the recorded outside-floor/exemption census, then run the +focused negative controls before the repo suite. + +## Progress Log + +| Date | Slice | Step | Notes | +| --- | --- | --- | --- | +| 2026-08-12 | Phase 1 | Bootstrap | `supervisor.md` written first; branch and baseline match dispatch. | +| 2026-08-12 | Phase 1 | Research | Live issue read; workspace, 578-fence corpus, checkers, and workflows inspected. | +| 2026-08-12 | Phase 1 | Original Plan & Design | Cycle-1 submission locked 35 Tier-1 = 18 checked + 17 exempt; superseded after evaluator probes. | +| 2026-08-12 | PLAN-EVAL cycle 1 | `FAIL_PLAN` | Opposite-family evaluator built and ran D2, confirmed four blocking defects and four non-blocking gaps; no implementation started. | +| 2026-08-12 | Phase 1 revision | Plan & Design | D2 now uses a copied temporary lock without `--frozen` plus root-catalog fallback; containment/Fresh guards remain; `typescript` is covered; the barrel bug and two island examples move to checked. Revised floor is 35 = 21 checked + 14 exempt. | +| 2026-08-12 | PLAN-EVAL cycle 2 | `PASS` | Fresh opposite-family session verified every cycle-1 fix by execution. Two mandatory additive D2 mechanics were recorded before implementation: carry the root catalog section and canonicalize equivalent import ranges. | +| 2026-08-12 | Slice 1 | Extractor contract | Added stable source/fence provenance, backtick/tilde parsing, `ts`/`tsx`/`typescript` recognition, exact reason markers, bidirectional coverage floors, the real-corpus census test, and the empty-reason CLI fixture. | +| 2026-08-12 | Slice 2 | Compiler contract | Added exact public-export resolution, canonicalized root/member imports, root-catalog fallback and copied catalog, disposable copied-lock compilation, page-isolated modules, typed query support, and real export/dialect controls. | +| 2026-08-12 | Slice 3 | Tier-1 and demotion | Applied 14 structural reasons, fixed the barrel binding, compiled 21 fences, wrote the five-wave ratchet, retained vocabulary/Fresh/export guards, and removed only the named positive needles. | +| 2026-08-12 | Slice 4 | Pages and final gates | Added both package/plugin trigger arms, the root snippet step before Lume, an unconditional structural workflow test, three raw red exits, and a green 3,193-test repository verdict. | +| 2026-08-12 | Phase 2 handoff | STOP | All four slices committed/pushed/commented; PR body and all 11 truthfully completed #1374 acceptance boxes carry evidence; draft remains at `status:impl` for separate IMPL-EVAL. | +| 2026-08-12 | Pre-merge F-1 | Remediation | Made missing/unknown negative fixture names fail closed with complete usage; added exact task-level regression coverage. Draft remains at `status:impl`; readiness stays orchestrator-owned. | + +## Decisions + +| Decision | Reason | Source | +| --- | --- | --- | +| Exact declared-export import map | Prevent private subpath/symbol false greens while retaining `@netscript/*`. | `plan.md` D2; doctrine A1/A14 | +| Disposable copied lock plus root-catalog fallback | Preserve dependency pins without an impossible frozen synthetic workspace, and resolve catalog-only transitive bare imports. | PLAN-EVAL B1; `plan.md` D2 | +| Page-isolated OS-temp modules | Preserve local imports without cross-page contamination or tracked noise. | `plan.md` D1 | +| Pages owns the package/plugin trigger | Site-owning workflow runs on drafts; core CI currently does not. | `plan.md` D4; workflow research | +| 14 reasoned exemptions | Evaluator classification leaves only structural fragments exempt; the real barrel bug is fixed and both harness-fit island examples receive typed support. | PLAN-EVAL B4/N1; `research.md`; `plan.md` marker table | +| Vocabulary/Fresh guards survive | Compilation cannot enforce dialect page placement, and site-wide compile coverage is not complete. | PLAN-EVAL B2/N2; `plan.md` D5 | + +## Drift + +| Drift | Severity | Logged in drift.md | +| --- | --- | --- | +| Cycle-1 plan assumptions failed concrete evaluator probes | Significant | Yes — D2 frozen lock, catalog fallback, containment, alias bypass, and exemption disposition are recorded. | +| Bare `docs:snippets:negative` fell through to the positive gate | Significant | Yes — missing/unknown cases now fail before analysis and are task-tested. | + +## Gate Results + +### Phase 1 artifact checks + +| Gate | Result | Notes | +| --- | --- | --- | +| Branch/base/raw status | PASS | Branch and both HEAD/base at `01aa12b67`; only run-dir files are new. | +| Fence census | PASS | 578 total; 288 `ts`/`tsx` plus 7 `typescript`; Tier-1 35. Read-only research command. | +| PLAN-EVAL cycle 1 | `FAIL_PLAN` | Four blocking and four non-blocking findings; all addressed in the revised plan artifacts. | +| PLAN-EVAL cycle 2 | PASS | No blocking findings; implementation authorized after the mandatory pre-slice-2 amendments. | +| Implementation gates | NOT_RUN | Correctly deferred until separate PLAN-EVAL `PASS`. | + +### Slice 1 — extractor, marker, and census + +| Gate | Result | Evidence | +| --- | --- | --- | +| Scoped type-check | PASS | Wrapper selected 17 docs-tool TS files; 0 diagnostics; exit 0. | +| Scoped lint | PASS | Wrapper selected 17 docs-tool TS files; 0 findings; exit 0. | +| Scoped format | PASS | Exact docs-tool wrapper selected 17 files; 0 findings after Deno-formatting the selected TS surface; exit 0. | +| Focused tests | PASS | `deno task docs:snippets:test`: 4 passed, 0 failed. | +| Raw empty-reason control | EXPECTED_RED | `deno task docs:snippets:negative empty-exemption-reason`: exit 1; `page.md:1: malformed ts fence: expected no-check:<nonblank reason>`. | +| Lock hygiene | PASS | No `deno.lock` change. | + +Slice review: parser provenance is stable for #1378; recognized aliases share one strict marker +grammar; missing candidates, checked-count loss, and exemption growth are independent failures. No +ambient or cast-based legalization exists. The exact format gate also normalized pre-existing Deno +format drift in `check-exports-drift.ts` and its test without semantic edits. + +Reconcile: PR #1537 remains draft with `status:impl`, `Closes #1374`, milestone `0.0.6`, and the +required type/area/priority labels. New comments contained only the already-recorded #1378 extractor +consumer boundary; no scope or issue-state readjustment was required. + +### Slice 2 — synthetic compiler and public entrypoints + +| Gate | Result | Evidence | +| --- | --- | --- | +| Scoped type-check | PASS | Wrapper selected 21 docs-tool TS files; 0 diagnostics; exit 0. | +| Scoped lint | PASS | Wrapper selected 21 docs-tool TS files; 0 findings; exit 0. | +| Scoped format | PASS | Wrapper selected 21 docs-tool TS files; 0 findings; exit 0. | +| Focused tests | PASS | `deno task docs:snippets:test`: 8 passed, 0 failed. | +| Copied-lock green control | PASS | Dialect-A positional fixture exits 0 with 37 discovered members and 38 copied catalog entries; temporary lock is rewritten while tracked root lock bytes remain unchanged. | +| Raw non-exported-symbol control | EXPECTED_RED | `deno task docs:snippets:negative non-exported-symbol`: exit 1; source fence `page.md:1`; TS2305 names `DefinitelyNotExportedByNetScript`. | +| Raw dialect-A wrong-shape control | EXPECTED_RED | `deno task docs:snippets:negative dialect-a-object-input`: exit 1; source fence `page.md:1`; TS2353 says `input` does not exist in `{ id: string }`. | +| Dialect positive controls | PASS | Direct dialect-A positional and dialect-B `{ input }` fixture invocations both exit 0 through the same typed public-factory support. | +| Lock hygiene | PASS | `deno.lock` is clean after all compiler/test/control runs. | + +Slice review: the generated configuration exposes only exact workspace-declared `@netscript/*` +exports, materializes catalog-only transitive imports, copies the 38-entry root catalog, and compares +canonicalized package requirements. Page-local imports resolve within isolated temporary roots; +diagnostics map generated paths back to source fence lines. Typed support contains no `any` or casts, +and the wrong dialect remains a real excess-property error. + +Reconcile: PR #1537 remains draft at `status:impl`; the only cross-lane instruction still applies: +#1378 consumes the stable extractor provenance surface and does not fork the parser. No new reviewer +comment changed the locked slice plan. + +### Slice 3 — Tier-1 floor, expansion plan, and accuracy demotion + +| Gate | Result | Evidence | +| --- | --- | --- | +| Tier-1 snippet gate | PASS | `deno task docs:snippets`: exit 0; `scanned=578 ts=211 tsx=77 typescript=7 ts_like=295 tier1=35 checked=21 exempt=14 outside_floor=260 malformed=0`; all 14 reasons printed. | +| Focused snippet tests | PASS | `deno task docs:snippets:test`: 8 passed, 0 failed, including independent candidate/checked/exemption floors and the `typescript` alias regression. | +| Accuracy unit tests | PASS | Focused accuracy test: 4 passed, 0 failed; covers stale saga claims, golden-path containment, Fresh-root imports, and mutation-map columns. | +| Docs accuracy | PASS | `deno task docs:accuracy`: exit 0; 4 saga pages, 192 published pages, exact query exception page, mutation columns, 3 Fresh-root imports; export drift subprocess remains green. | +| Docs links | PASS | `deno task docs:links`: exit 0; 102 docs, 0 broken links, 0 broken anchors. | +| Scoped type-check | PASS | Wrapper selected 21 docs-tool TS files; 0 diagnostics; exit 0. | +| Scoped lint | PASS | Wrapper selected 21 docs-tool TS files; 0 findings; exit 0. | +| Scoped format | PASS | Wrapper selected 21 docs-tool TS files; 0 findings; exit 0. | +| Lock hygiene | PASS | `deno.lock` remains clean. | + +The first real-corpus compile was usefully red: TS2307 named the missing documented relative support +modules at `web-layer/examples.md:40` and `web-layer/interactive.md:92`. Both are now generated from +typed public contracts/client/query factories with no casts or `any`; the checked count stayed 21. +The add-service barrel correction compiles unmarked. The expansion document records all five waves +and the deliberate wave-4 positive-reference-presence window. + +Accuracy demotion preserved the four orphan-send forms, object-form `defineSaga`, the stale service +kind, all golden-path forbidden aliases/CSS-client claims, exact one-page +`createServiceQueryUtils` containment, the five mutation-map columns, site-wide Fresh-root imports, +and the unchanged export-drift subprocess. Only the named positive API/presence needles were removed. + +### Slice 4 — Pages trigger and final gate set + +| Gate | Result | Evidence | +| --- | --- | --- | +| Pages workflow contract | PASS | `pages-workflow_test.ts` structurally parses both event path lists and build steps; both arms include docs/package/plugin/tool/config paths, and root `docs:snippets` precedes the `docs/site` Lume build. | +| Focused snippet/workflow tests | PASS | `deno task docs:snippets:test`: 9 passed, 0 failed. | +| Scoped type-check | PASS | Wrapper selected 22 docs-tool TS files; 0 diagnostics; exit 0. | +| Scoped lint | PASS | Wrapper selected 22 docs-tool TS files; 0 findings; exit 0. | +| Scoped format | PASS | Wrapper selected 22 docs-tool TS files; 0 findings; exit 0. | +| Docs links | PASS | Exit 0; 102 docs, 0 broken links, 0 broken anchors. | +| Docs accuracy | PASS | Exit 0; textual policies and export drift green. | +| Tier-1 snippet gate | PASS | Exit 0; exact census `578/211/77/7/295/35/21/14/260/0`; all 14 exemption reasons printed. | +| Raw empty-reason control | EXPECTED_RED | Exit 1; `page.md:1: malformed ts fence: expected no-check:<nonblank reason>`. | +| Raw non-exported-symbol control | EXPECTED_RED | Exit 1; source fence `page.md:1`; TS2305 names `DefinitelyNotExportedByNetScript`. | +| Raw dialect-A wrong-shape control | EXPECTED_RED | Exit 1; source fence `page.md:1`; TS2353 names the unsupported `input` property. | +| Repository suite | PASS | `deno task test`: 3,193 passed (617 steps), 0 failed, 17 ignored; exit 0 in 3m13s. | +| Lock hygiene | PASS | The full gate set left tracked `deno.lock` clean. | + +Final review: `.github/workflows/pages.yml` owns the compile step and both package/plugin trigger +arms; `.github/workflows/ci.yml` is deliberately unchanged and retains its existing accuracy job. +The structural assertion is part of both `docs:snippets:test` and root test discovery. Generated +modules remain temporary and compilation-only. No service, network validation, or E2E CLI suite was +run; those are outside this slice and the brief explicitly excludes `e2e:cli`. + +Reconcile: PR #1537 remains draft at `status:impl` with `Closes #1374`, milestone `0.0.6`, and the +required type/area/priority labels. No reviewer comment altered scope. Phase 2 is complete locally; +the next authority is a fresh opposite-family IMPL-EVAL session. + +External evidence mirror: the draft PR body now records commits, census, gates, red controls, drift, +and the unchecked IMPL-EVAL step. All 11 #1374 acceptance boxes are checked with one evidence line +each. The issue remains open and the PR remains draft; no readiness or merge transition was made. + +### Orchestrator pre-merge F-1 — fail-closed negative task + +| Gate | Result | Evidence | +| --- | --- | --- | +| Bare negative task | EXPECTED_RED | Raw `deno task docs:snippets:negative`: exit 1; `missing negative fixture name`; complete five-case usage printed. | +| Unknown negative fixture | EXPECTED_RED | Raw task with `not-a-fixture`: exit 1; `unknown negative fixture name`; complete five-case usage printed. | +| Focused snippet tests | PASS | `deno task docs:snippets:test`: 10 passed, 0 failed; exact task-level missing/unknown regression included. | +| Scoped type-check | PASS | Wrapper selected 22 docs-tool TS files; 0 diagnostics; exit 0. | +| Scoped lint | PASS | Wrapper selected 22 docs-tool TS files; 0 findings; exit 0. | +| Scoped format | PASS | Wrapper selected 22 docs-tool TS files; 0 findings after formatting only the two owned TS files; exit 0. | +| Lock hygiene | PASS | Raw `git diff --exit-code -- deno.lock`: exit 0. | + +Remediation review: `--negative` is now a strict mode switch, not an optional fixture hint. Its +argument is validated against the finite five-case vocabulary before site-root selection or +coverage analysis, so neither omission nor invention can reach the positive PASS path. The test +spawns the exact root task for both failures and verifies the complete usage list. + +## Handoff Notes + +- Phase 2 is complete; Slice 4 is `5ce34c089` and its push/comment boundary is closed. +- F-1 is focused, gated, committed, explicitly pushed, and recorded in the final PR implementation + comment. +- Keep the PR draft. The orchestrator owns the ready transition, which starts the automatic initial + IMPL-EVAL; this session must not dispatch a duplicate evaluator or merge. diff --git a/.llm/tools/agentic/openhands/impl-eval-prompt.md b/.llm/tools/agentic/openhands/impl-eval-prompt.md new file mode 100644 index 0000000000..25db641ec5 --- /dev/null +++ b/.llm/tools/agentic/openhands/impl-eval-prompt.md @@ -0,0 +1,22 @@ +use harness + +## SKILL + +- `netscript-harness` — apply the formal IMPL-EVAL protocol and verdict vocabulary. +- `openhands-handoff` — publish one machine-readable OpenHands verdict. +- `netscript-tools` — run the smallest decisive repository-native gates without mutating source. +- `netscript-doctrine` — apply package/plugin doctrine when the changed surface requires it. + +Act as the formal IMPL-EVAL session for this pull request. Do not edit files, create commits, push, +or repair findings. The trigger metadata supplies the trusted base SHA and immutable head SHA: read +the evaluator protocol, verdict definitions, and selected profiles from that base commit, then +evaluate the PR body, linked issues, run artifacts, final diff, review threads, and architecture +debt at the immutable head. Verify the approved plan or recorded `PLAN-EVAL: N/A`, design +checkpoint, acceptance criteria, static/runtime/consumer gates, public surface, lock hygiene, and +false-done states. For documentation changes, also read every changed document fully and hand-test +representative executable claims. + +Return concise, severity-ranked findings with exact evidence and required action. End with exactly +one supported verdict line using `OPENHANDS_VERDICT: PASS`, `OPENHANDS_VERDICT: FAIL_FIX`, +`OPENHANDS_VERDICT: FAIL_RESCOPE`, `OPENHANDS_VERDICT: FAIL_DEBT`, or +`OPENHANDS_VERDICT: FAIL_PLAN`. Write the same verdict to `OPENHANDS_SUMMARY_PATH`. diff --git a/.llm/tools/agentic/openhands/phase-eval-workflow_test.ts b/.llm/tools/agentic/openhands/phase-eval-workflow_test.ts new file mode 100644 index 0000000000..bc26216e53 --- /dev/null +++ b/.llm/tools/agentic/openhands/phase-eval-workflow_test.ts @@ -0,0 +1,183 @@ +import { assert, assertEquals, assertRejects, assertStringIncludes } from '@std/assert'; +import { OPEN_EVALUATOR_MODEL_IDS, OPENROUTER_MODEL_IDS } from '../config/models.ts'; + +type Event = { + action: 'ready_for_review' | 'labeled' | 'synchronize'; + label?: string; + labels: string[]; + draft?: boolean; +}; + +type PhaseDecision = 'dispatch-plan' | 'dispatch-impl' | 'skip-impl' | 'ignore'; + +function decision(event: Event): PhaseDecision { + if (event.action === 'ready_for_review') { + return event.labels.includes('impl-eval:skip') ? 'skip-impl' : 'dispatch-impl'; + } + if (event.action !== 'labeled') return 'ignore'; + const hasOpenHands = event.labels.includes('openhands'); + const completesPlanPair = hasOpenHands && + (event.label === 'status:plan-eval' || + (event.label === 'openhands' && event.labels.includes('status:plan-eval'))); + if (completesPlanPair) return 'dispatch-plan'; + return !event.draft && event.label === 'status:impl-eval' ? 'dispatch-impl' : 'ignore'; +} + +const MODELS = new Map([ + ['eval:model:minimax', `openrouter/${OPENROUTER_MODEL_IDS.minimax}`], + ['eval:model:deepseek', `openrouter/${OPENROUTER_MODEL_IDS.deepseekV4Flash0731}`], + ['eval:model:qwen', `openrouter/${OPENROUTER_MODEL_IDS.qwen}`], +]); + +async function selectModel(phase: 'plan' | 'impl', labels: string[]): Promise<string> { + const selected = labels.filter((label) => label.startsWith('eval:model:')); + if (selected.some((label) => !MODELS.has(label))) throw new Error('unknown evaluator label'); + if (selected.length > 1) throw new Error('conflicting evaluator labels'); + return selected.length + ? MODELS.get(selected[0])! + : phase === 'plan' + ? `openrouter/${OPENROUTER_MODEL_IDS.minimax}` + : `openrouter/${OPENROUTER_MODEL_IDS.deepseekV4Flash0731}`; +} + +function nextStatus( + phase: 'plan' | 'impl', + verdict: 'PASS' | 'FAIL_FIX' | 'FAIL_RESCOPE' | 'FAIL_DEBT' | 'FAIL_PLAN' | 'NONE', +): string | null { + if (verdict === 'NONE') return null; + if (phase === 'plan') return verdict === 'PASS' ? 'status:impl' : 'status:plan'; + if (verdict === 'PASS') return 'status:augment-review'; + return verdict === 'FAIL_RESCOPE' || verdict === 'FAIL_PLAN' ? 'status:plan' : 'status:impl'; +} + +Deno.test('phase evaluator event matrix dispatches only deliberate transitions', () => { + assertEquals(decision({ action: 'ready_for_review', labels: [] }), 'dispatch-impl'); + assertEquals( + decision({ action: 'ready_for_review', labels: ['impl-eval:skip'] }), + 'skip-impl', + ); + assertEquals( + decision({ + action: 'labeled', + label: 'status:plan-eval', + labels: ['status:plan-eval', 'openhands'], + draft: true, + }), + 'dispatch-plan', + ); + assertEquals( + decision({ + action: 'labeled', + label: 'openhands', + labels: ['status:plan-eval', 'openhands'], + draft: true, + }), + 'dispatch-plan', + ); + assertEquals( + decision({ action: 'labeled', label: 'status:plan-eval', labels: ['status:plan-eval'] }), + 'ignore', + ); + assertEquals( + decision({ + action: 'labeled', + label: 'status:impl-eval', + labels: ['status:impl-eval'], + draft: false, + }), + 'dispatch-impl', + ); + assertEquals( + decision({ + action: 'labeled', + label: 'status:impl-eval', + labels: ['status:impl-eval', 'openhands'], + draft: true, + }), + 'ignore', + ); + assertEquals( + decision({ action: 'labeled', label: 'eval:model:qwen', labels: ['eval:model:qwen'] }), + 'ignore', + ); + assertEquals(decision({ action: 'synchronize', labels: ['status:impl-eval'] }), 'ignore'); +}); + +Deno.test('phase evaluator model labels are exact, optional, and mutually exclusive', async () => { + assertEquals(await selectModel('plan', []), `openrouter/${OPENROUTER_MODEL_IDS.minimax}`); + assertEquals( + await selectModel('impl', []), + `openrouter/${OPENROUTER_MODEL_IDS.deepseekV4Flash0731}`, + ); + for (const [label, model] of MODELS) assertEquals(await selectModel('impl', [label]), model); + await assertRejects(() => selectModel('impl', ['eval:model:not-real'])); + await assertRejects(() => selectModel('impl', ['eval:model:deepseek', 'eval:model:qwen'])); +}); + +Deno.test('formal verdicts advance or return the harness status deterministically', () => { + assertEquals(nextStatus('plan', 'PASS'), 'status:impl'); + assertEquals(nextStatus('plan', 'FAIL_PLAN'), 'status:plan'); + assertEquals(nextStatus('plan', 'FAIL_RESCOPE'), 'status:plan'); + assertEquals(nextStatus('impl', 'PASS'), 'status:augment-review'); + assertEquals(nextStatus('impl', 'FAIL_FIX'), 'status:impl'); + assertEquals(nextStatus('impl', 'FAIL_DEBT'), 'status:impl'); + assertEquals(nextStatus('impl', 'FAIL_RESCOPE'), 'status:plan'); + assertEquals(nextStatus('impl', 'FAIL_PLAN'), 'status:plan'); + assertEquals(nextStatus('impl', 'NONE'), null); +}); + +Deno.test('workflow source encodes trusted, exactly-once phase dispatch', async () => { + const phase = await Deno.readTextFile('.github/workflows/openhands-phase-eval.yml'); + const runner = await Deno.readTextFile('.github/workflows/openhands-agent.yml'); + assertStringIncludes(phase, 'types: [labeled, ready_for_review]'); + assert(!phase.includes('synchronize')); + assertStringIncludes(phase, "contains(github.event.pull_request.labels.*.name, 'openhands')"); + assertStringIncludes( + phase, + "contains(github.event.pull_request.labels.*.name, 'impl-eval:skip')", + ); + assertStringIncludes(phase, 'ref: `heads/${pr.base.ref}`'); + assertStringIncludes(phase, 'ref: trustedBaseSha'); + assertStringIncludes(phase, 'Trusted base SHA: ${trustedBaseSha}'); + assert(!phase.includes('ref: pr.base.sha')); + assertStringIncludes(phase, 'openhands-phase-eval generation=${generationEvent.id}'); + assertStringIncludes(phase, '@openhands-agent model=${model}'); + assertStringIncludes(phase, 'head=${pr.head.sha}'); + assertStringIncludes(phase, 'name: selectedLabel'); + assertStringIncludes(phase, 'github-token: ${{ secrets.PAT_TOKEN }}'); + assertStringIncludes(runner, "steps.request.outputs.eval_phase != ''"); + assertStringIncludes(runner, "steps.request.outputs.eval_phase == ''"); + assertStringIncludes(runner, "'status:augment-review'"); + assertStringIncludes(runner, 'pr.head.sha !== evaluatedHead'); + assertStringIncludes(runner, 'github-token: ${{ secrets.PAT_TOKEN }}'); + assertStringIncludes( + runner, + "label.name == 'openhands' && github.event_name == 'issues'", + ); + assert(!runner.includes('types: [labeled, ready_for_review]')); +}); + +Deno.test('generic OpenHands stays fail-closed to current open evaluator models', async () => { + const workflow = await Deno.readTextFile('.github/workflows/openhands-agent.yml'); + for (const model of OPEN_EVALUATOR_MODEL_IDS) { + assertStringIncludes(workflow, `'openrouter/${model}'`); + } + assertStringIncludes(workflow, 'uv pip install --system fastapi'); + assert(!workflow.includes("['sonnet', 'anthropic/claude-sonnet-4']")); + assert(!workflow.includes("['gpt', 'openai/gpt-5.1']")); + assert(!workflow.includes("['gemini', 'gemini/gemini-2.5-pro']")); +}); + +Deno.test('formal evaluator prompts are trusted read-only harness contracts', async () => { + for ( + const path of [ + '.llm/tools/agentic/openhands/plan-eval-prompt.md', + '.llm/tools/agentic/openhands/impl-eval-prompt.md', + ] + ) { + const prompt = await Deno.readTextFile(path); + assert(prompt.startsWith('use harness\n')); + assertStringIncludes(prompt, 'Do not edit files, create commits, push'); + assertStringIncludes(prompt, 'OPENHANDS_SUMMARY_PATH'); + } +}); diff --git a/.llm/tools/agentic/openhands/plan-eval-prompt.md b/.llm/tools/agentic/openhands/plan-eval-prompt.md new file mode 100644 index 0000000000..26b96a5890 --- /dev/null +++ b/.llm/tools/agentic/openhands/plan-eval-prompt.md @@ -0,0 +1,18 @@ +use harness + +## SKILL + +- `netscript-harness` — apply the Plan-Gate and preserve evaluator independence. +- `openhands-handoff` — publish one machine-readable OpenHands verdict. +- `netscript-tools` — use repository-native, read-only evidence commands. + +Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push, +or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head +SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then +evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture, +tests, consumer proof, release risk, and false-done states. Verify claims directly where useful. + +Return concise findings with exact evidence and required amendments. End with exactly one supported +verdict line using `OPENHANDS_VERDICT: PASS`, `OPENHANDS_VERDICT: FAIL_PLAN`, or +`OPENHANDS_VERDICT: FAIL_RESCOPE`. Write the same verdict to `OPENHANDS_SUMMARY_PATH`. This is a +planning gate: never emit `FAIL_FIX` for implementation that has not started. diff --git a/.llm/tools/docs/check-snippets.ts b/.llm/tools/docs/check-snippets.ts new file mode 100644 index 0000000000..cbde5a1ea7 --- /dev/null +++ b/.llm/tools/docs/check-snippets.ts @@ -0,0 +1,70 @@ +import { dirname, fromFileUrl, join } from '@std/path'; +import { compileSnippetAnalysis } from './snippet-compiler.ts'; +import { analyzeSnippetSite, formatSnippetCensus } from './snippet-policy.ts'; + +const repositoryRoot = dirname(dirname(dirname(dirname(fromFileUrl(import.meta.url))))); + +const NEGATIVE_FIXTURE_NAMES = [ + 'non-exported-symbol', + 'empty-exemption-reason', + 'dialect-a-object-input', + 'dialect-a-positional', + 'dialect-b-object-input', +] as const; +const negativeFixtureNames = new Set<string>(NEGATIVE_FIXTURE_NAMES); +const negativeUsage = `usage: deno task docs:snippets:negative <case>\ncases: ${ + NEGATIVE_FIXTURE_NAMES.join(', ') +}`; + +function argumentAfter(args: string[], flag: string): string | undefined { + const index = args.indexOf(flag); + return index >= 0 ? args[index + 1] : undefined; +} + +function requestedNegativeFixture(args: string[]): string | undefined { + if (!args.includes('--negative')) { + return undefined; + } + + const name = argumentAfter(args, '--negative'); + if (name === undefined || name.startsWith('--')) { + throw new Error(`missing negative fixture name\n${negativeUsage}`); + } + if (!negativeFixtureNames.has(name)) { + throw new Error(`unknown negative fixture name ${JSON.stringify(name)}\n${negativeUsage}`); + } + return name; +} + +async function main(args: string[]): Promise<void> { + const negativeCase = requestedNegativeFixture(args); + const siteRoot = negativeCase + ? join(repositoryRoot, '.llm/tools/docs/fixtures', negativeCase, 'docs/site') + : argumentAfter(args, '--site-root') ?? join(repositoryRoot, 'docs/site'); + const analysis = await analyzeSnippetSite(siteRoot, { + enforceCoverage: negativeCase === undefined, + }); + + if (!args.includes('--extract-only')) { + const result = await compileSnippetAnalysis(analysis, repositoryRoot); + if (result.code !== 0) { + console.error(formatSnippetCensus(analysis.census, 'FAIL')); + console.error(result.diagnostics.trimEnd()); + Deno.exit(result.code); + } + } + + console.log(formatSnippetCensus(analysis.census)); + for (const block of analysis.exemptions) { + console.log(`${block.sourcePath}:${block.openingLine} — ${block.exemptionReason}`); + } +} + +if (import.meta.main) { + try { + await main(Deno.args); + } catch (error) { + console.error(`docs snippets: FAIL ${error instanceof Error ? error.message : String(error)}`); + Deno.exit(1); + } +} diff --git a/.llm/tools/docs/fixtures/dialect-a-object-input/docs/site/page.md b/.llm/tools/docs/fixtures/dialect-a-object-input/docs/site/page.md new file mode 100644 index 0000000000..4da0ee6aa3 --- /dev/null +++ b/.llm/tools/docs/fixtures/dialect-a-object-input/docs/site/page.md @@ -0,0 +1,5 @@ +```ts +import { docsQueries } from '../lib/docs.ts'; + +docsQueries.getById.queryOptions({ input: { id: 'doc-1' } }); +``` diff --git a/.llm/tools/docs/fixtures/dialect-a-positional/docs/site/page.md b/.llm/tools/docs/fixtures/dialect-a-positional/docs/site/page.md new file mode 100644 index 0000000000..d4894452b1 --- /dev/null +++ b/.llm/tools/docs/fixtures/dialect-a-positional/docs/site/page.md @@ -0,0 +1,5 @@ +```ts +import { docsQueries } from '../lib/docs.ts'; + +docsQueries.getById.queryOptions({ id: 'doc-1' }); +``` diff --git a/.llm/tools/docs/fixtures/dialect-b-object-input/docs/site/page.md b/.llm/tools/docs/fixtures/dialect-b-object-input/docs/site/page.md new file mode 100644 index 0000000000..e4ce43fdf3 --- /dev/null +++ b/.llm/tools/docs/fixtures/dialect-b-object-input/docs/site/page.md @@ -0,0 +1,7 @@ +```ts +import { createServiceQueryUtils } from '@netscript/sdk/query-client'; +import { docsClient } from '../lib/docs.ts'; + +const docsQueryUtils = createServiceQueryUtils(docsClient); +docsQueryUtils.getById.queryOptions({ input: { id: 'doc-1' } }); +``` diff --git a/.llm/tools/docs/fixtures/empty-exemption-reason/docs/site/page.md b/.llm/tools/docs/fixtures/empty-exemption-reason/docs/site/page.md new file mode 100644 index 0000000000..3c1a633e17 --- /dev/null +++ b/.llm/tools/docs/fixtures/empty-exemption-reason/docs/site/page.md @@ -0,0 +1,3 @@ +```ts no-check: +const shouldNeverBeSilentlySkipped = true; +``` diff --git a/.llm/tools/docs/fixtures/non-exported-symbol/docs/site/page.md b/.llm/tools/docs/fixtures/non-exported-symbol/docs/site/page.md new file mode 100644 index 0000000000..68675ed10f --- /dev/null +++ b/.llm/tools/docs/fixtures/non-exported-symbol/docs/site/page.md @@ -0,0 +1,5 @@ +```ts +import { DefinitelyNotExportedByNetScript } from '@netscript/sdk'; + +void DefinitelyNotExportedByNetScript; +``` diff --git a/.llm/tools/docs/pages-workflow_test.ts b/.llm/tools/docs/pages-workflow_test.ts new file mode 100644 index 0000000000..f8cadc9cbe --- /dev/null +++ b/.llm/tools/docs/pages-workflow_test.ts @@ -0,0 +1,128 @@ +import { assert, assertEquals } from '@std/assert'; + +interface SourceLine { + number: number; + indent: number; + text: string; +} + +interface WorkflowStep { + name: string; + run?: string; + workingDirectory?: string; +} + +interface PagesWorkflowContract { + eventPaths: Record<'pull_request' | 'push', string[]>; + buildSteps: WorkflowStep[]; +} + +function sourceLines(source: string): SourceLine[] { + return source.split('\n').flatMap((raw, index) => { + if (raw.trim() === '' || raw.trimStart().startsWith('#')) return []; + const indent = raw.length - raw.trimStart().length; + if (indent % 2 !== 0) throw new Error(`pages.yml:${index + 1}: odd indentation`); + return [{ number: index + 1, indent, text: raw.trim() }]; + }); +} + +function sectionEnd(lines: SourceLine[], start: number): number { + const parentIndent = lines[start].indent; + const next = lines.findIndex((line, index) => index > start && line.indent <= parentIndent); + return next < 0 ? lines.length : next; +} + +function childIndex( + lines: SourceLine[], + start: number, + end: number, + indent: number, + key: string, +): number { + const matches = lines.flatMap((line, index) => + index > start && index < end && line.indent === indent && line.text === `${key}:` ? [index] : [] + ); + if (matches.length !== 1) { + throw new Error(`pages.yml: expected one ${key} section at indent ${indent}`); + } + return matches[0]; +} + +function scalar(line: SourceLine, key: string): string | undefined { + const prefix = `${key}:`; + return line.text.startsWith(prefix) ? line.text.slice(prefix.length).trim() : undefined; +} + +function parsePagesWorkflow(source: string): PagesWorkflowContract { + const lines = sourceLines(source); + const root = (key: string) => childIndex(lines, -1, lines.length, 0, key); + const on = root('on'); + const onEnd = sectionEnd(lines, on); + const eventPaths = {} as Record<'pull_request' | 'push', string[]>; + + for (const event of ['pull_request', 'push'] as const) { + const eventStart = childIndex(lines, on, onEnd, 2, event); + const eventEnd = sectionEnd(lines, eventStart); + const pathsStart = childIndex(lines, eventStart, eventEnd, 4, 'paths'); + const pathsEnd = sectionEnd(lines, pathsStart); + eventPaths[event] = lines.slice(pathsStart + 1, pathsEnd).map((line) => { + if (line.indent !== 6 || !line.text.startsWith('- ')) { + throw new Error(`pages.yml:${line.number}: malformed ${event}.paths entry`); + } + return line.text.slice(2); + }); + } + + const jobs = root('jobs'); + const build = childIndex(lines, jobs, sectionEnd(lines, jobs), 2, 'build'); + const steps = childIndex(lines, build, sectionEnd(lines, build), 4, 'steps'); + const stepsEnd = sectionEnd(lines, steps); + const stepStarts = lines.flatMap((line, index) => + index > steps && index < stepsEnd && line.indent === 6 && line.text.startsWith('- name:') + ? [index] + : [] + ); + const buildSteps = stepStarts.map((start, ordinal) => { + const end = stepStarts[ordinal + 1] ?? stepsEnd; + const name = lines[start].text.slice('- name:'.length).trim(); + const fields = lines.slice(start + 1, end).filter((line) => line.indent === 8); + return { + name, + run: fields.map((line) => scalar(line, 'run')).find((value) => value !== undefined), + workingDirectory: fields.map((line) => scalar(line, 'working-directory')).find((value) => + value !== undefined + ), + }; + }); + + return { eventPaths, buildSteps }; +} + +Deno.test('Pages structurally revalidates docs for package and plugin changes before Lume', async () => { + const workflow = parsePagesWorkflow(await Deno.readTextFile('.github/workflows/pages.yml')); + const requiredPaths = [ + 'docs/site/**', + 'packages/**', + 'plugins/**', + '.llm/tools/docs/**', + 'deno.json', + 'deno.lock', + '.github/workflows/pages.yml', + ]; + assertEquals(workflow.eventPaths.pull_request, requiredPaths); + assertEquals(workflow.eventPaths.push, requiredPaths); + + const snippetIndex = workflow.buildSteps.findIndex((step) => + step.run === 'deno task docs:snippets' + ); + const lumeIndex = workflow.buildSteps.findIndex((step) => step.run === 'deno task build'); + assert(snippetIndex >= 0, 'Pages build job must invoke deno task docs:snippets'); + assert(lumeIndex >= 0, 'Pages build job must invoke the Lume build'); + assert(snippetIndex < lumeIndex, 'docs:snippets must run before the Lume build'); + assertEquals( + workflow.buildSteps[snippetIndex].workingDirectory, + undefined, + 'docs:snippets must run from the repository root', + ); + assertEquals(workflow.buildSteps[lumeIndex].workingDirectory, 'docs/site'); +}); diff --git a/.llm/tools/docs/snippet-compiler.ts b/.llm/tools/docs/snippet-compiler.ts new file mode 100644 index 0000000000..20166a4405 --- /dev/null +++ b/.llm/tools/docs/snippet-compiler.ts @@ -0,0 +1,177 @@ +import { dirname, join, relative, toFileUrl } from '@std/path'; +import type { FencedBlock } from './snippet-extractor.ts'; +import type { SnippetSiteAnalysis } from './snippet-policy.ts'; +import { + materializePageSupports, + materializeSharedSupports, + writeSnippetFile, +} from './snippet-supports.ts'; +import { resolveWorkspaceSurface } from './snippet-workspace.ts'; + +interface SyntheticModule { + block: FencedBlock; + path: string; + headerLines: number; +} + +/** Result of one compilation-only documentation check. */ +export interface SnippetCompilationResult { + code: number; + stdout: string; + diagnostics: string; + memberCount: number; + catalogCount: number; + rootLockUnchanged: boolean; + temporaryLockRewritten: boolean; +} + +function pageDirectory(tempRoot: string, sourcePath: string): string { + return join(tempRoot, 'pages', sourcePath.replaceAll(/[^A-Za-z0-9._-]/g, '_')); +} + +function explicitModulePath(block: FencedBlock): string | undefined { + const firstLine = block.body.split('\n', 1)[0] ?? ''; + const match = /^\/\/\s+([^\s]+\.tsx?)(?:\s|$)/.exec(firstLine); + if (!match) return undefined; + const path = match[1].replaceAll('\\', '/'); + if (path.startsWith('/') || path.split('/').includes('..')) { + throw new Error( + `${block.sourcePath}:${block.openingLine}: unsafe synthetic module path ${path}`, + ); + } + return path.replace(/^\.\//, ''); +} + +function relativeImport(fromPath: string, toPath: string): string { + const path = relative(dirname(fromPath), toPath).replaceAll('\\', '/'); + return path.startsWith('.') ? path : `./${path}`; +} + +async function materializeModules( + tempRoot: string, + blocks: FencedBlock[], + preamblePath: string, +): Promise<SyntheticModule[]> { + const modules: SyntheticModule[] = []; + const occupied = new Set<string>(); + for (const block of blocks) { + if (!block.compilationExtension || block.exemptionReason !== undefined) continue; + const pageRoot = pageDirectory(tempRoot, block.sourcePath); + await materializePageSupports(pageRoot); + const relativePath = explicitModulePath(block) ?? + `blocks/${ + block.sourcePath.replaceAll(/[^A-Za-z0-9._-]/g, '_') + }-L${block.openingLine}-B${block.fenceOrdinal}.${block.compilationExtension}`; + const path = join(pageRoot, relativePath); + if (occupied.has(path)) { + throw new Error( + `${block.sourcePath}:${block.openingLine}: duplicate synthetic path ${relativePath}`, + ); + } + occupied.add(path); + await writeSnippetFile( + path, + `// Source fence: ${block.sourcePath}:${block.openingLine}\nimport '${ + relativeImport(path, preamblePath) + }';\n${block.body}\n`, + ); + modules.push({ block, path, headerLines: 2 }); + } + return modules; +} + +function escapeRegExp(value: string): string { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +function mapDiagnostics(raw: string, modules: SyntheticModule[]): string { + let diagnostics = raw; + for (const module of modules) { + for (const generatedPath of [module.path, toFileUrl(module.path).href]) { + diagnostics = diagnostics.replace( + new RegExp(`${escapeRegExp(generatedPath)}:(\\d+):(\\d+)`, 'g'), + (_match, lineText: string, column: string) => { + const generatedLine = Number(lineText); + const sourceLine = module.block.codeStartLine + generatedLine - module.headerLines - 1; + return `${module.block.sourcePath}:${ + Math.max(module.block.openingLine, sourceLine) + }:${column}`; + }, + ); + diagnostics = diagnostics.replaceAll(generatedPath, module.block.sourcePath); + } + } + const fences = modules.map((module) => `${module.block.sourcePath}:${module.block.openingLine}`) + .join(', '); + return `source fences: ${fences}\n${diagnostics}`; +} + +/** Assemble page-isolated synthetic modules and run Deno check without executing snippets. */ +export async function compileSnippetAnalysis( + analysis: SnippetSiteAnalysis, + repositoryRoot: string, +): Promise<SnippetCompilationResult> { + const tempRoot = await Deno.makeTempDir({ prefix: 'netscript-doc-snippets-' }); + const rootLockPath = join(repositoryRoot, 'deno.lock'); + const rootLockBefore = await Deno.readTextFile(rootLockPath); + try { + const preamblePath = join(tempRoot, '_shared/preamble.ts'); + await writeSnippetFile(preamblePath, 'export {};\n'); + const supportImports = await materializeSharedSupports(tempRoot); + const workspace = await resolveWorkspaceSurface(repositoryRoot, supportImports); + const modules = await materializeModules(tempRoot, analysis.tier1Blocks, preamblePath); + const configPath = join(tempRoot, 'deno.json'); + await Deno.writeTextFile( + configPath, + `${ + JSON.stringify( + { + compilerOptions: { + strict: true, + noImplicitAny: true, + noImplicitReturns: true, + isolatedDeclarations: false, + jsx: 'precompile', + jsxImportSource: 'preact', + }, + imports: workspace.imports, + catalog: workspace.catalog, + }, + null, + 2, + ) + }\n`, + ); + const tempLockPath = join(tempRoot, 'deno.lock'); + await Deno.copyFile(rootLockPath, tempLockPath); + const output = await new Deno.Command(Deno.execPath(), { + cwd: repositoryRoot, + args: [ + 'check', + '--unstable-kv', + '--lock', + tempLockPath, + '--config', + configPath, + ...modules.map((module) => module.path), + ], + stdout: 'piped', + stderr: 'piped', + }).output(); + const rootLockAfter = await Deno.readTextFile(rootLockPath); + const tempLockAfter = await Deno.readTextFile(tempLockPath); + const rootLockUnchanged = rootLockBefore === rootLockAfter; + if (!rootLockUnchanged) throw new Error('tracked root deno.lock changed during snippet check'); + return { + code: output.code, + stdout: new TextDecoder().decode(output.stdout), + diagnostics: mapDiagnostics(new TextDecoder().decode(output.stderr), modules), + memberCount: workspace.memberCount, + catalogCount: Object.keys(workspace.catalog).length, + rootLockUnchanged, + temporaryLockRewritten: rootLockBefore !== tempLockAfter, + }; + } finally { + await Deno.remove(tempRoot, { recursive: true }); + } +} diff --git a/.llm/tools/docs/snippet-compiler_test.ts b/.llm/tools/docs/snippet-compiler_test.ts new file mode 100644 index 0000000000..0ee9043753 --- /dev/null +++ b/.llm/tools/docs/snippet-compiler_test.ts @@ -0,0 +1,61 @@ +import { assert, assertEquals, assertStringIncludes } from '@std/assert'; +import { dirname, fromFileUrl, join } from '@std/path'; +import { compileSnippetAnalysis } from './snippet-compiler.ts'; +import { analyzeSnippetSite } from './snippet-policy.ts'; +import { canonicalizePackageSpecifier, resolveWorkspaceSurface } from './snippet-workspace.ts'; + +const repositoryRoot = dirname(dirname(dirname(dirname(fromFileUrl(import.meta.url))))); +const fixtureRoot = join(repositoryRoot, '.llm/tools/docs/fixtures'); + +async function compileFixture(name: string) { + const analysis = await analyzeSnippetSite(join(fixtureRoot, name, 'docs/site'), { + enforceCoverage: false, + }); + return await compileSnippetAnalysis(analysis, repositoryRoot); +} + +Deno.test('workspace resolver exposes exact exports, catalog fallbacks, and canonical ranges', async () => { + const surface = await resolveWorkspaceSurface(repositoryRoot); + assertEquals(surface.memberCount, 37); + assertEquals(Object.keys(surface.catalog).length, 38); + assert(surface.imports['@netscript/sdk/client'].endsWith('/packages/sdk/src/client/mod.ts')); + assertEquals(surface.imports['@netscript/sdk/src/client/mod.ts'], undefined); + assertEquals(surface.imports['@opentelemetry/api'], 'npm:@opentelemetry/api@^1.9.1'); + assertEquals(canonicalizePackageSpecifier('jsr:@std/assert@1'), 'jsr:@std/assert@^1'); + assertEquals( + canonicalizePackageSpecifier('jsr:@std/assert@^1'), + canonicalizePackageSpecifier('jsr:@std/assert@1'), + ); + assertEquals(surface.declaredConflictCount, 0); + assertEquals(surface.catalogConflictCount, 0); +}); + +Deno.test('copied-lock compiler is green with root catalog and leaves tracked lock unchanged', async () => { + const lockBefore = await Deno.readTextFile(join(repositoryRoot, 'deno.lock')); + const result = await compileFixture('dialect-a-positional'); + assertEquals(result.code, 0, result.diagnostics); + assertEquals(result.catalogCount, 38); + assertEquals(result.memberCount, 37); + assert(result.rootLockUnchanged); + assert(result.temporaryLockRewritten); + assertEquals(await Deno.readTextFile(join(repositoryRoot, 'deno.lock')), lockBefore); +}); + +Deno.test('actual compiler graph rejects a non-exported SDK symbol', async () => { + const result = await compileFixture('non-exported-symbol'); + assertEquals(result.code, 1); + assertStringIncludes(result.diagnostics, 'page.md:1'); + assertStringIncludes(result.diagnostics, 'DefinitelyNotExportedByNetScript'); +}); + +Deno.test('dialect controls discriminate typed APIs through the same docs fixture', async () => { + const red = await compileFixture('dialect-a-object-input'); + assertEquals(red.code, 1); + assertStringIncludes(red.diagnostics, 'page.md:1'); + assertStringIncludes(red.diagnostics, 'TS2353'); + + const greenA = await compileFixture('dialect-a-positional'); + assertEquals(greenA.code, 0, greenA.diagnostics); + const greenB = await compileFixture('dialect-b-object-input'); + assertEquals(greenB.code, 0, greenB.diagnostics); +}); diff --git a/.llm/tools/docs/snippet-coverage.md b/.llm/tools/docs/snippet-coverage.md new file mode 100644 index 0000000000..220f615c97 --- /dev/null +++ b/.llm/tools/docs/snippet-coverage.md @@ -0,0 +1,74 @@ +# Documentation snippet compile coverage + +`deno task docs:snippets` extracts every fenced block under `docs/site`, then type-checks the +unexempted TypeScript blocks on the current coverage floor against exact public `@netscript/*` +entrypoints. It compiles snippets only; it does not execute them or start services. + +## Day-one floor + +The initial floor covers these #1373 Tier-1 pages: + +- `quickstart.vto` +- `index.vto` (zero TypeScript fences) +- `services-sdk/sdk.md` +- `services-sdk/how-to/add-a-service.md` +- `web-layer/query.md` +- `web-layer/examples.md` +- `web-layer/interactive.md` +- `web-layer/form.md` +- `web-layer/query-bridge.md` + +The checked-in baseline is: + +```text +scanned=578 ts=211 tsx=77 typescript=7 ts_like=295 tier1=35 checked=21 exempt=14 outside_floor=260 malformed=0 +``` + +All 35 candidates occur on eight of the nine pages; `index.vto` contributes none. The checker fails +below 35 candidates or 21 checked blocks, above 14 exemptions, or on a malformed/reasonless marker. +Every exemption is printed as `page:line — reason`, so the 14-block prose-only remainder stays +visible and may shrink without weakening the gate. + +## Fixture controls + +Run one fixture explicitly as `deno task docs:snippets:negative <case>`, where `<case>` is one of: + +- `non-exported-symbol` +- `empty-exemption-reason` +- `dialect-a-object-input` +- `dialect-a-positional` +- `dialect-b-object-input` + +The first three are required raw red controls; the last two prove the checker accepts the valid +dialect-A and dialect-B shapes. A missing or unknown case exits non-zero and prints this complete +usage list. The `:negative` task never falls back to the ordinary positive corpus gate. + +## Expanding the floor + +Expansion proceeds in coherent page families. Each wave adds pages to the checked policy, compiles +every new unmarked block, marks only deliberate fragments with a specific reason, lowers +`outside_floor`, and records the checked/exempt delta. The exemption baseline must not increase +without explicit reviewer-approved rationale. + +1. Remaining golden-path families: the rest of `web-layer/**`, `services-sdk/**`, and + `quickstart/**`. +2. Runtime guides: `ai/**`, `background-processing/**`, `data-persistence/**`, + `durable-workflows/**`, `identity-access/**`, `observability/**`, + `orchestration-runtime/**`, and `explanation/**`. +3. Tutorials: `tutorials/**`, one tutorial track per change so multi-file context stays reviewable. +4. Reference prose fences: `reference/**`, coordinated with #1108 but without expanding that + issue's export-table scope. Canonicalize the seven `typescript` tags to `ts`; they already compile + as checked aliases, so that edit is tag cleanup rather than new coverage. +5. Non-published source/templates under underscore directories, reported separately. Package + READMEs remain #1377. + +## Known ratchet window + +The demoted positive-presence needles `queryOptions({ input })`, `queryOptions(input)`, and +`no server KV tier` live on `docs/site/reference/sdk/index.md`, outside the day-one floor until wave +4. Their positive presence is therefore not asserted in the interim. This is deliberate: literal +presence was a false-green API check and compilation is its replacement. + +The retained exact one-page `createServiceQueryUtils` containment rule still prevents dialect B +from appearing on golden-path pages. Only positive presence on its sanctioned reference page is in +the temporary window; dialect placement remains enforced. diff --git a/.llm/tools/docs/snippet-extractor.ts b/.llm/tools/docs/snippet-extractor.ts new file mode 100644 index 0000000000..b34bbf8b97 --- /dev/null +++ b/.llm/tools/docs/snippet-extractor.ts @@ -0,0 +1,114 @@ +/** TypeScript-like fence tags compiled by the documentation snippet gate. */ +export const CHECKED_LANGUAGES = ['ts', 'tsx', 'typescript'] as const; + +/** A recognized documentation fence language. */ +export type CheckedLanguage = (typeof CHECKED_LANGUAGES)[number]; + +/** Stable source provenance and content for one fenced documentation block. */ +export interface FencedBlock { + sourcePath: string; + fenceOrdinal: number; + openingLine: number; + codeStartLine: number; + closingLine: number; + delimiter: '`' | '~'; + delimiterLength: number; + infoString: string; + language: string; + checkedLanguage?: CheckedLanguage; + compilationExtension?: 'ts' | 'tsx'; + exemptionReason?: string; + body: string; +} + +const OPENING_FENCE = /^ {0,3}(`{3,}|~{3,})(.*)$/; +const REASONED_MARKER = /^(ts|tsx|typescript)[ \t]+no-check:[ \t]*(\S(?:.*\S)?)[ \t]*$/; + +function isCheckedLanguage(value: string): value is CheckedLanguage { + return (CHECKED_LANGUAGES as readonly string[]).includes(value); +} + +function classifyInfoString( + infoString: string, + sourcePath: string, + openingLine: number, +): Pick<FencedBlock, 'language' | 'checkedLanguage' | 'compilationExtension' | 'exemptionReason'> { + const language = infoString.split(/[ \t]/, 1)[0] ?? ''; + if (!isCheckedLanguage(language)) return { language }; + + const compilationExtension = language === 'tsx' ? 'tsx' : 'ts'; + if (infoString === language) return { language, checkedLanguage: language, compilationExtension }; + + const marker = REASONED_MARKER.exec(infoString); + if (marker) { + return { + language, + checkedLanguage: language, + compilationExtension, + exemptionReason: marker[2].trim(), + }; + } + + const reasonHint = infoString.includes('no-check') + ? 'expected no-check:<nonblank reason>' + : 'extra fence attributes are not allowed'; + throw new Error(`${sourcePath}:${openingLine}: malformed ${language} fence: ${reasonHint}`); +} + +/** Extract Markdown/Vento fences with stable source provenance and strict TS marker validation. */ +export function extractFencedBlocks(source: string, sourcePath: string): FencedBlock[] { + const lines = source.replaceAll('\r\n', '\n').replaceAll('\r', '\n').split('\n'); + const blocks: FencedBlock[] = []; + + for (let index = 0; index < lines.length; index++) { + const opening = OPENING_FENCE.exec(lines[index]); + if (!opening) continue; + + const fence = opening[1]; + const delimiter = fence[0] as '`' | '~'; + const delimiterLength = fence.length; + const infoString = opening[2].trim(); + const openingLine = index + 1; + const classification = classifyInfoString(infoString, sourcePath, openingLine); + const bodyStart = index + 1; + let closingIndex = -1; + + for (let candidate = bodyStart; candidate < lines.length; candidate++) { + const closing = /^ {0,3}(`{3,}|~{3,})(.*)$/.exec(lines[candidate]); + if (!closing || closing[1][0] !== delimiter || closing[1].length < delimiterLength) continue; + + const trailing = closing[2].trim(); + if (trailing.length === 0) { + closingIndex = candidate; + break; + } + if (trailing.includes('no-check')) { + throw new Error( + `${sourcePath}:${candidate + 1}: no-check marker belongs on the opening fence`, + ); + } + } + + if (closingIndex < 0) { + throw new Error( + `${sourcePath}:${openingLine}: unclosed ${delimiter.repeat(delimiterLength)} fence`, + ); + } + + blocks.push({ + sourcePath, + fenceOrdinal: blocks.length + 1, + openingLine, + codeStartLine: openingLine + 1, + closingLine: closingIndex + 1, + delimiter, + delimiterLength, + infoString, + ...classification, + body: lines.slice(bodyStart, closingIndex).join('\n'), + }); + index = closingIndex; + } + + return blocks; +} diff --git a/.llm/tools/docs/snippet-extractor_test.ts b/.llm/tools/docs/snippet-extractor_test.ts new file mode 100644 index 0000000000..ef7cfb7918 --- /dev/null +++ b/.llm/tools/docs/snippet-extractor_test.ts @@ -0,0 +1,184 @@ +import { assertEquals, assertRejects, assertStringIncludes } from '@std/assert'; +import { dirname, fromFileUrl, join } from '@std/path'; +import { extractFencedBlocks } from './snippet-extractor.ts'; +import { analyzeSnippetSite } from './snippet-policy.ts'; + +const repositoryRoot = dirname(dirname(dirname(dirname(fromFileUrl(import.meta.url))))); + +Deno.test('extractor preserves provenance and treats typescript as checked ts', () => { + const blocks = extractFencedBlocks( + '# Example\n\n~~~typescript\nconst answer = 42;\n~~~\n\n```tsx no-check: partial JSX\n<div />\n```\n', + 'guide.md', + ); + assertEquals(blocks, [ + { + sourcePath: 'guide.md', + fenceOrdinal: 1, + openingLine: 3, + codeStartLine: 4, + closingLine: 5, + delimiter: '~', + delimiterLength: 3, + infoString: 'typescript', + language: 'typescript', + checkedLanguage: 'typescript', + compilationExtension: 'ts', + body: 'const answer = 42;', + }, + { + sourcePath: 'guide.md', + fenceOrdinal: 2, + openingLine: 7, + codeStartLine: 8, + closingLine: 9, + delimiter: '`', + delimiterLength: 3, + infoString: 'tsx no-check: partial JSX', + language: 'tsx', + checkedLanguage: 'tsx', + compilationExtension: 'tsx', + exemptionReason: 'partial JSX', + body: '<div />', + }, + ]); +}); + +Deno.test('recognized TS fences reject missing reasons and extra attributes', () => { + for (const info of ['ts no-check', 'ts no-check:', 'tsx twoslash']) { + let message = ''; + try { + extractFencedBlocks(`\`\`\`${info}\nconst x = 1;\n\`\`\`\n`, 'bad.md'); + } catch (error) { + message = error instanceof Error ? error.message : String(error); + } + assertStringIncludes(message, 'bad.md:1: malformed'); + } +}); + +Deno.test('real corpus census recognizes aliases and protects the Tier-1 candidate floor', async () => { + const analysis = await analyzeSnippetSite(join(repositoryRoot, 'docs/site')); + assertEquals(analysis.census, { + scanned: 578, + ts: 211, + tsx: 77, + typescript: 7, + tsLike: 295, + tier1: 35, + checked: 21, + exempt: 14, + outsideFloor: 260, + malformed: 0, + }); + + const temp = await Deno.makeTempDir(); + try { + const site = join(temp, 'docs/site'); + for ( + const page of [ + 'quickstart.vto', + 'index.vto', + 'services-sdk/sdk.md', + 'services-sdk/how-to/add-a-service.md', + 'web-layer/query.md', + 'web-layer/examples.md', + 'web-layer/interactive.md', + 'web-layer/form.md', + 'web-layer/query-bridge.md', + ] + ) { + const target = join(site, page); + await Deno.mkdir(dirname(target), { recursive: true }); + await Deno.writeTextFile( + target, + page === 'quickstart.vto' ? '```js\nconst bypass = true;\n```\n' : '', + ); + } + await assertRejects( + () => analyzeSnippetSite(site), + Error, + 'candidate count 0 is below floor 35', + ); + + const quickstart = join(site, 'quickstart.vto'); + await Deno.writeTextFile( + quickstart, + `${'```ts no-check:structural fragment\nvalue;\n```\n'.repeat(15)}${ + '```ts\nconst value = 1;\n```\n'.repeat(20) + }`, + ); + await assertRejects( + () => analyzeSnippetSite(site), + Error, + 'checked count 20 is below floor 21', + ); + + await Deno.writeTextFile( + quickstart, + `${'```ts no-check:structural fragment\nvalue;\n```\n'.repeat(15)}${ + '```ts\nconst value = 1;\n```\n'.repeat(21) + }`, + ); + await assertRejects( + () => analyzeSnippetSite(site), + Error, + 'exemption count 15 exceeds budget 14', + ); + + await Deno.writeTextFile( + quickstart, + '```typescript\nconst value = 1;\n```\n'.repeat(35), + ); + assertEquals((await analyzeSnippetSite(site)).census.checked, 35); + } finally { + await Deno.remove(temp, { recursive: true }); + } +}); + +Deno.test('empty-reason fixture exits the actual CLI non-zero and names its fence', async () => { + const output = await new Deno.Command(Deno.execPath(), { + cwd: repositoryRoot, + args: [ + 'run', + '--allow-read', + '.llm/tools/docs/check-snippets.ts', + '--negative', + 'empty-exemption-reason', + '--extract-only', + ], + stdout: 'piped', + stderr: 'piped', + }).output(); + assertEquals(output.code, 1); + assertStringIncludes(new TextDecoder().decode(output.stderr), 'page.md:1: malformed ts fence'); +}); + +Deno.test('negative task rejects missing and unknown fixture names with complete usage', async () => { + for ( + const args of [ + ['task', 'docs:snippets:negative'], + ['task', 'docs:snippets:negative', 'not-a-fixture'], + ] + ) { + const output = await new Deno.Command(Deno.execPath(), { + cwd: repositoryRoot, + args, + stdout: 'piped', + stderr: 'piped', + }).output(); + const stderr = new TextDecoder().decode(output.stderr); + + assertEquals(output.code, 1); + assertStringIncludes(stderr, 'usage: deno task docs:snippets:negative <case>'); + for ( + const fixture of [ + 'non-exported-symbol', + 'empty-exemption-reason', + 'dialect-a-object-input', + 'dialect-a-positional', + 'dialect-b-object-input', + ] + ) { + assertStringIncludes(stderr, fixture); + } + } +}); diff --git a/.llm/tools/docs/snippet-policy.ts b/.llm/tools/docs/snippet-policy.ts new file mode 100644 index 0000000000..e38012c709 --- /dev/null +++ b/.llm/tools/docs/snippet-policy.ts @@ -0,0 +1,127 @@ +import { relative } from '@std/path'; +import { extractFencedBlocks, type FencedBlock } from './snippet-extractor.ts'; + +/** The issue-mandated Tier-1 source pages, including zero-fence index.vto. */ +export const TIER_1_PAGES = [ + 'quickstart.vto', + 'index.vto', + 'services-sdk/sdk.md', + 'services-sdk/how-to/add-a-service.md', + 'web-layer/query.md', + 'web-layer/examples.md', + 'web-layer/interactive.md', + 'web-layer/form.md', + 'web-layer/query-bridge.md', +] as const; + +/** Initial coverage ratchet approved by PLAN-EVAL. */ +export const TIER_1_FLOOR = { + minimumCandidates: 35, + minimumChecked: 21, + maximumExempt: 14, +} as const; + +/** Corpus and coverage totals printed by the documentation snippet gate. */ +export interface SnippetCensus { + scanned: number; + ts: number; + tsx: number; + typescript: number; + tsLike: number; + tier1: number; + checked: number; + exempt: number; + outsideFloor: number; + malformed: number; +} + +/** Extracted site state used by the compiler and other checked-in quality consumers. */ +export interface SnippetSiteAnalysis { + blocks: FencedBlock[]; + tier1Blocks: FencedBlock[]; + exemptions: FencedBlock[]; + census: SnippetCensus; +} + +async function collectSourceFiles(directory: string): Promise<string[]> { + const files: string[] = []; + for await (const entry of Deno.readDir(directory)) { + const path = `${directory}/${entry.name}`; + if (entry.isDirectory) files.push(...await collectSourceFiles(path)); + else if (entry.isFile && (entry.name.endsWith('.md') || entry.name.endsWith('.vto'))) { + files.push(path); + } + } + return files.sort(); +} + +function toSourcePath(siteRoot: string, path: string): string { + return relative(siteRoot, path).replaceAll('\\', '/'); +} + +/** Scan a docs site and enforce the checked/exempt/candidate coverage ratchet. */ +export async function analyzeSnippetSite( + siteRoot: string, + options: { enforceCoverage?: boolean } = {}, +): Promise<SnippetSiteAnalysis> { + const files = await collectSourceFiles(siteRoot); + const sourcePaths = new Set(files.map((path) => toSourcePath(siteRoot, path))); + if (options.enforceCoverage !== false) { + for (const page of TIER_1_PAGES) { + if (!sourcePaths.has(page)) throw new Error(`${page}: Tier-1 page missing from ${siteRoot}`); + } + } + + const blocks: FencedBlock[] = []; + for (const path of files) { + const sourcePath = toSourcePath(siteRoot, path); + blocks.push(...extractFencedBlocks(await Deno.readTextFile(path), sourcePath)); + } + + const tier1Pages = new Set<string>(TIER_1_PAGES); + const tsLikeBlocks = blocks.filter((block) => block.checkedLanguage !== undefined); + const tier1Blocks = options.enforceCoverage === false + ? tsLikeBlocks + : tsLikeBlocks.filter((block) => tier1Pages.has(block.sourcePath)); + const exemptions = tier1Blocks.filter((block) => block.exemptionReason !== undefined); + const checked = tier1Blocks.length - exemptions.length; + + if (options.enforceCoverage !== false && tier1Blocks.length < TIER_1_FLOOR.minimumCandidates) { + throw new Error( + `Tier-1 candidate count ${tier1Blocks.length} is below floor ${TIER_1_FLOOR.minimumCandidates}`, + ); + } + if (options.enforceCoverage !== false && checked < TIER_1_FLOOR.minimumChecked) { + throw new Error( + `Tier-1 checked count ${checked} is below floor ${TIER_1_FLOOR.minimumChecked}`, + ); + } + if (options.enforceCoverage !== false && exemptions.length > TIER_1_FLOOR.maximumExempt) { + throw new Error( + `Tier-1 exemption count ${exemptions.length} exceeds budget ${TIER_1_FLOOR.maximumExempt}`, + ); + } + + const census: SnippetCensus = { + scanned: blocks.length, + ts: blocks.filter((block) => block.checkedLanguage === 'ts').length, + tsx: blocks.filter((block) => block.checkedLanguage === 'tsx').length, + typescript: blocks.filter((block) => block.checkedLanguage === 'typescript').length, + tsLike: tsLikeBlocks.length, + tier1: tier1Blocks.length, + checked, + exempt: exemptions.length, + outsideFloor: tsLikeBlocks.length - tier1Blocks.length, + malformed: 0, + }; + + return { blocks, tier1Blocks, exemptions, census }; +} + +/** Render the stable one-line census used in local and CI evidence. */ +export function formatSnippetCensus( + census: SnippetCensus, + status: 'PASS' | 'FAIL' = 'PASS', +): string { + return `docs snippets: ${status} scanned=${census.scanned} ts=${census.ts} tsx=${census.tsx} typescript=${census.typescript} ts_like=${census.tsLike} tier1=${census.tier1} checked=${census.checked} exempt=${census.exempt} outside_floor=${census.outsideFloor} malformed=${census.malformed}`; +} diff --git a/.llm/tools/docs/snippet-supports.ts b/.llm/tools/docs/snippet-supports.ts new file mode 100644 index 0000000000..2308db3a1b --- /dev/null +++ b/.llm/tools/docs/snippet-supports.ts @@ -0,0 +1,171 @@ +import { dirname, join, toFileUrl } from '@std/path'; + +/** Write a generated support/module file after creating its parent directory. */ +export async function writeSnippetFile(path: string, content: string): Promise<void> { + await Deno.mkdir(dirname(path), { recursive: true }); + await Deno.writeTextFile(path, content); +} + +/** Materialize strongly typed shared aliases used by checked Tier-1 examples. */ +export async function materializeSharedSupports( + tempRoot: string, +): Promise<Record<string, string>> { + const supportRoot = join(tempRoot, '_support'); + const supports: Record<string, string> = { + '@database/zod': join(supportRoot, 'database-zod.ts'), + '@playground/contracts': join(supportRoot, 'playground-contracts.ts'), + '@my-app/contracts': join(supportRoot, 'my-app-contracts.ts'), + '@app/utils.ts': join(supportRoot, 'app-utils.ts'), + '@app/lib/contacts.ts': join(supportRoot, 'contacts.ts'), + }; + + await writeSnippetFile( + supports['@database/zod'], + `import { z } from 'zod'; +export const UserSchema = z.object({ + id: z.number().int().positive(), + name: z.string(), + createdAt: z.date(), + updatedAt: z.date(), +}); +`, + ); + await writeSnippetFile( + supports['@playground/contracts'], + `import { oc } from '@orpc/contract'; +import { z } from 'zod'; +export const ordersContract = { + list: oc.route({ method: 'POST' }) + .input(z.object({ limit: z.number().int().positive().optional() })) + .output(z.array(z.object({ id: z.string(), total: z.number() }))), +}; +`, + ); + await writeSnippetFile( + supports['@my-app/contracts'], + `import { oc } from '@orpc/contract'; +import { implement } from '@orpc/server'; +import { z } from 'zod'; + +export const WidgetsContractV1 = { + list: oc.route({ method: 'POST' }) + .input(z.object({})) + .output(z.array(z.object({ id: z.string(), name: z.string() }))), +}; + +const UsersContractV1 = { + health: { + check: oc.route({ method: 'GET' }) + .input(z.object({}).optional()) + .output(z.object({ status: z.literal('healthy'), service: z.string() })), + }, + list: oc.route({ method: 'POST' }) + .input(z.object({ limit: z.number().int().positive().optional() })) + .output(z.object({ items: z.array(z.object({ id: z.number(), name: z.string() })) })), +}; +export const v1 = { users: implement(UsersContractV1) }; +`, + ); + await writeSnippetFile( + supports['@app/utils.ts'], + `export { definePage } from '@netscript/fresh/builders'; +`, + ); + await writeSnippetFile( + supports['@app/lib/contacts.ts'], + `export const contactsClient = { + async create(input: { email: string; message: string }) { + return { id: 'contact-1', ...input }; + }, + async invalidateList(): Promise<void> {}, +}; +`, + ); + + return Object.fromEntries( + Object.entries(supports).map(([alias, path]) => [alias, toFileUrl(path).href]), + ); +} + +/** Materialize page-relative, public-factory-derived support modules for complete examples. */ +export async function materializePageSupports(pageRoot: string): Promise<void> { + await writeSnippetFile( + join(pageRoot, 'lib/docs.ts'), + `import { oc } from '@orpc/contract'; +import { z } from 'zod'; +import { createServiceClient } from '@netscript/sdk/client'; +import { createQueryFactories } from '@netscript/sdk/query'; + +export const DocsContract = { + getById: oc.route({ method: 'POST' }) + .input(z.object({ id: z.string() })) + .output(z.object({ status: z.enum(['pending', 'embedding', 'ready']) })), +}; +export const docsClient = createServiceClient({ contract: DocsContract, serviceName: 'docs' }); +export const docsQueries = createQueryFactories({ + docs: { contract: DocsContract, client: docsClient }, +}).docs; +`, + ); + await writeSnippetFile( + join(pageRoot, 'apps/dashboard/lib/todos.ts'), + `import { oc } from '@orpc/contract'; +import { z } from 'zod'; +import { createServiceClient } from '@netscript/sdk/client'; +import { createQueryFactories } from '@netscript/sdk/query'; + +const Todo = z.object({ id: z.string(), title: z.string(), done: z.boolean() }); +const TodosContract = { + list: oc.route({ method: 'POST' }).input(z.object({})).output(z.array(Todo)), + update: oc.route({ method: 'POST' }) + .input(z.object({ id: z.string(), done: z.boolean() })) + .output(Todo), +}; +const todosClient = createServiceClient({ contract: TodosContract, serviceName: 'todos' }); +export const todosQueries = createQueryFactories({ + todos: { contract: TodosContract, client: todosClient }, +}).todos; +`, + ); + await writeSnippetFile( + join(pageRoot, 'apps/dashboard/lib/orders.ts'), + `import { oc } from '@orpc/contract'; +import { z } from 'zod'; +import { createServiceClient } from '@netscript/sdk/client'; +import { createQueryFactories } from '@netscript/sdk/query'; + +const OrdersContract = { + list: oc.route({ method: 'POST' }) + .input(z.object({ limit: z.number().int().positive() })) + .output(z.array(z.object({ id: z.string(), reference: z.string() }))), +}; +const ordersClient = createServiceClient({ contract: OrdersContract, serviceName: 'orders' }); +export const ordersQueries = createQueryFactories({ + orders: { contract: OrdersContract, client: ordersClient }, +}).orders; +`, + ); + await writeSnippetFile( + join(pageRoot, 'apps/dashboard/lib/widgets.ts'), + `import { oc } from '@orpc/contract'; +import { z } from 'zod'; +import { createServiceClient } from '@netscript/sdk/client'; +import { createQueryFactories } from '@netscript/sdk/query'; + +const WidgetsContract = { + list: oc.route({ method: 'POST' }) + .input(z.object({})) + .output(z.array(z.object({ id: z.string(), name: z.string() }))), +}; +const widgetsClient = createServiceClient({ contract: WidgetsContract, serviceName: 'widgets' }); +export const widgetsQueries = createQueryFactories({ + widgets: { contract: WidgetsContract, client: widgetsClient }, +}).widgets; +`, + ); + await writeSnippetFile( + join(pageRoot, 'routes/(_components)/ContactForm.tsx'), + `export default function ContactForm() { return null; } +`, + ); +} diff --git a/.llm/tools/docs/snippet-workspace.ts b/.llm/tools/docs/snippet-workspace.ts new file mode 100644 index 0000000000..b0a382d305 --- /dev/null +++ b/.llm/tools/docs/snippet-workspace.ts @@ -0,0 +1,252 @@ +import { join, toFileUrl } from '@std/path'; + +interface JsonObject { + [key: string]: unknown; +} + +interface WorkspaceMember { + directory: string; + configPath: string; + config: JsonObject; +} + +/** Resolved synthetic configuration inputs derived from the real workspace. */ +export interface WorkspaceSurface { + imports: Record<string, string>; + catalog: Record<string, string>; + memberCount: number; + declaredConflictCount: number; + catalogConflictCount: number; +} + +function isJsonObject(value: unknown): value is JsonObject { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +async function readJsonObject(path: string): Promise<JsonObject> { + const value: unknown = JSON.parse(await Deno.readTextFile(path)); + if (!isJsonObject(value)) throw new Error(`${path}: expected a JSON object`); + return value; +} + +function stringRecord(value: unknown, label: string): Record<string, string> { + if (value === undefined) return {}; + if (!isJsonObject(value)) throw new Error(`${label}: expected an object`); + const record: Record<string, string> = {}; + for (const [key, entry] of Object.entries(value)) { + if (typeof entry !== 'string') throw new Error(`${label}.${key}: expected a string`); + record[key] = entry; + } + return record; +} + +function stringArray(value: unknown, label: string): string[] { + if (!Array.isArray(value) || value.some((entry) => typeof entry !== 'string')) { + throw new Error(`${label}: expected a string array`); + } + return value; +} + +function exportTarget(value: unknown, label: string): string { + if (typeof value === 'string') return value; + if (!isJsonObject(value)) throw new Error(`${label}: expected a string or conditional export`); + for (const condition of ['deno', 'import', 'default']) { + if (condition in value) return exportTarget(value[condition], `${label}.${condition}`); + } + throw new Error(`${label}: no deno/import/default target`); +} + +async function pathExists(path: string): Promise<boolean> { + try { + await Deno.stat(path); + return true; + } catch (error) { + if (error instanceof Deno.errors.NotFound) return false; + throw error; + } +} + +async function discoverWorkspaceMembers( + repositoryRoot: string, + patterns: string[], +): Promise<WorkspaceMember[]> { + const directories = new Set<string>(); + for (const pattern of patterns) { + if (pattern.endsWith('/*')) { + const parent = join(repositoryRoot, pattern.slice(0, -2)); + if (!await pathExists(parent)) continue; + for await (const entry of Deno.readDir(parent)) { + if (entry.isDirectory) directories.add(join(parent, entry.name)); + } + } else { + directories.add(join(repositoryRoot, pattern)); + } + } + + const members: WorkspaceMember[] = []; + for (const directory of [...directories].sort()) { + const configPath = join(directory, 'deno.json'); + if (!await pathExists(configPath)) continue; + members.push({ directory, configPath, config: await readJsonObject(configPath) }); + } + return members; +} + +function normalizeRange(range: string): string { + return /^\d+(?:\.\d+)?$/.test(range) ? `^${range}` : range; +} + +/** Canonicalize equivalent JSR/npm major shorthand before merge conflict comparison. */ +export function canonicalizePackageSpecifier(specifier: string): string { + const match = /^(jsr|npm):(@[^/]+\/[^@/]+|[^@/]+)(?:@([^/]+))?(\/.*)?$/.exec(specifier); + if (!match) return specifier; + const [, protocol, packageName, range, subpath = ''] = match; + return `${protocol}:${packageName}${range ? `@${normalizeRange(range)}` : ''}${subpath}`; +} + +function addImport( + imports: Record<string, string>, + origins: Map<string, string>, + alias: string, + specifier: string, + origin: string, +): void { + const existing = imports[alias]; + if (existing === undefined) { + imports[alias] = specifier; + origins.set(alias, origin); + return; + } + if (canonicalizePackageSpecifier(existing) !== canonicalizePackageSpecifier(specifier)) { + throw new Error( + `conflicting import ${alias}: ${existing} (${ + origins.get(alias) + }) vs ${specifier} (${origin})`, + ); + } +} + +function resolveCatalogImport( + alias: string, + specifier: string, + catalog: Record<string, string>, + origin: string, +): string { + if (specifier !== 'catalog:') return specifier; + const range = catalog[alias]; + if (!range) throw new Error(`${origin}: ${alias} references missing root catalog entry`); + return `npm:${alias}@${range}`; +} + +/** Derive exact public workspace exports and canonicalized external imports for synthetic checks. */ +export async function resolveWorkspaceSurface( + repositoryRoot: string, + supportImports: Record<string, string> = {}, +): Promise<WorkspaceSurface> { + const rootPath = join(repositoryRoot, 'deno.json'); + const rootConfig = await readJsonObject(rootPath); + const catalog = stringRecord(rootConfig.catalog, `${rootPath}.catalog`); + const members = await discoverWorkspaceMembers( + repositoryRoot, + stringArray(rootConfig.workspace, `${rootPath}.workspace`), + ); + const imports: Record<string, string> = {}; + const origins = new Map<string, string>(); + const publicWorkspaceSpecifiers = new Set<string>(); + + for (const member of members) { + const name = member.config.name; + if (typeof name !== 'string' || !name.startsWith('@netscript/')) continue; + const exportsValue = member.config.exports; + if (typeof exportsValue === 'string') { + addImport( + imports, + origins, + name, + toFileUrl(join(member.directory, exportsValue)).href, + member.configPath, + ); + publicWorkspaceSpecifiers.add(name); + continue; + } + if (!isJsonObject(exportsValue)) continue; + for (const [key, value] of Object.entries(exportsValue)) { + if (key !== '.' && !key.startsWith('./')) continue; + const publicSpecifier = key === '.' ? name : `${name}/${key.slice(2)}`; + addImport( + imports, + origins, + publicSpecifier, + toFileUrl( + join(member.directory, exportTarget(value, `${member.configPath}.exports.${key}`)), + ) + .href, + member.configPath, + ); + publicWorkspaceSpecifiers.add(publicSpecifier); + } + } + + const configs: Array<readonly [string, JsonObject]> = [ + [rootPath, rootConfig], + ...members.map((member) => [member.configPath, member.config] as const), + ]; + for (const [configPath, config] of configs) { + for ( + const [alias, rawSpecifier] of Object.entries( + stringRecord(config.imports, `${configPath}.imports`), + ) + ) { + if (alias.startsWith('@netscript/')) continue; + addImport( + imports, + origins, + alias, + resolveCatalogImport(alias, rawSpecifier, catalog, configPath), + configPath, + ); + } + } + + for (const [packageName, range] of Object.entries(catalog)) { + addImport( + imports, + origins, + packageName, + `npm:${packageName}@${range}`, + `${rootPath}.catalog`, + ); + } + + for (const [alias, specifier] of Object.entries(supportImports)) { + if (publicWorkspaceSpecifiers.has(alias)) { + throw new Error(`support import ${alias} would shadow a public workspace export`); + } + addImport(imports, origins, alias, specifier, 'snippet support'); + } + + const preactRange = catalog.preact; + if (!preactRange) throw new Error(`${rootPath}.catalog: preact entry missing`); + addImport( + imports, + origins, + 'preact/jsx-runtime', + `npm:preact@${preactRange}/jsx-runtime`, + 'snippet JSX runtime', + ); + addImport( + imports, + origins, + 'preact/jsx-dev-runtime', + `npm:preact@${preactRange}/jsx-dev-runtime`, + 'snippet JSX runtime', + ); + + return { + imports, + catalog, + memberCount: members.length, + declaredConflictCount: 0, + catalogConflictCount: 0, + }; +} diff --git a/.llm/tools/release/publish-workspace_test.ts b/.llm/tools/release/publish-workspace_test.ts new file mode 100644 index 0000000000..567d796246 --- /dev/null +++ b/.llm/tools/release/publish-workspace_test.ts @@ -0,0 +1,116 @@ +import { assertEquals, assertStringIncludes } from '@std/assert'; + +import { publishMemberDryRun, publishWorkspace } from './publish-workspace.ts'; + +Deno.test('publish dry-run isolates catalog and Deno manifest rewrites in a throwaway workspace', async () => { + const sourceRoot = await Deno.makeTempDir({ prefix: 'netscript-publish-source-' }); + try { + await writeFixture(sourceRoot); + const servicePath = `${sourceRoot}/packages/service/deno.json`; + const mcpPath = `${sourceRoot}/packages/mcp/deno.json`; + const lockPath = `${sourceRoot}/deno.lock`; + const before = await readFiles([servicePath, mcpPath, lockPath]); + + await publishWorkspace({ + mode: 'dry-run', + root: sourceRoot, + commandRunner: async ({ args, cwd }) => { + assertEquals(args, ['publish', '--allow-dirty', '--dry-run']); + + const materializedService = await Deno.readTextFile( + `${cwd}/packages/service/deno.json`, + ); + assertStringIncludes(materializedService, '"zod": "npm:zod@^4.4.3"'); + + const mcpConfig = JSON.parse( + await Deno.readTextFile(`${cwd}/packages/mcp/deno.json`), + ) as Record<string, unknown>; + mcpConfig.publish = ['deno.json', 'mod.ts', 'expanded-by-deno.ts']; + await Deno.writeTextFile( + `${cwd}/packages/mcp/deno.json`, + `${JSON.stringify(mcpConfig, null, 2)}\n`, + ); + await Deno.writeTextFile(`${cwd}/deno.lock`, 'mutated by dry-run'); + assertEquals(cwd.startsWith(sourceRoot), false); + + return { code: 0, stdout: new Uint8Array(), stderr: new Uint8Array() }; + }, + }); + + assertEquals(await readFiles([servicePath, mcpPath, lockPath]), before); + } finally { + await Deno.remove(sourceRoot, { recursive: true }); + } +}); + +Deno.test('package dry-run isolates MCP publish array rewrites', async () => { + const sourceRoot = await Deno.makeTempDir({ prefix: 'netscript-publish-member-source-' }); + try { + await writeFixture(sourceRoot); + const mcpPath = `${sourceRoot}/packages/mcp/deno.json`; + const lockPath = `${sourceRoot}/deno.lock`; + const before = await readFiles([mcpPath, lockPath]); + + await publishMemberDryRun(sourceRoot, 'packages/mcp', async ({ args, cwd }) => { + assertEquals(args, ['publish', '--allow-dirty', '--dry-run']); + const mcpConfig = JSON.parse(await Deno.readTextFile(`${cwd}/deno.json`)) as Record< + string, + unknown + >; + mcpConfig.publish = ['deno.json', 'mod.ts', 'expanded-by-deno.ts']; + await Deno.writeTextFile(`${cwd}/deno.json`, `${JSON.stringify(mcpConfig, null, 2)}\n`); + await Deno.writeTextFile(`${cwd}/../../deno.lock`, 'mutated by package dry-run'); + assertEquals(cwd.startsWith(sourceRoot), false); + return { code: 0, stdout: new Uint8Array(), stderr: new Uint8Array() }; + }); + + assertEquals(await readFiles([mcpPath, lockPath]), before); + } finally { + await Deno.remove(sourceRoot, { recursive: true }); + } +}); + +async function writeFixture(root: string): Promise<void> { + await Deno.mkdir(`${root}/packages/service`, { recursive: true }); + await Deno.mkdir(`${root}/packages/mcp`, { recursive: true }); + await Deno.mkdir(`${root}/plugins`, { recursive: true }); + await Deno.writeTextFile( + `${root}/deno.json`, + `${JSON.stringify({ catalog: { zod: '^4.4.3' } }, null, 2)}\n`, + ); + await Deno.writeTextFile(`${root}/deno.lock`, '{\n "version": "5"\n}\n'); + await Deno.writeTextFile( + `${root}/packages/service/deno.json`, + `${ + JSON.stringify( + { + name: '@netscript/service', + imports: { zod: 'catalog:' }, + exports: './mod.ts', + }, + null, + 2, + ) + }\n`, + ); + await Deno.writeTextFile(`${root}/packages/service/mod.ts`, 'export const service = true;\n'); + await Deno.writeTextFile( + `${root}/packages/mcp/deno.json`, + `${ + JSON.stringify( + { + name: '@netscript/mcp', + publish: ['deno.json', 'mod.ts'], + exports: './mod.ts', + }, + null, + 2, + ) + }\n`, + ); + await Deno.writeTextFile(`${root}/packages/mcp/mod.ts`, 'export const mcp = true;\n'); +} + +async function readFiles(paths: readonly string[]): Promise<readonly string[]> { + return await Promise.all(paths.map((path) => Deno.readTextFile(path))); +} diff --git a/packages/plugin-streams-core/src/application/stream-browser-environment.d.ts b/packages/plugin-streams-core/src/application/stream-browser-environment.d.ts new file mode 100644 index 0000000000..0e359a2b60 --- /dev/null +++ b/packages/plugin-streams-core/src/application/stream-browser-environment.d.ts @@ -0,0 +1,6 @@ +interface ImportMeta { + readonly env: { + readonly VITE_services__streams__http__0?: string; + readonly VITE_STREAMS_URL?: string; + }; +} diff --git a/packages/plugin-streams-core/tests/application/stream-url-resolver_test.ts b/packages/plugin-streams-core/tests/application/stream-url-resolver_test.ts new file mode 100644 index 0000000000..148600efd0 --- /dev/null +++ b/packages/plugin-streams-core/tests/application/stream-url-resolver_test.ts @@ -0,0 +1,78 @@ +import { assertEquals, assertNotMatch, assertStringIncludes } from '@std/assert'; +import { getBrowserStreamsUrlFromEnv } from '../../src/application/stream-url-resolver.ts'; + +const RESOLVER_URL = new URL( + '../../src/application/stream-url-resolver.ts', + import.meta.url, +); +const CONSTANTS_URL = new URL('../../src/domain/constants.ts', import.meta.url); + +Deno.test('browser streams URL lookup prefers the full Aspire key over shorthand', () => { + assertEquals( + getBrowserStreamsUrlFromEnv({ + VITE_services__streams__http__0: 'http://browser-full.example', + VITE_STREAMS_URL: 'http://browser-short.example', + }), + 'http://browser-full.example', + ); +}); + +Deno.test('browser streams URL lookup falls back to the shorthand key', () => { + assertEquals( + getBrowserStreamsUrlFromEnv({ + VITE_STREAMS_URL: 'http://browser-short.example', + }), + 'http://browser-short.example', + ); +}); + +Deno.test('browser streams URL lookup returns undefined when both keys are absent', () => { + assertEquals(getBrowserStreamsUrlFromEnv({}), undefined); +}); + +Deno.test('browser streams URL reader preserves Vite-substitutable static member expressions', async () => { + const source = await Deno.readTextFile(RESOLVER_URL); + + assertStringIncludes( + source, + 'import.meta.env.VITE_services__streams__http__0', + ); + assertStringIncludes(source, 'import.meta.env.VITE_STREAMS_URL'); + assertNotMatch(source, /\w+\(\s*import\.meta\s*\)/); + assertNotMatch(source, /\benv\s*\[/); +}); + +Deno.test('getStreamsUrl reaches the browser reader and preserves browser key order', async () => { + const keys = ['DURABLE_STREAMS_URL', 'services__streams__http__0'] as const; + const previous = new Map(keys.map((key) => [key, Deno.env.get(key)])); + const tempDir = await Deno.makeTempDir(); + + try { + for (const key of keys) Deno.env.delete(key); + + const source = (await Deno.readTextFile(RESOLVER_URL)) + .replace("'../domain/constants.ts'", JSON.stringify(CONSTANTS_URL.href)) + .replaceAll( + 'import.meta.env.VITE_services__streams__http__0', + JSON.stringify('http://browser-full.example'), + ) + .replaceAll( + 'import.meta.env.VITE_STREAMS_URL', + JSON.stringify('http://browser-short.example'), + ); + const fixtureUrl = new URL(`file://${tempDir}/stream-url-resolver.ts`); + await Deno.writeTextFile(fixtureUrl, source); + + const fixture: { getStreamsUrl(): string } = await import( + `${fixtureUrl.href}?test=${crypto.randomUUID()}` + ); + assertEquals(fixture.getStreamsUrl(), 'http://browser-full.example'); + } finally { + await Deno.remove(tempDir, { recursive: true }); + for (const key of keys) { + const value = previous.get(key); + if (value === undefined) Deno.env.delete(key); + else Deno.env.set(key, value); + } + } +}); diff --git a/plugins/workers/tests/runtime/background-stream-hook_test.ts b/plugins/workers/tests/runtime/background-stream-hook_test.ts new file mode 100644 index 0000000000..d3790c32eb --- /dev/null +++ b/plugins/workers/tests/runtime/background-stream-hook_test.ts @@ -0,0 +1,46 @@ +import { assert, assertEquals, assertNotEquals } from '@std/assert'; +import { type ExecutionMutationHook, KvExecutionState } from '@netscript/plugin-workers-core/state'; +import { startCombinedProcess } from '../../bin/runtime.ts'; +import { Scheduler, Worker } from '../../worker/mod.ts'; + +Deno.test('startCombinedProcess installs the execution stream mutation hook', async () => { + const originalSetMutationHook = KvExecutionState.prototype.setMutationHook; + const originalSchedulerStart = Scheduler.prototype.start; + const originalWorkerStart = Worker.prototype.start; + const previousStreamsUrl = Deno.env.get('DURABLE_STREAMS_URL'); + let installedHook: ExecutionMutationHook | undefined; + let installationCount = 0; + + KvExecutionState.prototype.setMutationHook = function ( + this: KvExecutionState, + hook: ExecutionMutationHook, + ): void { + installationCount++; + installedHook = hook; + originalSetMutationHook.call(this, hook); + }; + Scheduler.prototype.start = function (): Promise<void> { + return Promise.resolve(); + }; + Worker.prototype.start = function (): Promise<void> { + return Promise.resolve(); + }; + Deno.env.set('DURABLE_STREAMS_URL', 'http://127.0.0.1:1'); + + try { + await startCombinedProcess({ definitions: new Map() }); + + assertEquals(installationCount, 1); + assertNotEquals(installationCount, 0); + assert(typeof installedHook === 'function'); + } finally { + KvExecutionState.prototype.setMutationHook = originalSetMutationHook; + Scheduler.prototype.start = originalSchedulerStart; + Worker.prototype.start = originalWorkerStart; + if (previousStreamsUrl === undefined) { + Deno.env.delete('DURABLE_STREAMS_URL'); + } else { + Deno.env.set('DURABLE_STREAMS_URL', previousStreamsUrl); + } + } +}); From 8b8362f07ff87ecc44083b0bd1cc8cbe2d99c431 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 14:29:25 +0200 Subject: [PATCH 30/57] chore(harness): rebind the two Fable review lanes to their documented Opus fallbacks Fable is prohibited for this lane until lifted. Nothing had to be stopped: zero live sessions in the lane and no Fable dispatch was ever made, the deep_analysis lane having been carried as optional and never used. Three bindings named Fable and each moves to the route lane-policy already documents -- review_codex to Opus 5 low, review_codex_complex unused because the rescope dropped PR-D to Sol medium, and deep_analysis to Opus if needed. Every substitution is deliberately Claude-family, because the policy exists to stop an OpenAI-authored change being reviewed by an OpenAI-family model. The cheap move under a Claude capacity squeeze is to review Codex work with Codex, and that is the one substitution forbidden, so it is named here rather than left implicit. The automatic evaluator is unaffected, verified at source: the phase-eval dispatcher can select only minimax, deepseek or qwen and throws on anything else. No Fable path exists. Refs #1403, #1380, #1549 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 35 +++++++++++++++++++ .../supervisor.md | 8 ++--- 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 64b8676fb9..6bf0398947 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -997,3 +997,38 @@ completed/failure 59e435c5d 12:03:58Z it in advance bought. - **Lane status:** #1436, #1415, #1530 closed. Remaining in 0.0.6: **#1403** (PR-B, next), **#1380** (PR-C), **#1549** (PR-D). #1378 and #1545 in 0.0.7. + +## D-31 — Fable prohibited for this lane; two review bindings moved to their documented Opus fallbacks + +- **Severity:** significant (route change), **minor in effect** (no dispatch was pending on Fable) +- **Recorded:** 2026-08-12, owner directive — Fable fully prohibited for the 0.0.6 lane until explicitly + lifted (95% quota until Saturday). No Fable for planning, research, implementation, review or evaluation; + if a configured route would select Fable, stop that dispatch and report it. +- **Nothing to stop.** Zero live sessions in this lane at the time of the directive (`agentic:codex-status` + filtered to this lane's worktrees → 0). PR-E was already merged; no Fable dispatch had ever been made in + this run — the `deep_analysis` lane was carried as "optional" and never used. +- **Three bindings named Fable; each rebound to the route `lane-policy.md` already documents:** + +| Lane | Was | Now | Why this is the sanctioned substitution | +| --- | --- | --- | --- | +| `review_codex` (PR-C, PR-D) | Fable 5 · low | **Opus 5 · low** | The § Review-pairing ladder lists Opus 5 · low as this lane's token-limit fallback | +| `review_codex_complex` | Fable 5 · medium | **not used** | PR-D dropped to Sol·medium in the rescope, so it pairs on `review_codex`. Had it stayed Sol·high the fallback is Opus 5 · medium | +| `deep_analysis` | Fable 5 · medium | **Opus 5** if needed | Kept Claude-family deliberately — see below | + +- **The invariant that must not be lost in the substitution.** `lane-policy.md` is explicit that the + Codex-review lanes fall back to **Claude · Opus** rather than to a Codex model, "so an OpenAI-authored + change is never reviewed by an OpenAI-family model — opposite-family review is never traded away for a + token-limit fallback". Every substitution above is Claude-family for exactly that reason. The tempting + cheap move — reviewing Codex work with Codex Sol because Claude capacity is constrained — is the one thing + the policy forbids, and it is worth naming so a future capacity squeeze does not quietly take it. + Note `deep_analysis`'s *published* fallback is Codex Sol · high; that is fine for orchestrator analysis in + general, but in this lane deep analysis would be analysing **Codex-authored** artifacts, so it stays on + Opus. +- **The automatic evaluator is unaffected, verified at source rather than assumed.** + `openhands-phase-eval.yml:103-106` maps only `eval:model:minimax` → `minimax-m3`, + `eval:model:deepseek` → `deepseek-v4-flash-0731`, `eval:model:qwen` → `qwen3.8-max`, and throws on any + unknown label. **There is no Fable path.** So PLAN-EVAL and IMPL-EVAL continue on the automated route + untouched — as already observed: MiniMax M3 for PLAN, DeepSeek V4 Flash 0731 for IMPL. +- **Effect on the remaining rail:** none on capability. PR-B is `light_implementation` (Sol · low), whose + pairing is `review_codex_light` → **Opus 5 · high**, which never involved Fable. PR-C and PR-D now review + on Opus 5 · low. Implementation stays Codex; formal evaluation stays automated. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md b/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md index d8dc8e84e2..9db74b2f1e 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/supervisor.md @@ -33,15 +33,15 @@ milestone by the owner — this lane does not touch it. | Task lane | Provider / model / effort | Role in this run | | --- | --- | --- | | `planning_decisions` | Claude · Anthropic · Opus 5 · high | This orchestrator session. Coordinates; does not write framework or tooling implementation code. | -| `deep_analysis` | Claude · Anthropic · Fable 5 · medium | Optional sub-agent for rail design questions (rename-vs-deletion policy, export-reachability strategy). | +| ~~`deep_analysis`~~ | **PROHIBITED — Fable barred for this lane (owner, 2026-08-12)**. Never dispatched in this run. If deep analysis is needed it stays Claude-family on **Opus 5**, so opposite-family analysis of Codex work is not traded away. | | `light_implementation` | Codex · OpenAI · GPT-5.6 Sol · low | Default implementer: PR-A, PR-B. Mobile-visible WSL daemon-attached thread. | | `normal_implementation` | Codex · OpenAI · GPT-5.6 Sol · medium | PR-C (decision-heavy: per-row rename-vs-deletion, RFC-location resolution). | | `complex_implementation` | Codex · OpenAI · GPT-5.6 Sol · high | PR-D (#1378 — export-reachability via `deno doc --json`, allowance registry, docs-fence extraction). | | `review_codex_light` | Claude · Anthropic · Opus 5 · high | Orchestrator slice review of Sol·low work (PR-A, PR-B). | -| `review_codex` | Claude · Anthropic · Fable 5 · low | Adversarial review of Sol·medium work (PR-C). | -| `review_codex_complex` | Claude · Anthropic · Fable 5 · medium | Adversarial review of Sol·high work (PR-D). | +| `review_codex` | ~~Fable 5 · low~~ → **Claude · Anthropic · Opus 5 · low** | Adversarial review of Sol·medium work (PR-C, PR-D). Rebound to the **documented token-limit fallback** in `lane-policy.md` § Review-pairing ladder, which is Claude-family by design. | +| ~~`review_codex_complex`~~ | **not used** | PR-D dropped to Sol·medium in the rescope (export-reachability moved to 0.0.7), so its pairing is `review_codex` above. Had it stayed Sol·high, the binding would be **Opus 5 · medium**, the documented fallback. | | `formal_plan_evaluation` | Codex · OpenAI · GPT-5.6 Sol · high | PLAN-EVAL of the quality-rail plan — Claude-authored plan ⇒ opposite family is Codex. | -| `formal_impl_evaluation` | Codex · OpenAI · GPT-5.6 Sol · xhigh **or** Claude · Fable 5 · medium | Per PR: whichever family is opposite the author of that PR's code. Codex-authored PRs ⇒ Fable 5 · medium. | +| `formal_impl_evaluation` | **automated — model resolved by the workflow** | Verified at source: `openhands-phase-eval.yml:103-106` can select only `minimax-m3`, `deepseek-v4-flash-0731`, or `qwen3.8-max`. **No Fable path exists**, so the prohibition does not touch the automatic evaluator. Observed: MiniMax M3 for PLAN, DeepSeek V4 Flash 0731 for IMPL. | | `light_implementation` (watcher) | Codex · OpenAI · GPT-5.6 Sol · low | CI watcher / minor green-up to merge-ready. | Routes are read from `.llm/harness/workflow/lane-policy.md`; the table above records only which From 72cef4e920ccf47eb71c5220f0344083f1f36b8b Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 14:50:49 +0200 Subject: [PATCH 31/57] chore(harness): record that #1566's 404 was a symptom of bookkeeping failing dispatch The reported defect was a label-removal race off an event snapshot. Review found two larger things: the fix self-blocked, because it imports a module absent from main so the introducing PR's own transition aborted the job and no evaluator was dispatched; and the structural defect was that housekeeping could fail dispatch at all, which is the same pathology as #1566 one level up. Resolved non-fatally and non-silently: the transition still rethrows so its outcome is truthful, an attributed step records the checkout outcome and a bounded reason, and dispatch depends on neither. The fix retires its own bootstrap problem, so #1567 demonstrates it on its own CI run. Refs #1566 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 39 +++++ .../slices/pr-f-1566/codex-thread-ids.md | 16 ++ .../slices/pr-f-1566/implement.md | 160 ++++++++++++++++++ 3 files changed, 215 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 6bf0398947..ca5633646c 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1032,3 +1032,42 @@ completed/failure 59e435c5d 12:03:58Z - **Effect on the remaining rail:** none on capability. PR-B is `light_implementation` (Sol · low), whose pairing is `review_codex_light` → **Opus 5 · high**, which never involved Fable. PR-C and PR-D now review on Opus 5 · low. Implementation stays Codex; formal evaluation stays automated. + +## D-32 — PR-F (#1566): the reported 404 was a symptom; the defect was bookkeeping able to fail dispatch + +- **Severity:** significant (structural fix beyond the reported symptom) +- **Recorded:** 2026-08-12 +- **Reported defect:** `openhands-phase-eval.yml`'s status cleanup removed `status:*` labels read from the + **event-payload snapshot**, so a concurrent dispatch's `removeLabel` returned `404 Label does not exist` + and reddened a run whose evaluation had succeeded. Verified before filing: runs `31596291515` (red) and + `31596293364` (green) two seconds apart on PR #1541 head `0503991ab`, and **exactly-once held** — one + trigger marker, `generation=29339092792`. Filed as **#1566**. +- **Two findings from the orchestrator review, the second larger than the issue:** + 1. **Bootstrap self-block.** The fix imports a new module from a **trusted base checkout**, and that module + is absent on `main`, so the introducing PR's own transition step threw `MODULE_NOT_FOUND`, aborted the + job, and the dispatch step never ran — no evaluator, so no PASS, so #1567 could not merge by its own + change. Explicitly **not** fixed by falling back to the PR-head copy, which would restore the very + escalation the trusted checkout prevents (this job holds `issues: write`). + 2. **The real defect is structural.** The job exists to dispatch exactly one evaluator; the label + transition is bookkeeping. A bookkeeping failure aborted the job and prevented the dispatch — the same + pathology as #1566 one level up, and a red run implying dispatch had failed when it had not. Fixing only + the 404 would have left that intact for the next hiccup. +- **Resolution:** both bookkeeping steps are `continue-on-error`; the transition still **rethrows** so its + outcome is truthfully `failure`; a dedicated step records actor / PR / head / **trusted-checkout outcome** / + bounded reason to `GITHUB_STEP_SUMMARY`; dispatch is gated only on `!cancelled()` and the chain-token check. + Non-fatal **and** non-silent, with the checkout outcome reported separately so the bootstrap case is + distinguishable from a permissions failure. +- **Consequence worth noting:** the fix retires its own bootstrap problem. #1567's ready flip now fails one + step, records the attributed diagnostic, and dispatches anyway — so **the PR demonstrates its own fix on its + own CI run**, and the `labeled`-path workaround is unnecessary. It self-heals on merge. +- **Two unprompted improvements by the implementer, both accepted:** the trusted-base checkout with + `persist-credentials: false` (and pinned to `base.ref`, the branch tip, not the recorded `base.sha` — the + #1552/#1564 lesson applied without being told), and sanitisation of the failure reason before it reaches a + Markdown step summary. +- **Epistemic honesty, self-reported:** the new workflow-policy test extracts named step blocks and asserts + the declared policy — static evidence that the YAML *declares* independence, not that the runner honours it. + Recorded as such in the slice `drift.md` rather than counted as behavioural coverage. The runner behaviour is + proven by #1567's own run. +- **Acceptance:** box 1 stands as worded — the concurrent-removal case is handled inside the caller so the + transition completes normally; the non-fatal path applies only to other failures. The implementer defended + the stronger reading with a reason instead of stretching the box. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/codex-thread-ids.md new file mode 100644 index 0000000000..9dc2c1177d --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-prf — Codex implementation thread +- **Thread / session id:** `019ff5f6-7626-7902-b922-1faf128c682f` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T14-33-17-019ff5f6-7626-7902-b922-1faf128c682f.jsonl` +- **Worktree:** `/home/codex/repos/ns006-labelrace` +- **Branch:** `fix/1566-phase-eval-label-race` @ `fe1d3b5e8` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1566-phase-eval-label-race`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-prf-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff5f6-7626-7902-b922-1faf128c682f -- "<follow-up>" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/implement.md new file mode 100644 index 0000000000..2cdbd92e8c --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/implement.md @@ -0,0 +1,160 @@ +use harness + +# PR-F — #1566: phase-eval status cleanup races on an event snapshot and 404s the run + +You are the **implementation agent** for a small, deterministic automation fix. The defect is fully +characterised and reproduced; your job is the fix plus the tests that prove it, not investigation. + +Your orchestrator is a Claude Opus 5 high session in `/home/codex/repos/netscript-006-internals`. It +holds merge authority. + +## SKILL + +- `netscript-harness` — run artifacts, slice discipline, commit trail. +- `netscript-tools` — scoped validation wrappers; what is a verdict and what is not. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` block. +- `openhands-handoff` — the phase-eval dispatch contract you must not break. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-labelrace` | +| Branch | `fix/1566-phase-eval-label-race` | +| Base | `e67c1ba13` (= `origin/main`) | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/` | +| Closes | #1566 | +| Route | Codex · gpt-5.6-sol · **low** | + +Work only in that worktree. No rebase, no force-push. Push with an explicit refspec. + +## The defect, already reproduced — do not re-investigate + +`.github/workflows/openhands-phase-eval.yml`, step **"Enter IMPL-EVAL status on ready transition"**: + +```js +const labels = context.payload.pull_request.labels.map((label) => label.name); +for (const label of labels.filter((name) => name.startsWith('status:'))) { + await github.rest.issues.removeLabel({ owner, repo, issue_number, name: label }); +} +await github.rest.issues.addLabels({ owner, repo, issue_number, labels: ['status:impl-eval'] }); +``` + +`context.payload.pull_request.labels` is a **snapshot from event-creation time**; the loop then issues +unconditional deletes against live state. + +Observed on PR #1541, head `0503991ab`, two dispatch runs **two seconds apart**: + +```text +31596291515 failure 12:24:36Z DELETE …/issues/1541/labels/status%3Aimpl - 404 +31596293364 success 12:24:38Z posted the authoritative trigger +``` + +**Exactly-once was not violated** — verified: one trigger marker for that head, +`generation=29339092792`. The generation dedup worked. Only the cleanup raced. So this is a **false red on +a PR whose evaluation succeeded**, which is the stimulus that teaches operators to discount red runs. It has +already cost this milestone an investigation into an apparent duplicate evaluator run that turned out not to +exist. + +## Design decision, made for you — the logic must become testable + +#1566's acceptance requires a **race regression test**. Inline `actions/github-script` JS in a workflow +cannot be unit-tested. So extract the decision into a checked-in module under `.github/scripts/` and have +the workflow call it — following the precedent already in that directory (`ci-classify-changes.ts` + +`ci-classify-changes.test.ts`, `draft-workflow-policy.test.ts`, `e2e-cli-event-policy.test.ts`). + +Shape that keeps it testable: a **pure** function deciding *which labels to remove and add* given the live +label set, plus a thin caller that performs the API calls and applies the narrow error tolerance. Inject the +GitHub client (or just the two operations you need) so the test can drive a client that throws on demand. +Do not invent a framework; match the neighbouring files' style. + +## Contract + +### C1 — cleanup is idempotent + +Read **live** labels immediately before removing (`issues.listLabelsOnIssue`) and remove only what is +actually present. Reading live is the primary fix; the tolerance in C2 is the belt to that braces, because +even a live read is racy in principle. + +### C2 — tolerance is narrow, and this is the part that is easy to get wrong + +Tolerate **only** a `404` that means *this label is not on this issue*. Rethrow everything else. + +A blanket `try { … } catch { }` around `removeLabel` **fails this slice**. It would swallow a `403` from a +permissions regression and a `404` from a wrong `issue_number`, converting a real failure into a silent +pass — which is the same false-green class the whole 0.0.6 internals lane exists to remove. Do not ship the +convenient version. + +Leave the workflow's `retry-exempt-status-codes` behaviour untouched; 404 being retry-exempt is correct. + +### C3 — generation deduplication is unchanged + +It already works under a genuine race. Do not restructure it, do not "improve" it. Prove it still holds +(acceptance box 4). + +### C4 — terminal state is exactly one `status:` label + +Per the taxonomy's single-status rule. After cleanup, `status:impl-eval` and nothing else `status:`-prefixed. + +## Acceptance mapping + +#1566 has **6** boxes. Read them from the live issue. Provide a fenced `acceptance-evidence` block in the PR +body using **`box-index: 1..6`** — **not** exact box text. Reason, learned expensively on PR #1560 two hours +ago: `acceptanceCheckboxes` keeps only each checkbox's **first raw line, backticks preserved**, so any box +that wraps in the issue body is unmatchable by exact text, and the author cannot see the wrapping. `box-index` +is stable against it. Do not repeat that failure. + +Box 2 is **proven RED** and box 3 is a **narrow-tolerance assertion** — both need tests that fail before your +change. Commit the failing tests, then the fix. + +## Gates — deliverables, not a checklist + +Paste real output with exit codes into your per-slice PR comment. + +| # | Gate | Command | +| --- | --- | --- | +| 1 | script tests | `deno test --allow-read --allow-env --allow-write --allow-run .github/scripts/` | +| 2 | scoped type-check | `.llm/tools/run-deno-check.ts --root .github/scripts --ext ts` | +| 3 | scoped lint | `.llm/tools/run-deno-lint.ts --root .github/scripts --ext ts` | +| 4 | scoped format | `.llm/tools/run-deno-fmt.ts --root .github/scripts --ext ts` | +| 5 | **asset-barrel freshness** | `deno task gen:assets-barrel`, then `git status --porcelain` **must be empty** | +| 6 | workflow YAML still parses | confirm the edited workflow loads (a `gh workflow view` or a YAML parse is fine) | + +Gate 5 is not optional and is not obvious: tool sources are embedded as strings in +`packages/cli/src/kernel/assets/*.generated.ts`, so touching a bundled file makes them stale and reds +`ci.yml`'s `quality` job. This cost PR-E a full CI cycle. Run it **before** you consider the slice done — +if `.github/scripts/` turns out not to be bundled, the command is a no-op and costs nothing, and the empty +`git status` is your proof either way. + +`deno task e2e:cli` is out of scope. Do **not** apply `ci:skip-e2e`/`ci:skip-scaffold` yourself; the +orchestrator decides labels. + +## PR mechanics + +1. First commit is the slice-dir bootstrap; open the **draft PR** in that same session and comment per slice. +2. Slice artifacts in `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/`: + `worklog.md`, `context-pack.md`, `drift.md`, updated in the **same commit** as the code they describe. +3. `## Scope` carries `Closes #1566` on its own line. Nothing else gets a closing keyword. +4. Labels: `type:fix`, `area:tooling`, `priority:p2`, `status:impl`, milestone `0.0.6`. Exactly one + `status:`. Do **not** apply `status:ready-merge` or `status:impl-eval`. +5. **Leave the PR in draft.** Draft → ready fires the formal IMPL-EVAL and is the orchestrator's action. +6. Resolve commit hashes in a **separate shell step** and paste the literal value — a previous slice posted + `(git rev-parse --short=10 HEAD)` unexpanded into two comments. + +## Boundaries + +- Touch only `.github/workflows/openhands-phase-eval.yml`, new/edited files under `.github/scripts/`, and + your slice dir. +- Do **not** change the workflow's `on:` triggers or dispatch conditions. +- Do **not** touch model resolution, the `eval:model:*` mapping, or the trusted-base-ref logic from #1552. +- Do **not** retrigger PR #1541 or comment on it. Its evaluation succeeded and its trigger is authoritative. +- Do **not** widen into #1564 (stale `base.sha` range computations). Same defect class, different surface. +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or `quality-allow:`. +- Do **not** merge. + +## Escalate instead of going idle + +If a gate is red and you cannot green it, or a contract here is wrong, write it in your slice `drift.md`, +post it as a PR comment, and continue with what is not blocked. On this lane escalation has twice found the +orchestrator's brief wrong rather than the code — that is a good outcome, so raise it. From 175d167d23de47b381f5df4542b1551236b96cfe Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 14:53:16 +0200 Subject: [PATCH 32/57] chore(harness): record that dispatch depends on the transition via event history My review predicted continue-on-error would make the bootstrap failure cosmetic. Wrong. The design worked -- transition failed truthfully, diagnostic fired, dispatch ran -- and dispatch then failed on its own with "No labeled-event generation found for status:impl-eval", because it requires a generation that only exists if the label was actually applied. So the steps are not independent: the dependency runs through GitHub's event history, not through step conditions. The implementer's independence claim was true of the conditions and false of the data flow, my review repeated it, and the slice's static workflow-policy test could not see it -- a limitation the implementer had itself flagged, which turned out to be the decisive one. Third time this session I reasoned one step past what I verified. Choosing the self-contained first landing, which removes the import rather than working around it, and declining the offered IMPL-EVAL waiver because this PR changes the evaluator dispatch path and running it is what has caught both defects. Refs #1566 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 56 +++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index ca5633646c..fd65885560 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1071,3 +1071,59 @@ completed/failure 59e435c5d 12:03:58Z - **Acceptance:** box 1 stands as worded — the concurrent-removal case is handled inside the caller so the transition completes normally; the non-fatal path applies only to other failures. The implementer defended the stronger reading with a reason instead of stretching the box. + +## D-33 — the two steps were never independent: dispatch depends on the transition through GitHub's event history + +- **Severity:** significant (falsifies my own review claim and the slice's independence test) +- **Recorded:** 2026-08-12, run `31598386001`, PR #1567 head `5b4d8caf5` +- **What I predicted:** that `continue-on-error` on the transition step would let dispatch proceed, so the + bootstrap `ERR_MODULE_NOT_FOUND` would be cosmetic and the PR would demonstrate its own fix. **Wrong.** +- **What actually happened, step by step:** + +```text +4. success Check out trusted phase-eval scripts +5. success Enter IMPL-EVAL status on ready transition (conclusion success, outcome failure) +6. success Record attributed IMPL-EVAL status-transition failure <- the diagnostic fired correctly +7. FAILURE Resolve and dispatch exactly one evaluator + Error: No labeled-event generation found for status:impl-eval. +``` + + So the `continue-on-error` design worked exactly as built — the transition failed truthfully, the attributed + diagnostic fired, and **dispatch did run**. Dispatch then failed on its own. + +- **The real dependency, and why it was invisible.** Dispatch resolves + `expectedStatus = phase === 'plan' ? 'status:plan-eval' : 'status:impl-eval'` and then requires a + **labeled-event generation** for it. That generation only exists if the label was actually applied, + producing a `labeled` event. The transition failed, so `status:impl-eval` was never applied, so no + generation existed, so dispatch threw. + + The two steps are therefore **not independent**: dispatch has a runtime data dependency on the transition, + mediated through **GitHub's event history** rather than through step `if:` conditions. The implementer's + claim that dispatch "has no dependency on checkout or transition outcomes" was true of the *conditions* and + false of the *data flow*, and my review repeated it. +- **This is exactly the gap the slice's own test could not close, and the implementer said so.** Its + workflow-policy test extracts named step blocks and asserts *declared* independence; it flagged that as + "static workflow-policy evidence, not a simulation of runner semantics". That caveat turned out to name the + decisive limitation. A test that asserts what the YAML declares cannot see a dependency carried in + repository state. +- **My error, for the third time this session:** I traced the step conditions, saw no dependency, and asserted + dispatch would *succeed* — reasoning one step past what I had verified. The first two were asserting an + absence without the probe that finds the presence (the sagas supersession record) and asserting a mechanism + without the probe that identifies it (the stale-base attribution). Same shape each time, and each time the + missing step was cheap: read the run, run the range, trace the data flow. +- **Owner direction:** return #1567 to draft/`status:impl` (done), fix the bootstrap shape before retry — + either keep the trusted workflow **self-contained** for the first landing (inline the narrow + live-label/idempotent cleanup, with its helper contract still independently tested) or split a helper-only + landing ahead of workflow adoption. No manual OpenHands trigger, no Fable. +- **Chosen: self-contained first landing.** It removes the import, so there is no bootstrap problem to work + around rather than a smaller one; the helper and its tests still land so acceptance boxes 2–5 stay provable; + and it is one PR rather than two. A follow-up can switch the workflow to import the helper **once it is + reachable from trusted `main`**, deleting the inline copy — the ordering that was wrong here, done in the + right order. +- **Declining the offered IMPL-EVAL waiver.** The owner offered `impl-eval:skip` under the small-deterministic + waiver. **Not taking it.** This PR changes the evaluator dispatch path itself, and `drift.md` D-14 records + this lane's position that a PR changing gate semantics is exactly the class needing an independent pass — a + position that has now been vindicated twice on this PR alone, since both the bootstrap self-block and this + event-history dependency were found by running the thing rather than reading it. With the import removed the + transition succeeds, the label is applied, the generation exists, and automatic DeepSeek runs exactly once. + The waiver would save one run and remove the only check that has caught anything here. From 763c8f1739b893ea8dc98fda3ccf5bf70d3a45b1 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 15:10:46 +0200 Subject: [PATCH 33/57] chore(harness): record the hard Fable prohibition in the run worklog Recorded at the owner's explicit instruction; the rebinding rationale stays in drift D-31 and the live bindings in supervisor.md. Compliance verified rather than asserted: codex-status shows only sol and luna models and zero Fable sessions, and this orchestrator has spawned no Claude subagents for the entire run -- every delegation went to Codex Sol through the launcher, and the deep_analysis lane was never used. Nothing to unwind, nothing in flight. Also names the substitution that must not be made: the Codex-review lanes fall back to Claude Opus rather than a Codex model, so an OpenAI-authored change is never reviewed by an OpenAI-family model. Under a Claude squeeze that is the cheap move and the one the policy forbids. Refs #1403, #1380, #1549, #1566 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../slices/pr-b-1403/codex-thread-ids.md | 16 ++ .../slices/pr-b-1403/implement.md | 182 ++++++++++++++++++ .../worklog.md | 44 +++++ 3 files changed, 242 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/codex-thread-ids.md new file mode 100644 index 0000000000..5188deb323 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-prb — Codex implementation thread +- **Thread / session id:** `019ff615-1929-7193-9dad-efa0c352e17f` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T15-06-45-019ff615-1929-7193-9dad-efa0c352e17f.jsonl` +- **Worktree:** `/home/codex/repos/ns006-qualitygate` +- **Branch:** `fix/1403-quality-gate-coverage` @ `059576fcd` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1403-quality-gate-coverage`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-prb-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff615-1929-7193-9dad-efa0c352e17f -- "<follow-up>" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/implement.md new file mode 100644 index 0000000000..00da49741f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/implement.md @@ -0,0 +1,182 @@ +use harness + +# PR-B — #1403: make `quality:gate` informative, in all three of the ways it currently is not + +You are the **implementation agent** for the p0 of the 0.0.6 internals quality rail. The plan passed a +formal PLAN-EVAL (cycle 5, `PASS`); your job is to implement it, not to redesign it. + +Your orchestrator is a Claude Opus 5 high session in `/home/codex/repos/netscript-006-internals`. It holds +merge authority and owns the draft → ready flip. + +## SKILL + +- `netscript-harness` — run artifacts, slice discipline, commit trail. +- `netscript-doctrine` — `arch:check`'s role, archetypes, fitness gates. **Read before touching + `check-doctrine.ts`.** +- `netscript-tools` — scoped wrappers; what is a verdict and what is not. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` block. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-qualitygate` | +| Branch | `fix/1403-quality-gate-coverage` | +| Base | `3c9dc1f39` (= `origin/main`) | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/` | +| Closes | #1403 | +| Route | Codex · gpt-5.6-sol · **low** | +| Plan | `plan-quality-rail.md` revision 4 (`PASS`), slices **B1–B3** in `worklog.md` § Design | + +Measured at your base: `deno task arch:check` exit **0**, `deno task quality:scan:repo` exit **0**. +Both green, so any red you produce is yours. + +## What is actually wrong — three independent defects, one gate + +#1403 was filed about **root lists**. Cross-lane work then found two more, and all three are now acceptance +boxes. A fix for one alone leaves a gate that looks covered and is not — which is the class this issue exists +to close, so do not stop at the first. + +### Defect 1 — the curated root list omits a package + +`deno.json`'s `arch:check` is `deps:check` plus **16** hand-listed `check-doctrine.ts --root` invocations in +one shell string. `packages/plugin-streams-core` is the only `plugin-*-core` package absent — an omission, +not a decision. + +### Defect 2 — the PR gate never scans `.llm/tools/**`, and skips entirely when nothing else changed + +`.github/workflows/code-quality.yml:36-42`: + +```bash +mapfile -t files < <(git diff --name-only --diff-filter=ACMR "$BASE" "$SHA" -- packages plugins) +args=(); for file in "${files[@]}"; do args+=(--changed-file "$file"); done +if ((${#args[@]})); then deno task quality:scan --pretty "${args[@]}"; fi +``` + +The pathspec is `-- packages plugins`, so a `.llm/tools/**` change can never enter the set; and +`if ((${#args[@]}))` means an empty set runs **no command** and reports success. Every PR in this rail — +including this one — is `.llm/tools`-only, so that step has been reporting success having executed nothing. + +### Defect 3 — the range is two-dot, so a stale base scans other lanes' merged work + +Same line: the two SHAs are passed as separate arguments, which is two-dot semantics — a literal tree +comparison. On PR #1539 that enumerated **nine** already-merged files belonging to other PRs and **zero** +lines of the PR under review. Audited across `main`, this is the **only** affected site: + +```text +surface-diff.yml:54 "$BASE_SHA...$HEAD_SHA" three-dot — safe +ci.yml:142 "$BASE_SHA...$HEAD_SHA" three-dot — safe +e2e-cli.yml:140 "$BASE_SHA...$HEAD_SHA" three-dot — safe +code-quality.yml:39 "$BASE" "$SHA" TWO-DOT — fix this one +``` + +`A...B` diffs from the **merge-base**, so a stale base self-heals — it is always a former `main` commit and +therefore an ancestor. Demonstrated on identical inputs: `cd24e1679 2a4102600` → 9 files; +`cd24e1679...2a4102600` → 0 files. + +**#1564** owns this root cause across consumers. **You fix only `code-quality.yml:39`.** Do not touch the +three safe sites; do not widen into #1564. + +## Contract + +### C1 — one transition to discovered roots, not two (rail `R-6`) + +Introduce `discoverDoctrineRoots()` in `.llm/tools/fitness/check-doctrine.ts` returning the **final** root +set, and repoint `arch:check` at it **in one step**. No interim list, no checked-in root data file. Two +earlier PLAN-EVAL cycles rejected a two-step version; do not reintroduce it. + +### C2 — the selector is the 36 top-level units, not every workspace member (rail `R-4`) + +Expanded top-level `packages/*` + `plugins/*` — **30 + 6 = 36**. Root `deno.json`'s workspace list also +includes `packages/cli/e2e`, `examples/*` and `apps/*`; those are **not** doctrine roots. +`packages/cli/e2e` is **excluded**, and that exclusion must be **stated in the doctrine document**, not left +implicit in code. + +### C3 — the coverage test must not ask the implementation what to expect + +Derive the expected set **independently** — enumerate publishable units from the filesystem or the workspace +list — then assert `discoverDoctrineRoots()` equals it. A test that computes its expectation *by calling the +function under test* cannot fail; PLAN-EVAL cycle 3 caught exactly that and it is a blocking defect here. + +### C4 — the PR gate must execute on a `.llm/tools`-only diff + +Widen the changed-file computation so `.llm/tools/**` is in scope, and make the **empty set** fail closed or +report "not scanned" explicitly — never silently green. Prove both red-first. + +### C5 — the range becomes three-dot + +One character at `code-quality.yml:39`, plus a fixture proving a stale recorded base no longer admits foreign +already-merged files. + +### C6 — surfaced findings are triaged, never fixed here (rail `R-8`) + +Running the repaired gate over newly covered surfaces will surface real findings. **Do not fix any of them.** +Write them into a triage list in your slice dir with file, line, rule, and a one-line assessment, and say +plainly how many there are. #1403 box 5 requires exactly this; a diff that "helpfully" cleans them up fails +the slice. If a finding is severe enough that you think it must be fixed now, say so and stop — the +orchestrator decides. + +## Acceptance mapping + +#1403 has **8** boxes (5 original + 3 added today; read them from the live issue). Provide a fenced +`acceptance-evidence` block using **`box-index: 1..8`** — **not** exact box text. `acceptanceCheckboxes` +keeps only each checkbox's **first raw line, backticks preserved**, so any box that wraps is unmatchable by +exact text and you cannot see the wrapping. This cost PR #1560 a full failed IMPL-EVAL cycle earlier today. + +## Gates — deliverables, not a checklist. Paste real output with exit codes. + +| # | Gate | Command | +| --- | --- | --- | +| 1 | fitness + quality tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/fitness/ .llm/tools/quality/` | +| 2 | doctrine (curated) | `deno task arch:check` — must stay **exit 0** | +| 3 | quality gate | `deno task quality:gate` — exit 0 | +| 4 | repo quality scan | `deno task quality:scan:repo` — must stay **exit 0** (it is green at your base; PR-E fixed it) | +| 5 | scoped check / lint / fmt | `.llm/tools/run-deno-{check,lint,fmt}.ts --root .llm/tools --ext ts` | +| 6 | **asset-barrel freshness** | `deno task gen:assets-barrel`, then `git status --porcelain` **must be empty** | +| 7 | new-workflow sanity | confirm `code-quality.yml` still parses | + +Gate 6 is mandatory and non-obvious: tool sources are embedded as strings in +`packages/cli/src/kernel/assets/*.generated.ts`, so editing a bundled tool makes them stale and reds +`ci.yml`'s `quality` job. This cost PR-E a CI cycle. The empty `git status` on a **second** run is also your +idempotence proof. + +Run **all** gates before you report done, so the head is final when the orchestrator flips to ready — that +flip triggers the formal IMPL-EVAL, and a commit landing after it invalidates the verdict. + +## PR mechanics + +1. First commit is the slice-dir bootstrap; open the **draft PR** in that same session, comment per slice. +2. Slice artifacts (`worklog.md`, `context-pack.md`, `drift.md`) updated in the **same commit** as the code + they describe. +3. `## Scope` carries `Closes #1403` on its own line. Reference `#1564` **without** a closing keyword — it + owns the shared root cause and stays open. +4. Labels: `type:fix`, `area:tooling`, `area:packages`, `priority:p0`, `status:impl`, milestone `0.0.6`. + Exactly one `status:`. +5. **Leave the PR draft.** The flip is the orchestrator's action. +6. **State gate claims as evidence, not buckets.** If a scaffold tier reports SUCCESS, say which step number + ran and whether step 2 was "Skipped by policy" — `scaffold-runtime: SUCCESS` is not a provable claim. + Likewise do **not** cite `quality:gate` as coverage of your own diff; on a `.llm/tools` change it is + precisely the defect you are fixing. +7. Resolve commit hashes in a separate shell step and paste literal values. + +## Boundaries + +- Touch only `.llm/tools/fitness/**`, `.llm/tools/quality/**`, `deno.json` tasks, + `.github/workflows/code-quality.yml`, `docs/architecture/doctrine/` for the C2 exclusion statement, and + your slice dir. +- Do **not** change `arch:check:repo`'s behaviour, the A14 rule, or the doctrine verdict table — that is + PR-C (#1380), which consumes your `discoverDoctrineRoots()` unchanged. +- Do **not** add export-awareness, allowance issue-links, `--max-allow` wiring, or docs-fence scanning — + that is PR-D (#1549). +- Do **not** touch `surface-diff.yml`, `ci.yml` or `e2e-cli.yml`; all three are already three-dot. +- Do **not** fix findings the repaired gate surfaces (C6). +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or `quality-allow:`. +- Do **not** merge, flip to ready, or apply `status:ready-merge` / `status:impl-eval` / `impl-eval:skip`. + +## Escalate instead of going idle + +If a gate is red and you cannot green it, or a contract here is wrong, write it in your slice `drift.md`, +post it as a PR comment, and continue with what is not blocked. On this lane escalation has three times found +the orchestrator's brief wrong rather than the code — missing `--allow-write`, then `--allow-run`, then the +asset-barrel gate. Raising it is the expected behaviour, not a failure. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index 1924ccf761..8a87ce2197 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -387,3 +387,47 @@ To add a doctrine-gated unit: create `packages/<name>/` with a `deno.json`; `dis picks it up and `arch:check` gates it with no task edit. To add a quality allowance: append `// quality-allow: <reason> (#<open issue>)` on the offending line and raise `--max-allow` in the same PR as the issue link — the budget can only fall otherwise. + +--- + +## Constraint: Fable fully prohibited for all remaining 0.0.6 work (owner, 2026-08-12) + +Recorded here in the worklog at the owner's explicit instruction; the route rebinding and its rationale are +in `drift.md` **D-31**, and `supervisor.md` § Routes in force carries the live bindings. + +**The policy.** Fable is prohibited for all remaining 0.0.6 work until explicitly lifted (quota at 95% until +Saturday). No Fable session or subagent may be launched **or resumed**. Continue on native Opus 5 +orchestration and subagents, Codex GPT-5.6 Sol implementation, and the **automatic** evaluator workflow only. +Never manually trigger OpenHands. + +**Compliance, verified rather than asserted.** Checked at the time of the directive: + +```text +deno task agentic:codex-status → models seen: ['gpt-5.6-luna', 'gpt-5.6-sol'] + FABLE sessions: 0 +``` + +No Fable session exists to stop or avoid resuming. This orchestrator has spawned **no Claude subagents at +all** for the entire run — every delegation went to Codex Sol through `agentic:launch-codex-slice`, and the +`deep_analysis` (Fable 5 · medium) lane was carried as optional in `supervisor.md` and never used. So there is +no Fable work to unwind and none in flight. + +**Bindings in force for the remaining rail** (PR-B/C/D and #1566): + +| Role | Route | +| --- | --- | +| Orchestration | Claude · Opus 5 · high — this session | +| Implementation | Codex · GPT-5.6 Sol · low/medium | +| Adversarial review of Sol·low work | Claude · Opus 5 · high (`review_codex_light`) | +| Adversarial review of Sol·medium work | Claude · Opus 5 · low — was Fable 5 · low, rebound to `lane-policy.md`'s documented fallback | +| Formal PLAN/IMPL-EVAL | **automatic** phase-eval workflow; model resolved by it (verified at source: only `minimax-m3`, `deepseek-v4-flash-0731`, `qwen3.8-max` — no Fable path exists) | +| Deep analysis, if needed | Claude · Opus 5 — deliberately **not** the published Codex Sol fallback, because it would be analysing Codex-authored artifacts | + +**The substitution that must not be made.** `lane-policy.md` requires the Codex-review lanes to fall back to +**Claude · Opus**, not to a Codex model, "so an OpenAI-authored change is never reviewed by an OpenAI-family +model". Under a Claude capacity squeeze the cheap move is to review Codex work with Codex; that is the one +substitution the policy forbids, and it is named here so a later squeeze does not quietly take it. + +**No manual OpenHands trigger has been or will be issued.** Every formal evaluation in this run reached the +evaluator through the `openhands` + `status:plan-eval` label pair or the draft → ready transition. That is +also why the earlier "duplicate run" scare resolved as no-op skip events rather than duplicate execution. From c21b56891a8ec85f72ffc99865a89e402a84f150 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 15:14:35 +0200 Subject: [PATCH 34/57] chore(harness): record that Canary.3 cut ownership is the runtime lane's Supersedes part of D-26 on who cuts, not on the duty to report. This lane declares no canary and runs no publish step, per D-3, so nothing had to stand down. One coordination action was warranted: the fixes lane had told me it would bring a first-coherent-checkpoint recommendation to the owner once #1539 landed, and #1539 has landed. Notifying them that Canary.3 authority sits with the runtime lane prevents two lanes arriving at the owner with cut recommendations, which is the collision the directive guards against. Refs #1403, #1380, #1549, #1566 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index fd65885560..19f97e4bdc 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1127,3 +1127,22 @@ completed/failure 59e435c5d 12:03:58Z event-history dependency were found by running the thing rather than reading it. With the import removed the transition succeeds, the label is applied, the generation exists, and automatic DeepSeek runs exactly once. The waiver would save one run and remove the only check that has caught anything here. + +## D-34 — Canary.3 cut ownership sits with the runtime lane; this lane does not cut and does not compete + +- **Severity:** minor (coordination), supersedes part of **D-26** +- **Recorded:** 2026-08-12, owner directive +- **Policy:** the **runtime** lane alone owns the Canary.3 cut, after merged **#1558**. No competing release is + to be started. This lane continues its assigned issues. +- **Effect on D-26.** D-26 recorded that 0.0.6 may cut intermediary canaries and that this lane **reports** + merged gate-trust checkpoints to the fixes/release coordinator. The reporting duty stands; what changes is + **who cuts**. D-26 pointed reporting at the fixes lane, which had described itself as release-checkpoint + coordinator and was assembling a cut recommendation for the owner. Canary.3 authority is the runtime lane's. +- **This lane's position is unchanged and was never at risk:** `drift.md` D-3 recorded on day one that this + lane declares no canary point and runs no publish step. Nothing to stand down. +- **Action taken to prevent an accidental competing process:** notified the fixes lane that Canary.3 cut + authority is the runtime lane's, because they had explicitly told me they would bring a + first-coherent-checkpoint recommendation to the owner once #1539 landed — and #1539 **has** landed + (`3c9dc1f39`). Without that note, two lanes could arrive at the owner with cut recommendations, which is the + exact collision the directive guards against. Payload facts were sent to them; the cut decision is not + this lane's to route. From 2cac13d0b5ca7e713c52eb5b90f9b23b0eee01a4 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 15:17:17 +0200 Subject: [PATCH 35/57] chore(harness): record that resume is prohibited too and this lane has nothing resumable The fixes lane pointed out that a SendMessage to a completed Fable evaluator would restart a Fable session without looking like a launch, and marked their wave-1 evaluator agents dead for that reason. Checked rather than assumed: every recorded thread in this run is openai/gpt-5.6-sol at one effort, no Fable thread id appears anywhere, and this orchestrator has spawned no Claude subagents at all. Nothing to mark dead. Keeping the general rule: a completed agent is a resumable agent, so "I am not launching anything" is not the same claim as "I am not starting a prohibited model". Refs #1403, #1380, #1549, #1566 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../worklog.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index 8a87ce2197..a93e39be20 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -428,6 +428,30 @@ no Fable work to unwind and none in flight. model". Under a Claude capacity squeeze the cheap move is to review Codex work with Codex; that is the one substitution the policy forbids, and it is named here so a later squeeze does not quietly take it. +### Resume is prohibited too, and this lane has nothing resumable — checked + +The fixes lane raised a subtlety worth adopting: the prohibition covers **resume**, not only launch, because a +`SendMessage` to a completed Fable evaluator agent would restart a Fable session **without looking like a +launch**. They marked wave 1's three completed Fable evaluator agents dead to their run for exactly that +reason. + +Checked here rather than assumed. Every recorded implementation thread in this run: + +```text +grep "Requested route\|Observed route" .llm/runs/…/slices/*/codex-thread-ids.md | sort -u +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=low +``` + +One distinct route across every slice, and no Fable thread id recorded anywhere. Combined with the fact that +this orchestrator has spawned **no Claude subagents at all** — every delegation went through +`agentic:launch-codex-slice` — there is no Fable session in this lane's history that a resume could restart. +Nothing to mark dead. + +Worth keeping as a rule regardless: **a completed agent is a resumable agent.** "I am not launching anything" +is not the same claim as "I am not starting a prohibited model", and the second is the one the policy is +about. + **No manual OpenHands trigger has been or will be issued.** Every formal evaluation in this run reached the evaluator through the `openhands` + `status:plan-eval` label pair or the draft → ready transition. That is also why the earlier "duplicate run" scare resolved as no-op skip events rather than duplicate execution. From 1961e7790cc98a6b88f338ce8756ec2bafba18fe Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 15:46:37 +0200 Subject: [PATCH 36/57] chore(harness): record #1566 merged and the follow-up now unblocked PR #1567 merged at b79eca5d6, closing #1566. Merged by the release coordinator during a Claude 529 outage and re-verified here rather than accepted: main contains it, the helper is present on trusted main, both items are status:shipped. The deliberate follow-up is now valid and only now: the workflow shipped a transcription rather than an import, because importing a module absent from trusted main is what made the first attempt self-block. The general shape is that a change introducing a trusted-path dependency cannot also be the change that first consumes it. Refs #1566, #1403, #1380, #1549 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../cut-trace.md | 1 + .../drift.md | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 7e5802e7d9..867ab8bd71 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,6 +8,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | +| 3 | 2026-08-12 13:40Z | `b79eca5d6` | **#1567** | **#1566** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` 0 current failures at `f6def9946`; `close-gate` PASS; 6/6 issue boxes mirrored; the one PR-body DoD box untickable at hand-off ticked by the orchestrator with run identities as evidence; `review-threads` PASS. **Merged by the release coordinator during a Claude 529 outage**, independently re-verified here: main contains it, `phase-eval-status.mjs` is present on trusted main, both items `status:shipped`. Verdict was head-matched to `f6def9946` before consumption (one trigger `gen=29340872564`, one verdict run `31599209037`). | | 2 | 2026-08-12 ~12:24Z | `e67c1ba13` | **#1560** | **#1530** (auto-closed `COMPLETED`) | 7/7 pass — `pr-checks` **23 checks, 0 current failures** at `28fc1b423`; `close-gate` green after label + workflow **re-run** (not a push); boxes 1–6 mirrored, box 7 `[post-merge]` verified after merge and ticked with evidence; `review-threads` PASS. **Two IMPL-EVAL verdicts:** `PASS` at `49e2b86e9` (pre barrel fix) then **`FAIL_FIX`** at `9ab361440` on a real close-gate defect, then `PASS` at `28fc1b423`. Consuming the first would have merged a red gate. | | 1 | 2026-08-12 ~08:31Z | `63cd1cd58` | **#1527** | **#1436**, **#1415** (both auto-closed `COMPLETED`) | 7/7 pass — full record in the PR's `[PRE-MERGE GATE]` comment. `pr-checks` **15/15 `current-pass`, 0 current failures** at `dfda54a16`; `close-gate` green; #1415 4/4 boxes mirrored with linked evidence; #1436 has 0 boxes so the PR body is its record; no new ignores/casts, no lock churn; both probes re-run independently by the orchestrator; `review-threads` PASS (0 threads). | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 19f97e4bdc..c5abf04f17 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1146,3 +1146,21 @@ completed/failure 59e435c5d 12:03:58Z (`3c9dc1f39`). Without that note, two lanes could arrive at the owner with cut recommendations, which is the exact collision the directive guards against. Payload facts were sent to them; the cut decision is not this lane's to route. + +## D-35 — #1566 closed; the deliberate follow-up is now unblocked, in the order that failed the first time + +- **Recorded:** 2026-08-12 +- **Merge:** PR #1567 → `b79eca5d6`, closing **#1566** (`CLOSED/COMPLETED`, `status:shipped`). Merged by the + release coordinator during a Claude 529 outage on this session. **Independently re-verified here rather than + accepted:** `origin/main` contains `b79eca5d6`; `.github/scripts/phase-eval-status.mjs` is present on main; + #1566 and PR #1567 both carry `status:shipped`. +- **The follow-up is now valid, and only now.** PR #1567 landed the workflow carrying a **transcription** of + the helper rather than importing it, because importing a module absent from trusted `main` is what made the + first attempt self-block (`ERR_MODULE_NOT_FOUND` → no `status:impl-eval` → no labeled-event generation → no + evaluator). The helper is now reachable from trusted `main`, so a follow-up may switch the workflow to + import it and delete the inline copy. Same two stages, in the order that works. +- **Worth keeping as the general shape:** a change that introduces a trusted-path dependency cannot also be + the change that first consumes it. The consuming edit has to come after the dependency is on the protected + branch. That is not a quirk of this workflow — it applies to any `getContent`/checkout-from-base pattern, + and this lane paid one full ready-flip cycle to learn it. +- **Lane state:** #1436, #1415, #1530, #1566 **closed**. Remaining: **#1403** (PR #1570), **#1380**, **#1549**. From ba80d2a8161204f58cb67b3f86ce316d765927a8 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 15:53:29 +0200 Subject: [PATCH 37/57] chore(harness): record #1580 as the owner of #1570's fresh-ui red, and hold #1570 immutable Verified rather than accepted: #1570 touches zero fresh-ui files, the fresh-ui lock is untouched since #1558, and the failing step is the frozen type-check reporting a stale private lock. #1580 owns it and the runtime lane repairs it. Not absorbing it even though the fix is one line: it would put a packages/fresh-ui change on a .llm/tools gate-coverage PR, which is the class this lane's own pre-merge check 6 exists to catch, and #1570's diff would stop matching its stated scope. Immutability is the operative constraint -- #1570 is mid-IMPL-EVAL at c740ff6e0 and any push moves the head and invalidates the verdict, which this lane has already paid for twice. Sequence is verdict, then #1580, then sync, then re-run. Refs #1403, #1580 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index c5abf04f17..792376bdbc 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1164,3 +1164,37 @@ completed/failure 59e435c5d 12:03:58Z branch. That is not a quirk of this workflow — it applies to any `getContent`/checkout-from-base pattern, and this lane paid one full ready-flip cycle to learn it. - **Lane state:** #1436, #1415, #1530, #1566 **closed**. Remaining: **#1403** (PR #1570), **#1380**, **#1549**. + +## D-36 — #1570's `fresh-ui-quality` red is a foreign defect (#1580); PR held immutable through the evaluator + +- **Severity:** minor for this lane, **p0 for the repo** (owned elsewhere) +- **Recorded:** 2026-08-12, owner directive +- **Directive:** #1570's `fresh-ui-quality` failure is proven unrelated to its tooling diff. New **p0 #1580** + in 0.0.6 owns the missing generated `packages/fresh-ui/deno.lock` dependency entry, absent since #1558, and + the **runtime lane** repairs it. Do **not** absorb it into #1570. Keep #1570 **immutable** during the + evaluator run. After #1580 merges, use the normal current-main sync / status lifecycle to obtain a green + Fresh UI quality. +- **Independently verified rather than accepted**, three ways: + +```text +git diff --name-only origin/main...HEAD | grep -c fresh-ui → 0 (#1570 touches no fresh-ui file) +git diff --name-only 5705aeb19..origin/main -- packages/fresh-ui/deno.lock → (empty) + (lock untouched since #1558, as reported) +fresh-ui-quality log: "Fresh UI private lock is stale. … run deno task --cwd packages/fresh-ui lock:update" + step: "Frozen package type-check", deno check --lock=deno.lock --frozen +``` + + #1580 is live: `OPEN`, milestone 0.0.6, `priority:p0`, `area:fresh-ui`+`area:deps`. + +- **Why not absorbing it is the right call even though the fix is one line.** The remedy is a generated-lock + regeneration in a package this PR does not touch. Folding it in would put a `packages/fresh-ui` change on a + `.llm/tools` gate-coverage PR — the #1020/#1079 class this lane's own pre-merge check 6 exists to catch, and + it would make #1570's diff no longer match its stated scope. A one-line fix in the wrong PR is still the + wrong PR. +- **Immutability matters here specifically.** #1570 is mid-IMPL-EVAL at head `c740ff6e0`. Any push — including + a one-line lock fix or a main re-sync — moves the head and invalidates the verdict, which this lane has + already paid for twice (`drift.md` D-21, D-23). So the sequence is: verdict first, then #1580 merges, then + sync forward, then re-run the gate. Not the other way round. +- **Consequence for the merge order.** #1570 cannot merge green until #1580 lands, so this lane's #1403 now + has an external dependency it did not have an hour ago. Recorded rather than worked around; nothing in + #1570 changes. From d9c306ca9b222b3acdc0f4992aedf8de68d55433 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 16:16:53 +0200 Subject: [PATCH 38/57] chore(harness): record the control rule and the merge-forward safety test #1580 closed on this lane's evidence. Two reusable rules came out of it. A one-sided after-measurement is an assertion; a control makes it evidence. The runtime lane had the green run, this lane had the red run at the head immediately before, and neither could produce the pair alone. And a merge-forward on another lane's PR is safe precisely when the owned paths are byte-identical -- one diff decides whether the same head move is harmless or verdict-destroying. That is why #1570 owes no re-evaluation despite its head moving mid-eval, where twice before on this lane it did. Also records the post-merge-marker versus Refs distinction for circular acceptance boxes, which went back the other way. Refs #1403, #1580 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 792376bdbc..a0454e7c40 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1198,3 +1198,49 @@ fresh-ui-quality log: "Fresh UI private lock is stale. … run deno task --cwd p - **Consequence for the merge order.** #1570 cannot merge green until #1580 lands, so this lane's #1403 now has an external dependency it did not have an hour ago. Recorded rather than worked around; nothing in #1570 changes. + +## D-37 — two reusable rules from the #1580 exchange, both earned rather than asserted + +- **Recorded:** 2026-08-12, from the runtime lane's #1580 closure +- **Outcome:** #1580 closed `COMPLETED` on this lane's evidence — `fresh-ui-quality` run `31605497993` at + head `807d29003` (success), against the prior `31603374728` at `c740ff6e0` (failure, "Fresh UI private lock + is stale", `failedBatches` 2/2). #1570's `fresh-ui-quality` is green and the transition spans exactly their + fix. + +### Rule 1 — a control turns an after-measurement into evidence + +The runtime lane had the **after** (green on fixed main). This lane supplied the **before** (the red run at +the pre-fix head, with the failing step and its message). Their framing, worth keeping verbatim in substance: +*a one-sided after is an assertion; a control is evidence.* Neither lane could have produced the pair alone — +they could not see my PR's earlier run in context, and I could not have known their fix was the only variable. + +This generalises past locks: for any "X is now green" claim, the cheap upgrade is to name the red run at the +head immediately before, so the transition brackets exactly one change. + +### Rule 2 — a merge-forward on another lane's PR is safe precisely when the owned paths are unchanged + +`gh pr update-branch` on #1570 moved its head mid-evaluation. That is normally a verdict-invalidating event on +this lane (`drift.md` D-21, D-23 — twice paid). It was **not** this time, and the difference is one command: + +```text +git diff --stat c740ff6e0..807d29003 -- .llm/tools/quality .llm/tools/fitness deno.json \ + .github/workflows/code-quality.yml docs/architecture/doctrine → EMPTY +``` + +Byte-identical owned paths ⇒ the evaluated implementation is the shipped implementation ⇒ the `PASS` at +`c740ff6e0` (run `31603391461`, trigger `gen=29343539580 head=c740ff6e0`) still names what merges, and no +re-evaluation is owed. Provenance preserved **on proof**, not on assumption, and no evaluator capacity spent +re-running an identical tree. + +The runtime lane adopted this as a rule and noted the ordering correction themselves: run the diff **before** +the merge-forward, not after. Recorded here because the asymmetry is the useful part — the same head move is +either harmless or verdict-destroying depending on a fact that takes one command to establish. + +### Reciprocal: `[post-merge]` vs `Refs` for a circular acceptance box + +They hit boxes that cannot be true until the enabling fix merges (#1571 wanting terminal-green Canary.3, +#1580 wanting green `fresh-ui-quality` on #1570) and resolved with `Refs` plus a follow-through section, which +defers the close by hand. This lane's `[post-merge]` marker (#1530 box 7, working precedent — the mirror +prints "excluded post-merge box(es) … verify in a follow-up comment and tick after merge") keeps the closing +keyword and the auto-close instead. The distinction they drew back is the right one: the marker fits a +**genuinely post-merge-only** fact; `Refs` fits a box blocked on **another PR**. From ffb5879ef53e2bb2532937340eb15baea7de0828 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 16:32:36 +0200 Subject: [PATCH 39/57] chore(harness): record #1403 closed and arch:check gating 36 units PR #1570 merged at e391f3aec. Verified on merged main rather than inferred from the PR: arch:check exit 0 while now iterating 36 discovered roots, and quality:scan:repo exit 0. Green at 36 with A14 origin-aware is a materially different claim from the old green at 16 hand-listed roots, which excluded packages/cli and therefore hid 54 A14 findings. Three defects on one gate are closed: the root omission, the PR gate never scanning .llm/tools and skipping entirely on an empty set, and the two-dot range that enumerated other lanes' merged files. Records that R-5 moved into PR-B after the implementer caught the plan's incoherent split, so #1380 box 5 is implemented there and PR-C ticks it citing this commit. Refs #1403, #1380, #1549 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .github/scripts/phase-eval-status.mjs | 59 ++++ .github/scripts/phase-eval-status.test.ts | 166 ++++++++++ .llm/runs/docs-1377-content--leaf/audit.md | 48 +++ .../docs-1377-content--leaf/context-pack.md | 85 +++++ .llm/runs/docs-1377-content--leaf/drift.md | 124 +++++++ .../docs-1377-content--leaf/supervisor.md | 49 +++ .llm/runs/docs-1377-content--leaf/worklog.md | 306 ++++++++++++++++++ .../slices/implement-1459.md | 133 ++++++++ .../slices/plan-1459.md | 258 +++++++++++++++ .../slices/research-1459.md | 121 +++++++ .../slices/worklog-1459.md | 143 ++++++++ .../cut-trace.md | 3 +- .../drift.md | 29 ++ .../slices/pr-b-1403/context-pack.md | 35 ++ .../slices/pr-b-1403/drift.md | 57 ++++ .../slices/pr-b-1403/triage.md | 31 ++ .../slices/pr-b-1403/worklog.md | 108 +++++++ .../slices/pr-f-1566/context-pack.md | 44 +++ .../slices/pr-f-1566/drift.md | 23 ++ .../slices/pr-f-1566/worklog.md | 155 +++++++++ .llm/tools/fitness/check-doctrine_test.ts | 72 +++++ .llm/tools/quality/changed-source-files.ts | 39 +++ .../quality/changed-source-files_test.ts | 89 +++++ .../site/reference/plugin-sagas-core/index.md | 156 +++++++++ .../reference/plugin-streams-core/index.md | 180 +++++++++++ .../reference/plugin-triggers-core/index.md | 259 +++++++++++++++ .../reference/plugin-workers-core/index.md | 148 +++++++++ .../fresh/src/application/defer/island.ts | 8 + .../tests/defer-island-client-bundle_test.ts | 64 ++++ .../fixtures/defer-island-client/client.ts | 1 + .../fixtures/defer-island-client/main.ts | 5 + .../fixtures/defer-island-client/otel-api.ts | 50 +++ .../defer-island-client/vite.config.ts | 17 + 33 files changed, 3064 insertions(+), 1 deletion(-) create mode 100644 .github/scripts/phase-eval-status.mjs create mode 100644 .github/scripts/phase-eval-status.test.ts create mode 100644 .llm/runs/docs-1377-content--leaf/audit.md create mode 100644 .llm/runs/docs-1377-content--leaf/context-pack.md create mode 100644 .llm/runs/docs-1377-content--leaf/drift.md create mode 100644 .llm/runs/docs-1377-content--leaf/supervisor.md create mode 100644 .llm/runs/docs-1377-content--leaf/worklog.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/implement-1459.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/plan-1459.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/research-1459.md create mode 100644 .llm/runs/release-0.0.6-features--orchestration/slices/worklog-1459.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/context-pack.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/drift.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/triage.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/worklog.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/context-pack.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/drift.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/worklog.md create mode 100644 .llm/tools/fitness/check-doctrine_test.ts create mode 100644 .llm/tools/quality/changed-source-files.ts create mode 100644 .llm/tools/quality/changed-source-files_test.ts create mode 100644 docs/site/reference/plugin-sagas-core/index.md create mode 100644 docs/site/reference/plugin-streams-core/index.md create mode 100644 docs/site/reference/plugin-triggers-core/index.md create mode 100644 docs/site/reference/plugin-workers-core/index.md create mode 100644 packages/fresh/src/application/defer/island.ts create mode 100644 packages/fresh/tests/defer-island-client-bundle_test.ts create mode 100644 packages/fresh/tests/fixtures/defer-island-client/client.ts create mode 100644 packages/fresh/tests/fixtures/defer-island-client/main.ts create mode 100644 packages/fresh/tests/fixtures/defer-island-client/otel-api.ts create mode 100644 packages/fresh/tests/fixtures/defer-island-client/vite.config.ts diff --git a/.github/scripts/phase-eval-status.mjs b/.github/scripts/phase-eval-status.mjs new file mode 100644 index 0000000000..7348d6f4ff --- /dev/null +++ b/.github/scripts/phase-eval-status.mjs @@ -0,0 +1,59 @@ +/** Prefix reserved for the single lifecycle status label. */ +export const STATUS_PREFIX = 'status:'; + +/** Terminal status entered before dispatching IMPL-EVAL. */ +export const IMPL_EVAL_STATUS = 'status:impl-eval'; + +/** GitHub's exact response message when a label is absent from an issue. */ +export const MISSING_LABEL_MESSAGE = 'Label does not exist'; + +/** + * Decide the idempotent status-label transition from a live issue-label set. + * + * @param {readonly string[]} liveLabels + * @returns {{ remove: string[], add: string[] }} + */ +export function decideImplEvalStatusTransition(liveLabels) { + return { + remove: liveLabels.filter((label) => label.startsWith(STATUS_PREFIX)), + add: [IMPL_EVAL_STATUS], + }; +} + +/** + * Apply the IMPL-EVAL transition through injected GitHub label operations. + * + * @param {{ + * listLabelsOnIssue: () => Promise<string[]>, + * removeLabel: (label: string) => Promise<void>, + * addLabels: (labels: string[]) => Promise<void>, + * }} operations + */ +export async function applyImplEvalStatusTransition(operations) { + const liveLabels = await operations.listLabelsOnIssue(); + const decision = decideImplEvalStatusTransition(liveLabels); + + for (const label of decision.remove) { + try { + await operations.removeLabel(label); + } catch (error) { + if (!isMissingLabelError(error)) throw error; + } + } + + await operations.addLabels(decision.add); +} + +/** @param {unknown} error */ +function isMissingLabelError(error) { + if (!isRecord(error) || error.status !== 404) return false; + const response = error.response; + if (!isRecord(response)) return false; + const data = response.data; + return isRecord(data) && data.message === MISSING_LABEL_MESSAGE; +} + +/** @param {unknown} value */ +function isRecord(value) { + return typeof value === 'object' && value !== null; +} diff --git a/.github/scripts/phase-eval-status.test.ts b/.github/scripts/phase-eval-status.test.ts new file mode 100644 index 0000000000..4f1fd9f923 --- /dev/null +++ b/.github/scripts/phase-eval-status.test.ts @@ -0,0 +1,166 @@ +import { assertEquals, assertRejects, assertStringIncludes } from '@std/assert'; +import { + applyImplEvalStatusTransition, + decideImplEvalStatusTransition, + IMPL_EVAL_STATUS, + MISSING_LABEL_MESSAGE, +} from './phase-eval-status.mjs'; + +interface IssueLabelOperations { + listLabelsOnIssue(): Promise<string[]>; + removeLabel(label: string): Promise<void>; + addLabels(labels: string[]): Promise<void>; +} + +function workflowStep(source: string, name: string): string { + const lines = source.split('\n'); + const start = lines.indexOf(` - name: ${name}`); + if (start < 0) throw new Error(`Missing workflow step: ${name}`); + let end = lines.length; + for (let index = start + 1; index < lines.length; index += 1) { + if (lines[index].startsWith(' - name: ')) { + end = index; + break; + } + } + return lines.slice(start, end).join('\n'); +} + +function operations( + labels: string[], + removeError?: (label: string) => unknown, +) { + const removed: string[] = []; + const added: string[][] = []; + const client: IssueLabelOperations = { + listLabelsOnIssue: () => Promise.resolve(labels), + removeLabel: (label: string) => { + const error = removeError?.(label); + if (error !== undefined) return Promise.reject(error); + removed.push(label); + return Promise.resolve(); + }, + addLabels: (next: string[]) => { + added.push(next); + return Promise.resolve(); + }, + }; + return { client, removed, added }; +} + +Deno.test('race regression: a concurrently removed status label does not fail cleanup', async () => { + const { client, removed, added } = operations( + ['status:impl', 'area:tooling'], + () => ({ status: 404, response: { data: { message: 'Label does not exist' } } }), + ); + + await applyImplEvalStatusTransition(client); + + assertEquals(removed, []); + assertEquals(added, [['status:impl-eval']]); +}); + +Deno.test('narrow tolerance: permission failures still fail cleanup', async () => { + const { client } = operations( + ['status:impl'], + () => ({ + status: 403, + response: { data: { message: 'Resource not accessible by integration' } }, + }), + ); + + await assertRejects(() => applyImplEvalStatusTransition(client)); +}); + +Deno.test('narrow tolerance: an unrelated 404 still fails cleanup', async () => { + const { client } = operations( + ['status:impl'], + () => ({ status: 404, response: { data: { message: 'Not Found' } } }), + ); + + await assertRejects(() => applyImplEvalStatusTransition(client)); +}); + +Deno.test('terminal decision contains exactly one status label', () => { + const decision = decideImplEvalStatusTransition([ + 'type:fix', + 'status:impl', + 'status:plan-eval', + 'area:tooling', + ]); + + assertEquals(decision, { + remove: ['status:impl', 'status:plan-eval'], + add: ['status:impl-eval'], + }); + const terminal = [ + 'type:fix', + 'area:tooling', + ...decision.add, + ]; + assertEquals(terminal.filter((label) => label.startsWith('status:')), ['status:impl-eval']); +}); + +Deno.test('generation deduplication remains before trigger creation', async () => { + const workflow = await Deno.readTextFile('.github/workflows/openhands-phase-eval.yml'); + const marker = + 'const marker = `<!-- openhands-phase-eval generation=${generationEvent.id} phase=${phase} head=${pr.head.sha} -->`;'; + const claim = "String(comment.body ?? '').includes(marker)"; + const earlyReturn = 'if (existing) {'; + const create = 'github.rest.issues.createComment({'; + + assertStringIncludes(workflow, marker); + assertStringIncludes(workflow, claim); + assertStringIncludes(workflow, earlyReturn); + assertStringIncludes(workflow, create); + assertEquals(workflow.indexOf(marker) < workflow.indexOf(claim), true); + assertEquals(workflow.indexOf(claim) < workflow.indexOf(earlyReturn), true); + assertEquals(workflow.indexOf(earlyReturn) < workflow.indexOf(create), true); +}); + +Deno.test('status bookkeeping failures are attributed and dispatch remains conditionally eligible', async () => { + const workflow = await Deno.readTextFile('.github/workflows/openhands-phase-eval.yml'); + const transition = workflowStep(workflow, 'Enter IMPL-EVAL status on ready transition'); + const diagnostic = workflowStep( + workflow, + 'Record attributed IMPL-EVAL status-transition failure', + ); + const dispatch = workflowStep(workflow, 'Resolve and dispatch exactly one evaluator'); + + assertStringIncludes(transition, 'id: enter_impl_eval_status'); + assertStringIncludes(transition, 'continue-on-error: true'); + assertStringIncludes(transition, 'core.setOutput('); + assertStringIncludes(transition, "'failure_reason'"); + assertStringIncludes(diagnostic, "steps.enter_impl_eval_status.outcome == 'failure'"); + assertStringIncludes(diagnostic, 'evaluator dispatch attempt continues'); + assertStringIncludes(diagnostic, 'REQUEST_ACTOR: ${{ github.actor }}'); + assertStringIncludes(diagnostic, 'FAILURE_REASON:'); + assertStringIncludes(dispatch, '!cancelled()'); + assertStringIncludes( + dispatch, + "steps.require_chainable_trigger_token.outcome == 'success'", + ); + assertEquals(dispatch.includes('enter_impl_eval_status.outcome'), false); +}); + +Deno.test('inline cleanup transcription matches helper contract literals', async () => { + const workflow = await Deno.readTextFile('.github/workflows/openhands-phase-eval.yml'); + const transition = workflowStep(workflow, 'Enter IMPL-EVAL status on ready transition'); + + assertEquals(workflow.includes('Check out trusted phase-eval scripts'), false); + assertEquals(transition.includes('await import('), false); + assertStringIncludes(transition, 'github.rest.issues.listLabelsOnIssue'); + assertStringIncludes( + transition, + `const IMPL_EVAL_STATUS = '${IMPL_EVAL_STATUS}';`, + ); + assertStringIncludes( + transition, + `const MISSING_LABEL_MESSAGE = '${MISSING_LABEL_MESSAGE}';`, + ); + assertStringIncludes( + transition, + 'error?.response?.data?.message === MISSING_LABEL_MESSAGE', + ); + assertStringIncludes(transition, 'labels: [IMPL_EVAL_STATUS]'); +}); diff --git a/.llm/runs/docs-1377-content--leaf/audit.md b/.llm/runs/docs-1377-content--leaf/audit.md new file mode 100644 index 0000000000..8a597222f6 --- /dev/null +++ b/.llm/runs/docs-1377-content--leaf/audit.md @@ -0,0 +1,48 @@ +# Full-changeset documentation audit — PR #1541 + +## Verdict + +**PASS after in-place documentation corrections.** + +This was the opposite-family `docs_audit` pass over `cd24e1679..f3893df5b` as one changeset, +followed by a complete rerun with the audit fixes applied and focused reruns after the +orchestrator's xref-registry and plugin/core canonical-location findings. The generator's worklog +and PR claims were treated as input only. Evidence below came from this Codex session's commands +against the checked-out public package, CLI, and documentation wiring surfaces. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings | Proceeded | +| --- | --- | --- | --- | --- | --- | +| 1. Links | `rtk proxy deno task docs:links` | Full documentation tree with the complete PR-C changeset and audit fixes applied | **PASS** — exit 0; `docs=102 broken-links=0 broken-anchors=0 orphans=0` | No dangling links, anchors, or reported orphans. | Yes — continued to the remaining gates. | +| 2. Site build | `(cd docs/site && rtk proxy deno task build)` | Full Lume site, including all four new core-reference routes and changed navigation copy | **PASS** — exit 0; source format OK; Lume generated 630 files; rendered-output check passed for 224 HTML files with four documented syntax allowances | The recurring warning that `docs/site/deno.json` is outside the root workspace is non-fatal and pre-existing. | Yes — no build fix required. | +| 3. Internal wording | `git diff --unified=0 cd24e1679 -- README.md packages/sdk/README.md docs/site`; changed-line extraction with `sed`; internal-term scan with `rg` | Changed public lines only, including audit corrections | **PASS after fix** — no internal identifier/process leak remains; the sole final lexical hit is public phrase “test harness” | Found `(US-2)` and “tracked separately” in the changed reference index. The whole-site comparison also exposed legacy `US-2`/`US-8` wording on reference claims touched while reconciling Gate 9. | Yes — removed internal story/process wording in place. | +| 4. Versioned specifiers | Changed-line and whole-changed-file `rg` scans for bare `jsr:@netscript/<package>`; `rg 'releaseSpecifier' docs/site/cli-reference.md`; `(cd /tmp && deno info jsr:@netscript/cli@0.0.5)` | All original public files in the PR-C changeset, plus root install instructions | **PASS** — no newly introduced bare pinnable package specifier; site commands concatenate `releaseSpecifier`; root README explicitly requires substituting `<version>`; registry resolution of current `0.0.5` succeeded | The root README is static Markdown, so `<version>` cannot use the site data variable; its substitution instruction and live release/JSR links make the placeholder explicit. | Yes — no specifier correction required; retained the explicit placeholder. | +| 5. Command/API accuracy | `deno run --no-lock --allow-read --allow-run=deno .llm/tmp/1377-doc-surface-audit.ts`; fresh `deno doc --json` for every declared subpath and `deno doc --filter` checks for the described root semantics; live `deno run -A packages/cli/bin/netscript.ts … --help` matrix; `deploy list --json`; router-source inspection; targeted CLI tests (10 passed); SDK query-factory tests (3 passed); `rtk proxy deno task docs:accuracy` | Every subpath/count and root table on the four exhaustive pages; all six newly documented CLI verbs, seven routed deploy families, deploy registry/router intersection, SDK query dialect, root README install target | **PASS after fixes** — exact package results: sagas 19/19 entrypoints and 41/41 root symbols; streams 4/4 and 51/51; triggers 12/12 and 106/106; workers 17/17 and 32/32; zero missing/extra entrypoints or symbols, count mismatches, or kind mismatches. CLI behavior tests: 10/10; SDK query tests: 3/3; docs accuracy PASS. | Found five documentation defects: nonexistent host-level `plugin ai --help`; `deploy list` advertised operations described as callable operations; desktop package target/default and release flags incomplete; jobs/tasks falsely split into handler-only versus entrypoint-only; root `createWorkersRuntime` falsely described as dependency-injected. Re-derived deploy surface as router intersection: Docker/Compose five verbs; five cloud targets three; `emit` advertised but unrouted. | Yes — fixed all five in place, including exact desktop flags/defaults and the root-versus-`/runtime` workers distinction. Source defect `emit` remains out of scope. | +| 6. Template/generated drift | `rtk proxy deno task check:assets-barrel`; immediate `git diff` of generated barrels and `rtk git status --short` | Repo-owned generated asset barrels affected by documentation/README assets | **PASS** — exit 0 and no generated-barrel diff | The gate did not alter any generated assets. A later targeted CLI invocation added one incidental `deno.lock` integrity row. | Yes — removed the incidental lock entry and retained no cache/lock change. | +| 7. Nav/front matter | Front-matter inspection for the four new pages; publishable-page inventory; `_config.ts`/`_data.ts` nav-source inspection; `deno eval` comparison of `docs/site/reference/` directories with exported `ref:` keys; Gates 1–2 rerun after the registry fix | New core pages, folder-derived navigation, xref registry, front matter, and path convention | **PASS after follow-up fix** — all four pages have `layout: layouts/base.vto` and exact package titles; Lume rendered all four; 35 publishable workspace members have pages; all 36 reference directories have exactly 36 registered `ref:` keys, with zero missing or extra entries | The initial audit verified folder-derived nav but missed the separate hand-maintained `REFERENCE_UNITS` registry. The four new `plugin-*-core` pages lacked xref keys. `_data.ts` no longer contains a mirrored executable list; it explicitly delegates `ref:` keys to `_data/xref.ts`. Two comments in `xref.ts` also carried stale counts (`32` and `28`) and one retained false “generated” provenance. | Yes — registered all four missing units, corrected both counts to 36, and documented that nav is independently folder-derived while the xref registry must stay aligned with `docs/site/reference/`. | +| 8. Prose quality | `deno run --no-lock --allow-read docs/site/_plugins/check-source-format.ts .`; heading-level Deno scan; `deno fmt --check README.md packages/sdk/README.md`; `git diff --check cd24e1679`; changed-line filler/callout/command scans; live command checks from Gate 5 | All changed public prose, headings, tables, callouts, and shown commands | **PASS after fixes** — source formatting OK; 45 changed Markdown files, zero heading jumps; both READMEs formatted; diff check clean | The shown `netscript plugin ai --help` command failed because the host pass-through does not own `--help`. Root README still contained separate hardcoded plugin-file/helper totals. Desktop wording omitted explicit-target behavior. | Yes — removed the invalid shown command, removed all hardcoded root-README file/directory/helper totals, qualified host targeting as the default, and added exact release setup flags. | +| 9. Cross-page contradiction | Whole-tree `rg` for generated-reference claims, deploy verb claims, SDK query shapes, scaffold totals, internal identifiers, and obsolete plugin/core convention statements; focused repo search for a reference writer; comparison to live surfaces from Gate 5; `rtk proxy deno task docs:accuracy` | Every changed claim against the rest of the public documentation tree | **PASS after follow-up fixes** — no obsolete “not a separate top-level reference entry,” “single-page internals convention,” or “not part of the public plugin contract” claim remains; docs accuracy exits 0 | The rewritten index said reference pages are hand-written, while many existing public pages still called them generated. A later polish pass found a second contradiction the initial audit missed: the four deployable-plugin pages duplicated their core packages' exhaustive surfaces, and the sagas/streams pages denied those separate core references should exist. Historical `_plan/` files retain their original planning language and are not public documentation. | Yes — selected one publishable package/one canonical reference page, retained focused cross-package examples, replaced four duplicated core tables with links to the canonical `plugin-*-core` pages, recorded the rule and rationale in the index, and retained the saga `spawn(...): never` contract required by the accuracy gate. The `Group F`/`T1` descriptions remain unchanged because they match authoritative published JSDoc and are tracked in #1554. | + +## Fix disposition + +Fixed in place: + +- removed internal `(US-2)` / process wording from the changed reference index; +- corrected the CLI reference for the `plugin ai` pass-through, deploy registry versus routed verbs, + desktop package/release flags and defaults, and five-versus-three routed deploy surface; +- corrected workers builder and runtime-composition claims against `deno doc` and the live types; +- removed all hardcoded file/directory/helper totals from the root README; +- reconciled public “generated reference” wording with the verified hand-maintained Markdown model; +- changed the public Reference-lane subtitle to “Every published symbol”; and +- registered the four new core-reference pages in the hand-maintained `ref:` xref surface, verified + exact 36-directory/36-key parity, and corrected its stale count/provenance comments; +- made each separately published `plugin-*-core` page canonical for its own exports, replacing the + four deployable-plugin pages' duplicated exhaustive tables with links and recording the convention + in the reference index; and +- removed incidental `deno.lock` churn. + +Flagged but not fixed: none. The unrouted adapter `emit` operation remains the separately owned +source defect and was not changed. Historical files under `docs/site/_plan/` are retained as records, +not treated as current public claims. The published-JSDoc terms `Group F` and `T1` remain in the +triggers-core page pending the separately filed source correction #1554. diff --git a/.llm/runs/docs-1377-content--leaf/context-pack.md b/.llm/runs/docs-1377-content--leaf/context-pack.md new file mode 100644 index 0000000000..298390798c --- /dev/null +++ b/.llm/runs/docs-1377-content--leaf/context-pack.md @@ -0,0 +1,85 @@ +# Context pack — docs-1377-content--leaf + +Resumable summary. Read with `supervisor.md`, `worklog.md`, `drift.md`. + +## What this run is + +PR-C of #1377 — the **content** half. Documentation authoring only. The gate half (a +`publish-readiness.ts` reference-page check that currently runs over zero packages, a +command-coverage gate, and the reference path-convention decision) is PR-D and is **out of scope**. + +`Refs #1377`, never `Closes` — the issue's gate acceptance rows are not this PR's to complete. + +## Coordinates + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-1377-content` | +| Branch | `docs/1377-reference-ia-readme-truth` | +| Baseline | `cd24e1679` | +| PR | **#1541** (draft) | +| Run dir | `.llm/runs/docs-1377-content--leaf/` | + +Push with an explicit refspec: `git push origin HEAD:refs/heads/docs/1377-reference-ia-readme-truth`. + +## Slices + +| Slice | Commit | State | +| --- | --- | --- | +| S1 four `-core` reference pages + reference index | `e22a2f952` | landed, gated, commented | +| S2 `packages/sdk/README.md` query dialect | `68cf6f2e5` | landed, gated, commented | +| S3 root `README.md` scaffold count + specifier | `617b74884` | landed, gated, commented | +| S4 CLI reference — six verbs + two wrong claims | `f3893df5b` | landed, gated, commented | +| Audit corrections | `077a75716`, `13b6d118a`, `ef2b05670` | landed, gated, commented | +| Plugin/core canonical-location follow-up | this commit | four deployable-plugin pages point to canonical `plugin-*-core` pages; gated | + +## Measurements that must not be re-derived + +- **Alias row of #1377 is already satisfied.** `@contracts`, `@/lib/`, `api-clients` are at zero + under `docs/site/`, and `check-accuracy-and-discoverability.ts:57-58` actively forbids them. Tick + on measurement; do not "fix". +- **Reference pages: 35 effective-publish members, 0 missing** after S1 (was 4 missing). +- **Path convention:** 31 name-exact, 4 exceptions — the *deployable* plugins drop `plugin-`. +- **No generator writes into `docs/site/reference/`.** Pages are hand-written. +- **Reference xref registry:** navigation is folder-derived, but `_data/xref.ts` separately registers + all 36 reference directories. The audit follow-up restored exact 36-directory/36-key parity and + added the four new `plugin-*-core` keys. +- **Canonical reference location:** each separately published `plugin-*-core` page owns its + exhaustive exported-API documentation. The corresponding deployable-plugin page owns manifest and + integration coverage, links to the core page, and may retain focused cross-package examples. This + removes the prior duplicated tables and obsolete single-page-internals claims. +- **Deploy target operations, measured from source:** + + | Target(s) | Class | Operations | + | --- | --- | --- | + | `compose`, `docker` | `AspireComposeDeployTarget:64` | plan · emit · up · down · status · logs (**6**) | + | `kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks`, `cloud-run` | `AspireCloudDeployTarget:125` | plan · emit · up · down (**4**) | + | `deno-deploy` | `deno-deploy-target.ts:57` | plan · up · down · status · logs (5) | + | `windows-service`, `linux-service` | `service-deploy-target.ts:21` | plan · emit · up · down · status · logs (6), +`rollback` with an `ActivationPort`, +`secrets` with a `SecretsStorePort` | + + Verb subcommands are generated from the target's `operations` + (`target-deploy-command.ts:58-66`), so both wrong claims are decidable from these arrays. +- **Six verbs confirmed undocumented** in `docs/site/`: `agent drift`, `deploy desktop`, + `deploy package-cli`, `deploy list`, `config list` → zero occurrences; `plugin ai` → one, at + `reference/plugin-ai/index.md:311`, absent from both command references. + +## Hard boundaries + +- **Do not open** the nine `docs/site` Tier-1 files PR-B is editing concurrently: `quickstart.vto`, + `index.vto`, `services-sdk/sdk.md`, `services-sdk/how-to/add-a-service.md`, and + `web-layer/{query,examples,interactive,form,query-bridge}.md`. +- No gate code, no checkers, no negative tests — PR-D. +- No `packages/**` or `plugins/**` change except `README.md`. +- Do not mark ready for review. Do not merge. A separate opposite-family Codex audit runs after this, + then a Fable prose polish. + +## Gate commands for this run + +```text +deno task docs:links +deno task docs:accuracy +deno task quality:gate +deno task publish:dry-run +cd docs/site && deno run --no-lock --allow-read _plugins/check-source-format.ts . +deno fmt <README paths> # docs/site/**/*.md is excluded — see drift DR-4 +``` diff --git a/.llm/runs/docs-1377-content--leaf/drift.md b/.llm/runs/docs-1377-content--leaf/drift.md new file mode 100644 index 0000000000..2fc5e37ac5 --- /dev/null +++ b/.llm/runs/docs-1377-content--leaf/drift.md @@ -0,0 +1,124 @@ +# Drift — docs-1377-content--leaf (PR-C of #1377) + +Append-only. Everything found and **not** fixed in this slice, with why. + +## DR-1 — `reference/index.md` claimed pages are generated; nothing generates them + +**Severity:** significant. **Status:** fixed in this PR (in scope per the brief). + +`docs/site/reference/index.md:6-7` asserted the reference pages "are generated from the source code +with `deno doc`, so they always describe the published surface". No script writes into +`docs/site/reference/`; every tooling reference to that path reads or checks. Evidence in +`worklog.md` § D-1. Corrected. + +## DR-2 — The same false generation claim survives inside two existing pages + +**Severity:** minor. **Status:** NOT fixed — out of this slice's footprint. + +- `docs/site/reference/plugin-ai-core/index.md:10-11` — "This page is generated from the package's + public surface with `deno doc` (US-2)." +- `docs/site/reference/plugin-auth-core/index.md:9-10` — "This page is generated from the package's + public surface with `deno doc`." + +Both are the per-page form of DR-1 and are equally untrue. The four pages added here deliberately say +"written against the package's published exports and its `deno doc` surface" instead. Left alone +because editing pages this slice does not otherwise touch widens the diff into files a sibling slice +or PR-D may reach; a one-line correction each is enough whenever someone owns them. + +## DR-3 — `docs:accuracy` hardcodes a short reference path, and #1377 does not name it + +**Severity:** significant for PR-D. **Status:** NOT fixed — PR-D owns the decision. + +`.llm/tools/docs/check-accuracy-and-discoverability.ts:126` reads +`'docs/site/reference/sagas/index.md'` and `:29` reads `'docs/site/reference/sdk/index.md'`. #1377 +names only `.llm/tools/release/publish-readiness.ts` as the gate hardcoding a path. If the +path-convention decision moves the IA to the gate's name-exact rule, this file breaks too, along with +`publish-readiness.ts:302` and every inbound link. Carried forward from the orchestrator's research +(C-3) and re-confirmed by reading the file in this worktree. + +## DR-4 — `deno fmt` covers READMEs by explicit path but never `docs/site/**/*.md` + +**Severity:** minor, process-relevant. **Status:** NOT a defect to fix here. + +**Corrected after S2/S3.** The first recording of this entry (during S1) over-generalized from one +observation; the measured behaviour is: + +- `docs/site/**/*.md` — `deno fmt <path>` exits **1** with `No target files found`, because + `docs/site/deno.json` `fmt.exclude` lists `**/*.md` and an **exclude beats an explicit path + argument**. `docs/site/_plugins/check-source-format.ts` is the real formatting authority here. +- `README.md` files — `deno fmt <path>` **does** format them (exit 0). The root `fmt.include` is + `packages/**` / `plugins/**` TypeScript only, so a bare `deno fmt` never *discovers* a README, but + an explicit path argument is honoured because nothing excludes it. + +Consequence: READMEs are formattable but not covered by any discovery-based gate, so their formatting +is enforced by nobody. Both READMEs touched by this PR were formatted explicitly and re-verified with +`--check`. Worth deciding deliberately rather than discovering again — flagged for the orchestrator, +not changed here (fmt config is not this slice's scope). + +## DR-6 — `docs:readme:check` is red on `main` + +**Severity:** minor. **Status:** NOT fixed — pre-existing and out of scope (#767). + +`deno task docs:readme:check` exits **1** at the baseline commit and still does: + +```text +A2 README standard FAIL - 1/36 non-conformant: + packages/bench/README.md + - [install-section] missing '## Install' section +``` + +`packages/bench` is `"publish": false`, so this is not a publish blocker. README-standard +conformance is #767's territory and is explicitly out of this slice's scope; recorded rather than +swept in so the audit does not read it as regression from this PR. + +## DR-7 — `emit` is implemented on three adapters but unreachable from the CLI + +**Severity:** significant. **Status:** NOT fixed — source, not docs. Found while verifying #1377's +deploy claims. + +`ROUTED_OPERATIONS` (`packages/cli/src/public/features/deploy/target/target-deploy-command.ts:15-23`) +is `['plan', 'up', 'down', 'status', 'logs', 'rollback', 'secrets']` — **no `emit`**. The router +generates a subcommand only for an operation in that list that the adapter also advertises (`:59`). + +But `emit` **is** advertised and implemented: + +- `AspireComposeDeployTarget` — advertises it (`:64-71`), implements it (`:96`); +- `AspireCloudDeployTarget` — advertises it (`:125`), implements it (`:162`); +- `ServiceDeployTarget` — advertises it (`SERVICE_DEPLOY_OPERATIONS`, `:21-29`). + +It even has a router description (`OPERATION_DESCRIPTIONS.emit = 'Emit deployment artifacts'`, `:28`). +The only other `'emit'` under `packages/cli/src/public/` is in a test. So every adapter's `emit` +handler is dead code from the CLI's point of view, and `deploy list` reports an operation the user +cannot invoke — the same class of honesty defect that `service-deploy-target.ts:14-20` explicitly +guards against for `rollback`/`secrets` ("LD-4: omit rather than silent no-op"), applied in reverse. + +Either `emit` belongs in `ROUTED_OPERATIONS` or it should stop being advertised. Deciding that is a +source change and out of this slice's scope. The documentation this PR writes describes the +**reachable** surface and does not mention `emit` as a verb. + +## DR-8 — The research's C-2 correction was itself inaccurate + +**Severity:** significant, process. **Status:** resolved by measurement; recorded so the audit can +check the reasoning rather than the conclusion. + +The orchestrator's research told this slice that `reference/cli/commands.md:209` ("the same +three-verb lifecycle — `plan`, `up`, `down`") was "contradicted by the same array" and must be fixed, +and that the docker/compose surface "is six" against #1377's "five". + +Measured (see `worklog.md` § D-5): the CLI verb surface is the **intersection** of the adapter's +advertised `operations` with the router's `ROUTED_OPERATIONS`, and `emit` is absent from the latter. +So the cloud targets expose exactly three verbs — `:209` was **correct**, and applying the instructed +fix would have replaced a true statement with a false one. Docker/compose expose five, matching +#1377's original wording; the research's "six" counts advertised operations, not commands. + +Only the `cli-reference.md:246-249` "not wired — they only print help" claim was actually false, and +that one is fixed. `:209`'s three-verb statement is preserved verbatim inside a rewritten section. + +## DR-5 — Maintainer CLI reports `version('1.0.0')` + +**Severity:** significant. **Status:** NOT fixed — source, not docs. + +`.../maintainer-command-tree.ts:32` sets `.version('1.0.0')` while the public tree uses +`CLI_PACKAGE_VERSION`. Carried in from the orchestrator's research (item 9). The brief explicitly +routes this here rather than to a fix: it is framework source, and this run authors documentation +only. Needs a WSL Codex slice. diff --git a/.llm/runs/docs-1377-content--leaf/supervisor.md b/.llm/runs/docs-1377-content--leaf/supervisor.md new file mode 100644 index 0000000000..78ca9c4bcf --- /dev/null +++ b/.llm/runs/docs-1377-content--leaf/supervisor.md @@ -0,0 +1,49 @@ +# Supervisor — docs-1377-content--leaf + +Leaf documentation-authoring run for **PR-C of #1377** (content half) in the NetScript 0.0.6 +documentation lane. Authored under the CLAUDE.md **documentation-authoring exception (2026-06-18)**: +Markdown/prose only, no framework source, validated afterwards by a separate opposite-family session. + +## Identity + +| Field | Value | +| --- | --- | +| Run id | `docs-1377-content--leaf` | +| Role | Leaf implementation agent (generator only — does not self-certify) | +| Model / lane | Claude Opus 5, Tier-B documentation authoring | +| Host | WSL2 Linux, `codex@` | +| Worktree | `/home/codex/repos/ns006-1377-content` | +| Branch | `docs/1377-reference-ia-readme-truth` (no upstream at dispatch, by design) | +| Baseline | `cd24e1679f8732837883aaa84ab16aa61d733714` (= `origin/main` at dispatch) | +| Control worktree (read-only to this run) | `/home/codex/repos/netscript-006-docs` | +| Brief | `.llm/runs/release-0.0.6-docs--orchestration/slices/1377-content/implement.md` (control worktree) | +| Research input | `.llm/runs/release-0.0.6-docs--orchestration/slices/1377-content/research.md` (control worktree) | +| Issue | #1377 — **`Refs`, not `Closes`**; the gate half is PR-D | + +## Overlay and archetype + +- Scope overlay: `SCOPE-docs.md`. No archetype selected — this run authors Markdown only and changes + no `packages/**` or `plugins/**` source. The only files touched under `packages/**` are + `README.md` files, which the brief explicitly permits. +- PLAN-EVAL: **N/A**. The plan is the orchestrator's brief; the decision-heavy parts (path + convention, gate shape) are explicitly withheld from this slice and belong to PR-D. Recorded here + before implementation per the harness checklist. + +## Lanes + +| Lane | Session | Role | +| --- | --- | --- | +| Orchestrator | 0.0.6 docs orchestration supervisor | brief, merge authority, pre-merge `packages/**` audit | +| Generator (this run) | Claude Opus 5 | authoring, gates, draft PR, per-slice comments | +| Audit | separate opposite-family Codex session | full-changeset audit after this run | +| Prose polish | Fable session | after the audit | + +No lane self-certifies. This run stops at **draft** — it does not mark ready for review and does not +merge. + +## Sibling-slice conflict boundary + +PR-B is concurrently editing nine `docs/site` Tier-1 files +(`quickstart.vto`, `index.vto`, `services-sdk/sdk.md`, `services-sdk/how-to/add-a-service.md`, +`web-layer/{query,examples,interactive,form,query-bridge}.md`). This run does not open any of them. +Anything found there goes to `drift.md`. diff --git a/.llm/runs/docs-1377-content--leaf/worklog.md b/.llm/runs/docs-1377-content--leaf/worklog.md new file mode 100644 index 0000000000..6313533f9c --- /dev/null +++ b/.llm/runs/docs-1377-content--leaf/worklog.md @@ -0,0 +1,306 @@ +# Worklog — docs-1377-content--leaf (PR-C of #1377) + +## Design + +Four decisions were made before authoring; each is recorded with the measurement that settled it. + +### D-1 — The reference pages are hand-written; the index's generator claim was false + +The brief required resolving this before authoring. Searched the repository for anything that +*writes* into `docs/site/reference/`: + +```text +grep -rn "docs/site/reference" --include=*.ts --include=*.js --include=*.json --include=*.mjs . +``` + +Every hit is a **reader**, not a writer: + +- `.llm/tools/docs/check-exports-drift.ts` — checks 8 hardcoded pages against declared entrypoints. +- `.llm/tools/docs/check-accuracy-and-discoverability.ts:29,:126` — reads `reference/sdk/index.md` + and `reference/sagas/index.md`. +- `.llm/tools/release/publish-readiness.ts:302` — asserts a page path exists. +- `packages/mcp/tests/registry_test.ts:86,:121` — reads `reference/mcp/index.md`. +- `.llm/runs/beta5-impl--supervisor/slices/479-ai-docs/workflow.js:136-138` — a **past authoring + run** that hand-wrote `reference/ai/`, `reference/plugin-ai/`, and `reference/plugin-ai-core/`. + This is positive evidence of the hand-written route, not a generator. + +The only `deno doc` consumers in tooling are `.llm/tools/run-deno-doc-lint.ts` (a `deno doc --lint` +runner) and `.llm/tools/release/surface-diff.ts` (`deno doc --json` surface snapshots). Neither emits +Markdown. + +**Verdict: no generator exists.** `docs/site/reference/index.md:6-7` claimed the pages "are generated +from the source code with `deno doc`, so they always describe the published surface". Both halves +were false — nothing generates them, and therefore nothing guarantees they stay current. Correcting +that sentence was in scope per the brief; adding four hand-written pages beneath an unretracted +generation claim was not an option. + +### D-2 — Page depth and structure modelled on the existing `-core` pages + +`reference/plugin-auth-core/index.md` (88 lines, grouped symbol tables) and +`reference/plugin-ai-core/index.md` (266 lines, entrypoint table plus per-surface tables) are the two +existing pages for this archetype. The four new pages use both patterns: an **Entrypoints** table +with a measured export count per subpath, then grouped root-surface symbol tables, then the +behavioural notes that a symbol table alone cannot carry (ack-then-process ordering, replay-commit +semantics, typestate gating, synchronous-handler discipline). + +Every symbol row is derived from `deno doc --json` output for the package's **declared** exports — +not from source reads. Root-surface tables are exhaustive against that output; each page's grouped +tables sum to the count `deno doc` reports: + +| Package | Root exports | Rows on page | +| --- | --- | --- | +| `@netscript/plugin-sagas-core` | 41 | 41 | +| `@netscript/plugin-streams-core` | 51 | 51 | +| `@netscript/plugin-triggers-core` | 106 | 106 | +| `@netscript/plugin-workers-core` | 32 | 32 | + +Subpath entrypoints are described by their module doc plus their measured export names, so no page +names a symbol the package does not export. + +### D-3 — Path convention is described, not legislated + +Measured over the whole workspace: 35 effective-publish members, 31 pages at the name-exact segment +(`@netscript/<x>` → `/reference/<x>/`), 4 exceptions where the **deployable** plugins drop the +`plugin-` prefix (`@netscript/plugin-sagas` → `/reference/sagas/`, and the same for streams, +triggers, workers). Their `-core` counterparts are name-exact, which is what makes the IA internally +inconsistent rather than uniformly divergent. + +The index now records exactly that, and states explicitly that it is a description of today's site, +not a rule to follow when the two forms disagree — reconciling them is PR-D's decision. + +### D-4 — `deno fmt` governs the READMEs but not `docs/site/**/*.md` + +The brief's formatting gate is `deno fmt <touched files>`. Measured, it splits in two: + +- **`docs/site/**/*.md`** — exits **1** with `error: No target files found`, because + `docs/site/deno.json` `fmt.exclude` lists `**/*.md`, `**/*.mdx`, `**/*.vto`, and an exclude beats + an explicit path argument. The real formatting gate here is + `docs/site/_plugins/check-source-format.ts` (`deno task check:source-format` in that directory), + which this run uses in its place. +- **`README.md` files** — exit **0** and are reformatted. The root `fmt.include` is + `packages/**/*.{ts,tsx}` and `plugins/**/*.{ts,tsx}`, so a bare `deno fmt` never discovers a + README, but an explicit path is honoured because nothing excludes it. + +The S1 record of this decision claimed `deno fmt` covered neither, generalizing from the `docs/site` +observation before a README had been touched. Corrected here and in `drift.md` DR-4 during S2. The +rewrap hazard the brief warns about is therefore **real for the READMEs**: both were reformatted by +`deno fmt`, and the edited passages were re-read and grepped afterwards to confirm the edits survived +(they did). + +## Gate baselines (before any edit, at `cd24e1679`) + +| Gate | Exit | Result | +| --- | --- | --- | +| `deno task docs:links` | 0 | `docs=102 broken-links=0 broken-anchors=0 orphans=0` | +| `deno task docs:accuracy` | 0 | `PASS (… 192 published source pages …)` | + +Reference-page count before: **35 effective-publish members, 4 missing pages** +(`plugin-{sagas,streams,triggers,workers}-core`). + +## Slice 1 — four reference pages + reference index + +Files: + +- `docs/site/reference/plugin-sagas-core/index.md` (new) +- `docs/site/reference/plugin-streams-core/index.md` (new) +- `docs/site/reference/plugin-triggers-core/index.md` (new) +- `docs/site/reference/plugin-workers-core/index.md` (new) +- `docs/site/reference/index.md` (generator claim corrected; path convention recorded) + +| Gate | Command | Exit | Result | +| --- | --- | --- | --- | +| Internal doc links | `deno task docs:links` | 0 | `docs=102 broken-links=0 broken-anchors=0 orphans=0` | +| Docs accuracy | `deno task docs:accuracy` | 0 | `PASS (… 196 published source pages …)` — up 4, one per new page | +| Docs source format | `docs/site/_plugins/check-source-format.ts .` | 0 | `Docs source format: OK` | +| `deno fmt` (touched files) | `deno fmt <5 files>` | 1 | `No target files found` — see D-4; excluded for `docs/site` Markdown | +| Reference-page existence | own count | — | before 31/35, after **35/35**; 0 missing | + +## Slice 2 — `packages/sdk/README.md` (the JSR landing page) + +`jsr-package-settings.json:6` sets `readmeSource: "readme"`, so this file is the consumer-facing JSR +page. + +Two measurements that shaped the edit: + +1. **`createQueryFactory` is a real export**, not a wrong symbol name + (`packages/sdk/src/query/query-factory.ts:41`). It is the single-resource form; + `createQueryFactories` (`:192`) is the map form. The README's defect was emphasis, not naming. +2. **`createServiceQueryUtils` cannot simply be demoted out of the page.** `defineServices` builds + its returned `queryUtils.*` from it (`packages/sdk/src/presets/define-services.ts:9,:120`), so the + README's own Quick example already exercises both dialects. Removing the symbol would have made + the page describe an API the package does not have. + +Result: the golden path leads, and a new "Two query dialects" section frames `createServiceQueryUtils` +as the narrower helper — `queryOptions({ input })` versus `queryOptions(input)`, no server KV tier — +matching `docs/site/reference/sdk/index.md:111` rather than inventing a second framing. Call shapes +verified against source (`query-factory.ts:140-146`), not copied from the reference page. + +The `docs:accuracy` single-page exception for `createServiceQueryUtils` +(`check-accuracy-and-discoverability.ts:63-74`) walks `docs/site` only, so a mention in +`packages/sdk/README.md` does not contend with it — confirmed by the gate still passing. + +## Slice 3 — root `README.md` + +- Removed "The scaffold reports **183 files, 44 directories**", which contradicted + `quickstart.vto:51` ("treat the printed result—not a static number in this guide—as the + authority"). Replaced with a statement that the scaffold prints its own totals and why they vary. + `README.md:78-80` already described `--dry-run` as reporting the counts it would create, so the two + statements now agree. +- The install line's `jsr:@netscript/cli@<version>` is left as a placeholder with an explicit + substitution instruction and a pointer to the latest release / JSR page, plus the note that bare + `jsr:@netscript/*` specifiers do not resolve on the pre-release line. Pinning it to `0.0.5` would + have reproduced the hardcoded-count defect one release later; the site can derive its specifier + (`_data.ts:30-33`) and a static README cannot. +- **Not changed:** `:148` "29 packages and 6 first-party plugins". Re-measured — 35 effective-publish + members, 6 of them under `plugins/`. The claim is correct. + +### Gates for slices 2 and 3 + +| Gate | Command | Exit | Result | +| --- | --- | --- | --- | +| Code quality | `deno task quality:gate` | 0 | pass; doctrine readiness `FAIL=0` for every package (WARN/INFO pre-existing) | +| Publish dry-run | `deno task publish:dry-run` | 0 | `Success Dry run complete` | +| Markdown format | `deno fmt README.md packages/sdk/README.md` | 0 | reformatted, `--check` clean, edits re-verified after rewrap | +| Internal doc links | `deno task docs:links` | 0 | `broken-links=0 broken-anchors=0 orphans=0` | +| Docs accuracy | `deno task docs:accuracy` | 0 | `PASS` | + +**#1417 side effect did not occur.** `git status --short` immediately after `publish:dry-run` was +empty — no catalog-backed manifest rewritten, nothing to revert, `deno.lock` untouched. + +## Slice 4 — CLI reference: six verbs and the deploy verb surface + +### D-5 — The deploy verb counts, measured + +The brief said to verify the operations array myself and not to copy #1377's wording. Doing so +produced a result that contradicts **the orchestrator's own research correction**, so it is set out +in full. + +There are **two** arrays, and only their intersection reaches the CLI: + +1. Each adapter advertises `operations`: + + | Target(s) | Source | Advertised | + | --- | --- | --- | + | `compose`, `docker` | `aspire-compose-deploy-target.ts:64-71` | plan · emit · up · down · status · logs | + | `kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks`, `cloud-run` | `aspire-cloud-deploy-target.ts:125` | plan · emit · up · down | + | `deno-deploy` | `deno-deploy-target.ts:57-63` | plan · up · down · status · logs | + | `windows-service`, `linux-service` | `service-deploy-target.ts:21-29` | plan · emit · up · down · status · logs (+`rollback` with an `ActivationPort`, +`secrets` with a `SecretsStorePort`) | + +2. The router walks a **fixed candidate list** and skips anything the adapter does not advertise + (`target-deploy-command.ts:15-23`, gate at `:59`): + + ```ts + const ROUTED_OPERATIONS: readonly DeployOperation[] = [ + 'plan', 'up', 'down', 'status', 'logs', 'rollback', 'secrets', + ]; + ``` + + **`emit` is not in that list.** It has an entry in `OPERATION_DESCRIPTIONS` (`:28`) and is + implemented on the compose, cloud, and service adapters, but no public CLI path exposes it — the + only other `'emit'` occurrence under `packages/cli/src/public/` is in a test. + +So the CLI verb surface is the intersection: + +| Group | CLI verbs | Count | +| --- | --- | --- | +| `deploy docker`, `deploy compose` | plan · up · down · status · logs | **5** | +| `deploy kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks`, `cloud-run` | plan · up · down | **3** | + +Consequences for the two claims this slice was sent to fix: + +- **`cli-reference.md:246-249` — wrong, corrected.** "`deploy docker` and `deploy compose` … are not + wired — they only print help" is false: each exposes five working verbs routed to + `AspireComposeDeployTarget`, whose `plan`/`emit`/`up`/`down`/`status`/`logs` methods are all + implemented (`:91`–`:152`). The bare group does print help, but so does every command group in the + CLI, including `deploy` itself — that is the default action, not evidence of a stub. +- **`reference/cli/commands.md:209` — CORRECT; deliberately not "fixed".** "the same three-verb + lifecycle — `plan`, `up`, `down`" describes the five cloud targets, and three is exactly what they + expose, because their advertised `emit` is not routed. The orchestrator's research (C-2) called + this claim contradicted and instructed fixing it; measuring shows the claim was right and the + correction would have introduced the error. The section was rewritten for coverage — it now names + `docker`/`compose` and their two extra verbs, and explains *why* the verb lists differ — but the + three-verb statement for the cloud targets was preserved because it is true. +- **#1377's "five-verb surface" for docker/compose — also correct**, at the CLI level. The research's + "it is six" counts the adapter's advertised operations, which is a different (and unreachable-by-one) + number. Both were describing real arrays; neither said which one the user types. + +### Content added + +`docs/site/reference/cli/commands.md` (the exhaustive page) gains all six previously undocumented +verbs, `docs/site/cli-reference.md` (the curated page) gains the five that fit its narrative: + +| Verb | Curated page | Exhaustive page | +| --- | --- | --- | +| `agent drift` (+ `drift record`) | yes | yes, with the 15-minute receipt gate | +| `plugin ai` | yes | yes, framed as a pass-through | +| `deploy list` | yes | yes, with the ten default targets | +| `deploy desktop` (+ `package`, `release`) | yes | yes | +| `deploy package-cli` | yes | yes, with all four flags and defaults | +| `config list` | — (no config section on the curated page) | yes, incl. the `(not read by the generator)` marker | + +Two behaviours were read from source rather than paraphrased from a description string, because the +description alone would have been misleading: + +- `agent drift record` refuses unless a diagnostic receipt for `--resource` exists, exited `0`, and + is under `DIAGNOSTIC_RECEIPT_TTL_MS` = **15 minutes** (`record-drift-flow.ts:5,:31-42`). The same + gate backs the MCP `record_drift` tool. +- `plugin ai` uses `.useRawArgs()`, strips only `--project-root`, and forwards everything else to the + installed `@netscript/plugin-ai` CLI in a child `deno run` (`ai-plugin-command.ts:49-68,:92-113`). + Its verbs therefore belong to that plugin's release, not to this reference — documenting a verb + list here would go stale on the plugin's schedule, not NetScript's. + +### Gates for slice 4 + +| Gate | Command | Exit | Result | +| --- | --- | --- | --- | +| Internal doc links | `deno task docs:links` | 0 | `broken-links=0 broken-anchors=0 orphans=0` | +| Docs accuracy | `deno task docs:accuracy` | 0 | `PASS` | +| Docs source format | `docs/site/_plugins/check-source-format.ts .` | 0 | `Docs source format: OK` | +| Six-verb coverage | own grep over `docs/site/` | — | all six present in `reference/cli/commands.md`; five also on `cli-reference.md` | +| Stale-claim removal | own grep | — | `not wired` / `only print help` gone from `docs/site/` | + +## Opposite-family audit follow-up — xref wiring + +The orchestrator found that Gate 7 had verified folder-derived navigation but not the independent +hand-maintained `REFERENCE_UNITS` registry in `docs/site/_data/xref.ts`. The four new core-reference +directories were absent, so their `ref:` shortcuts could not resolve. The same file also contained +stale `32` and `28 generated` count/provenance comments. + +`_data.ts` has no mirrored executable reference-unit list: its source comment explicitly says the +nav is folder-derived and the `ref:` keys live in `_data/xref.ts`. The fix therefore keeps behavior +unchanged, adds the four missing registry entries, and documents the real two-surface relationship. + +| Gate | Command | Exit | Result | +| --- | --- | --- | --- | +| Xref parity | `deno eval` import of `xref` plus directory/key set comparison | 0 | 36 directories, 36 registered `ref:` units, zero missing/extra; all four new URLs exact | +| Internal doc links | `deno task docs:links` | 0 | `docs=102 broken-links=0 broken-anchors=0 orphans=0` | +| Site build | `cd docs/site && deno task build` | 0 | source format OK; 630 files generated; rendered output OK for 224 HTML files | + +## Opposite-family audit follow-up — canonical plugin/core references + +The polish lane found a Gate 9 contradiction the first audit did not catch. The four new +`plugin-*-core` pages were intended to satisfy the one-page-per-publishable-package acceptance row, +but the deployable sagas and streams pages still denied that such top-level references should exist. +All four deployable-plugin pages also repeated their core package's exhaustive API documentation. + +Decision: **the separately published core package's page is canonical for its exported surface**. +This is the arrangement most consistent with #1377: one publishable workspace member has one +canonical reference page, so an API claim has one place to remain accurate. The four deployable +plugin pages retain their manifest/integration coverage and focused examples, but their exhaustive +core sections are now short pointers to `plugin-{sagas,streams,triggers,workers}-core`. The reference +index records this one-package/one-canonical-page rule and its rationale. + +`docs:accuracy` initially exposed one legacy checker dependency on the sagas page's exact +`options?: SpawnOptions): never` marker. The page therefore retains a focused explanation of the +unsupported `spawn(...)` contract while leaving the exhaustive core table canonical on +`plugin-sagas-core`. No gate code was changed. + +The `Group F` and `T1` descriptions in `plugin-triggers-core` were deliberately left unchanged: +they match `deno doc` and the published source JSDoc. Their source-level cleanup is tracked in #1554. + +| Gate | Command | Exit | Result | +| --- | --- | --- | --- | +| Internal doc links | `rtk proxy deno task docs:links` | 0 | `docs=102 broken-links=0 broken-anchors=0 orphans=0` | +| Docs accuracy | `rtk proxy deno task docs:accuracy` | 0 | PASS; 196 published source pages, spawn contract, preferred paths, and CLI mutation families checked | +| Obsolete convention scan | `rtk grep -n -E 'not a separate top-level reference entry\|single-page internals convention\|not part of the public plugin contract' docs/site/reference/*/index.md` | 1 | Expected no-match result; all three contradictory statements are absent | +| Core-page preservation | `rtk git diff -- docs/site/reference/plugin-{sagas,streams,triggers,workers}-core/index.md` | 0 | No diff; exhaustive canonical pages and `Group F`/`T1` descriptions untouched | diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1459.md b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1459.md new file mode 100644 index 0000000000..f9b94dacc5 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/implement-1459.md @@ -0,0 +1,133 @@ +use harness + +# Slice brief — #1459 the deferred refresh coordinator is never hydrated + +**Codex · GPT-5.6 Sol · high** (`complex_implementation`). The plan **passed PLAN-EVAL** (run +`31593309658`, verdict PASS in the body). Implement it; do not re-decide it. + +| Field | Value | +| --- | --- | +| Issue | #1459 (`priority:p1`) | +| PR | **#1558** — already open as a draft at plan phase; commit onto its branch | +| Worktree | `/home/codex/repos/ns006-1459` | +| Branch | `fix/1459-defer-island-hydration` | +| Base | `origin/main@59e435c5d` | + +**Read first, in order:** + +1. `slices/plan-1459.md` — read the **whole** file. It has three sections: the original plan, then + **`# Plan v2`** (the revision after `FAIL_PLAN`), then **`## PLAN-EVAL v2 result`** with + amendments **B1–B4**. **v2 and B1–B4 are binding**; where the original plan disagrees with v2, v2 + wins. +2. `slices/research-1459.md`. + +If they are not on your branch: `git show chore/release-0.0.6-runtime-reopen:<path>`. + +## SKILL + +- `deno-fresh` — Fresh 2.x islands, partials, client navigation, and the Vite plugin. +- `netscript-doctrine` — `packages/fresh` is framework code. +- `netscript-cli` — the scaffold template surface, if D1 touches it. +- `netscript-tools`, `netscript-pr`, `netscript-harness`. + +## The defect — two inseparable halves + +1. **`DeferIsland.tsx` is not an island at all.** No island registration exists anywhere. It is a + *named* export (`DeferIsland.tsx:111`); the scaffold calls `fresh()` with **no options** + (`packages/cli/src/kernel/assets/app/vite.config.ts.template:41`); `createNetScriptVitePlugin` has + **zero** island logic. It is reached only through a server-render path, so it never enters the + client bundle. +2. **`f-client-nav` is false in exactly the failing case.** `DeferIsland.tsx:222` sets + `f-client-nav={!(isPartialRequest && !hasCachedData)}` — that condition **is** `partial-miss`. + PLAN-EVAL verified against Fresh core that `client.ts:256` gates form submission on + `checkClientNavEnabled`, so with it false the `requestSubmit()` produces a **full document + navigation**, not a `/partials/**` request. It also confirmed the client refresh mechanism is + form/anchor interception only — **no background poll, no auto-refresh timer**, so nothing else + rescues a missed region. + +Fixing (1) without (2) leaves the reported symptom unfixed. + +## LOCKED decisions + +- **D1 — register the island via `islandSpecifiers`.** `@fresh/plugin-vite` exposes + `islandSpecifiers?: string[]`, documented as *"Treat these specifiers as island files. This is used + to declare islands from remote packages."* + **B1 (binding):** the scaffold pins **`jsr:@fresh/plugin-vite@^1.1.2`** — verify against **1.1.2** + (`utils.ts:59-63`; mechanism at `src/mod.ts:234-237`, `fConfig.islandSpecifiers.set(spec, name)`), + **not** 1.0.8. Copy-mode and a consumer-owned shim are **rejected**. +- **B2 (binding) — the specifier must be created, not just referenced.** `packages/fresh/deno.json` + has **no** sub-export for the defer island today. You must **(a)** add the sub-export and **(b)** + name that specifier in `vite.config.ts.template`'s `fresh({ islandSpecifiers: [...] })`. Choose the + concrete specifier string and say why. +- **D2 — fix `f-client-nav`** so `partial-miss` enables client nav. Semantics are already verified; + do not re-litigate them. +- **D3 — move the coordinator form *inside* the region's `<Partial>`.** It currently renders as a + sibling outside it (`DeferPage.tsx:257-275`), so a region swap leaves stale DOM and a page swap + re-renders an inert form. **Fallback if that breaks the fallback-render contract:** stable `key` + + remount. `SlotRef` is not pursued. State which landed and why. +- **D4 — do not change `decideDeferClientAction`'s policy logic** (`policy.ts:177-208`). It is correct + and unit-covered; it is simply never executed. If your fix requires changing it, **stop and report**. +- **D5 — scope.** There is exactly **one** render site (`DeferPage.tsx:263`), and the streaming path + provably never renders it (`runtime/mod.tsx:178`, `!shouldStream && …`). Out of scope: the dead + `debug` prop (`DeferIsland.tsx:54`), and the #1457/#1548 surfaces (both already merged). +- **B4 — migration.** `islandSpecifiers` is a `fresh()` option, so this is a **scaffold-template + change**: already-generated apps do **not** get the fix until regenerated. Carry either a + template-regeneration commit or a tracked follow-up issue, and **say which in the PR**. + +## Required tests + +1. **Client-bundle presence — B3 (binding).** Build the client bundle and assert the defer island is + in it. **You must commit to a fixture location and add it in this PR** — there is no `vite build` + fixture under `packages/fresh/tests/` today. Either a fixture under `packages/fresh/tests/fixtures/` + with a `Deno.test` wrapper, or a scaffolded-app fixture under `packages/cli/e2e/`. This attacks the + exact evidence the issue reports: *"the generated client bundle contains none of `DeferComponent`, + `decideDeferClientAction`, or `partial-miss`."* +2. **`f-client-nav` across all four** `isPartialRequest` × `hasCachedData` combinations, pinning that + `partial-miss` **enables** client nav. This is the guard that would have caught defect (2). +3. **Island marker in server output**, reusing the JSX-tree harness + (`define-page/tests/search-params.test.tsx:90-134`). +4. **Partial-swap behaviour** for whichever D3 technique lands. + +Each must fail if its own defect returns. Where a shape matters more than a value, assert the shape — +the #1548 slice in this lane proved that a behaviour-only suite stays green through a silent +regression. + +## Gates — hardcoded, not conditional + +```bash +deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/fresh --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root packages/fresh --ext ts,tsx +deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/fresh --ext ts,tsx +deno task --cwd packages/fresh test +deno task doc:lint --root packages/fresh --pretty +``` + +`arch:check` (`deno.json:156`) **does not cover `packages/fresh`** (verified). Run `quality:gate` +**and** an explicit target quality scan over `packages/fresh/src`, and state in the PR that the +package-quality verdict rests on the explicit scan. If you touch the scaffold template, add the +`packages/cli` scoped wrappers and expect `scaffold-static` to exercise it in CI. + +Never a bare `deno test <path>` — it omits `--allow-env` and exits 1 on `NotCapable`. + +## PR contract — read this carefully + +PR **#1558** exists. Its body already carries **`Refs #1459`, deliberately not a closing keyword**, +because the client-bundle *navigation* criterion is split to **#1557**. **Do not change `Refs` to +`Closes`.** If you believe your work fully resolves #1459 including browser-navigation proof, say so +in your report and let the orchestrator decide — do not decide it in the body. + +Do **not** emit an `acceptance-evidence` block with an empty entry list; the mirror's parser throws +on it (#1561). If there is nothing to map, omit the block and say why. + +Commit per slice, push by explicit refspec, post `[PHASE: IMPL]` with commit hash and **pasted real +gate output**, and move the label from `status:plan-eval` to `status:impl`. + +## Reporting contract + +Report what changed and where; the concrete specifier you chose and the fixture location you +committed to; the exact test names and what each catches; verbatim gate output; and **anything you +could not do, could not verify, or that surprised you**. If a gate goes red, report the red with its +output — do not work around it silently. + +Do **not** flip the PR to ready (that fires the automatic IMPL-EVAL, which is the orchestrator's +trigger) and do **not** merge. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/plan-1459.md b/.llm/runs/release-0.0.6-features--orchestration/slices/plan-1459.md new file mode 100644 index 0000000000..5cc3070d61 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/plan-1459.md @@ -0,0 +1,258 @@ +# Plan — #1459 `DeferComponent` is not hydrated, so partial-miss regions never refresh + +Lane: 0.0.6 runtime reopen. Control PR #1555. Research: `slices/research-1459.md`. +Branch: `fix/1459-defer-island-hydration`. **Goes to PLAN-EVAL before implementation** via the +`openhands` + `status:plan-eval` label pair. + +## What the research changed + +The issue frames this as "the component is not hydrated". The actual finding is narrower and harder: +**`DeferIsland.tsx` is not an island at all** — there is no island registration for it anywhere +(`DeferIsland.tsx:111` is a *named* export; the scaffold calls `fresh()` with no options at +`vite.config.ts.template:41`; `createNetScriptVitePlugin` has zero island logic across all 438 lines). +Islands in this repo reach the client by **directory convention** or by being **copied into the +consumer app** (`fresh-ui`), and the one JSR-shipped island that works — `QueryIsland` — works only +because the *consumer's own* island file imports it (`query-island.tsx:26-36`). + +And a **second, independent defect** sits in the same component: `DeferIsland.tsx:222` sets +`f-client-nav={!(isPartialRequest && !hasCachedData)}`, whose condition is **exactly** `partial-miss`. +So in precisely the failing case, `f-client-nav` is `false` while `f-partial` is set (`:221`). +Hydration alone may therefore still produce a full document navigation instead of the `/partials/**` +request. These two are **technically inseparable**: fixing hydration without this leaves the issue's +observable symptom unfixed. + +## Scope decision — split, and move one criterion with a written reason + +The issue's acceptance asks for a regression test that *"build[s] the Fresh client bundle, navigate[s] +into a page with a cache-miss deferred layer, assert[s] the partial endpoint is requested, and +assert[s] the named boundary swaps exactly once."* + +This repo has **no browser driver** (Playwright appears only in docs and in the CLI *scaffolding* for +consumer projects) and **no gate that builds or inspects a client bundle**. That criterion therefore +needs **net-new test capability**, which is its own piece of work. + +**Decision: it moves to a follow-up issue with a written reason (#1090 pattern), and this PR does not +tick it.** A p1 runtime blocker — a documented pattern that paints a skeleton which never resolves — +should not wait on net-new browser-test infrastructure. Per the honesty rule, the criterion travels +with a reason rather than being ticked on weaker evidence. + +What this PR **does** prove is stated below and is deliberately not dressed up as the missing test. + +## Locked decisions + +- **D1 — the island must reach the client without depending on a consumer-owned file.** Preference + order, to be settled by the implementer against verified feasibility: + 1. register the island from `createNetScriptVitePlugin` (the only option requiring no consumer + file) — **feasibility is unverified**; the research could not confirm `@fresh/plugin-vite` + supports registering an island from a JSR specifier, and proved only that *this repo passes no + such option*. **Verify this first.** + 2. if (1) is not supported, the `fresh-ui` **copy-mode** precedent (`registry.generated.ts:334-340`). + A consumer-side shim that a consumer can delete — silently reintroducing the bug — is the **least** + preferred and must be argued for if chosen. +- **D2 — fix `f-client-nav` (`DeferIsland.tsx:222`).** Its current condition disables client nav in + exactly the `partial-miss` case it needs to be enabled for. **Verify Fresh's actual + `f-client-nav`/`f-partial` form-interception semantics before changing it** — the research flagged + this as inference, not fact. If Fresh does intercept `f-partial` regardless of `f-client-nav`, say + so and leave the line alone with evidence. +- **D3 — address the partial-swap case.** The coordinator form renders as a **sibling outside** the + `<Partial>` (`DeferPage.tsx:257-275`), so a region swap leaves the form's DOM in place while a + page-level swap re-renders an inert form. The acceptance explicitly requires the coordinator to work + *after a page partial swap*; a fix that only works on first paint does not satisfy it. +- **D4 — no change to `decideDeferClientAction`'s policy logic** (`policy.ts:177-208`). It is correct + and already unit-covered; it is simply never executed. If the fix requires changing it, stop and + report. +- **D5 — out of scope:** the `streams`/`shouldStream` sibling path (`runtime/mod.tsx:178`), the dead + `debug` prop (`DeferIsland.tsx:54`), and #1457/#1548's surfaces. + +## Tests — what this PR can honestly prove + +1. **Island marker in server output.** Reuse the existing JSX-tree harness + (`define-page/tests/search-params.test.tsx:93-99,115-134`) to assert the rendered tree carries + whatever marker the chosen D1 mechanism produces. Fails if registration regresses. +2. **`f-client-nav` condition** (D2) asserted directly for the four combinations of + `isPartialRequest` × `hasCachedData`, pinning that `partial-miss` enables client nav. This is the + guard that would have caught the second defect. +3. **`buildDeferFormState` / decision plumbing** extended so `partial-miss → submit` is exercised + with the real inputs `DeferPage` passes (`DeferPage.tsx:267-268`), not synthetic ones. +4. If D1 lands via the Vite plugin, a unit test that the plugin **emits the island registration**. + +**None of these prove the client bundle contains the island or that a browser issues the partial +request.** That is the moved criterion. Say so plainly in the PR body; do not let tests 1–4 be read as +satisfying it. + +## Gates + +Scoped check/lint/fmt over `packages/fresh`, `deno task quality:gate` (**verify it covers +`packages/fresh`; it demonstrably omits several packages — #1542**), `deno task --cwd packages/fresh +test`, `deno task doc:lint`. If D1 changes the CLI registry or scaffold template, add the +`packages/cli` scoped wrappers and expect `scaffold-static` to exercise it in CI. + +## Risks + +- **R1** — D1's preferred option has **unverified feasibility**. If it fails, the fallback changes the + distribution model (copy-mode), which is a materially different PR. That branch point is why this + plan goes to PLAN-EVAL. +- **R2** — the `f-client-nav` claim is inference. Acting on it without verifying Fresh's semantics + risks changing a correct line. +- **R3** — copy-mode (D1.2) means framework fixes stop reaching already-generated apps; that is a + doctrine-relevant trade, not just an implementation detail. + +## Acceptance mapping (#1459) + +| Criterion | This PR | Evidence | +| --- | --- | --- | +| Coordinator registered/hydrated as a real Fresh island, including after a partial swap | **yes** (D1 + D3) | server-output marker test; plugin-emission test | +| A cache miss triggers the configured partial request and the region settles | **partially** — mechanism fixed (D2), not observed in a browser | `f-client-nav` condition tests | +| Regression test builds the client bundle, navigates, asserts the partial request and a single swap | **no — moved to a follow-up with a written reason** | needs net-new browser/bundle capability this repo lacks | + +--- + +# Plan v2 — revised after PLAN-EVAL `FAIL_PLAN` + +The evaluator verified the second defect **against Fresh core source** rather than leaving it as my +inference: `client.ts:256` gates form submission on `checkClientNavEnabled(el)`, and `shared.ts` +confirms `CLIENT_NAV_ATTR = "f-client-nav"`. So `f-client-nav={false}` in the `partial-miss` case +**does** suppress the interception — the second defect is now **fact, not inference**, and R2 is +closed. It also confirmed the client refresh mechanism is form/anchor interception only: **no +background poll, no auto-refresh timer.** Nothing else will save a missed region. + +Amendments, in the order the verdict raised them. + +## A1 — D1 feasibility is RESOLVED, not punted + +I read `@fresh/plugin-vite` v1.0.8 from the JSR cache. It exposes a public config option whose +documented purpose is exactly this case: + +```ts +/** + * Treat these specifiers as island files. This is used to declare + * islands from remote packages. + */ +islandSpecifiers?: string[]; +``` + +(`https://jsr.io/@fresh/plugin-vite/1.0.8/src/utils.ts`, cached; consumed in the plugin's `options()` +hook, which adds each specifier to `islands` and, in non-dev builds, emits a +`fresh-island::<name>` input entry.) + +**D1 is therefore LOCKED to the specifier route** — no copy-mode, no consumer-owned shim. The +scaffold's `vite.config.ts.template:41` currently calls `fresh()` with **no options**; it must pass +`islandSpecifiers` naming the defer island's specifier. + +**A1a — migration is real and must be named (verdict item 6).** Because `islandSpecifiers` is a +`fresh()` option and not a Vite-level setting, `createNetScriptVitePlugin` cannot inject it by +merging Vite config. It is a **scaffold-template change**, so **already-generated apps do not get the +fix until regenerated**. The PR must state this plainly and either carry a CLI-registry/template +commit or file a tracked migration follow-up. It must not be left implicit. + +## A2 — D3 picks a technique + +**Locked: move the coordinator form *inside* the region's `<Partial>`** (`DeferPage.tsx:257-275` +currently renders it as a sibling outside). That makes a region swap re-render the form, which is the +behaviour the acceptance requires, and it is the minimal structural change. + +**Fallback if that breaks the fallback-render contract:** give the form a stable `key` and force a +remount. `SlotRef` is not pursued. Whichever lands must be stated in the PR with the reason. + +## A3 — D5 call sites enumerated, not assumed + +Every non-test reference to `DeferComponent` in `packages/`: + +- `DeferIsland.tsx:21,111,122` — its own declaration. +- `DeferPage.tsx:8` (import) and `:263` (the **only** render site). + +There is exactly **one** render site. `DeferPage` is reached only from +`builders/define-page/runtime/mod.tsx`, where `shouldDefer = !shouldStream && !!descriptor.config.partial +&& (descriptor.config.delivery ?? 'defer') === 'defer'` (`runtime/mod.tsx:178`). So the streaming path +provably never renders `DeferComponent`, and D5's exclusion is **verified rather than asserted**. + +## A4 — the test commitment, and a correction to the verdict + +The verdict proposed `probe-project-boundary-dev.ts:43` + `FetchHttpAdapter` as an +already-available, non-net-new way to drive the partial request end-to-end. **I checked it, and it +does not do that.** That probe boots a Fresh dev server and issues +`fetch('http://127.0.0.1:${port}/')` in a liveness loop (`:41-52`). There is no browser and no JS +execution, so it can prove the partial **endpoint answers** but cannot prove the **client issues the +request** — and "the client never issues the request" is the entire defect. A server-side fetch +firing the request would be evidence of nothing. + +**What this PR commits to instead**, which is stronger than v1's server-only tests and cheaper than a +browser: + +- **Build the client bundle and assert the defer island is present in it.** With D1 locked to + `islandSpecifiers`, the non-dev build emits a `fresh-island::<name>` entry, so this is directly + checkable. It is modest net-new capability (a `vite build` plus an assertion — **not** a browser + driver), and it attacks the exact evidence the issue reports: *"the generated client bundle + contains none of `DeferComponent`, `decideDeferClientAction`, or `partial-miss`."* +- Plus the v1 tests: `f-client-nav` condition across all four `isPartialRequest` × `hasCachedData` + combinations, and the island-marker assertion in server output. + +**Still moved to #1557:** browser-driven navigation asserting the `/partials/**` request fires and the +boundary swaps exactly once. That needs a real browser and stays out. + +## A5 — closing keyword (verdict item 5): already corrected + +PR #1558's body carried `Closes #1459` while this plan moves one of that issue's criteria — a +self-contradiction, and the #1092 precedent. **Fixed:** the body now reads `Refs #1459` with the +reason stated. The closing keyword goes on whichever PR truthfully completes the issue. + +## A6 — gate commands hardcoded (verdict item 7) + +`arch:check` (`deno.json:156`) does not cover every package, so the gate list is fixed rather than +conditional: run `deno task --cwd packages/fresh test`, the scoped check/lint/fmt wrappers over +`packages/fresh`, **and** an explicit target quality scan over `packages/fresh/src`. If the D1 +template change lands, add the `packages/cli` scoped wrappers and expect `scaffold-static` to +exercise it. + +## Revised acceptance mapping (#1459) + +| Criterion | This PR | Evidence | +| --- | --- | --- | +| Coordinator registered/hydrated as a real Fresh island, incl. after a partial swap | **yes** | `islandSpecifiers` registration + client-bundle presence assertion + form moved inside the `<Partial>` | +| A cache miss triggers the configured partial request and the region settles | **mechanism proven, not observed in a browser** | `f-client-nav` corrected against verified Fresh semantics (`client.ts:256`) | +| Regression test builds the client bundle, navigates, asserts the request and a single swap | **bundle half yes; navigation half moved to #1557** | stated in the PR body without overclaiming | + +--- + +## PLAN-EVAL v2 result — PASS (recorded with an extraction discrepancy) + +Run `31593309658`, MiniMax M3, on plan v2 commit `63ae41bb4`. + +**The verdict is PASS.** The summary body states `Verdict: **PASS**` and ends with +`## OPENHANDS_VERDICT: PASS`. The machine marker nevertheless recorded +`"verdict":"NONE","verdict_source":"none"` — the token was emitted as a **markdown heading**, so the +extractor did not match it. Filed as **#1563**. + +**Not re-run.** Re-dispatching an evaluator purely to obtain a parseable token would be duplicate +spend of exactly the kind the phase-eval policy forbids, to work around a formatting bug rather than +an evaluation failure. The body verdict is accepted and the discrepancy is recorded instead. + +### What it independently verified + +- A1's mechanism, the streaming exclusion (A3), the sibling-form structure (A2), the `f-client-nav` + condition, and the `arch:check` omission of `packages/fresh` — all confirmed against the head SHA. +- PR #1558 correctly carries `Refs #1459`, not a closing keyword. + +### Amendments to fold into implementation + +**B1 — version citation corrected (my error).** Plan v2 cited `@fresh/plugin-vite@1.0.8`, read from +the local Deno cache. The scaffold actually pins **`jsr:@fresh/plugin-vite@^1.1.2`**. The evaluator +verified `islandSpecifiers?: string[]` exists in **1.1.2** with identical JSDoc +(`https://jsr.io/@fresh/plugin-vite/1.1.2/src/utils.ts:59-63`) and confirmed the mechanism at +`src/mod.ts:234-237` (`fConfig.islandSpecifiers.set(spec, name)`). **The D1 conclusion stands; the +citation was to the wrong version.** The implementer must check any 1.0.8-specific assumption against +1.1.2. + +**B2 — the specifier must be created, not just named.** `packages/fresh/deno.json` has **no** +sub-export for the defer island today. A1 therefore requires **(a)** adding the sub-export and +**(b)** naming that specifier in `vite.config.ts.template`'s `fresh({ islandSpecifiers: [...] })`. +This is an implementer decision, but it is a required step, not an optional detail. + +**B3 — the bundle fixture needs a committed location.** A4 says "build the client bundle and assert +the island is present" without saying where the fixture lives. There is **no `vite build` fixture +under `packages/fresh/tests/` today**. The implementer must commit to a location — a fixture under +`packages/fresh/tests/fixtures/` with a `Deno.test` wrapper, or a scaffolded-app fixture under +`packages/cli/e2e/` — and add it **in the same PR**, not defer it. + +**B4 — migration accepted as scoped.** Already-generated apps do not auto-migrate; carrying either a +template-regeneration commit or a tracked migration issue is acceptable. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/research-1459.md b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1459.md new file mode 100644 index 0000000000..8379c20733 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/research-1459.md @@ -0,0 +1,121 @@ +# Research — #1459 `DeferComponent` is not hydrated + +Delegated read-only sub-agent (Claude · Opus 5, `drift.md` D-1), bounded brief — 27 tool calls, +~3 min, within budget. Static analysis only; nothing built, run, or reproduced. + +## The reported defect is confirmed, and its cause is simpler than "hydration is broken" + +**`DeferIsland.tsx` is not an island at all.** There is no island registration for it anywhere in the +repo. + +- It is a **named** export, not a default export + (`packages/fresh/src/application/defer/DeferIsland.tsx:111`), re-exported as an ordinary module + symbol (`defer/mod.ts:12`, reachable as `@netscript/fresh/defer`). +- **The mechanism in this repo is directory convention**, resolved by the stock Fresh Vite plugin. + The scaffold calls `fresh()` with **no options** + (`packages/cli/src/kernel/assets/app/vite.config.ts.template:41`), and `createNetScriptVitePlugin` + contains **zero** island logic — no `islands` match across all 438 lines of + `packages/fresh/src/application/vite/vite.ts`. `define-fresh-app.ts:121-135` registers fs-routes + only and never passes a `loadIsland`. +- Every island that actually works in a scaffolded app lives in a **scanned directory**: the app + template's `(_islands)/` folders, and `@netscript/fresh-ui` islands, which are **copied into the + consumer app** (`packages/fresh-ui/registry/islands/`, registered as copy templates at + `registry.generated.ts:334-340`) rather than imported from the package. +- The contrast that proves the pattern: `QueryIsland` + (`packages/fresh/src/application/query/query-island.tsx:39`) ships from the same JSR package and + **does** reach the client — because its documented contract is that the **consumer's own** island + file imports it (`:26-36`), giving it a scanned entry point. `DeferComponent` has no such entry. + +So the component is reached **only** through a server-render path +(`DeferPage.tsx:8` ← `builders/define-page/runtime/mod.tsx:4`), and nothing in a scanned directory +transitively imports it. Both candidate causes hold and compound. + +## A second, independent defect inside the same issue + +`DeferIsland.tsx:222` sets: + +```tsx +f-client-nav={!(isPartialRequest && !hasCachedData)} +``` + +`isPartialRequest && !hasCachedData` **is exactly the `partial-miss` condition**. So in precisely the +case this issue is about, `f-client-nav` is `false` while `f-partial` is set (`:221`). If Fresh only +intercepts `f-partial` on forms under active client navigation, `requestSubmit()` would trigger a +**full document navigation**, not the `/partials/**` request the issue requires. + +**This is not verified** — Fresh core's form/partial interception was not read, so it is inference +from the attribute values. But it means **hydration alone may not fix the issue**, and any fix must +test this line rather than assume it. + +## Why no existing test catches this + +`decideDeferClientAction` (`policy.ts:177`, `partial-miss` branch at `:178-182`) has **exactly one +caller** — the client effect at `DeferIsland.tsx:188`. There is no server-side caller. In the shipped +product it is therefore **never executed at runtime on either side**; its unit coverage is the only +thing exercising it. + +The input plumbing is correct end-to-end (`runtime/mod.tsx:220` → `DeferPage.tsx:170,267-268`), which +is why this looks healthy from the server side. `DeferIsland.test.ts:1` imports only +`buildDeferFormState` and `sanitizeDeferSearchParams` and never renders the component. + +## Partial-swap behaviour + +From `DeferPage.tsx:257-275`: the coordinator form is rendered **as a sibling outside** the +`<Partial>` — `<div><Partial name={name}>…</Partial><DeferComponent …/></div>`. + +- A swap of the region's own partial replaces the content but **leaves the form's DOM node in + place**, so even a working island would not naturally re-run its effect (deps at `:201-214` are all + props). +- A **page-level** partial swap re-renders the whole `<div>` including a freshly server-rendered, + inert form — with no island marker, Fresh has nothing to hydrate into it. + +Nothing in the defer code addresses island persistence or re-hydration across `<Partial>` swaps: no +`key`, no persistence attribute, no remount handling. The docs describe the transport and make no +persistence claim (`docs/site/web-layer/defer-streaming-ui.md:238-245`). + +## The regression test the issue asks for requires net-new capability + +The issue's acceptance says a test should *"build the Fresh client bundle, navigate into a page with +a cache-miss deferred layer, assert the partial endpoint is requested, and assert the named boundary +swaps exactly once."* + +What exists: + +- Pure-function unit tests (`DeferIsland.test.ts`) — never render the component. +- JSX-tree prop assertions (`define-page/tests/search-params.test.tsx:93-99,115-134`) — could assert + an island marker in **server** output, but cannot prove a client bundle. +- A live Fresh dev server over HTTP (`probe-project-boundary-dev.ts:43` with `FetchHttpAdapter`) — + the closest existing "drive a real app". + +What does **not** exist: + +- **No browser driver.** Playwright appears only in docs and in the CLI *scaffolding* Playwright + config for consumer projects (`init-agent.ts`) — not test infra this repo runs. +- **No gate builds or inspects a client bundle** anywhere in `packages/cli/e2e`. + +So the acceptance's regression test needs net-new capability at whichever level it targets. + +## Options (research made no recommendation) + +| Option | Trade-off | +| --- | --- | +| Consumer-side island shim from the scaffold (`routes/(_islands)/DeferIsland.tsx` re-exporting `DeferComponent`) | matches the `QueryIsland`/`fresh-ui` precedent, but every existing scaffolded app needs regeneration, and a consumer who deletes the file silently reintroduces the bug | +| Copy-mode distribution like `fresh-ui` (`registry.generated.ts`) | consistent with `ui:add`, but forks the island per project so framework fixes stop reaching existing apps | +| Declare the island from `createNetScriptVitePlugin` | the only option needing no consumer file — but **feasibility unconfirmed**, depends on a `@fresh/plugin-vite` capability not verified | +| Drop the island; resolve server-side on `ctx.isPartial && !component` | removes the hydration dependency and the `f-client-nav` hazard, but collapses the defer model — the policy branches (`policy.ts:184-208`) have no server equivalent | +| Inline `<script>` beside the form | cheapest and immune to island-scan mechanics, but forfeits typed policy logic, likely trips CSP, and is doctrine-hostile for `packages/` | + +**Every option must additionally resolve the `f-client-nav` question**, or hydration alone produces a +document navigation instead of the partial request. + +## Explicitly not verified + +- Whether `@fresh/plugin-vite` exposes any mechanism to register an island from a JSR/npm specifier. + The Deno-cache grep was inconclusive; **only "this repo passes no such option" is proven.** +- Fresh's actual `f-client-nav` / `f-partial` form interception semantics — so the + document-navigation claim is inference, not fact. +- No reproduction: nothing built, no browser, no gates. +- The `streams`/`shouldStream` sibling path (`runtime/mod.tsx:178`) — deferral only applies when + `!shouldStream`, and whether streaming delivery has the same gap is unexamined. + +Minor: the `debug` prop is declared (`DeferIsland.tsx:54`) but never destructured (`:111-122`) — dead. diff --git a/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1459.md b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1459.md new file mode 100644 index 0000000000..3b11c73a79 --- /dev/null +++ b/.llm/runs/release-0.0.6-features--orchestration/slices/worklog-1459.md @@ -0,0 +1,143 @@ +# Worklog — #1459 deferred refresh coordinator hydration + +Implementation lane: Codex · GPT-5.6 Sol · high (`complex_implementation`). +Branch: `fix/1459-defer-island-hydration`. +Baseline: `origin/main@59e435c5d`. + +## Design + +- **Profile:** `@netscript/fresh` remains Archetype 4 with the frontend overlay. This slice adds a + narrow package subpath and changes the browser behavior materialized by the existing page DSL. +- **Public surface:** add `@netscript/fresh/defer/island`, backed by + `src/application/defer/island.ts`. The entry re-exports only `DeferComponent`, so Fresh 1.1.2 does + not mistake the helper functions in `DeferIsland.tsx` for additional islands. +- **Scaffold contract:** generated app import maps resolve that same specifier in JSR and local + source modes, and the Vite template passes it to `fresh({ islandSpecifiers: [...] })`. +- **Client navigation:** preserve the three already-enabled request/cache combinations and enable + the missing fourth (`partial-miss`) by rendering `f-client-nav` unconditionally. The locked + `decideDeferClientAction` policy remains unchanged. +- **Partial lifecycle:** move the existing coordinator vnode inside the named `<Partial>`. A + miss-to-hit region swap therefore replaces the coordinator with current props and gives Fresh the + registered component identity to hydrate again. The stable-key fallback is unnecessary. +- **Bundle fixture:** `packages/fresh/tests/fixtures/defer-island-client/`, driven by + `packages/fresh/tests/defer-island-client-bundle_test.ts`, builds with the pinned + `@fresh/plugin-vite@1.1.2` and inspects the production client manifest and island chunk. +- **Commit slice:** one implementation slice covers D1–D3 because hydration and client navigation + are inseparable. Its gates are the four required regression tests, scoped Fresh/CLI wrappers, + package tests, doc lint, `quality:gate`, and the explicit `packages/fresh/src` quality scan. +- **Migration:** regenerate and commit the embedded CLI template asset. Already-generated apps do + not receive the new `fresh()` option; they must regenerate their app scaffold. +- **Deferred scope:** browser-driven navigation and exact swap-count proof remain in #1557; the + dead `debug` prop, streaming delivery, and #1457/#1548 surfaces remain untouched. + +## Implementation + +- Added the published `./defer/island` export. Its runtime surface contains only + `DeferComponent`; the component props and their policy dependencies are type-only exports so + Fresh cannot register the helper functions in `DeferIsland.tsx` as islands. +- Added `@netscript/fresh/defer/island` to the kernel, maintainer-local, and public-JSR resolvers; + generated app manifests now carry the literal import-map key in both source modes. +- Updated the Fresh-init fallback from its stale `@fresh/plugin-vite@^1.0.8` literal to the + canonical `SCAFFOLD_APP_CATALOG.FRESH_PLUGIN_VITE` (`^1.1.2`). +- Regenerated `embedded.generated.ts` and `agent-docs.generated.ts`. The lock change is the + `packages/fresh` dependency edge for `jsr:@fresh/plugin-vite@^1.1.2` plus that plugin's resolved + Fresh/Vite/Babel graph; no cache or unrelated source dependency churn is present. The graph is + required so root `deno ci` sees the new package dependency rather than a dirty or incomplete lock. +- The bundle fixture aliases only `@opentelemetry/api` to a no-op fixture module. This avoids an + unrelated workspace-only telemetry-resolution failure while compiling the real defer island, + real policy, and Fresh 1.1.2 client entry. + +## Regression tests + +- `vite build emits the registered defer island in the client bundle` — builds the committed Vite + fixture, asserts exactly one `fresh-island__*` manifest entry, and verifies its emitted chunk + contains `partial-miss`. +- `DeferComponent enables f-client-nav for every request and cache combination` — checks all four + `isPartialRequest` × `hasCachedData` combinations, including the formerly-disabled partial miss. +- `DeferPage server tree carries the registered defer island component identity` — uses the + precompiled JSX-tree shape and requires the exact function identity exported by the island + subpath. +- `DeferPage keeps the coordinator inside the named partial across miss-to-hit swaps` — proves the + named `<Partial>` is the only outer dynamic child and that both miss and hit trees contain a + coordinator with current request/cache props. +- `published manifest declares every catalog-backed Fresh runtime dependency` now also pins the + `./defer/island` export shape. CLI generator/resolver and embedded-template tests cover the JSR, + local-source, Fresh-init fallback, and regenerated asset paths. + +## Validation + +All final commands exited 0. Verbatim terminal output or terminal excerpts follow; the full +package-test run reported `227 passed | 0 failed`, and the targeted CLI run reported +`10 passed (23 steps) | 0 failed`. + +```text +$ deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/fresh --ext ts,tsx +{"source":{"mode":"selection","cwd":"/home/codex/repos/ns006-1459"},"command":"deno check --unstable-kv <files>","selection":{"filesSelected":188,"batches":2,"failedBatches":0},"summary":{"totalOccurrences":0,"uniqueOccurrences":0,"uniqueCodes":0,"uniquePaths":0},"groups":[]} + +$ deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root packages/fresh --ext ts,tsx +{"source":{"mode":"command","cwd":"/home/codex/repos/ns006-1459","exitCode":0},"selection":{"filesSelected":188,"batches":1},"summary":{"totalOccurrences":0,"uniqueOccurrences":0,"uniqueRules":0,"uniquePaths":0},"groups":[]} + +$ deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/fresh --ext ts,tsx +{"command":"deno fmt --check","cwd":"/home/codex/repos/ns006-1459","mode":"check","summary":{"filesSelected":188,"batches":1,"failedBatches":0,"findings":0,"ignoredFindings":0},"findings":[]} + +$ deno task --cwd packages/fresh test +ok | 227 passed | 0 failed (8s) + +$ deno task doc:lint --root packages/fresh --pretty +"summary": { + "totalPackages": 1, + "totalErrors": 44, + "totalPrivateTypeRef": 27, + "totalMissingJSDoc": 17, + "totalOther": 0 +} +... +{ + "path": "./src/application/defer/island.ts", + "privateTypeRef": 0, + "missingJSDoc": 0, + "other": 0, + "total": 0 +} + +$ deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/cli --ext ts,tsx +{"source":{"mode":"selection","cwd":"/home/codex/repos/ns006-1459"},"command":"deno check --unstable-kv <files>","selection":{"filesSelected":861,"batches":8,"failedBatches":0},"summary":{"totalOccurrences":0,"uniqueOccurrences":0,"uniqueCodes":0,"uniquePaths":0},"groups":[]} + +$ deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root packages/cli --ext ts,tsx +{"source":{"mode":"command","cwd":"/home/codex/repos/ns006-1459","exitCode":0},"selection":{"filesSelected":861,"batches":5},"summary":{"totalOccurrences":0,"uniqueOccurrences":0,"uniqueRules":0,"uniquePaths":0},"groups":[]} + +$ deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/cli --ext ts,tsx +{"command":"deno fmt --check","cwd":"/home/codex/repos/ns006-1459","mode":"check","summary":{"filesSelected":861,"batches":5,"failedBatches":0,"findings":0,"ignoredFindings":0},"findings":[]} + +$ deno run --allow-read .llm/tools/quality/scan-code-quality.ts --root packages/fresh/src +{"ok":true,"mode":"repository","scanned":["packages/fresh/src"],"findings":[],"allowCount":1,"allowances":[{"file":"packages/fresh/src/application/builders/define-page/builder/route-support.ts","line":96,"reason":"DefinePageWithRouteContract preserves prior path/search output when either optional schema is omitted, but BoundRouteContract maps an omitted schema to EmptyRecord; TypeScript cannot equate those conditional states without presence-specific legacy builder overloads"}]} + +$ deno task quality:gate +Task quality:gate deno task quality:scan && deno task arch:check +Task quality:scan deno run --allow-read .llm/tools/quality/scan-code-quality.ts +{"ok":true,"mode":"repository","scanned":["packages/cli/src","plugins"],"findings":[],"allowCount":7,"allowances":[...]} +Task arch:check deno task deps:check && deno run --allow-read .llm/tools/fitness/check-doctrine.ts ... +``` + +The package-quality verdict for `packages/fresh` rests on the explicit target scan above because +root `arch:check` does not include this package. + +Two required gates were red during implementation and were corrected rather than bypassed: + +```text +$ deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root packages/fresh --ext ts,tsx +{"source":{"mode":"selection","cwd":"/home/codex/repos/ns006-1459"},"command":"deno check --unstable-kv <files>","selection":{"filesSelected":188,"batches":2,"failedBatches":1},"summary":{"totalOccurrences":2,"uniqueOccurrences":2,"uniqueCodes":2,"uniquePaths":1},"groups":[{"code":"TS9036","message":"Move the expression in default export to a variable and add a type annotation to it.","count":1,"paths":[{"path":"/home/codex/repos/ns006-1459/packages/fresh/tests/fixtures/defer-island-client/vite.config.ts","count":1,"locations":[{"line":5,"column":1}]}]},{"code":"TS9037","message":"Default exports can't be inferred with --isolatedDeclarations.","count":1,"paths":[{"path":"/home/codex/repos/ns006-1459/packages/fresh/tests/fixtures/defer-island-client/vite.config.ts","count":1,"locations":[{"line":5,"column":16}]}]}]} + +$ deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root packages/fresh --ext ts,tsx +{"command":"deno fmt --check","cwd":"/home/codex/repos/ns006-1459","mode":"check","summary":{"filesSelected":188,"batches":1,"failedBatches":1,"findings":1,"ignoredFindings":0},"findings":[{"path":"/home/codex/repos/ns006-1459/packages/fresh/src/application/defer/island.ts","reason":"-export type {"}]} +``` + +The first was fixed by assigning an explicitly typed `UserConfig` before default export; the +second by formatting the type-only exports. Both green reruns are included above. + +## Not verified in this lane + +- No browser-driven navigation or exact one-swap assertion was run; #1557 owns that criterion. +- `scaffold-static` was not run locally; the scoped CLI wrappers, targeted generator/registry + tests, and regenerated embedded asset are green, and CI is expected to exercise that suite. +- The PR remains draft and no IMPL-EVAL was triggered. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 867ab8bd71..6072176fe2 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,6 +8,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | +| 4 | 2026-08-12 ~14:33Z | `e391f3aec` | **#1570** | **#1403** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **22 checks, 0 current failures** at `807d29003`; `close-gate` green after label + workflow re-run; **8/8** issue boxes mirrored from `box-index` evidence; `review-threads` PASS. Verified **on merged main**: `arch:check` exit 0 **at 36 roots** (was green only at 16), `quality:scan:repo` exit 0. Evaluator provenance preserved on proof: `PASS` at `c740ff6e0`, head moved to `807d29003` by another lane's `update-branch`, owned-path diff **empty**, so the evaluated implementation is the shipped implementation. | | 3 | 2026-08-12 13:40Z | `b79eca5d6` | **#1567** | **#1566** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` 0 current failures at `f6def9946`; `close-gate` PASS; 6/6 issue boxes mirrored; the one PR-body DoD box untickable at hand-off ticked by the orchestrator with run identities as evidence; `review-threads` PASS. **Merged by the release coordinator during a Claude 529 outage**, independently re-verified here: main contains it, `phase-eval-status.mjs` is present on trusted main, both items `status:shipped`. Verdict was head-matched to `f6def9946` before consumption (one trigger `gen=29340872564`, one verdict run `31599209037`). | | 2 | 2026-08-12 ~12:24Z | `e67c1ba13` | **#1560** | **#1530** (auto-closed `COMPLETED`) | 7/7 pass — `pr-checks` **23 checks, 0 current failures** at `28fc1b423`; `close-gate` green after label + workflow **re-run** (not a push); boxes 1–6 mirrored, box 7 `[post-merge]` verified after merge and ticked with evidence; `review-threads` PASS. **Two IMPL-EVAL verdicts:** `PASS` at `49e2b86e9` (pre barrel fix) then **`FAIL_FIX`** at `9ab361440` on a real close-gate defect, then `PASS` at `28fc1b423`. Consuming the first would have merged a red gate. | | 1 | 2026-08-12 ~08:31Z | `63cd1cd58` | **#1527** | **#1436**, **#1415** (both auto-closed `COMPLETED`) | 7/7 pass — full record in the PR's `[PRE-MERGE GATE]` comment. `pr-checks` **15/15 `current-pass`, 0 current failures** at `dfda54a16`; `close-gate` green; #1415 4/4 boxes mirrored with linked evidence; #1436 has 0 boxes so the PR body is its record; no new ignores/casts, no lock churn; both probes re-run independently by the orchestrator; `review-threads` PASS (0 threads). | @@ -18,7 +19,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | --- | --- | --- | --- | --- | | 1 | PR-A `fix/1436-1415-close-gate-trust` → **PR #1527** | #1436, #1415 | Sol · low | **MERGED** `63cd1cd58` — thread `019ff4f4-1fce-7253-a7e0-d718c65b39cc`, worktree `/home/codex/repos/ns006-gatetrust`, 7 commits | | 2 | PR-E `fix/1530-type-fixture-scan-scope` → **PR #1560** | #1530 | Sol · low | **MERGED** `e67c1ba13` — thread `019ff5b2-7d02-…`, worktree `/home/codex/repos/ns006-typefixtures`, 6 commits. Restored `main`'s blocking `code-quality-repo` job: first green in **nine** consecutive push-to-main runs. | -| 2 | PR-B `fix/1403-quality-gate-coverage` | #1403 | Sol · low | not yet dispatched | +| 2 | PR-B `fix/1403-quality-gate-coverage` → **PR #1570** | #1403 | Sol · low | **MERGED** `e391f3aec` — absorbed rail `R-5` (A14 origin-awareness) after the implementer caught the plan's incoherent split; 3 defects fixed on one gate | | 2 | PR-C `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | not yet dispatched | | 2 | PR-D `fix/1378-quality-scan-rule-power` | #1378 | Sol · high | not yet dispatched | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index a0454e7c40..691a85a6d4 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1244,3 +1244,32 @@ defers the close by hand. This lane's `[post-merge]` marker (#1530 box 7, workin prints "excluded post-merge box(es) … verify in a follow-up comment and tick after merge") keeps the closing keyword and the auto-close instead. The distinction they drew back is the right one: the marker fits a **genuinely post-merge-only** fact; `Refs` fits a box blocked on **another PR**. + +## D-38 — #1403 closed; `arch:check` now gates 36 units, verified on merged main + +- **Recorded:** 2026-08-12 +- **Merge:** PR #1570 → `e391f3aec`, closing **#1403** (`CLOSED/COMPLETED`, `status:shipped`), 8/8 boxes + mirrored from `box-index` evidence. +- **Verified on merged `main`, not inferred from the PR:** + +```text +deno task arch:check → exit 0 ← now iterating 36 discovered roots +deno task quality:scan:repo → exit 0 +``` + + `arch:check` was previously green only because it gated **16** hand-listed roots; `packages/cli` was not + among them, which is why 54 A14 findings were invisible to it. Green at 36 with A14 origin-aware is a + materially different claim from green at 16. +- **Three defects on one gate, all closed:** the curated root omission; the PR gate never scanning + `.llm/tools/**` **and** skipping entirely on an empty set (`if ((${#args[@]}))` — no command, reported + success); and the two-dot range that enumerated other lanes' merged files. +- **The plan error, and why the escalation mattered.** `R-6` put the 36-root expansion here and `R-5` (A14 + origin-awareness) in PR-C. Incoherent: the expansion is exactly what surfaces A14's false positives to + `arch:check`. The implementer escalated rather than silently fixing A14 or shipping a red gate, `R-5` moved + into PR-B, and the gate is green. **#1380 box 5 is therefore implemented by PR-B**, referenced without a + closing keyword; PR-C ticks it citing `e391f3aec`. +- **Allowance count rises 8 → 10**, stated on the PR rather than discovered: two `explicit-any` false + positives where the scanner matches the English word "any" in comments, routed to **#1549**, deleted when + that lands comment-awareness. First time this lane's count has risen, and it is reversible by construction. +- **Lane state: #1436, #1415, #1530, #1566, #1403 closed.** Remaining owned: **#1380** (PR-C), **#1549** + (PR-D). diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/context-pack.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/context-pack.md new file mode 100644 index 0000000000..f0198199b1 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/context-pack.md @@ -0,0 +1,35 @@ +# Context pack: PR-B #1403 + +## Current state + +- Draft PR #1570 is open from bootstrap commit `059576fcd`. +- The formal quality-rail plan revision 4 passed PLAN-EVAL cycle 5. +- RED-first fixtures cover B1 and B3 and currently fail because both planned contracts are absent. +- B1/B3 implementations make the focused fixtures green. B2 exposes the passed-plan contradiction + recorded as `drift.md` D-1: the final 36 roots contain 54 known A14 failures while this slice is + forbidden to change A14 and is required to keep `arch:check` green. + +## Locked implementation + +- Discover exactly the 36 top-level package/plugin units; never use every workspace member. +- `packages/cli/e2e` is nested and excluded, with the reason written into doctrine. +- `arch:check` consumes discovery in the same change that removes the curated task list. +- The PR selector includes `packages`, `plugins`, and `.llm/tools`, reports empty explicitly, and + diffs `BASE...HEAD`. +- Findings are triaged only; package/plugin source is out of scope. +- `triage.md` records 1 actionable `plugin-streams-core` doctrine warning plus a 2-entry temporary + #1549 allowance register for changed-tool comment false positives. The focused package quality + scan is green with zero findings and zero allowances. + +## Orchestrator decisions applied + +- R-5 moved into PR-B. A14 is lexical-origin-aware and retains a synthetic unresolved RED case. +- The two tool-comment false positives now have reversible #1549 per-line allowances; current + triage is 1 actionable package finding plus a 2-entry temporary allowance register. +- Final wrapper roots are the owned `.llm/tools/quality` and `.llm/tools/fitness` trees. + +## Next + +Commit and push the resolved B1–B3 implementation with generated assets, rerun the final-head +idempotence/status check, and update draft PR #1570. The orchestrator then re-syncs against main and +owns the ready transition plus separate-session IMPL-EVAL. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/drift.md new file mode 100644 index 0000000000..6718274c15 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/drift.md @@ -0,0 +1,57 @@ +# Drift: PR-B #1403 + +Append-only. No drift from the passed plan at bootstrap or RED-first fixture time. + +## D-1 — significant: final 36-root selector conflicts with required green `arch:check` + +After implementing the locked R-6 transition, `deno task arch:check` exits **1**. Discovery reaches +all 36 intended roots, but 54 pre-existing A14 findings become blocking: 52 under `packages/cli`, +one under `packages/database`, and one under `packages/mcp`. This is the same known population the +passed plan records inside the baseline `arch:check:repo` result (55 total = 54 A14 + root-level +A1). + +The brief simultaneously requires the final 36-root selector, requires `arch:check` to remain exit +0, forbids changing A14 (PR-C #1380 owns it), and forbids fixing surfaced findings. Those conditions +cannot all hold. No suppression or source fix was applied. B1/B3 continue; B2's final gate is +escalated to the orchestrator. + +## D-2 — minor: mandatory root formatter has unrelated pre-existing red + +The exact scoped check and lint wrappers over `.llm/tools --ext ts` pass. After formatting every +PR-B-owned TypeScript file, the exact format wrapper still exits **1** solely for the pre-existing, +out-of-scope `.llm/tools/harness/extract-verdict.ts`. PR-B boundaries allow changes only under the +fitness and quality tool subtrees, so this slice does not edit that file. A focused format check of +all owned TypeScript is green; the root-wrapper residue is escalated rather than folded into this +PR. + +## D-3 — significant: repaired PR scan truthfully reds on two pre-existing comment false positives + +The exact workflow-equivalent changed-file scan at `ca52c3a8f` executes and exits **1** on +`.llm/tools/fitness/check-doctrine.ts:169` and `:237`. Both lines contain the English word “any” in +comments; neither is TypeScript `any`. They predate PR-B's semantic changes but become visible +because the repaired gate scans the changed tool file. C6 forbids fixing surfaced findings here, +and the boundaries forbid allowance comments, so both are recorded in `triage.md` without a fix. +This is evidence that C4 is no longer silently green, but it also prevents the workflow job from +being green at this head. + +## D-4 — orchestrator resolution: R-5 moves from PR-C to PR-B + +The orchestrator confirmed D-1 was a plan-ordering defect. The 36-root transition and A14 +origin-awareness are a matched pair, so R-5 now lands in PR-B. `resolveIdentifierOrigin()` performs +lexical import and local-binding collection and returns `imported | locally-bound | unresolved`; +A14 fires only on `unresolved`. One test exercises all three origins through the actual CLI, with a +synthetic unresolved fixture that exits 1. `deno task arch:check` now exits 0 across all 36 roots. +#1380 remains open; its box 5 implementation is provided here for PR-C to cite and tick. + +## D-5 — orchestrator resolution: temporary #1549 allowances + +D-3's two comment false positives receive exactly two per-line `quality-allow:` comments. Each +reason says the scanner matched an English comment word rather than TypeScript `any` and routes the +durable comment-awareness fix to #1549. The PR-owned repo scan allowance count rises **8 → 10**; +both additions are designed to be deleted by #1549. + +## D-6 — orchestrator correction: wrapper scope is the owned tool trees + +The brief's `.llm/tools` wrapper root was too broad. Final wrapper evidence uses only +`.llm/tools/quality` and `.llm/tools/fitness`; the pre-existing unformatted +`.llm/tools/harness/extract-verdict.ts` remains untouched and outside PR-B. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/triage.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/triage.md new file mode 100644 index 0000000000..ae23477c56 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/triage.md @@ -0,0 +1,31 @@ +# Newly surfaced findings triage + +The repaired scans leave **1 actionable finding** in `packages/plugin-streams-core`. Two additional +scanner reports in the changed tool file were classified as comment false positives and carry +temporary, per-line allowances linked to #1549. No package finding is fixed in PR-B. + +| File | Line | Rule | Assessment | +| --- | ---: | --- | --- | +| `packages/plugin-streams-core/src/application/durable-stream-producer-supervisor.ts` | 501 | `A8/AP-1/F-1` | The file is 515 lines, crossing the doctrine's 500-line warning threshold. This is pre-existing decomposition debt and should be handled in a package-owned follow-up, not in the gate-coverage PR. | + +Focused quality scan evidence: + +```text +deno task quality:scan --pretty --root packages/plugin-streams-core +exit 0; findings=0; allowCount=0 +``` + +Focused doctrine evidence reports the one warning above and one informational A9 reminder that the +package has no `docs/architecture.md`. The A9 record is informational rather than a finding, so it +is not counted in the triage total. + +## Temporary allowance register + +| File | Line | Rule | Assessment | +| --- | ---: | --- | --- | +| `.llm/tools/fitness/check-doctrine.ts` | 210 | `explicit-any` | The scanner matches the English word “any” in an existing comment (`any export abstract class`), not a TypeScript `any`. A per-line allowance names #1549; delete it when that issue adds comment-awareness. | +| `.llm/tools/fitness/check-doctrine.ts` | 278 | `explicit-any` | The scanner matches the English word “any” in an existing heuristic comment (`any class chain`), not a TypeScript `any`. The same reversible #1549 allowance applies. | + +The committed pre-allowance changed-file run at `b64550722` exited 1 on these two lines. That is the +red-first proof that a `.llm/tools`-only PR now executes and reports; before PR-B the workflow ran no +command and returned success. The final scan is green with two reported allowances. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/worklog.md new file mode 100644 index 0000000000..8d15b34991 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-b-1403/worklog.md @@ -0,0 +1,108 @@ +# Worklog: PR-B #1403 quality-gate coverage + +## Identity + +- Worktree: `/home/codex/repos/ns006-qualitygate` +- Branch: `fix/1403-quality-gate-coverage` +- Base: `3c9dc1f3907c605d2d30d76f5a20ade1e4754736` +- Draft PR: #1570 +- Route: Codex · GPT-5.6 Sol · low +- PLAN-EVAL: PASS, quality-rail revision 4, cycle 5 + +## Design + +The parent orchestration worklog is authoritative. This leaf executes its locked B1–B3 slices: + +1. B1 — export `discoverDoctrineRoots()` with the final 36-unit top-level `packages/*` + + `plugins/*` selector and compare it with an independently enumerated expected set. +2. B2 — repoint `arch:check` to that function in one transition; state why nested + `packages/cli/e2e` is outside doctrine-root scope. +3. B3 — make changed-file selection include `.llm/tools/**`, report an empty set as not scanned, + and use three-dot merge-base semantics; triage findings without source fixes. + +No package/plugin public surface changes; archetype and jsr-audit are N/A. Doctrine A14 and F-19 +govern the gate-truth changes. + +## RED-first evidence + +Command: + +```text +deno test --allow-read --allow-env --allow-write --allow-run \ + .llm/tools/fitness/check-doctrine_test.ts \ + .llm/tools/quality/changed-source-files_test.ts +``` + +Exit **1**. Type checking reports both missing contracts: + +```text +TS2307: Cannot find module '.llm/tools/quality/changed-source-files.ts'. +TS2305: check-doctrine.ts has no exported member 'discoverDoctrineRoots'. +``` + +This single committed fixture set proves the doctrine selector and PR changed-file behavior red +before either implementation exists. The `.llm/tools`-only and stale-base cases are explicit test +fixtures, not inferred from the final implementation. + +## Reconcile notes + +- Bootstrap: live issue #1403 has 8 acceptance boxes; draft PR #1570 carries `Closes #1403`, a + non-closing reference to #1564, the required labels, exactly one `status:impl`, and milestone + 0.0.6. + +## Gates + +- `deno task arch:check` after the 36-root transition — exit **1**, 54 known pre-existing A14 + findings. See `drift.md` D-1; no findings fixed or suppressed. +- `deno task quality:scan --pretty --root packages/plugin-streams-core` — exit **0**, 0 findings, + 0 allowances. +- Focused doctrine result for `plugin-streams-core` — 0 FAIL, 1 WARN, 1 INFO. The single actionable + warning is recorded in `triage.md`; no package source was edited. +- Scoped check wrapper (`--root .llm/tools --ext ts`) — exit **0**, 287 files, 0 diagnostics. +- Scoped lint wrapper (`--root .llm/tools --ext ts`) — exit **0**, 287 files, 0 diagnostics. +- Scoped format wrapper (`--root .llm/tools --ext ts`) — exit **1** solely for pre-existing, + out-of-scope `.llm/tools/harness/extract-verdict.ts`; every owned TS file passes. See D-2. +- `code-quality.yml` — `@std/yaml` parse exit **0**; draft workflow policy tests 3/3 pass. +- Workflow-equivalent changed-file scan at `ca52c3a8f` — exit **1** on 2 pre-existing comment false + positives, both recorded in `triage.md`; see D-3. +- `deno task quality:gate` — exit **1**: default quality scan is green, then the discovered-root + doctrine half fails on the 54 known A14 findings from D-1. +- `deno task quality:scan:repo` — exit **0**, 0 findings, 8 allowances. +- `deno task gen:assets-barrel` second run — exit **0** and `git status --porcelain` empty; + generated assets are fresh and idempotent. + +## Orchestrator rescope + +- R-5 moved from PR-C into PR-B because 36-root discovery and A14 origin-awareness cannot be split. +- Actual-cli three-origin fixture: imported exit 0, locally bound exit 0, unresolved exit 1 with + `FAIL A14`. +- `deno task arch:check` after R-5 — exit **0** over all 36 roots. +- Two authorized comment false positives carry temporary #1549 allowances; the PR-owned repo scan + allowance census is **8 → 10**. +- Wrapper scope corrected to `.llm/tools/quality` + `.llm/tools/fitness`; no change to the unrelated + formatter residue. + +## Final gate evidence after orchestrator decisions + +| Gate | Result | Evidence | +| --- | --- | --- | +| Fitness + quality tests | PASS, exit 0 | 15 passed, 0 failed; includes 36-root census and the imported/local/unresolved A14 CLI fixture | +| `deno task arch:check` | PASS, exit 0 | all 36 roots; CLI/database/MCP A14 false positives eliminated by origin resolution | +| `deno task quality:gate` | PASS, exit 0 | default quality scan followed by the green 36-root doctrine gate | +| `deno task quality:scan:repo` | PASS, exit 0 | 0 findings, allowCount **10** (base 8 + 2 reversible #1549 comment allowances) | +| Owned scoped check | PASS, exit 0 | roots `.llm/tools/quality` + `.llm/tools/fitness`, 10 files, 0 diagnostics | +| Owned scoped lint | PASS, exit 0 | same 10 files, 0 diagnostics | +| Owned scoped format | PASS, exit 0 | same 10 files, 0 findings | +| Workflow-equivalent PR scan | PASS, exit 0 | `.llm/tools` files scanned; 0 findings, 2 reported allowances | +| Workflow sanity | PASS, exit 0 | draft policy 3/3; `code-quality.yml` parses through `@std/yaml` | +| Asset barrel generation | PASS, exit 0 | generated tool embedding refreshed; final second-run cleanliness checked after commit | + +## Final reconcile + +- #1403 remains the sole closing issue and has eight index-based evidence entries on draft PR #1570. +- #1380 is referenced without a closing keyword: box 5's A14 implementation lands here for PR-C to + cite and tick; no #1380 checkbox was changed by this lane. +- #1549 remains open and owns deletion of the two temporary comment allowances when scanner + comment-awareness lands. +- The actionable `plugin-streams-core` 515-line A8 warning remains unchanged in `triage.md`. +- Draft PR #1570 stays `status:impl`; the orchestrator retains ready/merge authority. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/context-pack.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/context-pack.md new file mode 100644 index 0000000000..21dd8e1ba4 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/context-pack.md @@ -0,0 +1,44 @@ +# PR-F #1566 Context Pack + +## Objective + +Make the ready-for-review IMPL-EVAL status cleanup read live labels and tolerate only the specific +missing-label 404 race, while preserving generation deduplication and the single-status taxonomy. + +## Current state + +- Branch is based on `e67c1ba13` (`origin/main`). +- The supplied implementation brief is committed at `fe1d3b5e8` and pushed. +- Live issue #1566 has six acceptance boxes; PR evidence must map them with `box-index: 1..6`. +- Draft PR #1567 is open with `type:fix`, `area:tooling`, `priority:p2`, `status:impl`, and milestone + `0.0.6`; it remains draft. +- The S1 test file defines the extracted module contract, race regression, narrow 403/unrelated-404 + failures, terminal single-status state, and a guard for the unchanged generation-dedup ordering. +- S2 implementation and all functional/static gates are green. The asset generator produced no + generated-file drift; a final post-commit clean status remains to capture before handoff. +- Orchestrator review found that bookkeeping failures could still abort dispatch and that this PR + cannot bootstrap the trusted-base module on its own ready event. The labeled evaluation path is + orchestrator-owned; the review-fix slice makes checkout/transition failures non-blocking and + attributed without weakening the trusted-base boundary. +- Review-fix commit `7170d574b3` is pushed. Gates are green: 66 script tests plus scoped + check/lint/format, YAML parsing, and post-commit asset generation with an empty working tree. +- The PR body and S3 phase comment state the box-1 interpretation and bootstrap limitation. PR + #1567 remains draft with exactly `status:impl` and milestone `0.0.6`. +- Run `31598386001` showed the hidden event-history dependency: dispatch ran after the non-fatal + transition failure, then failed because no `status:impl-eval` labeled-event generation existed. + The owner-directed next landing removes checkout/import and transcribes the tested cleanup inline; + the helper and unit tests remain. +- Self-contained implementation commit `d7ea38f1cd` is pushed. All six local gates are green: 67 + script tests, scoped check/lint/format, YAML parsing, and post-commit asset generation followed by + an empty working-tree proof. + +## Locked boundaries + +Only the phase-eval workflow, `.github/scripts/`, and this slice directory may change. Do not alter +dispatch deduplication, triggers, conditions, model/trusted-base logic, #1564, or PR #1541. Do not +merge or mark the PR ready. + +## Next action + +Update the PR body/evidence and phase comment, then stop. The orchestrator owns the ready flip and +automatic DeepSeek retry; this agent must not trigger or merge. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/drift.md new file mode 100644 index 0000000000..b31baa606f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/drift.md @@ -0,0 +1,23 @@ +# PR-F #1566 Drift Log + +## Entries + +- 2026-08-12: The orchestrator-provided first bootstrap commit contained `implement.md` only. + Completed the mandatory harness artifact set in an immediate bootstrap follow-up before tests or + implementation; no product scope changed. +- 2026-08-12: Orchestrator review identified a bootstrap limitation and widened the reliability + invariant from one known 404 to all status-bookkeeping failures. `phase-eval-status.mjs` is absent + on `origin/main`, so PR #1567 cannot import it from the trusted base during its own ready event; + the orchestrator will evaluate this PR through the labeled path. The durable fix keeps trusted + base execution and makes checkout/transition failures attributed but non-blocking for dispatch. + The new independence test statically validates named workflow step contracts and dependencies; + it cannot simulate GitHub Actions runner status semantics locally, so its evidence is policy + structure plus YAML parsing rather than an end-to-end Actions execution. +- 2026-08-12: Run `31598386001` corrected the prior interpretation. Dispatch was conditionally + eligible and did run, but its data dependency on a `status:impl-eval` labeled-event generation + made successful dispatch impossible after transition failure. The static policy test's recorded + limitation was decisive; it is retained but no longer cited as end-to-end independence evidence. + Owner directed a self-contained first landing: inline the tested cleanup in the workflow and keep + the helper as its independently tested contract. Importing the helper is deferred to a follow-up + only after this PR merges and the helper is reachable from trusted `main`; no PR-head fallback is + permitted. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/worklog.md new file mode 100644 index 0000000000..cd1f7ca60b --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-f-1566/worklog.md @@ -0,0 +1,155 @@ +# PR-F #1566 Worklog + +## Identity + +- Worktree: `/home/codex/repos/ns006-labelrace` +- Branch: `fix/1566-phase-eval-label-race` +- Base: `e67c1ba13` +- Implementation route: Codex · GPT-5.6 Sol · low +- Supervising orchestrator: Claude · Opus 5 · high, `/home/codex/repos/netscript-006-internals` + +## Plan gate + +PLAN-EVAL: N/A. This is a small deterministic automation fix with the defect, design, scope, +acceptance criteria, and required gates fully specified in issue #1566 and `implement.md`. + +## Design + +- Public surface: `.github/scripts/` exports a pure label-transition decision and a thin injected + GitHub-operation caller; the workflow imports and invokes the caller. +- Domain vocabulary: live issue-label names, the `status:` prefix, the terminal + `status:impl-eval` label, and the missing-label REST error classification. +- Ports: injected `listLabelsOnIssue`, `removeLabel`, and `addLabels` operations provide the only + external seam used by tests and the workflow caller. +- Constants: the status prefix and terminal label are named module constants. +- Commit slices: + 1. Bootstrap the tracked slice artifacts and draft PR. + 2. Add RED regression tests for the removal race, narrow tolerance, generation dedup, and terminal + single-status state. + 3. Implement live-label cleanup and workflow integration, then run the six required gates. +- Deferred scope: generation dispatch logic, workflow triggers/conditions, model resolution, + trusted-base resolution, #1564 range computation, and PR #1541 are unchanged. +- Contributor path: extend the decision/caller module and its adjacent test file; keep workflow + inline code limited to client adaptation and invocation. + +## Progress + +- Bootstrap: complete; draft PR #1567 opened with required metadata and six indexed acceptance + mappings. +- Tests: RED regression suite added. Before implementation it fails because the extracted + `phase-eval-status.ts` production module does not yet exist; the named race and narrow-tolerance + assertions define the required caller contract. Generation dedup is guarded structurally in the + unchanged workflow script. +- Implementation: complete and pushed through the orchestrator review fix. +- Gates: complete, including post-commit asset-barrel generation and empty-status proof. +- IMPL-EVAL: owned by the separate orchestrator/evaluator transition; this agent leaves the PR draft. + +## RED evidence + +- `deno test --allow-read .github/scripts/phase-eval-status.test.ts` — exit 1 before the production + module exists (`TS2307 Cannot find module .github/scripts/phase-eval-status.ts`). + +## Implementation + +- Added `phase-eval-status.mjs`: pure transition decision plus an injected operation caller. +- The caller reads live labels, removes only live `status:` labels, tolerates only an Octokit-style + `404` with response message `Label does not exist`, and adds `status:impl-eval` once. +- The workflow checks out the live protected base ref with credentials disabled, imports that + trusted module, and adapts the existing GitHub client operations. Dispatch/dedup code is unchanged. +- Targeted regression suite after implementation: 5 passed, 0 failed. + +## Gate evidence + +| Gate | Result | +| --- | --- | +| Script tests | `deno test --allow-read --allow-env --allow-write --allow-run .github/scripts/` — exit 0; 65 passed, 0 failed | +| Scoped type-check | `deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Scoped lint | `deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Scoped format | Initial exit 1 on the new test import layout; formatted only the owned test/module, rerun exit 0; 6 files, 0 findings | +| Asset barrel | `deno task gen:assets-barrel` — exit 0; no generated file appeared in status. Final clean-status proof runs after the implementation commit. | +| Workflow YAML | `deno eval --no-lock` with `jsr:@std/yaml@^1.0.10` parsed `openhands-phase-eval.yml` — exit 0, `YAML_PARSE_OK` | + +## Reconcile + +- Slice S1: issue #1566 remained open; PR #1567 carries the sole closing keyword, six indexed + evidence entries, and exactly one lifecycle label (`status:impl`). No new comments changed scope. +- Slice S2: no issue/PR feedback required readjustment. The PR remains draft for orchestrator-owned + IMPL-EVAL; no skip, ready-merge, or impl-eval label was applied. + +## Orchestrator review fix + +- Finding 1 confirmed: because `phase-eval-status.mjs` is not yet on `main`, this PR's own + ready-for-review event cannot import the trusted-base module. The orchestrator will use the + existing labeled path for this PR's evaluation; this implementation does not trigger it. +- Finding 2 accepted: evaluator dispatch is the primary work; status mutation and its trusted + checkout are bookkeeping. Both bookkeeping steps now use `continue-on-error`, while the dispatch + step explicitly depends only on a successful chain-token check and `!cancelled()`—not on checkout + or transition outcomes. This preserves a hard failure when the required PAT is absent. +- The transition catches its error only to publish a `failure_reason` output, then rethrows so the + step retains a truthful failure outcome. A following attributed summary step records actor, PR, + head, checkout outcome, and reason before dispatch proceeds. +- Static regression coverage extracts the named workflow step blocks and asserts the non-blocking + edges, diagnostic fields, trusted-base/credential boundary, and absence of bookkeeping outcome + dependencies from dispatch. This proves the declared workflow policy; it is not a GitHub runner + simulation. +- Acceptance reading: live issue #1566 box 1 remains truthful. Its specific concurrent-removal race + is narrowly tolerated inside the caller, so that transition completes normally and applies + `status:impl-eval` exactly once. Other errors still fail the transition step truthfully but no + longer suppress evaluator dispatch. + +### Review-fix gate evidence + +| Gate | Result | +| --- | --- | +| Script tests | `deno test --allow-read --allow-env --allow-write --allow-run .github/scripts/` — exit 0; 66 passed, 0 failed | +| Scoped type-check | `deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Scoped lint | `deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Scoped format | `deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Asset barrel | `deno task gen:assets-barrel` — exit 0; after review-fix commit `7170d574b3`, `git status --porcelain` was empty with exit 0 | +| Workflow YAML | `deno eval --no-lock` with `jsr:@std/yaml@^1.0.10` — exit 0, `YAML_PARSE_OK` | + +- Review-fix reconcile: live issue wording supports box 1 without amendment; PR #1567 remains draft + with exactly `status:impl`. No trigger, condition, model, trusted-base lookup, #1541, or #1564 + scope changed. +- Review-fix implementation commit: `7170d574b3`; pushed with explicit refspec. The orchestrator + retains the labeled IMPL-EVAL transition and merge authority. + +## Evaluator-run correction and self-contained landing + +- Run `31598386001` disproved the prior end-to-end independence claim. `continue-on-error` worked: + the failed transition remained legible, its attributed diagnostic ran, and the dispatch step + started. Dispatch then failed because it requires a `status:impl-eval` labeled-event generation, + which cannot exist when the transition does not apply the label. The dependency is through + GitHub event history, not the step's declared `if:` condition. +- Owner-directed design: remove the checkout/import bootstrap and perform the cleanup inline in the + trusted `github-script` step. The inline code paginates live labels, removes only live `status:` + labels, tolerates only status 404 with exact message `Label does not exist`, rethrows everything + else, and adds only `status:impl-eval`. +- `.github/scripts/phase-eval-status.mjs` remains the independently unit-tested behavioral contract. + The workflow currently carries a transcription rather than importing it because this first + landing must be self-contained before the helper exists on trusted `main`. +- The existing workflow-policy test is retained with its precise scope: it proves the failed + transition is attributed and the dispatch step remains eligible under its declared conditions. + It does not prove the event-history generation dependency is satisfied. A separate explicitly + string-based parity assertion checks that the inline transcription and helper use the same exact + missing-label message and terminal label, including their comparison/addition sites. +- `continue-on-error` and the attributed failure summary remain. They do not make label generation + optional; they make future failures observable and allow the dispatch step to expose its own + generation precondition instead of being skipped. + +### Self-contained landing gate evidence + +| Gate | Result | +| --- | --- | +| Script tests | `deno test --allow-read --allow-env --allow-write --allow-run .github/scripts/` — exit 0; 67 passed, 0 failed | +| Scoped type-check | `deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Scoped lint | `deno run --allow-read --allow-run .llm/tools/run-deno-lint.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Scoped format | `deno run --allow-read --allow-run .llm/tools/run-deno-fmt.ts --root .github/scripts --ext ts` — exit 0; 6 files, 0 findings | +| Asset barrel | `deno task gen:assets-barrel` — exit 0; after implementation commit `d7ea38f1cd`, `git status --porcelain` was empty with exit 0 | +| Workflow YAML | `deno eval --no-lock` with `jsr:@std/yaml@^1.0.10` — exit 0, `YAML_PARSE_OK` | + +- Self-contained landing reconcile: PR #1567 is draft with exactly `status:impl`; the failed run + facts changed the design but not issue #1566's six-box acceptance mapping. No manual OpenHands + trigger, ready transition, waiver label, merge, #1541 action, or out-of-scope change occurred. +- Self-contained implementation commit: `d7ea38f1cd`; pushed with explicit refspec. The orchestrator + owns the ready flip and automatic DeepSeek retry. diff --git a/.llm/tools/fitness/check-doctrine_test.ts b/.llm/tools/fitness/check-doctrine_test.ts new file mode 100644 index 0000000000..b05d9b1df2 --- /dev/null +++ b/.llm/tools/fitness/check-doctrine_test.ts @@ -0,0 +1,72 @@ +import { assertEquals, assertStringIncludes } from '@std/assert'; +import { join } from '@std/path'; +import { discoverDoctrineRoots } from './check-doctrine.ts'; + +async function expectedDoctrineRoots(repoRoot: string): Promise<string[]> { + const roots: string[] = []; + for (const parent of ['packages', 'plugins']) { + for await (const entry of Deno.readDir(join(repoRoot, parent))) { + if (!entry.isDirectory) continue; + try { + const config = JSON.parse( + await Deno.readTextFile(join(repoRoot, parent, entry.name, 'deno.json')), + ) as { name?: string }; + if (config.name) roots.push(`${parent}/${entry.name}`); + } catch (error) { + if (!(error instanceof Deno.errors.NotFound)) throw error; + } + } + } + return roots.sort(); +} + +Deno.test('doctrine root discovery equals the independently enumerated publishable top-level units', async () => { + const repoRoot = Deno.cwd(); + const expected = await expectedDoctrineRoots(repoRoot); + const actual = await discoverDoctrineRoots(repoRoot); + + assertEquals(expected.length, 36); + assertEquals(actual, expected); + assertEquals(actual.includes('packages/plugin-streams-core'), true); + assertEquals(actual.includes('packages/cli/e2e'), false); +}); + +Deno.test('A14 distinguishes imported, locally-bound, and unresolved test identifiers', async () => { + const fixtureRoot = await Deno.makeTempDir(); + const tool = join(Deno.cwd(), '.llm/tools/fitness/check-doctrine.ts'); + const cases = [ + { + name: 'imported', + source: + "import { describe, it } from '@std/testing/bdd';\ndescribe('suite', () => it('works', () => {}));\n", + code: 0, + }, + { + name: 'locally-bound', + source: "const describe = (value: string) => value;\ndescribe('local helper');\n", + code: 0, + }, + { + name: 'unresolved', + source: "describe('bare global', () => {});\n", + code: 1, + }, + ] as const; + + for (const fixture of cases) { + const root = join(fixtureRoot, fixture.name); + await Deno.mkdir(join(root, 'tests'), { recursive: true }); + await Deno.writeTextFile(join(root, 'mod.ts'), '/** @module */\nexport const value = true;\n'); + await Deno.writeTextFile(join(root, 'tests', 'origin_test.ts'), fixture.source); + const output = await new Deno.Command(Deno.execPath(), { + args: ['run', '--allow-read', tool, '--root', root], + }).output(); + assertEquals(output.code, fixture.code, fixture.name); + const stdout = new TextDecoder().decode(output.stdout); + if (fixture.name === 'unresolved') { + assertStringIncludes(stdout, "FAIL A14: unresolved Jest/Vitest global 'describe'"); + } else { + assertEquals(stdout.includes('FAIL A14'), false, fixture.name); + } + } +}); diff --git a/.llm/tools/quality/changed-source-files.ts b/.llm/tools/quality/changed-source-files.ts new file mode 100644 index 0000000000..fbef47198c --- /dev/null +++ b/.llm/tools/quality/changed-source-files.ts @@ -0,0 +1,39 @@ +#!/usr/bin/env -S deno run --allow-run +/** Source paths whose changes belong to the pull-request quality scan. */ +export async function collectChangedSourceFiles( + repoRoot: string, + base: string, + head: string, +): Promise<string[]> { + const output = await new Deno.Command('git', { + cwd: repoRoot, + args: [ + 'diff', + '--name-only', + '--diff-filter=ACMR', + `${base}...${head}`, + '--', + 'packages', + 'plugins', + '.llm/tools', + ], + }).output(); + if (!output.success) { + throw new Error(new TextDecoder().decode(output.stderr).trim()); + } + return new TextDecoder().decode(output.stdout).split(/\r?\n/).filter(Boolean).sort(); +} + +if (import.meta.main) { + const [base, head] = Deno.args; + if (!base || !head) { + console.error('usage: changed-source-files.ts <base> <head>'); + Deno.exit(2); + } + const files = await collectChangedSourceFiles(Deno.cwd(), base, head); + if (files.length === 0) { + console.error('not scanned: no changed source files matched packages, plugins, or .llm/tools'); + Deno.exit(2); + } + console.log(files.join('\n')); +} diff --git a/.llm/tools/quality/changed-source-files_test.ts b/.llm/tools/quality/changed-source-files_test.ts new file mode 100644 index 0000000000..671d3cf112 --- /dev/null +++ b/.llm/tools/quality/changed-source-files_test.ts @@ -0,0 +1,89 @@ +import { assertEquals, assertStringIncludes } from '@std/assert'; +import { join } from '@std/path'; +import { collectChangedSourceFiles } from './changed-source-files.ts'; + +async function git(cwd: string, ...args: string[]): Promise<string> { + const output = await new Deno.Command('git', { cwd, args }).output(); + if (!output.success) throw new Error(new TextDecoder().decode(output.stderr)); + return new TextDecoder().decode(output.stdout).trim(); +} + +async function write(root: string, path: string, text: string): Promise<void> { + const target = join(root, path); + await Deno.mkdir(join(target, '..'), { recursive: true }); + await Deno.writeTextFile(target, text); +} + +Deno.test('changed-source selector includes .llm/tools-only diffs and reports an empty set', async () => { + const root = await Deno.makeTempDir(); + await git(root, 'init', '-q'); + await git(root, 'config', 'user.email', 'fixture@example.test'); + await git(root, 'config', 'user.name', 'Fixture'); + await write(root, 'README.md', 'base\n'); + await git(root, 'add', '.'); + await git(root, 'commit', '-qm', 'base'); + const base = await git(root, 'rev-parse', 'HEAD'); + + await write(root, '.llm/tools/quality/new-rule.ts', 'const unsafe: any = 1;\n'); + await git(root, 'add', '.'); + await git(root, 'commit', '-qm', 'tool change'); + const head = await git(root, 'rev-parse', 'HEAD'); + + assertEquals(await collectChangedSourceFiles(root, base, head), [ + '.llm/tools/quality/new-rule.ts', + ]); + assertEquals(await collectChangedSourceFiles(root, head, head), []); + + const empty = await new Deno.Command(Deno.execPath(), { + cwd: root, + args: [ + 'run', + '--allow-run', + join(Deno.cwd(), '.llm/tools/quality/changed-source-files.ts'), + head, + head, + ], + }).output(); + assertEquals(empty.code, 2); + assertEquals( + new TextDecoder().decode(empty.stderr).trim(), + 'not scanned: no changed source files matched packages, plugins, or .llm/tools', + ); +}); + +Deno.test('changed-source selector uses merge-base and excludes foreign merged files', async () => { + const root = await Deno.makeTempDir(); + await git(root, 'init', '-q'); + await git(root, 'config', 'user.email', 'fixture@example.test'); + await git(root, 'config', 'user.name', 'Fixture'); + await write(root, 'README.md', 'base\n'); + await git(root, 'add', '.'); + await git(root, 'commit', '-qm', 'common base'); + const common = await git(root, 'rev-parse', 'HEAD'); + + await git(root, 'switch', '-qc', 'feature', common); + await write(root, '.llm/tools/quality/owned.ts', 'export const owned = true;\n'); + await git(root, 'add', '.'); + await git(root, 'commit', '-qm', 'owned change'); + const head = await git(root, 'rev-parse', 'HEAD'); + + await git(root, 'switch', '-q', 'master'); + await write(root, 'packages/foreign/mod.ts', 'export const foreign = true;\n'); + await git(root, 'add', '.'); + await git(root, 'commit', '-qm', 'foreign merged change'); + const recordedBase = await git(root, 'rev-parse', 'HEAD'); + + assertEquals(await collectChangedSourceFiles(root, recordedBase, head), [ + '.llm/tools/quality/owned.ts', + ]); +}); + +Deno.test('code-quality workflow executes the selector for every .llm/tools change', async () => { + const workflow = await Deno.readTextFile('.github/workflows/code-quality.yml'); + assertStringIncludes(workflow, "- '.llm/tools/**'"); + assertStringIncludes( + workflow, + 'changed_files=$(deno run --allow-run .llm/tools/quality/changed-source-files.ts', + ); + assertStringIncludes(workflow, 'deno task quality:scan --pretty "${args[@]}"'); +}); diff --git a/docs/site/reference/plugin-sagas-core/index.md b/docs/site/reference/plugin-sagas-core/index.md new file mode 100644 index 0000000000..55a3890f1f --- /dev/null +++ b/docs/site/reference/plugin-sagas-core/index.md @@ -0,0 +1,156 @@ +--- +layout: layouts/base.vto +title: "@netscript/plugin-sagas-core" +--- + +# `@netscript/plugin-sagas-core` + +Saga DSL, runtime ports, adapters, telemetry, config, and testing primitives for NetScript sagas +plugins. This page is written against the package's public surface reported by `deno doc`. +For the full index of packages and plugins return to the [reference overview](/reference/). + +Sagas are the honest answer to distributed transactions: a sequence of steps, each with a +compensation, driven by messages that may arrive twice or out of order. `defineSaga` builds a +frozen, typed definition — state, handlers, compensations, signals, queries — and the runtime drives +it through explicit ports for storage, transport, clock, and idempotency. Nothing is global: +applications inject their own durability, and tests inject deterministic in-memory doubles. + +This is the core that the deployable [`@netscript/plugin-sagas`](/reference/sagas/) plugin binds to a +NetScript host. Use it directly for custom hosts, libraries, and tests. + +## Entrypoints + +The package publishes nineteen entrypoints. The root path carries the userland DSL; the remaining +subpaths expose the layers a host, adapter author, or test harness composes. + +| Export specifier | Module | Exports | Purpose | +| --- | --- | --- | --- | +| `@netscript/plugin-sagas-core` | `./mod.ts` | 41 | The userland saga DSL — `defineSaga`, the cascaded-message constructors, signals, queries, and the definition types they produce (documented below). | +| `@netscript/plugin-sagas-core/builders` | `./src/builders/mod.ts` | 27 | The builder layer behind the DSL, for tooling that constructs definitions programmatically. | +| `@netscript/plugin-sagas-core/domain` | `./src/domain/mod.ts` | 44 | Saga domain vocabulary and policy defaults (`DEFAULT_RETRY_POLICY`, `DEFAULT_IDEMPOTENCY_WINDOW_MS`, `DEFAULT_RETRY_MAX_ATTEMPTS`). | +| `@netscript/plugin-sagas-core/ports` | `./src/ports/mod.ts` | 61 | The port interfaces the runtime depends on — store, bus, transport, clock, idempotency, telemetry. | +| `@netscript/plugin-sagas-core/runtime` | `./src/runtime/mod.ts` | 83 | The engine: `createSagaRuntime`, `createSagaEngine`, `createSagaCompensator`, `createSagaScheduler`, and the idempotency-key helpers. | +| `@netscript/plugin-sagas-core/adapters` | `./src/adapters/mod.ts` | 68 | Concrete port adapters, including `createSagaBusBridge`. | +| `@netscript/plugin-sagas-core/transports` | `./src/transports/mod.ts` | 49 | Saga bus transports (`createNetScriptRedisTransport`, `createGarnetListTransport`) with their message and delayed-entry codecs. | +| `@netscript/plugin-sagas-core/stores` | `./src/stores/mod.ts` | 56 | KV-backed instance and applied-key stores, `openSagaRuntimeKv`, and `resolveSagaStoreBackend`. | +| `@netscript/plugin-sagas-core/middleware` | `./src/middleware/mod.ts` | 30 | Host middleware — `createSagaMiddleware`, `createSSEEventsMiddleware`, `emitSagaEvent`. | +| `@netscript/plugin-sagas-core/integration/workers` | `./src/integration/workers/mod.ts` | 14 | Explicit workers-port helpers (`triggerJob`, `triggerTask`, `createWorkerTriggers`) that dispatch work **outside** synchronous saga handlers. | +| `@netscript/plugin-sagas-core/integration/publisher` | `./src/integration/publisher/mod.ts` | 10 | Publisher port contracts for submitting saga messages from plugin surfaces. | +| `@netscript/plugin-sagas-core/telemetry` | `./src/telemetry/mod.ts` | 43 | Telemetry attributes and instrumentation helpers, including an OpenTelemetry tracer factory. | +| `@netscript/plugin-sagas-core/config` | `./src/config/mod.ts` | 24 | `defineSagaConfig` and the saga runtime configuration schemas. | +| `@netscript/plugin-sagas-core/contracts/v1` | `./src/contracts/v1/mod.ts` | 29 | Version 1 saga API schemas and contract route types (`sagasContract`, `sagasContractV1`). | +| `@netscript/plugin-sagas-core/streams` | `./src/streams/mod.ts` | 17 | Durable stream schemas for projected saga instance records (`sagasStreamSchema`). | +| `@netscript/plugin-sagas-core/presets` | `./src/presets/mod.ts` | 9 | Preset composition helpers — `startSagas`, `startSagaHandlers`. | +| `@netscript/plugin-sagas-core/abstracts` | `./src/abstracts/mod.ts` | 58 | Abstract runtime contracts and reserved extension-point base classes. | +| `@netscript/plugin-sagas-core/testing` | `./src/testing/mod.ts` | 59 | `createTestSagaRuntime` plus in-memory bus and store doubles for deterministic verification. | +| `@netscript/plugin-sagas-core/agent` | `./src/agent/mod.ts` | 2 | `defineAgent` — the agent-shaped builder over the same saga definition. | + +Export counts are the symbol counts `deno doc` reports for each entrypoint; subpaths overlap where a +type is re-exported through more than one layer. + +## Root surface (`@netscript/plugin-sagas-core`) + +### Defining a saga + +| Symbol | Kind | Description | +| --- | --- | --- | +| `defineSaga` | function | Start a userland saga definition chain. | +| `SagaBuilder` | interface | The userland fluent saga builder. | +| `SagaBuilderPhase` | type alias | Typestate phase for the userland saga builder. | +| `SagaDefinition` | type alias | Frozen saga definition produced by the fluent DSL. | +| `SagaState` | type alias | Base state shape accepted by saga definitions. | +| `SagaContext` | type alias | Handler context passed to pure saga projections. | +| `SagaHandler` | type alias | Synchronous saga handler that returns cascaded messages. | +| `SagaEvent` | type alias | Event shape inferred by `defineSaga().on(type, handler)`. | +| `SagaMessage` | type alias | Base event or command delivered to a saga handler. | + +### Cascaded messages + +A handler is a pure projection: it returns cascaded messages rather than performing effects. These +constructors are the only side-effect ledger a handler produces. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `send` | function | Create a cascade that republishes an internal saga message onto the saga bus. | +| `schedule` | function | Create a cascaded scheduled message. | +| `sagaComplete` | function | Create a terminal saga completion message. | +| `sagaFail` | function | Create a terminal saga failure message. | +| `sagaCompensate` | function | Create a cascaded compensation message. | +| `spawn` | function | **Rejects** an unsupported child-saga spawn request — spawn cascades are not implemented. | +| `CASCADED_MESSAGE_KINDS` | variable | Cascaded message kinds emitted by saga handlers. | +| `CascadedMessage` | type alias | Message emitted by a saga handler as its only side-effect ledger. | +| `CascadedMessageKind` | type alias | Cascaded message discriminator. | +| `CascadedMessageOptions` | type alias | Common options accepted by cascaded message constructors. | +| `CascadedMessageTarget` | type alias | Cascaded message target for jobs, sagas, or arbitrary runtime adapters. | +| `SendOptions` | type alias | Options for republishing an internal saga message onto the saga bus. | +| `SagaScheduleDelay` | type alias | Delay accepted by the `schedule()` cascaded-message constructor. | +| `SpawnOptions` | type alias | Options reserved for the unsupported `spawn()` cascade. | + +### Signals and queries + +| Symbol | Kind | Description | +| --- | --- | --- | +| `defineSignal` | function | Define a signal that can be sent to a running saga instance. | +| `defineQuery` | function | Define a synchronous read-only query for a running saga instance. | +| `SignalDefinition` | type alias | Signal definition reserved by the public DSL. | +| `QueryDefinition` | type alias | Query definition reserved by the public DSL. | +| `SagaSignalHandler` | type alias | Signal handler reserved by the userland saga DSL. | +| `SagaQueryHandler` | type alias | Synchronous query handler reserved by the userland saga DSL. | +| `SyncQueryResult` | type alias | Synchronous query result accepted by `onQuery`; promises are rejected at type level. | + +### Correlation and identity + +| Symbol | Kind | Description | +| --- | --- | --- | +| `SagaCorrelation` | type alias | Extracts a correlation key from an incoming saga message. | +| `SagaCorrelationKey` | type alias | Branded correlation key used to route messages to saga instances. | +| `SagaCorrelationRule` | type alias | Named correlation rule stored on a saga definition. | +| `SagaId` | type alias | Branded saga definition identifier. | +| `SagaInstanceId` | type alias | Branded saga instance identifier. | +| `SagaMessageId` | type alias | Branded message identifier for runtime and diagnostics records. | + +### Policies + +| Symbol | Kind | Description | +| --- | --- | --- | +| `RetryPolicy` | type alias | Retry policy for saga handlers and cascaded messages. | +| `SagaConcurrencyOptions` | type alias | Concurrency options accepted by the saga builder. | +| `SagaConcurrencyPolicy` | type alias | Concurrency policy for a saga definition. | +| `SAGA_DURABILITY_TIERS` | variable | Durability tiers supported by saga definitions. | +| `SagaDurabilityTier` | type alias | Saga durability tier. | + +## Handlers are synchronous + +A saga handler is synchronous and returns cascaded messages; it does not `await`. That is what makes +replay deterministic and what keeps the compensation path a pure function of the transcript. Work +that must happen outside the handler — enqueueing a worker job, calling a service — is expressed as a +cascaded message the runtime dispatches, or through the explicit +`@netscript/plugin-sagas-core/integration/workers` helpers. + +`spawn()` is present in the surface but **rejects**: child-saga spawn cascades are unsupported, and +calling it raises rather than silently succeeding. Its return type is `never`, so a handler that +returns `spawn(...)` does not type-check into the cascade union by accident. + +## Composing a runtime + +`@netscript/plugin-sagas-core/runtime` exposes `createSagaRuntime`, which takes explicit ports rather +than reaching for globals. The store, transport, clock, and idempotency edges are all injected, so +the same definitions run against Redis or Garnet in production +(`@netscript/plugin-sagas-core/transports`) and against in-memory doubles in tests +(`@netscript/plugin-sagas-core/testing`). + +`@netscript/plugin-sagas-core/presets` collapses the common case: `startSagas` composes a runtime and +starts it, and `startSagaHandlers` binds a definition set to an already-composed runtime. + +## Related pages + +- [`@netscript/plugin-sagas`](/reference/sagas/) — the deployable plugin that binds this core to a + NetScript host. +- [`@netscript/plugin-workers-core`](/reference/plugin-workers-core/) — the worker primitives the + `integration/workers` helpers dispatch to. +- [`@netscript/plugin-streams-core`](/reference/plugin-streams-core/) — the producer behind the + projected instance stream. + +--- + +Back to the [reference overview](/reference/). diff --git a/docs/site/reference/plugin-streams-core/index.md b/docs/site/reference/plugin-streams-core/index.md new file mode 100644 index 0000000000..d524d90d5c --- /dev/null +++ b/docs/site/reference/plugin-streams-core/index.md @@ -0,0 +1,180 @@ +--- +layout: layouts/base.vto +title: "@netscript/plugin-streams-core" +--- + +# `@netscript/plugin-streams-core` + +Schema, producer, config, telemetry, testing, and diagnostics primitives for NetScript streams. This +page is written against the package's public surface reported by `deno doc`. For the full +index of packages and plugins return to the [reference overview](/reference/). + +Publishing change events sounds trivial until it has to be safe: typed payloads, idempotent appends, +one producer per stream path, and a clean flush on shutdown. `defineStreamSchema` declares the +collections a stream carries with standard-schema validation and a configured primary key; +`createDurableStream` returns a path-singleton producer whose `upsert`/`delete` appends are +idempotent and auto-claimed; and the diagnostics helpers inspect a schema or resolve the stream +endpoint without opening a socket. + +This is the layer the deployable [`@netscript/plugin-streams`](/reference/streams/) service builds +on, and the layer the other NetScript plugins use when they project entities — executions, saga +instances, sessions — into durable topics. + +## Entrypoints + +| Export specifier | Module | Exports | Purpose | +| --- | --- | --- | --- | +| `@netscript/plugin-streams-core` | `./mod.ts` | 51 | Schema definition, the durable producer, endpoint resolution, diagnostics, and the v1 producer port vocabulary (documented below). | +| `@netscript/plugin-streams-core/sse` | `./src/sse/mod.ts` | 33 | The single versioned authority for the stream SSE wire contract: named-frame parsing, validated consumer outcomes, and replay state. | +| `@netscript/plugin-streams-core/telemetry` | `./src/telemetry/mod.ts` | 33 | Telemetry registration, span names, attribute keys, and the meter/counter/gauge ports used by reconnect metrics. | +| `@netscript/plugin-streams-core/testing` | `./src/testing/mod.ts` | 4 | An in-memory producer and a small schema fixture for tests that must not open network sockets. | + +Export counts are the symbol counts `deno doc` reports for each entrypoint; a few domain types are +re-exported through more than one subpath. + +## Root surface (`@netscript/plugin-streams-core`) + +### Schema definition + +| Symbol | Kind | Description | +| --- | --- | --- | +| `defineStreamSchema` | function | Define a type-safe durable stream schema. | +| `StreamStateDefinition` | type alias | Input map accepted by `defineStreamSchema`. | +| `StateSchema` | type alias | Schema map returned by `defineStreamSchema`. | +| `CollectionDefinition` | interface | A single collection definition inside a durable stream schema. | +| `CollectionEventHelpers` | interface | Helper methods attached to collections by `@durable-streams/state`. | +| `CollectionWithHelpers` | type alias | Collection definition after durable-streams helper methods are attached. | +| `StreamStandardSchema` | interface | Package-owned Standard Schema surface used by durable stream collections. | +| `StreamSchemaIssue` | interface | One validation issue returned by a Standard Schema validator. | +| `StreamSchemaValidationOptions` | interface | Validation options accepted by Standard Schema validators. | +| `StreamSchemaValidationResult` | type alias | Result returned by a Standard Schema validator. | + +### Producers + +| Symbol | Kind | Description | +| --- | --- | --- | +| `createDurableStream` | function | Create or reuse a compatible durable stream producer for one stream path. | +| `createServiceStreamProducer` | function | Create a durable stream producer from a backend Service. | +| `DurableStreamProducer` | class | Server-side writer for a named durable stream. | +| `DurableStreamProducerOptions` | interface | Options accepted by `DurableStreamProducer`. | +| `ServiceStreamProducerOptions` | type alias | Options accepted by `createServiceStreamProducer`. | + +`createDurableStream` is a **path singleton**: two calls for the same stream path with compatible +options return the same producer rather than opening a second writer. + +### Events + +| Symbol | Kind | Description | +| --- | --- | --- | +| `ChangeEvent` | interface | Entity change event emitted by durable stream producers. | +| `ControlEvent` | interface | Control event emitted by durable streams for non-entity lifecycle changes. | +| `StateEvent` | type alias | Durable stream event union. | +| `Operation` | type alias | State Protocol operation names supported by durable streams. | + +### Endpoint resolution and diagnostics + +| Symbol | Kind | Description | +| --- | --- | --- | +| `getStreamsUrl` | function | Resolve the base URL of the durable streams server. | +| `getStreamsAuth` | function | Resolve authentication headers for the durable streams server. | +| `buildStreamUrl` | function | Build the full stream URL for a NetScript stream path. | +| `inspectStreamTopic` | function | Inspect a stream schema and optional producer metadata. | +| `StreamTopicInspectionInput` | interface | Input accepted by `inspectStreamTopic`. | +| `StreamTopicInspectionReport` | interface | Diagnostic report returned by `inspectStreamTopic`. | + +`inspectStreamTopic` is a pure diagnostic: it reports on a schema and optional producer metadata +without connecting to the streams server. + +### Producer port contract (v1) + +The `*V1` names are the versioned port contract a producer implementation satisfies and a supervisor +consumes. They describe write acceptance, delivery outcome, reconnection, and buffering as data, +which is what lets the retry policy live outside the transport. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `StreamProducerPort` | interface | Port implemented by stream producers that publish State Protocol changes. | +| `StreamProducerTransportPort` | interface | Durable-stream protocol edge consumed by the producer supervisor. | +| `StreamProducerClockPort` | interface | Clock edge used by reconnect backoff. | +| `StreamProducerRandomPort` | interface | Randomness edge used only to jitter reconnect delays. | +| `StreamProducerIdentityV1` | interface | Exact producer identity retained across an append retry. | +| `StreamProducerConnectInputV1` | interface | Connection input for the durable stream transport edge. | +| `StreamProducerAppendInputV1` | interface | Append input retained byte-for-byte until acknowledgement. | +| `StreamProducerCloseInputV1` | interface | Terminal close input using the next sequence in the same producer epoch. | +| `StreamProducerAcknowledgementV1` | interface | Acknowledgement returned by an append or terminal close. | +| `StreamProducerStateSnapshotV1` | interface | Snapshot of producer readiness and buffered work. | +| `StreamProducerReadinessOptionsV1` | interface | Options for waiting on the next ready transition. | +| `StreamProducerBufferPolicyV1` | interface | Dual queue bounds for accepted durable stream writes. | +| `StreamProducerReconnectPolicyV1` | interface | Finite retry policy used for connection and delivery attempts. | +| `StreamProducerTransportFailureV1` | interface | Transport failure understood by the application supervisor. | +| `StreamProducerTransportFailureKindV1` | type alias | Stable failure categories translated by the durable-stream transport adapter. | +| `StreamProducerTransportResultV1` | type alias | Result of one transport operation without policy or retry decisions. | +| `StreamProducerLifecycleStateV1` | type alias | One legal lifecycle state for a reconnecting durable stream producer. | +| `STREAM_PRODUCER_LIFECYCLE_STATES_V1` | variable | Legal lifecycle states for a reconnecting durable stream producer. | +| `DEFAULT_STREAM_PRODUCER_BUFFER_POLICY_V1` | variable | Default bounded producer buffer policy. | +| `DEFAULT_STREAM_PRODUCER_RECONNECT_POLICY_V1` | variable | Default bounded reconnect policy. | + +### Write outcomes + +| Symbol | Kind | Description | +| --- | --- | --- | +| `StreamWriteContextV1` | interface | Per-write correlation and message identity accepted by stream producers. | +| `StreamWriteReceiptV1` | interface | Immediate acceptance plus eventual terminal outcome for one write. | +| `StreamWriteOutcomeV1` | type alias | Terminal outcome of one durable stream write. | +| `StreamWriteRejectionReasonV1` | type alias | Reasons a producer rejects a write before accepting it. | +| `StreamWriteCancellationReasonV1` | type alias | Reasons an accepted write is cancelled before its first delivery attempt. | +| `StreamWriteUnknownReasonV1` | type alias | Reasons an attempted write cannot be reported as delivered or rejected. | + +A write returns a **receipt**, not a boolean: acceptance is immediate, and the terminal outcome — +delivered, rejected, cancelled, or unknown — settles later. `unknown` is a distinct outcome rather +than a failure, because a producer that lost its connection mid-append cannot honestly report either +success or rejection. + +## The SSE contract (`@netscript/plugin-streams-core/sse`) + +The `./sse` subpath is the single runtime authority for NetScript's versioned stream SSE envelope. +It distinguishes the **wire** event names the durable-stream server emits from the **consumer** +event names a validated binding delivers, so a consumer never branches on an unvalidated frame. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `STREAM_SSE_CONTRACT_V1` | variable | Single runtime authority for NetScript's versioned stream SSE envelope. | +| `STREAM_SSE_PROTOCOL_VERSION_V1` | variable | Version identifier for the first NetScript stream SSE consumer contract. | +| `STREAM_SSE_WIRE_EVENT_NAMES_V1` | variable | Actual event names emitted by the durable-stream SSE wire protocol. | +| `STREAM_SSE_CONSUMER_EVENT_NAMES_V1` | variable | Validated outcomes delivered by the NetScript SSE consumer binding. | +| `parseStreamSseEventV1` | function | Parse a named SSE frame through the single v1 contract authority. | +| `bindStreamEventSourceV1` | function | Bind named `data` and `control` listeners using schema-validated v1 outcomes. | +| `createStreamSseReplayStateV1` | function | Create an empty or caller-seeded v1 replay snapshot. | +| `reduceStreamSseReplayStateV1` | function | Apply one valid frame while committing replay progress only on a control frame. | +| `StreamEventSourceV1` | interface | Minimal native `EventSource` surface used at the browser edge and in tests. | +| `StreamEventSourceBindingV1` | interface | Disposable browser binding and immutable replay snapshot accessor. | +| `StreamSseReplayStateV1` | interface | Replay snapshot consumed by reconnect policy without owning that policy. | +| `StreamSseOffsetV1` | type alias | A server-owned replay token — opaque, never parsed or incremented by a consumer. | + +Replay progress commits **only on a control frame**: `reduceStreamSseReplayStateV1` applies a valid +data frame without advancing the committed offset, so a consumer that reconnects mid-batch replays +from the last acknowledged control point rather than skipping the tail of an interrupted batch. +Errors are normalized into `StreamSseErrorPayloadV1` and never advance replay state. + +## Telemetry and testing + +`@netscript/plugin-streams-core/telemetry` exposes `streamsInstrumentation` — a registration object +matching the minimal contract NetScript telemetry hosts understand — plus `StreamSpanNames`, +`StreamAttributes`, and `StreamProducerMetricNames`. The meter, counter, gauge, tracer, and span +ports are declared here as interfaces, so a producer emits reconnect metrics without importing an +OpenTelemetry SDK. + +`@netscript/plugin-streams-core/testing` provides `MemoryStreamProducer` (records +`MemoryStreamEvent`s instead of opening a socket) and `createStreamTopicFixture` (a small schema with +one `execution` collection). Tests written against these run without network permissions. + +## Related pages + +- [`@netscript/plugin-streams`](/reference/streams/) — the deployable streams plugin and service. +- [`@netscript/plugin-workers-core`](/reference/plugin-workers-core/) and + [`@netscript/plugin-sagas-core`](/reference/plugin-sagas-core/) — both project their records into + durable topics through this producer. + +--- + +Back to the [reference overview](/reference/). diff --git a/docs/site/reference/plugin-triggers-core/index.md b/docs/site/reference/plugin-triggers-core/index.md new file mode 100644 index 0000000000..6c7b3ff344 --- /dev/null +++ b/docs/site/reference/plugin-triggers-core/index.md @@ -0,0 +1,259 @@ +--- +layout: layouts/base.vto +title: "@netscript/plugin-triggers-core" +--- + +# `@netscript/plugin-triggers-core` + +Trigger DSL, runtime ports, adapters, telemetry, config, and testing primitives for NetScript trigger +plugins. This page is written against the package's public surface reported by `deno doc`. For +the full index of packages and plugins return to the [reference overview](/reference/). + +The hard part of triggers is not receiving them — it is surviving them: duplicate webhooks, senders +that retry on a slow response, crashes between the acknowledgement and the work. `defineWebhook`, +`defineScheduledTrigger`, and `defineFileWatch` take the handler first and a frozen spec second; +ingress verifies and persists an event before responding `202`; and the processor applies +idempotency, retry policy, bounded concurrency, dead-lettering, and circuit-breaking around every +dispatch — all through explicit ports you can swap. + +This is the core that the deployable [`@netscript/plugin-triggers`](/reference/triggers/) plugin binds +to a NetScript host. Use it directly for custom hosts, libraries, and tests. + +## Entrypoints + +| Export specifier | Module | Exports | Purpose | +| --- | --- | --- | --- | +| `@netscript/plugin-triggers-core` | `./mod.ts` | 106 | The full public surface — builders, runtime factories, ports, and the event model (documented below). | +| `@netscript/plugin-triggers-core/public` | `./src/public/mod.ts` | 106 | The same curated public surface the root re-exports, for consumers that prefer to name it explicitly. | +| `@netscript/plugin-triggers-core/builders` | `./src/builders/mod.ts` | 50 | The three definition builders and the handler-action constructors (`enqueueJob`). | +| `@netscript/plugin-triggers-core/domain` | `./src/domain/mod.ts` | 69 | Trigger domain vocabulary and policy defaults (concurrency limit, idempotency TTL, circuit-breaker thresholds, backoff multiplier). | +| `@netscript/plugin-triggers-core/ports` | `./src/ports/mod.ts` | 80 | The port interfaces the runtime depends on — event store, DLQ, idempotency, enabled-state, scheduler, verifier, subscription. | +| `@netscript/plugin-triggers-core/runtime` | `./src/runtime/mod.ts` | 97 | The runtime factories — `createTriggerIngress`, `createTriggerProcessor`, `createManualDispatcher`, `createEventSubscription`, `defaultRetryPolicy`. | +| `@netscript/plugin-triggers-core/adapters` | `./src/adapters/mod.ts` | 28 | Concrete adapters: the cron scheduler adapter, the file-watcher port, and the HMAC-SHA256 webhook verifier. | +| `@netscript/plugin-triggers-core/stores` | `./src/stores/mod.ts` | 40 | KV-backed store implementations and `openTriggerRuntimeKv`. | +| `@netscript/plugin-triggers-core/config` | `./src/config/mod.ts` | 17 | `defineTriggers` and the trigger configuration schemas. | +| `@netscript/plugin-triggers-core/contracts/v1` | `./src/contracts/v1/mod.ts` | 35 | Version 1 trigger API schemas and contract route types (`triggersContract`, `triggersContractV1`). | +| `@netscript/plugin-triggers-core/telemetry` | `./src/telemetry/mod.ts` | 37 | `createTriggerInstrumentation`, span names, and attribute keys for trigger dispatch. | +| `@netscript/plugin-triggers-core/testing` | `./src/testing/mod.ts` | 79 | In-memory and KV store doubles for deterministic runtime verification. | + +Export counts are the symbol counts `deno doc` reports for each entrypoint; the layered subpaths +re-export shared vocabulary, so the counts overlap rather than sum. + +## Root surface (`@netscript/plugin-triggers-core`) + +### Defining triggers + +Each builder takes the **handler first** and an immutable spec second, and returns a frozen +definition the runtime walker discovers. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `defineWebhook` | function | Define a webhook trigger from a handler and static spec. | +| `WebhookHandler` | type alias | Webhook handler signature used by `defineWebhook`. | +| `WebhookSpec` | type alias | Webhook definition fields accepted by `defineWebhook`. | +| `WebhookDefinition` | type alias | Webhook trigger definition discovered by the runtime walker. | +| `defineScheduledTrigger` | function | Define a scheduled trigger from a handler and static cron spec. | +| `ScheduledTriggerHandler` | type alias | Scheduled trigger handler signature used by `defineScheduledTrigger`. | +| `DefineScheduledTriggerSpec` | type alias | Scheduled trigger definition fields accepted by `defineScheduledTrigger`. | +| `ScheduledTriggerSpec` | type alias | Static scheduled trigger spec consumed by scheduler ports and builders. | +| `ScheduledTriggerDefinition` | type alias | Scheduled trigger definition discovered by the runtime walker. | +| `defineFileWatch` | function | Define a file-watch trigger from a handler and static spec. | +| `FileWatchHandler` | type alias | File-watch handler signature used by `defineFileWatch`. | +| `FileWatchSpec` | type alias | File-watch definition fields accepted by `defineFileWatch`. | +| `FileWatchDefinition` | type alias | File-watch trigger definition discovered by the runtime walker. | +| `FileWatchLifecycle` | type alias | File lifecycle event names supported by file-watch triggers. | +| `FileWatchStabilityThreshold` | type alias | Stability threshold for network-filesystem tolerant file-watch triggers. | +| `CronExpression` | type alias | Cron expression accepted by scheduled trigger definitions. | + +### Handler context and actions + +A handler does not perform work directly; it returns **actions** the dispatcher carries out. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `TriggerContext` | type alias | Context passed to trigger handlers by the processor. | +| `TriggerHandler` | type alias | Handler invoked by the processor for a trigger event. | +| `TriggerActionResult` | type alias | Action result emitted by trigger handlers. | +| `TriggerActionDispatcher` | type alias | Dispatches actions emitted by trigger handlers. | +| `enqueueJob` | function | Create an action that enqueues a typed worker job from a trigger handler. | +| `EnqueueJobAction` | type alias | Action emitted when a trigger should enqueue a worker job. | +| `EnqueueJobOptions` | type alias | Options for dispatching a worker job from a trigger handler. | +| `DeferAction` | type alias | Action emitted when a trigger yields without holding a worker slot. | +| `JobDefinition` | type alias | Root-surface job definition derived from the thin public schema. | +| `JobId` | type alias | Branded worker job identifier. | + +### Definition union and kinds + +| Symbol | Kind | Description | +| --- | --- | --- | +| `TriggerDefinition` | type alias | Trigger definitions known by the Group F public surface. | +| `TriggerDefinitionBase` | type alias | Common immutable fields shared by trigger definitions. | +| `TriggerKind` | type alias | Open trigger discriminator. | +| `TriggerKnownKind` | type alias | Canonical known trigger kind. | +| `TRIGGER_KINDS` | variable | Canonical trigger kinds known by Group F. | +| `ProcessableTriggerDefinition` | type alias | Trigger definition accepted by runtime processor ports. | +| `RuntimeTriggerDefinition` | type alias | Trigger definitions implemented by the Group F runtime. | +| `RuntimeWebhookDefinition` | type alias | Webhook definition shape accepted by the trigger ingress runtime. | +| `TriggerId` | type alias | Branded trigger definition identifier. | +| `WebhookId` | type alias | Branded webhook definition identifier. | + +`TriggerKind` is deliberately **open** while `TriggerKnownKind` is closed: a host may carry a kind the +core does not implement, and only the known kinds narrow into `RuntimeTriggerDefinition`. + +### Payloads + +| Symbol | Kind | Description | +| --- | --- | --- | +| `TriggerPayload` | type alias | Payload union for known Group F trigger kinds. | +| `WebhookTriggerPayload` | type alias | HTTP payload captured by webhook ingress before processing. | +| `ScheduledTriggerPayload` | type alias | Scheduled payload emitted by a scheduler adapter. | +| `FileWatchTriggerPayload` | type alias | Filesystem payload captured by a file-watch adapter. | + +### Reserved kinds + +These are declared in the type surface but **not executed** by the current runtime. They exist so a +host can carry the shape without the core pretending to run it. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `ManualTriggerDefinition` | type alias | Reserved manual trigger definition for CLI/API fire paths. | +| `ManualTriggerPayload` | type alias | Reserved manual-fire payload for CLI and API dispatch. | +| `QueueTriggerDefinition` | type alias | Reserved queue-source trigger definition; runtime execution is deferred. | +| `QueueTriggerPayload` | type alias | Reserved queue-source payload; runtime execution is deferred. | +| `StreamTriggerDefinition` | type alias | Reserved stream-source trigger definition; runtime execution is deferred. | +| `StreamTriggerPayload` | type alias | Reserved stream-source payload; runtime execution is deferred. | + +### Ingress + +| Symbol | Kind | Description | +| --- | --- | --- | +| `createTriggerIngress` | function | Create an ack-then-process webhook ingress boundary. | +| `TriggerIngressPort` | interface | Fast ack-then-process ingress boundary. | +| `TriggerIngressOptions` | type alias | Options accepted by the trigger ingress composition root. | +| `TriggerIngressRequest` | type alias | Ingress request passed to trigger ingress adapters. | +| `TriggerIngressResponse` | type alias | Fast ack response returned by ingress adapters. | +| `TriggerIngressEventIdFactory` | type alias | Generates event ids for accepted ingress events. | + +Ingress **verifies the signature and persists the event before responding**. That ordering is the +contract: a slow handler never blocks the sender, and a crash after the acknowledgement replays from +the stored event rather than losing it. + +### Processor + +| Symbol | Kind | Description | +| --- | --- | --- | +| `createTriggerProcessor` | function | Create a trigger processor runtime from explicit dependencies. | +| `TriggerProcessor` | class | T1 trigger processor with idempotency, retry, concurrency, DLQ, and circuit breaker handling. | +| `TriggerProcessorPort` | interface | Processes unified trigger events through the T1 dispatch pipeline. | +| `TriggerProcessorOptions` | type alias | Options accepted by the trigger processor runtime. | +| `TriggerProcessorStopOptions` | type alias | Stop options for processor drain. | +| `TriggerProcessResult` | type alias | Result returned after processing a trigger event. | + +### Dispatch policies + +| Symbol | Kind | Description | +| --- | --- | --- | +| `TriggerRetryPolicy` | type alias | Retry policy applied by the trigger processor before DLQ handoff. | +| `TriggerConcurrencySpec` | type alias | Bounded dispatch concurrency for a trigger definition. | +| `TriggerDeduplicationSpec` | type alias | Event-boundary deduplication policy. | +| `TriggerCircuitBreakerSpec` | type alias | Circuit breaker policy for repeated trigger dispatch failures. | +| `TriggerDurabilityTier` | type alias | Trigger durability tier. | +| `TRIGGER_DURABILITY_TIERS` | variable | Durability tiers supported by trigger definitions. | + +### Events, stores, and idempotency + +| Symbol | Kind | Description | +| --- | --- | --- | +| `TriggerEvent` | type alias | Unified event envelope consumed by every trigger processor path. | +| `TriggerEventId` | type alias | Branded trigger event identifier. | +| `TriggerEventStatus` | type alias | Trigger event lifecycle status. | +| `TRIGGER_EVENT_STATUSES` | variable | Trigger event lifecycle statuses. | +| `TriggerEventStorePort` | interface | Persistent trigger event store boundary. | +| `TriggerEventListOptions` | type alias | Event store list filters. | +| `TriggerDlqPort` | interface | Dead-letter queue boundary for exhausted trigger events. | +| `TriggerDlqEntry` | type alias | Dead-letter entry recorded after trigger retry exhaustion. | +| `TriggerDlqListOptions` | type alias | Dead-letter list filters. | +| `TriggerIdempotencyPort` | interface | Event-boundary idempotency store with a TTL window. | +| `TriggerIdempotencyClaim` | type alias | Idempotency claim result. | +| `TriggerIdempotencyKeyInput` | type alias | Idempotency key resolution input. | +| `TriggerEnabledStatePort` | interface | Persistent enabled-state boundary for trigger enable/disable routes. | +| `TriggerEnabledStateOverride` | type alias | Stored enabled-state override for a trigger definition. | +| `createKvTriggerEnabledStateStore` | function | Create a KV-backed enabled-state store over the supplied adapter. | + +Every kind — webhook, scheduled, file-watch — converges on the same `TriggerEvent` envelope, which is +why one processor pipeline covers all of them. + +### Live event subscription + +| Symbol | Kind | Description | +| --- | --- | --- | +| `createEventSubscription` | function | Create a single-replica in-process trigger event subscription hub. | +| `TriggerEventSubscriptionPort` | interface | In-process live trigger event subscription boundary. | +| `TriggerEventSubscriptionFilter` | type alias | Subscription filter for live trigger event streams. | +| `TriggerEventSubscriptionMessage` | type alias | Message yielded by trigger event subscription ports. | +| `TriggerEventSubscriptionOptions` | type alias | Subscribe options for live trigger event streams. | +| `TriggerEventSubscriptionType` | type alias | Trigger lifecycle event emitted to live subscribers. | + +`createEventSubscription` is explicitly **single-replica** and in-process: it is a live-tail hub for +one host, not a distributed fan-out. + +### Scheduling + +| Symbol | Kind | Description | +| --- | --- | --- | +| `TriggerSchedulerPort` | interface | Scheduler boundary for scheduled trigger definitions. | +| `TriggerSchedulerStopOptions` | type alias | Scheduler stop options. | +| `ScheduledTriggerHandle` | type alias | Scheduled trigger handle returned by scheduler adapters. | +| `computeNextFireTimes` | function | Compute upcoming fire times for a 5-field scheduled trigger spec. | + +`computeNextFireTimes` is a pure preview over a **5-field** cron spec — it starts no scheduler and is +what the CLI and dashboards use to show "next runs". + +### Manual dispatch + +| Symbol | Kind | Description | +| --- | --- | --- | +| `createManualDispatcher` | function | Create a manual trigger dispatcher from explicit runtime ports. | +| `ManualDispatcher` | interface | Runtime port for explicit manual trigger dispatch. | +| `ManualDispatcherOptions` | type alias | Options accepted by the manual trigger dispatcher factory. | +| `ManualTriggerFireInput` | type alias | Manual trigger fire request consumed by the runtime dispatcher. | +| `ManualTriggerFireResponse` | type alias | Manual trigger fire response returned by the runtime dispatcher. | +| `ManualTriggerEventIdFactory` | type alias | Generates event ids for manual trigger fire events. | + +### Webhook verification and test delivery + +| Symbol | Kind | Description | +| --- | --- | --- | +| `WebhookVerifierPort` | interface | Verifies inbound webhook authenticity and extracts provider event ids. | +| `WebhookVerifierKind` | type alias | Webhook verifier selector declared by a webhook trigger. | +| `WebhookVerificationRequest` | type alias | Request shape passed to a webhook verifier adapter. | +| `WebhookVerificationResult` | type alias | Result returned by a webhook verifier adapter. | +| `createWebhookTestDelivery` | function | Create a webhook test-delivery helper over an ingress port. | +| `WebhookTestDelivery` | interface | Runtime helper for sending signed synthetic webhook test requests. | +| `WebhookTestDeliveryDefinition` | type alias | Webhook definition shape accepted by the test-delivery helper. | +| `WebhookTestDeliveryInput` | type alias | Webhook test delivery request consumed by the runtime helper. | +| `WebhookTestDeliveryOptions` | type alias | Options accepted by the webhook test-delivery helper factory. | +| `WebhookTestDeliveryResponse` | type alias | Webhook test delivery response mapped to the trigger fire contract shape. | + +A verifier extracts the **provider event id** as well as validating the signature; that id is what +lets deduplication recognize the sender's retry as the same event. + +`@netscript/plugin-triggers-core/adapters` ships `HmacSha256WebhookVerifier` as the default +implementation of `WebhookVerifierPort`. + +### Logging + +| Symbol | Kind | Description | +| --- | --- | --- | +| `LoggerPort` | interface | Structured logger boundary consumed by trigger runtime code. | + +## Related pages + +- [`@netscript/plugin-triggers`](/reference/triggers/) — the deployable plugin that binds this core to + a NetScript host. +- [`@netscript/plugin-workers-core`](/reference/plugin-workers-core/) — the job surface `enqueueJob` + hands work to. + +--- + +Back to the [reference overview](/reference/). diff --git a/docs/site/reference/plugin-workers-core/index.md b/docs/site/reference/plugin-workers-core/index.md new file mode 100644 index 0000000000..adcf5219bd --- /dev/null +++ b/docs/site/reference/plugin-workers-core/index.md @@ -0,0 +1,148 @@ +--- +layout: layouts/base.vto +title: "@netscript/plugin-workers-core" +--- + +# `@netscript/plugin-workers-core` + +Job, task, workflow, runtime, config, and testing primitives for NetScript workers plugins. This page +is written against the package's public surface reported by `deno doc`. For the full index of +packages and plugins return to the [reference overview](/reference/). + +Background-job definitions fail in two places: at definition time, when a job is missing an +execution target and nobody notices until production; and at runtime, when the executor and its +storage are welded together and untestable. This package attacks both. `defineJob`, `defineTask`, +and `defineWorkflow` are **typestate-gated** builders — `build()` only exists once an entrypoint or +handler is set, so an incomplete definition is a compile error rather than a runtime surprise. The +runtime composes from injected registry, worker, and storage ports with memory-backed defaults, so +the same definitions run in production and in permission-free tests. + +This is the core that the deployable [`@netscript/plugin-workers`](/reference/workers/) plugin binds +to a NetScript host. Use it directly for custom hosts, libraries, and tests. + +## Entrypoints + +The package publishes seventeen entrypoints. The root path carries the authoring surface; the +subpaths expose the runtime layers a host composes. + +| Export specifier | Module | Exports | Purpose | +| --- | --- | --- | --- | +| `@netscript/plugin-workers-core` | `./mod.ts` | 32 | The authoring surface — the three typestate builders, handler results and tools, schedule and permission presets, inspection, and the runtime entry points (documented below). | +| `@netscript/plugin-workers-core/builders` | `./src/builders/mod.ts` | 28 | The builder layer behind the DSL, including the builder-state types tooling needs. | +| `@netscript/plugin-workers-core/runtime` | `./src/runtime/mod.ts` | 132 | The full runtime: `createWorkersRuntime`, the in-process dispatcher and runner, execution records, and `resolveWorkerIdempotencyKey`. | +| `@netscript/plugin-workers-core/presets` | `./src/presets/mod.ts` | 28 | `startWorkers` and the runtime port shapes its default composition fills in. | +| `@netscript/plugin-workers-core/executor` | `./src/executor/mod.ts` | 33 | The multi-runtime task executor and its per-runtime adapters (Deno, .NET, `cmd`, generic executable) plus `runProcess`. | +| `@netscript/plugin-workers-core/workflow` | `./src/workflow/mod.ts` | 27 | The workflow builder, workflow events, workflow clock, and an in-memory workflow state store. | +| `@netscript/plugin-workers-core/registry` | `./src/registry/mod.ts` | 22 | KV-backed job and task registries with their filter and selector types. | +| `@netscript/plugin-workers-core/state` | `./src/state/mod.ts` | 9 | Execution state: records, statuses, trigger types, and the KV execution state store. | +| `@netscript/plugin-workers-core/stores` | `./src/stores/mod.ts` | 14 | The KV-backed worker idempotency store and its atomic KV primitives. | +| `@netscript/plugin-workers-core/streams` | `./src/streams/mod.ts` | 20 | Durable stream projection of executions and jobs (`workersStreamSchema`, `createWorkersStreamProducer`, the mutation hook). | +| `@netscript/plugin-workers-core/shutdown` | `./src/shutdown/mod.ts` | 5 | `ShutdownManager` and the resource/report types behind graceful drain. | +| `@netscript/plugin-workers-core/schemas` | `./src/domain/public-schema.ts` | 13 | Public structural schemas for worker definitions. | +| `@netscript/plugin-workers-core/contracts/v1` | `./src/contracts/v1/mod.ts` | 27 | Version 1 workers API schemas and contract route types (`workersContract`, `workersContractV1`). | +| `@netscript/plugin-workers-core/config` | `./src/config/mod.ts` | 32 | `defineWorkers`, `defineJobs`, and the queue-provider configuration schemas. | +| `@netscript/plugin-workers-core/telemetry` | `./src/telemetry/mod.ts` | 26 | Worker instrumentation abstractions and `applyWorkerInstrumentations`. | +| `@netscript/plugin-workers-core/abstracts` | `./src/abstracts/mod.ts` | 31 | Abstract runtime contracts and reserved extension-point base classes. | +| `@netscript/plugin-workers-core/testing` | `./src/testing/mod.ts` | 46 | `createTestWorkersRuntime` plus job, result, and execution-record fixtures. | + +Export counts are the symbol counts `deno doc` reports for each entrypoint; the layered subpaths +re-export shared vocabulary, so the counts overlap rather than sum. + +## Root surface (`@netscript/plugin-workers-core`) + +### Definition builders + +Each builder is typestate-gated: `build()` becomes available only after the definition is complete. + +| Symbol | Kind | Description | +| --- | --- | --- | +| `defineJob` | function | Start a worker job definition chain. | +| `JobBuilder` | interface | Root-surface job builder typestate API. | +| `JobDefinition` | type alias | Root-surface job definition derived from the thin public schema. | +| `defineJobHandler` | function | Define a worker job handler. | +| `JobHandlerContext` | type alias | Context passed to root-surface job handlers. | +| `JobResult` | type alias | Result returned by worker job handlers. | +| `JobId` | type alias | Branded worker job identifier. | +| `defineTask` | function | Start a worker task definition chain. | +| `TaskBuilder` | interface | Root-surface task builder typestate API. | +| `TaskDefinition` | type alias | Root-surface task definition derived from the thin public schema. | +| `TaskId` | type alias | Branded worker task identifier. | +| `defineWorkflow` | function | Start a worker workflow definition chain. | +| `WorkflowBuilder` | interface | Root-surface workflow builder typestate API. | +| `WorkflowDefinition` | type alias | Root-surface workflow definition derived from the thin public schema. | + +Jobs and tasks both accept either an in-process handler or a runtime-specific entrypoint; workflows +sequence their definitions. The three builders share the same typestate discipline. + +### Handler results and tools + +| Symbol | Kind | Description | +| --- | --- | --- | +| `createSuccessResult` | function | Create a successful job result. | +| `createFailureResult` | function | Create a failed job result. | +| `createJobTools` | function | Create handler tools backed by the active worker telemetry context. | +| `JobTools` | type alias | Runtime tools exposed to worker job handlers. | +| `JobToolSpan` | type alias | Span operations exposed to worker job handlers. | + +`JobTools` is how a handler reaches telemetry without importing an SDK: the span operations it +exposes are bound to the active worker instrumentation context. + +### Schedules and permissions + +| Symbol | Kind | Description | +| --- | --- | --- | +| `cron` | variable | Cron schedule helpers for worker jobs. | +| `CronHelpers` | type alias | Cron schedule helper surface for worker jobs. | +| `permissions` | variable | Permission presets for worker jobs and tasks. | +| `PermissionPresets` | type alias | Worker permission preset surface for common job execution modes. | + +### Runtime + +| Symbol | Kind | Description | +| --- | --- | --- | +| `startWorkers` | function | Create and start a workers runtime using default composition. | +| `createWorkersRuntime` | function | Create an unstarted workers runtime with the root surface's minimal options. | + +Two entry styles for one runtime: root `startWorkers()` creates and starts the memory-backed preset, +while root `createWorkersRuntime()` creates the same runtime without starting it. Import +`createWorkersRuntime` from `/runtime` when registries, executors, storage, or other runtime ports +must be injected explicitly. + +### Inspection + +| Symbol | Kind | Description | +| --- | --- | --- | +| `inspectJob` | function | Inspect a job definition without starting a runtime. | +| `inspectTask` | function | Inspect a task definition without starting a runtime. | +| `inspectWorkflow` | function | Inspect a workflow definition without starting a runtime. | + +These are what CLI listings and doctor checks call: they read a definition and report on it without +starting a runtime or touching storage. + +### Idempotency + +| Symbol | Kind | Description | +| --- | --- | --- | +| `WorkerIdempotencyPort` | interface | Durable applied-keys store used to make worker effects exactly-once-effective. | +| `WorkerIdempotencyInput` | type alias | Input used to resolve and claim an applied key for one worker delivery. | +| `WorkerIdempotencyClaim` | type alias | Result returned when a worker delivery attempts to claim an applied key. | +| `WorkerIdempotencySource` | type alias | How a worker delivery idempotency key was resolved. | + +Delivery is at-least-once; effects are exactly-once-effective. A key is resolved from a caller-supplied +key, the message id, or a payload hash — `WorkerIdempotencySource` records which — and then claimed +against the port before the effect runs. + +## Related pages + +- [`@netscript/plugin-workers`](/reference/workers/) — the deployable plugin that binds this core to a + NetScript host. +- [`@netscript/plugin-triggers-core`](/reference/plugin-triggers-core/) — trigger handlers enqueue + these jobs through `enqueueJob`. +- [`@netscript/plugin-sagas-core`](/reference/plugin-sagas-core/) — its `integration/workers` helpers + dispatch jobs and tasks from saga cascades. +- [`@netscript/plugin-streams-core`](/reference/plugin-streams-core/) — the producer behind the + projected execution stream. + +--- + +Back to the [reference overview](/reference/). diff --git a/packages/fresh/src/application/defer/island.ts b/packages/fresh/src/application/defer/island.ts new file mode 100644 index 0000000000..d2bc5be3f1 --- /dev/null +++ b/packages/fresh/src/application/defer/island.ts @@ -0,0 +1,8 @@ +/** + * Fresh island entry for the deferred refresh coordinator. + * + * @module + */ + +export { DeferComponent, type DeferComponentProps } from './DeferIsland.tsx'; +export type { DeferPolicyInput, DeferPolicyProfile, DeferPolicyResolved } from './policy.ts'; diff --git a/packages/fresh/tests/defer-island-client-bundle_test.ts b/packages/fresh/tests/defer-island-client-bundle_test.ts new file mode 100644 index 0000000000..62f82454ac --- /dev/null +++ b/packages/fresh/tests/defer-island-client-bundle_test.ts @@ -0,0 +1,64 @@ +import { assert, assertEquals, assertStringIncludes } from '@std/assert'; +import { fromFileUrl, join } from '@std/path'; + +interface ViteManifestEntry { + readonly file: string; + readonly isEntry?: boolean; + readonly name?: string; + readonly src?: string; +} + +const FIXTURE_ROOT = fromFileUrl( + new URL('./fixtures/defer-island-client/', import.meta.url), +); + +Deno.test('vite build emits the registered defer island in the client bundle', async () => { + const outputDir = await Deno.makeTempDir({ prefix: 'netscript-defer-island-' }); + + try { + const output = await new Deno.Command(Deno.execPath(), { + args: [ + 'run', + '--no-lock', + '-A', + 'npm:vite@7.2.2', + 'build', + '--config', + 'vite.config.ts', + '--outDir', + outputDir, + ], + cwd: FIXTURE_ROOT, + stdout: 'piped', + stderr: 'piped', + }).output(); + + const stderr = new TextDecoder().decode(output.stderr); + assertEquals(output.code, 0, `Vite fixture build failed:\n${stderr}`); + + const manifest = JSON.parse( + await Deno.readTextFile(join(outputDir, 'client', '.vite', 'manifest.json')), + ) as Record<string, ViteManifestEntry>; + const islandEntries = Object.values(manifest).filter((entry) => + entry.name?.startsWith('fresh-island__') + ); + + assertEquals(islandEntries.length, 1, 'Expected exactly one registered island client entry'); + const islandEntry = islandEntries[0]; + assert(islandEntry.isEntry, 'Expected the defer island chunk to be a client entry'); + assertStringIncludes( + `${islandEntry.name ?? ''} ${islandEntry.src ?? ''}`, + 'defer', + 'Expected the emitted island entry metadata to identify the defer island', + ); + + const islandBundle = await Deno.readTextFile(join(outputDir, 'client', islandEntry.file)); + assertStringIncludes( + islandBundle, + 'partial-miss', + 'Expected the defer client policy to be present in the emitted island bundle', + ); + } finally { + await Deno.remove(outputDir, { recursive: true }); + } +}); diff --git a/packages/fresh/tests/fixtures/defer-island-client/client.ts b/packages/fresh/tests/fixtures/defer-island-client/client.ts new file mode 100644 index 0000000000..cb0ff5c3b5 --- /dev/null +++ b/packages/fresh/tests/fixtures/defer-island-client/client.ts @@ -0,0 +1 @@ +export {}; diff --git a/packages/fresh/tests/fixtures/defer-island-client/main.ts b/packages/fresh/tests/fixtures/defer-island-client/main.ts new file mode 100644 index 0000000000..65bc813870 --- /dev/null +++ b/packages/fresh/tests/fixtures/defer-island-client/main.ts @@ -0,0 +1,5 @@ +import { App } from 'fresh'; + +export const app: App<unknown> = new App(); + +export default app; diff --git a/packages/fresh/tests/fixtures/defer-island-client/otel-api.ts b/packages/fresh/tests/fixtures/defer-island-client/otel-api.ts new file mode 100644 index 0000000000..6c7d615c8f --- /dev/null +++ b/packages/fresh/tests/fixtures/defer-island-client/otel-api.ts @@ -0,0 +1,50 @@ +type SpanCallback<T> = () => T; + +interface FixtureSpan { + addEvent(): void; + end(): void; + recordException(): void; + setAttributes(): void; + setStatus(): void; +} + +const span: FixtureSpan = { + addEvent() {}, + end() {}, + recordException() {}, + setAttributes() {}, + setStatus() {}, +}; + +/** Minimal OpenTelemetry context surface required by the client bundle fixture. */ +export const context: { + active(): object; + with<T>(value: object, callback: SpanCallback<T>): T; +} = { + active: () => ({}), + with: (_value, callback) => callback(), +}; + +/** Minimal propagation surface retained by the telemetry facade exports. */ +export const propagation: { + extract(value: object): object; + inject(value: object): void; +} = { + extract: (value) => value, + inject: () => {}, +}; + +/** Minimal OpenTelemetry trace surface required by the client bundle fixture. */ +export const trace: { + getActiveSpan(): undefined; + getSpanContext(): undefined; + getTracer(): { startSpan(): FixtureSpan }; + setSpan(value: object, activeSpan: FixtureSpan): object; + setSpanContext(value: object): object; +} = { + getActiveSpan: () => undefined, + getSpanContext: () => undefined, + getTracer: () => ({ startSpan: () => span }), + setSpan: (value) => value, + setSpanContext: (value) => value, +}; diff --git a/packages/fresh/tests/fixtures/defer-island-client/vite.config.ts b/packages/fresh/tests/fixtures/defer-island-client/vite.config.ts new file mode 100644 index 0000000000..c6263daa41 --- /dev/null +++ b/packages/fresh/tests/fixtures/defer-island-client/vite.config.ts @@ -0,0 +1,17 @@ +import { fresh } from 'jsr:@fresh/plugin-vite@1.1.2'; +import { resolve } from 'node:path'; +import type { UserConfig } from 'vite'; + +const config: UserConfig = { + root: import.meta.dirname, + resolve: { + alias: { + '@opentelemetry/api': resolve(import.meta.dirname!, 'otel-api.ts'), + }, + }, + plugins: [ + fresh({ islandSpecifiers: ['@netscript/fresh/defer/island'] }), + ], +}; + +export default config; From ceb47f99df189d5431dfa1f3fe4058893a995216 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 17:45:38 +0200 Subject: [PATCH 40/57] chore(harness): record the shallow-clone trap and withdraw the false history claim This checkout is shallow. 317e4b509 is the shallow boundary, not a root, and has canonical parent 6a4ca79de. So every history-shaped probe I ran was unreliable, and my public rebuttal of the evaluator's F1 was false. Retracted on PR #1585 with the citations. The lesson is not "run more probes": I ran three and called them independent when they were three expressions of one premise -- that the local object graph is complete. Independence of method is not independence of premise, and a shallow clone breaks that premise silently. Fourth instance of one pattern on this lane, and the sharpest, because the wrong answer was stable across repetition and therefore persuasive. Withdraws the false framing from the rail plan and records the standing rule: check is-shallow-repository before any ancestry claim, and use the compare API for anything load-bearing. Refs #1380 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../plan-quality-rail.md | 4 +- .../slices/pr-c-1380/codex-thread-ids.md | 16 ++ .../slices/pr-c-1380/implement.md | 206 ++++++++++++++++++ .../worklog.md | 67 ++++++ 4 files changed, 291 insertions(+), 2 deletions(-) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md index b781eb847e..c3dacad853 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/plan-quality-rail.md @@ -50,7 +50,7 @@ Measured at `01aa12b67`, re-confirmed at `84dd44ae7` (contains PR #1527). | Verdict table | 28 rows; **6** name non-live units; **14** live units have no row | | `*-soundness_test.ts` / `*_type.ts` | **6** / **12** (all under `tests/type-fixtures/`, **3** with `@ts-expect-error`) | | `deno doc --json` over all 30 export maps | **3.733 s**, exit 0, **567** `Failed resolving types` warnings | -| Repository history | root `317e4b509` (2026-07-06, "cut 0.0.1-beta.5"); **374** commits **measured at `84dd44ae7`** — the count moves with `main`, so it is only meaningful pinned to a sha (cycle 3 finding 7) | +| Repository history | ~~root `317e4b509`; 374 commits~~ **WITHDRAWN — measured in a SHALLOW clone and false.** `git rev-parse --is-shallow-repository` → **true**; `317e4b509` is the shallow **boundary**, not a root, and has canonical parent `6a4ca79de`. Canonical ancestry via the compare API: `0ef13de359...4a5a28b8` → `merge_base=0ef13de359b`, `ahead_by=2050`. See `worklog.md` § Diagnostic trap. | ### A14 has three identifier origins, not two @@ -82,7 +82,7 @@ beta.5 release cut) with 374 commits, and `git ls-tree 317e4b509:packages/` alre | `@netscript/workers` | same, per `arch-debt.md:561` → `packages/plugin-workers-core` | | `@netscript/sagas` | not present in this history; **a checked-in supersession record exists** — `arch-debt.md:583-584` reads "the top-level `packages/sagas` directory named in this heading no longer exists — the code and this resolved debt live entirely in `packages/plugin-sagas-core`". Cycle 3 finding 3: revision 3 asserted no record existed, which was **false**. Verified by the orchestrator. | | `@netscript/streams` | not present in this history; successor `packages/plugin-streams-core` exists; no supersession record found — PR-C states that absence **after** running the same `arch-debt.md` probe that found the sagas one | -| `@netscript/shared` | not present in HEAD's history; **added at `0ef13de35`, deleted at `fd8259b76`** (`feat(contracts): consolidate shared foundation package`, 2026-06-05, which deletes `packages/shared/deno.json` and 25 further `packages/shared/**` paths) — both on **non-ancestor** history. Cycle 3 finding 3: revision 3 said only "no removal commit on `main`", true but omitting the load-bearing commit. PR-C cites the commit **and** the ancestry qualifier. | +| `@netscript/shared` | **CORRECTED — it DID exist in reachable history.** Added `0ef13de35` (2026-06-05), deleted `fd8259b76`; **both are canonical ancestors** (compare API `merge_base` confirms each). The earlier "non-ancestor" qualifier was a shallow-clone artefact. Original text: added at `0ef13de35`, deleted at `fd8259b76` (`feat(contracts): consolidate shared foundation package`, 2026-06-05, which deletes `packages/shared/deno.json` and 25 further `packages/shared/**` paths) — both on **non-ancestor** history. Cycle 3 finding 3: revision 3 said only "no removal commit on `main`", true but omitting the load-bearing commit. PR-C cites the commit **and** the ancestry qualifier. | | `plugins/hello-world` | not present in this history; no successor and no supersession record | #1380 box 2 was **amended with owner authorization** to require per-row evidence and admit "never present diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/codex-thread-ids.md new file mode 100644 index 0000000000..b27fcba86f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-prc — Codex implementation thread +- **Thread / session id:** `019ff678-ce45-71e1-8823-b195f7b26735` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T16-55-40-019ff678-ce45-71e1-8823-b195f7b26735.jsonl` +- **Worktree:** `/home/codex/repos/ns006-doctrine` +- **Branch:** `fix/1380-doctrine-verdict-and-repo-gate` @ `8ddc17abb` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1380-doctrine-verdict-and-repo-gate`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=medium +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=medium +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-prc-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff678-ce45-71e1-8823-b195f7b26735 -- "<follow-up>" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/implement.md new file mode 100644 index 0000000000..12fb45262f --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/implement.md @@ -0,0 +1,206 @@ +use harness + +# PR-C — #1380: make the doctrine verdict table a measurement again, and `arch:check:repo` a verdict + +You are the **implementation agent** for the last large slice of the 0.0.6 internals quality rail. The plan +passed PLAN-EVAL (`PASS`, cycle 5). Most of the gate work you might expect is **already done by PR-B** — +read the baseline below before planning anything. + +Your orchestrator is a Claude Opus 5 high session in `/home/codex/repos/netscript-006-internals`. It holds +merge authority and owns the draft → ready flip. + +## SKILL + +- `netscript-doctrine` — **read first.** You are editing the doctrine's own verdict table and its fitness + gate. This skill is the navigator for what those documents mean. +- `netscript-harness` — run artifacts, slice discipline, commit trail. +- `netscript-tools` — scoped wrappers; verdict vs non-verdict. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` block. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-doctrine` | +| Branch | `fix/1380-doctrine-verdict-and-repo-gate` | +| Base | `fa5d0d411` (= `origin/main`) | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-c-1380/` | +| Closes | #1380 | +| Route | Codex · gpt-5.6-sol · **medium** (per-row provenance judgement and the RFC resolution are real decisions) | + +## Baseline, measured at your base — this is much better than #1380 describes + +```text +deno task arch:check → exit 0 +deno task arch:check:repo → exit 1, FAIL=2, WARN=307 + 1 × A1 : mod.ts missing — required canonical entrypoint + 1 × A14: unresolved Jest/Vitest global 'it' + (.llm/tools/fitness/check-doctrine_test.ts:41) +``` + +#1380 records `FAIL=53`. **PR-B (`e391f3aec`) already took it to 2**, by landing `discoverDoctrineRoots()` +and A14 origin-awareness. So: + +- **#1380 box 5** — "the A14 rule does not fire on a test importing `describe`/`it` from `@std/testing/bdd`" + — is **already implemented**. Tick it citing `e391f3aec`; do **not** reimplement it. +- What remains on the gate is **two findings**, and both have a precise cause. Read on. + +### The residual A1 is the root-as-package defect + +`deno.json`'s `arch:check:repo` runs `check-doctrine.ts` with **no `--root`**, so the checker evaluates the +repository root as a single package and walks trees that are not packages (`.llm/tmp/`, `docs/site/`, +`.llm/tools/`). Fix: have `arch:check:repo` iterate **`discoverDoctrineRoots()`** — the function PR-B already +landed. That kills the A1 and satisfies boxes 4 and 6 in one change. Do not write a second selector. + +### The residual A14 is self-referential, and the fix already exists elsewhere in the repo + +```text +.llm/tools/fitness/check-doctrine_test.ts:51 + source: "describe('bare global', () => {});\n", +``` + +That is **PR-B's own negative-control fixture** — the synthetic bare global that must stay red — sitting in +the test file as a **string literal**. A14 is scanning its own test data and reporting it as production code. +The rule is working; its input is wrong. + +`scan-code-quality.ts:47` already solves this exact problem for the other scanner: it skips any line +beginning with a quote or backtick, because *template and fixture source strings are data, not syntax in the +scanned module*. Apply the same guard to `check-doctrine.ts`. Keep it narrow — a leading-quote/backtick line +only — and add a test proving a bare global in **real** source still fails, so the guard cannot swallow the +signal it exists to preserve. + +## Contract + +### C1 — the verdict table becomes a measurement of this tree (boxes 1, 2, 3, 11, 12) + +Re-walk to the **36** live units (30 `packages/*` + 6 `plugins/*`) and nothing else. Sync +`06-archetypes.md`'s archetype assignment table to match. Add two tests: one failing if a verdict row names a +directory that does not exist, one failing if a live `packages/*`/`plugins/*` unit has no row. + +### C2 — per-row provenance, with the reconciliation already done for you (box 2, amended) + +Box 2 was **amended with owner authorization** to require per-row git evidence and to admit "never present in +this repository under that name" (issue comments `5264580324`, corrected by `5264832009`). Do not re-derive +this; it was got wrong twice already. The correct record per removed row: + +| Row | What to write | +| --- | --- | +| `@netscript/triggers` | not present anywhere in this repository's history (begins `317e4b509`, 2026-07-06); **superseded** by `packages/plugin-triggers-core` per **`arch-debt.md:385`**, an event predating this history | +| `@netscript/workers` | same, per **`arch-debt.md:561`** → `packages/plugin-workers-core` | +| `@netscript/sagas` | not present in this history; **a checked-in supersession record exists** — `arch-debt.md:583-584` reads "the top-level `packages/sagas` directory named in this heading no longer exists — the code and this resolved debt live entirely in `packages/plugin-sagas-core`" | +| `@netscript/streams` | not present in this history; successor `packages/plugin-streams-core` exists; **no** supersession record found — state that absence **after** running the same `arch-debt.md` probe that found the sagas one | +| `@netscript/shared` | **added at `0ef13de35`, deleted at `fd8259b76`** (`feat(contracts): consolidate shared foundation package`, which deletes `packages/shared/deno.json` and 25 further paths) — both on **non-ancestor** history; cite the commits **and** the ancestry qualifier | +| `plugins/hello-world` | not present in this history; no successor and no supersession record | + +**Why the qualifier matters:** this repository's history begins at `317e4b509` (a beta.5 cut) with a truncated +past, and `git ls-tree 317e4b509:packages/` already contains the full `plugin-*-core` tier. So "never +present" is a statement about **this repo's history**, not about the packages. Both records are true; write +both. + +### C3 — `arch:check:repo` becomes a verdict (boxes 4, 6, 13) + +Iterate `discoverDoctrineRoots()`; stop walking `.llm/tmp/`, `docs/`, `.llm/tools/`. Then `arch:check:repo` +must **exit 0**, or its residue must be enumerated in `arch-debt.md` as named debt with owners. With A1 and +the self-referential A14 both addressed, exit 0 should be reachable — measure it and report the real number +either way. `arch:check` must **stay exit 0**. + +### C4 — the accepted-red debt entry closes or gets a dated plan (box 7) + +`arch-debt.md`'s entry ("repo doctrine task — full historical scan remains red", created **2026-06-21**, +`DEBT_ACCEPTED`, closing gate "reduce unrelated root failures **or** replace the legacy root scan with +debt-aware package selection"). The second branch is what this PR does. Close it, citing the change, or leave +a dated closure plan naming both mechanical causes. + +### C5 — state which units are gated, and why one is not (box 8) + +`10-…md` gains a section naming which of the 36 units `arch:check` gates and why **`packages/cli/e2e`** is +excluded (it is a nested e2e harness, not one of the 36 top-level units). PR-B already stated this exclusion +in `09-anti-patterns-and-fitness-functions.md` — cross-reference rather than duplicate. Add a test that fails +if the documented gated set and `discoverDoctrineRoots()` disagree. + +### C6 — the engineering-reference gap gets a dated plan, not silence (box 9) + +`10-…md:79-181` specifies ten required contents; §6 is partial and §1–§5/§8–§10 do not exist. The box asks +for **a dated plan**, not the reference. Write one, authored *from* the refactors as a byproduct. Add a test +asserting the section exists and carries a date. + +### C7 — the RFC divergence is resolved by recording what the repo already chose (box 10) + +#1380 says "Zero numbered RFCs have ever landed". **False now.** `rfcs/` contains `0001-*.md` through +`0005-*.md`, all `status: Accepted`, and `rfcs/0005-devtools-contribution.md:10-18` names `rfcs/README.md` +canonical. So: record `rfcs/NNNN-*.md` as the canonical location, classify `.llm/runs/*/design/canonical/` +bundles as provenance/draft artifacts, and map the five `DECISION_PENDING` entries +(`CRON-SUBSYSTEM-DUP`, `RUN-ARTIFACT-ARCHIVAL-POLICY`, `PAGEBUILDER-LEGACY-COMPAT-TREE`, +`FORMPAGEPROPS-PLAYGROUND-MIGRATION`, `REDIS-LEGACY-VALUE-FALLBACK`) onto it. **Do not file those five +RFCs** — #1380's Boundaries forbid it. Add a test asserting all five ids appear. + +### C8 — the `labeled`-trigger documentation fix (rail `R-11`) + +Three statements are true and the current documents assert something weaker: + +1. `openhands-phase-eval.yml` **does** listen to `labeled`. +2. `ci.yml:41` **does not** — and `ci.yml` is what runs `close-gate` and the acceptance mirror. +3. To make the close-gate and mirror observe a new label, **re-run the existing run** (`gh run rerun`), which + re-reads live labels — *not* push, because a push moves the head and invalidates an existing IMPL-EVAL + verdict. + +`netscript-pr` `SKILL.md:169-170` currently says "applying `status:ready-merge` itself triggers a fresh run +(the workflow listens to `labeled`)", which is now **half true** and therefore worse than false: an operator +applies the label, sees a run appear, and concludes the close-gate was re-evaluated. It was not. Correct that +sentence and `check-close-gate.ts`'s repair hint, then regenerate the `.claude/skills/` mirror +(`deno task agentic:sync-claude`) and validate with `deno task agentic:check-claude`. **Change no workflow +trigger.** + +## Acceptance mapping + +#1380 has **13** boxes. Read them live. Provide a fenced `acceptance-evidence` block using +**`box-index: 1..13`** — **not** exact box text; several boxes wrap, and wrapped boxes are unmatchable by +exact text. Box 5 is evidenced by PR-B's commit `e391f3aec`, not by work in this PR — say so in the evidence. + +## Gates — paste real output with exit codes + +| # | Gate | Command | +| --- | --- | --- | +| 1 | fitness tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/fitness/` | +| 2 | doctrine curated | `deno task arch:check` — must stay **exit 0** | +| 3 | doctrine repo | `deno task arch:check:repo` — **exit 0**, or residue enumerated in `arch-debt.md` | +| 4 | quality gate | `deno task quality:gate` — exit 0 | +| 5 | scoped check/lint/fmt | the wrappers with `--root .llm/tools/fitness --root .llm/tools/validation --ext ts` (owned roots **only** — do not sweep all of `.llm/tools`, it contains pre-existing residue you do not own) | +| 6 | **asset-barrel freshness** | `deno task gen:assets-barrel`, then `git status --porcelain` **empty** | +| 7 | Claude surface | `deno task agentic:check-claude` after `agentic:sync-claude` | +| 8 | doc tests | the new existence/coverage/date/RFC-mapping tests | + +Run **all** gates before reporting done, so the head is final when the orchestrator flips to ready — that +flip triggers IMPL-EVAL and a commit after it invalidates the verdict. + +## PR mechanics + +1. First commit is the slice-dir bootstrap; open the **draft PR** in that same session; comment per slice. +2. `## Scope` carries `Closes #1380` on its own line. Reference `#1403`/`e391f3aec` for box 5 **without** a + closing keyword. +3. Labels: `type:docs`, `area:docs`, `area:tooling`, `area:packages`, `priority:p2`, `status:impl`, milestone + `0.0.6`. Exactly one `status:`. **This is not a docs-lane PR** despite `type:docs` — its acceptance + requires tooling code, so do **not** apply `ci:skip-e2e`/`ci:skip-scaffold` on the basis of the labels. +4. **Leave the PR draft.** The orchestrator owns the flip and will re-sync against main immediately before it. +5. Resolve commit hashes in a separate step; paste literal values. + +## Boundaries + +- Do **not** perform the six open verdict-Refactor/Restructure package refactors (`packages/database`, + `packages/kv`, `packages/service`, `packages/workers`, `plugins/triggers`, `plugins/workers`). This issue + re-establishes the denominator; the refactors are separate slices. +- Do **not** file the five `DECISION_PENDING` RFCs. Do **not** duplicate #1093, #1280, #1320, #232, #301. +- Do **not** reimplement A14 origin-awareness or `discoverDoctrineRoots()`; both are on main. +- Do **not** widen the A14 string-literal guard beyond a leading quote/backtick line, and prove a real bare + global still fails. +- Do **not** touch `.llm/tools/quality/**` (PR-D's surface) or any workflow trigger. +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or `quality-allow:`. +- Do **not** merge or flip to ready. + +## Escalate instead of going idle + +If a gate is red and you cannot green it, or a contract here is wrong, write it in your slice `drift.md`, post +it as a PR comment, and continue with what is not blocked. On this lane escalation has four times found the +orchestrator's brief wrong rather than the code — including one incoherent sequencing decision that would +have shipped a red gate. Raising it is the expected behaviour. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md index a93e39be20..8fad6deb55 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/worklog.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/worklog.md @@ -455,3 +455,70 @@ about. **No manual OpenHands trigger has been or will be issued.** Every formal evaluation in this run reached the evaluator through the `openhands` + `status:plan-eval` label pair or the draft → ready transition. That is also why the earlier "duplicate run" scare resolved as no-op skip events rather than duplicate execution. + +--- + +## Diagnostic trap: this checkout is SHALLOW, and every history-shaped git answer here is unreliable + +Recorded at the owner's explicit instruction. This is the most consequential methodological error of the run +and it invalidated a public rebuttal I posted against a correct formal verdict. + +### The trap + +```text +git rev-parse --is-shallow-repository → true +gh api /repos/rickylabs/netscript/commits/317e4b509 → parents: 6a4ca79de +``` + +`317e4b509` is **not** a root commit. It is the **shallow boundary**, and it has a canonical parent. Git +reports it as a root because the parent object is simply absent locally — **with no error and no warning.** + +### What it made false + +| Probe I ran | What I concluded | Why it was wrong | +| --- | --- | --- | +| `git rev-list --max-parents=0 HEAD` → `317e4b509` | "history begins 2026-07-06" | reports the shallow boundary as a root | +| `git merge-base --is-ancestor 0ef13de35 HEAD` → false | "non-ancestor history" | returns false for a commit whose connecting history is absent | +| `git log HEAD -- packages/shared/deno.json` → 0 | "never in reachable history" | cannot traverse past the boundary | +| `git for-each-ref --contains 0ef13de35` → 3 branches | "lives only on other refs" | true, and irrelevant to canonical ancestry | + +### The canonical answer, via the API rather than the clone + +```text +gh api compare/0ef13de359...4a5a28b8 → status=ahead merge_base=0ef13de359b ahead_by=2050 +gh api compare/fd8259b76...4a5a28b8 → status=ahead merge_base=fd8259b76d8 +``` + +Both are canonical ancestors. `@netscript/shared` **did** exist in reachable history — added `0ef13de35` +(2026-06-05), deleted `fd8259b76`. + +### The lesson, which is not "run more probes" + +I ran **three** probes and called them independent. They were three expressions of **one premise** — that the +local object graph is complete. **Independence of method is not independence of premise.** Adding a fourth +probe of the same kind would have produced a fourth wrong answer with more confidence. + +This is the fourth instance of one pattern on this lane, and the sharpest: + +1. asserted an absence (`packages/sagas` supersession record) without the probe that finds the presence; +2. asserted a mechanism (the `code-quality` changed-file range) without running the range; +3. asserted an outcome (dispatch would proceed) without tracing the data flow; +4. asserted ancestry from a clone that structurally cannot answer it. + +Each time the missing step was one command. The difference here is that the wrong answer was *stable across +repetition*, which is what made it persuasive. + +### Standing rules taken + +- **Before any history-shaped claim** — ancestry, "first commit", "never existed", "not reachable" — run + `git rev-parse --is-shallow-repository`. If true, the clone cannot answer it. +- For anything load-bearing, use the **compare API** (`gh api /repos/:o/:r/compare/A...B` → `status`, + `merge_base`, `ahead_by`), which is computed on the canonical graph. +- Treat `--all` / `for-each-ref --contains` results as statements about **local refs**, never about ancestry. +- When several probes agree, ask what premise they share **before** treating agreement as corroboration. + +### Blast radius, being corrected + +The false framing reached `plan-quality-rail.md` (4 occurrences), the doctrine document and doc-contract test +in PR #1585, box 2's acceptance evidence, and **#1380 comment `5264832009`**. All are corrected — the run +record must not carry the claim that a formal evaluator correctly rejected. From 814aefff2c13ad29254018595108f0452e9555e3 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 17:53:35 +0200 Subject: [PATCH 41/57] chore(harness): record the shallow-clone trap and gh-watch's third stale match F1 was correct and my rebuttal was false. These worktrees are shallow, so 317e4b509 is the boundary rather than a root, and both shared-package commits are canonical ancestors per the compare API. Corrected across all three surfaces F1 named, with a diagnostic note so the next re-walk does not repeat it. The finding worth keeping is methodological: I ran three probes and called them independent when they shared one premise -- that the local object graph is complete. A shallow clone breaks that silently, so the wrong answer was stable across repetition and therefore persuasive. Also records gh-watch reporting a superseded verdict as terminal in 0s for the third time. Consuming it would have blocked a PR whose finding was already fixed. Refs #1380 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 691a85a6d4..838869e656 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1273,3 +1273,53 @@ deno task quality:scan:repo → exit 0 that lands comment-awareness. First time this lane's count has risen, and it is reversible by construction. - **Lane state: #1436, #1415, #1530, #1566, #1403 closed.** Remaining owned: **#1380** (PR-C), **#1549** (PR-D). + +## D-39 — the shallow-clone trap, and `gh-watch`'s stale match for the third time + +- **Severity:** architectural (methodology), recorded at the owner's instruction +- **Recorded:** 2026-08-12, PR #1585 + +### F1 was correct and my rebuttal was false + +The evaluator flagged the doctrine's `@netscript/shared` provenance as a false-done state. I rebutted it with +three probes and was **wrong**. These worktrees are **shallow clones**: + +```text +git rev-parse --is-shallow-repository → true +gh api …/commits/317e4b509 → parents: 6a4ca79de ← the shallow BOUNDARY, not a root +gh api compare/0ef13de359...4a5a28b8 → merge_base=0ef13de359b ahead_by=2050 +gh api compare/fd8259b76...4a5a28b8 → merge_base=fd8259b76d8 +``` + +Both commits **are** canonical ancestors, so `@netscript/shared` did exist in reachable history. Corrected at +`b409498a8` across all three surfaces F1 named (doctrine prose, the doc-contract test's expected strings — it +would otherwise have kept a false statement green — and box-2 evidence), plus a diagnostic note in the +doctrine so the next re-walk does not reach for the same commands. Rebuttal retracted publicly +(`5269078936`); provenance corrected at the issue (`5269095079`); the run's own artifacts corrected. + +### The methodological finding, which is the one worth keeping + +I ran **three** probes and called them independent. They were three expressions of **one premise** — that the +local object graph is complete. **Independence of method is not independence of premise.** A shallow clone +breaks that premise *silently*: `rev-list --max-parents=0` reports the boundary as a root and +`merge-base --is-ancestor` returns false for absent objects, with no error. So the wrong answer was **stable +across repetition**, which is what made it persuasive enough to rebut a correct verdict. + +Fourth instance of one pattern on this lane: an absence asserted without the probe that finds the presence; +a mechanism asserted without running the range; an outcome asserted without tracing the data flow; and now +ancestry asserted from a clone that structurally cannot answer it. **Standing rule:** check +`git rev-parse --is-shallow-repository` before any history-shaped claim, and use +`gh api /repos/:o/:r/compare/A...B` for anything load-bearing. Treat `--all` and `for-each-ref --contains` as +statements about **local refs**, never ancestry. + +### `gh-watch` matched a superseded verdict — third occurrence + +After re-triggering at head `0b34371c2`, `gh-watch` printed `TERMINAL PR #1585: FAIL_FIX after 0s`, citing run +`31611977212` — the **cycle-1** verdict at the old head `4a5a28b8a`. The cycle-2 evaluator (`31614677830`) was +still in progress and had emitted nothing. + +That is the same defect as D-24, now seen three times on this lane. Consuming it would have re-consumed the +verdict the re-trigger existed to replace — and this time it would have blocked a PR whose finding was +already fixed. The workaround remains: **wait on the Actions run id**, then match the verdict comment's +`run_id` against it. One trigger marker was verified at the new head (`gen=29350385980`), so exactly-once +holds. From 83015d073b529d0b8f2a12ab2dcf2493dc7b8e83 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 18:13:37 +0200 Subject: [PATCH 42/57] chore(harness): record #1380 closed and dispatch the lane's last issue PR #1585 merged at eb373db29, closing #1380 with 13/13 boxes. Verified on merged main: arch:check:repo exit 0, closing an entry that had been DEBT_ACCEPTED red since 2026-06-21. Records honestly that it took two evaluator cycles because the orchestrator rebutted a correct false-done finding using probes run in a shallow clone. #1549 dispatched as the final slice, consuming #1537's extractFencedBlocks rather than writing a second fence parser. Refs #1380, #1549 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../cut-trace.md | 5 +- .../slices/pr-d-1549/codex-thread-ids.md | 16 ++ .../slices/pr-d-1549/implement.md | 181 ++++++++++++++++++ 3 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 6072176fe2..1c58f9de76 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,6 +8,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | +| 5 | 2026-08-12 ~16:12Z | `eb373db29` | **#1585** | **#1380** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **26 checks, 0 current failures** at `0b34371c2`; **13/13** boxes mirrored; `review-threads` PASS. Verified on merged main: **`arch:check:repo` exit 0**, closing an entry `DEBT_ACCEPTED` red since **2026-06-21**. Two eval cycles: `FAIL_FIX` on a false-done provenance claim the orchestrator **wrongly rebutted** from a shallow clone, then `PASS` at `0b34371c2` after correction. | | 4 | 2026-08-12 ~14:33Z | `e391f3aec` | **#1570** | **#1403** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **22 checks, 0 current failures** at `807d29003`; `close-gate` green after label + workflow re-run; **8/8** issue boxes mirrored from `box-index` evidence; `review-threads` PASS. Verified **on merged main**: `arch:check` exit 0 **at 36 roots** (was green only at 16), `quality:scan:repo` exit 0. Evaluator provenance preserved on proof: `PASS` at `c740ff6e0`, head moved to `807d29003` by another lane's `update-branch`, owned-path diff **empty**, so the evaluated implementation is the shipped implementation. | | 3 | 2026-08-12 13:40Z | `b79eca5d6` | **#1567** | **#1566** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` 0 current failures at `f6def9946`; `close-gate` PASS; 6/6 issue boxes mirrored; the one PR-body DoD box untickable at hand-off ticked by the orchestrator with run identities as evidence; `review-threads` PASS. **Merged by the release coordinator during a Claude 529 outage**, independently re-verified here: main contains it, `phase-eval-status.mjs` is present on trusted main, both items `status:shipped`. Verdict was head-matched to `f6def9946` before consumption (one trigger `gen=29340872564`, one verdict run `31599209037`). | | 2 | 2026-08-12 ~12:24Z | `e67c1ba13` | **#1560** | **#1530** (auto-closed `COMPLETED`) | 7/7 pass — `pr-checks` **23 checks, 0 current failures** at `28fc1b423`; `close-gate` green after label + workflow **re-run** (not a push); boxes 1–6 mirrored, box 7 `[post-merge]` verified after merge and ticked with evidence; `review-threads` PASS. **Two IMPL-EVAL verdicts:** `PASS` at `49e2b86e9` (pre barrel fix) then **`FAIL_FIX`** at `9ab361440` on a real close-gate defect, then `PASS` at `28fc1b423`. Consuming the first would have merged a red gate. | @@ -20,8 +21,8 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | 1 | PR-A `fix/1436-1415-close-gate-trust` → **PR #1527** | #1436, #1415 | Sol · low | **MERGED** `63cd1cd58` — thread `019ff4f4-1fce-7253-a7e0-d718c65b39cc`, worktree `/home/codex/repos/ns006-gatetrust`, 7 commits | | 2 | PR-E `fix/1530-type-fixture-scan-scope` → **PR #1560** | #1530 | Sol · low | **MERGED** `e67c1ba13` — thread `019ff5b2-7d02-…`, worktree `/home/codex/repos/ns006-typefixtures`, 6 commits. Restored `main`'s blocking `code-quality-repo` job: first green in **nine** consecutive push-to-main runs. | | 2 | PR-B `fix/1403-quality-gate-coverage` → **PR #1570** | #1403 | Sol · low | **MERGED** `e391f3aec` — absorbed rail `R-5` (A14 origin-awareness) after the implementer caught the plan's incoherent split; 3 defects fixed on one gate | -| 2 | PR-C `fix/1380-doctrine-verdict-and-repo-gate` | #1380 | Sol · medium | not yet dispatched | -| 2 | PR-D `fix/1378-quality-scan-rule-power` | #1378 | Sol · high | not yet dispatched | +| 2 | PR-C `fix/1380-doctrine-verdict-and-repo-gate` → **PR #1585** | #1380 | Sol · medium | **MERGED** `eb373db29` — verdict table restored to 36 units; `arch:check:repo` green after 7 weeks accepted-red; RFC location resolved; shallow-clone diagnostic recorded | +| 2 | PR-D `fix/1549-quality-scan-provable-half` | **#1549** | Sol · medium | **dispatched** — consumes #1537's `extractFencedBlocks`; last open issue in the lane | ## Re-planning events diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/codex-thread-ids.md new file mode 100644 index 0000000000..e6d8aff1bc --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/codex-thread-ids.md @@ -0,0 +1,16 @@ +# ns006-prd — Codex implementation thread +- **Thread / session id:** `019ff6bf-fa81-7262-b4a2-edd0e37c7c13` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T18-13-24-019ff6bf-fa81-7262-b4a2-edd0e37c7c13.jsonl` +- **Worktree:** `/home/codex/repos/ns006-scanrail` +- **Branch:** `fix/1549-quality-scan-provable-half` @ `f862fb570` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1549-quality-scan-provable-half`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=medium +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=medium +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/ns006-prd-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff6bf-fa81-7262-b4a2-edd0e37c7c13 -- "<follow-up>" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement.md new file mode 100644 index 0000000000..6b3c55334d --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement.md @@ -0,0 +1,181 @@ +use harness + +# PR-D — #1549: the provable half of the quality-scan rail + +You are the **implementation agent** for the **last** slice of the 0.0.6 internals lane. The plan passed +PLAN-EVAL (`PASS`, cycle 5). Everything here is deliberately scoped to what can be *proven* at this baseline — +the unprovable half (export-reachability, live issue-state verification) was moved to 0.0.7 with #1378 after it +was measured, so do not reach for it. + +Your orchestrator is a Claude Opus 5 high session in `/home/codex/repos/netscript-006-internals`. It holds +merge authority and owns the draft → ready flip. + +## SKILL + +- `netscript-harness` — run artifacts, slice discipline, commit trail. +- `netscript-tools` — scoped wrappers; verdict vs non-verdict. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` block. +- `netscript-deno-toolchain` — deterministic `deno test`, task semantics. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-scanrail` | +| Branch | `fix/1549-quality-scan-provable-half` | +| Base | **`eb373db29`** = `origin/main` at dispatch. Verified there: `arch:check` 0, `arch:check:repo` **0**, `quality:scan:repo` ok / `allowCount` **10**. All three green, so any red is yours. | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/` | +| Closes | #1549 | +| Route | Codex · gpt-5.6-sol · **medium** | + +## The extractor already exists — consume it, do not write a second one + +This is rail decision **R-10** and the reason this lane coordinated with the docs lane. #1537 landed +(`d558f9ab2`), so the primary path is available and the fallback is not needed. + +**Consume `.llm/tools/docs/snippet-extractor.ts`:** + +```ts +export function extractFencedBlocks(source: string, sourcePath: string): FencedBlock[] + +export interface FencedBlock { + sourcePath: string; fenceOrdinal: number; openingLine: number; codeStartLine: number; + closingLine: number; delimiter: '`' | '~'; delimiterLength: number; + infoString: string; language: string; checkedLanguage?: CheckedLanguage; + compilationExtension?: 'ts' | 'tsx'; exemptionReason?: string; body: string; +} +``` + +That gives you everything needed: `compilationExtension` identifies the TypeScript fences, +`sourcePath` + `fenceOrdinal` + `codeStartLine` give **stable per-snippet provenance** so a finding attributes +back to the documented line, and `exemptionReason` already carries the docs lane's exemption convention — +**respect it** rather than re-deciding what is exempt. + +**A second fence parser is a slice failure.** Two parsers with different fence rules would disagree about what +counts as a snippet, and each gate would pass on the corpus it happened to parse — invisibly. Add a test +asserting no second fence-parsing implementation exists under `.llm/tools/quality/**`. + +## Baseline, measured at dispatch + +```text +deno task quality:scan:repo → exit 0, findings 0, allowCount 10 +deno task quality:scan → exit 0, allowCount 7 +deno task arch:check → exit 0 (36 discovered roots, since PR-B) +deno task arch:check:repo → exit 0 (since PR-C) +``` + +`allowCount 10` includes **two allowances PR-B added deliberately** at +`.llm/tools/fitness/check-doctrine.ts` — `explicit-any` firing on the **English word** "any" in comments +(`any export abstract class`, `any class chain`). PR-B routed the durable fix here. See C4. + +## Contract + +### C1 — docs fences are scanned (boxes 1, 2) + +Fenced TypeScript under `docs/site/**` is scanned by the existing rule set, via `extractFencedBlocks`. Prove +**red-first**: an `as any` inside a `docs/site/**` fence must fail `quality:scan` before the change passes +after it. `*_test.ts` companions under `docs/site/**` are **docs fixtures, not exempt tests**. + +### C2 — the soundness/type-fixture exemptions are asserted by rule (box 3) + +The six `*-soundness_test.ts` files stay green with their `@ts-expect-error` lines **unchanged**, and a test +asserts that exemption rather than relying on filename luck. (The `tests/type-fixtures/**/*_type.ts` exemption +landed in #1530; assert it still holds rather than re-implementing it.) + +### C3 — `--max-allow` is wired at the measured count (box 4) + +Wire it into **both** `quality:scan` and `quality:scan:repo` at the count you measure **in this PR** — not at +a number copied from this brief. Add a budget-overflow fixture that fails. + +**In 0.0.6 there is no issue-id requirement.** The linked-open-milestoned-issue rule moved to 0.0.7 with +#1378, because the scanner runs `--allow-read` only and cannot observe live issue state. Do **not** implement +an id-presence check and call it registration — that would satisfy a test while violating the contract, which +is the unearned-green pattern this lane exists to remove. + +### C4 — comment-awareness, which lets the budget fall (box 4, and PR-B's routed finding) + +`explicit-any` currently matches the English word "any" in a comment. `scan-code-quality.ts:47` already skips +lines beginning with a quote or backtick for the same class of reason — *fixture and template source is data, +not syntax*. Extend that narrowly to **comment** lines for the `explicit-any` rule. + +Then **delete PR-B's two now-redundant `// quality-allow:` lines** in `.llm/tools/fitness/check-doctrine.ts` +and show `allowCount` falling **10 → 8**. Keep the guard narrow and prove it does not leak: a real +`any` in code on the *same line as a trailing comment* must still be found. + +### C5 — the same-PR budget-link predicate (box 5) + +A budget increase must carry an issue link in the same PR. That is a property of a **diff**, which a file +scanner cannot observe, so it is proved by **one added step in the existing `code-quality` pull-request job** +(rail `R-12`, owner-approved). Compare the `--max-allow` delta against issue links in the same diff and fail +when the budget rose without one. Provide a **missing-link RED** control and a **linked GREEN** control. + +Do not change the workflow's triggers or any skip semantics. + +### C6 — the triggers reference and its executable twin are typed (box 6) + +`docs/site/reference/triggers/index.md:310` (`const observedEvents: any[] = []`) and +`docs/site/reference/triggers/examples_test.ts:65` become properly typed and both compile. This is the +consumer-visible half of the issue: an agent copying the triggers reference should not inherit `any[]`. + +### C7 — the gate pair (box 7) + +`deno task quality:scan:repo` and `deno task arch:check` both green after the change. + +## Acceptance mapping + +#1549 has **7** boxes. Read them live. Use a fenced `acceptance-evidence` block with **`box-index: 1..7`** — +**not** exact box text; wrapped boxes are unmatchable by exact text and this cost PR #1560 a failed IMPL-EVAL +cycle. + +## Gates — paste real output with exit codes + +| # | Gate | Command | +| --- | --- | --- | +| 1 | quality + docs tool tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/quality/ .llm/tools/docs/` | +| 2 | repo scan | `deno task quality:scan:repo` — exit 0, and report `allowCount` **before and after** (expect 10 → 8) | +| 3 | default scan | `deno task quality:scan` — exit 0 | +| 4 | doctrine gates | `deno task arch:check` and `deno task arch:check:repo` — both **exit 0** (they are green at your base; keep them so) | +| 5 | docs snippet gate | the #1537 gate still passes — you are adding a consumer, not changing its behaviour | +| 6 | scoped check/lint/fmt | wrappers with `--root .llm/tools/quality --root .llm/tools/docs --ext ts` (**owned roots only** — do not sweep all of `.llm/tools`, it holds pre-existing residue you do not own) | +| 7 | **asset-barrel freshness** | `deno task gen:assets-barrel`, then `git status --porcelain` **empty** | +| 8 | triggers docs compile | the reference and its twin type-check | + +Run **all** gates before reporting done, so the head is final when the orchestrator flips to ready — that flip +triggers IMPL-EVAL and any commit after it invalidates the verdict. + +## PR mechanics + +1. First commit is the slice-dir bootstrap; open the **draft PR** in that same session; comment per slice. +2. `## Scope` carries `Closes #1549` on its own line. Reference `#1378` and `#1545` **without** closing + keywords — both are 0.0.7 and stay open. +3. Labels: `type:chore`, `area:tooling`, `area:docs`, `priority:p1`, `status:impl`, milestone `0.0.6`. Exactly + one `status:`. +4. **Leave the PR draft.** The orchestrator owns the flip and re-syncs against main immediately before it. +5. **State gate claims as evidence, not buckets.** For any scaffold tier, give the **wall time** and whether + step 2 was "Skipped by policy" — a `scaffold.runtime` that returns in seconds did nothing whatever the + rollup says. Do **not** cite `quality:gate` as coverage of your own diff without saying what it covers. +6. Resolve commit hashes in a separate step; paste literal values. + +## Boundaries + +- Touch only `.llm/tools/quality/**`, `.github/workflows/code-quality.yml` (the C5 step), the two named + `docs/site/reference/triggers/` files, the two `// quality-allow:` lines named in C4, and your slice dir. +- Do **not** implement export-reachability or allowance issue-state verification. Both moved to 0.0.7 with + #1378 after being measured unimplementable here (567 `deno doc` warnings over 1,714 published symbol + records; `--allow-read`-only scanner). +- Do **not** modify `.llm/tools/docs/**` — you are a **consumer** of the extractor. If its surface is + insufficient, say so and stop; do not fork it. +- Do **not** re-implement the `tests/type-fixtures` exemption (#1530) or `discoverDoctrineRoots()` (#1403). +- Do **not** fix findings the widened scan surfaces — triage them into your slice dir with file, line, rule and + assessment, and say how many. +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or a new `quality-allow:`. +- Do **not** merge, flip to ready, or apply `status:ready-merge` / `status:impl-eval` / `impl-eval:skip`. + +## Escalate instead of going idle + +If a gate is red and you cannot green it, or a contract here is wrong, write it in your slice `drift.md`, post +it as a PR comment, and continue with what is not blocked. On this lane escalation has **five** times found the +orchestrator's brief or plan wrong rather than the code — three missing gate commands, one incoherent +sequencing decision that would have shipped a red gate, and one false provenance claim I defended before +checking. Raising it is the expected behaviour, not a failure. From 551fb8c4e12912bd172ce11e54bc2878e832cacb Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 18:28:12 +0200 Subject: [PATCH 43/57] chore(harness): record the explicit scope authorization on PR #1596 Both escalations were contradictions in this lane's brief rather than defects in the work, and the implementer blocked on direction instead of resolving either silently. D-2 authorized: commit the generated agent-tools barrel, a mandatory derivative of the owned embedded tool change, with a second generator run proving an empty status. My gate list demanded that empty status while my boundary list omitted the file -- and this is the same coupling as D-22, which already cost PR-E a cycle and which I recorded and then failed to apply. D-1 authorized: fix exactly the three explicit-any findings the widened docs scan exposed, with sound narrowing and no casts, suppressions or allowances. A gate that deliberately widens coverage cannot ship red while deferring the first findings that widening exposes, so the fix is in scope by consequence -- and a fourth finding would not be. Refs #1549 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 838869e656..0dc3f4cbea 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1323,3 +1323,53 @@ verdict the re-trigger existed to replace — and this time it would have blocke already fixed. The workaround remains: **wait on the Actions run id**, then match the verdict comment's `run_id` against it. One trigger marker was verified at the new head (`gen=29350385980`), so exactly-once holds. + +## D-40 — explicit orchestrator scope authorization on PR #1596 (#1549), both escalations granted narrowly + +- **Severity:** significant (authorized scope expansion, recorded rather than absorbed) +- **Recorded:** 2026-08-12, owner decision relayed as orchestrator authorization +- Both escalations are contradictions **in this lane's brief**, not defects in the implementation. The + implementer refused to resolve either silently and blocked on direction, which is the correct behaviour and + the fifth time on this lane that escalation has found the brief wrong rather than the code. + +### D-2 — the generated derivative: AUTHORIZED + +`deno task gen:assets-barrel` regenerates `packages/cli/src/kernel/assets/agent-tools.generated.ts` (2 +insertions, 2 deletions) because the embedded `quality/scan-code-quality.ts` payload and bundle hash changed. +My brief's **gate** list demanded an empty `git status` after the generator while my **boundary** list omitted +that file — self-contradictory by construction. + +Authorized: commit the deterministic output. It is a **mandatory generated derivative of the owned embedded +tool change**, not scope drift. Constraint: only the generator output, with a **second** generator run proving +`git status --porcelain` empty — the idempotence proof is what distinguishes a generated commit from a +hand-edit. + +**This is the same coupling as `drift.md` D-22**, which cost PR-E a CI cycle and which I then recorded — and I +still failed to add the generated file to PR-D's boundary list while copying the gate across. Recording a +lesson is not the same as applying it. + +### D-1 — the three newly surfaced findings: AUTHORIZED, exactly three + +Treating `docs/site/**/*_test.ts` companions as docs fixtures rather than exempt tests surfaced three +pre-existing `explicit-any` violations in `docs/site/reference/contracts/examples_test.ts` (lines 13, 22, 43 — +Prisma-shaped fixture signatures, a sort callback, a count signature). My brief simultaneously required all +docs companions to be scanned, forbade fixing surfaced findings, and required both scans green. Those cannot +all hold at this baseline. + +Authorized: **fix exactly those three**, with sound `unknown`/input narrowing or a properly typed fixture. +**No casts, suppressions, or allowances** — any of those would satisfy the scan while defeating its purpose. + +The governing reason, which generalises past this PR: **a gate that deliberately widens coverage cannot ship +red while deferring the first findings that widening exposes.** Landing it red would have the PR prove the +gate now sees docs companions *and* demonstrate the repo cannot satisfy it — the false-done shape inverted. +Fixing them is the smallest connected integrity fix, so it is in scope **by consequence** rather than by +original scope. Anything beyond those three is not: a fourth finding is triaged, not fixed. + +Additional constraint recorded because the file is documentation: it must still compile **and still teach what +the page teaches**. If proper typing would change what the snippet demonstrates, stop and report rather than +silently alter documentation semantics. + +### Accepted drift, not a defect + +The trigger defect moved from `index.md:310` to line 173; symbol and executable twin still match. My brief +pinned a stale line number. The identity of a defect is the symbol, not the line. From 5b5f91be2792cfdb228b2a05537e8b4a104ed4e2 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 18:30:29 +0200 Subject: [PATCH 44/57] chore(harness): record that the authorization steer never arrived The D-1/D-2 authorization was refused with a thread-store conflict because the thread was mid-turn, and the turn-boundary watcher then observed the escalation record completing rather than the authorized work. On that basis I reported the work as in progress when no authorization had reached the agent and the head had not moved. D-9 already records that resume happens at the turn boundary and that active-writer is the not-yet signal. The failure was sending immediately on receiving a decision and treating urgency as a reason to skip the check. Rule tightened: after a resume, confirm delivery by a fresh agent message or a moved head before reporting on it. A zero exit from the wrapper is not delivery. Refs #1549 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 0dc3f4cbea..be808020f3 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1373,3 +1373,31 @@ silently alter documentation semantics. The trigger defect moved from `index.md:310` to line 173; symbol and executable twin still match. My brief pinned a stale line number. The identity of a defect is the symbol, not the line. + +## D-41 — the authorization steer silently did not arrive; I reported it as in progress + +- **Severity:** minor (supervision), but the reporting error matters more than the mechanics +- **Recorded:** 2026-08-12, PR #1596 +- **What happened:** the D-1/D-2 scope authorization was sent while the implementation thread was still + mid-turn writing its escalation record. `codex exec resume` refused it: + +```text +Error: thread/resume: thread/resume failed: thread 019ff6bf-… already has an active writer (code -32600) +``` + + The message was never delivered. I then armed a turn-boundary watcher, which returned + `{"turnComplete":true,"alreadyIdle":true}` — the turn it observed completing was the **escalation record**, + not the authorized work. +- **The reporting error.** On that basis I told the owner "the authorized work is in progress". It was not: no + authorization had reached the agent and the head had not moved. Corrected in the next report. A failed send + plus an idle-thread signal is indistinguishable from a completed instruction **unless you check the head**, + and I checked the head only afterwards. +- **Why this recurs on this lane.** `drift.md` D-9 already records that resume must happen **at the turn + boundary** and that `active writer` is the mechanical signal for "not there yet". The failure is not + ignorance of the rule — it is that I sent immediately on receiving the owner's decision, treating urgency as + a reason to skip the check. +- **Rule tightened:** after any `codex exec resume`, confirm delivery before reporting on it — either a fresh + agent message or a moved head. Exit code 0 from the wrapper is not delivery; in this case the wrapper's own + output carried the refusal while the surrounding command still succeeded. +- **Resent** once the thread was genuinely idle (0 active sessions in that worktree), with the authorization + unchanged plus a note that the earlier send never arrived, so the agent does not treat it as a repeat. From 2dc604a666b0038ecf03ea347e5cd0480aa5451c Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 18:48:42 +0200 Subject: [PATCH 45/57] docs(harness): record D-42 watch false-green and draft the lane retrospective D-42: agentic:gh-watch reported success twice while doing nothing -- once on a rejected `--` separator, once exiting 4 with no token -- and both times the task notification read exit 0 because the call was wrapped in a pipe. A watcher is a wake-up mechanism, never evidence; verdicts get confirmed by matching the verdict comment to the evaluated head. Also records the good half of the same observation: the ready-flip double-fired the phase-eval dispatch 8s apart and both runs succeeded with exactly one trigger posted, which is #1566's fix confirmed in production on the next occurrence. Retrospective drafted while PR #1596's IMPL-EVAL runs; merge figures land at close. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 32 +++++ .../retrospective.md | 119 ++++++++++++++++++ 2 files changed, 151 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/retrospective.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index be808020f3..5cbfecaa99 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1401,3 +1401,35 @@ Error: thread/resume: thread/resume failed: thread 019ff6bf-… already has an a output carried the refusal while the surrounding command still succeeded. - **Resent** once the thread was genuinely idle (0 active sessions in that worktree), with the authorization unchanged plus a note that the earlier send never arrived, so the agent does not treat it as a repeat. + +## D-42 — my own supervision watch reported success twice while doing nothing (severity: moderate) + +Watching for PR #1596's IMPL-EVAL verdict, `agentic:gh-watch` produced a **false green twice inside five +minutes**, and both times the harness task notification read `exit code 0`: + +1. **`--` separator rejected.** I invoked `deno task agentic:gh-watch -- --repo … --pr 1596`, copying the + `--`-separated form other `agentic:*` tasks use. This task passes argv straight through, so `--` arrived as + an argument: `Unknown argument: --`. The watch never started. +2. **No token.** Re-invoked without `--`, it exited **4** — `No valid GitHub token resolved`. `gh` itself is + authenticated in this session, but the task shell does not inherit a token; `GH_TOKEN=$(gh auth token)` is + required, exactly as the mirror pre-flight needs. + +**Why both looked green:** I wrapped the call as `timeout … | tail -12`, so the reported status was the +*pipeline's*, and `tail` always succeeds. `${PIPESTATUS[0]}` did print the real code into the log — I had the +evidence and read the notification instead of the output. + +**The uncomfortable part:** this run's entire thesis is that a gate reporting success without executing is +worse than a red one, and my own supervision loop did precisely that while I was auditing others for it. Had I +trusted it, I would have proceeded to the pre-merge gate believing a verdict existed. The second occurrence +also has a superseded-verdict precedent — D-24 — so `gh-watch` output has now misled this run in two distinct +ways and gets no benefit of the doubt. + +**Rule:** never accept a watch's exit status from a pipeline or a task notification. Read the tool's own final +line, and confirm the verdict independently — the verdict *comment* matched against the evaluated head — before +acting on it. A watcher is a wake-up mechanism, never evidence. + +**Also observed, and this one is good news:** the ready-flip fired **two** phase-eval dispatch runs 8s apart +(`31618717550` 16:39:58, `31618729177` 16:40:06) — the same double-fire shape as #1566. **Both exited +`success`**, and exactly **one** trigger comment was posted (`head=7264ce6aa`). Before PR-F this pattern +produced one red 404 run beside one success. That is #1566's fix confirmed in production on the next +occurrence, and it is why the dedup evidence is worth citing at the cut. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md b/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md new file mode 100644 index 0000000000..a094ba90f0 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md @@ -0,0 +1,119 @@ +# Retrospective — 0.0.6 chores/internals lane + +Written at lane close by the orchestrating session. Draft completed while PR #1596's IMPL-EVAL was running; +final merge figures filled in at close. Measured, not celebratory: the useful output of a run like this is the +list of things it got wrong early enough to fix, plus the assumptions it destroyed. + +## What the lane actually delivered + +Six PRs closing seven issues, all against `main`, no umbrella implementation. The three results worth naming +are the ones that changed what a gate *means* rather than adding one: + +- **`main`'s `code-quality-repo` went green after nine consecutive red push runs** (PR-E/#1530). It had been + red long enough to be background noise, which is the state in which a gate stops being read. +- **`arch:check` went from 16 hand-listed roots to 36 discovered ones** (PR-B/#1403), and + **`arch:check:repo` went green after seven weeks `DEBT_ACCEPTED`** (PR-C/#1380). +- **`quality:gate` had three independent coverage defects, on one line** (PR-B): a `-- packages plugins` + pathspec that could never see `.llm/tools/**`, an `if ((${#args[@]}))` that reported success having run no + command, and two-dot diff semantics that on PR #1539 enumerated nine already-merged foreign files and zero + lines of the PR under review. Every PR in this rail was `.llm/tools`-only, so the gate had been reporting + success on its own repairs. + +The last point is the lane's thesis in miniature: **a gate that reports success without executing is worse +than a red one**, because a red gate gets investigated and a false green gets trusted. + +`allowCount` fell **10 → 8** on PR-D — the first downward movement of the run. Every prior change to the +allowance budget was upward or flat. A budget that only ratchets up is not a budget. + +## The mistake that matters most + +On PR #1585 an evaluator returned `FAIL_FIX` saying a doctrine claim was false. **I rebutted it publicly, with +three probes, and I was wrong.** All three probes ran in a **shallow** checkout: `git rev-parse +--is-shallow-repository` → `true`, so the "root commit" I cited (`317e4b509`) was the shallow boundary, whose +canonical parent is `6a4ca79de`. GitHub's compare API showed both commits as ancestors +(`merge_base=0ef13de359b`, `ahead_by=2050`). The owner corrected me; I withdrew the rebuttal publicly, fixed +the prose, tests, and box evidence, and recorded the trap. + +The lesson is sharper than "shallow clones lie": + +> **Independence of method is not independence of premise.** Three different commands sharing one corrupted +> premise produce three confirmations and zero information. I mistook agreement among my own probes for +> corroboration. + +The aggravating factor is that I posted the rebuttal *before* checking, so for 20 minutes the PR carried +confident, false provenance written by the session holding merge authority. Withdrawing it publicly was the +minimum, not a remedy. + +## My own supervision loop produced two false greens (D-42) + +While auditing other people's gates for reporting success without executing, `agentic:gh-watch` did exactly +that to me twice inside five minutes — once rejecting a `--` separator, once exiting `4` on a missing token — +and both times the harness task notification read `exit code 0`, because I had wrapped the call in a pipe and +was reading the *pipeline's* status. I had the real exit code in my own log and read the notification instead. + +**A watcher is a wake-up mechanism, never evidence.** Verdicts get confirmed by matching the verdict comment +to the evaluated head, which is also the fix for the separate defect where `gh-watch` reported a *superseded* +verdict as terminal in 0s, three times (D-24). + +## What the delegation model got right + +**Briefing the gate as a deliverable, with an explicit instruction to escalate rather than idle, worked — and +what it caught was mostly me.** Escalation found the orchestrator's brief or plan wrong rather than the code +**six** times: + +| # | What the implementer caught | Consequence had they idled or complied | +| --- | --- | --- | +| 1 | Gate 1 missing `--allow-write` (9 tests use `Deno.makeTempDir()`) | idle at a red gate | +| 2 | Same gate still missing `--allow-run` | idle again | +| 3 | Asset-barrel gate absent from the brief | a red `ci.yml quality` (later cost PR-E a CI cycle) | +| 4 | Rail sequencing incoherent: the 36-root expansion (PR-B) is what surfaces A14's false positives, but A14 was scheduled in PR-C | **would have shipped a red `arch:check`** | +| 5 | #1436's prescribed fix was a no-op — `\b` was already present and *is* the cause, since `-` is a non-word character | a "fix" that changed nothing, with tests written to pass | +| 6 | PR-D's brief simultaneously required the generator's `git status` clean, forbade touching the file it regenerates, required all docs companions scanned, and forbade fixing what that surfaced | self-contradictory; no compliant implementation exists | + +Six for six, the escalation was correct and I amended. An implementation lane that stops and says "your brief +is wrong" is worth more than one that produces something plausible from a contradictory spec — and #5 is the +clearest case: complying would have produced a green test suite around a fix that fixed nothing. + +Case 3 has an uncomfortable coda: I recorded the asset-barrel coupling as D-22 after it cost PR-E a cycle, +then **copied the gate into PR-D's brief without copying the boundary**, reproducing the contradiction I had +already paid to learn. Writing a lesson down is not the same as applying it. + +## Process findings worth keeping + +- **`box-index` beats exact box text, structurally.** `acceptanceCheckboxes` keeps only each checkbox's first + raw line with backticks preserved, so any box that *wraps in the issue body* is unmatchable by exact text — + and the author cannot see the wrapping. Five of #1530's six boxes wrapped, so five of six keys were + unmatchable by construction. This cost PR #1560 a full failed IMPL-EVAL cycle before the cause was clear. +- **Re-evaluate after a required fix; never reason around a stale PASS.** PR #1560's *first* verdict PASSed a + head whose close-gate was red; the re-run's `FAIL_FIX` caught a real defect. Consuming the first verdict + would have merged a red gate. The rule paid for itself the one time it was tested. +- **Label, then re-run — never push.** `status:ready-merge` does trigger `ci.yml` via `labeled`, but where a + re-run is needed, a push would move the head and invalidate the IMPL-EVAL verdict. +- **Pre-validate the evidence mapping locally.** The mirror's `--dry-run` short-circuits on the missing + `status:ready-merge` label *before* parsing, so it cannot validate a mapping pre-label; running + `validateEvidenceMapping` directly against the live PR body and issue boxes can, and catches #1560's failure + class without a CI round trip. +- **Distinguish a flake from a defect before touching anything.** PR-D's `scaffold-static` failure was + `Error: socket hang up` — infrastructure. Re-run, do not investigate the diff, and do not push. + +## Assumptions this run destroyed + +1. **"`status:ready-merge` does not trigger `ci.yml`"** — false, and I had propagated it. `labeled` *is* in + `pull_request.types`; the workflow's live reads observe the label. Corrected in the docs. +2. **"Local git ancestry is authoritative"** — false in this shared checkout, which is shallow. GitHub's + compare API is the canonical source for ancestry questions. +3. **"A green `deno task quality:gate` covers this PR's diff"** — false for any `.llm/tools`-only PR, which + was *every PR in this rail*. Cited as coverage, it is a false green. +4. **"One evaluator pass per head is enough"** — true only if the head does not move. Any required fix + invalidates the verdict. + +## What I would do differently + +- **Check before publishing a rebuttal.** Especially when contradicting an evaluator, and most especially from + the session that holds merge authority — the asymmetry means my confident wrong answer travels further than + theirs. +- **Verify the environment before deriving evidence from it.** `--is-shallow-repository` costs nothing and + would have prevented the run's worst error outright. +- **When a lesson is recorded, apply it to the *next* brief mechanically**, not from memory — D-22 was written + down and still reproduced two PRs later. +- **Read tool output, not task notifications.** Two false greens in five minutes came from that single habit. From c4367a4d70a3db4720afb35eb64e303ed47e6eb9 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 18:52:06 +0200 Subject: [PATCH 46/57] docs(harness): record the consumer-bundle closure gap and brief its fix D-43: PR #1596's widened scanner imports ../docs/snippet-extractor.ts, but the consumer bundle enumerates files from consumer-tools.json with no transitive import resolution, so the installed quality/scan-code-quality.ts exits 1 with "Module not found". Reproduced locally. The reusable finding is the gate error, not the missing file: `gen:assets-barrel` plus an empty `git status` proves the barrel is current with respect to the manifest, not that the bundle is complete with respect to its own imports. It passed twice while the bundle was broken. The barrel gate is two gates and I had written only the first. Fix brief requires a third manifest category for module dependencies plus a closure test with a negative control -- not a tools entry with an invented symptom, and never a second fence parser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 47 ++++++ .../slices/pr-d-1549/implement-closure.md | 138 ++++++++++++++++++ 2 files changed, 185 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 5cbfecaa99..cfee7d0f3b 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1433,3 +1433,50 @@ acting on it. A watcher is a wake-up mechanism, never evidence. `success`**, and exactly **one** trigger comment was posted (`head=7264ce6aa`). Before PR-F this pattern produced one red 404 run beside one success. That is #1566's fix confirmed in production on the next occurrence, and it is why the dedup evidence is worth citing at the cut. + +## D-43 — the asset-barrel gate proves currency, not closure (severity: significant) + +PR #1596's `check-test` failed at +`packages/cli/src/public/features/agent/init/init-agent_test.ts:580` — the installed consumer bundle's +`quality/scan-code-quality.ts --help` exits 1. Reported by the owner, **reproduced locally**, real, and mine. + +**Mechanism.** `renderAgentToolEmbeddedContent()` bundles exactly what `.llm/tools/consumer-tools.json` +enumerates, with **no transitive import resolution**. When C1/R-10 made the scanner *consume* +`../docs/snippet-extractor.ts` rather than fork a second fence parser, the embedded scanner gained an import the +bundle cannot satisfy: + +```text +error: Module not found "file:///tmp/…/docs/snippet-extractor.ts". + at file:///tmp/…/quality/scan-code-quality.ts:2:37 +bundled paths: [… "quality/scan-code-quality.ts" …] ← docs/snippet-extractor.ts absent +``` + +The manifest's two categories both exclude this case: `supportFiles` is metadata, `tools` are runnable +diagnostics carrying `symptom` + `permissions` and surfaced as consumer commands. A library module is neither. + +**Why my gate missed it — the reusable part.** Gate 7 (`gen:assets-barrel`, then `git status --porcelain` +empty) **passed, twice**, including as an idempotence proof. The result was true and answered the wrong +question: it proves the barrel is **current with respect to the manifest**, not that the bundle is **complete +with respect to its own imports**. The generator faithfully embedded a file whose second line cannot resolve. + +This is the lane's own thesis turned on me: a gate reporting success on a property *adjacent* to the one that +matters. It happened on my leaf, one PR after I recorded the asset-barrel coupling as D-22 and then failed to +apply it. Two distinct failures now trace to the same generated-asset boundary, and neither was caught by the +gate I added for it. + +**Fix shape (not the one-liner).** Listing the extractor under `tools` would misfile a non-runnable module as a +runnable diagnostic with an invented `symptom`/`permissions` and expose it as a consumer command. Correct: a +**third manifest category for bundled module dependencies** — in the file set and the ordered-path hash, out of +the runnable surface — plus **a test asserting every relative import of every bundled file resolves to a bundled +path**, with a negative control. The test is the durable deliverable; without it the next tool that grows an +import breaks consumers silently and `gen:assets-barrel` stays green throughout. Inlining a second copy of the +extractor is excluded: two fence parsers that can drift is what R-10 forbids. + +**Sequencing (owner instruction).** Evaluated head `7264ce6aa` frozen until the active DeepSeek IMPL-EVAL is +terminal; correction lands afterward on a new immutable head, then exactly one re-triggered IMPL-EVAL. +`scaffold-static` at this head is unrelated and genuinely infrastructural (`Error: socket hang up`) — re-run, +do not investigate. + +**Gate correction for future briefs:** the asset-barrel gate is two gates, and I had been writing only the +first. (a) regenerate and assert an empty `git status` — currency; (b) install the bundle to a temp dir and +execute each runnable tool's `--help` — closure. (b) is what a consumer actually does, and only (b) fails here. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md new file mode 100644 index 0000000000..56077c022e --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md @@ -0,0 +1,138 @@ +use harness + +# PR-D follow-up — the consumer bundle ships an unresolvable import + +You are continuing the **same leaf** (`fix/1549-quality-scan-provable-half`, PR #1596) in the **same worktree**. +This is a correction to work you already landed, dispatched after the automatic IMPL-EVAL on head `7264ce6aa` +went terminal. Everything already merged in this PR stands — the defect is one missing capability in the +consumer-bundle generator, plus the guard that makes the class non-recurring. + +Do **not** revert the extractor consumption. Consuming `.llm/tools/docs/snippet-extractor.ts` instead of forking +a second fence parser is rail decision **R-10** and stays. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-scanrail` (unchanged) | +| Branch | `fix/1549-quality-scan-provable-half` (unchanged) | +| PR | **#1596** — already open, currently non-draft | +| Base | current `origin/main` — **re-sync before you start**; it has moved | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/` | +| Route | Codex · gpt-5.6-sol · **medium** (a manifest schema decision, not a mechanical edit) | + +## The defect, already reproduced — do not re-investigate + +`packages/cli/src/public/features/agent/init/init-agent_test.ts:580` +("installed consumer tools resolve from the project when process CWD differs") fails: the installed +`quality/scan-code-quality.ts --help` exits 1. + +```text +error: Module not found "file:///tmp/…/docs/snippet-extractor.ts". + at file:///tmp/…/quality/scan-code-quality.ts:2:37 + +bundled paths: ["consumer-tools.json","README.md","release.json","run-deno-check.ts","run-deno-lint.ts", + "run-deno-doc-lint.ts","validation/check-aspire-host-ports.ts","quality/scan-code-quality.ts", + "deps/outdated.ts","deps/why.ts","e2e/scaffold-e2e-test.ts"] +``` + +`renderAgentToolEmbeddedContent()` in `.llm/tools/generate-cli-assets-barrel.ts` bundles **exactly** what +`.llm/tools/consumer-tools.json` enumerates, with **no transitive import resolution**: + +```ts +for (const tool of manifest.tools) { + files[tool.path] = await Deno.readTextFile(new URL(`../../${tool.source}`, import.meta.url)); +} +``` + +Your scanner's line 2 import is therefore embedded but unsatisfiable in the installed tree. + +## Why the obvious fix is wrong + +Adding `.llm/tools/docs/snippet-extractor.ts` to `tools` would work and is **rejected**: every `tools` entry is +a runnable diagnostic carrying a `symptom` and a `permissions` list, and is surfaced to consumers as a command. +The extractor is a **library module**. Misfiling it means inventing a `symptom` for something a consumer can +never usefully run, and polluting the tool listing. + +`supportFiles` is also wrong — that is bundle metadata (`consumer-tools.json`, `README.md`, `release.json`). + +## Contract + +### C1 — a third manifest category for bundled module dependencies + +Add one, named for what it is (e.g. `modules`), holding `{ source, path }` pairs. Requirements: + +- included in the embedded file set, so the relative import resolves in the installed tree; +- included in the canonical **ordered-path hash** — a bundle whose content changes must change its hash, or + the hash stops being a bundle identity; +- **excluded** from the runnable tool surface: not listed as a consumer command, no `symptom`, no + `permissions`; +- `schemaVersion` handled deliberately. Decide whether this is a compatible addition or a version bump, state + which and why in your slice `worklog.md`, and make any reader of the manifest agree with your decision. + +Then register the extractor as `{ source: '.llm/tools/docs/snippet-extractor.ts', path: 'docs/snippet-extractor.ts' }` +so `../docs/snippet-extractor.ts` resolves from `quality/`. + +### C2 — the guard that makes this class non-recurring (the real deliverable) + +A test asserting that **every relative import of every bundled file resolves to a bundled path**. Derive it +from the bundle itself — scan each embedded file's import specifiers, resolve each relative one against its own +bundled path, and assert the target is present in the bundle. + +It must have a **negative control**: with a required module omitted, the test fails. Prove that before you make +it pass. A closure test that cannot fail is worth nothing, and this lane has already shipped one test whose +expectation was computed by calling the function under test (PLAN-EVAL cycle 3 caught it) — do not repeat that. + +Do **not** implement this by hardcoding "the scanner needs the extractor". The next tool to grow an import must +be caught without anyone remembering to update a list. + +### C3 — the end-to-end proof is executing the installed bundle + +The property that matters is what a consumer does: install the bundle and run the tool. Prove it by installing +`EMBEDDED_AGENT_TOOL_FILES` to a temp dir and executing each runnable tool's `--help`, asserting exit 0. + +This is the gate my brief got wrong and is worth stating plainly: `gen:assets-barrel` + an empty +`git status` proves the barrel is **current with respect to the manifest**; it does **not** prove the bundle is +**complete with respect to its own imports**. It passed twice here while the bundle was broken. If +`init-agent_test.ts:580` already provides adequate end-to-end coverage, say so and do not duplicate it — extend +it if it only covers one tool. + +### C4 — nothing else changes + +No revert of the extractor consumption; no second fence parser; no change to the scanner's rules, `--max-allow` +wiring, the docs-fence behaviour, or the budget-link workflow step. `allowCount` stays at **8** and the repo +scan stays green. + +## Gates — paste real output with exit codes + +| # | Gate | Command | +| --- | --- | --- | +| 1 | the failing test now passes | `deno test --allow-all packages/cli/src/public/features/agent/init/init-agent_test.ts` | +| 2 | closure guard + negative control | your new test, **plus** pasted proof of the control failing before the fix | +| 3 | generator tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/` for the generator's own suite | +| 4 | quality + docs tool tests | `deno test --allow-read --allow-env --allow-write --allow-run .llm/tools/quality/ .llm/tools/docs/` | +| 5 | barrel **currency** | `deno task gen:assets-barrel`, then `git status --porcelain` empty | +| 6 | barrel **closure** | install the bundle to a temp dir; every runnable tool's `--help` exits 0 | +| 7 | repo scan unchanged | `deno task quality:scan:repo` exit 0, `allowCount` still **8** | +| 8 | doctrine gates | `deno task arch:check` and `deno task arch:check:repo` — both exit 0 | +| 9 | scoped check/lint/fmt | wrappers over the roots you actually touched | + +Gates 5 and 6 are **different gates**. Running only 5 is what let this defect through. + +## Boundaries + +- Touch `.llm/tools/consumer-tools.json`, `.llm/tools/generate-cli-assets-barrel.ts`, its test surface, the + regenerated `packages/cli/src/kernel/assets/agent-tools.generated.ts`, and your slice dir. +- `packages/cli/src/public/features/agent/init/**` only if the end-to-end coverage in C3 genuinely belongs + there; prefer extending the existing test over adding a parallel one. +- Do **not** modify `.llm/tools/docs/snippet-extractor.ts` — still a consumer of it. +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or a new `quality-allow:`. +- Do **not** flip to ready, relabel, or merge. The orchestrator owns the ready-flip that re-triggers IMPL-EVAL + exactly once. + +## Escalate instead of going idle + +If a contract here is wrong, say so and stop — on this lane escalation has found the orchestrator's brief or +plan wrong **six** times, most recently the self-contradictory boundary list in your own previous brief. The +gate error that produced this very defect was mine too: I wrote the currency half of the barrel gate and +omitted the closure half. Raising a contradiction is the expected behaviour. From 86c23f05a6dd9ac6f372354c274631bcfc5645dc Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 18:52:55 +0200 Subject: [PATCH 47/57] docs(harness): measure the bundle closure before briefing its guard Audited every relative import of every bundled tool: exactly one gap exists and it is this PR's own (quality/scan-code-quality.ts -> ../docs/snippet-extractor.ts), and the extractor itself has zero imports, so registering it terminates the closure with no cascade. Both facts go in the brief because a closure test failing for pre-existing reasons would have made it self-contradictory -- the brief error this lane has already made twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../slices/pr-d-1549/implement-closure.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md index 56077c022e..ce95abb364 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md @@ -56,6 +56,19 @@ never usefully run, and polluting the tool listing. `supportFiles` is also wrong — that is bundle metadata (`consumer-tools.json`, `README.md`, `release.json`). +## Two facts measured for you, so C2 is not a trap + +I audited the whole bundle before writing this, because a closure test that fails for **pre-existing** reasons +would make this brief self-contradictory — which has already happened twice on this lane. + +1. **There is exactly one gap, and it is yours.** Every relative import of every bundled `.ts` file resolves to + a bundled path except `quality/scan-code-quality.ts -> ../docs/snippet-extractor.ts`. So C2 goes green on + registering that one module; it will not surface unrelated debt. +2. **The extractor has zero imports of its own** — no `import` or re-export lines at all. Registering it + therefore **terminates** the closure; there is no transitive cascade to chase. + +If your own measurement disagrees with either, trust yours and say so. + ## Contract ### C1 — a third manifest category for bundled module dependencies From 915ea61dea7d3638a689890fe22f01001afc2ab5 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 19:01:10 +0200 Subject: [PATCH 48/57] docs(harness): record gh-watch matching the orchestrator's own review as a verdict D-44: with the automatic DeepSeek run cancelled and no IMPL-EVAL comment on the PR at all, gh-watch reported "TERMINAL PASS after 724s". The only VERDICT: PASS text on the PR is this session's own REVIEW comment. Consuming it would have merged a PR carrying a reproducible consumer-facing defect on the orchestrator's own certification -- the one thing the harness forbids outright. A verdict now requires all three: a [PHASE: IMPL-EVAL] comment, an author other than this session, and an evaluated head equal to the head under consideration. gh-watch wakes a turn and does nothing else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index cfee7d0f3b..d04af11145 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1480,3 +1480,49 @@ do not investigate. **Gate correction for future briefs:** the asset-barrel gate is two gates, and I had been writing only the first. (a) regenerate and assert an empty `git status` — currency; (b) install the bundle to a temp dir and execute each runnable tool's `--help` — closure. (b) is what a consumer actually does, and only (b) fails here. + +## D-44 — `gh-watch` reported my own review comment as the evaluator's PASS (severity: significant) + +Third distinct failure of the same tool in one hour, and the only dangerous one. + +The automatic DeepSeek IMPL-EVAL run `31618732014` was **cancelled** at 16:58:41Z after 20 minutes with no +verdict. `agentic:gh-watch`, watching the same PR, reported: + +```text +TERMINAL PR #1596: PASS (PASS) after 724s +GH_WATCH_REAL_EXIT=0 +``` + +There is **no IMPL-EVAL verdict on #1596 at all** — verified directly: + +```text +gh pr view 1596 --json comments --jq '[.comments[]|select(.body|test("PHASE: IMPL-EVAL"))]|length' +=> 0 +``` + +The only `VERDICT: PASS` text on the PR is **my own** `[PHASE: REVIEW] [VERDICT: PASS — flipping to ready]` +comment. The watcher matched the orchestrator's review as the evaluator's verdict. + +**Why this one is worse than D-42.** Those were false *negatives-as-greens*: a watch that did nothing while +claiming success, costing time. This is a false *authority*: had I consumed it, I would have run the pre-merge +gate believing an independent evaluator had passed the head, and merged **on my own certification** — the one +thing the harness forbids without exception (`generator ≠ evaluator`, no lane self-certifies). The PR would +have merged with a reproducible consumer-facing defect (D-43) and a fabricated evaluator provenance. + +Three failure modes now recorded for this tool: superseded verdict as terminal (D-24), silent no-op / no-token +with a pipeline-masked exit code (D-42), and **non-evaluator comment matched as a verdict (this entry)**. It has +been wrong in every way a verdict watcher can be wrong. + +**Rule, now unconditional:** a verdict exists only when a comment matching `[PHASE: IMPL-EVAL]` exists, its +author is not this session, and its evaluated head equals the head under consideration. Assert all three by +direct query before the pre-merge gate. `gh-watch` may be used to wake a turn and for nothing else. + +**What made this catchable:** the owner reported the run as cancelled, which contradicted the watcher. Without +that contradiction the false PASS was plausible — it arrived at 724s, a believable evaluation duration, with a +believable verdict. That is the shape of the failures this lane exists to remove, and my own supervision loop +produced it twice in one afternoon. + +Fallback authorized by the owner and dispatched: one fresh native Opus 5 medium **read-only** session against +the immutable head `7264ce6aa`, trigger-immune single comment, no label cycle, no paid retrigger. It is briefed +to treat the orchestrator's claims as claims, warned that this checkout is shallow, and required to reach its +own conclusion on the D-43 closure failure. From 524d0bdc5fd59ec37ed5978aa06d830823118fb9 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 19:08:10 +0200 Subject: [PATCH 49/57] docs(harness): correct the verdict-detection rule and map the correction to F1/F2 D-45: the cancelled run did publish a formal FAIL_FIX, by editing the existing openhands-agent-summary comment in place. Two facts I had wrong: a verdict can arrive on a cancelled run (verdict_source=summary-file, written before cancellation), and it can arrive by comment edit rather than a new comment. D-44's rule -- keyed to a [PHASE: IMPL-EVAL] comment -- would have hidden exactly the verdict it existed to protect. Corrected to anchor on OPENHANDS_VERDICT: or the <!-- openhands-run --> marker's run_id, and to re-fetch before concluding "no verdict". D-44's substance stands; its rule is superseded. The verdict's F1/F2 independently match D-43, so the prepared brief needed only a mapping of its contracts to the two findings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 43 +++++++++++++++++++ .../slices/pr-d-1549/implement-closure.md | 17 ++++++-- 2 files changed, 57 insertions(+), 3 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index d04af11145..8f0a5db1e6 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1526,3 +1526,46 @@ Fallback authorized by the owner and dispatched: one fresh native Opus 5 medium the immutable head `7264ce6aa`, trigger-immune single comment, no label cycle, no paid retrigger. It is briefed to treat the orchestrator's claims as claims, warned that this checkout is shallow, and required to reach its own conclusion on the D-43 closure failure. + +## D-45 — my D-44 verdict-detection rule was too narrow to catch the real verdict (severity: moderate) + +D-44 concluded: "a verdict exists only when a comment matching `[PHASE: IMPL-EVAL]` exists…". That rule is +**wrong**, and it would have hidden the authoritative verdict it was written to protect against missing. + +The cancelled run `31618732014` **did** publish a complete formal `FAIL_FIX`, by **editing the existing +`<!-- openhands-agent-summary -->` comment in place** rather than posting a new one: + +```text +<!-- openhands-run: {"run_id":31618732014,"attempt":1,"conclusion":"cancelled","state":"not-run", + "verdict":"FAIL_FIX","verdict_source":"summary-file"} --> +OPENHANDS_VERDICT: FAIL_FIX +# IMPL-EVAL — PR #1596 … +``` + +My query `select(.body|test("PHASE: IMPL-EVAL"))` returned **0** twice, because the machine format is +`OPENHANDS_VERDICT:` plus a `# IMPL-EVAL` heading — not the `**[PHASE: IMPL-EVAL]**` token from the human phase +convention. Two structural facts I had not internalised: + +1. **A verdict can arrive on a `cancelled` run.** `conclusion=cancelled` with `state=not-run` still carries + `verdict_source=summary-file`: the agent wrote its summary file before the job was cancelled, and + housekeeping published it. "Run cancelled" is therefore **not** equivalent to "no verdict" — I had treated + them as the same thing and dispatched a fallback on that basis. +2. **A verdict can arrive by comment *edit*.** Any detection keyed to new-comment creation, or to `createdAt` + ordering, misses it. The `<!-- openhands-run: … -->` marker is the reliable machine anchor. + +**Corrected rule.** A verdict exists when a PR comment carries `OPENHANDS_VERDICT:` or an +`<!-- openhands-run: … -->` marker whose `run_id` matches the dispatch under consideration, **or** a +`[PHASE: IMPL-EVAL]` comment authored by a session other than this one. Match the marker's `head`/dispatch +identity, never the comment's position or `createdAt`. Re-fetch before concluding "no verdict" — the body is +mutable in place. + +**Cost of the error.** I launched an authorized native Opus fallback against a verdict that already existed. The +owner caught it and I stopped that agent before it posted anything, so exactly one verdict stands and no second +evaluation was published — but the token burn was real and avoidable, and the trigger was my own too-narrow +detection rule written one entry earlier. D-44's *substance* holds (gh-watch matched my own review comment as a +PASS; that remains a false authority); its *rule* is superseded by this entry. + +**What the verdict confirms.** F1/F2 match D-43 independently: F1 the closure defect, verified **introduced by +this PR** (base `agent-tools.generated.ts` had zero `snippet-extractor` occurrences); F2 names the asset-freshness +gate as answering the wrong question, the same finding I recorded against myself. The prepared correction brief +already covers both and needed no change beyond mapping its contracts to F1/F2. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md index ce95abb364..9447f674db 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md @@ -3,9 +3,20 @@ use harness # PR-D follow-up — the consumer bundle ships an unresolvable import You are continuing the **same leaf** (`fix/1549-quality-scan-provable-half`, PR #1596) in the **same worktree**. -This is a correction to work you already landed, dispatched after the automatic IMPL-EVAL on head `7264ce6aa` -went terminal. Everything already merged in this PR stands — the defect is one missing capability in the -consumer-bundle generator, plus the guard that makes the class non-recurring. +This is a correction to work you already landed. The formal IMPL-EVAL on head `7264ce6aa` returned +**`FAIL_FIX`** with exactly two findings, and this brief is scoped to closing both. Everything else in this PR +**stands** — the evaluator independently re-ran the rail and confirmed it green, including `quality:scan:repo` +exit 0 / findings 0 / `allowCount` **8** (10 → 8), `quality:scan` at 7, `arch:check` and `arch:check:repo` +exit 0, `docs:snippets` exit 0, 21 quality tests passing, the D-1 typed fixture free of casts/suppressions, and +the D-2 derivative idempotent. Do not revisit any of that. + +| Verdict finding | What it requires | Contract here | +| --- | --- | --- | +| **F1** HIGH — embedded consumer bundle incomplete wrt its own imports; `init-agent_test.ts` red; confirmed **introduced by this PR** (base had zero `snippet-extractor` occurrences) | third manifest category for bundled module dependencies + the durable import-closure regression guard with a negative control | **C1 + C2** | +| **F2** MEDIUM false-done — the asset-freshness gate answered the wrong question: `gen:assets-barrel` + empty `git status` proves currency wrt the manifest, not completeness wrt the bundle's imports, so F1 passed straight through it | the closure test is the durable fix and must land before merge | **C2 + C3** | + +The evaluator explicitly rules out inlining a second fence parser (it would violate R-10 and #1549 box 2), which +matches this brief. Do not revert the extractor consumption. Do **not** revert the extractor consumption. Consuming `.llm/tools/docs/snippet-extractor.ts` instead of forking a second fence parser is rail decision **R-10** and stays. From 756260911d689067c520f6f48b279b2e71834b88 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 19:09:22 +0200 Subject: [PATCH 50/57] docs(harness): map the correction brief to verdict findings F1/F2 Adds the required ## SKILL chapter (the launcher's brief contract rejected the first form) and records that the authoritative FAIL_FIX confirmed the rest of the rail green by independent execution, so the correction is scoped to F1 and F2 only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../slices/pr-d-1549/implement-closure.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md index 9447f674db..3e02cfdf85 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-d-1549/implement-closure.md @@ -21,6 +21,15 @@ matches this brief. Do not revert the extractor consumption. Do **not** revert the extractor consumption. Consuming `.llm/tools/docs/snippet-extractor.ts` instead of forking a second fence parser is rail decision **R-10** and stays. +## SKILL + +- `netscript-harness` — run artifacts, slice discipline, commit trail. +- `netscript-tools` — scoped validation wrappers; what is a verdict and what is not. +- `netscript-pr` — PR comment/label mechanics and the fenced `acceptance-evidence` block (the existing + `box-index: 1..7` mapping stays valid; do not rewrite it). +- `netscript-deno-toolchain` — deterministic `deno test`, task semantics, `deno doc` for surface questions. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + ## Identity | Field | Value | From 3c22aaa6e41a83971e4538440d8401ee8ef91382 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 19:12:34 +0200 Subject: [PATCH 51/57] docs(harness): record that re-syncing a leaf orphans the hashes its evidence cites D-46: the correction rebase put the branch on current main and orphaned the evaluated head 7264ce6aa, which PR #1596's body still cites in three places -- including the box-index 7 gate evidence that mirrors into #1549 and outlives the PR. The claim stays true; the pointer rots. An unverifiable gate evidence link is the same false-provenance class as the shallow-clone rebuttal, so the body's hashes must be refreshed to the corrected head before the mirror runs -- verified with `git merge-base --is-ancestor`, not by eye. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 8f0a5db1e6..0d47c9b826 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1569,3 +1569,37 @@ PASS; that remains a false authority); its *rule* is superseded by this entry. this PR** (base `agent-tools.generated.ts` had zero `snippet-extractor` occurrences); F2 names the asset-freshness gate as answering the wrong question, the same finding I recorded against myself. The prepared correction brief already covers both and needed no change beyond mapping its contracts to F1/F2. + +## D-46 — re-syncing a leaf orphans every commit hash its evidence cites (severity: moderate) + +The correction slice rebased `fix/1549-quality-scan-provable-half` onto current `main` +(`f542f31cb`), which is correct — the merge-base is now exactly `origin/main`. Side effect: the evaluated head +`7264ce6aa` and all three S3 commits are **orphaned**, and PR #1596's body still cites them in three places: + +```text +line 18 - [x] S3 … `bd95998fd9…`, `b82d2086ea…`, `7264ce6aac…` +line 22 Final head: `7264ce6aac21eecade916ea4b0332f5a1912e0c3` +line 68 box-index: 7 → "At final head 7264ce6aac21…: quality:scan:repo, quality:scan, arch:check … exit 0" +``` + +`git merge-base --is-ancestor 7264ce6aa HEAD` → **false**. + +**Why it matters rather than being cosmetic.** Box 7 is a `gate:` box. Its evidence mirrors into **#1549**, where +it outlives the PR. After merge, a reader checking that evidence would look up a commit that is not in the +branch's history and cannot verify the claim — the gate result becomes **unverifiable**, which is the same +false-provenance class as the shallow-clone rebuttal earlier in this run. The claim was true when made; the +pointer is what rots. + +**Generalisation:** evidence pinned to a commit hash has a lifetime bounded by the next history rewrite. Any +re-sync, rebase, or squash of a leaf invalidates every hash its PR body and acceptance evidence cite. This lane +already had one case of a head moving under a verdict (PR-B, where another lane's `update-branch` moved it and +the owned-path diff had to be shown empty); this is the same hazard reached from the leaf's own side. + +**Action, owned by the orchestrator, not the implementer:** after the correction lands and the final head is +fixed, update the PR body's `## Slices` hashes, `Final head:`, and the `box-index: 7` evidence to the corrected +head **before** the single authorized evaluation generation and before the mirror runs. Doing it after the mirror +would leave #1549 permanently citing an orphan. Verify with `git merge-base --is-ancestor <cited> HEAD` for every +hash the body cites — not by eye. + +**Sequencing note:** this repair changes only the PR **body**, never the head, so it does not disturb the +evaluated tree or consume an evaluation generation. From 85a235892a25d37214facc22b49fbe7018b7af59 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 19:30:01 +0200 Subject: [PATCH 52/57] docs(harness): record the flake-consumed evaluation and my own poller bug D-47: the authorized generation for head a1010e314 fired correctly and delivered no verdict -- toolchain bootstrap died on `socket hang up` fetching Deno, the same flake that killed scaffold-static 45 minutes earlier. "Generation spent" and "evaluation delivered" are different facts and only the second licenses a merge; a policy counting triggers rather than verdicts eventually merges on a flake. Re-entered the documented label lifecycle once, since a bootstrap that never ran the agent duplicates nothing. D-48: my replacement poller paired a marker from one comment with a verdict from another and reported FAIL_PLAN from an unrelated comment against a still-running run -- the same conflation I had documented one entry earlier. Parse per-comment; never grep two facts from a concatenated blob and treat them as related. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 0d47c9b826..ecffcd2c38 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1603,3 +1603,61 @@ hash the body cites — not by eye. **Sequencing note:** this repair changes only the PR **body**, never the head, so it does not disturb the evaluated tree or consume an evaluation generation. + +## D-47 — the authorized evaluation generation was consumed by a network flake (severity: significant) + +The single authorized evaluation generation for corrected head `a1010e314` fired correctly (one trigger comment, +dispatch `31622678606` success) and then **produced no verdict**: + +```text +<!-- openhands-run: {"run_id":31622691685,"attempt":1,"conclusion":"failure","state":"not-run", + "verdict":"NONE","verdict_source":"none"} --> +## OpenHands Agent — Did not run +Toolchain bootstrap failed, so the agent never ran. +This is a workflow failure, not a task verdict. +``` + +Root cause, from the run log — **the same flake that killed `scaffold-static` 45 minutes earlier**: + +```text +Downloading Deno from https://github.com/denoland/deno/releases/download/v2.9.5/… +socket hang up → wait 13s → socket hang up → wait 10s → ##[error]Error: socket hang up +``` + +Two independent job failures in one hour from `socket hang up` against GitHub-hosted downloads. Neither is a +task outcome; both are transport. + +**Decision and its justification.** Re-entered the documented label lifecycle **once** to obtain the evaluation. +A bootstrap failure that never started the agent is not a duplicate evaluation — there was no evaluation to +duplicate — and the label cycle is the documented rerun mechanism, not a manual dispatch. The alternatives were +both worse: merging with no independent verdict on the corrected head would be **self-certification** (the +harness's one unconditional prohibition), and the only verdict on record, `FAIL_FIX`, was against the superseded +head `7264ce6aa`. + +**Distinction worth preserving:** "the evaluation generation was spent" and "an evaluation was delivered" are +different facts, and only the second one licenses a merge decision. A `state:not-run` / `verdict:NONE` marker is +the machine-readable form of "spent, not delivered". Any policy that counts triggers rather than verdicts will +eventually merge on a flake. + +## D-48 — my own verdict poller reproduced the exact bug I had just documented (severity: moderate) + +Having recorded in D-45 that verdict detection must anchor on the `<!-- openhands-run -->` marker, I wrote a +poller that concatenated **all** matching comment bodies and then grepped the marker and the verdict +**independently**, each with `tail -1`. It paired a marker from one comment with a verdict from another and +reported: + +```text +TERMINAL_FOR_NEW_GENERATION: OPENHANDS_VERDICT: FAIL_PLAN +``` + +`FAIL_PLAN` came from an unrelated older comment; the marker it was paired with said `"conclusion":"running"`. +The exit condition also matched any `3162…` run id, so it fired on a run that was still in progress. + +That is the same defect class as D-42/D-44 — a detector conflating independent fields and reporting a confident +verdict that does not exist — committed by me **immediately after** writing the rule against it. Four instances +now in one afternoon: `gh-watch` three times, my own poller once. + +**Rule:** parse per-comment and keep fields from a single comment together; require the marker's `run_id` to +equal the generation under consideration **and** its `conclusion` to be terminal **and** the verdict to be in +**that same** comment body. Never grep two facts out of a concatenated blob and treat them as related — that is +not a detection heuristic, it is a coincidence generator. From b725f24632a2af5c759704de7a8bc41edfdc0146 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 19:56:52 +0200 Subject: [PATCH 53/57] docs(harness): record the transport flake window and its recovery rules D-49: `socket hang up` killed scaffold-static, the evaluator's setup-deno bootstrap, and close-gate inside 75 minutes. All three died before producing any work product, which is what separates them from task outcomes. Rules confirmed: read a red as transport only when the job produced nothing; recover with `gh run rerun` never a push, since a push invalidates the PASS bound to the head; `--failed` is refused while any job in the run is still in flight; and verify partial mutation (#1549 was 0/7 ticked) before re-running a job that runs after the mirror. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index ecffcd2c38..1eb756691e 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1661,3 +1661,30 @@ now in one afternoon: `gh-watch` three times, my own poller once. equal the generation under consideration **and** its `conclusion` to be terminal **and** the verdict to be in **that same** comment body. Never grep two facts out of a concatenated blob and treat them as related — that is not a detection heuristic, it is a coincidence generator. + +## D-49 — a GitHub transport flake window cost three job outcomes in 75 minutes (severity: moderate) + +Extends D-47. `Error: socket hang up` killed three unrelated jobs between 16:40Z and 17:54Z: + +| Time | Job | What died | Consequence | +| --- | --- | --- | --- | +| 16:40:56Z | `scaffold-static (deno-only)` on PR #1596 | job setup | a red check on a head I then deliberately did not re-run, because the correction was going to produce a new head anyway | +| 17:27:57Z | `OpenHands Agent` bootstrap, run `31622691685` | `setup-deno` downloading `deno-x86_64-unknown-linux-gnu.zip`, 3 attempts | **consumed the authorized evaluation generation** and delivered `verdict:NONE` | +| 17:53:58Z | `close-gate` on the ready-merge re-run | job step | blocked merge readiness; **#1549 boxes confirmed 0/7 ticked**, so the mirror never ran and no partial state needed repair | + +All three died **before doing any work**, which is what makes them cleanly separable from task outcomes — and why the +middle one self-describes as "a workflow failure, not a task verdict". + +**Operational rules this window confirms.** + +1. **Check whether a red is transport before reading it as a finding.** The distinguishing evidence is that the + job produced no work product: no test output, no scan result, no verdict body. +2. **Re-run, never push.** A push would have moved the head and invalidated the IMPL-EVAL `PASS` bound to + `a1010e314`. Every recovery here used `gh run rerun`. +3. **`gh run rerun --failed` is refused while any job in the run is still in flight** ("This workflow is already + running"), so recovery waits on the whole run, not just the failed job. +4. **Verify partial mutation before re-running a mutating job.** `close-gate` runs *after* the acceptance mirror, + so a mid-run death could have left #1549 half-ticked. Confirming 0/7 first is what made a plain re-run safe + rather than a guess. + +No code, plan, or evidence defect is implicated in any of the three. From 66e2f884e18cce5e63da13b1ceb171f18d6dca5c Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 20:08:24 +0200 Subject: [PATCH 54/57] docs(harness): close the 0.0.6 internals lane -- merge 6, assumptions, retrospective Records PR #1596 merged at c7fc45318 closing #1549, the fourth and last owned issue. All four (#1380, #1403, #1549, #1566) are CLOSED/COMPLETED, milestone 0.0.6, exactly one status:shipped each. Verified on merged main: quality:scan:repo exit 0 allowCount 8, arch:check 0, arch:check:repo 0, import-closure guard 3/3. Four more falsified assumptions (A-7..A-10): a clean regenerated asset does not prove the generated surface sound; a cancelled evaluator run can still carry a verdict; a spent generation is not a delivered evaluation; evidence pinned to a commit hash dies at the next history rewrite. Retrospective finalised: seven brief errors caught by escalation (three of them test-command permissions I wrote without running), four false verdict signals from my own supervision loop including one that matched my own review comment as the evaluator's PASS, and the transport flake window that consumed an authorized evaluation generation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../cut-trace.md | 5 + .../retrospective.md | 98 +++++++++++++++++-- 2 files changed, 95 insertions(+), 8 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 1c58f9de76..3911221702 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,6 +8,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | +| 6 | 2026-08-12 18:04:40Z | `c7fc45318` | **#1596** | **#1549** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **32 checks, 0 current failures** at `a1010e314`; `close-gate` green after **two** `socket hang up` deaths recovered by `gh run rerun` (never a push, which would have invalidated the verdict); **7/7** boxes mirrored from `box-index` evidence; `review-threads` PASS; all three scaffold tiers **executed and passed**. Verified **on merged main** `c7fc45318`: `quality:scan:repo` exit 0 `allowCount` **8**, `arch:check` 0, `arch:check:repo` 0, closure guard 3/3. **Three evaluation generations, one verdict:** `FAIL_FIX` (`31618732014`, authoritative for superseded head `7264ce6aa`), `NONE` (`31622691685` — `setup-deno` died on `socket hang up`, **generation spent, evaluation not delivered**), `PASS` (`31622903944`, `verdict_source: pr-comment`, bound to `a1010e314`). The authorized native fallback was stopped before posting, so no second verdict exists for any head. | | 5 | 2026-08-12 ~16:12Z | `eb373db29` | **#1585** | **#1380** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **26 checks, 0 current failures** at `0b34371c2`; **13/13** boxes mirrored; `review-threads` PASS. Verified on merged main: **`arch:check:repo` exit 0**, closing an entry `DEBT_ACCEPTED` red since **2026-06-21**. Two eval cycles: `FAIL_FIX` on a false-done provenance claim the orchestrator **wrongly rebutted** from a shallow clone, then `PASS` at `0b34371c2` after correction. | | 4 | 2026-08-12 ~14:33Z | `e391f3aec` | **#1570** | **#1403** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **22 checks, 0 current failures** at `807d29003`; `close-gate` green after label + workflow re-run; **8/8** issue boxes mirrored from `box-index` evidence; `review-threads` PASS. Verified **on merged main**: `arch:check` exit 0 **at 36 roots** (was green only at 16), `quality:scan:repo` exit 0. Evaluator provenance preserved on proof: `PASS` at `c740ff6e0`, head moved to `807d29003` by another lane's `update-branch`, owned-path diff **empty**, so the evaluated implementation is the shipped implementation. | | 3 | 2026-08-12 13:40Z | `b79eca5d6` | **#1567** | **#1566** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` 0 current failures at `f6def9946`; `close-gate` PASS; 6/6 issue boxes mirrored; the one PR-body DoD box untickable at hand-off ticked by the orchestrator with run identities as evidence; `review-threads` PASS. **Merged by the release coordinator during a Claude 529 outage**, independently re-verified here: main contains it, `phase-eval-status.mjs` is present on trusted main, both items `status:shipped`. Verdict was head-matched to `f6def9946` before consumption (one trigger `gen=29340872564`, one verdict run `31599209037`). | @@ -56,4 +57,8 @@ needed. | A-3 | An issue's measured counts stay valid for the few days between filing and implementation | implicit in working from an issue body | **falsified, twice** | `arch:check:repo` FAIL moved 53 → **55** in four days, and `quality:scan:repo` moved from green to **RED exit 1** — the latter making one of #1378's own acceptance boxes unsatisfiable as written. Both found by executing, neither visible in the issue. | | A-4 | A doctrine verdict row that names a missing directory was renamed or deleted | #1380 D6 ("plausibly renamed into the `plugin-*-core` tier") | **falsified for 5 of 6 rows** | `git log --all --diff-filter=A` shows `packages/{streams,triggers,workers,sagas}` and `plugins/hello-world` **never existed** in this repo; only `@netscript/shared` ever did (`0ef13de35 chore: genesis eject`). A third state — *authored against a layout that never landed* — is required, and a rename note would have fabricated provenance. | | A-6 | A `cancelled` expensive gate on a PR means the gate did not run | pre-merge gate check 4 read naively | **falsified, and it nearly manufactured a false red** | `scaffold-runtime` and `scaffold-runtime-sqlite` first evaluated as `cancelled`. Applying the #1142 rule — only the latest run per check name — resolved both to real `success` (`08:26:44Z`, `08:22:52Z`). Blocking on the cancelled pair would have been the mirror image of check 4's false greens. | +| A-7 | A regenerated-and-clean generated asset proves the generated surface is sound | this lane's own gate 7, written after D-22 | **falsified** | `gen:assets-barrel` + empty `git status` passed **twice** while the installed consumer bundle could not resolve its own import. Currency wrt the manifest ≠ completeness wrt imports. Found by CI (`init-agent_test.ts:580`), confirmed by IMPL-EVAL F2, recorded D-43. The gate is two gates; only executing the installed bundle catches it. | +| A-8 | A cancelled evaluator run means no verdict was produced | my own inference, acted on | **falsified** | `31618732014` was `conclusion:cancelled, state:not-run` and still carried a complete `FAIL_FIX` via `verdict_source: summary-file`, published by **editing an existing comment in place**. My `[PHASE: IMPL-EVAL]` query returned 0 twice, so I dispatched an unnecessary fallback. Detection must anchor on the `<!-- openhands-run -->` marker and re-fetch. D-45. | +| A-9 | A spent evaluation generation is an evaluation | implicit in "exactly one generation, then merge" | **falsified** | `31622691685` fired correctly and delivered `verdict:NONE` — `setup-deno` died on `socket hang up`. "Generation spent" and "evaluation delivered" are different facts; only the second licenses a merge. A policy counting triggers rather than verdicts eventually merges on a flake. D-47. | +| A-10 | Evidence pinned to a commit hash stays valid for the life of the issue | implicit in every `box-index` evidence line citing a head | **falsified** | The correction rebase orphaned **all six** hashes PR #1596's body cited, including box 7's `gate:` evidence that mirrors into #1549 and outlives the PR. The claim stayed true; the pointer rotted. Any re-sync invalidates every hash the body cites — verify with `git merge-base --is-ancestor`, not by eye. D-46. | | A-5 | The escalate-don't-idle brief instruction changes behaviour | `agent-milestone-orchestrator` § Delegation (4 idle-at-red-gate occurrences in 0.0.4) | **confirmed** | PR-A's agent hit a red Gate 1, diagnosed it as a pre-existing permission gap, refused to weaken unrelated tests, recorded it, escalated, and continued unblocked work. `drift.md` D-8. | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md b/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md index a094ba90f0..e0c5ee262a 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md @@ -1,13 +1,17 @@ # Retrospective — 0.0.6 chores/internals lane -Written at lane close by the orchestrating session. Draft completed while PR #1596's IMPL-EVAL was running; -final merge figures filled in at close. Measured, not celebratory: the useful output of a run like this is the -list of things it got wrong early enough to fix, plus the assumptions it destroyed. +Written at lane close by the orchestrating session. Measured, not celebratory: the useful output of a run like +this is the list of things it got wrong early enough to fix, plus the assumptions it destroyed. + +**Final state.** Six PRs merged, seven issues closed (#1436, #1415, #1530, #1566, #1403, #1380, #1549), all +directly against `main`, no umbrella implementation. The four owned issues (#1380, #1403, #1549, #1566) are each +`CLOSED/COMPLETED`, milestone 0.0.6, exactly one `status:shipped`. 49 drift entries. Verified on merged `main` +`c7fc45318`: `quality:scan:repo` exit 0 `allowCount` **8**, `arch:check` 0, `arch:check:repo` 0, import-closure +guard 3/3. ## What the lane actually delivered -Six PRs closing seven issues, all against `main`, no umbrella implementation. The three results worth naming -are the ones that changed what a gate *means* rather than adding one: +The three results worth naming are the ones that changed what a gate *means* rather than adding one: - **`main`'s `code-quality-repo` went green after nine consecutive red push runs** (PR-E/#1530). It had been red long enough to be background noise, which is the state in which a gate stops being read. @@ -22,6 +26,17 @@ are the ones that changed what a gate *means* rather than adding one: The last point is the lane's thesis in miniature: **a gate that reports success without executing is worse than a red one**, because a red gate gets investigated and a false green gets trusted. +And then the lane's final PR proved the thesis against the lane itself. My asset-freshness gate — +`gen:assets-barrel` then assert an empty `git status` — **passed twice, including as an idempotence proof, while +the installed consumer bundle could not resolve its own import**. The result was true and answered the wrong +question: it proves the barrel is *current with respect to the manifest*, never that the bundle is *complete with +respect to its own imports*. CI caught it (`init-agent_test.ts:580`), the formal IMPL-EVAL named it independently +as F2, and the fix is the durable one — a generic import-closure guard with a negative control, so the next tool +that grows an import is caught without anyone remembering. + +**The barrel gate is two gates, and I had only ever written the first.** Currency and closure. Only closure +executes what a consumer actually does. + `allowCount` fell **10 → 8** on PR-D — the first downward movement of the run. Every prior change to the allowance budget was upward or flat. A budget that only ratchets up is not a budget. @@ -44,7 +59,7 @@ The aggravating factor is that I posted the rebuttal *before* checking, so for 2 confident, false provenance written by the session holding merge authority. Withdrawing it publicly was the minimum, not a remedy. -## My own supervision loop produced two false greens (D-42) +## My own supervision loop produced four false verdict signals (D-42, D-44, D-45, D-48) While auditing other people's gates for reporting success without executing, `agentic:gh-watch` did exactly that to me twice inside five minutes — once rejecting a `--` separator, once exiting `4` on a missing token — @@ -55,11 +70,35 @@ was reading the *pipeline's* status. I had the real exit code in my own log and to the evaluated head, which is also the fix for the separate defect where `gh-watch` reported a *superseded* verdict as terminal in 0s, three times (D-24). +It got worse before it got better, and the pattern is mine rather than the tool's: + +| # | Signal | What it actually was | +| --- | --- | --- | +| D-42a | `gh-watch` "success" | `--` separator rejected; the watch never started | +| D-42b | `gh-watch` "success" | exit **4**, no token; masked because I read a pipeline's status | +| D-44 | `gh-watch` "TERMINAL PASS after 724s" | **my own `[PHASE: REVIEW] [VERDICT: PASS]` comment**, matched as the evaluator's verdict, with zero IMPL-EVAL comments on the PR | +| D-48 | my replacement poller "TERMINAL FAIL_PLAN" | a marker from one comment paired with a verdict from **another**, on a run still `"conclusion":"running"` | + +D-44 is the dangerous one: it manufactured *authority*. Consumed, I would have run the pre-merge gate believing +an independent evaluator had passed the head, and merged **on my own certification** — the harness's single +unconditional prohibition — with a reproducible consumer-facing defect in the diff. What made it catchable was +the owner reporting the run as cancelled, which contradicted the watcher. On its own it was entirely plausible: +724s is a believable duration and PASS a believable verdict. + +D-48 is the humbling one. I wrote it **immediately after** recording the rule against exactly that error. Four +false verdict signals in one afternoon, three from a tool and one from the replacement I built to be more careful. +The common defect is a detector that conflates independent fields and then reports confidence. + +**Rule that survived:** parse per-comment; require the marker's `run_id`, its terminal `conclusion`, and the +verdict to come from the **same** comment body, and the evaluated head to equal the head under consideration. +Never grep two facts out of a concatenated blob and treat them as related — that is not a heuristic, it is a +coincidence generator. + ## What the delegation model got right **Briefing the gate as a deliverable, with an explicit instruction to escalate rather than idle, worked — and what it caught was mostly me.** Escalation found the orchestrator's brief or plan wrong rather than the code -**six** times: +**seven** times: | # | What the implementer caught | Consequence had they idled or complied | | --- | --- | --- | @@ -69,11 +108,16 @@ what it caught was mostly me.** Escalation found the orchestrator's brief or pla | 4 | Rail sequencing incoherent: the 36-root expansion (PR-B) is what surfaces A14's false positives, but A14 was scheduled in PR-C | **would have shipped a red `arch:check`** | | 5 | #1436's prescribed fix was a no-op — `\b` was already present and *is* the cause, since `-` is a non-word character | a "fix" that changed nothing, with tests written to pass | | 6 | PR-D's brief simultaneously required the generator's `git status` clean, forbade touching the file it regenerates, required all docs companions scanned, and forbade fixing what that surfaced | self-contradictory; no compliant implementation exists | +| 7 | The correction brief's "generator's own suite" command was not a suite selector — it swept **724** tests, two of which need `--allow-net` | exit 1 on unrelated pre-existing tests; either a false red or a silent substitution of a narrower command | -Six for six, the escalation was correct and I amended. An implementation lane that stops and says "your brief +Seven for seven, the escalation was correct and I amended. An implementation lane that stops and says "your brief is wrong" is worth more than one that produces something plausible from a contradictory spec — and #5 is the clearest case: complying would have produced a green test suite around a fix that fixed nothing. +**Three of the seven were permissions or targets on a test command I wrote without running it.** That is not bad +luck, it is a specific habit: I authored gate commands from memory of what the repo's commands look like. Verifying +a command before briefing it costs seconds; each of these cost a round trip and one risked a false red. + Case 3 has an uncomfortable coda: I recorded the asset-barrel coupling as D-22 after it cost PR-E a cycle, then **copied the gate into PR-D's brief without copying the boundary**, reproducing the contradiction I had already paid to learn. Writing a lesson down is not the same as applying it. @@ -96,6 +140,23 @@ already paid to learn. Writing a lesson down is not the same as applying it. - **Distinguish a flake from a defect before touching anything.** PR-D's `scaffold-static` failure was `Error: socket hang up` — infrastructure. Re-run, do not investigate the diff, and do not push. +## Infrastructure is a first-class failure mode, and it cost three outcomes + +A `socket hang up` window between 16:40Z and 17:54Z killed three unrelated jobs — `scaffold-static`, the +evaluator's `setup-deno` bootstrap, and `close-gate`. All three died **before producing any work product**, which +is exactly what separates transport from a finding: no test output, no scan result, no verdict body. + +The middle one is the expensive one: it **consumed the single authorized evaluation generation** and delivered +`verdict:NONE`. That forced a distinction worth keeping — **"the generation was spent" and "an evaluation was +delivered" are different facts, and only the second licenses a merge decision.** A policy that counts triggers +rather than verdicts eventually merges on a flake. + +Recovery rules that held: read a red as transport only when the job produced nothing; recover with `gh run rerun`, +**never a push**, because a push moves the head and invalidates the verdict bound to it; `--failed` is refused +while any job in the run is still in flight; and check for partial mutation before re-running a job that runs +*after* the acceptance mirror (#1549 was confirmed 0/7 ticked, which is what made a plain re-run safe rather than +a guess). + ## Assumptions this run destroyed 1. **"`status:ready-merge` does not trigger `ci.yml`"** — false, and I had propagated it. `labeled` *is* in @@ -106,6 +167,13 @@ already paid to learn. Writing a lesson down is not the same as applying it. was *every PR in this rail*. Cited as coverage, it is a false green. 4. **"One evaluator pass per head is enough"** — true only if the head does not move. Any required fix invalidates the verdict. +5. **"A regenerated, clean generated asset proves the generated surface is sound"** — false. It proves currency + against the manifest, nothing about import closure. The shipped guard is what makes it true going forward. +6. **"A cancelled evaluator run produced no verdict"** — false. `conclusion:cancelled, state:not-run` still + carried a complete `FAIL_FIX` via `verdict_source: summary-file`, published by **editing an existing comment + in place**. My detection rule missed it and I dispatched an unnecessary fallback as a result. +7. **"Evidence pinned to a commit hash stays valid"** — false past the next history rewrite. A rebase orphaned + all six hashes the PR body cited, including gate evidence that mirrors into the issue and outlives the PR. ## What I would do differently @@ -117,3 +185,17 @@ already paid to learn. Writing a lesson down is not the same as applying it. - **When a lesson is recorded, apply it to the *next* brief mechanically**, not from memory — D-22 was written down and still reproduced two PRs later. - **Read tool output, not task notifications.** Two false greens in five minutes came from that single habit. +- **Re-fetch before concluding a thing does not exist.** "No verdict" was wrong because the verdict arrived by + editing a comment I had already read. An absence proved by one stale query is not an absence, and here it cost + an unnecessary fallback dispatch. +- **Verify a command before briefing it as a gate.** Three of seven brief errors were test-command permissions or + targets written from memory. + +## What I would keep unchanged + +- **Escalate-don't-idle in every brief.** Seven corrections, all of them mine, none discovered at merge time. +- **Re-execute the decisive claim rather than relaying it.** Every substantive defect in this lane — the no-op + fix, the coverage defects, the closure gap — was found by running something, never by reading it. +- **Re-evaluate after a required fix, and bind the verdict to a head.** It caught a red close-gate on PR #1560 the + one time it was tested, and it is the reason the flake window could not smuggle a stale PASS into a merge. +- **`box-index` over exact box text**, and pre-validating the mapping locally before spending a CI round trip. From 1f7546bfab98d8358868ecb58f259193daa05526 Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 20:34:32 +0200 Subject: [PATCH 55/57] docs(harness): dispatch PR-G for #1612 with the sweep already measured Records PLAN-EVAL N/A for a one-line prose fix, the independently reproduced main red at 6aee2b414, and the repo-wide sweep result: exactly one occurrence exists, with none latent outside the guard's published-closure scope, so acceptance box 3 rests on evidence rather than assertion. Two brief decisions are about not defeating the guard: a backticked issue ref would pass because inline code spans are stripped, and is forbidden; and the reword must add no factual claim, since the preceding sentence already states the whole mechanism. Also records that `git worktree add -b <branch> origin/main` makes the leaf track main, so a bare push would have hit main -- caught by the launcher's git-safety probe and fixed with `git branch --unset-upstream` before launch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../drift.md | 56 +++++++ .../leak-report.md | 7 + .../slices/pr-g-1612/codex-thread-ids.md | 16 ++ .../slices/pr-g-1612/implement.md | 148 ++++++++++++++++++ 4 files changed, 227 insertions(+) create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/leak-report.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/codex-thread-ids.md create mode 100644 .llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/implement.md diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index 1eb756691e..cc868f2e17 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1688,3 +1688,59 @@ middle one self-describes as "a workflow failure, not a task verdict". rather than a guess. No code, plan, or evidence defect is implicated in any of the three. + +## D-50 — PR-G / #1612 dispatched: `main` red on the published-JSDoc codename guard + +New leaf taken after lane closeout, on owner instruction. **PLAN-EVAL: N/A** — a single-line prose fix against a +deterministic guard; recording the waiver here rather than running ceremony. + +**Verified independently before briefing**, on current `main` `6aee2b414`: + +```text +deno test --allow-all .llm/tools/fitness/check-public-jsdoc-codenames_test.ts + Error: Published JSDoc contains internal codenames: + packages/cli/.../netscript-web-runtime-closure.ts:6 #1589 + FAILED | 3 passed | 1 failed +``` + +**The sweep, done by the orchestrator so the brief is not a discovery exercise.** The guard only scans published +*entrypoint closures*, so an occurrence in a file outside one is latent — real but not yet caught. I applied the +guard's own `INTERNAL_CODENAME` class to **every** `.ts`/`.tsx` under `packages` and `plugins` with closure +membership ignored: + +```text +repo-wide JSDoc findings (guard class, closure-membership ignored): 1 + packages/cli/.../netscript-web-runtime-closure.ts:6 #1589 +``` + +**Exactly one occurrence repo-wide**, and no latent ones. Acceptance box 3 is therefore satisfiable by evidence +rather than by assertion, and the leaf's scope is genuinely one line. + +**Two decisions recorded in the brief, both about not defeating the guard.** + +1. A backticked `` `#1589` `` *would* pass, because the scanner strips inline code spans before matching. That is + forbidden here: it keeps an unresolvable internal pointer in published JSR output while satisfying the letter + of the guard. The fix is the mechanism reword with no issue reference in the published comment. +2. The reword must introduce **no new factual claim**. The comment already states the full reason in the + preceding sentence (fresh-ui's SDK imports are limited to desktop/auto-update, which import no cache or query + modules, and it imports no Fresh runtime), so `in #1589` is a dangling pointer on a complete explanation. + Guard-clean but factually wrong published JSDoc would be a worse outcome than the current red, so the brief + instructs escalate-and-stop if the implementer thinks the mechanism differs. + +**Launch safety catch worth keeping.** `launch-codex-slice` refused the first attempt: + +```text +FAIL git-safety: worktree has upstream 'origin/main' — a bare push could corrupt it + (push-safety requires NONE; push via explicit refspec) +``` + +`git worktree add -b <branch> origin/main` sets the new branch to **track `origin/main`**, so a bare `git push` +from the slice would have targeted `main` directly. Unset with `git branch --unset-upstream` before launch. This +is a real hazard the launcher caught for me, not a formality — and it argues for creating leaf branches with the +upstream explicitly unset as a standing step. + +**Impl-eval decision deferred, with its condition stated in advance:** an `impl-eval:skip` is defensible for a +one-line prose change proven by a deterministic guard plus an exhaustive sweep — but only if the delivered wording +adds no claim beyond what the surrounding comment and this file's code already support, which the orchestrator can +verify directly at slice review. If the implementer introduces a new mechanism claim, the automatic evaluation +runs instead. Deciding after reading the diff, not before. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/leak-report.md b/.llm/runs/release-0.0.6-internals--orchestration/leak-report.md new file mode 100644 index 0000000000..a097409668 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/leak-report.md @@ -0,0 +1,7 @@ +# Run resource leak report + +Generated: 2026-08-12T18:08:37.588Z +Worktree: `/home/codex/repos/ns006-scanrail` +Aspire probe: ok +Docker probe: ok +No surviving Aspire resources found. diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/codex-thread-ids.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/codex-thread-ids.md new file mode 100644 index 0000000000..81286da1db --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/codex-thread-ids.md @@ -0,0 +1,16 @@ +# pr-g-1612 — Codex implementation thread +- **Thread / session id:** `019ff740-c0bb-7342-b1ef-9877a73b90cb` +- **Rollout:** `/home/codex/.codex/sessions/2026/08/12/rollout-2026-08-12T20-34-03-019ff740-c0bb-7342-b1ef-9877a73b90cb.jsonl` +- **Worktree:** `/home/codex/repos/ns006-jsdoc` +- **Branch:** `fix/1612-published-jsdoc-codename` @ `6aee2b414` (NO upstream by design). +- **Push rule:** explicit refspec only — `git push origin HEAD:refs/heads/fix/1612-published-jsdoc-codename`. +- **Requested route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Observed route:** provider=openai · model=gpt-5.6-sol · effort=low +- **Route verdict:** matched +- **Runtime:** approval=never · sandbox=dangerFullAccess +- **Brief (staged):** `/home/codex/pr-g-1612-brief.md` +## Steering (same thread — never a second send-message-v2 at this worktree) +```bash +codex exec resume 019ff740-c0bb-7342-b1ef-9877a73b90cb -- "<follow-up>" +``` +_Written by `.llm/tools/agentic/codex/launch-codex-slice.ts`._ \ No newline at end of file diff --git a/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/implement.md b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/implement.md new file mode 100644 index 0000000000..bed0b5b259 --- /dev/null +++ b/.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/implement.md @@ -0,0 +1,148 @@ +use harness + +# PR-G — #1612: `main` is red on the published-JSDoc codename guard + +You are the implementation agent for a **one-line documentation fix that is currently breaking +`deno task test` on `main`**. The defect is fully characterised and the sweep is already done; your job is the +reword plus the evidence, not investigation. + +Your orchestrator is a Claude Opus 5 high session in `/home/codex/repos/netscript-006-internals`. It holds merge +authority. + +## SKILL + +- `netscript-harness` — run artifacts, slice discipline, commit trail. +- `netscript-tools` — scoped validation wrappers; what is a verdict and what is not. +- `netscript-pr` — branch/PR/label mechanics, closing keywords, the fenced `acceptance-evidence` block. +- `netscript-deno-toolchain` — deterministic `deno test`, task semantics. +- `rtk` — prefix read-heavy `git`/`gh`/`grep`. + +## Identity + +| Field | Value | +| --- | --- | +| Worktree | `/home/codex/repos/ns006-jsdoc` (fresh, created for this leaf) | +| Branch | `fix/1612-published-jsdoc-codename` | +| Base | `6aee2b414` = `origin/main` at dispatch | +| Slice dir | `.llm/runs/release-0.0.6-internals--orchestration/slices/pr-g-1612/` | +| Closes | #1612 | +| Route | Codex · gpt-5.6-sol · **low** | +| PLAN-EVAL | **N/A** — single-line prose fix against a deterministic guard. Recorded by the orchestrator. | + +## The defect — reproduced, do not re-investigate + +`packages/cli/src/kernel/domain/dependency-closures/netscript-web-runtime-closure.ts:6`: + +```text + * runtime. A second fresh-ui instance therefore cannot own the cache-provider singleton in #1589. +``` + +The guard (`.llm/tools/fitness/check-public-jsdoc-codenames_test.ts`, shipped by #1554) fails on current `main`: + +```text +Error: Published JSDoc contains internal codenames: +packages/cli/src/kernel/domain/dependency-closures/netscript-web-runtime-closure.ts:6 #1589 +FAILED | 3 passed | 1 failed +``` + +This is the guard **working**, not a false positive: a bare `#1589` renders on JSR and a consumer cannot resolve +it. #1554 defines the class as internal workstream names, wave/epic codenames, **and issue numbers** in published +JSDoc. + +## The sweep is already complete — box 3 is measured, not yours to discover + +I ran the guard's own class over **every** `.ts`/`.tsx` under `packages` and `plugins`, ignoring closure +membership (so it covers files the guard does not yet reach because they are outside a published entrypoint +closure, where an occurrence would be latent): + +```text +repo-wide JSDoc findings (guard class, closure-membership ignored): 1 + packages/cli/src/kernel/domain/dependency-closures/netscript-web-runtime-closure.ts:6 #1589 +``` + +**Exactly one occurrence exists repo-wide, and it is this one.** Re-run the sweep yourself to confirm and to +produce your own evidence for box 3 — do not take my number on trust — but do not go looking for a second site. + +## Contract + +### C1 — the reword states the mechanism, and introduces no new claim + +The current comment **already states the complete reason** in the sentence before the defect: + +> `@netscript/fresh-ui` is intentionally excluded: its SDK imports are limited to desktop and auto-update, whose +> implementations do not import cache or query modules, and it imports no Fresh runtime. + +So the trailing `in #1589` is a dangling pointer on an already-complete explanation. Replace it with +consumer-facing wording that says **why** a second `fresh-ui` instance cannot own the cache-provider singleton — +the mechanism being that it never imports the cache or query modules, so it cannot instantiate a second +cache-provider at all. + +**Do not introduce a factual claim that is not already supported by the surrounding comment or the code in this +file.** If you believe the accurate mechanism differs from the above, say so and stop rather than writing prose +you cannot substantiate — published JSDoc that is merely guard-clean but wrong is a worse outcome than the +current red. + +Keep it short. This is a doc comment, not an essay. + +### C2 — do not defeat the guard instead of satisfying it + +A backticked `` `#1589` `` **would** pass, because inline code spans are stripped before scanning. **Do not use +that escape here.** It keeps an unresolvable internal pointer in published output and satisfies the letter of the +guard while missing its purpose. The issue itself offers it only as an option for maintainer value; the +orchestrator's decision is the mechanism reword, with no issue reference in the published comment at all. + +Equally: do not add the file to any exclusion list, do not weaken `INTERNAL_CODENAME`, and do not touch the guard +or its fixtures. + +### C3 — the guard passes and nothing else regresses + +`deno test --allow-all .llm/tools/fitness/check-public-jsdoc-codenames_test.ts` → **4 passed / 0 failed**. + +## Acceptance mapping + +#1612 has **3** boxes. Read them live. Use a fenced `acceptance-evidence` block with **`box-index: 1..3`** — not +exact box text. Box text wrapping is unmatchable by exact text and has already cost this lane a failed IMPL-EVAL +cycle on PR #1560. + +Box 2 says the guard passes "on `main`" — before merge you can only prove it on this branch at the merge base. +State it that way honestly: green on this branch over base `6aee2b414`. The orchestrator re-verifies on merged +`main` after the squash and records that. + +## Gates — paste real output with exit codes + +| # | Gate | Command | +| --- | --- | --- | +| 1 | the codename guard | `deno test --allow-all .llm/tools/fitness/check-public-jsdoc-codenames_test.ts` — **4 passed** | +| 2 | repo-wide sweep | your own re-run of the guard class over `packages` + `plugins`; report the count and paste it | +| 3 | the file still type-checks | `deno check --unstable-kv packages/cli/src/kernel/domain/dependency-closures/netscript-web-runtime-closure.ts` | +| 4 | scoped fmt/lint | wrappers over the one file's root — a doc comment reflow must not leave `fmt` dirty | +| 5 | package tests unaffected | the CLI kernel tests that cover this module, if any; if none exist say so plainly rather than implying coverage | + +`deno task test` repo-wide is **not** your gate — it is a large suite and CI runs it. Do not claim it. + +## PR mechanics + +1. First commit is the slice-dir bootstrap; open the **draft PR** in that same session; comment per slice. +2. `## Scope` carries `Closes #1612` on its own line. Reference **#1554** without a closing keyword — it shipped + the guard, is closed, and is not defective. +3. Labels: `type:fix`, `area:cli`, `area:docs`, `priority:p1`, `status:impl`, milestone `0.0.6`. Exactly one + `status:`. +4. **Leave the PR draft.** The orchestrator owns the ready flip and the evaluation decision. +5. Resolve commit hashes in a separate step and paste literal values. +6. State gate claims as evidence with exit codes, never as buckets. + +## Boundaries + +- Touch **only** `packages/cli/src/kernel/domain/dependency-closures/netscript-web-runtime-closure.ts` and your + slice dir. +- Do **not** touch `.llm/tools/fitness/check-public-jsdoc-codenames_test.ts`, its fixtures, or any exclusion list. +- Do **not** fix unrelated prose, reflow untouched comments, or tidy the export lists in this file. +- Do **not** add `deno-lint-ignore`, `@ts-ignore`, `as any`, `as unknown as`, or `quality-allow:`. +- Do **not** merge, flip to ready, or apply `status:ready-merge` / `status:impl-eval` / `impl-eval:skip`. + +## Escalate instead of going idle + +If a contract here is wrong — in particular if the mechanism in C1 is not what the code actually does — write it +in your slice `drift.md`, post it as a PR comment, and stop. On this lane escalation has found the +orchestrator's brief or plan wrong **seven** times rather than the code, three of them gate commands I wrote +without running. Raising it is the expected behaviour. From e7ab15b12d6296d14df2a6f970ac5b20f1069b8e Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 20:55:18 +0200 Subject: [PATCH 56/57] docs(harness): record PR-G merge and the duplicate-evidence failure mode #1612 closed by 6b29d12ea; guard verified 4 passed / 0 failed on merged main, bare #1589 gone, repo-wide sweep 0. Unblocks #1599. D-51 records a failure mode new to this lane: a fenced acceptance-evidence block repeated in a phase comment as well as the PR body makes the mirror parse 6 entries for 3 boxes and fail on duplicates. box-index does not protect against this -- the mapping is per-box-per-document-set, not per-document. Caught by the local validateEvidenceMapping pre-flight before CI, now two-for-two. Also records the impl-eval:skip decision with its pre-committed condition and the verification of the prose against fresh-ui's import map, the PR-body claim the skip would have falsified, and resolving duplicate check names by the #1142 latest-run rule rather than by eye. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../cut-trace.md | 1 + .../drift.md | 46 +++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md index 3911221702..510407c829 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/cut-trace.md @@ -8,6 +8,7 @@ Root 0.0.6 orchestration owns canary payload computation and the stable cut. | # | Merged (UTC) | Commit on `origin/main` | PR | Issues closed | Pre-merge gate record | | --- | --- | --- | --- | --- | --- | +| 7 | 2026-08-12 18:54:05Z | `6b29d12ea` | **#1614** | **#1612** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **40 checks, 0 current failures, 0 pending** at `2ae64cc00`; `close-gate` green after one rerun; **3/3** boxes mirrored; `review-threads` PASS. **PLAN-EVAL N/A** and **`impl-eval:skip`** applied with written justification (owner-authorized), the skip condition fixed *before* reading the diff: the only evaluation surface was whether the new prose is true, verified against `packages/fresh-ui/deno.json` exposing only `auto-update`/`desktop` with no `sdk/cache`|`sdk/query` import. Skip honoured — zero trigger comments, zero markers. Verified **on merged main**: codename guard **4 passed / 0 failed** (was 3 passed / 1 FAILED), `git grep '#1589' -- packages plugins` empty, repo-wide sweep 0. Unblocks #1599 (3340/3340 green but this guard). | | 6 | 2026-08-12 18:04:40Z | `c7fc45318` | **#1596** | **#1549** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **32 checks, 0 current failures** at `a1010e314`; `close-gate` green after **two** `socket hang up` deaths recovered by `gh run rerun` (never a push, which would have invalidated the verdict); **7/7** boxes mirrored from `box-index` evidence; `review-threads` PASS; all three scaffold tiers **executed and passed**. Verified **on merged main** `c7fc45318`: `quality:scan:repo` exit 0 `allowCount` **8**, `arch:check` 0, `arch:check:repo` 0, closure guard 3/3. **Three evaluation generations, one verdict:** `FAIL_FIX` (`31618732014`, authoritative for superseded head `7264ce6aa`), `NONE` (`31622691685` — `setup-deno` died on `socket hang up`, **generation spent, evaluation not delivered**), `PASS` (`31622903944`, `verdict_source: pr-comment`, bound to `a1010e314`). The authorized native fallback was stopped before posting, so no second verdict exists for any head. | | 5 | 2026-08-12 ~16:12Z | `eb373db29` | **#1585** | **#1380** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **26 checks, 0 current failures** at `0b34371c2`; **13/13** boxes mirrored; `review-threads` PASS. Verified on merged main: **`arch:check:repo` exit 0**, closing an entry `DEBT_ACCEPTED` red since **2026-06-21**. Two eval cycles: `FAIL_FIX` on a false-done provenance claim the orchestrator **wrongly rebutted** from a shallow clone, then `PASS` at `0b34371c2` after correction. | | 4 | 2026-08-12 ~14:33Z | `e391f3aec` | **#1570** | **#1403** (`CLOSED/COMPLETED`) | 7/7 — `pr-checks` **22 checks, 0 current failures** at `807d29003`; `close-gate` green after label + workflow re-run; **8/8** issue boxes mirrored from `box-index` evidence; `review-threads` PASS. Verified **on merged main**: `arch:check` exit 0 **at 36 roots** (was green only at 16), `quality:scan:repo` exit 0. Evaluator provenance preserved on proof: `PASS` at `c740ff6e0`, head moved to `807d29003` by another lane's `update-branch`, owned-path diff **empty**, so the evaluated implementation is the shipped implementation. | diff --git a/.llm/runs/release-0.0.6-internals--orchestration/drift.md b/.llm/runs/release-0.0.6-internals--orchestration/drift.md index cc868f2e17..b5e4723a7e 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/drift.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/drift.md @@ -1744,3 +1744,49 @@ one-line prose change proven by a deterministic guard plus an exhaustive sweep adds no claim beyond what the surrounding comment and this file's code already support, which the orchestrator can verify directly at slice review. If the implementer introduces a new mechanism claim, the automatic evaluation runs instead. Deciding after reading the diff, not before. + +## D-51 — PR-G / #1612 merged; the duplicate-evidence trap caught locally this time + +Merged `6b29d12ea` at 18:54:05Z. `main` no longer fails the published-JSDoc codename guard: **4 passed / 0 +failed** on merged main, `git grep "#1589" -- packages plugins` empty, repo-wide sweep 0. #1612 +`CLOSED/COMPLETED`, single `status:shipped` (it had retained `status:triage` from filing). Unblocks **#1599**, +whose CI was 3340/3340 green except this guard. + +**The one real defect, and it was caught before CI this time.** close-gate first failed because the fenced +`acceptance-evidence` block existed **twice** — canonical in the PR body *and* repeated in the `[PHASE: IMPL]` +comment. The mirror parses the body **plus every comment**, so it saw **6 entries for 3 boxes** and failed on +duplicates: + +```text +parsed evidence entries: 6 +#1612 acceptance boxes: 3 +VALIDATION FAILED: box "No bare issue reference remains…" has duplicate evidence +``` + +This is a **new** failure mode for this lane, distinct from #1560's (which was exact-box-text mismatch, fixed by +`box-index`). `box-index` does not protect against duplication — the mapping is per-box-per-*document-set*, not +per-document. **A per-slice comment must not repeat the body's evidence block.** + +What made it cheap: running the mirror's own `validateEvidenceMapping` against the live PR **before** labeling, +the pre-flight established after #1560. It cost one command instead of a CI round trip, and it is now two-for-two +at catching mapping defects that CI would otherwise have found. + +**Impl-eval skip, recorded as a decision with a pre-committed condition.** Owner authorized documenting a skip +"if warranted". I fixed the condition before reading the diff: for a one-line prose change the only genuine +evaluation surface is *whether the prose is true*. The delivered reword — `in #1589` → "cannot **instantiate** a +cache-provider singleton" — introduces no claim, being entailed by the preceding sentence, and I verified it +against the package rather than the prose (`packages/fresh-ui/deno.json` exposes exactly `auto-update` and +`desktop`; no `sdk/cache`/`sdk/query` import anywhere in `fresh-ui`). Had it added an unsupported mechanism claim +the automatic evaluation would have run. Verified the label was honoured: zero trigger comments, zero markers. + +**Two smaller items worth keeping.** + +- **The PR body asserted something the merge would falsify.** It carried "Do not merge until the mandatory + separate-session IMPL-EVAL is complete" and a DoD box to match. Applying a skip without editing that would have + shipped a body contradicting what happened — pre-merge check 7 exists for exactly this. Corrected to record the + skip and its justification. +- **Duplicate check names resolved by rule, not by eye.** Every expensive check appeared twice at the head + (`skipped` + `success`) because `ci:skip-*` was applied after a run had already executed them. Under the #1142 + latest-run-per-name rule the head is 0 failures; taken from `pr-checks` rather than inspected manually, because + this lane already had a near-miss reading duplicate names naively (`cut-trace` A-6). The skipped duplicates are + **not** claimed as green. From bddad90c2da1e70f56650b056dfd9d6589094ace Mon Sep 17 00:00:00 2001 From: Rickylabs <eric.chautems@gmail.com> Date: Wed, 12 Aug 2026 22:27:23 +0200 Subject: [PATCH 57/57] docs(harness): finalise the 0.0.6 internals retrospective Final tallies: seven PRs merged, eight issues closed, 51 drift entries, every issue CLOSED/COMPLETED at milestone 0.0.6 with one status:shipped. Records the post-merge verification per claim, including the published-JSDoc guard going 3-passed-1-failed to 4-passed-0-failed on merged main, which unblocked #1599. Adds the two-failure-modes analysis of the acceptance mirror: exact-box-text brittleness (#1560) and duplicated evidence blocks (#1614) are different defects, box-index only fixes the first, and the local validateEvidenceMapping pre-flight is now two-for-two at catching both classes before CI. Adds the skip rationale for #1612 -- condition fixed before reading the diff, prose verified against fresh-ui's import map, the guard escape refused -- and a closing note on control PR #1553 including the status-taxonomy gap for a by-design non-merge record. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1uTFgh4emCPxSs7m72Pqf --- .../retrospective.md | 75 +++++++++++++++++-- 1 file changed, 70 insertions(+), 5 deletions(-) diff --git a/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md b/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md index e0c5ee262a..42f3ede1ca 100644 --- a/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md +++ b/.llm/runs/release-0.0.6-internals--orchestration/retrospective.md @@ -3,11 +3,19 @@ Written at lane close by the orchestrating session. Measured, not celebratory: the useful output of a run like this is the list of things it got wrong early enough to fix, plus the assumptions it destroyed. -**Final state.** Six PRs merged, seven issues closed (#1436, #1415, #1530, #1566, #1403, #1380, #1549), all -directly against `main`, no umbrella implementation. The four owned issues (#1380, #1403, #1549, #1566) are each -`CLOSED/COMPLETED`, milestone 0.0.6, exactly one `status:shipped`. 49 drift entries. Verified on merged `main` -`c7fc45318`: `quality:scan:repo` exit 0 `allowCount` **8**, `arch:check` 0, `arch:check:repo` 0, import-closure -guard 3/3. +**Final state.** **Seven PRs merged, eight issues closed** (#1436, #1415, #1530, #1566, #1403, #1380, #1549, +#1612), all directly against `main`, no umbrella implementation. Every one is `CLOSED/COMPLETED`, milestone 0.0.6, +with exactly one `status:shipped`. **51 drift entries.** Verified on merged `main`: + +| Claim | Verified at | Result | +| --- | --- | --- | +| `quality:scan:repo` | `c7fc45318` | exit 0, `allowCount` **8** (from 10) | +| `arch:check` / `arch:check:repo` | `c7fc45318` | exit 0 / exit 0 | +| consumer bundle import-closure guard | `c7fc45318` | 3 passed / 0 failed | +| published-JSDoc codename guard | `6b29d12ea` | **4 passed / 0 failed** (main was 3 passed / 1 **failed**) | + +The last row is the one that matters operationally: the lane's final leaf (#1612) took `main` from red to green on +`deno task test`, which unblocked #1599's otherwise-3340/3340 run. ## What the lane actually delivered @@ -199,3 +207,60 @@ a guess). - **Re-evaluate after a required fix, and bind the verdict to a head.** It caught a red close-gate on PR #1560 the one time it was tested, and it is the reason the flake window could not smuggle a stale PASS into a merge. - **`box-index` over exact box text**, and pre-validating the mapping locally before spending a CI round trip. + +## The acceptance mirror failed twice, in two different ways + +Worth separating, because fixing the first does nothing for the second and I initially assumed it would: + +| | #1560 | #1614 | +| --- | --- | --- | +| Symptom | mapping matched **0 of 6** boxes | mapping parsed **6 entries for 3** boxes | +| Cause | exact box text is brittle — `acceptanceCheckboxes` keeps only each box's **first raw line, backticks preserved**, so any box that wraps in the issue body is unmatchable, and the author cannot see the wrapping | the fenced block existed **twice** — canonical in the PR body *and* repeated in a `[PHASE: IMPL]` comment; the mirror parses the body **plus every comment** | +| Fix | `box-index: 1..N` | exactly one parseable block per issue, canonically in the PR body | +| Cost | a full failed IMPL-EVAL cycle | one close-gate rerun — caught by local pre-flight before CI | + +**`box-index` does not protect against duplication.** The mapping is per-box-per-*document-set*, not +per-document. Both rules are needed, and the standing form is: index the boxes, and never let a per-slice comment +repeat the body's evidence block. + +The local `validateEvidenceMapping` pre-flight — established after #1560 — is now **two for two** at catching +mapping defects before CI. It is the cheapest gate this lane added, and the only reason the second failure cost a +rerun rather than a cycle. Note the mirror's own `--dry-run` cannot substitute for it: it short-circuits on the +missing `status:ready-merge` label *before* parsing, so it validates nothing pre-label. + +## Skipping an evaluation, done as a decision rather than a shortcut + +The lane skipped exactly one IMPL-EVAL (#1612), under owner authorization to document a skip "if warranted". What +made it defensible was **fixing the condition before reading the diff**: for a one-line prose change the only real +evaluation surface is *whether the prose is true*, everything else being machine-checked by a deterministic guard +and an exhaustive sweep. + +The reword introduced no claim — it was entailed by the preceding sentence — and I verified it against the package +rather than the prose: `packages/fresh-ui/deno.json` exposes exactly `auto-update` and `desktop`, with no +`sdk/cache`/`sdk/query` import anywhere in `fresh-ui`. Had it added an unsupported mechanism claim, the automatic +evaluation would have run. + +Two details that keep a skip honest rather than convenient: + +- **The escape was available and refused.** A backticked `` `#1589` `` passes the guard, because the scanner strips + inline code spans. Using it would have satisfied the letter of the guard while leaving an unresolvable internal + pointer in published JSR output. Guard-clean is not the goal; resolvable published documentation is. +- **The PR body had to be corrected, not just labelled.** It asserted "do not merge until the mandatory + separate-session IMPL-EVAL is complete". Applying a skip without editing that would have merged a body + contradicting what happened — which is precisely what pre-merge check 7 exists to catch. + +## Closing note on the control PR + +This lane's control PR (**#1553**) carried orchestration evidence only — the tracked run dir — with **no** +implementation and **no** closing keyword, and was **closed unmerged by design**. Every issue closed through its +own leaf PR against `main`, which is what kept each merge independently revertible and each verdict bound to a +real implementation head. + +Worth flagging as a small taxonomy gap rather than silently picking a label: the `status:` lifecycle defines a +*completed* close (terminal `status:shipped`) and a *not-planned/wontfix* close (no `status:` label, "because it +did not ship"), and a by-design non-merge record is neither. Literally it shipped no code; in substance it did +exactly what it existed for. I applied `status:shipped` on the reasoning that the exception is scoped +specifically to not-planned/wontfix, and leaving it unlabelled would file a completed orchestration record +alongside abandoned work. The milestone stays `0.0.6` because that is the release it records. + +Release ownership sits with **ns006-fixes**. This lane dispatched no release and holds no publish authority.