From 7b5b13330a197d18d6191f022224b2cb3995cb5e Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 16:58:08 +0200 Subject: [PATCH 1/6] plan(release): lock #1377 reference gate scope --- .llm/runs/fix-1377-gate--leaf/context-pack.md | 77 ++++++++ .llm/runs/fix-1377-gate--leaf/drift.md | 18 ++ .llm/runs/fix-1377-gate--leaf/plan.md | 178 ++++++++++++++++++ .llm/runs/fix-1377-gate--leaf/research.md | 49 +++++ .llm/runs/fix-1377-gate--leaf/supervisor.md | 24 +++ .llm/runs/fix-1377-gate--leaf/worklog.md | 124 ++++++++++++ 6 files changed, 470 insertions(+) create mode 100644 .llm/runs/fix-1377-gate--leaf/context-pack.md create mode 100644 .llm/runs/fix-1377-gate--leaf/drift.md create mode 100644 .llm/runs/fix-1377-gate--leaf/plan.md create mode 100644 .llm/runs/fix-1377-gate--leaf/research.md create mode 100644 .llm/runs/fix-1377-gate--leaf/supervisor.md create mode 100644 .llm/runs/fix-1377-gate--leaf/worklog.md diff --git a/.llm/runs/fix-1377-gate--leaf/context-pack.md b/.llm/runs/fix-1377-gate--leaf/context-pack.md new file mode 100644 index 0000000000..4f520ba093 --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/context-pack.md @@ -0,0 +1,77 @@ +# Context Pack: #1377 gate half + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `fix-1377-gate--leaf` | +| Branch | `fix/1377-docs-reference-gate-scope` | +| Current phase | `plan-eval` pending | +| Archetype | 6 — CLI / tooling | +| Scope overlays | Docs | + +## Current State + +Phase 1 is plan-complete at baseline `fa5d0d411`. No implementation file has changed. The plan +chooses a four-entry alias map, separates reference existence from first-publish policy, and designs +a tree-derived 91-path root/direct-subcommand docs gate. A separate-session PLAN-EVAL must PASS +before implementation. + +## Completed + +- Required skills and harness policy read. +- `supervisor.md` created first. +- Carried research, live #1377, comments, merged PR-C #1541, reference index, release tool/tests, + docs checker/tests, and live public command catalog inspected. +- 35/35 alias-resolved reference arrival coverage verified. +- Plan, Design checkpoint, risk register, exact files, negative assertions, and commit gates locked. + +## In Progress + +- Commit/push plan and open draft PR at `status:plan`. + +## Next Steps + +1. Orchestrator launches fresh native Claude/Fable PLAN-EVAL and writes `plan-eval.md`. +2. Resolve any `FAIL_PLAN`; do not implement meanwhile. +3. Before S2, orchestrator resolves the four PR-C prose findings (`deploy start/stop/status/uninstall`) + or explicitly rescopes; never weaken the strict predicate. +4. On PASS, advance to `status:impl` and implement S1–S3 in order with per-slice review/push/comment. + +## Key Decisions + +| Decision | Source | Notes | +| --- | --- | --- | +| Alias map | `plan.md` D-1 | Four exact deployable-plugin aliases; no URL moves | +| Whole-set placement | `plan.md` D-2 | Directly after publish-set; separate evidence row | +| Release unblock | `plan.md` D-3 | Canonical page or explicit tracked stub; no bypass | +| Command coverage | `plan.md` D-4–D-6 | Materialized tree, colon-safe, 91 exact obligations | +| Prose boundary | `plan.md` D-7 | Finding for PR-C/orchestrator, not this agent's prose edit | + +## Files Changed + +Only `.llm/runs/fix-1377-gate--leaf/{supervisor,research,plan,worklog,context-pack,drift}.md` are +created in phase 1. + +## Gates + +| Gate family | Current status | Evidence | +| --- | --- | --- | +| Plan | READY_FOR_EVAL | Plan-Gate inputs complete | +| Static / Fitness | NOT_RUN | Phase 2 | +| Docs / Release | NOT_RUN | Phase 2 | +| Negative controls | NOT_RUN | Phase 2; raw exits mandatory | + +## Open Questions + +- None for implementation design. The four prose gaps are a recorded external sequencing + dependency. + +## Drift and Debt + +- Drift: one significant baseline finding, recorded in `drift.md`. +- Debt: none created. + +## Commits + +- See the draft PR commit list + per-slice phase comments after push. diff --git a/.llm/runs/fix-1377-gate--leaf/drift.md b/.llm/runs/fix-1377-gate--leaf/drift.md new file mode 100644 index 0000000000..dd155f1fc1 --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/drift.md @@ -0,0 +1,18 @@ +# Drift Log: #1377 gate half + +Append-only. + +## 2026-08-12 — strict direct-subcommand census finds four additional prose gaps + +- **What:** Executing the existing public command catalog and checking root/immediate-child paths + against the public CLI reference surfaces found four paths with no exact occurrence: + `netscript deploy start`, `netscript deploy stop`, `netscript deploy status`, and + `netscript deploy uninstall`. +- **Source:** `createPublicCommandRegistry()` materialized through `PublicCliCommandCatalog`; focused + search over `docs/site/reference/cli/commands.md`, `docs/site/cli-reference.md`, and `docs/site`. +- **Expected:** PR-C had landed all content required for the strict subcommand gate to be green. +- **Actual:** 87 of 91 root/direct-child paths have an exact occurrence; these four do not. +- **Severity:** significant. +- **Action:** defer to PR-C/orchestrator as a sequencing dependency. PR-D will neither author the + prose nor weaken the predicate. PLAN-EVAL must assess the boundary before implementation. +- **Evidence:** `research.md` finding 9; `plan.md` D-7. diff --git a/.llm/runs/fix-1377-gate--leaf/plan.md b/.llm/runs/fix-1377-gate--leaf/plan.md new file mode 100644 index 0000000000..d8df129112 --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/plan.md @@ -0,0 +1,178 @@ +# Plan: #1377 reference and CLI coverage gates + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `fix-1377-gate--leaf` | +| Branch | `fix/1377-docs-reference-gate-scope` | +| Phase | `plan` — selected PLAN-EVAL is a hard stop | +| Target | Release-readiness tooling and docs-accuracy tooling | +| Archetype | 6 — CLI / tooling (internal tooling application) | +| Scope overlays | Docs | + +## Archetype and Doctrine + +Archetype 6 is the closest fit because both owned production files are executable repository tools: +one is a release-readiness CLI and the other is a docs verification CLI. No `packages/cli` product +source changes are planned; the public tree is consumed through its existing registry/catalog +surface. The docs overlay applies to the path convention and command-reference coverage. + +The doctrine verdict labels `@netscript/cli` **Restructure**, but this slice does not deepen or +restructure that package. It reuses its existing registry-derived catalog rather than adding a +second command vocabulary. Relevant avoidance rules are AP-2 (parallel implementation), AP-9 +(stringly-typed finite vocabularies), AP-18 (opaque snapshots), AP-23 (command composition), and the +docs overlay's source-alignment and scope-separation rules. + +## Goal + +Make reference-page existence a visible release-readiness check over all 35 effective publish +members, align its path resolver with the convention PR-C recorded, and add a command-reference +coverage check derived from the materialized public command tree. Both required negative controls +must exit non-zero with package/command diagnostics. + +## Scope + +- Split reference-page existence out of `auditFirstPublishPackages` and run it directly over + `publishSet.effective`. +- Declare four exact package-to-page aliases matching `docs/site/reference/index.md`. +- Add tree-derived root/immediate-subcommand coverage to `docs:accuracy` with an asserted census. +- Add focused positive and negative tests and preserve raw non-zero negative-control evidence. +- Clarify the reference-index convention once so it describes the gate contract, not only observed + layout. + +## Non-Scope + +- Reference page content/export-map verification (#1108) or `check-exports-drift` expansion. +- README standard/staleness (#767), fenced-block extraction (#1374), installed-artifact proof + (#1343), package renames, URL moves, or command behavior changes. +- Authoring missing package pages or filling PR-C command prose. Missing content is reported to the + orchestrator. +- The unreachable deploy `emit` operation (#1544). + +## Locked Decisions + +| ID | Decision | Criterion and rationale | +| --- | --- | --- | +| D-1 | Use a declared four-entry alias map; do not move the IA. | Criterion: minimize irreversible public-URL and cross-gate change while making the release predicate agree with the already published site. Aliases change one resolver, preserve inbound links and `docs:accuracy`, and match PR-C's enumerated convention. Moving four URLs would create avoidable user-facing breakage and broader proof obligations. The inconsistency is bounded and explicit rather than inferred. | +| D-2 | Add a separate `docs-reference` readiness check immediately after `publish-set`. | `publishSet.effective` is the authoritative coordinated publish set. Placing the check here makes it independent of JSR registry discovery and prevents it being skipped with first-publish checks. `auditFirstPublishPackages` retains only README/tagline/license/export policy. If publish-set evidence is unavailable, `docs-reference` is explicitly `SKIP`; publish-set itself is already `FAIL`, so readiness cannot falsely pass. | +| D-3 | A missing page is release-blocking, with a bounded content escape hatch rather than a bypass flag. | A publishable member without a page turns `docs-reference` red. A maintainer unblocks by adding the canonical page resolved by the convention; when full prose cannot land before a cut, the release skill permits an explicit stub that names and links the tracked content follow-up. No ignore list, environment override, or silent exemption is added. #1108 remains responsible for content/export fidelity. | +| D-4 | Derive command obligations from the materialized public tree through `PublicCliCommandCatalog`; never parse `.command()` source and never hardcode verbs. | This executes the same tree users/MCP see and already walks every child. The checker selects all root entries plus each root entry's immediate children. That is the acceptance level that catches a new group subcommand while respecting compact documentation of deeper generated command families. | +| D-5 | Treat colon-form `ui:*` names as complete root command paths. | A tree path `ui:add` renders as `netscript ui:add`; it must not be rewritten to `netscript ui add`. The normal nested form remains `netscript `. | +| D-6 | Assert the exact audited census, not a neighboring or merely printed count. | Production evidence reports and asserts 91 root-or-immediate-child command paths on this baseline (15 roots plus 76 direct children). The test also asserts the derived path set; any public-tree growth changes the count and requires corresponding documentation. The count is computed from the tree and compared to the ratified floor, not used as a literal verb list. | +| D-7 | PR-D will not author four newly discovered deploy lifecycle prose entries. | The focused arrival check found `deploy start`, `deploy stop`, `deploy status`, and `deploy uninstall` absent as exact public command paths. PR-C owns prose. The orchestrator must land/authorize its content follow-up before the implementation slice can claim `docs:accuracy` green. The gate remains strict; it will not be weakened to hide these four paths. | +| D-8 | Raw negative controls invoke the exported gate functions one case at a time. | A unit test that expects a rejection exits zero and is insufficient evidence. Each control command seeds exactly one missing page/command, invokes the real exported audit, prints its diagnostic, and leaves the process at raw exit 1. Tests separately lock the diagnostic contract. | + +The reference-index change records D-1 once: normal scoped-name stripping plus exactly four declared +aliases. It does not rename or add pages. + +## Open-Decision Sweep + +| Decision | Status | Notes | +| --- | --- | --- | +| Path convention | resolved now | D-1 | +| Readiness placement and skip behavior | resolved now | D-2 | +| Missing-page release escape | resolved now | D-3 | +| Command-tree depth and `ui:*` normalization | resolved now | D-4/D-5 | +| Census assertion | resolved now | D-6 | +| Four prose gaps | resolved now, external dependency | D-7; do not start implementation if the strict baseline cannot be made green without violating scope | +| Deeper recursive command families | safe to defer | They are represented by compact family tables; a later structural Markdown grammar may expand from 91 to the full 149 without changing this slice's direct-subcommand acceptance. | +| Content correctness | safe to defer | Owned by #1108; this slice enforces existence only. | + +No unresolved decision remains that would force implementation rework. D-7 is a sequencing +dependency, not permission to modify the predicate. + +## Exact Files + +| File | Planned change | +| --- | --- | +| `.llm/tools/release/publish-readiness.ts` | Alias resolver, separate whole-publish-set audit/dependency/evidence row, remove docs check from first-publish audit. | +| `.llm/tools/release/publish-readiness_test.ts` | Ordered evidence update; 35/35-style whole-set positive; published-member missing-page negative; first-publish regression remains registry-scoped. | +| `.llm/tools/docs/check-accuracy-and-discoverability.ts` | Tree-derived command-reference coverage, colon-aware rendering, asserted census, `runAccuracyCheck()` integration. | +| `.llm/tools/docs/check-accuracy-and-discoverability_test.ts` | Synthetic public-tree positive and missing-direct-subcommand negative; colon-form control; census behavior. | +| `docs/site/reference/index.md` | One bounded convention sentence declaring that the four listed paths are gate aliases. No page content or URL move. | +| `.llm/runs/fix-1377-gate--leaf/{supervisor,research,plan,worklog,context-pack,drift}.md` | Harness decisions, evidence, per-slice results, resume state, and drift. | + +If implementation requires any other product or prose file, stop and rescope through the +orchestrator rather than silently widening the slice. + +## The Two Negative Tests + +1. **Missing reference page:** fixture has a member in `publishSet.effective`, registry metadata says + it is already published, and its resolved canonical page is absent. The report must contain + `docs-reference: FAIL`, package name, resolved path, rule `docs-reference`, and `ok=false`. The raw + one-case command must exit 1. +2. **Missing command-tree subcommand:** a synthetic materialized tree adds a direct child under an + otherwise documented group without adding reference prose. Coverage must identify the rendered + `netscript ` path, report the audited/covered counts, and fail. A separate + colon control proves `ui:add` stays colon-form. The raw one-case command must exit 1. + +## Commit Slices + +| # | What the slice proves | Files | Named gate before commit | +| --- | --- | --- | --- | +| P | The decision-locked plan is reviewable and implementation has not begun. | Run artifacts only. | Plan-Gate checklist review; separate-session PLAN-EVAL must return PASS before S1. | +| S1 | Every effective publish member is audited at the PR-C path convention, independently of registry/new-package state, and a missing page fails. | Release tool/test, reference index, run artifacts. | Focused `deno test --allow-all .llm/tools/release/publish-readiness_test.ts`; raw missing-page control exit 1; scoped release check/lint/fmt wrappers. | +| S2 | Every public root/direct-child command is derived from the live tree, colon-safe, census-asserted, and an undocumented child fails. | Docs checker/test, run artifacts. | Focused `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts`; raw missing-subcommand control exit 1; `rtk proxy deno task docs:accuracy`; scoped check/lint/fmt wrappers over owned tool roots. | +| S3 | The composed release/docs gate set stays green and leaves no incidental manifest/lock churn. | Run artifacts only unless reviewed fixes are required. | `docs:links`, `docs:accuracy`, `publish:dry-run`, repo tests, scoped release check/lint/fmt; immediate raw git status after dry-run. | + +Each implementation slice updates `worklog.md` and `context-pack.md`, receives a separate +opposite-family substantive review before its sign-off commit, then is pushed and commented on the +draft PR before the next slice. + +## Validation Plan + +| Order | Gate | Command / evidence | Expected result | +| --- | --- | --- | --- | +| 1 | Release unit | `deno test --allow-all .llm/tools/release/publish-readiness_test.ts` | PASS | +| 2 | Docs checker unit | `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts` | PASS | +| 3 | Missing-page raw negative | focused exported-audit command recorded verbatim in worklog | exit 1 with package + resolved page | +| 4 | Missing-command raw negative | focused exported-coverage command recorded verbatim in worklog | exit 1 with full rendered command path | +| 5 | Scoped type check | `deno run --allow-read --allow-run .llm/tools/run-deno-check.ts --root .llm/tools/release --ext ts,tsx` plus the same wrapper for `.llm/tools/docs` | PASS | +| 6 | Scoped lint | corresponding `run-deno-lint.ts` wrappers | PASS | +| 7 | Scoped format | corresponding `run-deno-fmt.ts` wrappers | PASS | +| 8 | Docs links | `rtk proxy deno task docs:links` | PASS | +| 9 | Docs accuracy | `rtk proxy deno task docs:accuracy` | PASS, strict command census asserted | +| 10 | Publish dry-run | `rtk proxy deno task publish:dry-run` | PASS; inspect status immediately, restore #1417 churn, no lock drift committed | +| 11 | Repo tests | `rtk proxy deno task test` | PASS | + +The release-gate class (`scaffold.runtime`/production E2E) is N/A: this changes a readiness +predicate but not publish shape, generated output, CLI behavior, DB/Aspire wiring, or an actual +release cut. + +## Risk Register + +| Risk | Mitigation | +| --- | --- | +| False red blocks a canary/stable cut. | Audit `publishSet.effective`, preserve alias resolver in one constant, retain independent evidence, test published and new members. | +| Registry outage suppresses reference coverage. | Place check before and outside registry discovery. | +| Alias drift creates another hidden convention. | Exact four-entry constant plus index table and tests for alias and name-exact `-core` paths. | +| Command gate is inert or literal-list based. | Consume materialized tree; raw synthetic child negative; assert 91 obligations, not only print it. | +| `ui:*` false negative. | Dedicated colon-form test and path renderer rule. | +| Compact nested command families create false reds. | Lock this slice to root/immediate-child acceptance and defer recursive family grammar explicitly. | +| Existing four prose gaps make strict gate red. | D-7 orchestrator dependency; do not weaken or author prose in this slice. | +| `publish:dry-run` rewrites manifests/lock. | Immediate raw status/diff and restore only identified incidental churn; never commit it. | + +## Arch-Debt Implications + +No new architecture debt is planned. Existing `@netscript/cli` restructuring debt is unchanged; +the slice consumes its established public composition/catalog seam. Any need for a literal command +allowlist or a release bypass would be plan drift and requires rescope, not a debt entry hidden in +the implementation. + +## Dependencies and Drift Watch + +- Hard dependency: separate-session PLAN-EVAL PASS. +- Sequencing dependency: the orchestrator resolves D-7 before strict `docs:accuracy` can be claimed + green without PR-D writing prose. +- Log drift if effective publish count differs from 35, command census differs from 91, aliases + differ from PR-C's four, another file hardcodes the path convention, or implementation needs a + third production file. + +## Deferred Scope + +- Full recursive 149-command structural coverage grammar. +- Reference content/export validation (#1108). +- Post-merge JSR landing-page observation; leave its #1377 box unticked and route verification to a + follow-up issue. +- Any acceptance-row mutation until implementation evidence exists. diff --git a/.llm/runs/fix-1377-gate--leaf/research.md b/.llm/runs/fix-1377-gate--leaf/research.md new file mode 100644 index 0000000000..21677d1572 --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/research.md @@ -0,0 +1,49 @@ +# Research — #1377 gate half + +## Re-baseline + +- Carried-in source: + `/home/codex/repos/netscript-006-docs/.llm/runs/release-0.0.6-docs--orchestration/slices/1377-content/research.md`, live issue #1377, and merged PR-C #1541. +- Re-derived against dispatched `main` baseline + `fa5d0d411054ba8aea272df392eb4e85b57c0d41` on 2026-08-12. +- PR-C merge commit `db1d79c68f7861a43902313f5d7a68274b0ef12a` is an ancestor of the baseline. +- The carried-in measurements remain authoritative except where the dispatch amendment supplied a + newer state. This run performed only the requested arrival checks and focused design inspection; + it did not redo the completed corpus audit. + +## Findings + +| # | Finding | How to verify | +| --- | --- | --- | +| 1 | The branch and baseline match dispatch, with no upstream assumed. | `git branch --show-current`; `git rev-parse HEAD` | +| 2 | PR-C recorded the current path convention in `docs/site/reference/index.md`: name-exact after stripping `@netscript/`, except four deployable-plugin aliases (`plugin-{sagas,streams,triggers,workers}` → short segment). | `docs/site/reference/index.md` § Page paths | +| 3 | Applying those four aliases gives 35/35 effective publish members a reference page at the baseline. | `auditPublishSet(Deno.cwd())` plus existence check; result `effective=35 present=35 missing=[]` | +| 4 | `docs-reference` is still inside `auditFirstPublishPackages`; `collectPublishReadiness` invokes that audit only with registry-absent `newPackages`. All effective members being published therefore leaves the check inert. | `.llm/tools/release/publish-readiness.ts:159-204,264-319` | +| 5 | Release readiness is composed and fail-closed. A new whole-publish-set reference check belongs directly after `publish-set`, before registry discovery; first-publish README/tagline/manifest/export checks remain registry-dependent. | `.agents/skills/netscript-release/SKILL.md`; `collectPublishReadiness()` | +| 6 | The existing `firstPublishFixture()` can seed a missing reference page. Tests can reuse it after reference existence is separated from first-publish policy. | `.llm/tools/release/publish-readiness_test.ts:369-415` | +| 7 | The live public command tree already has a recursive, tree-derived catalog: `PublicCliCommandCatalog` walks `getCommands()` and emits paths. No source parser or literal command list is needed. | `packages/cli/src/public/features/agent/mcp/cli-mcp-adapters.ts:7-51` | +| 8 | The live catalog currently has 15 root entries, 91 root-or-immediate-child entries, and 149 recursive entries. The five `ui:*` commands are colon-form root entries, not nested `ui` children. | focused `deno eval --unstable-kv` against `createPublicCommandRegistry()` and `PublicCliCommandCatalog` | +| 9 | A literal full-path search across the exhaustive and curated CLI pages finds 87/91 root-or-immediate-child paths. Four real immediate children (`deploy start`, `deploy stop`, `deploy status`, `deploy uninstall`) have no exact public-doc occurrence. This is a PR-C content finding, not permission for PR-D to write prose. | focused catalog evaluation plus `rg -F` over `docs/site`; see plan dependency D-7 | +| 10 | The command-reference checker is naturally part of `docs:accuracy`, whose implementation and tests already own textual command-policy checks. | `deno.json:82`; `.llm/tools/docs/check-accuracy-and-discoverability.ts` and `_test.ts` | +| 11 | PR-C's short sagas path remains consumed by `docs:accuracy`. Choosing aliases preserves it; moving the IA would widen scope and change published URLs. | `.llm/tools/docs/check-accuracy-and-discoverability.ts`; PR-C drift DR-3 | +| 12 | `publish:dry-run` is a required final gate but may rewrite catalog-backed manifests under #1417. Status must be inspected immediately and incidental manifest/lock churn restored without committing it. | dispatch boundary; `.agents/skills/netscript-tools/SKILL.md` lock hygiene | + +## jsr-audit surface scan + +- N/A: this slice changes repository release/docs tooling and tests, not a package/plugin export, + manifest, JSDoc surface, or publish payload. +- Release risk still applies: a false positive in the new reference check blocks canary and stable + readiness. The plan therefore preserves first-publish semantics and makes the whole-set check a + separate evidence row. + +## Open questions resolved by the plan + +- Alias map versus IA move: locked to a four-entry alias map. +- Missing-page unblock path: add the canonical page; a clearly marked stub linked to a tracking + issue is the bounded release escape hatch, while content/export fidelity remains #1108's gate. +- Command coverage level: root entries plus immediate child commands, derived from the materialized + public tree, with a census assertion over that exact set. Recursive grandchildren are deferred + because the reference intentionally documents generated deploy families compactly rather than as + every cross-product path. +- Four missing deploy lifecycle strings: orchestrator/PR-C follow-up dependency, not prose authored + by this implementation agent. diff --git a/.llm/runs/fix-1377-gate--leaf/supervisor.md b/.llm/runs/fix-1377-gate--leaf/supervisor.md new file mode 100644 index 0000000000..a057e347d5 --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/supervisor.md @@ -0,0 +1,24 @@ +# Supervisor Identity — fix-1377-gate--leaf + +Written at run start per `workflow/lane-policy.md` § Supervisor identity. + +| Field | Value | +| --- | --- | +| Model | Codex · OpenAI · GPT-5.6 Sol | +| Session | Current Codex implementation session (thread identifier not exposed) | +| Host | `YogaBook9i` · Linux/WSL · `codex` | +| Checkout | `/home/codex/repos/netscript-547-lffix` (shared Git common directory) | +| Worktree | `/home/codex/repos/ns006-1377-gate` | +| Branch | `fix/1377-docs-reference-gate-scope` | +| Baseline | `fa5d0d411054ba8aea272df392eb4e85b57c0d41` from `main`, dispatched 2026-08-12 | +| Run ID | `fix-1377-gate--leaf` | + +## Routes in force + +| Task lane | Provider / model / effort | Role in this run | +| --- | --- | --- | +| `normal_implementation` | Codex · OpenAI · GPT-5.6 Sol · medium | Plan author and, only after PLAN-EVAL PASS, implementation agent | +| `review_codex` | Claude · Anthropic · Fable 5 · low | Opposite-family slice review / IMPL-EVAL route selected by the orchestrator | +| formal PLAN-EVAL | Fresh native Claude · Anthropic · Fable 5 · medium | Separate-session plan evaluator selected and launched by the orchestrator | + +Reference `.llm/harness/workflow/lane-policy.md`; no lane overrides are in force. diff --git a/.llm/runs/fix-1377-gate--leaf/worklog.md b/.llm/runs/fix-1377-gate--leaf/worklog.md new file mode 100644 index 0000000000..caf624c87f --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/worklog.md @@ -0,0 +1,124 @@ +# Worklog: #1377 gate half + +## Run Metadata + +| Field | Value | +| --- | --- | +| Run ID | `fix-1377-gate--leaf` | +| Branch | `fix/1377-docs-reference-gate-scope` | +| Archetype | 6 — CLI / tooling | +| Scope overlays | Docs | + +## Design + +Recorded before implementation. PLAN-EVAL is selected and implementation is forbidden until a +separate session writes `plan-eval.md` with `PASS`. + +### Public Surface + +- `collectPublishReadiness()` — gains an ordered `docs-reference` evidence row over the effective + publish set. +- `auditReferencePages()` (planned name) — exported pure filesystem audit used by readiness, tests, + and raw negative evidence. +- `auditFirstPublishPackages()` — preserves README/tagline/license/export first-publish policy and + no longer owns reference existence. +- `checkPublicCommandReference()` (planned name) — exported docs-policy audit accepting a command + tree/catalog and reference Markdown, invoked by `runAccuracyCheck()`. +- No new end-user CLI command, option, package export, or page URL. + +### Domain Vocabulary + +- `REFERENCE_PAGE_ALIASES` — exact package-name → path-segment exceptions. +- `ReferencePageViolation` — package, resolved path, rule, message (may reuse/rename the existing + violation shape without widening semantics). +- `CommandReferenceResult` — audited count, documented count, missing rendered command paths. +- `CommandPath` — token lineage from the materialized Cliffy tree; colon-form root ids remain one + token. +- `DIRECT_COMMAND_DEPTH` — root plus one child, the coverage boundary for this slice. + +### Ports + +- `auditPublishSet` dependency — authoritative publish membership. +- Existing `exists` filesystem seam — reference-page presence. +- Existing `PublicCliCommandCatalog` / enumerable Cliffy command seam — live public command tree. +- Markdown text input — injected in tests; production reads the exhaustive public command reference + (and any explicitly declared companion source needed by the ratified predicate). +- No new network, registry, filesystem abstraction, or source parser. + +### Constants + +- `REFERENCE_PAGE_ALIASES` — four entries: sagas, streams, triggers, workers. +- `EXPECTED_PUBLIC_DIRECT_COMMAND_COUNT` — `91`, asserted against the exact root/direct-child set. +- `NETSCRIPT_COMMAND_PREFIX` — `netscript` for rendered diagnostics. +- Existing gate ids plus new `docs-reference`; no duplicate string list of commands. + +### Command Surface / Composition Contract + +- Composition authority stays + `packages/cli/src/public/features/root/public-command-tree.ts`. +- Enumeration reuses `PublicCliCommandCatalog`; the docs checker does not import group factories or + scrape `.command()` source. +- Current vertical-feature catalog, spine abstracts, registries, adapters, and ports are unchanged; + this is a consumer of the existing tree, not an Archetype-6 restructuring slice. +- No layer-2 abstract or generated output is introduced. + +### Commit Slices + +| # | Slice | Gate | Files | +| --- | --- | --- | --- | +| P | Decision-locked plan and Design checkpoint | Separate-session PLAN-EVAL | Run artifacts | +| S1 | Whole-publish-set path coverage and alias convention | Release unit + raw negative + scoped release wrappers | Release tool/test, index, run artifacts | +| S2 | Tree-derived direct-subcommand coverage | Docs unit + raw negative + docs accuracy + scoped docs wrappers | Docs checker/test, run artifacts | +| S3 | Composed merge-readiness evidence and lock hygiene | Docs links/accuracy, publish dry-run, repo tests, scoped wrappers | Run artifacts only unless reviewed fix required | + +### Deferred Scope + +- Full recursive command-family expansion — direct-subcommand acceptance catches the documented + defect without inventing a Markdown DSL for generated grandchildren. +- Reference content fidelity, README checker, snippet extractor, installed-artifact proof, deploy + operation reachability, and post-merge JSR observation — owned by their cited issues. + +### Contributor Path + +To add a publishable package, add its canonical reference page at the scoped-name segment unless it +is one of the four declared aliases; run publish readiness. To add a public root/direct child +command, register it in the public tree and document the rendered `netscript …` path; the tree census +and coverage diagnostic name any missing path. Do not edit a command list in the gate. + +## Progress Log + +| Time | Slice | Step | Notes | +| --- | --- | --- | --- | +| 2026-08-12 | P | Bootstrap | Wrote `supervisor.md` first; loaded named skills and harness policy. | +| 2026-08-12 | P | Arrival checks | Baseline exact; PR-C ancestor; alias convention yields 35/35 reference coverage. | +| 2026-08-12 | P | Design | Locked alias, check placement, release unblock, command-tree scope, census, and negative controls. | + +## Decisions + +| Decision | Reason | Source | +| --- | --- | --- | +| Four aliases, no IA move | Preserve public URLs and agree with PR-C | `plan.md` D-1 | +| Separate whole-set readiness check | Prevent registry/new-package skip | `plan.md` D-2 | +| Page/stub with follow-up is unblock; no bypass | Release gate needs a bounded repair path | `plan.md` D-3; release skill | +| Live tree, root/direct children, colon-safe | Predicate must be executable and nonliteral | `plan.md` D-4–D-6 | +| Four deploy prose gaps return to orchestrator | PR-C owns prose; strict gate must not be weakened | `plan.md` D-7 | + +## Gate Results + +All implementation gates are `NOT_RUN` by design in phase 1. + +| Gate | Result | Evidence / notes | +| --- | --- | --- | +| Baseline identity | PASS | branch `fix/1377-docs-reference-gate-scope`; head `fa5d0d411…` | +| PR-C landed | PASS | `db1d79c68…` is an ancestor of HEAD | +| Reference arrival coverage | PASS | 35 effective, 35 present under locked alias resolver, 0 missing | +| PLAN-EVAL | PENDING | Must be a fresh native Claude/Fable session; this generator cannot self-evaluate | +| Type/lint/fmt/docs/publish/tests | NOT_RUN | Phase 2 only after PLAN-EVAL PASS | +| Raw negative controls | NOT_RUN | Phase 2 only; both raw exit codes and diagnostics required | + +## Handoff Notes + +- Evaluator should challenge D-4/D-6 first: whether 91 is the correct direct-subcommand surface and + whether compact deeper command families are safely deferred. +- Evaluator should verify D-7 is treated as a real dependency, not an excuse to reduce coverage. +- No implementation file has been edited. From 5ba4bc339bff5084a313e2a0afa69592a9de959b Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 17:15:54 +0200 Subject: [PATCH 2/6] chore(harness): route plan evaluation to the automatic evaluator --- .llm/runs/fix-1377-gate--leaf/context-pack.md | 2 +- .llm/runs/fix-1377-gate--leaf/supervisor.md | 4 ++-- .llm/runs/fix-1377-gate--leaf/worklog.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.llm/runs/fix-1377-gate--leaf/context-pack.md b/.llm/runs/fix-1377-gate--leaf/context-pack.md index 4f520ba093..5c9d1bda49 100644 --- a/.llm/runs/fix-1377-gate--leaf/context-pack.md +++ b/.llm/runs/fix-1377-gate--leaf/context-pack.md @@ -32,7 +32,7 @@ before implementation. ## Next Steps -1. Orchestrator launches fresh native Claude/Fable PLAN-EVAL and writes `plan-eval.md`. +1. Orchestrator triggers the automatic status-driven PLAN-EVAL via the status-label mechanism and records the verdict. 2. Resolve any `FAIL_PLAN`; do not implement meanwhile. 3. Before S2, orchestrator resolves the four PR-C prose findings (`deploy start/stop/status/uninstall`) or explicitly rescopes; never weaken the strict predicate. diff --git a/.llm/runs/fix-1377-gate--leaf/supervisor.md b/.llm/runs/fix-1377-gate--leaf/supervisor.md index a057e347d5..737d474d79 100644 --- a/.llm/runs/fix-1377-gate--leaf/supervisor.md +++ b/.llm/runs/fix-1377-gate--leaf/supervisor.md @@ -18,7 +18,7 @@ Written at run start per `workflow/lane-policy.md` § Supervisor identity. | Task lane | Provider / model / effort | Role in this run | | --- | --- | --- | | `normal_implementation` | Codex · OpenAI · GPT-5.6 Sol · medium | Plan author and, only after PLAN-EVAL PASS, implementation agent | -| `review_codex` | Claude · Anthropic · Fable 5 · low | Opposite-family slice review / IMPL-EVAL route selected by the orchestrator | -| formal PLAN-EVAL | Fresh native Claude · Anthropic · Fable 5 · medium | Separate-session plan evaluator selected and launched by the orchestrator | +| `review_codex` | Claude · Anthropic · Opus 5 · low | Fable prohibited this milestone; this is the declared in-plan fallback in `routing-policy.ts:441` | +| formal PLAN-EVAL | Automatic status-driven evaluator (MiniMax M3) | Triggered by the orchestrator through the `openhands` + `status:plan-eval` label pair. The orchestrator does not select, launch, or resume an evaluator model. | Reference `.llm/harness/workflow/lane-policy.md`; no lane overrides are in force. diff --git a/.llm/runs/fix-1377-gate--leaf/worklog.md b/.llm/runs/fix-1377-gate--leaf/worklog.md index caf624c87f..c9a05ca674 100644 --- a/.llm/runs/fix-1377-gate--leaf/worklog.md +++ b/.llm/runs/fix-1377-gate--leaf/worklog.md @@ -112,7 +112,7 @@ All implementation gates are `NOT_RUN` by design in phase 1. | Baseline identity | PASS | branch `fix/1377-docs-reference-gate-scope`; head `fa5d0d411…` | | PR-C landed | PASS | `db1d79c68…` is an ancestor of HEAD | | Reference arrival coverage | PASS | 35 effective, 35 present under locked alias resolver, 0 missing | -| PLAN-EVAL | PENDING | Must be a fresh native Claude/Fable session; this generator cannot self-evaluate | +| PLAN-EVAL | PENDING | Must be the automatic status-driven plan evaluator; this generator cannot self-evaluate | | Type/lint/fmt/docs/publish/tests | NOT_RUN | Phase 2 only after PLAN-EVAL PASS | | Raw negative controls | NOT_RUN | Phase 2 only; both raw exit codes and diagnostics required | From 706c2bf05cd1a10268c7b9659fab5e7804e80fdf Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 18:12:17 +0200 Subject: [PATCH 3/6] plan(release): resolve #1377 plan-eval findings --- .llm/runs/fix-1377-gate--leaf/context-pack.md | 32 ++++----- .llm/runs/fix-1377-gate--leaf/drift.md | 19 +++++ .llm/runs/fix-1377-gate--leaf/plan.md | 69 ++++++++++--------- .llm/runs/fix-1377-gate--leaf/research.md | 10 +-- .llm/runs/fix-1377-gate--leaf/worklog.md | 20 +++--- 5 files changed, 90 insertions(+), 60 deletions(-) diff --git a/.llm/runs/fix-1377-gate--leaf/context-pack.md b/.llm/runs/fix-1377-gate--leaf/context-pack.md index 5c9d1bda49..fa35710b06 100644 --- a/.llm/runs/fix-1377-gate--leaf/context-pack.md +++ b/.llm/runs/fix-1377-gate--leaf/context-pack.md @@ -12,10 +12,12 @@ ## Current State -Phase 1 is plan-complete at baseline `fa5d0d411`. No implementation file has changed. The plan -chooses a four-entry alias map, separates reference existence from first-publish policy, and designs -a tree-derived 91-path root/direct-subcommand docs gate. A separate-session PLAN-EVAL must PASS -before implementation. +Fallback PLAN-EVAL returned `FAIL_PLAN` on `5ba4bc339`; the plan is revised and no implementation +file has changed. It retains the confirmed alias and whole-publish-set design, locks command +coverage to the exact union of `docs/site/reference/cli/commands.md` and +`docs/site/cli-reference.md`, requires structural root matching plus tokenized path resolution and +exact equality with 91 root/direct obligations, and makes the four missing deploy rows bounded S2 +scope. A new automatic PLAN-EVAL on this immutable head must PASS before implementation. ## Completed @@ -24,19 +26,18 @@ before implementation. - Carried research, live #1377, comments, merged PR-C #1541, reference index, release tool/tests, docs checker/tests, and live public command catalog inspected. - 35/35 alias-resolved reference arrival coverage verified. -- Plan, Design checkpoint, risk register, exact files, negative assertions, and commit gates locked. +- FAIL_PLAN B1–B3 and non-blocking findings resolved in the revised plan. ## In Progress -- Commit/push plan and open draft PR at `status:plan`. +- Commit/push the revised plan; stop for a new automatic PLAN-EVAL. ## Next Steps -1. Orchestrator triggers the automatic status-driven PLAN-EVAL via the status-label mechanism and records the verdict. -2. Resolve any `FAIL_PLAN`; do not implement meanwhile. -3. Before S2, orchestrator resolves the four PR-C prose findings (`deploy start/stop/status/uninstall`) - or explicitly rescopes; never weaken the strict predicate. -4. On PASS, advance to `status:impl` and implement S1–S3 in order with per-slice review/push/comment. +1. Orchestrator triggers the automatic status-driven PLAN-EVAL via the status-label mechanism and + records the verdict. +2. Resolve any further `FAIL_PLAN`; do not implement meanwhile. +3. On PASS, the orchestrator may advance the lifecycle; implement S1–S3 in order only after resume. ## Key Decisions @@ -45,8 +46,8 @@ before implementation. | Alias map | `plan.md` D-1 | Four exact deployable-plugin aliases; no URL moves | | Whole-set placement | `plan.md` D-2 | Directly after publish-set; separate evidence row | | Release unblock | `plan.md` D-3 | Canonical page or explicit tracked stub; no bypass | -| Command coverage | `plan.md` D-4–D-6 | Materialized tree, colon-safe, 91 exact obligations | -| Prose boundary | `plan.md` D-7 | Finding for PR-C/orchestrator, not this agent's prose edit | +| Command coverage | `plan.md` D-4–D-8 | Materialized tree, two-page union, structural/tokenized match, colon-safe, exact 91 obligations | +| Prose unblock | `plan.md` D-9 | Four bounded deploy lifecycle rows are S2 scope | ## Files Changed @@ -64,12 +65,11 @@ created in phase 1. ## Open Questions -- None for implementation design. The four prose gaps are a recorded external sequencing - dependency. +- None for implementation design; no external content predecessor remains. ## Drift and Debt -- Drift: one significant baseline finding, recorded in `drift.md`. +- Drift: PLAN-EVAL correction recorded append-only in `drift.md`. - Debt: none created. ## Commits diff --git a/.llm/runs/fix-1377-gate--leaf/drift.md b/.llm/runs/fix-1377-gate--leaf/drift.md index dd155f1fc1..7e49dece0e 100644 --- a/.llm/runs/fix-1377-gate--leaf/drift.md +++ b/.llm/runs/fix-1377-gate--leaf/drift.md @@ -16,3 +16,22 @@ Append-only. - **Action:** defer to PR-C/orchestrator as a sequencing dependency. PR-D will neither author the prose nor weaken the predicate. PLAN-EVAL must assess the boundary before implementation. - **Evidence:** `research.md` finding 9; `plan.md` D-7. + +## 2026-08-12 — PLAN-EVAL corrects corpus scope, unblock ownership, and path-consumer premise + +- **What:** The first plan left the command corpus implicit, treated four missing deploy rows as an + ownerless external dependency, and inherited a false claim that the accuracy checker hardcodes + the short sagas reference path. +- **Source:** fallback PLAN-EVAL `FAIL_PLAN` on `5ba4bc339`; independently confirmed corpus counts + 25/25/4 and checker search. +- **Expected:** A decision-complete plan whose S2 can reach green and whose alias decision rests on + true consumers. +- **Actual:** `commands.md` and `cli-reference.md` each miss 25/91 while their union misses 4/91; + no open content owner existed; `check-accuracy-and-discoverability.ts` has no sagas path. +- **Severity:** significant. +- **Action:** fix. Lock the two-page union, ratify it in the reference index, let S2 add the four + bounded rows, require structural longest-path matching and exact 91 equality, and restate the + alias rationale without the retracted premise. This entry supersedes the earlier action that + deferred the four rows externally; the original finding remains as historical provenance. +- **Evidence:** revised `research.md` findings 9/11 and `plan.md` D-1/D-4/D-6–D-9 on the + superseding plan head. diff --git a/.llm/runs/fix-1377-gate--leaf/plan.md b/.llm/runs/fix-1377-gate--leaf/plan.md index d8df129112..547dd80349 100644 --- a/.llm/runs/fix-1377-gate--leaf/plan.md +++ b/.llm/runs/fix-1377-gate--leaf/plan.md @@ -36,35 +36,39 @@ must exit non-zero with package/command diagnostics. - Split reference-page existence out of `auditFirstPublishPackages` and run it directly over `publishSet.effective`. - Declare four exact package-to-page aliases matching `docs/site/reference/index.md`. -- Add tree-derived root/immediate-subcommand coverage to `docs:accuracy` with an asserted census. +- Add tree-derived root/immediate-subcommand coverage to `docs:accuracy` over the locked two-page + command corpus with an exact asserted census. - Add focused positive and negative tests and preserve raw non-zero negative-control evidence. -- Clarify the reference-index convention once so it describes the gate contract, not only observed - layout. +- Clarify the reference index once for both page paths and the two-page command corpus. +- Add the four missing deploy lifecycle rows and correct the command-reference completeness claim + so the strict gate is green on arrival. ## Non-Scope - Reference page content/export-map verification (#1108) or `check-exports-drift` expansion. - README standard/staleness (#767), fenced-block extraction (#1374), installed-artifact proof (#1343), package renames, URL moves, or command behavior changes. -- Authoring missing package pages or filling PR-C command prose. Missing content is reported to the - orchestrator. +- Authoring package/API reference content beyond the bounded four command rows needed to activate + the gate. - The unreachable deploy `emit` operation (#1544). ## Locked Decisions | ID | Decision | Criterion and rationale | | --- | --- | --- | -| D-1 | Use a declared four-entry alias map; do not move the IA. | Criterion: minimize irreversible public-URL and cross-gate change while making the release predicate agree with the already published site. Aliases change one resolver, preserve inbound links and `docs:accuracy`, and match PR-C's enumerated convention. Moving four URLs would create avoidable user-facing breakage and broader proof obligations. The inconsistency is bounded and explicit rather than inferred. | +| D-1 | Use a declared four-entry alias map; do not move the IA. | Criterion: preserve published URLs while making the release predicate agree with the measured site. Name-exact resolution misses exactly the four deployable plugins; the four aliases produce 35/35 coverage and match PR-C's enumerated convention. Moving four URLs would create avoidable user-facing breakage. The retracted claim that `check-accuracy-and-discoverability.ts` consumes the short sagas path is not part of this decision: it does not. The real hardcoded-path consumer, `check-exports-drift.ts`, covers eight other paths and remains #1108's untouched scope. | | D-2 | Add a separate `docs-reference` readiness check immediately after `publish-set`. | `publishSet.effective` is the authoritative coordinated publish set. Placing the check here makes it independent of JSR registry discovery and prevents it being skipped with first-publish checks. `auditFirstPublishPackages` retains only README/tagline/license/export policy. If publish-set evidence is unavailable, `docs-reference` is explicitly `SKIP`; publish-set itself is already `FAIL`, so readiness cannot falsely pass. | | D-3 | A missing page is release-blocking, with a bounded content escape hatch rather than a bypass flag. | A publishable member without a page turns `docs-reference` red. A maintainer unblocks by adding the canonical page resolved by the convention; when full prose cannot land before a cut, the release skill permits an explicit stub that names and links the tracked content follow-up. No ignore list, environment override, or silent exemption is added. #1108 remains responsible for content/export fidelity. | -| D-4 | Derive command obligations from the materialized public tree through `PublicCliCommandCatalog`; never parse `.command()` source and never hardcode verbs. | This executes the same tree users/MCP see and already walks every child. The checker selects all root entries plus each root entry's immediate children. That is the acceptance level that catches a new group subcommand while respecting compact documentation of deeper generated command families. | +| D-4 | Derive command obligations from the materialized public tree through `PublicCliCommandCatalog`; never parse `.command()` source and never hardcode verbs. | This executes the same tree users/MCP see and already walks every child. The checker selects all root entries plus each root entry's immediate children. This deliberately diverges from #1377's acceptance wording, which says command **group**: all 15 groups already have prose, so a group-only predicate is inert. The negative test therefore removes a direct **subcommand** from the documentation and audits the correction explicitly when acceptance evidence is posted. Deeper generated command families remain compactly documented and out of this slice's predicate. | | D-5 | Treat colon-form `ui:*` names as complete root command paths. | A tree path `ui:add` renders as `netscript ui:add`; it must not be rewritten to `netscript ui add`. The normal nested form remains `netscript `. | -| D-6 | Assert the exact audited census, not a neighboring or merely printed count. | Production evidence reports and asserts 91 root-or-immediate-child command paths on this baseline (15 roots plus 76 direct children). The test also asserts the derived path set; any public-tree growth changes the count and requires corresponding documentation. The count is computed from the tree and compared to the ratified floor, not used as a literal verb list. | -| D-7 | PR-D will not author four newly discovered deploy lifecycle prose entries. | The focused arrival check found `deploy start`, `deploy stop`, `deploy status`, and `deploy uninstall` absent as exact public command paths. PR-C owns prose. The orchestrator must land/authorize its content follow-up before the implementation slice can claim `docs:accuracy` green. The gate remains strict; it will not be weakened to hide these four paths. | -| D-8 | Raw negative controls invoke the exported gate functions one case at a time. | A unit test that expects a rejection exits zero and is insufficient evidence. Each control command seeds exactly one missing page/command, invokes the real exported audit, prints its diagnostic, and leaves the process at raw exit 1. Tests separately lock the diagnostic contract. | +| D-6 | Assert exact equality with the audited census, not a floor or a merely printed count. | Production code and tests require exactly 91 root-or-immediate-child paths on this baseline: 15 roots plus 76 direct children. A tree addition makes the census assertion red even if prose happens to match, forcing maintainers to inspect and ratify the new obligation and update the expected count. The constant is a count, never a literal verb list. Recursive census 149 is reported for context but is not the asserted coverage set. | +| D-7 | Lock the command coverage corpus to the union of exactly two files: `docs/site/reference/cli/commands.md` and `docs/site/cli-reference.md`. | Measured separately, each file misses 25 of 91 obligations; their union misses 4. The gate reads both and no other docs. The reference index records that the public command contract is the two-page union, while `commands.md` is amended so it no longer falsely promises completeness by itself. This makes the corpus stable and auditable instead of dependent on an implementer's interpretation of “the command reference.” | +| D-8 | Match documented commands structurally, never with raw substring `includes`. | Tokenize `netscript` invocations across the full locked Markdown corpus and resolve each against the recursive tree catalog. A root obligation is credited only by an explicit root declaration (a matching command heading/top-level entry or an exact root invocation), never by a descendant invocation: `netscript agent init` cannot satisfy `netscript agent`. A direct-child obligation is credited by that exact invocation or a resolved descendant whose immediate prefix is that child, so compact deeper-family examples still establish their direct parent. Tests lock root non-prefix behavior, sibling isolation, direct-parent projection, and colon-form ids as single exact tokens. Coverage remains the root/direct 91; recursive 149 is matching context, not a neighboring asserted census. | +| D-9 | S2 owns four bounded gate-enabling command rows. | `deploy start`, `deploy stop`, `deploy status`, and `deploy uninstall` are real registered direct children with dedicated handlers. S2 adds their rows to `commands.md`, changes its self-description to agree with the two-page-union convention, and records that convention in the reference index. This is the smallest executable unblock; it does not reopen PR-C's broader prose scope or touch the unrelated unrouted `emit` defect (#1544). | +| D-10 | Raw negative controls invoke the exported gate functions one case at a time. | A unit test that expects a rejection exits zero and is insufficient evidence. Each control command seeds exactly one missing page/command, invokes the real exported audit, prints its diagnostic, and leaves the process at raw exit 1. Tests separately lock the diagnostic contract. | -The reference-index change records D-1 once: normal scoped-name stripping plus exactly four declared -aliases. It does not rename or add pages. +The reference-index change records D-1 once for page paths and D-7 once for command coverage. It +does not rename or add package pages. ## Open-Decision Sweep @@ -75,12 +79,13 @@ aliases. It does not rename or add pages. | Missing-page release escape | resolved now | D-3 | | Command-tree depth and `ui:*` normalization | resolved now | D-4/D-5 | | Census assertion | resolved now | D-6 | -| Four prose gaps | resolved now, external dependency | D-7; do not start implementation if the strict baseline cannot be made green without violating scope | +| Command corpus and matching semantics | resolved now | D-7/D-8 | +| Four prose gaps | resolved now, owned by this slice | D-9; bounded S2 rows make the strict baseline green | | Deeper recursive command families | safe to defer | They are represented by compact family tables; a later structural Markdown grammar may expand from 91 to the full 149 without changing this slice's direct-subcommand acceptance. | | Content correctness | safe to defer | Owned by #1108; this slice enforces existence only. | -No unresolved decision remains that would force implementation rework. D-7 is a sequencing -dependency, not permission to modify the predicate. +No unresolved decision or external predecessor remains that would prevent S1 or S2 from starting +after PLAN-EVAL PASS. ## Exact Files @@ -88,13 +93,15 @@ dependency, not permission to modify the predicate. | --- | --- | | `.llm/tools/release/publish-readiness.ts` | Alias resolver, separate whole-publish-set audit/dependency/evidence row, remove docs check from first-publish audit. | | `.llm/tools/release/publish-readiness_test.ts` | Ordered evidence update; 35/35-style whole-set positive; published-member missing-page negative; first-publish regression remains registry-scoped. | -| `.llm/tools/docs/check-accuracy-and-discoverability.ts` | Tree-derived command-reference coverage, colon-aware rendering, asserted census, `runAccuracyCheck()` integration. | -| `.llm/tools/docs/check-accuracy-and-discoverability_test.ts` | Synthetic public-tree positive and missing-direct-subcommand negative; colon-form control; census behavior. | -| `docs/site/reference/index.md` | One bounded convention sentence declaring that the four listed paths are gate aliases. No page content or URL move. | +| `.llm/tools/docs/check-accuracy-and-discoverability.ts` | Tree-derived command-reference coverage over the exact two-file corpus, structural root matching, tokenized recursive-path resolution, colon-aware rendering, exact census assertion, and `runAccuracyCheck()` integration. | +| `.llm/tools/docs/check-accuracy-and-discoverability_test.ts` | Synthetic public-tree positive and missing-direct-subcommand negative; root-prefix false-positive control; colon-form control; exact census behavior. | +| `docs/site/reference/index.md` | Declare the four page aliases and state once that CLI command coverage is the union of the curated and detailed command pages. No package page or URL move. | +| `docs/site/reference/cli/commands.md` | Add the four missing deploy lifecycle rows and replace the false claim that this page alone contains every command with the ratified two-page-union contract. | | `.llm/runs/fix-1377-gate--leaf/{supervisor,research,plan,worklog,context-pack,drift}.md` | Harness decisions, evidence, per-slice results, resume state, and drift. | -If implementation requires any other product or prose file, stop and rescope through the -orchestrator rather than silently widening the slice. +`docs/site/cli-reference.md` is an input to the gate but is not edited. If implementation requires +any other product or prose file, stop and rescope through the orchestrator rather than silently +widening the slice. ## The Two Negative Tests @@ -102,10 +109,11 @@ orchestrator rather than silently widening the slice. it is already published, and its resolved canonical page is absent. The report must contain `docs-reference: FAIL`, package name, resolved path, rule `docs-reference`, and `ok=false`. The raw one-case command must exit 1. -2. **Missing command-tree subcommand:** a synthetic materialized tree adds a direct child under an - otherwise documented group without adding reference prose. Coverage must identify the rendered - `netscript ` path, report the audited/covered counts, and fail. A separate - colon control proves `ui:add` stays colon-form. The raw one-case command must exit 1. +2. **Missing command-tree subcommand:** the synthetic two-page corpus documents the group but omits + one direct child from a materialized tree. Coverage must identify the rendered + `netscript ` path, report the audited/covered counts, and fail. Separate + controls prove `netscript ` cannot satisfy the root or missing child, + and that `ui:add` stays colon-form. The raw one-case command must exit 1. ## Commit Slices @@ -113,7 +121,7 @@ orchestrator rather than silently widening the slice. | --- | --- | --- | --- | | P | The decision-locked plan is reviewable and implementation has not begun. | Run artifacts only. | Plan-Gate checklist review; separate-session PLAN-EVAL must return PASS before S1. | | S1 | Every effective publish member is audited at the PR-C path convention, independently of registry/new-package state, and a missing page fails. | Release tool/test, reference index, run artifacts. | Focused `deno test --allow-all .llm/tools/release/publish-readiness_test.ts`; raw missing-page control exit 1; scoped release check/lint/fmt wrappers. | -| S2 | Every public root/direct-child command is derived from the live tree, colon-safe, census-asserted, and an undocumented child fails. | Docs checker/test, run artifacts. | Focused `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts`; raw missing-subcommand control exit 1; `rtk proxy deno task docs:accuracy`; scoped check/lint/fmt wrappers over owned tool roots. | +| S2 | The ratified two-page corpus covers every live public root/direct child, exact matching is prefix-safe and colon-safe, the exact 91 census is asserted, and an undocumented child fails. | Docs checker/test, `docs/site/reference/index.md`, `docs/site/reference/cli/commands.md`, run artifacts. | Focused `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts`; raw missing-subcommand control exit 1; `rtk proxy deno task docs:accuracy`; `rtk proxy deno task docs:links`; scoped check/lint/fmt wrappers over owned tool roots. | | S3 | The composed release/docs gate set stays green and leaves no incidental manifest/lock churn. | Run artifacts only unless reviewed fixes are required. | `docs:links`, `docs:accuracy`, `publish:dry-run`, repo tests, scoped release check/lint/fmt; immediate raw git status after dry-run. | Each implementation slice updates `worklog.md` and `context-pack.md`, receives a separate @@ -147,10 +155,11 @@ release cut. | False red blocks a canary/stable cut. | Audit `publishSet.effective`, preserve alias resolver in one constant, retain independent evidence, test published and new members. | | Registry outage suppresses reference coverage. | Place check before and outside registry discovery. | | Alias drift creates another hidden convention. | Exact four-entry constant plus index table and tests for alias and name-exact `-core` paths. | -| Command gate is inert or literal-list based. | Consume materialized tree; raw synthetic child negative; assert 91 obligations, not only print it. | +| Command gate is inert, prefix-vacuous, or literal-list based. | Consume the materialized tree; require structural root matches and tokenized command-path resolution; run the raw synthetic-child negative; assert exact equality with 91 obligations, not only print it. | | `ui:*` false negative. | Dedicated colon-form test and path renderer rule. | +| Corpus selection silently changes the result. | Read exactly the two locked files; index states their union; tests pass inputs separately and together. | | Compact nested command families create false reds. | Lock this slice to root/immediate-child acceptance and defer recursive family grammar explicitly. | -| Existing four prose gaps make strict gate red. | D-7 orchestrator dependency; do not weaken or author prose in this slice. | +| Existing four prose gaps make strict gate red. | S2 owns exactly four rows and the completeness-contract correction; no external predecessor remains. | | `publish:dry-run` rewrites manifests/lock. | Immediate raw status/diff and restore only identified incidental churn; never commit it. | ## Arch-Debt Implications @@ -163,11 +172,9 @@ the implementation. ## Dependencies and Drift Watch - Hard dependency: separate-session PLAN-EVAL PASS. -- Sequencing dependency: the orchestrator resolves D-7 before strict `docs:accuracy` can be claimed - green without PR-D writing prose. - Log drift if effective publish count differs from 35, command census differs from 91, aliases - differ from PR-C's four, another file hardcodes the path convention, or implementation needs a - third production file. + differ from PR-C's four, either locked command-corpus path changes, or implementation needs an + additional production/prose file. ## Deferred Scope diff --git a/.llm/runs/fix-1377-gate--leaf/research.md b/.llm/runs/fix-1377-gate--leaf/research.md index 21677d1572..5de3c77297 100644 --- a/.llm/runs/fix-1377-gate--leaf/research.md +++ b/.llm/runs/fix-1377-gate--leaf/research.md @@ -23,9 +23,9 @@ | 6 | The existing `firstPublishFixture()` can seed a missing reference page. Tests can reuse it after reference existence is separated from first-publish policy. | `.llm/tools/release/publish-readiness_test.ts:369-415` | | 7 | The live public command tree already has a recursive, tree-derived catalog: `PublicCliCommandCatalog` walks `getCommands()` and emits paths. No source parser or literal command list is needed. | `packages/cli/src/public/features/agent/mcp/cli-mcp-adapters.ts:7-51` | | 8 | The live catalog currently has 15 root entries, 91 root-or-immediate-child entries, and 149 recursive entries. The five `ui:*` commands are colon-form root entries, not nested `ui` children. | focused `deno eval --unstable-kv` against `createPublicCommandRegistry()` and `PublicCliCommandCatalog` | -| 9 | A literal full-path search across the exhaustive and curated CLI pages finds 87/91 root-or-immediate-child paths. Four real immediate children (`deploy start`, `deploy stop`, `deploy status`, `deploy uninstall`) have no exact public-doc occurrence. This is a PR-C content finding, not permission for PR-D to write prose. | focused catalog evaluation plus `rg -F` over `docs/site`; see plan dependency D-7 | +| 9 | Corpus choice is load-bearing: `docs/site/reference/cli/commands.md` alone misses 25/91, `docs/site/cli-reference.md` alone misses 25/91, and their exact union misses 4/91 (`deploy start`, `deploy stop`, `deploy status`, `deploy uninstall`). | focused tree-derived census against each file and their union; evaluator-confirmed FAIL_PLAN B1/B2 | | 10 | The command-reference checker is naturally part of `docs:accuracy`, whose implementation and tests already own textual command-policy checks. | `deno.json:82`; `.llm/tools/docs/check-accuracy-and-discoverability.ts` and `_test.ts` | -| 11 | PR-C's short sagas path remains consumed by `docs:accuracy`. Choosing aliases preserves it; moving the IA would widen scope and change published URLs. | `.llm/tools/docs/check-accuracy-and-discoverability.ts`; PR-C drift DR-3 | +| 11 | **Retraction:** carried research C-3 and this run's first plan incorrectly claimed `check-accuracy-and-discoverability.ts` hardcodes `docs/site/reference/sagas/index.md`. It does not; its only hardcoded reference path is the SDK page. The real second hardcoded-path consumer is `check-exports-drift.ts`, whose eight paths include none of the four aliases and whose expansion remains #1108's scope. | `rg "reference/sagas" .llm/tools/` → no matches; focused reads of both checkers | | 12 | `publish:dry-run` is a required final gate but may rewrite catalog-backed manifests under #1417. Status must be inspected immediately and incidental manifest/lock churn restored without committing it. | dispatch boundary; `.agents/skills/netscript-tools/SKILL.md` lock hygiene | ## jsr-audit surface scan @@ -45,5 +45,7 @@ public tree, with a census assertion over that exact set. Recursive grandchildren are deferred because the reference intentionally documents generated deploy families compactly rather than as every cross-product path. -- Four missing deploy lifecycle strings: orchestrator/PR-C follow-up dependency, not prose authored - by this implementation agent. +- Command corpus: exact union of `docs/site/reference/cli/commands.md` and + `docs/site/cli-reference.md`, ratified in the reference index. +- Four missing deploy lifecycle strings: bounded gate-enabling rows owned by S2, so no external + predecessor remains. diff --git a/.llm/runs/fix-1377-gate--leaf/worklog.md b/.llm/runs/fix-1377-gate--leaf/worklog.md index c9a05ca674..7c8b70f2a7 100644 --- a/.llm/runs/fix-1377-gate--leaf/worklog.md +++ b/.llm/runs/fix-1377-gate--leaf/worklog.md @@ -23,7 +23,7 @@ separate session writes `plan-eval.md` with `PASS`. - `auditFirstPublishPackages()` — preserves README/tagline/license/export first-publish policy and no longer owns reference existence. - `checkPublicCommandReference()` (planned name) — exported docs-policy audit accepting a command - tree/catalog and reference Markdown, invoked by `runAccuracyCheck()`. + tree/catalog and the exact two-file Markdown corpus, invoked by `runAccuracyCheck()`. - No new end-user CLI command, option, package export, or page URL. ### Domain Vocabulary @@ -41,8 +41,8 @@ separate session writes `plan-eval.md` with `PASS`. - `auditPublishSet` dependency — authoritative publish membership. - Existing `exists` filesystem seam — reference-page presence. - Existing `PublicCliCommandCatalog` / enumerable Cliffy command seam — live public command tree. -- Markdown text input — injected in tests; production reads the exhaustive public command reference - (and any explicitly declared companion source needed by the ratified predicate). +- Markdown corpus — injected in tests; production reads exactly + `docs/site/reference/cli/commands.md` plus `docs/site/cli-reference.md`. - No new network, registry, filesystem abstraction, or source parser. ### Constants @@ -50,6 +50,7 @@ separate session writes `plan-eval.md` with `PASS`. - `REFERENCE_PAGE_ALIASES` — four entries: sagas, streams, triggers, workers. - `EXPECTED_PUBLIC_DIRECT_COMMAND_COUNT` — `91`, asserted against the exact root/direct-child set. - `NETSCRIPT_COMMAND_PREFIX` — `netscript` for rendered diagnostics. +- `COMMAND_REFERENCE_PATHS` — the exact two-file corpus, not a broad docs glob. - Existing gate ids plus new `docs-reference`; no duplicate string list of commands. ### Command Surface / Composition Contract @@ -68,7 +69,7 @@ separate session writes `plan-eval.md` with `PASS`. | --- | --- | --- | --- | | P | Decision-locked plan and Design checkpoint | Separate-session PLAN-EVAL | Run artifacts | | S1 | Whole-publish-set path coverage and alias convention | Release unit + raw negative + scoped release wrappers | Release tool/test, index, run artifacts | -| S2 | Tree-derived direct-subcommand coverage | Docs unit + raw negative + docs accuracy + scoped docs wrappers | Docs checker/test, run artifacts | +| S2 | Tree-derived direct-subcommand coverage plus bounded four-row unblock | Docs unit + raw negative + docs accuracy/links + scoped docs wrappers | Docs checker/test, reference index, commands page, run artifacts | | S3 | Composed merge-readiness evidence and lock hygiene | Docs links/accuracy, publish dry-run, repo tests, scoped wrappers | Run artifacts only unless reviewed fix required | ### Deferred Scope @@ -92,6 +93,7 @@ and coverage diagnostic name any missing path. Do not edit a command list in the | 2026-08-12 | P | Bootstrap | Wrote `supervisor.md` first; loaded named skills and harness policy. | | 2026-08-12 | P | Arrival checks | Baseline exact; PR-C ancestor; alias convention yields 35/35 reference coverage. | | 2026-08-12 | P | Design | Locked alias, check placement, release unblock, command-tree scope, census, and negative controls. | +| 2026-08-12 | P | FAIL_PLAN revision | Locked the two-page corpus, exact 91 equality, structural root plus tokenized path matching, four-row S2 unblock, and retracted the false sagas-path premise. | ## Decisions @@ -101,7 +103,8 @@ and coverage diagnostic name any missing path. Do not edit a command list in the | Separate whole-set readiness check | Prevent registry/new-package skip | `plan.md` D-2 | | Page/stub with follow-up is unblock; no bypass | Release gate needs a bounded repair path | `plan.md` D-3; release skill | | Live tree, root/direct children, colon-safe | Predicate must be executable and nonliteral | `plan.md` D-4–D-6 | -| Four deploy prose gaps return to orchestrator | PR-C owns prose; strict gate must not be weakened | `plan.md` D-7 | +| Exact two-page corpus and structural matching | Prevent implicit corpus drift and root-prefix false greens | `plan.md` D-7/D-8 | +| Four deploy rows owned by S2 | Make the strict gate executable without an ownerless predecessor | `plan.md` D-9 | ## Gate Results @@ -112,13 +115,12 @@ All implementation gates are `NOT_RUN` by design in phase 1. | Baseline identity | PASS | branch `fix/1377-docs-reference-gate-scope`; head `fa5d0d411…` | | PR-C landed | PASS | `db1d79c68…` is an ancestor of HEAD | | Reference arrival coverage | PASS | 35 effective, 35 present under locked alias resolver, 0 missing | -| PLAN-EVAL | PENDING | Must be the automatic status-driven plan evaluator; this generator cannot self-evaluate | +| PLAN-EVAL | FAIL_PLAN → REVISION_READY | Fallback evaluation failed head `5ba4bc339`; revised head requires a new automatic status-driven evaluation; this generator cannot self-evaluate | | Type/lint/fmt/docs/publish/tests | NOT_RUN | Phase 2 only after PLAN-EVAL PASS | | Raw negative controls | NOT_RUN | Phase 2 only; both raw exit codes and diagnostics required | ## Handoff Notes -- Evaluator should challenge D-4/D-6 first: whether 91 is the correct direct-subcommand surface and - whether compact deeper command families are safely deferred. -- Evaluator should verify D-7 is treated as a real dependency, not an excuse to reduce coverage. +- Evaluator should verify the exact two-file corpus, structural root plus tokenized path matching, + exact 91 equality, and bounded four-row ownership satisfy B1–B3 without expanding into #1108. - No implementation file has been edited. From dedd7804508f6cbe0d1ce97174cb7642f49dc96a Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 18:38:13 +0200 Subject: [PATCH 4/6] fix(release): audit reference pages across publish set --- .llm/runs/fix-1377-gate--leaf/context-pack.md | 32 +++++---- .llm/runs/fix-1377-gate--leaf/drift.md | 12 ++++ .llm/runs/fix-1377-gate--leaf/plan-eval.md | 41 +++++++++++ .llm/runs/fix-1377-gate--leaf/plan.md | 10 +-- .llm/runs/fix-1377-gate--leaf/supervisor.md | 2 +- .llm/runs/fix-1377-gate--leaf/worklog.md | 20 +++++- .llm/tools/release/publish-readiness.ts | 71 +++++++++++++++++-- .llm/tools/release/publish-readiness_test.ts | 69 +++++++++++++++--- docs/site/reference/index.md | 5 +- 9 files changed, 222 insertions(+), 40 deletions(-) create mode 100644 .llm/runs/fix-1377-gate--leaf/plan-eval.md diff --git a/.llm/runs/fix-1377-gate--leaf/context-pack.md b/.llm/runs/fix-1377-gate--leaf/context-pack.md index fa35710b06..6933e5a009 100644 --- a/.llm/runs/fix-1377-gate--leaf/context-pack.md +++ b/.llm/runs/fix-1377-gate--leaf/context-pack.md @@ -6,18 +6,17 @@ | --- | --- | | Run ID | `fix-1377-gate--leaf` | | Branch | `fix/1377-docs-reference-gate-scope` | -| Current phase | `plan-eval` pending | +| Current phase | `implement` — S1 | | Archetype | 6 — CLI / tooling | | Scope overlays | Docs | ## Current State -Fallback PLAN-EVAL returned `FAIL_PLAN` on `5ba4bc339`; the plan is revised and no implementation -file has changed. It retains the confirmed alias and whole-publish-set design, locks command -coverage to the exact union of `docs/site/reference/cli/commands.md` and -`docs/site/cli-reference.md`, requires structural root matching plus tokenized path resolution and -exact equality with 91 root/direct obligations, and makes the four missing deploy rows bounded S2 -scope. A new automatic PLAN-EVAL on this immutable head must PASS before implementation. +Fallback PLAN-EVAL cycle 1 returned `FAIL_PLAN` on `5ba4bc339`; cycle 2 returned `PASS` on +`706c2bf05`. Phase 2 is active in S1. The implementation retains the confirmed alias and +whole-publish-set design, locks command coverage to the exact two-page union, requires structural +root matching plus tokenized path resolution and exact equality with 91 root/direct obligations, +and makes the four missing deploy rows bounded S2 scope. ## Completed @@ -27,17 +26,17 @@ scope. A new automatic PLAN-EVAL on this immutable head must PASS before impleme docs checker/tests, and live public command catalog inspected. - 35/35 alias-resolved reference arrival coverage verified. - FAIL_PLAN B1–B3 and non-blocking findings resolved in the revised plan. +- Native Opus 5 fallback PLAN-EVAL cycle 2 passed immutable head `706c2bf05`. ## In Progress -- Commit/push the revised plan; stop for a new automatic PLAN-EVAL. +- S1 is gate-green and ready for its sign-off commit/push/comment. ## Next Steps -1. Orchestrator triggers the automatic status-driven PLAN-EVAL via the status-label mechanism and - records the verdict. -2. Resolve any further `FAIL_PLAN`; do not implement meanwhile. -3. On PASS, the orchestrator may advance the lifecycle; implement S1–S3 in order only after resume. +1. Complete S1 gates, commit, explicit push, and PR comment. +2. Complete S2 command-coverage gate and bounded docs corrections. +3. Complete S3 composed gates and stop for native Opus 5 IMPL-EVAL fallback. ## Key Decisions @@ -51,18 +50,21 @@ scope. A new automatic PLAN-EVAL on this immutable head must PASS before impleme ## Files Changed -Only `.llm/runs/fix-1377-gate--leaf/{supervisor,research,plan,worklog,context-pack,drift}.md` are -created in phase 1. +- S1: release readiness tool/test, reference index convention, and run artifacts including + `plan-eval.md`. ## Gates | Gate family | Current status | Evidence | | --- | --- | --- | -| Plan | READY_FOR_EVAL | Plan-Gate inputs complete | +| Plan | PASS | Native Opus 5 fallback, cycle 2, head `706c2bf05` | | Static / Fitness | NOT_RUN | Phase 2 | | Docs / Release | NOT_RUN | Phase 2 | | Negative controls | NOT_RUN | Phase 2; raw exits mandatory | +S1 release unit, scoped check/lint/format, and raw missing-page exit 1 are PASS/expected-fail with +diagnostics recorded in `worklog.md`. + ## Open Questions - None for implementation design; no external content predecessor remains. diff --git a/.llm/runs/fix-1377-gate--leaf/drift.md b/.llm/runs/fix-1377-gate--leaf/drift.md index 7e49dece0e..283b58fd7e 100644 --- a/.llm/runs/fix-1377-gate--leaf/drift.md +++ b/.llm/runs/fix-1377-gate--leaf/drift.md @@ -35,3 +35,15 @@ Append-only. deferred the four rows externally; the original finding remains as historical provenance. - **Evidence:** revised `research.md` findings 9/11 and `plan.md` D-1/D-4/D-6–D-9 on the superseding plan head. + +## 2026-08-12 — cloud evaluators stalled; native fallback completed PLAN-EVAL + +- **What:** Forward-looking run artifacts named an automatic status-driven evaluator and paid + retrigger that did not complete either plan cycle. +- **Source:** orchestrator Phase-2 resume after native Opus 5 read-only fallback verdicts. +- **Expected:** A separate evaluator verdict for each immutable plan head. +- **Actual:** Cloud evaluators stalled and were cancelled; the orchestrator dispatched native Opus 5 + fallbacks without label cycling. Cycle 1 failed `5ba4bc339`; cycle 2 passed `706c2bf05`. +- **Severity:** procedural. +- **Action:** correct active routing instructions and preserve both verdicts in `plan-eval.md`. +- **Evidence:** `supervisor.md`, `plan-eval.md`, `worklog.md`, and `context-pack.md`. diff --git a/.llm/runs/fix-1377-gate--leaf/plan-eval.md b/.llm/runs/fix-1377-gate--leaf/plan-eval.md new file mode 100644 index 0000000000..4ea20b9ae0 --- /dev/null +++ b/.llm/runs/fix-1377-gate--leaf/plan-eval.md @@ -0,0 +1,41 @@ +# PLAN-EVAL — fix-1377-gate--leaf + +- Plan evaluator: native Claude · Anthropic · Opus 5 read-only fallback +- Run: `fix-1377-gate--leaf` +- Surface / archetype: release tooling and public CLI docs gate / Archetype 6 +- Scope overlay: docs +- Dispatch: orchestrator-dispatched per immutable head; no label cycling or paid retrigger + +## Verdict cycles + +| Cycle | Immutable head | Verdict | Outcome | +| --- | --- | --- | --- | +| 1 | `5ba4bc339` | `FAIL_PLAN` | Lock the exact command corpus; reconcile the two-page IA contract; give the four missing deploy rows an executable owner; retract the false sagas-path consumer premise; clarify census and matching semantics. | +| 2 | `706c2bf05` | `PASS` | Cycle-1 blockers verified fixed by execution; counts 15 roots / 76 direct children / 91 root+direct / 149 recursive and five colon roots reproduced from the live tree. | + +## Cycle 2 checklist results + +| Plan-Gate item | Result | Evidence / location | +| --- | --- | --- | +| Research present and current | PASS | `research.md`, including finding 11 retraction | +| Decisions locked | PASS | `plan.md` D-1–D-10 | +| Open-decision sweep | PASS | `plan.md` Open-Decision Sweep | +| Commit slices (< 30, gate + files each) | PASS | `plan.md` S1–S3 | +| Risk register | PASS | `plan.md` Risk Register | +| Gate set selected | PASS | `plan.md` Validation Plan | +| Deferred scope explicit | PASS | `plan.md` Deferred Scope | +| jsr-audit surface scan (pkg/plugin) | N/A | Tooling/docs gate; no package export or manifest surface change | + +## Open-decision sweep (evaluator-run) + +None. The two-page corpus, alias resolver, whole-publish-set placement, subcommand depth, colon +rendering, exact census, four-row content owner, and raw negative controls are decision-complete. + +## Verdict + +`PASS` + +## Notes + +The evaluator confirmed cycle 1's three blockers by executing the live tree/corpus measurements, +not only by reading the revised diff. Implementation was forbidden until this cycle-2 verdict. diff --git a/.llm/runs/fix-1377-gate--leaf/plan.md b/.llm/runs/fix-1377-gate--leaf/plan.md index 547dd80349..145e615a7b 100644 --- a/.llm/runs/fix-1377-gate--leaf/plan.md +++ b/.llm/runs/fix-1377-gate--leaf/plan.md @@ -62,7 +62,7 @@ must exit non-zero with package/command diagnostics. | D-4 | Derive command obligations from the materialized public tree through `PublicCliCommandCatalog`; never parse `.command()` source and never hardcode verbs. | This executes the same tree users/MCP see and already walks every child. The checker selects all root entries plus each root entry's immediate children. This deliberately diverges from #1377's acceptance wording, which says command **group**: all 15 groups already have prose, so a group-only predicate is inert. The negative test therefore removes a direct **subcommand** from the documentation and audits the correction explicitly when acceptance evidence is posted. Deeper generated command families remain compactly documented and out of this slice's predicate. | | D-5 | Treat colon-form `ui:*` names as complete root command paths. | A tree path `ui:add` renders as `netscript ui:add`; it must not be rewritten to `netscript ui add`. The normal nested form remains `netscript `. | | D-6 | Assert exact equality with the audited census, not a floor or a merely printed count. | Production code and tests require exactly 91 root-or-immediate-child paths on this baseline: 15 roots plus 76 direct children. A tree addition makes the census assertion red even if prose happens to match, forcing maintainers to inspect and ratify the new obligation and update the expected count. The constant is a count, never a literal verb list. Recursive census 149 is reported for context but is not the asserted coverage set. | -| D-7 | Lock the command coverage corpus to the union of exactly two files: `docs/site/reference/cli/commands.md` and `docs/site/cli-reference.md`. | Measured separately, each file misses 25 of 91 obligations; their union misses 4. The gate reads both and no other docs. The reference index records that the public command contract is the two-page union, while `commands.md` is amended so it no longer falsely promises completeness by itself. This makes the corpus stable and auditable instead of dependent on an implementer's interpretation of “the command reference.” | +| D-7 | Lock the command coverage corpus to the union of exactly two files: `docs/site/reference/cli/commands.md` and `docs/site/cli-reference.md`. | Each page alone misses 25–33 obligations depending on crediting semantics; their union misses exactly 4 under every measured model. The gate reads both and no other docs. The reference index records that the public command contract is the two-page union, while both pages are amended so neither falsely promises that `commands.md` is complete by itself. This makes the corpus stable and auditable instead of dependent on an implementer's interpretation of “the command reference.” | | D-8 | Match documented commands structurally, never with raw substring `includes`. | Tokenize `netscript` invocations across the full locked Markdown corpus and resolve each against the recursive tree catalog. A root obligation is credited only by an explicit root declaration (a matching command heading/top-level entry or an exact root invocation), never by a descendant invocation: `netscript agent init` cannot satisfy `netscript agent`. A direct-child obligation is credited by that exact invocation or a resolved descendant whose immediate prefix is that child, so compact deeper-family examples still establish their direct parent. Tests lock root non-prefix behavior, sibling isolation, direct-parent projection, and colon-form ids as single exact tokens. Coverage remains the root/direct 91; recursive 149 is matching context, not a neighboring asserted census. | | D-9 | S2 owns four bounded gate-enabling command rows. | `deploy start`, `deploy stop`, `deploy status`, and `deploy uninstall` are real registered direct children with dedicated handlers. S2 adds their rows to `commands.md`, changes its self-description to agree with the two-page-union convention, and records that convention in the reference index. This is the smallest executable unblock; it does not reopen PR-C's broader prose scope or touch the unrelated unrouted `emit` defect (#1544). | | D-10 | Raw negative controls invoke the exported gate functions one case at a time. | A unit test that expects a rejection exits zero and is insufficient evidence. Each control command seeds exactly one missing page/command, invokes the real exported audit, prints its diagnostic, and leaves the process at raw exit 1. Tests separately lock the diagnostic contract. | @@ -97,11 +97,11 @@ after PLAN-EVAL PASS. | `.llm/tools/docs/check-accuracy-and-discoverability_test.ts` | Synthetic public-tree positive and missing-direct-subcommand negative; root-prefix false-positive control; colon-form control; exact census behavior. | | `docs/site/reference/index.md` | Declare the four page aliases and state once that CLI command coverage is the union of the curated and detailed command pages. No package page or URL move. | | `docs/site/reference/cli/commands.md` | Add the four missing deploy lifecycle rows and replace the false claim that this page alone contains every command with the ratified two-page-union contract. | +| `docs/site/cli-reference.md` | Correct only the two sentences that incorrectly route readers to `commands.md` as exhaustive; this file remains otherwise unchanged. | | `.llm/runs/fix-1377-gate--leaf/{supervisor,research,plan,worklog,context-pack,drift}.md` | Harness decisions, evidence, per-slice results, resume state, and drift. | -`docs/site/cli-reference.md` is an input to the gate but is not edited. If implementation requires -any other product or prose file, stop and rescope through the orchestrator rather than silently -widening the slice. +If implementation requires any other product or prose file, stop and rescope through the +orchestrator rather than silently widening the slice. ## The Two Negative Tests @@ -121,7 +121,7 @@ widening the slice. | --- | --- | --- | --- | | P | The decision-locked plan is reviewable and implementation has not begun. | Run artifacts only. | Plan-Gate checklist review; separate-session PLAN-EVAL must return PASS before S1. | | S1 | Every effective publish member is audited at the PR-C path convention, independently of registry/new-package state, and a missing page fails. | Release tool/test, reference index, run artifacts. | Focused `deno test --allow-all .llm/tools/release/publish-readiness_test.ts`; raw missing-page control exit 1; scoped release check/lint/fmt wrappers. | -| S2 | The ratified two-page corpus covers every live public root/direct child, exact matching is prefix-safe and colon-safe, the exact 91 census is asserted, and an undocumented child fails. | Docs checker/test, `docs/site/reference/index.md`, `docs/site/reference/cli/commands.md`, run artifacts. | Focused `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts`; raw missing-subcommand control exit 1; `rtk proxy deno task docs:accuracy`; `rtk proxy deno task docs:links`; scoped check/lint/fmt wrappers over owned tool roots. | +| S2 | The ratified two-page corpus covers every live public root/direct child, exact matching is prefix-safe and colon-safe, the exact 91 census is asserted, and an undocumented child fails. | Docs checker/test, `docs/site/reference/index.md`, `docs/site/reference/cli/commands.md`, the two authorised sentences in `docs/site/cli-reference.md`, run artifacts. | Focused `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts`; raw missing-subcommand control exit 1; `rtk proxy deno task docs:accuracy`; `rtk proxy deno task docs:links`; scoped check/lint/fmt wrappers over owned tool roots. | | S3 | The composed release/docs gate set stays green and leaves no incidental manifest/lock churn. | Run artifacts only unless reviewed fixes are required. | `docs:links`, `docs:accuracy`, `publish:dry-run`, repo tests, scoped release check/lint/fmt; immediate raw git status after dry-run. | Each implementation slice updates `worklog.md` and `context-pack.md`, receives a separate diff --git a/.llm/runs/fix-1377-gate--leaf/supervisor.md b/.llm/runs/fix-1377-gate--leaf/supervisor.md index 737d474d79..9b63688815 100644 --- a/.llm/runs/fix-1377-gate--leaf/supervisor.md +++ b/.llm/runs/fix-1377-gate--leaf/supervisor.md @@ -19,6 +19,6 @@ Written at run start per `workflow/lane-policy.md` § Supervisor identity. | --- | --- | --- | | `normal_implementation` | Codex · OpenAI · GPT-5.6 Sol · medium | Plan author and, only after PLAN-EVAL PASS, implementation agent | | `review_codex` | Claude · Anthropic · Opus 5 · low | Fable prohibited this milestone; this is the declared in-plan fallback in `routing-policy.ts:441` | -| formal PLAN-EVAL | Automatic status-driven evaluator (MiniMax M3) | Triggered by the orchestrator through the `openhands` + `status:plan-eval` label pair. The orchestrator does not select, launch, or resume an evaluator model. | +| formal PLAN-EVAL | Claude · Anthropic · Opus 5 · read-only | Native fallback dispatched by the orchestrator for each immutable head after the cloud evaluators stalled and were cancelled; no label cycling or paid retrigger. | Reference `.llm/harness/workflow/lane-policy.md`; no lane overrides are in force. diff --git a/.llm/runs/fix-1377-gate--leaf/worklog.md b/.llm/runs/fix-1377-gate--leaf/worklog.md index 7c8b70f2a7..7dae570da7 100644 --- a/.llm/runs/fix-1377-gate--leaf/worklog.md +++ b/.llm/runs/fix-1377-gate--leaf/worklog.md @@ -11,8 +11,8 @@ ## Design -Recorded before implementation. PLAN-EVAL is selected and implementation is forbidden until a -separate session writes `plan-eval.md` with `PASS`. +Recorded before implementation. Separate-session PLAN-EVAL cycle 2 passed head `706c2bf05`; Phase 2 +began only after that verdict. ### Public Surface @@ -94,6 +94,10 @@ and coverage diagnostic name any missing path. Do not edit a command list in the | 2026-08-12 | P | Arrival checks | Baseline exact; PR-C ancestor; alias convention yields 35/35 reference coverage. | | 2026-08-12 | P | Design | Locked alias, check placement, release unblock, command-tree scope, census, and negative controls. | | 2026-08-12 | P | FAIL_PLAN revision | Locked the two-page corpus, exact 91 equality, structural root plus tokenized path matching, four-row S2 unblock, and retracted the false sagas-path premise. | +| 2026-08-12 | P | PLAN-EVAL cycle 2 | Native Opus 5 read-only fallback returned PASS on immutable head `706c2bf05`; cloud evaluators had stalled and were cancelled. | +| 2026-08-12 | S1 | Start | Began whole-publish-set reference existence implementation after PASS. | +| 2026-08-12 | S1 | Implement | Added the independent `docs-reference` row over `publishSet.effective`, four declared aliases, and removed page existence from first-publish policy. | +| 2026-08-12 | S1 | Reconcile | PR #1586 remains draft with `Closes #1377`; issue #1377 remains open. Read current PR/issue comments; no new blocker or scope change. Labels were observed but not changed. | ## Decisions @@ -115,10 +119,20 @@ All implementation gates are `NOT_RUN` by design in phase 1. | Baseline identity | PASS | branch `fix/1377-docs-reference-gate-scope`; head `fa5d0d411…` | | PR-C landed | PASS | `db1d79c68…` is an ancestor of HEAD | | Reference arrival coverage | PASS | 35 effective, 35 present under locked alias resolver, 0 missing | -| PLAN-EVAL | FAIL_PLAN → REVISION_READY | Fallback evaluation failed head `5ba4bc339`; revised head requires a new automatic status-driven evaluation; this generator cannot self-evaluate | +| PLAN-EVAL | FAIL_PLAN → PASS | Native Opus 5 read-only fallback: cycle 1 failed `5ba4bc339`; cycle 2 passed `706c2bf05`. The generator did not self-evaluate. | | Type/lint/fmt/docs/publish/tests | NOT_RUN | Phase 2 only after PLAN-EVAL PASS | | Raw negative controls | NOT_RUN | Phase 2 only; both raw exit codes and diagnostics required | +### S1 gate evidence + +| Gate | Result | Evidence / notes | +| --- | --- | --- | +| Release unit | PASS | `deno test --allow-all .llm/tools/release/publish-readiness_test.ts` — 15 passed, 0 failed | +| Missing-page raw control | EXPECTED FAIL, exit 1 | Exported `auditReferencePages()` over published `@netscript/published`; diagnostic: `@netscript/published docs/site/reference/published/index.md [docs-reference] docs-site reference page is required` | +| Scoped release check | PASS | 42 files selected; 0 findings | +| Scoped release lint | PASS | 42 files selected; 0 findings | +| Scoped release format | PASS | 42 files selected; 0 findings after targeted formatting | + ## Handoff Notes - Evaluator should verify the exact two-file corpus, structural root plus tokenized path matching, diff --git a/.llm/tools/release/publish-readiness.ts b/.llm/tools/release/publish-readiness.ts index ad8898e0f5..09d92c0ee7 100644 --- a/.llm/tools/release/publish-readiness.ts +++ b/.llm/tools/release/publish-readiness.ts @@ -50,11 +50,22 @@ export interface FirstPublishViolation { readonly message: string; } +export interface ReferencePageViolation { + readonly packageName: string; + readonly path: string; + readonly rule: 'docs-reference'; + readonly message: string; +} + export interface PublishReadinessDependencies { readonly auditPublishSet: (root: string) => Promise; readonly auditMarkdownPins: (root: string, version: string) => Promise; readonly auditVersions: (root: string, version: string) => Promise; readonly scanSpecifiers: (roots: readonly string[], root: string) => Promise; + readonly auditReferencePages: ( + root: string, + members: readonly PublishableMember[], + ) => Promise; readonly readRegistryVersions: (packageName: string) => Promise; readonly auditFirstPublish: ( root: string, @@ -69,6 +80,7 @@ const defaultDependencies: PublishReadinessDependencies = { auditMarkdownPins, auditVersions: auditLockstepAndResidue, scanSpecifiers: scanNetscriptJsrSpecifiers, + auditReferencePages, readRegistryVersions, auditFirstPublish: auditFirstPublishPackages, runProvisioningDryCheck, @@ -81,6 +93,14 @@ const INTERNAL_SPECIFIER = const IMPORT_ATTRIBUTE_SUNSET = 'Import-attribute publication remains prohibited until denoland/deno#35546 is fixed, merged, released, and an authenticated canary text-import probe is green.'; +/** Declared package-name exceptions to the scoped-name reference path convention. */ +export const REFERENCE_PAGE_ALIASES: Readonly> = { + '@netscript/plugin-sagas': 'sagas', + '@netscript/plugin-streams': 'streams', + '@netscript/plugin-triggers': 'triggers', + '@netscript/plugin-workers': 'workers', +}; + /** Collect every pre-publish check without hiding later independent evidence. */ export async function collectPublishReadiness( root: string, @@ -108,6 +128,27 @@ export async function collectPublishReadiness( }; }); + if (!publishSet) { + checks.push(skip('docs-reference', 'publish-set evidence unavailable')); + } else { + await capture(checks, 'docs-reference', async () => { + const violations = await dependencies.auditReferencePages(root, publishSet!.effective); + if (violations.length > 0) { + throw new Error( + violations.map((violation) => + `${violation.packageName} ${violation.path} [${violation.rule}] ${violation.message}` + ).join('; '), + ); + } + return { + summary: `${publishSet!.effective.length} publishable package reference page(s) present`, + details: publishSet!.effective.map((member) => + `${member.name} (${referencePagePath(member.name)})` + ), + }; + }); + } + await capture(checks, 'markdown-pins', async () => { const result = await dependencies.auditMarkdownPins(root, version); if (result.violations.length > 0) { @@ -251,7 +292,7 @@ export async function auditLockstepAndResidue( return findings.sort((left, right) => left.path.localeCompare(right.path)); } -/** Apply first-publish-only README, tagline, manifest, export, and docs checks. */ +/** Apply first-publish-only README, tagline, manifest, and export checks. */ export async function auditFirstPublishPackages( root: string, members: readonly PublishableMember[], @@ -298,12 +339,25 @@ export async function auditFirstPublishPackages( if (!isJsonObject(manifest) || !hasExports(manifest.exports)) { violations.push(violation(member, 'deno.json', 'exports', 'non-empty exports are required')); } + } + return violations; +} - const docsPath = `docs/site/reference/${packageSegment(member.name)}/index.md`; - if (!(await exists(join(root, docsPath)))) { - violations.push( - violation(member, docsPath, 'docs-reference', 'docs-site reference page is required'), - ); +/** Audit reference-page existence for every member in the coordinated publish set. */ +export async function auditReferencePages( + root: string, + members: readonly PublishableMember[], +): Promise { + const violations: ReferencePageViolation[] = []; + for (const member of members) { + const path = referencePagePath(member.name); + if (!(await exists(join(root, path)))) { + violations.push({ + packageName: member.name, + path, + rule: 'docs-reference', + message: 'docs-site reference page is required', + }); } } return violations; @@ -414,6 +468,11 @@ function packageSegment(packageName: string): string { return segment; } +function referencePagePath(packageName: string): string { + const segment = REFERENCE_PAGE_ALIASES[packageName] ?? packageSegment(packageName); + return `docs/site/reference/${segment}/index.md`; +} + function publishRoots(result: PublishSetAuditResult): readonly string[] { return [ ...new Set( diff --git a/.llm/tools/release/publish-readiness_test.ts b/.llm/tools/release/publish-readiness_test.ts index acb73309c6..6b90d9ebe9 100644 --- a/.llm/tools/release/publish-readiness_test.ts +++ b/.llm/tools/release/publish-readiness_test.ts @@ -2,6 +2,7 @@ import { assert, assertEquals } from 'jsr:@std/assert@^1'; import { auditFirstPublishPackages, auditLockstepAndResidue, + auditReferencePages, collectPublishReadiness, type PublishReadinessDependencies, type ReadinessCheckEvidence, @@ -16,6 +17,7 @@ Deno.test('publish readiness emits ordered structured evidence for every compose assertEquals(report.ok, true); assertEquals(report.checks.map(({ id, status }) => ({ id, status })), [ { id: 'publish-set', status: 'PASS' }, + { id: 'docs-reference', status: 'PASS' }, { id: 'markdown-pins', status: 'PASS' }, { id: 'lockstep-residue', status: 'PASS' }, { id: 'versionless-specifiers', status: 'PASS' }, @@ -27,6 +29,62 @@ Deno.test('publish readiness emits ordered structured evidence for every compose assert(report.checks.every((check) => Number.isInteger(check.durationMs))); }); +Deno.test('reference-page audit resolves the four deployable aliases and name-exact core peers', async () => { + const root = await Deno.makeTempDir({ prefix: 'netscript-readiness-reference-aliases-' }); + const members = [ + { path: 'plugins/sagas', name: '@netscript/plugin-sagas' }, + { path: 'packages/sagas-core', name: '@netscript/plugin-sagas-core' }, + { path: 'plugins/streams', name: '@netscript/plugin-streams' }, + { path: 'packages/streams-core', name: '@netscript/plugin-streams-core' }, + { path: 'plugins/triggers', name: '@netscript/plugin-triggers' }, + { path: 'packages/triggers-core', name: '@netscript/plugin-triggers-core' }, + { path: 'plugins/workers', name: '@netscript/plugin-workers' }, + { path: 'packages/workers-core', name: '@netscript/plugin-workers-core' }, + ]; + try { + for ( + const segment of [ + 'sagas', + 'plugin-sagas-core', + 'streams', + 'plugin-streams-core', + 'triggers', + 'plugin-triggers-core', + 'workers', + 'plugin-workers-core', + ] + ) { + const directory = `${root}/docs/site/reference/${segment}`; + await Deno.mkdir(directory, { recursive: true }); + await Deno.writeTextFile(`${directory}/index.md`, `# ${segment}\n`); + } + assertEquals(await auditReferencePages(root, members), []); + } finally { + await Deno.remove(root, { recursive: true }); + } +}); + +Deno.test('publish readiness fails when a published effective member has no reference page', async () => { + const root = await Deno.makeTempDir({ prefix: 'netscript-readiness-reference-missing-' }); + try { + const report = await collectPublishReadiness( + root, + '0.0.2-canary.1', + dependencies({ + auditReferencePages, + readRegistryVersions: () => Promise.resolve(['0.0.1']), + }), + ); + assertEquals(report.ok, false); + assertFailed(report.checks, 'docs-reference', '@netscript/new'); + assertFailed(report.checks, 'docs-reference', 'docs/site/reference/new/index.md'); + assertEquals(report.checks.find((check) => check.id === 'new-packages')?.details, []); + assertEquals(report.checks.find((check) => check.id === 'first-publish')?.status, 'PASS'); + } finally { + await Deno.remove(root, { recursive: true }); + } +}); + Deno.test('publish readiness fails on a seeded workspace member omitted from the publish set', async () => { const missing = { path: 'plugins/new', name: '@netscript/plugin-new' }; const report = await collectPublishReadiness( @@ -140,19 +198,18 @@ Deno.test('first-publish checklist fails on a seeded missing README', async () = } }); -Deno.test('first-publish checklist fails over-cap tagline, missing license/exports, and docs pointer', async () => { +Deno.test('first-publish checklist fails over-cap tagline and missing license/exports', async () => { const root = await firstPublishFixture({ tagline: `**${'enterprise '.repeat(30)}**`, license: false, exports: false, - docs: false, }); try { const rules = (await auditFirstPublishPackages(root, [MEMBER])).map((finding) => finding.rule); assert(rules.includes('tagline-bytes')); assert(rules.includes('license')); assert(rules.includes('exports')); - assert(rules.includes('docs-reference')); + assertEquals(rules.includes('docs-reference'), false); } finally { await Deno.remove(root, { recursive: true }); } @@ -279,6 +336,7 @@ function dependencies( unknownExports: [], ranges: [], }), + auditReferencePages: () => Promise.resolve([]), readRegistryVersions: () => Promise.resolve([]), auditFirstPublish: () => Promise.resolve([]), runProvisioningDryCheck: () => Promise.resolve(), @@ -328,7 +386,6 @@ async function firstPublishFixture( tagline?: string; license?: boolean; exports?: boolean; - docs?: boolean; } = {}, ): Promise { const root = await Deno.makeTempDir({ prefix: 'netscript-readiness-first-' }); @@ -366,9 +423,5 @@ async function firstPublishFixture( ].join('\n'), ); } - if (options.docs !== false) { - await Deno.mkdir(`${root}/docs/site/reference/new`, { recursive: true }); - await Deno.writeTextFile(`${root}/docs/site/reference/new/index.md`, '# New\n'); - } return root; } diff --git a/docs/site/reference/index.md b/docs/site/reference/index.md index 62fbf8f26f..a644dbbce1 100644 --- a/docs/site/reference/index.md +++ b/docs/site/reference/index.md @@ -30,8 +30,9 @@ exceptions, the deployable plugins, which drop the `plugin-` prefix: Their `-core` counterparts use the name-exact form, so `@netscript/plugin-sagas-core` is at [`/reference/plugin-sagas-core/`](/reference/plugin-sagas-core/) while `@netscript/plugin-sagas` is at -`/reference/sagas/`. This is a description of what the site does today, not a rule to follow when the -two forms disagree. +`/reference/sagas/`. The release-readiness gate treats the four table entries as declared aliases; +all other publishable members use the name-exact segment. A maintainer adding or changing an +exception must update this table and the gate together. Each publishable workspace member's own page is canonical for its exported API. The four deployable plugin pages describe their manifests and integration entrypoints, then link to their separately From 29557faa858e832d8cabdc452ed4692ae26031e2 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 18:47:13 +0200 Subject: [PATCH 5/6] fix(docs): gate command reference against public tree --- .llm/runs/fix-1377-gate--leaf/context-pack.md | 18 +- .llm/runs/fix-1377-gate--leaf/research.md | 2 +- .llm/runs/fix-1377-gate--leaf/worklog.md | 18 +- .../check-accuracy-and-discoverability.ts | 208 +++++++++++++++++- ...check-accuracy-and-discoverability_test.ts | 52 +++++ docs/site/cli-reference.md | 8 +- docs/site/reference/cli/commands.md | 14 +- docs/site/reference/index.md | 7 + 8 files changed, 307 insertions(+), 20 deletions(-) diff --git a/.llm/runs/fix-1377-gate--leaf/context-pack.md b/.llm/runs/fix-1377-gate--leaf/context-pack.md index 6933e5a009..dad1867f12 100644 --- a/.llm/runs/fix-1377-gate--leaf/context-pack.md +++ b/.llm/runs/fix-1377-gate--leaf/context-pack.md @@ -6,7 +6,7 @@ | --- | --- | | Run ID | `fix-1377-gate--leaf` | | Branch | `fix/1377-docs-reference-gate-scope` | -| Current phase | `implement` — S1 | +| Current phase | `implement` — S2 ready to commit | | Archetype | 6 — CLI / tooling | | Scope overlays | Docs | @@ -30,13 +30,13 @@ and makes the four missing deploy rows bounded S2 scope. ## In Progress -- S1 is gate-green and ready for its sign-off commit/push/comment. +- S2 is gate-green and ready for its sign-off commit/push/comment. ## Next Steps -1. Complete S1 gates, commit, explicit push, and PR comment. -2. Complete S2 command-coverage gate and bounded docs corrections. -3. Complete S3 composed gates and stop for native Opus 5 IMPL-EVAL fallback. +1. Commit S2, explicit push, and PR comment. +2. Complete S3 composed gates and lock-hygiene inspection. +3. Finalize run/PR evidence and stop for native Opus 5 IMPL-EVAL fallback. ## Key Decisions @@ -52,15 +52,17 @@ and makes the four missing deploy rows bounded S2 scope. - S1: release readiness tool/test, reference index convention, and run artifacts including `plan-eval.md`. +- S2: docs accuracy tool/test, four deploy command rows, exact two-page union wording including the + two authorised `cli-reference.md` sentences, reference-index command contract, and run artifacts. ## Gates | Gate family | Current status | Evidence | | --- | --- | --- | | Plan | PASS | Native Opus 5 fallback, cycle 2, head `706c2bf05` | -| Static / Fitness | NOT_RUN | Phase 2 | -| Docs / Release | NOT_RUN | Phase 2 | -| Negative controls | NOT_RUN | Phase 2; raw exits mandatory | +| Static / Fitness | PARTIAL PASS | S1 release and S2 docs scoped wrappers green; S3 composition pending | +| Docs / Release | PARTIAL PASS | Docs accuracy/links green; publish dry-run pending S3 | +| Negative controls | COMPLETE | Both raw controls exited 1 with required diagnostics | S1 release unit, scoped check/lint/format, and raw missing-page exit 1 are PASS/expected-fail with diagnostics recorded in `worklog.md`. diff --git a/.llm/runs/fix-1377-gate--leaf/research.md b/.llm/runs/fix-1377-gate--leaf/research.md index 5de3c77297..da26607d56 100644 --- a/.llm/runs/fix-1377-gate--leaf/research.md +++ b/.llm/runs/fix-1377-gate--leaf/research.md @@ -23,7 +23,7 @@ | 6 | The existing `firstPublishFixture()` can seed a missing reference page. Tests can reuse it after reference existence is separated from first-publish policy. | `.llm/tools/release/publish-readiness_test.ts:369-415` | | 7 | The live public command tree already has a recursive, tree-derived catalog: `PublicCliCommandCatalog` walks `getCommands()` and emits paths. No source parser or literal command list is needed. | `packages/cli/src/public/features/agent/mcp/cli-mcp-adapters.ts:7-51` | | 8 | The live catalog currently has 15 root entries, 91 root-or-immediate-child entries, and 149 recursive entries. The five `ui:*` commands are colon-form root entries, not nested `ui` children. | focused `deno eval --unstable-kv` against `createPublicCommandRegistry()` and `PublicCliCommandCatalog` | -| 9 | Corpus choice is load-bearing: `docs/site/reference/cli/commands.md` alone misses 25/91, `docs/site/cli-reference.md` alone misses 25/91, and their exact union misses 4/91 (`deploy start`, `deploy stop`, `deploy status`, `deploy uninstall`). | focused tree-derived census against each file and their union; evaluator-confirmed FAIL_PLAN B1/B2 | +| 9 | Corpus choice is load-bearing. Before the bounded four-row repair, naive substring crediting reports 25/91 missing from each page; the locked structural/tokenized matcher reports 26/91 missing from `commands.md` and 32/91 from `cli-reference.md`. Their exact union misses the same 4/91 under every measured model (`deploy start`, `deploy stop`, `deploy status`, `deploy uninstall`). | focused tree-derived census against each file and their union; evaluator-confirmed FAIL_PLAN B1/B2; implementation matcher recheck | | 10 | The command-reference checker is naturally part of `docs:accuracy`, whose implementation and tests already own textual command-policy checks. | `deno.json:82`; `.llm/tools/docs/check-accuracy-and-discoverability.ts` and `_test.ts` | | 11 | **Retraction:** carried research C-3 and this run's first plan incorrectly claimed `check-accuracy-and-discoverability.ts` hardcodes `docs/site/reference/sagas/index.md`. It does not; its only hardcoded reference path is the SDK page. The real second hardcoded-path consumer is `check-exports-drift.ts`, whose eight paths include none of the four aliases and whose expansion remains #1108's scope. | `rg "reference/sagas" .llm/tools/` → no matches; focused reads of both checkers | | 12 | `publish:dry-run` is a required final gate but may rewrite catalog-backed manifests under #1417. Status must be inspected immediately and incidental manifest/lock churn restored without committing it. | dispatch boundary; `.agents/skills/netscript-tools/SKILL.md` lock hygiene | diff --git a/.llm/runs/fix-1377-gate--leaf/worklog.md b/.llm/runs/fix-1377-gate--leaf/worklog.md index 7dae570da7..919d6204c4 100644 --- a/.llm/runs/fix-1377-gate--leaf/worklog.md +++ b/.llm/runs/fix-1377-gate--leaf/worklog.md @@ -98,6 +98,9 @@ and coverage diagnostic name any missing path. Do not edit a command list in the | 2026-08-12 | S1 | Start | Began whole-publish-set reference existence implementation after PASS. | | 2026-08-12 | S1 | Implement | Added the independent `docs-reference` row over `publishSet.effective`, four declared aliases, and removed page existence from first-publish policy. | | 2026-08-12 | S1 | Reconcile | PR #1586 remains draft with `Closes #1377`; issue #1377 remains open. Read current PR/issue comments; no new blocker or scope change. Labels were observed but not changed. | +| 2026-08-12 | S2 | Implement | Added the two-file `COMMAND_REFERENCE_PATHS` gate over the live recursive catalog, exact 91 census, strict structural roots, tokenized direct-parent projection, colon handling, and new-path diagnostics. Added four deploy rows and bounded union-contract corrections in the two corpus pages plus the reference index. | +| 2026-08-12 | S2 | Toolchain seam | Static imports pulled unrelated existing CLI `isolatedDeclarations` failures into the focused docs test. Switched only module loading to dynamic structural types; runtime still materializes `createPublicCommandRegistry()` through `PublicCliCommandCatalog`, with no source parser or literal verb list. | +| 2026-08-12 | S2 | Reconcile | PR remains draft and issue #1377 remains open with `Closes #1377` in the PR body. No labels changed; the orchestrator retains lifecycle ownership. | ## Decisions @@ -121,7 +124,7 @@ All implementation gates are `NOT_RUN` by design in phase 1. | Reference arrival coverage | PASS | 35 effective, 35 present under locked alias resolver, 0 missing | | PLAN-EVAL | FAIL_PLAN → PASS | Native Opus 5 read-only fallback: cycle 1 failed `5ba4bc339`; cycle 2 passed `706c2bf05`. The generator did not self-evaluate. | | Type/lint/fmt/docs/publish/tests | NOT_RUN | Phase 2 only after PLAN-EVAL PASS | -| Raw negative controls | NOT_RUN | Phase 2 only; both raw exit codes and diagnostics required | +| Raw negative controls | COMPLETE | Missing page exit 1 in S1; missing command exit 1 in S2; diagnostics below | ### S1 gate evidence @@ -133,6 +136,19 @@ All implementation gates are `NOT_RUN` by design in phase 1. | Scoped release lint | PASS | 42 files selected; 0 findings | | Scoped release format | PASS | 42 files selected; 0 findings after targeted formatting | +### S2 gate evidence + +| Gate | Result | Evidence / notes | +| --- | --- | --- | +| Docs checker unit | PASS | `deno test --allow-all --unstable-kv .llm/tools/docs/check-accuracy-and-discoverability_test.ts` — 7 passed, 0 failed | +| Missing-command raw control | EXPECTED FAIL, exit 1 | Live recursive tree plus the exact locked two-page corpus with `netscript deploy uninstall` removed in memory; diagnostic: `public command reference covers 90/91; missing netscript deploy uninstall` | +| Corpus measurement | PASS | After the four-row repair: detailed page 69/91, curated page 59/91, union 91/91; live recursive census 149 | +| Docs accuracy | PASS | 91/91 root/direct commands from 149 recursive paths; baseline peer-dependency warning remained non-verdict | +| Docs links | PASS | 102 docs; 0 broken links, 0 broken anchors, 0 enforced orphans | +| Scoped docs check | PASS | 22 files selected; 0 findings | +| Scoped docs lint | PASS | 22 files selected; 0 findings | +| Scoped docs format | PASS | 22 files selected; 0 findings | + ## Handoff Notes - Evaluator should verify the exact two-file corpus, structural root plus tokenized path matching, diff --git a/.llm/tools/docs/check-accuracy-and-discoverability.ts b/.llm/tools/docs/check-accuracy-and-discoverability.ts index c535f88698..46cf2079f9 100644 --- a/.llm/tools/docs/check-accuracy-and-discoverability.ts +++ b/.llm/tools/docs/check-accuracy-and-discoverability.ts @@ -1,5 +1,7 @@ /** Guards genuinely textual documentation policy against surface drift. */ +import { resolve } from '@std/path'; + const root = new URL('../../../', import.meta.url); async function read(path: string, base: URL = root): Promise { @@ -28,11 +30,100 @@ export const ALLOWED_FRESH_ROOT_SYMBOLS: Set = new Set([ const GOLDEN_PATH_QUERY_UTILS_PAGE = 'docs/site/reference/sdk/index.md'; +/** The complete Markdown corpus accepted as public command documentation. */ +export const COMMAND_REFERENCE_PATHS: readonly string[] = [ + 'docs/site/reference/cli/commands.md', + 'docs/site/cli-reference.md', +]; + +/** Ratified root plus direct-child command census for the current public tree. */ +export const EXPECTED_PUBLIC_DIRECT_COMMAND_COUNT = 91; + export interface GoldenPathDocsResult { readonly pageCount: number; readonly queryUtilsPageCount: number; } +export interface CommandReferenceResult { + readonly auditedCount: number; + readonly documentedCount: number; + readonly recursiveCount: number; + readonly missing: readonly string[]; +} + +/** Audit root/direct-child command coverage against the locked Markdown corpus. */ +export function auditPublicCommandReference( + recursiveCommandPaths: readonly string[], + markdownSources: readonly string[], +): CommandReferenceResult { + const catalogPaths = [ + ...new Set(recursiveCommandPaths.map((path) => path.trim()).filter(Boolean)), + ] + .sort(); + const rootPaths = catalogPaths.filter((path) => commandDepth(path) === 1); + const directPaths = catalogPaths.filter((path) => commandDepth(path) === 2); + const auditedPaths = [...rootPaths, ...directPaths].sort(); + const resolvedInvocations = markdownSources.flatMap((source) => + resolveDocumentedInvocations(source, catalogPaths) + ); + const documented = new Set(); + + for (const rootPath of rootPaths) { + if ( + markdownSources.some((source) => hasStructuralRootDeclaration(source, rootPath)) || + resolvedInvocations.includes(rootPath) + ) { + documented.add(rootPath); + } + } + for (const directPath of directPaths) { + if ( + resolvedInvocations.some((resolvedPath) => + resolvedPath === directPath || resolvedPath.startsWith(`${directPath} `) + ) + ) { + documented.add(directPath); + } + } + + const missing = auditedPaths + .filter((path) => !documented.has(path)) + .map(renderCommandPath); + return { + auditedCount: auditedPaths.length, + documentedCount: auditedPaths.length - missing.length, + recursiveCount: catalogPaths.length, + missing, + }; +} + +/** Fail when the live command census or locked-corpus coverage drifts. */ +export function checkPublicCommandReference( + recursiveCommandPaths: readonly string[], + markdownSources: readonly string[], + expectedCount: number = EXPECTED_PUBLIC_DIRECT_COMMAND_COUNT, +): CommandReferenceResult { + const result = auditPublicCommandReference(recursiveCommandPaths, markdownSources); + if (result.auditedCount !== expectedCount) { + const lead = result.missing.length > 0 + ? `; unratified command path(s): ${result.missing.join(', ')}` + : `; audited paths: ${ + deriveAuditedPaths(recursiveCommandPaths).map(renderCommandPath).join(', ') + }`; + throw new Error( + `public command census expected ${expectedCount}, got ${result.auditedCount}${lead}`, + ); + } + if (result.missing.length > 0) { + throw new Error( + `public command reference covers ${result.documentedCount}/${result.auditedCount}; missing ${ + result.missing.join(', ') + }`, + ); + } + return result; +} + /** Enforce the published golden-path module, alias, and query-dialect vocabulary. */ export async function checkGoldenPathDocs( dirPath = 'docs/site', @@ -170,6 +261,13 @@ export async function runAccuracyCheck(): Promise { checkSagaVocabulary(publicDocs, sagaPagePaths); const cliReference = await read('docs/site/cli-reference.md'); + const commandReferenceSources = await Promise.all( + COMMAND_REFERENCE_PATHS.map((path) => read(path)), + ); + const commandReference = checkPublicCommandReference( + await listPublicCommandPaths(), + commandReferenceSources, + ); const goldenPathDocs = await checkGoldenPathDocs(); checkMutationMapColumns(cliReference); @@ -188,10 +286,118 @@ export async function runAccuracyCheck(): Promise { } console.log( - `docs accuracy: PASS (${publicDocs.length} saga pages checked for stale claims, ${goldenPathDocs.pageCount} published source pages, one query dialect exception page, mutation-map columns, ${checkedFreshRootImports} valid @netscript/fresh root imports checked)`, + `docs accuracy: PASS (${publicDocs.length} saga pages checked for stale claims, ${goldenPathDocs.pageCount} published source pages, one query dialect exception page, mutation-map columns, ${commandReference.documentedCount}/${commandReference.auditedCount} root/direct public commands from ${commandReference.recursiveCount} recursive paths, ${checkedFreshRootImports} valid @netscript/fresh root imports checked)`, ); } +/** Materialize the live public CLI registry through its recursive catalog adapter. */ +export async function listPublicCommandPaths(): Promise { + interface PublicCliHost { + readonly cwd: () => string; + readonly resolvePath: (path?: string) => string; + } + interface EnumerableCommand { + readonly getName: () => string; + readonly getDescription: () => string; + readonly getCommands: () => readonly EnumerableCommand[]; + } + interface PublicTreeModule { + readonly createPublicCommandRegistry: () => { + readonly program: (input: Record) => EnumerableCommand; + }; + } + interface PublicDependenciesModule { + readonly createPublicCommandDependencies: (host: PublicCliHost) => unknown; + } + interface PublicCatalogModule { + readonly PublicCliCommandCatalog: new (root: EnumerableCommand) => { + readonly listCommands: () => Promise; + }; + } + + const treeUrl = new URL( + '../../../packages/cli/src/public/features/root/public-command-tree.ts', + import.meta.url, + ).href; + const dependenciesUrl = new URL( + '../../../packages/cli/src/public/features/root/public-command-dependencies.ts', + import.meta.url, + ).href; + const catalogUrl = new URL( + '../../../packages/cli/src/public/features/agent/mcp/cli-mcp-adapters.ts', + import.meta.url, + ).href; + const [treeModule, dependenciesModule, catalogModule] = await Promise.all([ + import(treeUrl) as Promise, + import(dependenciesUrl) as Promise, + import(catalogUrl) as Promise, + ]); + const host: PublicCliHost = { + cwd: () => Deno.cwd(), + resolvePath: (path) => resolve(Deno.cwd(), path ?? '.'), + }; + const rootCommand = treeModule.createPublicCommandRegistry().program({ + name: 'netscript', + version: 'docs-accuracy', + description: 'Public command documentation accuracy check', + context: { + host, + dependencies: dependenciesModule.createPublicCommandDependencies(host), + }, + }); + return (await new catalogModule.PublicCliCommandCatalog(rootCommand).listCommands()).map( + ({ path }) => path, + ); +} + +function resolveDocumentedInvocations( + markdown: string, + catalogPaths: readonly string[], +): readonly string[] { + const catalogTokens = catalogPaths + .map((path) => ({ path, tokens: path.split(' ') })) + .sort((left, right) => right.tokens.length - left.tokens.length); + const resolved: string[] = []; + for ( + const match of markdown.matchAll( + /\bnetscript[ \t]+([a-z0-9][a-z0-9:-]*(?:[ \t]+[a-z0-9][a-z0-9:-]*)*)/gi, + ) + ) { + const tokens = match[1].toLowerCase().split(/[ \t]+/); + const command = catalogTokens.find(({ tokens: commandTokens }) => + commandTokens.every((token, index) => tokens[index] === token) + ); + if (command) resolved.push(command.path); + } + return resolved; +} + +function hasStructuralRootDeclaration(markdown: string, rootPath: string): boolean { + const escaped = escapeRegExp(rootPath); + return new RegExp( + `^#{1,6}[ \\t]+\\\`${escaped}\\\`(?:[ \\t]+—.*)?$`, + 'im', + ).test(markdown); +} + +function deriveAuditedPaths(recursiveCommandPaths: readonly string[]): readonly string[] { + return [...new Set(recursiveCommandPaths.map((path) => path.trim()).filter(Boolean))] + .filter((path) => commandDepth(path) <= 2) + .sort(); +} + +function commandDepth(path: string): number { + return path.split(' ').length; +} + +function renderCommandPath(path: string): string { + return `netscript ${path}`; +} + +function escapeRegExp(value: string): string { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + if (import.meta.main) { await runAccuracyCheck(); } diff --git a/.llm/tools/docs/check-accuracy-and-discoverability_test.ts b/.llm/tools/docs/check-accuracy-and-discoverability_test.ts index 6192418275..9eb11a9f7b 100644 --- a/.llm/tools/docs/check-accuracy-and-discoverability_test.ts +++ b/.llm/tools/docs/check-accuracy-and-discoverability_test.ts @@ -4,6 +4,7 @@ import { checkFreshRootImports, checkGoldenPathDocs, checkMutationMapColumns, + checkPublicCommandReference, checkSagaVocabulary, } from './check-accuracy-and-discoverability.ts'; @@ -133,3 +134,54 @@ Deno.test('checkMutationMapColumns requires the heading and all five policy colu ); } }); + +Deno.test('public command coverage uses structural roots, direct-parent projection, and colon roots', () => { + const paths = [ + 'agent', + 'agent drift', + 'agent drift record', + 'agent init', + 'ui:add', + ]; + const result = checkPublicCommandReference(paths, [ + [ + '## `agent` — agent tooling', + '`netscript agent init` installs tooling.', + '`netscript agent drift record` records drift.', + '`netscript ui:add` copies a registry item.', + ].join('\n'), + '# Curated companion\n', + ], 4); + assertEquals(result, { + auditedCount: 4, + documentedCount: 4, + recursiveCount: 5, + missing: [], + }); +}); + +Deno.test('public command coverage does not credit a descendant or sibling to a missing path', () => { + const paths = ['agent', 'agent init', 'agent mcp']; + assertThrows( + () => + checkPublicCommandReference(paths, [ + 'The command `netscript agent init` installs tooling.\n', + 'No group heading is present.\n', + ], 3), + Error, + 'missing netscript agent, netscript agent mcp', + ); +}); + +Deno.test('public command census drift names the newly appeared undocumented path', () => { + assertThrows( + () => + checkPublicCommandReference( + ['agent', 'agent init', 'agent mcp'], + ['## `agent`\n`netscript agent init`\n'], + 2, + ), + Error, + 'expected 2, got 3; unratified command path(s): netscript agent mcp', + ); +}); diff --git a/docs/site/cli-reference.md b/docs/site/cli-reference.md index 3df613a9a4..1e1f0661ff 100644 --- a/docs/site/cli-reference.md +++ b/docs/site/cli-reference.md @@ -285,15 +285,15 @@ See [Agent tooling](/ai/agent-tooling/) for the mental model. ## The full surface -This page is the curated common path. For every command, every subcommand, and every -flag — spelled exactly as the installed CLI prints it — go to the -[command reference](/reference/cli/commands/); for the embeddable package API, the +This page is the curated common path; together with the detailed +[command reference](/reference/cli/commands/), the two-page set covers every public command group +and direct subcommand. For the embeddable package API, use the [`@netscript/cli` package page]({{ "ref:cli" |> xref }}). {{ comp.featureGrid({ items: [ { title: "Command reference", - body: "The exhaustive command surface — every command, subcommand, and flag verbatim.", + body: "The detailed companion for extended command groups and flags; use it with this curated page for the complete public root/direct-command surface.", href: "/reference/cli/commands/", icon: "≡" }, diff --git a/docs/site/reference/cli/commands.md b/docs/site/reference/cli/commands.md index 197ea22ce7..8ebdd6bb82 100644 --- a/docs/site/reference/cli/commands.md +++ b/docs/site/reference/cli/commands.md @@ -5,11 +5,11 @@ title: "netscript command reference" # `netscript` command reference -The complete verb-and-flag surface of the published `netscript` binary -(`@netscript/cli`). The [CLI reference](/cli-reference/) is the curated -tour of the everyday path; this page is the exhaustive companion — every command group, -subcommand, and flag, spelled exactly as the installed CLI prints it. For the package API -surface (the embeddable helpers, not the command tree) see the +The detailed verb-and-flag companion for the published `netscript` binary +(`@netscript/cli`). Together with the curated [CLI reference](/cli-reference/), these two +pages cover every public command group and direct subcommand; this page concentrates the extended +groups and flags while the curated page carries the everyday flows. For the package API surface +(the embeddable helpers, not the command tree) see the [`@netscript/cli` package page](/reference/cli/). This page documents the **public `netscript` binary** — the CLI published as @@ -299,5 +299,9 @@ the bundle step fails, so `--no-bundle` is a size/behaviour trade-off rather tha | Command | Description | | --- | --- | | `netscript deploy copy` | Copy build artifacts to the install directory (no Servy registration). Flags: `--deploy-dir ` (default `./.deploy/windows`), `--install-dir ` (default auto-resolved from the manifest), `--verbose`, `--dry-run`. | +| `netscript deploy start [service]` | Start all registered Windows services, or one named service, via Servy. Flags: `--install-dir `, `--deploy-dir `, `--servy-cli `, `--no-health-check`, `--verbose`. | +| `netscript deploy stop [service]` | Stop all registered Windows services in reverse manifest order, or stop one named service. Flags: `--install-dir `, `--deploy-dir `, `--servy-cli `, `--verbose`. | +| `netscript deploy status [service]` | Show Servy status for all registered Windows services, or one named service. Flags: `--install-dir `, `--deploy-dir `, `--servy-cli `, `--verbose`. | | `netscript deploy logs ` | Show recent logs for a service. Flags: `--install-dir `, `--deploy-dir ` (default `./.deploy/windows`), `-n, --lines ` (default 50), `--errors` (show the error log instead of stdout), `--list` (list available log files), `-f, --follow` (tail the log). | +| `netscript deploy uninstall [service]` | Stop and remove all registered Windows services, or one named service, from the service manager. Flags: `--install-dir `, `--deploy-dir `, `--no-stop`. | | `netscript deploy upgrade` | Build, sync, reinstall, and restart services in one step. Flags: `--deploy-dir ` (default `./.deploy/windows`), `--install-dir `, `--servy-cli `, `--skip-compile`, `--skip-install`, `--skip-start`, `--verbose`, `--dry-run`. | diff --git a/docs/site/reference/index.md b/docs/site/reference/index.md index a644dbbce1..f09a44770b 100644 --- a/docs/site/reference/index.md +++ b/docs/site/reference/index.md @@ -43,3 +43,10 @@ publishable workspace surface and gives each API claim one place to stay current Navigation is derived from the folder tree, so a new page is discoverable as soon as its directory exists — no index or nav file lists the packages. + +## CLI command coverage + +The public command documentation is the union of exactly two pages: the curated +[CLI reference](/cli-reference/) and the detailed [`netscript` command +reference](/reference/cli/commands/). Together they cover every public root command and direct +subcommand derived from the installed command tree; neither page is treated as exhaustive alone. From ddd2c46b8d6873c5a474da642d64f37da20532cc Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Wed, 12 Aug 2026 19:02:02 +0200 Subject: [PATCH 6/6] chore(harness): record #1377 implementation evidence --- .llm/runs/fix-1377-gate--leaf/context-pack.md | 36 +++++++++++------- .llm/runs/fix-1377-gate--leaf/worklog.md | 37 ++++++++++++++++--- 2 files changed, 54 insertions(+), 19 deletions(-) diff --git a/.llm/runs/fix-1377-gate--leaf/context-pack.md b/.llm/runs/fix-1377-gate--leaf/context-pack.md index dad1867f12..4e412f3632 100644 --- a/.llm/runs/fix-1377-gate--leaf/context-pack.md +++ b/.llm/runs/fix-1377-gate--leaf/context-pack.md @@ -6,17 +6,17 @@ | --- | --- | | Run ID | `fix-1377-gate--leaf` | | Branch | `fix/1377-docs-reference-gate-scope` | -| Current phase | `implement` — S2 ready to commit | +| Current phase | `implement` — S3 complete; IMPL-EVAL pending | | Archetype | 6 — CLI / tooling | | Scope overlays | Docs | ## Current State Fallback PLAN-EVAL cycle 1 returned `FAIL_PLAN` on `5ba4bc339`; cycle 2 returned `PASS` on -`706c2bf05`. Phase 2 is active in S1. The implementation retains the confirmed alias and -whole-publish-set design, locks command coverage to the exact two-page union, requires structural -root matching plus tokenized path resolution and exact equality with 91 root/direct obligations, -and makes the four missing deploy rows bounded S2 scope. +`706c2bf05`. S1–S3 are implemented and gate-green. The implementation retains the confirmed alias +and whole-publish-set design, locks command coverage to the exact two-page union, requires +structural root matching plus tokenized path resolution and exact equality with 91 root/direct +obligations, and includes only the four authorised deploy rows as bounded prose scope. ## Completed @@ -27,16 +27,22 @@ and makes the four missing deploy rows bounded S2 scope. - 35/35 alias-resolved reference arrival coverage verified. - FAIL_PLAN B1–B3 and non-blocking findings resolved in the revised plan. - Native Opus 5 fallback PLAN-EVAL cycle 2 passed immutable head `706c2bf05`. +- S1 whole-publish-set reference coverage committed as `dedd7804508f6cbe0d1ce97174cb7642f49dc96a`. +- S2 tree-derived command coverage committed as `29557faa858e832d8cabdc452ed4692ae26031e2`. +- Both raw negative controls exited 1 and named the missing page/command. +- S3 composed gates passed, including publish dry-run with no worktree churn and the full repository + suite with 3,258 passed, 0 failed, and 17 ignored. ## In Progress -- S2 is gate-green and ready for its sign-off commit/push/comment. +- Final run-artifact commit, explicit push, PR/issue evidence update, then stop. ## Next Steps -1. Commit S2, explicit push, and PR comment. -2. Complete S3 composed gates and lock-hygiene inspection. -3. Finalize run/PR evidence and stop for native Opus 5 IMPL-EVAL fallback. +1. Commit and push the final S3 evidence. +2. Update PR #1586 and truthfully tick #1377's completed gate rows, leaving the post-merge row + unticked. +3. Stop for orchestrator-dispatched native Opus 5 IMPL-EVAL on the final immutable head. ## Key Decisions @@ -60,12 +66,12 @@ and makes the four missing deploy rows bounded S2 scope. | Gate family | Current status | Evidence | | --- | --- | --- | | Plan | PASS | Native Opus 5 fallback, cycle 2, head `706c2bf05` | -| Static / Fitness | PARTIAL PASS | S1 release and S2 docs scoped wrappers green; S3 composition pending | -| Docs / Release | PARTIAL PASS | Docs accuracy/links green; publish dry-run pending S3 | +| Static / Fitness | PASS | Release wrappers: 42 files; docs wrappers: 22 files; 0 findings | +| Docs / Release | PASS | Accuracy 91/91 from 149; links 102/0/0; publish dry-run exit 0 with clean status | | Negative controls | COMPLETE | Both raw controls exited 1 with required diagnostics | +| Repository tests | PASS | 3,258 passed (622 steps), 0 failed, 17 ignored in 8m46s | -S1 release unit, scoped check/lint/format, and raw missing-page exit 1 are PASS/expected-fail with -diagnostics recorded in `worklog.md`. +Gate counts, raw exits, and diagnostics are recorded in `worklog.md`. ## Open Questions @@ -78,4 +84,6 @@ diagnostics recorded in `worklog.md`. ## Commits -- See the draft PR commit list + per-slice phase comments after push. +- `dedd7804508f6cbe0d1ce97174cb7642f49dc96a` — S1 whole-publish-set reference audit. +- `29557faa858e832d8cabdc452ed4692ae26031e2` — S2 public-tree command-reference audit. +- S3 evidence commit: pending final sign-off commit. diff --git a/.llm/runs/fix-1377-gate--leaf/worklog.md b/.llm/runs/fix-1377-gate--leaf/worklog.md index 919d6204c4..bf48af3b9f 100644 --- a/.llm/runs/fix-1377-gate--leaf/worklog.md +++ b/.llm/runs/fix-1377-gate--leaf/worklog.md @@ -101,6 +101,9 @@ and coverage diagnostic name any missing path. Do not edit a command list in the | 2026-08-12 | S2 | Implement | Added the two-file `COMMAND_REFERENCE_PATHS` gate over the live recursive catalog, exact 91 census, strict structural roots, tokenized direct-parent projection, colon handling, and new-path diagnostics. Added four deploy rows and bounded union-contract corrections in the two corpus pages plus the reference index. | | 2026-08-12 | S2 | Toolchain seam | Static imports pulled unrelated existing CLI `isolatedDeclarations` failures into the focused docs test. Switched only module loading to dynamic structural types; runtime still materializes `createPublicCommandRegistry()` through `PublicCliCommandCatalog`, with no source parser or literal verb list. | | 2026-08-12 | S2 | Reconcile | PR remains draft and issue #1377 remains open with `Closes #1377` in the PR body. No labels changed; the orchestrator retains lifecycle ownership. | +| 2026-08-12 | S3 | Publish composition | `publish:dry-run` passed. The immediate raw status check was clean: #1417 produced no manifest or lock churn, so nothing was restored. | +| 2026-08-12 | S3 | Repository composition | Repeated both scoped wrapper families plus docs links/accuracy; all passed. The full repository suite passed 3,258 tests (622 steps), with 0 failures and 17 ignored. The snippet extractor's real-corpus test explicitly confirmed Lume `_site` output remains excluded. | +| 2026-08-12 | S3 | Reconcile | Final implementation evidence is complete. PR #1586 remains draft; no `status:` label, ready-state, or merge mutation was performed. Native Opus 5 IMPL-EVAL remains an orchestrator-owned next step on the final immutable head. | ## Decisions @@ -115,7 +118,7 @@ and coverage diagnostic name any missing path. Do not edit a command list in the ## Gate Results -All implementation gates are `NOT_RUN` by design in phase 1. +All planned implementation and composition gates completed after the cycle-2 PLAN-EVAL PASS. | Gate | Result | Evidence / notes | | --- | --- | --- | @@ -123,7 +126,7 @@ All implementation gates are `NOT_RUN` by design in phase 1. | PR-C landed | PASS | `db1d79c68…` is an ancestor of HEAD | | Reference arrival coverage | PASS | 35 effective, 35 present under locked alias resolver, 0 missing | | PLAN-EVAL | FAIL_PLAN → PASS | Native Opus 5 read-only fallback: cycle 1 failed `5ba4bc339`; cycle 2 passed `706c2bf05`. The generator did not self-evaluate. | -| Type/lint/fmt/docs/publish/tests | NOT_RUN | Phase 2 only after PLAN-EVAL PASS | +| Type/lint/fmt/docs/publish/tests | PASS | S1/S2 focused gates and S3 composed gates are recorded below | | Raw negative controls | COMPLETE | Missing page exit 1 in S1; missing command exit 1 in S2; diagnostics below | ### S1 gate evidence @@ -149,8 +152,32 @@ All implementation gates are `NOT_RUN` by design in phase 1. | Scoped docs lint | PASS | 22 files selected; 0 findings | | Scoped docs format | PASS | 22 files selected; 0 findings | +### S3 composed gate evidence + +| Gate | Result | Evidence / notes | +| --- | --- | --- | +| Docs accuracy | PASS | 91/91 root/direct commands from 149 recursive paths; baseline peer-dependency warning remained non-verdict | +| Docs links | PASS | 102 docs; 0 broken links, 0 broken anchors, 0 enforced orphans | +| Scoped release check/lint/format | PASS | 42 files selected per wrapper; 0 findings in each | +| Scoped docs check/lint/format | PASS | 22 files selected per wrapper; 0 findings in each | +| Publish dry-run | PASS, exit 0 | `rtk proxy deno task publish:dry-run`; dry run completed successfully | +| Immediate post-dry-run status | PASS | Raw `git status --short` was empty; no catalog-backed manifest churn and no `deno.lock` churn occurred | +| Repository tests | PASS, exit 0 | `rtk proxy deno task test` — 3,258 passed (622 steps), 0 failed, 17 ignored in 8m46s | +| Snippet `_site` exclusion | PASS | Repository suite ran `site analysis excludes Lume build output without shrinking source coverage`; it passed | + +### Raw negative-control evidence + +The controls invoked the exported production audits directly, one seeded failure at a time. They +were not tests that converted an expected failure into exit 0. + +| Control | Raw result | Diagnostic | +| --- | --- | --- | +| Published package without reference page | exit 1 | `@netscript/published docs/site/reference/published/index.md [docs-reference] docs-site reference page is required` | +| Live-tree subcommand absent from locked two-file corpus | exit 1 | `public command reference covers 90/91; missing netscript deploy uninstall` | + ## Handoff Notes -- Evaluator should verify the exact two-file corpus, structural root plus tokenized path matching, - exact 91 equality, and bounded four-row ownership satisfy B1–B3 without expanding into #1108. -- No implementation file has been edited. +- Native Opus 5 IMPL-EVAL should verify the exact two-file corpus, structural root plus tokenized + path matching, exact 91 equality, bounded four-row ownership, and both raw exit-1 controls. +- The PR remains draft. The orchestrator owns the IMPL-EVAL dispatch, any `status:` transition, and + the ready/merge decision.