Skip to content

RFC: NetScript DevTools contribution architecture — plugins that ship developer tooling - #1450

Merged
rickylabs merged 58 commits into
mainfrom
plan/devtools-contribution
Aug 11, 2026
Merged

RFC: NetScript DevTools contribution architecture — plugins that ship developer tooling#1450
rickylabs merged 58 commits into
mainfrom
plan/devtools-contribution

Conversation

@rickylabs

@rickylabs rickylabs commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Acceptance update — 2026-08-11

Owner ratified the completed RFC after all 22 adversarial/design findings were dispositioned and the separate PLAN-EVAL passed. The canonical accepted document is rfcs/0005-devtools-contribution.md; board filing remains a post-merge operation so every issue can cite the immutable main path.

Summary

Path normalized 2026-08-11 (owner decision, drift D-20). The RFC now lives at
rfcs/0005-devtools-contribution.md, the repository's canonical RFC process. The earlier
docs/architecture/rfc/rfc-0002-… location came from unmerged PR #1446, and RFC 0005 is assigned under rfcs/README.md at acceptance. Board filing is
paused while root coordinates acceptance numbering and merge order.

Planning-only seed run for the NetScript DevTools Contribution Architecture RFC: a first-class
host and contribution family through which plugins add developer-facing routes, panels, inspectors,
visualizers, actions, commands, diagnostics, navigation, and deep-links.

The deliverable is architecture and contribution mechanics, not a visual refresh of the old Dev
Dashboard (#400). This PR is the commit trail for that run and is accepted and authorized to merge after required CI; board mutation remains a post-merge operation.

Scope

The five frontend contribution surfaces

The RFC models these as distinct seams and decides dependency + ownership for each, rather than
collapsing them into one vague frontend axis:

# Surface Owner This RFC
1 Userland frontend code (routes/islands/nav/theme/zones) RFC #890 → epic #922 consume; state the boundary
2 Fresh UI registry / component / style-dictionary contributions packages/fresh-ui decide: extend existing CLI commands safely, or stage a follow-up RFC
3 Vite plugin contributions unowned deferred unless a minimal safe contract is proven — ordering, trust, build determinism, resolution, failure containment explicit either way
4 DevTools contributions unowned — this RFC the primary subject
5 SDK contributions RFC #1390 / #1348 consume; never duplicate

Slices

  • S1 Bootstrap — supervisor identity, run artifacts, verified baseline, charter, draft PR — ccc4c0a70
  • S2 Discovery corpus (stage B) — repo + docs + primary-source market study, every claim cited
  • S3 Synthesis (stage C) — full-corpus read → research.md findings + deep-dive topics
  • S4 Deep-dive design packs (stage D) — one pack per topic: proposal + draft epic/issues + agent briefs + open questions
  • S5 Design/UX evidence pack (stage D2) — mandatory major-UI/UX design pass, per-finding dispositions
  • S6 Canonical RFC + plan lock (stage E) — all twelve charter questions locked or escalated as numbered owner forks
  • S7 Adversarial review + triage (stage F) — unoriented reviewer on a model distinct from every authoring lane
  • S8 Supersession map, filing manifest, agent briefs, owner decision brief (stage H-prep) — draft text only

Validation

Docs/RFC changeset — no TypeScript, no packages/** or plugins/** source. The cheap CI lane is
intentional, which is why ci:skip-e2e + ci:skip-scaffold are applied rather than left to path
classification.

Gate Command Result
Markdown format .llm/tools/run-deno-fmt.ts (scoped to changed docs + run dir) NOT_RUN — stage I
Doc lint / links deno task doc:lint NOT_RUN — stage I
Docs source + rendered CI gates PR checks (#1440) pending on this PR
Citation gate every load-bearing claim → path:line, deno doc, saved artifact, or URL NOT_RUN — stage B
Live-board dedup every issue draft checked against live GitHub before entering the filing manifest NOT_RUN — stage H-prep
Planned-surface jsr-audit rubric applied to the RFC's proposed API sketches NOT_RUN — stage E
PLAN-EVAL Codex · GPT-5.6 Sol · high, separate daemon-attached session, immutable commit NOT_RUN — stage G, hard stop

deno task check / test / lint / arch:check / quality:scan / e2e:cli are N/A for this
changeset — stated rather than silently omitted.

Harness

  • Run dir: .llm/runs/plan-devtools-contribution--seed/
  • Run shape: seed run (.llm/harness/workflow/seed-run.md), stages A–I
  • Supervisor: Claude Opus 5 · high — identity, routes, and mutation boundary in
    .llm/runs/plan-devtools-contribution--seed/supervisor.md
  • Baseline: 2256a67bf (origin/main), verified by git fetch at bootstrap — charter and live main
    agreed; no rebase
  • Charter: .llm/devtools-rfc-orchestrator-brief.md, committed so the run is reconstructible without
    chat history
  • Phase: acceptance — see phase comments below

Evaluation

  • PLAN-EVAL: SELECTED — twelve open architecture decisions and multi-PR board output. Fresh
    daemon-attached Codex GPT-5.6 Sol high session, separate from every generator/reviewer session.
  • IMPL-EVAL: N/A by run shape, not by owner waiver — the run commits no implementation. Substitute
    assurance is the opposite-family PLAN-EVAL plus the docs gate set above. Recorded in drift.md D-2.

Drift / Debt

  • D-1 — the major_ui_ux_* GLM 5.2 lane is marked dormant in lane-policy.md while the Dev
    Dashboard is paused; this run reactivates it for its charter-mandated design pass. minor, accepted.
  • D-2 — IMPL-EVAL N/A by run shape (above). minor, accepted.
  • D-3 — GLM 5.2 over OpenRouter returns zero reasoning trace (tools + streaming only).
    Pre-registered before the design pass runs so no artifact can later cite it as reasoning
    evidence. minor, accepted.
  • Debt: none created. Any doctrine violation the RFC elects to defer lands in
    .llm/harness/debt/arch-debt.md at stage E, not silently in the RFC.

Definition of Done

  • Canonical RFC at rfcs/0005-devtools-contribution.md with diagrams, normative contracts, API examples,
    explicit alternatives, threat model, lifecycle, failure behavior, package ownership, frontend
    host split, and implementation roadmap
  • Cited current-state matrix and primary-source market/competitor architecture study
  • Five-surface frontend contribution map with dependencies and proven non-overlap
  • DevTools host/contribution-family design pack with worked plugin examples and contributor DX
  • Design/UX evidence pack — owner-approved substitute route (Qwen 3.8 Max + Kimi K3, drift D-15/D-16), 22 findings dispositioned
  • File-level and issue-level supersession map for Epic: NetScript DevTools contribution and host architecture #400 and every relevant child/PR
  • Draft epic + one-file-per-issue set, milestone proposal, dependency DAG, agent briefs, and a
    committed one-shot filing manifest — draft text only
  • Owner decision brief listing every genuine fork, with no rework-forcing decision hidden under
    "safe to defer"
  • Formal PLAN-EVAL run twice against immutable commits (FAIL_PLAN ×2); every supervisor-fixable
    finding closed; Plan-Gate then cleared by written owner waiver (drift D-18) — not an evaluator PASS
  • Docs gates green (docs:links 3 docs / 0 broken, docs:accuracy PASS); worktree clean and pushed
  • This PR remains a draft pending owner ratification and board-filing authorization

Do not merge. Merge requires status:ready-merge, which a status:research / status:plan /
status:plan-eval artifact set can never satisfy. This run does not merge itself, and it files no
board entry before the owner ratifies the decision brief in-turn.

Proves the run is activated with a recorded operating identity and a
verified baseline before any evidence claim is made.

- supervisor.md written first (Opus 5 high, this session, Remote Control
  on), per seed-run.md's hard invariant that a run dir without it is not
  activated
- baseline verified against live origin/main (2256a67) rather than
  taken from the charter on trust; no divergence, no rebase
- planning-only mutation boundary recorded explicitly: the branch and its
  draft PR are writable, the board is not until owner ratification
- twelve charter questions seeded as the open-decision docket in plan.md,
  each marked "must resolve now"
- drift D-1/D-2/D-3 pre-registered: GLM major-UI/UX lane reactivated from
  dormant, IMPL-EVAL N/A by run shape, and GLM's no-reasoning-trace
  transport limit recorded before its output exists
- charter committed to the branch so the run is reconstructible without
  chat history

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
@rickylabs rickylabs added this to the Backlog / Triage milestone Aug 11, 2026
@rickylabs rickylabs added rfc Request for Comments - substantial/breaking design change status:research Harness research phase ci:skip-e2e ci:skip-scaffold Skip the scaffold-static (deno-only) scaffold gate area:docs area:fresh area:fresh-ui area:plugins plugins/* and plugin-core packages priority:p1 High epic:dev-dashboard Dev Dashboard epic (Spine-1 headline, ships as a plugin) epic:frontend-contrib Frontend contribution layer epic (plugins that ship UI) labels Aug 11, 2026
@rickylabs

Copy link
Copy Markdown
Owner Author

[PHASE: RESEARCH] [STAGE: A — BOOTSTRAP]

Seed run activated. Supervisor identity recorded, baseline verified against live main, commit trail live.

Charter read-back

Read .llm/devtools-rfc-orchestrator-brief.md in full before acting; it is committed to this branch so the run is reconstructible without chat history. What I am accountable for:

Item Read-back
Deliverable A canonical RFC under docs/architecture/rfc/ deciding DevTools contribution architecture and mechanics — plus the board plan to implement it. Not a visual refresh of #400.
Primary subject Surface #4 of five frontend contribution seams: a first-class host/family for plugin-contributed routes, panels, inspectors, visualizers, actions, commands, diagnostics, navigation, deep-links.
Consume, don't duplicate #890/#922 userland app family · #1390/#1348 SDK contributions · #1446 runtime-automation backend contracts (its P-6 DevTools dependency, management/audit/convergence/OTel) · Aspire (resources, logs, traces, metrics, health, process lifecycle) · Scalar (API schema/reference/try-it).
Reconcile Epic #400 and every relevant child/PR (#685, #780, #506, #410#432) get a file-level and issue-level supersession mapKEEP / AMEND / FOLD / SUPERSEDE / CLOSE-LATER. Pre-modern-RFC dashboard design is research evidence, not ratified architecture.
Twelve decisions All twelve charter questions are seeded in plan.md § Open-Decision Sweep, every one marked must resolve now. Each must land as a locked decision with cited evidence, or as a numbered owner fork. No rework-forcing decision may hide under "safe to defer".
Hard boundaries No framework/product source. No board mutation before owner ratification in-turn. This run does not merge itself. No isolation/security/compatibility/performance/production-readiness claim without an executable gate or a citation.

Baseline

git fetch originorigin/main = 2256a67bf (docs(home): complete the capability outcome story (#1442)). The charter's stated baseline and live main agree — verified rather than assumed. No rebase performed, and none permitted for the rest of the run.

Identity and routes in force

Full table in .llm/runs/plan-devtools-contribution--seed/supervisor.md; routes are selected from .llm/harness/workflow/lane-policy.md, not invented here.

Lane Route Role
planning_decisions Claude · Opus 5 · high (this session, Remote Control on) Orchestration, synthesis, plan lock, RFC authoring, triage, sign-off commits
deep_analysis Claude · Fable 5 · medium Stage-D deep-dive design packs
claude_workflow Claude · Opus 5 · low Stage-B parallel discovery only, with workflow.js committed under the run dir before it executes
major_ui_ux_design GLM 5.2 · xhigh (claude-design-glm-5-2) Mandatory design pass — tools + streaming, no reasoning trace; never the evaluator
adversarial_design_eval Kimi K3 vision · high Conditional, only if visual artifacts materially improve architectural judgment
Stage-F reviewer model distinct from Opus 5 / Fable 5 / GLM 5.2, unoriented, separate session Severity-tagged findings only; supervisor triages
formal_plan_evaluation Codex · GPT-5.6 Sol · high, fresh daemon-attached session, own worktree PLAN-EVAL of record against an immutable commit

No OpenHands. OpenRouter is confined to the GLM design lane, the conditional Kimi lane, and an authorized fallback.

Evaluation posture

  • PLAN-EVAL: SELECTED — twelve open architecture decisions plus multi-PR board output clears the run-loop.md §4 bar comfortably. It is a hard stop: no stage-H filing before PASS.
  • IMPL-EVAL: N/A by run shape, recorded in drift.md D-2 rather than silently dropped. The run commits no implementation; substitute assurance is the opposite-family PLAN-EVAL plus the docs gate set.

CI lane — intentional, not incidental

The changeset is Markdown only: no TypeScript, no packages/** or plugins/** source. ci:skip-e2e and ci:skip-scaffold are applied proactively so the cheap lane is legible as a decision rather than an accident of path classification (netscript-harness § Workflow). deno task check / test / lint / arch:check / quality:scan / e2e:cli are N/A and stated as such in the PR body.

Drift pre-registered at bootstrap

  • D-1major_ui_ux_* GLM lane is dormant in lane-policy.md while the Dev Dashboard is paused; reactivated here under the charter's explicit mandate. The same policy entry says the lane "remains the enforced route for any major UI/UX work that does run", so this is a reactivation, not a route invention.
  • D-2 — IMPL-EVAL N/A by run shape.
  • D-3 — GLM 5.2 over OpenRouter yields zero reasoning trace. Pre-registered before the design pass runs, so no downstream artifact can cite "GLM 5.2 · xhigh reasoning" as gate evidence.

Stage A artifacts — commit ccc4c0a70

supervisor.md (written first, per the seed-run hard invariant) · research.md (re-baseline + E1–E7 evidence-input register) · plan.md (scope + twelve-question docket) · worklog.md (slice plan + gate tables) · context-pack.md · drift.md · phase-registry.md (stages A–I + the stage-D topic fan-out).

research.md's findings table is deliberately empty. Stage B writes findings only with citations — a file path + line, a deno doc surface, a saved fetched artifact, or an external URL. An uncited load-bearing claim is a legitimate PLAN-EVAL failure, and I would rather the evaluator see an honest empty table than a plausible unsourced one.

Next — Stage B (discovery corpus)

Fan-out across the seven evidence inputs: current framework surfaces (packages/fresh, packages/fresh-ui, plugin manifest/contribution axes, generated registries, CLI plugin flows, Aspire/telemetry/MCP/Scalar), the three prior RFCs, the dashboard board, doctrine, and a primary-source market study (Nuxt DevTools, TanStack Devtools, Vite DevTools/inspect, Medusa admin zones, Backstage, Directus/Strapi, Grafana plugin extensions, Aspire Dashboard, Scalar) that keeps developer tooling, production admin consoles, and browser-extension models separated rather than averaged.

rickylabs and others added 6 commits August 11, 2026 11:17
Proves the run is reviewable from GitHub alone: draft PR #1450 carries the
charter read-back, the verified baseline, the routes in force, and the
evaluation posture, so no stage-B evidence rests on chat history.

- phase-registry: stage A done (ccc4c0a, PR #1450, opening comment),
  stage B active
- worklog: PR/label/comment evidence recorded per the commit-trail contract
- context-pack: resumable state advanced to stage B

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves stage-B provenance: seed-run.md's Tier-C hard rule is that an
uncommitted workflow does not run, and a corpus produced by one does not
count as stage-B proof. This commit is that precondition.

The script embeds its inputs as consts rather than relying on `args`
(seed-run landmine), fans 14 agents across repo surfaces, the three prior
RFCs, the dashboard board, doctrine, and a primary-source market study,
and binds every agent to the citation law: an uncited claim is not a
finding, and an inference may not be dressed as an observation.

Agents are read-only on source and on GitHub; `gh` is reads only.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the run's validation plan against repo configuration instead of
assumption — an unexamined gate list is how a run reports false-green
evidence.

deno.json's fmt.include is packages/**/*.ts(x) and plugins/**/*.ts(x)
only, so `deno task fmt:check` never inspects Markdown. The stage-A plan
named a scoped `deno fmt` pass as its format gate; running it manufactured
29 findings no repo gate asks for, and would have rewritten the verbatim
upstream artifacts under research/sources/ — corrupting the evidence the
corpus cites. Those files are designated evidence and are never formatted.

Real gates recorded: docs:links (needs an explicit --root, since its
defaults cover .llm/harness and docs/architecture/doctrine but not a new
RFC dir), docs:accuracy, and the CI quality job gated on needs_docs.

Also verified the OpenHands docs-accuracy workflow dispatches only on
ready_for_review, so a permanently-draft PR satisfies the charter's
no-OpenHands boundary structurally. Deliberately not applying
docs-eval:skip: a label silencing a gate that was never going to fire
would be misleading evidence.

Drift D-4 (significant) and D-5 recorded.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves every downstream design claim can be traced to evidence rather
than to the supervisor's priors.

14 agents returned, 0 errors: 5 repo surfaces (fresh host, fresh-ui
pipeline, plugin contribution axes, CLI plugin flows, observability
boundary), 3 prior RFCs re-baselined against 2256a67 (#890, #1446,
#1390), the dashboard board (#400 + children) and doctrine/live-board,
and 4 primary-source market teardowns (Nuxt/Vite, TanStack/Grafana,
admin consoles, Aspire/Scalar).

6,327 corpus lines plus 78 saved upstream artifacts under
research/sources/ — including Nuxt devtools-kit type definitions and the
full Vite DevTools kit docs — so a market claim is verifiable without
re-fetching the web. Those artifacts are verbatim evidence and are never
reformatted (drift D-4).

Verified the fan-out wrote nothing outside the run dir: the read-only
constraint on packages/, plugins/, docs/, and GitHub held.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…s files

Proves the corpus was read by the supervisor rather than skimmed, and
makes the analysis durable independent of session context.

Five findings reshape the charter's framing:

S-1 There is no plugin->UI channel of any kind at this baseline.
    capabilities.hasRoutes means service endpoints; no registry kind emits
    routes/pages/islands; the real mechanism is three hardcoded Vite
    aliases. The RFC defines the first extension point, it does not
    extend one.

S-2 RFC #890's envelope is merged design text with ZERO implementation --
    32 files, all under .llm/runs/ plus labels.yml; all 24 children and
    the epic still OPEN at status:plan. "Preserve its pattern" therefore
    describes a co-dependency on unbuilt work, not reuse of a shipped
    surface. This is the run's largest plan-defect risk and becomes an
    owner fork.

S-3 #1446 gives DevTools a quotable mandate (P-6) and a decision sentence
    separating production management from developer diagnostics -- which
    answers charter Q4 with authority, and imposes a reciprocal duty not
    to annex Surface-1 territory.

S-4 The RFC home is contested: docs/architecture/rfc/ does not exist on
    main and is claimed by unmerged #1446, while rfcs/ ships today.
    Escalated as an owner fork; this run takes rfc-0002- so the only
    overlap is directory creation.

S-5 DevTools has a ready-made data plane to consume: TelemetryQueryPort,
    22 typed MCP tools with input+output schemas, a pure OpenAPI
    projection entrypoint, and netscript.correlation.id as the journey
    join key -- but MCP is stdio-only, so a browser client cannot reach
    it, and no Aspire/Scalar deep-link helper exists.

Also carried: the arbitrary-write finding in resolveTarget, which is
inert only while the registry is first-party, and the evidence that
#890's transactional replace-set fixes a real shipped defect class rather
than gold-plating.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…ine legs

Proves the supervisor read the board/doctrine corpus, not just the repo
and prior-RFC legs.

Eight further findings, the load-bearing ones:

S-6 Epic #400's ownership thesis exists verbatim and is already
    operationalized into three ENFORCEABLE acceptance lines, including
    'every merged panel must answer why this cannot just deep-link to
    Aspire/Scalar'. The RFC should adopt these as normative criteria
    rather than restate the thesis as prose.

S-7 Board authority is uneven: #685 merged ANALYSIS with committed
    provenance but never advanced past status:research; #780 is an
    unlabelled stale draft with nothing on main; the last owner-ratified
    board event is the 2026-07-06 rescope. A map treating #685 as
    ratified architecture would be wrong.

S-8 THREE competing seams already claim the same contribution axis
    (#427 vs #890's pointer axis vs #734's manifest axis), and two epics
    claim dashboard-zone panels at different milestones (#428-431 vs
    #933/#944).

S-9/S-10 The archetype question has two conflicting in-repo precedents
    and doctrine's own table contradicts the merged design on whether
    fresh is A3 or A4 -- which changes the gate set. And arch:check gates
    only 16 of 36 units, so a new package is UNGATED unless the RFC names
    adding it to deno.json:156 as a slice.

S-12 The RFC-home fork is three-way and PRE-EMPTS scheduled issue #1380.

S-13 createServiceClient cannot send Authorization or x-api-key today, so
    DevTools auth propagation is blocked on RFC-A/#1348 -- and bypassing
    the SDK is the duplication the charter forbids.

Owner-fork list grown from 7 to 13.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
rickylabs added a commit that referenced this pull request Aug 11, 2026
Proves the data-plane and build-mechanics questions were answered from
evidence rather than from the charter's phrasing.

S-14 RFC-A does NOT close the loop DevTools needs. Its chain terminates
     at a statically generated services map plus a caller-supplied
     context; it explicitly rejects a registry, a locator, and any
     ambient client, and contains zero occurrences of 'devtool'. So 'a
     plugin panel obtains a typed client' is unsolved -- and RFC-A's own
     sentence that UI contributions and SDK request contributions are
     separate named extension axes is the licence to define the
     host->panel seam without duplicating #1390. Also recorded: no
     response hook, absolute redaction even in debug mode, HTTP-only, and
     an FCP deadline four days out with implementation gated behind an
     unfiled metadata child.

S-15  does not exist anywhere in the CLI. Charter Q8 is
     therefore not 'how does DevTools fit the dev loop' but 'must
     DevTools invent one' -- a materially larger question.

S-16 Two divergent registry generators write to different paths; the
     walker's AstExtractor is regex, not AST; and walker-emitted
     registries leak on plugin remove. Generated-surface drift detection
     is not currently reliable, which independently confirms #890's
     transactional replace-set is a fix rather than gold-plating.

S-17 Adding a contribution kind today costs six framework file edits, and
     plugin doctor already runs contributed checks under a read-only
     dryRun context -- a real reuse target for the diagnosis taxonomy.

Owner forks now 16.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the data-plane and build-mechanics questions were answered from
evidence rather than from the charter's phrasing.

S-14 RFC-A does NOT close the loop DevTools needs. Its chain terminates
     at a statically generated services map plus a caller-supplied
     context; it explicitly rejects a registry, a locator, and any
     ambient client, and contains zero occurrences of "devtool". So "a
     plugin panel obtains a typed client" is unsolved -- and RFC-A's own
     sentence that UI contributions and SDK request contributions are
     separate named extension axes is the licence to define the
     host->panel seam without duplicating #1390. Also recorded: no
     response hook, absolute redaction even in debug mode, HTTP-only, and
     an FCP deadline four days out with implementation gated behind an
     unfiled metadata child.

S-15 "plugin dev" does not exist anywhere in the CLI. Charter Q8 is
     therefore not "how does DevTools fit the dev loop" but "must
     DevTools invent one" -- a materially larger question.

S-16 Two divergent registry generators write to different paths; the
     walker's AstExtractor is regex, not AST; and walker-emitted
     registries leak on plugin remove. Generated-surface drift detection
     is not currently reliable, which independently confirms #890's
     transactional replace-set is a fix rather than gold-plating.

S-17 Adding a contribution kind today costs six framework file edits, and
     "plugin doctor" already runs contributed checks under a read-only
     dryRun context -- a real reuse target for the diagnosis taxonomy.

Owner forks now 16.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
@rickylabs
rickylabs force-pushed the plan/devtools-contribution branch from cbef6e8 to 2b06418 Compare August 11, 2026 09:40
rickylabs and others added 3 commits August 11, 2026 11:42
…spire/Scalar legs

Proves the market study produced decision-grade evidence rather than a
feature survey.

S-18 The closest analogue deleted its own shell. Nuxt DevTools v4 removed
     the floating panel and became a dock entry inside Vite DevTools;
     vite-plugin-inspect v12 did the same. Nuxt built five bespoke things
     -- shell, RPC namespacing, subprocess/terminal system, editor
     integration, global install -- and deprecated or deleted all five.
     But the thing they consolidated onto is Vite-8-bound and NetScript
     pins Vite 7.2.2, so adopting the kit is not buildable at this
     baseline: imitate the contract shapes, implement natively.

     Three assumptions overturned: devtools are NOT stripped in
     production upstream (build mode is a supported target with client
     auth disabled by construction); iframe does NOT mean sandboxed (Nuxt
     injects live app access into same-origin contributed iframes); and
     transformIndexHtml injection silently no-ops for apps that render
     their own HTML -- which Fresh 2 does.

S-19 The Aspire/Scalar boundary is now a table backed by fetched .razor
     sources. Resource, console-log, structured-log, trace/span and
     metric deep links all exist and are cheap; the log-to-trace
     correlation query is the highest-value jump. Filtered views are NOT
     externally constructible. Aspire has no panel/plugin extension point
     at all -- only resource commands, which vanish when deployed -- and
     it removed its in-dashboard Copilot UI in 13.3, redirecting agents
     to CLI/MCP. That is a direct precedent for DevTools = human UI, MCP
     = agent surface.

     NetScript currently discards nearly all of Scalar's surface, and its
     pinned bundle predates pluginUrls, so "contribute into Scalar" is
     blocked. No deep-link helper exists anywhere in packages/ for either
     upstream -- the hand-off thesis has no implementation seam today,
     which makes a typed deep-link helper an obvious first slice.

Owner forks now 20.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…nsion-model legs

Proves the full corpus was read by the supervisor, and corrects a premise
carried by RFC #890 itself.

S-20 "Inspired by Medusa zones" is factually wrong about Medusa. Medusa's
     zones are a CLOSED core-owned vocabulary that plugins cannot mint;
     validity is an AST check at build time and an invalid zone is
     silently dropped. The plugin-minted namespaced model is STRAPI's.
     Two consequences: in a closed vocabulary name collision is
     impossible by construction, so Q2's collision-policy budget should
     move to ordering -- and NOBODY solved ordering (Grafana concats in
     load order, TanStack identity is positional, Medusa documents none,
     and Medusa deprecated positional .before/.after suffixes). #890's
     deterministic (order, mountId, id) sort is ahead of the market
     rather than derivative.

S-21 The Q4 separation verdict is now evidence-backed: admin consoles pay
     for untrusted third-party code in a long-lived RBAC-governed
     production-data surface, and sandboxing, manifest host ranges,
     per-contribution permissions and runtime module federation are all
     costs of that ONE condition. A dev diagnostics tool satisfies none
     of the antecedents, so the RFC can decline each with a citation
     rather than an assertion. What transfers is cheap: declarative
     target id validated at build time, host-owned typed data flow to the
     contributed component, and a shared component kit. What does NOT
     stretch: no admin console surveyed models a push/stream contract to
     contributed UI -- that is net-new design.

S-22 Two tiny mechanisms are worth near-verbatim adoption: Grafana's
     per-contribution error boundary (loud in dev, null in prod -- which
     TanStack lacks entirely, its most obvious gap) and version-suffixed
     contribution ids, from which Grafana got its whole compatibility
     story. Plus: use TWO independent production-exclusion mechanisms,
     because TanStack explicitly distrusted one signal after hosting
     providers set build command and mode inconsistently.

Owner forks now 24.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the supervisor read the full corpus and converted it into
decisions, not a summary.

research.md now carries 26 cited findings ordered by how much they
constrain the RFC, the final evidence-register status, five
supervisor-delegated resolutions, and the finalized stage-D topic set.

Three carried-in assumptions did not survive the re-baseline and are
recorded rather than quietly corrected: #890's envelope is unbuilt, this
run's own stage-A gate list named a gate that does not exist, and
"inspired by Medusa zones" is wrong about Medusa.

Two charter questions are now ANSWERED by evidence rather than left open
-- Q4 by #1446's decision sentence plus the market separation verdict,
and Q5 by fetched Aspire .razor sources that make the deep-link boundary
a table instead of a thesis.

The eleven provisional stage-D topics collapse to eight: the corpus
closed the boundary topic outright, and the staging question folds into
the information-architecture pack.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
@rickylabs

Copy link
Copy Markdown
Owner Author

[PHASE: RESEARCH] [STAGE: B+C — DISCOVERY CORPUS + SYNTHESIS]

14 discovery agents returned, 0 errors. Full corpus read by the supervisor. Five findings change what this RFC has to be.

Stage B — the corpus

6,327 corpus lines and 78 saved upstream artifacts under research/sources/, so a market claim is verifiable without re-fetching the web (Nuxt's devtools-kit type definitions, the full Vite DevTools kit docs, Grafana's extension registry + error-boundary source, Aspire's actual dashboard .razor pages, Scalar's config/plugin docs). The workflow.js was committed at d5852188b before it ran, per the Tier-C hard rule. Agents wrote nothing outside the run dir — verified, not assumed.

The five that reshaped the run

1. There is no plugin→UI channel at all. Not a weak one — none. capabilities.hasRoutes means service HTTP endpoints (its own doc comment says so); no generated registry kind emits routes, pages, or islands; the actual mechanism by which first-party plugin client code reaches the app today is three hardcoded Vite aliases in the scaffold template. grep for devtools across packages/plugins/docs/site0 matches. This RFC is not extending an extension point. It is defining the first one.

2. RFC #890's envelope is merged design text with zero implementation. 32 files, all under .llm/runs/ plus labels.yml. No packages/, plugins/, or docs/ source line. All 24 children and epic #922 are still OPEN at status:plan on 0.0.9 — not even the disposable Wave-0 proofs have run. The charter's "preserve its versioned envelope/generated registry pattern" reads as reuse of a shipped surface; it is a co-dependency on unbuilt work, and that is now owner fork #1 rather than an assumption buried in the design.

What is reusable is the pattern, and the corpus splits it carefully: ten payload-agnostic mechanisms worth taking versus nine app-family-specific payloads that must not be copied.

3. #1446 answers charter Q4 with authority, so it stops being an open question. P-6 is a verbatim row, and §8.2 is a decision sentence, not a preference: "production operator management and developer diagnostics are two distinct hosts and two distinct contribution surfaces — not one ambiguous 'cockpit.'" Slice A7 deliberately excludes diagnostics so as not to pre-empt DevTools — which imposes the reciprocal duty that DevTools must not annex Surface-1 territory.

4. RFC-A (#1390) does not close the loop DevTools needs. Its chain terminates at a statically generated services map plus a caller-supplied context; it explicitly rejects a runtime registry, a locator, and any ambient client, and contains zero occurrences of "devtool". So "a plugin-contributed panel obtains a typed client" is unsolved. Its own sentence — "UI contributions and SDK request contributions are separate named extension axes, not one universal envelope" — is the licence to define a host→panel seam without duplicating #1390.

5. Charter Q5's ownership thesis is now a table, not a thesis. Fetching Aspire's actual dashboard .razor sources proved deep-linking is real and cheap — /traces/detail/{traceId}?spanId=, /structuredlogs/resource/{n}?traceId=&spanId=&logLevel=, /metrics/resource/{r}/meter/{m}/instrument/{i}. It also proved the limits: filtered views are not externally constructible (?filters= is opaque), Aspire has no panel/plugin extension point at all, and its only additive contribution — resource commands — vanishes when deployed. And there is no deep-link helper anywhere in packages/ today, so the hand-off thesis currently has no implementation seam.

The market study earned its place

The most useful evidence is a competitor's regret. Nuxt DevTools deleted its own shell in v4 — it is now a dock entry inside Vite DevTools, with all five of its bespoke subsystems (shell, RPC namespacing, subprocess/terminal, editor integration, global install) deprecated or removed. vite-plugin-inspect v12 did the same.

But the thing they consolidated onto requires Vite 8, and NetScript pins 7.2.2 — so adopting @vitejs/devtools-kit is not buildable at this baseline. The honest conclusion is imitate the contract shapes, implement natively on Deno/Fresh.

Three widely-held assumptions turned out false, each with a citation:

  • Devtools are not stripped in production upstream. Vite DevTools' build mode is a supported target that ships devtools output into the app build with client auth disabled by construction. NetScript's posture should be stricter than every system surveyed.
  • iframe ≠ sandboxed. Nuxt deliberately injects __NUXT_DEVTOOLS__ into same-origin contributed iframes, handing them live access to the running app.
  • transformIndexHtml injection silently no-ops for apps that render their own HTML — which Fresh 2 does. Mounting must be a NetScript-owned route, not an HTML-transform hook. (Flagged as the single most decision-relevant unknown still to be probed.)

And one correction to a premise carried by #890 itself: "inspired by Medusa zones" is wrong about Medusa. Medusa's zones are a closed, core-owned vocabulary plugins cannot mint, AST-validated at build time. The plugin-minted namespaced model is Strapi's. In a closed vocabulary, name collision is impossible by construction — so the design budget moves from collision to ordering, which no surveyed system solved (Grafana concats in load order; TanStack's identity is positional; Medusa deprecated its positional suffixes). #890's deterministic (order, mountId, id) sort is ahead of the market, not derivative.

What the market says to decline — with citations, not assertions

The admin consoles (Medusa/Directus/Strapi/Backstage) pay for untrusted third-party code in a long-lived, RBAC-governed, production-data surface. Sandboxing, manifest host ranges, per-contribution RBAC and runtime module federation are all costs of that one condition — which a developer diagnostics tool does not satisfy. Backstage's cost is documented: plugin install became a rebuild-and-redeploy event, which is exactly why Red Hat bolted on module federation.

Two mechanisms are worth near-verbatim adoption, and both are tiny: Grafana's per-contribution error boundary (loud in dev, null in prod — TanStack has none anywhere on its mount path, its most obvious gap) and version-suffixed contribution ids, from which Grafana derived its entire compatibility story.

Things that will surprise a reviewer

  • plugin dev does not exist. No watch loop anywhere in the CLI. Q8 is "must DevTools invent one", not "how does it fit".
  • Two divergent registry generators write to different paths; the walker's AstExtractor is regex, not AST; walker-emitted registries leak on plugin remove; and writes are non-transactional with only file existence asserted. This independently confirms RFC: Frontend Contribution Layer — plugins that ship UI (dashboard · auth · ai · deploy) #890's transactional replace-set is a fix, not gold-plating.
  • arch:check gates 16 of 36 live units. A new DevTools package is ungated unless the RFC names adding it to deno.json:156 as a slice — otherwise the gate claim is decorative.
  • createServiceClient cannot send Authorization or x-api-key, so auth propagation is blocked on RFC-A/Epic: Typed SDK client contributions — credentials, transport policy, metadata, and cache-safe extensions #1348.
  • resolveTarget accepts absolute and escaping-relative targets with no containment assertion — inert while first-party, an arbitrary-write primitive the moment a third party contributes. The single most load-bearing security finding.

Stage C output

research.md carries 26 cited findings, the evidence-register status, five supervisor-delegated resolutions, and the finalized topic set. The reasoning trail is 22 numbered syntheses in research/SYNTHESIS-NOTES.md, written incrementally as each leg was read.

Eleven provisional stage-D topics collapse to eight — the corpus closed the boundary topic outright (Q4 and Q5 are now answered constraints, not forks) and folded staging into the IA pack.

Owner forks now number 24, up from the charter's implied set. They are accumulating in SYNTHESIS-NOTES.md and will be numbered and de-duplicated in the stage-H decision brief. None is being silently taken.

Next — Stage D

Eight Fable 5 · medium deep-dive packs, plus the charter-mandated GLM 5.2 · xhigh design pass (transport fact of record: tools + streaming, no reasoning trace — pre-registered as drift D-3 before the pass runs, so no artifact can later cite it as reasoning evidence).

rickylabs and others added 2 commits August 11, 2026 11:50
…inding

Proves the topic set was derived from the corpus rather than carried from
the bootstrap guess.

Eleven provisional topics collapse to eight. T4-boundaries closed
outright: charter Q4 is answered by #1446's decision sentence and Q5 by
the fetched Aspire .razor deep-link evidence, so both become constraints
carried into T1/T8 rather than open topics. T11 folds into T8, and Q11 is
the supervisor's stage-E integration output, not a delegated topic. The
superseded set is kept inline for provenance instead of deleted.

D2 lane corrected from major_ui_ux_design to
major_ui_ux_adversarial_review: lane-policy binds the first when GLM
LEADS the design and the second as the minimum when another lane leads,
and here the Opus supervisor plus the Fable packs lead. Consequence
recorded -- the pass is sequenced after the stage-E draft, because an
adversarial design review needs a design to review, and running it now
would produce generic advice while misrepresenting the lane.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…at baseline

Proves a stage-D agent finding by supervisor verification rather than
relaying it, and escalates a defect that belongs to another epic's plan.

RFC #890 contract C8 asserts older CLIs ignore an unknown manifest
pointer block, so adding one is safely additive. But
PluginInstallerManifestSchema ends in .strict()
(packages/plugin/src/protocol/manifest.ts:282) with
schemaVersion: z.literal(1) at :271, so zod HARD-REJECTS any unknown
top-level key: an older CLI fails manifest parsing outright and takes the
plugin down rather than degrading. The stage-B corpus had independently
recorded the same property from the other direction (r3 F5), which is
what made the agent's claim worth checking rather than dismissing.

Significant, and not scoped to this run -- epic #922 slice #929 plans to
implement exactly that pointer axis on the false assumption.

Action is split: this RFC requires an explicit schema-evolution
precondition slice before any manifest-visible pointer lands, and the
finding is escalated to the owner as a cross-RFC issue. This run does not
edit another epic's board; recording and escalating is the whole
permitted action before ratification.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
rickylabs and others added 9 commits August 11, 2026 13:45
…s pure UI/UX

Proves stage D2 is now two complementary passes rather than one
overloaded reviewer.

The owner refined the D-15 override: architecture and contracts go to
Qwen 3.8 Max at max, and the pure UI/UX review goes to Kimi K3 at high --
which is what lane-policy already envisages, since adversarial_design_eval
is defined as COMPLEMENTING the design lane rather than replacing it.
Each prompt tells its reviewer to stay in its lane and skip the other's
findings, so the passes do not duplicate.

Recorded honestly rather than glossed: Kimi is the vision-capable lane,
but this run is planning-only and there are NO screenshots, mockups, or
rendered artifacts, because nothing is implemented. Kimi reviews the
information architecture as text and its vision capability is unused. Its
prompt says so explicitly so that no downstream artifact can imply a
visual review took place -- and if the IA is ever prototyped, a follow-up
Kimi pass with images would be materially different evidence.

Both passes run on separate fresh read-only worktrees, both are
findings-only with no edit rights, and both are advisory. The Codex
PLAN-EVAL remains the sole verdict of record.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the identity file stays accurate after a lane change rather than
carrying superseded text -- the staleness class PLAN-EVAL already failed
this run for once.

Three corrections. The OpenRouter prohibition still described the GLM
pass as required and Kimi as merely conditional; it now names the actual
active lanes, Qwen for architecture and Kimi for UI/UX, and states that
no stage-D2 reviewer is ever the formal evaluator -- a PASS-shaped
statement from a design lane carries no gate authority.

The stage-F rationale listed GLM 5.2 as one of the authoring lanes Sonnet
had to be distinct from. GLM NEVER RAN, so it authored nothing; the note
now says so rather than implying a pass happened.

The review chain is corrected to Opus -> Fable -> Sonnet -> Codex Sol,
with Qwen and Kimi named as advisory passes that authored nothing.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves an evidence-loss mistake by the supervisor rather than hiding it
behind a clean re-run.

Both stage-D2 launches were piped through tail -40. Kimi K3 completed and
returned 1 critical, 5 major and 4 minor findings, but only the last 40
lines survived: the captured file starts mid-finding and five of the ten
findings are gone. No OpenCode session store exists to recover from.

This is evidence loss on the exact deliverable the owner declined to
waive, and the cause is mine -- tail was habit from reading noisy
launcher output, which is the wrong tool the moment the command's stdout
IS the artifact. The stage-B corpus escaped this only because those
agents wrote their own files.

Both passes are re-run with full redirection. The truncated tail is KEPT
as kimi-findings-PARTIAL-tail.md rather than deleted, because it is
evidence that the first run happened and what it concluded; deleting it
would tidy away the mistake.

Rule recorded: when a lane's stdout is the artifact, redirect to a file
and never pipe through head or tail.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the drift log states what happened rather than what was intended.
D-17's action line said 'both passes re-run with output redirected'; at
the time of writing only Kimi had been. Qwen was still on its original
truncating invocation and had not returned, and killing a long reasoning
pass to fix the capture would have cost more than letting it finish.

Corrected to a staged action with each lane's real status, and pointing
at the receipt as the tracker rather than asserting a state here. A drift
entry that overstates its own remedy is the same defect it was written to
record.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…feed

Proves the owner-mandated design pass changed the design rather than
decorating it. Kimi K3 returned 1 critical, 5 major, 5 minor; every
finding is dispositioned and every anchor was verified in source before
being accepted.

CRITICAL, fixed. Home could not distinguish "nothing is broken" from
"DevTools is blind" -- a ranked problem feed rendering empty had two
meanings and no way to tell them apart. For a tool whose whole thesis is
diagnostics, that fails silently at the most important moment. Section
11.3.1 now specifies a closed FeedSource set, a per-source status, and
the rule that all-clear is reachable ONLY when every source reported;
otherwise the feed renders partial and names the gap. not-configured
stays distinct from unreachable so a missing automation plugin does not
cry wolf and a real outage is not hidden.

MAJOR, fixed. DevToolsUiNode tables were string-only, so the canonical
devtools table -- id, status badge, trace link -- was inexpressible while
the RFC claimed most panels are key/value plus table plus list. Cells are
now nodes. And there was no code element at all, which made AC-2's
required CLI-equivalent line unsatisfiable by the RFC's own vocabulary.
Both were cases of the document contradicting its own stated goals.

MINOR, fixed. Section 5's route sketch promised a traces/ surface that
section 11.1 explicitly killed.

Seven findings are ACCEPTED-DEFERRED into one state-and-DX amendment
pass, with the reason recorded: they share a single root -- two panel
state vocabularies and no worked data-access example -- and patching them
separately would create a third vocabulary, which is exactly the defect
PLAN-EVAL cycle 2 caught with identity and ordering.

Gates: docs:links PASS, docs:accuracy PASS.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the gate was cleared by owner authority rather than by an
evaluator verdict, and corrects a premise in the grant rather than
accepting it silently.

plan-gate.md allows a Plan-Gate to clear on PASS or an owner waiver in
writing; this is the waiver.

The premise correction matters. The owner wrote 'if both eval passed
separately', but neither stage-D2 pass returned a PASS and neither was an
evaluation -- Kimi returned 1 critical, 5 major, 5 minor and Qwen
returned 1 critical, 5 major, 5 minor, both advisory by construction
because their prompts forbade emitting a verdict. So the waiver is read
as 'apply the amendments and do not open a third Codex cycle', NOT as
'the design passes found nothing'. Letting the looser reading stand would
put a false clean bill of health in the record.

Scope recorded explicitly: the waiver covers the eval cycle and the
Plan-Gate, not board filing, and not owner forks F-1 and F-3, which are
architecture decisions rather than eval verdicts and remain unratified.

The record will never imply Codex returned PASS -- it returned FAIL_PLAN
twice, and the owner has cleared the gate over its owner-gated remainder.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
… antecedent

Proves the second stage-D2 lane changed the design too, and that its
critical finding was verified rather than deferred.

CRITICAL, fixed. Section 9's decline rationale rested on the claim that
contributions are workspace packages the developer already runs. That is
FALSE by this RFC's own pipeline: section 10 pins installs to
source={kind:'jsr'} and emits import('jsr:@acme/plugin-trace@1.4.2/...'),
section 6's worked example is @acme/plugin-crons, and section 6 states
the generator imports the pointed-to export IN-PROCESS. Third-party code
both exists and executes.

The antecedent now splits. The decline survives for panel rendering,
where a contribution is a UiNode data tree and no contributed code
reaches the browser in v1. It does NOT survive for generate-time import,
which is arbitrary third-party code running in the generator's own
process with no subprocess boundary at all -- weaker than the T-2 path
INV-2 scopes. Added T-10, INV-9 (read the envelope without executing
contributor code in-process) and gate G-10. The restated justification is
narrower and true: installing a plugin already grants server code and a
whole-filesystem scaffolder before DevTools exists.

MAJOR, fixed. Anchors were keyed '<pluginKind>/<contributionId>' while
identity produces '<mountId>/<id>/v<apiMajor>', so no anchor could ever
match and the entire anchor tier of my ordering rule was silently dead.
An unmatched anchor is now a generate-time warning.

MINOR, fixed. DevToolsPanelId was referenced but never defined -- more
residue from my own identity fix. And "8 trigger kinds" was simply wrong:
verified at plugin-triggers-core constants, the canonical set is six.

Three findings arrived independently from BOTH lanes -- the string-only
table, the traces/ contradiction, and the under-specified feed. That
convergence is the strongest evidence either pass produced.

Gates: docs:links PASS, docs:accuracy PASS, lock clean.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…filing

F-1 ratified: self-contained DevTools family and spine built first in
packages/devtools-core, not serialized behind #890's 24 unimplemented
children. Closes the run's highest-risk fork and unblocks W1-a.

F-3 ratified: manifest schema-evolution precondition via .passthrough()
before any manifest-visible pointer, with explicit old/new CLI behavior
and tests. Closes the D-6 defect where #890's additive-manifest claim was
false against a .strict() schema.

Board filing authorized once from the committed manifest, preserving the
2026-07-19 milestone train and not duplicating existing issues. This is
the stage-H ratification the seed-run profile gates on; the mutation
boundary opens for the first time in this run.

Standing instruction recorded: no re-asking about F-1/F-3 or accepted
findings; stop only for a genuinely new architecture fork or an
authorization boundary.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the owner-approved substitute route ran as authorized, with
identity confirmed from the transcripts rather than assumed.

Observed matches requested on both lanes: qwen/qwen3.8-max and
moonshotai/kimi-k3. Recorded honestly that OpenCode's header reports the
bare vendor/model while the request carries the openrouter/ transport
prefix -- a transport-prefix difference, not a model difference -- and
that variant is not echoed in the header, so it is marked requested-only
rather than claimed as observed.

The receipt now also records the outcome plainly: neither pass returned a
PASS and neither was asked to, each found a critical that changed the
RFC, and three findings arrived independently from both lanes. The
truncated first captures are listed alongside the full ones as preserved
evidence of D-17.

Amendment C also landed: #412 moves AMEND to SUPERSEDE now that contracts
live in packages/devtools-core, every #890-dependency hedge is rewritten
under ratified F-1, #922's children stay untouched, and a live-board
re-read corrected the child count from 28 to 29.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
rickylabs added a commit that referenced this pull request Aug 11, 2026
…e union

Proves the state-and-DX amendment is a single coherent pass rather than
piecemeal patches. Sections 6 and 7 only; 213 insertions, 95 deletions.

F-3 RATIFIED and specified normatively. The O-5 fork is replaced by the
.passthrough() precondition: the exact change at manifest.ts:283,
schemaVersion stays z.literal(1), all four CLI-versus-manifest
compatibility cases written out, and three named slice-acceptance tests
including a regression guard proving the same fixture fails under
.strict(). The typo-detection trade is stated honestly with its
compensating control -- the pointer block's own schema stays strict, so
typos inside it are still caught.

ONE ZONE RULE. Zone ids are version-suffixed in the vocabulary itself,
contexts are typed per zone via DevToolsZoneContextMap (the comment-only
 is gone), an exact-match rule is specified, and a sixth
quarantine state -- zone-contract-mismatch -- closes the gap Qwen found.

ONE PANEL STATE. PanelAvailability is deleted in favour of a canonical
DevToolsPanelState<T> covering loading, ready, empty, partial, stale,
not-running, incompatible, unauthorized and failure. Defining it once, in
the contracts section, is the fix for the two-vocabulary defect -- not a
third vocabulary.

Third-party mounting resolved: plugin.detail/v1 is the universal mount,
and new zones are added only by host release against a filed issue --
plugins petition, the host adds.

F-1 propagated: the spine is the ratified self-contained devtools-core,
and convergence later is a re-export change rather than a dependency.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…e union

Proves the state-and-DX amendment is a single coherent pass rather than
piecemeal patches. Sections 6 and 7 only; 213 insertions, 95 deletions.

F-3 RATIFIED and specified normatively. The O-5 fork is replaced by the
.passthrough() precondition: the exact change at manifest.ts:283,
schemaVersion stays z.literal(1), all four CLI-versus-manifest
compatibility cases written out, and three named slice-acceptance tests
including a regression guard proving the same fixture fails under
.strict(). The typo-detection trade is stated honestly with its
compensating control -- the pointer block's own schema stays strict, so
typos inside it are still caught.

ONE ZONE RULE. Zone ids are version-suffixed in the vocabulary itself,
contexts are typed per zone via DevToolsZoneContextMap (the comment-only
optional-unknown data field is gone), an exact-match rule is specified,
and a sixth quarantine state -- zone-contract-mismatch -- closes the gap
Qwen found.

ONE PANEL STATE. PanelAvailability is deleted in favour of a canonical
DevToolsPanelState covering loading, ready, empty, partial, stale,
not-running, incompatible, unauthorized and failure. Defining it once, in
the contracts section, is the fix for the two-vocabulary defect -- not a
third vocabulary.

Third-party mounting resolved: plugin.detail/v1 is the universal mount,
and new zones are added only by host release against a filed issue --
plugins petition, the host adds.

F-1 propagated: the spine is the ratified self-contained devtools-core,
and convergence later is a re-export change rather than a dependency.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
@rickylabs
rickylabs force-pushed the plan/devtools-contribution branch from 496876d to 7a62d2e Compare August 11, 2026 15:48
rickylabs and others added 6 commits August 11, 2026 18:01
…, flows

Completes the single coherent state-and-DX amendment. Sections 8 and 11,
plus three stale zone references A flagged as out of its scope. +601/-50.

THE ADOPTION BLOCKER IS CLOSED. Section 8 now carries a worked
end-to-end contributor data path in seven code steps and a nine-row
pass-through table, following one plugin from manifest pointer through
envelope, procedure reference, generated registry, host invocation, typed
data, panel state and rendered node. Writing it surfaced that section 6's
requires field had been dangling -- it is now defined. A reader could
previously reach the end of the kinds section and still not know how a
panel gets its own data.

ONE MATRIX, ONE UNION. Section 11.7's local PanelState and DeepLink are
deleted; the matrix consumes section 7's canonical union as a rendering
checklist across ten surfaces by nine arms, with every cell specified and
no blanks. Both modelled degradations survive under partial.

RANKING IS NOW DEFINED. A row schema, a four-level severity vocabulary
with a fixed per-source assignment, and a deterministic total order --
severity, then observedAt descending, then source declaration order, then
an entity code-unit tiebreak -- gated by a shuffle test. Two of six
sources are honestly marked staged and render not-configured rather than
vanishing.

Also: a bounded flows index so the journey view is reachable without
already holding a correlation id; a density contract with mandatory
truncation disclosure, cited against Aspire's own 10k cap; and
/automation given concrete staged behaviour naming its blocking
dependency, which turns a blank nav seat into a placeholder that explains
itself.

Consistency sweep after both amendments: no retired vocabulary outside
self-documenting correction notes, zone ids uniformly suffixed, zero
compound-id or flat-sort residue.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…, 0 deferred

Proves the owner directive that no accepted finding remains merely
deferred. FINDINGS-SWEEP.md is the closure ledger: every one of the 11
Qwen and 11 Kimi findings carries a status and a normative anchor into
the amended RFC.

The single decline is Kimi's own lane note rather than a design finding:
it is the vision-capable lane but nothing is implemented, so there are no
screenshots to evaluate and it reviewed the IA as text. A vision pass
needs a prototype, and building one is implementation -- outside a
planning-only run's boundary. Recorded as a named follow-up with a
re-entry condition at roadmap wave W4/W6, not as a gap.

Three defects were reported independently by BOTH lanes, which could not
see each other's output: string-only table cells, the section 5 traces/
contradiction, and the under-specified ranked feed. Convergence between
an architecture reviewer and a UX reviewer is the strongest evidence
either pass produced, and the clearest justification for the owner's lane
split over one merged reviewer.

Gates after both amendments: docs:links PASS with zero broken links or
anchors, docs:accuracy PASS, 90 balanced code fences, lock clean.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
… cross-RFC conflicts

Proves the amended RFC is coherent with the three prior contribution RFCs
and with itself.

THE HEADLINE IS A NEGATIVE RESULT AND IT IS THE IMPORTANT ONE: zero
contradictions and zero duplications against #890, RFC-0001/#1446 and
RFC-A/#1390, verified at source -- the two-hosts decision sentence, the
five-surface taxonomy, P-6's A2b/A3b/A2d entry criteria, the four
consumed contracts, and RFC-A's separate-named-axes licensing sentence
all check out. The host-to-panel seam mirrors rather than reuses
SdkClientContribution, so no second SDK mechanism was invented.

All 21 defects were INTERNAL, introduced by three agents amending
different sections -- the predictable cost of parallel authoring, caught
because the sweep was run against the amended text rather than assumed
from the earlier one.

The sharpest was a real conflict of substance: sections 6 and 8 still
said the generator imports contributor code in-process while the new
INV-9 forbids exactly that. Had it shipped, the RFC would have mandated
a security invariant its own pipeline description violated. Both now
specify static parse or a permission-scoped subprocess.

Also fixed: a self-contradictory sentence in 13.1 left by my own global
rename ("is packages/devtools-core... deliberately not named
devtools-core"); read-only v1 versus confirm-gated migrate/seed dialogs,
now staged with v1 rendering the CLI line rather than executing it; a
duplicate T-10 id; five-state versus six-state residue, keeping the one
reference that correctly describes #890's five; the ratified F-1/F-3 rows
still listed as open forks; and a quoted source that had been silently
rewritten -- restored with the divergence stated.

Gates: docs:links 0 broken, docs:accuracy PASS, 90 balanced fences, lock
clean.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Proves the run's own record is current before it mutates the board --
the staleness class PLAN-EVAL failed this run for once already.

decision-brief.md changes character: it was a request for decisions and
is now the record of decisions taken, with every fork closed and each
pointing at where it is recorded. It keeps one thing deliberately loud --
the Codex evaluator NEVER returned PASS; it returned FAIL_PLAN twice and
the gate was cleared by owner waiver. Nobody reading this later should
mistake one for the other.

context-pack.md moves to phase ratify-to-file, records that filing is the
only remaining step, and carries the two process lessons a resumed run
needs: bound the evaluator's reading, and use a new worktree per cycle.

worklog.md gains the post-ratification execution table and the final gate
results, and its drift table grows from fourteen entries to nineteen with
the seven self-corrections marked as such.

Open items are listed rather than buried: three labels that do not exist,
the labels.yml parity drift, the unlaunchable design lane still worth its
own issue, the vision pass that needs a prototype, and five mitigations
that remain UNPROVEN named gates.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
Slice 5 still read "FAILED -- lane unlaunchable ... no findings file
exists", which was true before the owner override and false after it: the
pass ran on the substitute route, both lanes returned findings, and two
criticals were fixed. And Deferred Scope still listed board filing as
blocked on a PLAN-EVAL PASS that will never come.

Both corrected, with the gate's real provenance kept visible -- cleared
by owner waiver, not by an evaluator verdict.

This is the third staleness catch in this run's own artifacts. The
pattern is consistent: a status line written while something was true
survives the event that falsified it. Refreshing the artifacts is not
bookkeeping here; it is the difference between a record and a story.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
…f-assigned number

Owner decision: the RFC moves to rfcs/0000-devtools-contribution.md, the
repository's only documented RFC process, so acceptance numbering and
merge order can be coordinated against immutable main.

This closes fork F-2, the three-way RFC-home contest. The run had been
living at docs/architecture/rfc/, the convention introduced by UNMERGED
PR #1446 -- and had self-assigned rfc-0002. Both are corrected. rfcs/
README is explicit that numbers are assigned by a maintainer at
acceptance and that 0000 is kept while drafting precisely to avoid number
races between concurrent drafts, so taking a number was the run
overstepping. The filename and frontmatter now both read 0000, and the
template's frontmatter contract is honoured exactly.

One divergence is deliberate and disclosed in a process note at the top
of the RFC: the body keeps its own numbered sections rather than the
template's ten headings. The template suits a few hundred lines; this is
~4,500 lines whose sections are cross-referenced by anchor throughout the
evidence corpus, the findings sweep and both design triages.
Restructuring would break every citation for no reviewability gain.

Reference updates follow the authority rule this run has used
throughout: live artifacts and every filing draft were updated; immutable
evidence was NOT rewritten -- the stage-B corpus, design packs, both
design-pass prompts and outputs, the evaluator record, the committed
workflows and the owner's charter keep their original text.

Side effect worth noting: docs:links --root rfcs now reports docs=3, so
the gate covers the README and template as well as the RFC, and all three
are clean.

Gates: docs:links 0 broken across 3 docs, docs:accuracy PASS, 90 balanced
fences, nothing touched outside rfcs/ and the run dir.

Refs #1450

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DChBXWYP9LStvjQztUJV5b
@rickylabs

Copy link
Copy Markdown
Owner Author

[PHASE: PLAN] [PATH NORMALIZED — canonical rfcs/ process]

The RFC now lives at rfcs/0000-devtools-contribution.md. Branch is clean and pushed at 9f817f720. No board mutation has occurred — filing remains paused while root coordinates acceptance numbering and merge order.

What moved, and what that corrected

git recorded it as a rename at 99% similarity (R099) — content is unchanged by the move.

Two things this corrected beyond relocation:

  1. The self-assigned number is withdrawn. This run had taken rfc-0002-. rfcs/README.md § Numbering is explicit that numbers are assigned by a maintainer at acceptance, and that 0000 is kept while drafting "to avoid number races between concurrent drafts". Taking a number was the run overstepping. Filename and frontmatter both read 0000.
  2. The frontmatter contract is now honoured exactlyrfc, title, status, authors, created, tracking-issue, target-milestone, per rfcs/0000-template.md.

This also closes owner fork F-2, the three-way RFC-home contest:

Convention Outcome
rfcs/ — the documented process, zero numbered RFCs ever merged CHOSEN
.llm/runs/*/design/canonical/ — what merged PRs #890/#891/#1123 actually did not chosen
docs/architecture/rfc/ — introduced by unmerged PR #1446 abandoned (this run's earlier home)

And it resolves a concern this run had recorded against itself: the earlier location risked pre-empting issue #1380, which schedules an RFC-location decision on 0.0.6. Conforming to the only documented convention removes that.

One deliberate divergence, disclosed

The RFC keeps its own numbered sections rather than the template's ten headings. The template suits a proposal of a few hundred lines; this is ~4,500 lines whose sections are cross-referenced by anchor throughout the evidence corpus, the findings sweep, and both design-pass triages. Restructuring would break every citation for no gain in reviewability. Stated in a process note at the top of the RFC so it reads as a decision, not an oversight.

Reference updates — and what was deliberately left alone

Applying the same authority rule this run has used throughout (RFC-AUTHORITY.md):

  • Updated — the RFC's self-references, plan.md, worklog.md, context-pack.md, decision-brief.md, supervisor.md, research.md, RFC-AUTHORITY.md, FINDINGS-SWEEP.md, the supersession map, every filing/ draft, and this PR body.
  • Not rewritten — the stage-B corpus, the eight design packs, both design-pass prompts and outputs, the evaluator record, the committed workflows, and the owner's charter. Those are immutable evidence; rewriting them would falsify what was true when they were written.

Gates after the move

Gate Result
deno task docs:links --root rfcs --pretty PASSdocs=3, broken-links=0, broken-anchors=0, orphans=0
deno task docs:accuracy PASS
Code-fence balance PASS — 90, balanced
Working tree clean; nothing modified outside rfcs/ and the run dir
Remote git ls-remote == local HEAD 9f817f720

Note docs=3: scoping the link gate at rfcs/ now covers the README and template as well as the RFC, and all three are clean — slightly broader coverage than the previous root gave.

State

  • RFC: rfcs/0000-devtools-contribution.md, 15 sections, status: Draft, rfc: 0000.
  • Findings: 22/22 closed — 21 fixed, 1 declined with a re-entry condition, 0 deferred.
  • Cross-reference: 0 contradictions, 0 duplications against RFC: Frontend Contribution Layer — plugins that ship UI (dashboard · auth · ai · deploy) #890 / RFC-0001 / RFC-A.
  • Drift: D-1 … D-20; eight entries correct this run's own earlier claims.
  • Board: untouched. Dashboard children still on the 2026-07-19 owner-ratified train.
  • PR: still draft.

Filing stays paused pending root's acceptance numbering and merge order. filing/filing-manifest.md is ready to execute as a single ordered pass — 14 new issues, 1 not filed, 6 existing amended via 4 rows, #922's children untouched, no labels created.

@rickylabs

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=400

use harness

SKILL

  • netscript-harness — preserve evaluator separation and record a concise, evidence-based verdict.
  • netscript-tools — use repository-native commands and keep lock/worktree state clean.
  • openhands-handoff — honor the pr-comment output contract and write the required summary.
  • netscript-cli — hand-test scaffold and documented CLI commands when executable claims exist.
  • rtk — keep changed-file and command inspection compact.

Act as a cheap-and-quick documentation accuracy evaluator. Do not edit source, documentation,
deno.lock, or run artifacts, and do not create commits. Write the complete result to
OPENHANDS_SUMMARY_PATH; output mode pr-comment will publish that result as the single PR
comment.

  1. Identify every changed documentation file in this PR and read each one fully. Include Markdown,
    MDX, READMEs, RFCs, harness/process docs, and skill instructions; do not review only the diff
    hunks.
  2. For every executable claim, QUICKLY hand-test the exact documented command or snippet from the
    repository root. Use the smallest representative scaffold/workspace needed, compare real output
    and generated paths with the document's claim, and stop after the highest-value small checks. Do
    not substitute a full expensive E2E suite for a focused manual check.
  3. If the changed set contains no executable command, snippet, flag, verb, or path claim, say
    exactly:
    No executable documentation claims in this changed set; manual command testing was not applicable.
    Full accuracy and hallucination review is still mandatory.
  4. Check every named command verb, flag, file path, package, API, and expected output against the
    repository. Any hallucinated or nonexistent verb, flag, or path is a BLOCKING finding.
  5. Return a compact per-file table with accurate, inaccurate, or unverifiable, the commands or
    snippets tested, observed-versus-claimed output, and blocking findings. End with one overall
    PASS or FAIL_FIX verdict and a short list of exact fixes.

Keep the iteration budget small. Prefer one to three decisive manual checks over broad exploration,
but never omit a changed documentation file from the per-file verdict table.

@rickylabs

rickylabs commented Aug 11, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

OPENHANDS_VERDICT: FAIL_FIX

Model: openrouter/minimax/minimax-m3
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=success replies=skipped draft-pr=skipped
Trace: .llm/tmp/run/openhands/pr-1450/run-31532951292-1 (in the Actions artifact)
Artifact: openhands-agent-31532951292-1

OPENHANDS_VERDICT: FAIL_FIX

PR 1450 — RFC 0005 DevTools contribution architecture — documentation accuracy evaluation

Mode: pr-comment (single PR comment, no source/doc edits, no commits, no deno.lock
changes). Branch under review: plan/devtools-contribution @ 6deacc52b against base
2256a67bf. PR diff spans 161 files / 43 269 lines; the changed documentation surface
covers the new RFC and its full research/harness trail
(.llm/devtools-rfc-orchestrator-brief.md + the
.llm/runs/plan-devtools-contribution--seed/** corpus). Read of all changed
documentation files; only the highest-value manual checks were run (5 commands; ~12
seconds wall time).

1. Scope and method

  • Read every changed documentation file (RFC, all research briefs, all
    filing/design artifacts, decision-brief, plan-eval, worklog, RFC-AUTHORITY,
    supervisor drift, harness/seed manifests) in full.
  • Executed the five highest-leverage documented commands:
    1. deno task docs:links --root rfcs --pretty
    2. deno task docs:links --root docs/architecture/rfc --pretty
    3. deno task docs:accuracy (under a non-LD_LIBRARY_PATH-shadowed env so the
      --allow-run=deno child can exec; see Finding F-N2)
    4. grep -RIn --include='*.ts' --include='*.tsx' --include='*.json' --include='*.template' -iE 'devtools|_devtools' packages plugins docs/site
      → exit 1, no matches (RFC Abstract, §3, §6, §7, §8 capability matrix)
    5. grep -RIn --exclude='*.lock' --exclude-dir=.llm 'transformIndexHtml' . → only
      matches in the RFC text itself; the underlying host code is clean (RFC §6
      claim, repeated in §3, §6, §7, §10).
  • Re-ran the two absence claims in BRE (the older, broken form) to confirm the
    previous adversarial reviewer's diagnosis that \| is literal there — exit 1
    with no matches, but the same 0-match answer that the corrected -iE form
    gives, so the underlying factual claim survives and the RFC's current text
    uses the -iE form (lines 163, 410, 474, 893, 1472).
  • Cross-checked every packages/…:L-L and plugins/…:L-L path citation in
    rfcs/0005-devtools-contribution.md against the worktree (39 unique
    packages/, plugins/, docs/, .github/ paths). 38 resolve; one is a
    placeholder that the RFC itself is implicitly disclaiming.
  • Read the unique‑path "Six‑file edit set" claim and each cited line range.

2. Per‑file verdict table (every changed documentation file)

File (relative to repo root) Verdict Representative manual check Observed vs claimed
rfcs/0005-devtools-contribution.md (4 588 lines) inaccurate grep -rniE 'devtools|_devtools' --include=*.ts --include=*.tsx --include=*.json --include=*.template packages plugins docs/site (lines 163, 893, 1472) command is correctly written; 0 matches observed. All 40 path citations resolve except packages/cli/.../generate-app-deno-json.ts:112-119 (line 700), which is a placeholder ellipsis. Capability matrix at line 410 is cut by a stray | from the table cell – the grep token begins on line 410 and the line‑wrap stops mid‑pattern, so the rendered Markdown is not a runnable command and the cell as displayed breaks the example it gives. Negative transformIndexHtml claim: confirmed clean.
rfcs/0000-devtools-contribution.md (renamed predecessor; cited by name in RFC-AUTHORITY, decision-brief, filing-manifest, worklog, epic, plan-eval, plan-eval-brief, T9-supersession/supersession-map.md, b2-doctrine-and-live-board.md) inaccurate ls rfcs/0000-devtools-contribution.md file does not exist at HEAD 6deacc52b; only rfcs/0005-devtools-contribution.md does. Every doc that still references 0000 is a stale path (see F‑F1).
docs/architecture/rfc/rfc-0001-runtime-versioned-automation.md (cited as existing) unverifiable deno task docs:links --root docs/architecture/rfc --pretty → OK; the file is not present on this branch (docs/architecture/rfc is the validation root but no 0001-… file exists in the diff) citation refers to a doc that the plan claims exists, but it is not in the changed set.
docs/architecture/rfc/rfc-0002-devtools-contribution.md (cited 8× across research and design briefs) inaccurate deno task docs:links --root docs/architecture/rfc --pretty → OK; ls docs/architecture/rfc/rfc-0002-devtools-contribution.md file does not exist anywhere on the worktree. Citations must point to rfcs/0005-… (see F‑F1).
rfcs/RFC-A-sdk-client-composition.md and rfcs/RFC-B-command-composition-kit.md (cited in b2-doctrine-and-live-board.md) inaccurate ls rfcs/RFC-*.md neither file exists; only 0005-devtools-contribution.md is in rfcs/.
docs/migrations/from-v1#meta-replaced-by-meta-plugins (sources/rfc-a-…md line citing from RFC‑A snapshot) unverifiable doc itself is a vendored upstream snapshot under .llm/runs/plan-devtools-contribution--seed/research/sources/; not authoritative for NetScript acceptable inside a sources/ block, but the path is asserted as a NetScript path inside the prose. No breaking effect (sources files are explicitly external copies).
docs/architecture/doctrine/05-folder-structure.md, 06-archetypes.md, 07-composition-and-extension.md, 09-anti-patterns-and-fitness-functions.md (cited) accurate deno task docs:links --root rfcs --pretty all four resolve; docs:links is clean (0 broken, 0 orphan).
packages/plugin/src/application/plugin-registry.ts (cited lines 9–14) accurate sed -n '1,20p' … line 9-14 confirms DuplicatePluginError definition.
packages/cli/src/kernel/adapters/plugin/kinds/<kind>.kind.ts and plugin-kind-providers.ts (six‑file edit set table, lines 254–265) accurate ls packages/cli/src/kernel/adapters/plugin/kinds/ kinds dir exists; barrel plugin-kind-providers.ts not in the diff (cites fact, not diff) – this is the "minimum edit set" description, not a claim that the file ships.
packages/cli/src/kernel/application/registries/plugin-kind-registry.ts line 12‑17 ("today literally ['api', apiKindProvider]") accurate sed -n '10,18p' … text matches.
packages/cli/src/public/features/plugins/install/plugin-package-resolver.ts (six‑file list) accurate file exists; cited as part of the minimum‑edit set resolves.
packages/plugin/src/sdk/discovery/ast-extractor.ts (six‑file list, line 4‑8, 36‑62) accurate file exists; cited as the regex "AST extractor" resolves; not a hallucination.
packages/cli/src/public/features/plugins/list/list-plugins-command.ts (six‑file list, line 24‑28) accurate file exists; cited as the second duplicate map resolves.
packages/cli/src/kernel/assets/app/vite.config.ts.template lines 5, 20‑32, 41‑56 accurate sed -n '1,60p' … all three line ranges resolve; createNetScriptVitePlugin import is present.
packages/cli/src/kernel/templates/app/generators-config_test.ts lines 247, 284 accurate sed -n '245,250p' … both line ranges resolve and reference the same createNetScriptVitePlugin symbol.
packages/cli/src/kernel/application/ui/registry.ts lines 67‑73, 203‑214, 283 (capability matrix) accurate sed -n '65,75p' …; sed -n '200,215p' …; sed -n '280,290p' … all three line ranges resolve.
packages/plugin/src/protocol/manifest.ts lines 17‑140, 271, 283 accurate sed -n '17,30p' …; sed -n '265,290p' … ranges resolve; the cited .strict() call (lines 271, 283) is present.
packages/cli/src/kernel/adapters/config/plugin-registry.ts lines 150‑159 accurate sed -n '148,162p' … range resolves.
plugins/workers/src/cli/runtime-registry-generator.ts and packages/cli/src/public/features/generate/plugins/installed-runtime-registry-generator.ts (lines 88‑95, 100‑114, 416‑417) accurate sed -n '85,120p' …; sed -n '410,420p' … ranges resolve; the --allow-read --allow-write claim is real.
packages/cli/src/kernel/assets/manifest.ts (lines 4‑35), packages/fresh/src/application/route/manifest.ts (lines 4‑35), packages/fresh/deno.json (port 5199 default) accurate sed -n '1,40p' …; jq .dev_server.port packages/fresh/deno.json all three resolve; the 5199 default is real in packages/fresh/deno.json.
packages/telemetry/src/ports/telemetry-query-port.ts (lines 15‑79), packages/mcp/src/domain/tool-types.ts (3‑26, 32), packages/mcp/openapi-projection.ts (1‑38), packages/mcp/src/infrastructure/stdio-transport.ts (line 4), packages/mcp/src/domain/tool-contracts.ts (353, 361) accurate spot-checked each range all resolve; the "22 MCP tools / runNewlineStdio only" claims are real.
packages/service/src/primitives/scalar.generated.ts line 5 (predates pluginUrls) accurate sed -n '1,10p' …; grep -c pluginUrls packages/service/src/primitives/scalar.generated.ts resolved path; pluginUrls count is 0 as claimed.
plugins/workers/scaffold.runtime.json lines 24‑55 (no registry kind emits routes/pages/islands) accurate sed -n '24,55p' … range resolves and confirms the absence.
plugins/workers/scaffold.plugin.json and plugins/workers/src/public/mod.ts (cited) accurate sed -n '1,30p' …; ls … resolve.
packages/cli/src/kernel/templates/app/generators-config_test.ts (createNetScriptVitePlugin use) accurate sed -n '240,290p' … symbol is used as cited.
.llm/devtools-rfc-orchestrator-brief.md accurate git log -n 1 --pretty=oneline (last touched at ccc4c0a70) the only path claim (docs/architecture/rfc/rfc-0001-runtime-versioned-automation.md) is to a non‑existent file (see F‑F1) but the brief itself is internal; the broader doc that propagates the bad path is the RFC.
.llm/runs/plan-devtools-contribution--seed/plan.md, plan-eval.md, plan-eval-brief.md, decision-brief.md, worklog.md, RFC-AUTHORITY.md inaccurate ls rfcs/0000-devtools-contribution.md docs/architecture/rfc/rfc-0002-devtools-contribution.md all six cite either rfcs/0000-… or docs/architecture/rfc/rfc-0002-devtools-contribution.md; neither exists on the branch. Internal references to the orchestrator brief's docs/architecture/rfc/rfc-0001-runtime-versioned-automation.md are likewise unresolvable.
.llm/runs/plan-devtools-contribution--seed/filing/manifest.md (filing-manifest), filing/epic.md, filing/briefs/W0.mdW6.md, filing/issues/W1-*, W4-a, W6-a inaccurate same as above every filing brief restates the bad 0000/0002 paths and the unimplemented packages/devtools-core and plugins/devtools paths (these are future packages, acceptable as proposals but their claimed contracts/v1/panel.ts, links.ts files are not yet present in the diff).
.llm/runs/plan-devtools-contribution--seed/design/T1-*T9-* (proposals + supersession-map.md) inaccurate repeated 0000/0002 references cite the non‑existent RFC paths; T1-host-shape/proposal.md cites the future plugins/dev_server.ts and packages/devtools-core (acceptable as proposal scope, but must be marked as such).
.llm/runs/plan-devtools-contribution--seed/research/r1-fresh-host.md, r3-plugin-contribution-axes.md, r4-cli-plugin-flows.md, r5-observability-boundary.md, b1-dashboard-board.md, b2-doctrine-and-live-board.md, p1-rfc-890-frontend-contrib.md, p2-rfc-1446-runtime-automation.md, p3-rfc-1390-sdk.md, m1-nuxt-vite.md, m3-admin-consoles.md, m4-aspire-scalar.md, SYNTHESIS-NOTES.md inaccurate repeated RFC-path placeholders, plus vendored upstream snapshots (acceptable inside sources/) research files contain the most occurrences of the 0000/0002 placeholder, but the internal packages/cli/src/… paths they cite (e.g. generate-app-deno-json.ts:112-119) all resolve when the placeholder is replaced with the actual packages/cli/src/kernel/adapters/templates/app/generate-app-deno-json.ts.
.llm/runs/plan-devtools-contribution--seed/research/sources/** (vendored upstream docs from Nuxt, Vite, Strapi, Aspire, Scalar, RFC‑A snapshot, etc.) unverifiable spot-checked nuxt-devtools__docs_content_1.guide_0.getting-started.md, scalar/configuration.md, vite-devtools__docs_errors_DTK0008.md, m3-admin-consoles-fetch-log.md, r1-fresh-host.md (research r1) these are external copies kept under research/sources/; the internal claims they re-cite (e.g. packages/core/src/node/ws.ts, apps/playground/main.ts, docs/architecture/rfc/rfc-0002-devtools-contribution.md) are not present in NetScript. Acceptable as vendored external material, but the research notes re-assert them as if they lived in NetScript.
.llm/runs/plan-devtools-contribution--seed/design/ux-evidence/{qwen,glm,kimi}-*.md, FINDINGS-SWEEP.md inaccurate ls rfcs/0000-* prompt/findings files re-cite the placeholder 0000/0002 paths.
.llm/runs/docs-rfc-runtime-versioned-automation--supervisor/drift.md (supervisor drift) inaccurate ls docs/architecture/rfc/ drift entry cites docs/architecture/rfc paths that don't exist on the branch; the placeholders contracts/v1 and contracts/v1/data are likewise forward references.
Harness / seed manifests: chore(harness) seed activation commit, run metadata, OPENHANDS_SUMMARY_PATH plumbing accurate git log --oneline the harness wiring is internally consistent and uses run-scoped paths; no executable claim to mis-test.
All package.json / deno.json task manifests touched by the PR (deno task strings cited) accurate deno task docs:links --root rfcs --pretty; deno task docs:links --root docs/architecture/rfc --pretty; deno task docs:accuracy every cited task verb (docs:links, docs:accuracy, arch:check, quality:scan) is defined in deno.json and the two docs:links invocations return OK - no broken links, anchors, or orphans. The docs:accuracy task returns PASS once the env is set up to allow child deno exec (see F‑N2).
All sources/rfc-*.md and sources/nuxt-*, sources/vite-*, sources/scalar/*, sources/strapi/*, sources/m3-admin-consoles-fetch-log.md snapshots unverifiable these are intentionally vendored external content; "as-claimed" verification is out of scope for a NetScript accuracy pass the docs say they are copies of upstream sources. Acceptable as a research corpus, but the prose around them reuses their internal paths as if they were NetScript paths.
Test files referenced as evidence (generators-config_test.ts, fresh/Playwright configs, e2e scaffolds) accurate sed -n '240,290p' packages/cli/src/kernel/templates/app/generators-config_test.ts the only test cited in narrative (generators-config_test.ts 247, 284) resolves; no e2e harness is documented to depend on RFC #1450 yet.

3. Blocking findings

  • F‑F1 (BLOCKING) — RFC path placeholders. rfcs/0005-devtools-contribution.md and every
    internal doc that propagates it keep two stale path schemes that never existed on
    this branch and that the RFC's own filename refutes:

    • rfcs/0000-devtools-contribution.md (used in RFC-AUTHORITY, decision-brief,
      plan-eval, plan-eval-brief, worklog, filing-manifest, filing/epic.md,
      filing/briefs/W0.mdW4.md, T9-supersession/supersession-map.md,
      b2-doctrine-and-live-board.md, SYNTHESIS-NOTES.md).
    • docs/architecture/rfc/rfc-0002-devtools-contribution.md (used 8× across
      T1-*T9-* design proposals, plan-eval-brief.md, T9-supersession, all
      ux-evidence/* files, r1-fresh-host.md and other research briefs).
    • docs/architecture/rfc/rfc-0001-runtime-versioned-automation.md (referenced
      by the orchestrator brief and propagated through p1, p2, T8,
      RFC-AUTHORITY).
    • rfcs/RFC-A-sdk-client-composition.md and rfcs/RFC-B-command-composition-kit.md
      (referenced in b2-doctrine-and-live-board.md).
      Every occurrence must be rewritten to rfcs/0005-devtools-contribution.md (or
      to the actual filenames the correlated RFCs sit under, once those exist).
  • F‑F2 (BLOCKING) — RFC capability matrix broken table cell. The
    "Any DevTools host, path, or mode flag" cell in §8 (line 410) begins its
    inline‑code span on line 410 and is closed only on line 474, so the rendered
    table cell swallows the rest of the row and the example grep token is
    truncated mid‑pattern (grep -rniE 'devtools). The next two lines of the
    cell are filled with literal \| separators that the Markdown table parser
    treats as new column boundaries. Reformat the row so the inline code span
    fits on one line
    (or move the example to a fenced block below the table).

  • F‑F3 (BLOCKING) — RFC placeholder path packages/cli/.../generate-app-deno-json.ts.
    Line 700 cites the file with literal ... ellipsis instead of the real
    packages/cli/src/kernel/adapters/templates/app/generate-app-deno-json.ts,
    which the research corpus and the rest of the codebase already use. Any
    automated link checker that resolves the citation will see a 404; manual
    readers have to mentally expand the ellipsis every time.

  • F‑F4 (BLOCKING) — Future‑state paths asserted as if they existed. The
    RFC, the filing briefs, the design proposals, and the research notes all
    reference packages/devtools-core/** and plugins/devtools/** (e.g.
    packages/devtools-core/mod.ts, packages/devtools-core/contracts/v1/panel.ts,
    packages/devtools-core/contracts/v1/links.ts,
    packages/devtools-core/src/domain/deep-link.ts,
    plugins/workers/devtools/queue-inspector.tsx,
    plugins/dev_server.ts, plugins/installed-runtime-registry-generator.ts).
    These are the proposed target structure, not shipped code. The doc must
    label every such reference as a proposal (use a callout box or a leading
    "Target:" prefix) so reviewers do not read them as already shipped, and so
    the link checker (and any future tool that reads the RFC) does not flag them
    as broken.

  • F‑F5 (BLOCKING) — cli/docs:accuracy is a repo task that silently fails
    when the harness runs under LD_LIBRARY_PATH (e.g. Ubuntu self-hosted
    runners, which is exactly this environment). The --allow-run=deno child
    fails to exec with libdeno.so not found, and the task itself returns
    non‑zero with no actionable error. plan.md records this task as PASS
    in its command-by-command matrix (line 200) but on a clean machine it would
    fail; the harness task definition needs a Deno PATH assertion or to drop
    the LD_LIBRARY_PATH requirement.

  • F‑F6 (BLOCKING) — vite.config.ts.template:20-32 "three hardcoded aliases".
    Verified correct, but the RFC repeats this claim with slightly different
    alias counts in different places: §3 says "three hardcoded aliases" (line
    2998), §8 says "three hardcoded aliases + a manual import" (line 410), §6
    says "chain is static template text" (line 471). These are consistent, but
    the orchestrator brief and T1-host-shape/proposal.md use the more
    ambiguous phrase "five alias prefixes" (ui:add row in §8) without
    reconciling the two counts. Re‑align the counts (3 hardcoded + 5 alias
    prefixes serving ui:add is the only consistent reading) and add a single
    cross-reference.

  • F‑F7 (BLOCKING) — dev_server.ts host lifecycle claim. §6 asserts
    dev_server.ts is the canonical host entry, but the file does not exist
    (T1-host-shape/proposal.md cites it as a future file). The RFC must
    mark it explicitly as "shipped by this RFC, see W‑X" or remove the
    reference; today it is a future tense dressed up as a present fact.

  • F‑F8 (non-blocking) — m4-aspire-scalar.md "404 + gh api directory
    listing" claim.
    The negative claim that ?filters= is not a public
    Scalar file is plausible but unverifiable in the harness; recorded as
    unverifiable, not a finding.

  • F‑N1 (NON-blocking) — deno run --no-lock … warnings. The docs:links
    and docs:accuracy tasks are invoked with --no-lock, which means they
    bypass deno.lock resolution. Per the operational contract, the harness
    must not commit deno.lock churn — currently fine, but be aware that any
    future task that does re‑resolve the lock will trigger a contract
    violation unless explicitly requested.

  • F‑N2 (NON-blocking) — deno task docs:accuracy exec path bug. The
    task's --allow-run=deno child process fails to load when the runner
    exports LD_LIBRARY_PATH=/opt/hostedtoolcache/Python/3.13.14/x64/lib
    (the GitHub-hosted Python toolchain). With the env unset the task passes.
    Not a doc bug, but it makes the docs:accuracy claim in the
    plan-eval table non-reproducible from a clean Actions runner.

4. Commands / snippets that were hand-tested

# Documented command Source claim Observed
1 deno task docs:links --root rfcs --pretty plan.md line 199 ("scoped via --root rfcs") OK - no broken links, anchors, or orphans. (3 docs scanned).
2 deno task docs:links --root docs/architecture/rfc --pretty plan-eval-brief.md line 45 OK (0 docs scanned; docs/architecture/rfc is currently empty on the branch).
3 deno task docs:accuracy plan.md line 200; plan-eval-brief.md line 45 PASS only when LD_LIBRARY_PATH is unset; otherwise the inner deno child can't load libdeno.so.
4 grep -rniE 'devtools|_devtools' --include=*.ts --include=*.tsx --include=*.json --include=*.template packages plugins docs/site RFC lines 163, 410, 474, 893, 1472 (the corrected -iE form) exit 1, 0 matches — matches the RFC's headline claim.
5 grep -RIn --exclude='*.lock' --exclude-dir=.llm 'transformIndexHtml' . RFC lines 830, 1832, 2999, 4263, 4264 only the RFC text itself contains the string; no host code matches. Matches the claim.
6 Path-existence cross-check of all 39 unique packages/…:L-L, plugins/…:L-L, docs/…:L-L, .github/…:L-L citations in the RFC RFC §3, §4, §5, §6, §7, §8 38 resolve; 1 is a placeholder (packages/cli/.../generate-app-deno-json.ts:112-119).

5. Exact fixes (apply, in order)

  1. In rfcs/0005-devtools-contribution.md:
    1. Replace every rfcs/0000-devtools-contribution.md reference with
      rfcs/0005-devtools-contribution.md.
    2. Replace every docs/architecture/rfc/rfc-0002-devtools-contribution.md
      reference with rfcs/0005-devtools-contribution.md (or with the
      eventual real filename once RFC 0001 / 0002 / RFC-A / RFC-B are written).
    3. In line 700, replace packages/cli/.../generate-app-deno-json.ts:112-119
      with the full path packages/cli/src/kernel/adapters/templates/app/generate-app-deno-json.ts:112-119.
    4. In §8 (line 410), move the grep -rniE 'devtools|_devtools' … example
      out of the table cell into a fenced block below the table; otherwise
      the cell swallows the rest of the row.
    5. Add a single sentence that explicitly says packages/devtools-core/**
      and plugins/devtools/** (and the dev_server.ts host) are proposed
      target paths
      , not shipped code.
    6. Reconcile the alias counts between §3, §6, and §8 (3 hardcoded aliases
      in the app template + 5 alias prefixes for ui:add are the only
      consistent reading).
  2. In every .llm/runs/plan-devtools-contribution--seed/**.md and
    .llm/devtools-rfc-orchestrator-brief.md file that cites
    rfcs/0000-devtools-contribution.md or
    docs/architecture/rfc/rfc-0002-devtools-contribution.md: replace the
    citation as in (1.i)/(1.ii). The same sed pass is enough for ~30 files
    and prevents the placeholder from leaking into the filing trail.
  3. In deno.json (or in the docs:accuracy task wrapper), either unset
    LD_LIBRARY_PATH before the inner deno call, or stop shelling out to
    deno and re-implement the check as a TypeScript module that reuses the
    already-imported Deno APIs. The current task is non-reproducible from a
    clean Actions runner.
  4. In rfcs/0005-devtools-contribution.md §6 / §8, drop the literal
    dev_server.ts reference, or move it to a "shipped by this RFC" callout.

6. Overall verdict

OPENHANDS_VERDICT: FAIL_FIX — the RFC is well-evidenced and the
corrected absence claims are accurate, but the RFC and its harness
corpus still ship with stale path placeholders (rfcs/0000-…, docs/architecture/rfc/rfc-0002-…,
packages/cli/.../generate-app-deno-json.ts), a Markdown table cell that
truncates its own example, future‑tense paths dressed as present
(packages/devtools-core/**, plugins/devtools/**, dev_server.ts), and
a docs:accuracy task that fails under LD_LIBRARY_PATH. These are
blocking because they (a) make the RFC's link and path references
non-resolvable, (b) make the link-validation evidence in plan.md line
199-200 non-reproducible, and (c) cause the filing trail to point at files
that do not exist. The substantive design and the corrected grep -rniE
form are correct; the doc surface needs the four mechanical rewrites above
before the RFC can be merged.

7. Evaluator contract compliance

  • No source, doc, or deno.lock file modified; no commit, push, or draft PR
    created.
  • All 122 changed documentation-like files appear in the per-file verdict
    table (grouped by artifact class for compactness; all individually
    verifiable from the per-row source claim).
  • The two deno task docs:links invocations and the docs:accuracy
    invocation were the highest-value manual checks; iteration budget kept
    to under 12 seconds of wall time on the focused commands plus a one-shot
    read of every changed doc.
  • The single 4-line command OPENHANDS_VERDICT: FAIL_FIX header is on its
    own line outside code fences and is the first line of both this summary
    and the PR comment.

This summary was generated by an AI agent (OpenHands) on behalf of the
PR author as the documentation accuracy evaluator for PR 1450.

Run: https://github.com/rickylabs/netscript/actions/runs/31532951292

@rickylabs

Copy link
Copy Markdown
Owner Author

[PHASE: FILED] [BOARD MIGRATION EXECUTED — one shot, from the manifest]

RFC 0005 is merged on main (03680f6e8) and the board now matches it. Full record: filing/FILING-LOG.md.

Provenance, stated once so it is not inferred: the formal Codex GPT-5.6 Sol PLAN-EVAL returned FAIL_PLAN twice. Every supervisor-fixable finding was closed, then the owner cleared the gate by written waiver (drift D-18). There is no evaluator PASS — do not cite one.

Filed — 14 new issues

Draft Live Draft Live
DT-RFC #1468 ⚠️ DT-8 #1476
DT-1 #1469 DT-9 #1477
DT-2 #1470 DT-10 #1478
DT-3 #1471 DT-12 #1479
DT-4 #1472 DT-14 #1480
DT-5 #1473 DT-17 #1481
DT-6 #1474 DT-11/13/15/16 (no new issue — folded onto #427 / #423 / #428 / #429#431)
DT-7 #1475 DT-18 not filed

⚠️ #1468 was closed by you as DUPLICATE nine minutes after filing. Left closed — that is your call, not a filing error. The consequence: the RFC's tracking role sits on #400, and the RFC frontmatter's tracking-issue line on main still reads pending. Say the word if you want a follow-up PR pointing it at #400.

DT-18 was deliberately not filed. Its open question D-0b was never decided, and filing an issue whose scope is undecided produces a ticket nobody can start. The gap stays visible in drift D-10/D-15 rather than becoming a stale card.

Epic #400 — amended, not replaced

A second umbrella would fragment the very board this RFC de-fragments (standing decision D-11).

Preserved verbatim (checked present before the edit, not after): the ownership thesis, the three acceptance lines, and the killed-surfaces list. Rewritten: the invent-your-own-discovery premise, the dead beta.6 prose, and the S1–S13 screen list → RFC 0005 pointers. Milestone unchanged.

CR-DDX-HOSTAGNOSTIC — raised on #400 on 2026-07-06 from epic #510 and never resolved — is now recorded as accepted. That un-dangles #544, whose body was not edited: it belongs to #510's lane.

Successor-before-supersede — executed literally

Superseded Successor(s) Order used State
#412 #1471 successor filed first, then the comment open
#424 #1472, #1477 successors filed first, then the comment open

Zero filing-time closes. Every fold/supersede/close-later carries a precondition that filing does not satisfy, so each stays open with its trigger named. No seam is unowned in the interval.

Preservation guarantees — verified, not asserted

Guarantee Result
#922 and its 24 children untouched ✅ zero body/label/milestone/state changes; #933 and #944 received nothing
2026-07-19 milestone train no issue re-milestoned; new children default to 0.0.15, matching the train
No undocumented labels epic:devtools / area:devtools / area:frontend do not exist and were not created — epic:dev-dashboard used
No milestones created, no issue moved ✅ single edit: stripped the stale "Dev dashboard (thin, contribution-based)" clause from 0.0.14's description. Its 11 open issues verified unchanged
Closing-keyword hygiene Part of #400 only; grep across all 14 bodies for closes|fixes|resolves #0 hits

The one authorized #922-adjacent touch is the informational cross-post on #929: RFC #890's contract C8 claims a new manifest block is safely ignored by older CLIs, but PluginInstallerManifestSchema ends in .strict() (packages/plugin/src/protocol/manifest.ts:283), which hard-rejects unknown keys — an older CLI fails manifest parsing outright rather than degrading. RFC 0005 handles this for its own family via #1474 (fork F-3). The comment changes no label, milestone, or scope; how #922 responds is entirely that lane's call.

Two deviations — recorded, not hidden

  1. DT-8 filed as "six-state", manifest said "five-state." The manifest predates Amendment A, which added a sixth quarantine state (zone-contract-mismatch) to close a Qwen finding. Filing the manifest's literal title would have contradicted the merged RFC. Substance unchanged.
  2. Bodies cite rfcs/0005-…, manifest said rfcs/0000-… — per your instruction to use the canonical main link; the number was assigned at acceptance.

Open, and yours

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:docs area:fresh area:fresh-ui area:plugins plugins/* and plugin-core packages canary:0.0.6-canary.2 Published NetScript prerelease 0.0.6-canary.2 ci:skip-e2e ci:skip-scaffold Skip the scaffold-static (deno-only) scaffold gate epic:dev-dashboard Dev Dashboard epic (Spine-1 headline, ships as a plugin) epic:frontend-contrib Frontend contribution layer epic (plugins that ship UI) priority:p1 High rfc Request for Comments - substantial/breaking design change status:ready-merge type:docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant