Skip to content

docs(rfc): RFC-5 polyglot task protocol — ecosystem citizenship for foreign-language tasks - #1687

Open
rickylabs wants to merge 26 commits into
mainfrom
claude/harness-profile-rfc-benchmark-shzhgv
Open

docs(rfc): RFC-5 polyglot task protocol — ecosystem citizenship for foreign-language tasks#1687
rickylabs wants to merge 26 commits into
mainfrom
claude/harness-profile-rfc-benchmark-shzhgv

Conversation

@rickylabs

@rickylabs rickylabs commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Summary

Run 5 of the polyglot series — and a deliberate pivot. Runs 1–4 (#1678 scriptc, #1683 rust-workers, #1685 dotnet, #1686 golang) measured execution and inherited the polyglot contract ("TASK_ID/TASK_PAYLOAD env in, last JSON line of stdout out") as a fixed seam. This run challenges the seam itself (owner brief, 2026-08-20): evolve polyglot tasks from foreign-language runners into ecosystem citizens — interoperability, observability, communication layer, error & lifecycle management — as one generic, cross-language protocol.

Delivered: rfcs/0000-polyglot-task-protocol.md — the NetScript Task Protocol (NTP), revised in S10 (owner content review, drift R5-D-7) from a findings report into a full architectural design proposal (803 lines, 26 code fences, at the accepted-RFC 0001 bar): normative Zod wire schemas (EnvelopeV1, TaskFrameV1/HostFrameV1 unions, StructuredErrorV1 with RETRY|PAUSE|FAIL); TypeScript port interfaces on the auth-core blueprint (composite TaskProtocolBackendPort + narrow ports + typed unsupported errors + named registry); a file-level package map (plugin-workers-core/src/protocol/, testing/conformance/, citizen contracts, withTaskProtocol decorator, KV token store, scaffold-asset shims); a per-seam engine-integration table (file:line, refactored-vs-preserved, defect linkage D-1..D-14); bound builder generics + the TaskOutcomeV1 union; the citizen-surface oRPC contract with a route×capability table and numbered token-verification algorithm; lifecycle state machine; T2 handshake with zombie rules; conformance case grammar; six-axis extension model; edge-case resolutions; and a five-wave staged implementation plan (W1 protocol-core → W5 T2 worker) with acceptance bars tied to the spike numbers. Three conformance tiers (T0 legacy-forever / T1 structured / T2 duplex) are computed by the conformance suite; no TaskType vocabulary changes (series precedent, 5×).

Evidence (Appendix A): 32-file ratified research corpus (two workflow rounds, 25/25 agents), engine audit with file:line defect register, and six pre-registered spikes all passing their primary criteria — headline: the full T1 contract costs +0.41 ms through the real dispatch path (1920/1920 executions), sandbox-scoped loopback at 0.49 ms p50, in-band cancel acked at 3.5/30 ms p50 (Go/python3).

Scope

  • rfcs/0000-polyglot-task-protocol.md — the RFC (docs only).
  • .llm/runs/claude-harness-profile-rfc-benchmark-shzhgv--polyglot-protocol-rfc/ — run artifacts: supervisor/research/plan/plan-eval/evaluate/worklog/drift/context-pack, research-sources/ corpus (32 files), spike code K1–K6, raw JSONL + generated results-spikes.md.
  • No packages/ or plugins/ source is touched (G4; independently re-confirmed by both evaluator cycles).

Slices

S1 bootstrap · S2 research round 1 · S3 round 2 + synthesis · S4 plan + PLAN-EVAL (cycle-1 FAIL_PLAN → fixes → cycle-2 PASS, run 32343592955) · S5 K1+K2 · S6 K3+K5 · S7 K4+K6 · S8 spike synthesis + drift · S9 RFC (first authoring) · S10 owner-review revision (report → architectural design; R5-D-7) · S10b IMPL-EVAL cycle-2 fixes F1–F4. Per-slice commits + PR comments are the trail.

Validation

  • G1 spikes: PASS — all six on primary pre-registered criteria, script-generated results.
  • G2 RFC: PASS on the S10 head — fmt-clean, zero TBD; cycle-2 evaluator hand-tested the Zod 4.4.3 snippets, verified every seam file:line cite against source, and re-checked all spike figures against raw.
  • G3 run artifacts current per slice — restored by S10b (context-pack refreshed; cycle-2 finding F2).
  • G4 docs lane only — PASS (ci:skip-e2e + ci:skip-scaffold).
  • jsr-audit: N/A — docs-only PR.

Harness

  • Run dir: .llm/runs/claude-harness-profile-rfc-benchmark-shzhgv--polyglot-protocol-rfc/ (ARCHETYPE-3 + SCOPE-docs).
  • PLAN-EVAL: PASS (required; cycle 1 FAIL_PLAN → six fixes → cycle 2 PASS, run 32343592955, plan-eval.md).
  • IMPL-EVAL history: cycle 1 PASS on pre-revision head 70d101a (run 32346098261 — superseded by the S10 rewrite, kept as record); cycle 2 on S10 head 45b134a (run 32358375889) — FAIL_FIX with design judged "substantively sound and reference-complete" and four doc fixes (F1 corpus count, F2 context-pack, F3 this body, F4 seam cite path), all applied in c45d6c1; cycle 3 re-dispatched on the fixed head. Full chain mirrored in evaluate.md.
  • Owner-conducted Codex GPT 5.6 Sol Max adversarial pass follows the cycle-3 verdict (supervisor.md lane table).

Drift / Debt

R5-D-1 PLAN-EVAL cycle-1 fixes · R5-D-2 fd-3 infeasible on Deno host · R5-D-3 K6 replica · R5-D-4 UDS constraints · R5-D-5 container/Aspire/Windows untested · R5-D-6 zod lock-alias reverted · R5-D-7 owner content review → S10 redesign. Full detail in drift.md.

Definition of Done

  • Research corpus ratified (primary sources, extracts cited) + engine audit with defect register
  • plan.md locked with pre-registered verdict criteria + out-of-scope register; PLAN-EVAL PASS recorded
  • Spike measurements committed, script-generated
  • RFC authored to the architectural-design bar (S10); gates green
  • IMPL-EVAL verdict on the current head recorded (cycle 3 pending; cycle-1 PASS superseded by S10, cycle-2 FAIL_FIX fixes applied)

Refs #1679, #1684 (protocol intersects the monty sandbox spike and the web-worker pool).

🤖 Generated with Claude Code

https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

@rickylabs rickylabs added this to the Backlog / Triage milestone Aug 20, 2026 — with Claude
@rickylabs rickylabs added area:docs type:docs ci:skip-e2e area:plugins plugins/* and plugin-core packages status:research Harness research phase rfc Request for Comments - substantial/breaking design change ci:skip-scaffold Skip the scaffold-static (deno-only) scaffold gate labels Aug 20, 2026 — with Claude

Copy link
Copy Markdown
Owner Author

[PHASE: RESEARCH] Run 5 opened — RFC-5 polyglot task protocol (ecosystem citizenship).

  • Owner brief: runs 1–4 measured execution but left the polyglot contract unchallenged; RFC-5 owns interoperability, observability, the communication layer, and error/lifecycle management as one generic, cross-language protocol — plus the standing engine caveats (D-4 trace drop, stdout-hijacked result channel, untyped TaskResult, unvalidated fallback parse, one-shot-only spawning).
  • Research fan-out is running now (Opus 5 medium aggregators → Fable analyzers, per owner lane ruling): Faktory · Celery v2 · BullMQ sandboxed · Lambda Runtime API · LSP · Temporal · Hatchet/Inngest/Restate · sd_notify/gRPC-health/CloudEvents/W3C-trace-context · oRPC v1+v2 (message-port transfer, binary, websocket, stdio) · OpenAPI/NSwag codegen · FFI callback channels — plus a local engine audit (defect register incl. file:line cites) and the auth-plugin port/adapter blueprint.
  • Process correction from the series: plan.md will pre-register an out-of-scope register alongside verdict criteria, and PLAN-EVAL is required this time (decision-heavy protocol design) — hard stop before any implementation slice.
  • CI lane: ci:skip-e2e + ci:skip-scaffold (docs/run-dir slices only in this PR).

Next phase comment lands with research.md synthesis.


Generated by Claude Code

claude added 4 commits August 20, 2026 05:36
… lane split (run 5)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
…ps + engine audit

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
…ping, callback surface, restate/conformance raws

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
…tension register, defect map

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

Copy link
Copy Markdown
Owner Author

[PHASE: RESEARCH] Research complete — two workflow rounds, 25/25 agents, corpus committed (research-sources/, 32 files) and synthesized in research.md (d011621).

Round 1 (8 source groups + engine audit + adversarial critic): Faktory/Sidekiq · Celery v2/BullMQ sandboxed · Lambda Runtime API/LSP · Temporal + Hatchet/Inngest/Restate · sd_notify/gRPC-health/CloudEvents/W3C-trace-context · oRPC v1+v2 · OpenAPI/NSwag codegen · FFI callback channels. The engine audit upgraded our caveats into a 14-item defect register with file:line cites (new finds: full parent env leaked to every subprocess D-9, abort never kills the spawned child D-6, attempt hardcoded 0 D-5, timeout collapsed to failed on persist D-8, JS reportProgress dies at console.log D-12) and extracted the auth-plugin port/adapter blueprint as the package template.

Round 2 (critic-driven): security/token scoping corpus (Lambda per-invoke creds, Temporal task tokens, K8s bound SA tokens, RFC 8693, macaroons/Biscuit) — previously a zero-evidence pillar; task→engine callback-surface ruling (attempt-bound verbs on the task channel, everything else on an authenticated loopback oRPC surface — even Tier-0 tasks get full ecosystem access); Restate steals ratified (yanked-version registry hard-confirmed; terminal-frame discipline double-attested; cancellation settled as cooperative-only-on-duplex — T0/T1 keep OS signals, stated honestly); conformance harness designed from gRPC-interop/Autobahn/CloudEvents precedent (<tier>.<verb>.<behavior> cases, dual behavior+exit verdicts, mode inversion so the host is gateable too, generated capability matrix as the tier declaration).

Eight ecosystems converge on the same skeleton (research.md §2): protocol bytes off the payload channel, context in the envelope, structured errors with coordinator-decided retry, terminal-frame discipline, capabilities over version handshakes, heartbeat replies as the control channel, forward-only lifecycle DAG, absolute deadlines.

Next: plan.md locking the §6 tension register (5 spikes flagged: frame transport, token delivery, loopback TCP-vs-UDS + sandbox survival, stdin buffering, protocol overhead vs the run-1–4 baselines), with pre-registered verdict criteria and the out-of-scope register — then PLAN-EVAL dispatch (required this run; status:plan-eval). Hard stop before implementation.


Generated by Claude Code

…-scope register

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
@rickylabs rickylabs added status:plan-eval Awaiting plan evaluation openhands Enable status-driven OpenHands PLAN/IMPL evaluation or request a manual cloud run and removed status:research Harness research phase labels Aug 20, 2026 — with Claude

Copy link
Copy Markdown
Owner Author

[PHASE: PLAN] Plan locked (2a8fb8b) — PLAN-EVAL dispatch requested via openhands + status:plan-eval.

plan.md locks seven decisions from the ratified corpus (L1–L7): three conformance tiers with computed tier declaration; two-surface architecture (attempt-bound task-channel verbs + authenticated loopback oRPC citizen surface for every tier); closed versioned Zod envelope carrying context/attempt/deadline (kills D-1/D-2 by construction); structured errors with RETRY|PAUSE|FAIL behavior + terminal-frame discipline (exit without terminal frame ⇒ synthesized UnknownFailure); reserved NETSCRIPT_* env + allowlisted base env + per-attempt opaque capability token (kills D-9, exposes attempt per D-5); capability negotiation over version handshake with a yankable-version registry; auth-blueprint port/adapter package split with the conformance harness as the tier authority.

Six spikes (L8, K1–K6) carry pre-registered decision criteria — headline: K4 protocol-overhead bar (T1 framing adds ≤1.0ms to the Go subject's 6.2ms exec-wall p50, ≤5% e2e at c=16) so the protocol cannot silently eat the execution wins of #1678#1686; K5 may honestly fail (then T1 cancellation stays signal-only, with the Restate citation licensing exactly that). L9 pre-registers the RFC's own verdict criteria, and the out-of-scope register (9 items) applies the completeness-probe correction from the series miss — anything a reviewer thinks belongs in scope is a FAIL_RESCOPE conversation, not a silent change.

Hard stop is in effect: no spike slice starts before PLAN-EVAL PASS. Evaluator inputs: research.md, plan.md, worklog ## Design checkpoint, corpus under research-sources/.


Generated by Claude Code

@rickylabs

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan head=2a8fb8b5b9f079088694bd1d1746b2dbf76257fb

Trusted base SHA: 9634735
Evaluated head SHA: 2a8fb8b

use harness

SKILL

  • netscript-harness — apply the Plan-Gate and preserve evaluator independence.
  • openhands-handoff — publish one machine-readable OpenHands verdict.
  • netscript-tools — use repository-native, read-only evidence commands.

Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push,
or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head
SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then
evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture,
tests, consumer proof, release risk, and false-done states. Verify claims directly where useful.

Return concise findings with exact evidence and required amendments. End with exactly one supported
verdict line using OPENHANDS_VERDICT: PASS, OPENHANDS_VERDICT: FAIL_PLAN, or
OPENHANDS_VERDICT: FAIL_RESCOPE. Write the same verdict to OPENHANDS_SUMMARY_PATH. This is a
planning gate: never emit FAIL_FIX for implementation that has not started.

@rickylabs

rickylabs commented Aug 20, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

OPENHANDS_VERDICT: FAIL_PLAN

Model: openrouter/minimax/minimax-m3
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Verdict comment: #1687 (comment)
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=skipped replies=skipped draft-pr=skipped
Trace: .llm/tmp/run/openhands/pr-1687/run-32340551406-1 (in the Actions artifact)
Artifact: openhands-agent-32340551406-1

Summary — PLAN-EVAL for RFC-5 polyglot task protocol (PR #1687)

Summary

Formal PLAN-EVAL session for claude-harness-profile-rfc-benchmark-shzhgv--polyglot-protocol-rfc
on PR #1687 (HEAD 2a8fb8b5, base 9634735b). The plan is substantively excellent — research
is thorough (32 corpus files, two-round method, engine audit, D-1..D-14 register), L1–L7 are
locked with corpus citations, the iff-branched K1–K6 spike framing prescribes honest narrowing
on miss, and the out-of-scope register (9 items) closes the runs-1–4 completeness-probe miss.
However, the plan-gate checklist has 3 unchecked boxes: open-decision sweep (T-4, T-5, T-8
not addressed), commit-slices table (sibling-run precedent relies on it), and risk register
(sibling-run precedent relies on it). A minor fourth gap is a missing jsr-audit N/A line for
the docs-only PR. Per gates/plan-gate.md and evaluator/plan-protocol.md, the verdict is
FAIL_PLAN with six required fixes. After the fixes land, the plan re-submits via openhands

Changes

  • No code or doc edits. PLAN-EVAL is read-only.
  • Wrote plan-eval.md per templates/plan-eval.md with the checklist table, the open-decision
    sweep, and the six required fixes.
  • Wrote this summary at OPENHANDS_SUMMARY_PATH.

Validation

  • Walked the 8 plan-gate checklist boxes against plan.md / research.md / worklog.md /
    supervisor.md (read-only).
  • Cross-checked sibling-run plans (dotnet-rfc, rust-workers-rfc, scriptc-rfc-bench,
    golang-rfc) for ## Commit slices / ## Risk register / ## Open decisions sections;
    three of the four previous plans carry the slice + risk sections; the polyglot-protocol-rfc
    plan does not.
  • Verified the engine audit's D-1..D-14 lineage and the L7 conformance-harness split against
    the existing plugin-workers-core/contracts structural-server-contract debt entry
    (consistency confirmed, no new debt).
  • Confirmed the planned protocol/ area in plugin-workers-core and the per-language adapter
    packages are out-of-scope for this PR (docs-only G4); the jsr-audit applies in their own
    implementation waves.
  • Verified rfcs/0000-polyglot-task-protocol.md is correctly absent from the tree (deferred
    to a separate authoring PR per the plan; no scope drift on this PR).
  • Verified the L1–L7 locks are consistent with the corpus citations (callback-surface
    analysis, conformance-harness analysis, Temporal/Restate H2/H3 evidence, engine audit).

Responses to review comments or issue comments

None received on PR #1687 during this session. The PR is a draft still; the
openhands + status:plan-eval dispatch that triggered this run is the only review signal
on the head.

Remaining risks

  • T-4 / T-5 unresolved: MUST be locked in L1/L2 before the L8 spike slices begin, or the
    RFC authoring may pick a channel incompatible with the K1 outcome, forcing rework.
  • T-8 implicit deferral: should be made explicit in the out-of-scope register so the
    supervisor-side fix and the RFC stay aligned.
  • Commit-slices & risk register: leaving either out of the plan risks the L8 spike slice
    sequencing drifting from the declared gates, and the K4-narrow / K5-signal-only / K3-channel
    branches losing their attribution if the plan is later amended.
  • 2-cycle cap: a second FAIL_PLAN after the fix cycle escalates to the user per
    evaluator/plan-protocol.md. Plan author should treat the first round as the gate.
  • Series merge timing: RFC-5 ships before docs(rfc): Go task runtime paths — benchmark, official wasm/c-shared bridges, series close #1686 final; if the RFC needs revisits after
    docs(rfc): Go task runtime paths — benchmark, official wasm/c-shared bridges, series close #1686 merge, the follow-up implementation PR absorbs the change (out-of-scope register
    item 4).

Verdict

OPENHANDS_VERDICT: FAIL_PLAN

Run: https://github.com/rickylabs/netscript/actions/runs/32340551406

@rickylabs

Copy link
Copy Markdown
Owner Author

OPENHANDS_VERDICT: FAIL_PLAN

PLAN-EVAL checklist (gates/plan-gate.md):

Plan-Gate item Result Evidence
Research present and current PASS research.md §1–§8, 32 corpus files, 2-round method, engine audit D-1..D-14
Decisions locked PASS L1–L7 with corpus citations; K1–K6 iff-branches
Open-decision sweep FAIL T-4 (T2 reads in-band vs canonical loopback), T-5 (checkpoint redelivery placement), T-8 (D-11 slot accounting) unaddressed in plan.md
Commit slices (< 30, gate + files each) FAIL No ## Commit slices table; sibling runs dotnet-rfc / rust-workers-rfc / scriptc-rfc-bench carry this section
Risk register FAIL No ## Risk register; notable risks (K4 narrow, K5 signal-only, K3 channel-only, K6 replica drift, 2-cycle cap, series merge timing) uncollected
Gate set selected PASS G1–G4; package-quality gates N/A under G4 docs-lane
Deferred scope explicit PASS Out-of-scope register 9 items, with FAIL_RESCOPE redirect rule
jsr-audit surface scan (pkg/plugin) FAIL jsr-audit N/A line for docs-only PR is missing

The plan is substantively excellent — research is thorough, L1–L7 are locked with corpus
citations, the iff-branched K1–K6 spike framing prescribes honest narrowing on miss, and the
out-of-scope register closes the runs-1–4 completeness-probe miss. The three unchecked boxes
are checklist-form gaps, not design failures.

Open-decision sweep (evaluator-run):

  • T-4 (T2 reads in-band vs canonical loopback). L2's two-surface architecture is consistent
    with canonical-loopback for T2 reads but never explicitly states it. Must resolve now —
    lock in L2 or add a binding constraint for the RFC authoring slice.
  • T-5 (checkpoint redelivery placement). L1 mentions checkpoint as a T2 verb but does not
    name the channel; interacts with K1 (frame transport) and the overhead budget. Must resolve
    now — write a decision rule (envelope metadata for short, inbound frame for medium, artifacts
    ref for long; K1/K4 validate).
  • T-8 (D-11 slot accounting). research.md recommends supervisor-side deferral. Safe to defer
    — add a 10th item to the out-of-scope register so the RFC and the supervisor-side fix stay
    aligned.

Required fixes to unblock PASS:

  1. T-4 lock in L2 (T2 reads use canonical loopback; in-band reserved as v2 extension).
  2. T-5 lock in L1 (channel rule + size cap + K1/K4 validator).
  3. T-8 explicit deferral in the out-of-scope register (item 10).
  4. ## Commit slices table with the Slice / Proves / Gate / Files columns (S1–S9 suggested).
  5. ## Risk register table (Risk / Likelihood / Mitigation) covering the iff-branches and
    2-cycle cap, series merge timing, K1 adversarial regression.
  6. jsr-audit N/A line for the docs-only PR.

After Fix 1–6 land the plan re-submits via openhands + status:plan-eval on PR #1687. The
harness allows up to two FAIL_PLAN cycles before escalating to the user with the unresolved
items (evaluator/plan-protocol.md). The hard stop before L8 spike slices remains in force
(no implementation begins until PASS).

Full deliverable: .llm/tmp/run/openhands/pr-1687/run-32340551406-1/plan-eval.md.

@rickylabs rickylabs added status:plan Harness plan phase and removed status:plan-eval Awaiting plan evaluation labels Aug 20, 2026
…mit slices, risk register

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

Copy link
Copy Markdown
Owner Author

[PHASE: PLAN — cycle-1 fixes applied] All six PLAN-EVAL required fixes landed in f4ae089; requesting cycle-2 re-evaluation via the standing openhands + status:plan-eval labels.

  1. T-4 locked in L2: loopback surface is canonical for citizen access in every tier including T2; in-band citizen reads reserved as a fenced v2 extension.
  2. T-5 locked in L1: checkpoint channel rule — ≤8 KB rides the next attempt's envelope, 8–256 KB is an inbound frame (T2) / truncated-to-ref (T1), >256 KB goes through the loopback artifacts route; caps are protocol constants validated by K1 and counted in the K4 budget.
  3. T-8 explicitly deferred: out-of-scope register item 10 — supervisor-side D-11 fix owns slot accounting; no protocol slot verbs in v1.
  4. ## Commit slices table added: S1–S9 with Proves/Gate/Files, landed slices annotated with commit hashes.
  5. ## Risk register added: 8 rows covering every K-branch (K4 narrowing, K5 signal-only, K3 channel-only, K6 replica drift), the 2-cycle cap, series merge timing, and K1 adversarial regression.
  6. jsr-audit: N/A line recorded for the docs-only PR (audits belong to the implementation waves).

Cycle-1 verdict and dispositions mirrored in the run dir's plan-eval.md. Hard stop before the L8 spike slices remains in force pending the cycle-2 verdict.


Generated by Claude Code

@rickylabs rickylabs added status:plan-eval Awaiting plan evaluation and removed status:plan Harness plan phase labels Aug 20, 2026 — with Claude
@rickylabs

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment iterations=500 phase=plan head=f4ae0898e7d95427373af8efc0571da67bd8e629

Trusted base SHA: 9634735
Evaluated head SHA: f4ae089

use harness

SKILL

  • netscript-harness — apply the Plan-Gate and preserve evaluator independence.
  • openhands-handoff — publish one machine-readable OpenHands verdict.
  • netscript-tools — use repository-native, read-only evidence commands.

Act as the formal PLAN-EVAL session for this pull request. Do not edit files, create commits, push,
or continue implementation. The trigger metadata supplies the trusted base SHA and immutable head
SHA: read harness protocols, verdict definitions, and selected profiles from that base commit, then
evaluate the plan and artifacts at the immutable head. Challenge scope, dependencies, architecture,
tests, consumer proof, release risk, and false-done states. Verify claims directly where useful.

Return concise findings with exact evidence and required amendments. End with exactly one supported
verdict line using OPENHANDS_VERDICT: PASS, OPENHANDS_VERDICT: FAIL_PLAN, or
OPENHANDS_VERDICT: FAIL_RESCOPE. Write the same verdict to OPENHANDS_SUMMARY_PATH. This is a
planning gate: never emit FAIL_FIX for implementation that has not started.

@rickylabs
rickylabs marked this pull request as draft August 20, 2026 10:19
@rickylabs
rickylabs marked this pull request as ready for review August 20, 2026 10:19
@rickylabs

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/deepseek/deepseek-v4-flash-0731 output=pr-comment iterations=800 phase=impl head=45b134aadb477e6bcc5c0821fae28b2f7dea8f9d

Trusted base SHA: 9634735
Evaluated head SHA: 45b134a

use harness

SKILL

  • netscript-harness — apply the formal IMPL-EVAL protocol and verdict vocabulary.
  • openhands-handoff — publish one machine-readable OpenHands verdict.
  • netscript-tools — run the smallest decisive repository-native gates without mutating source.
  • netscript-doctrine — apply package/plugin doctrine when the changed surface requires it.

Act as the formal IMPL-EVAL session for this pull request. Do not edit files, create commits, push,
or repair findings. The trigger metadata supplies the trusted base SHA and immutable head SHA: read
the evaluator protocol, verdict definitions, and selected profiles from that base commit, then
evaluate the PR body, linked issues, run artifacts, final diff, review threads, and architecture
debt at the immutable head. Verify the approved plan or recorded PLAN-EVAL: N/A, design
checkpoint, acceptance criteria, static/runtime/consumer gates, public surface, lock hygiene, and
false-done states. For documentation changes, also read every changed document fully and hand-test
representative executable claims.

Return concise, severity-ranked findings with exact evidence and required action. End with exactly
one supported verdict line using OPENHANDS_VERDICT: PASS, OPENHANDS_VERDICT: FAIL_FIX,
OPENHANDS_VERDICT: FAIL_RESCOPE, OPENHANDS_VERDICT: FAIL_DEBT, or
OPENHANDS_VERDICT: FAIL_PLAN. Write the same verdict to OPENHANDS_SUMMARY_PATH.

@rickylabs

rickylabs commented Aug 20, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

OPENHANDS_VERDICT: FAIL_FIX

Model: openrouter/deepseek/deepseek-v4-flash-0731
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=skipped replies=skipped draft-pr=skipped
Trace: .llm/tmp/run/openhands/pr-1687/run-32358375889-1 (in the Actions artifact)
Artifact: openhands-agent-32358375889-1

OPENHANDS_VERDICT: FAIL_FIX

IMPL-EVAL — RFC-5 polyglot task protocol (PR #1687) — re-dispatch on S10 head

Summary

Formal IMPL-EVAL (separate evaluator session) for the RFC-5 polyglot-task-protocol deliverable at
immutable head 45b134aa (RFC S10 revision, 319→803 lines) against trusted base 9634735. This is
the re-dispatch that drift R5-D-7 mandated after the owner content review rejected the S9 "findings"
shape and required a reference-level redesign; the prior IMPL-EVAL PASS (run 32346098261, head
70d101a) does not cover the revised head.

The revised RFC is substantively sound and reference-complete: PLAN-EVAL chain on record (cycle-1
FAIL → fixes → cycle-2 PASS), all locked decisions L1–L8 intact (no RESCOPE), fmt-clean, zero
TBD/TODO, nondeterminism/symmetry clean-up to the accepted-RFC bar. I hand-verified the Zod 4.4.3
normative snippets (z.int/z.looseObject/z.toJSONSchema + EnvelopeV1 parse — all valid against
the workspace-pinned version), the engine seam file:line citations against source (D-1/D-2/D-3/D-4/
D-9/D-12 all confirm), the builder-generics motivation, and the K1–K6 spike headline numbers against
results-spikes.md/raw. G4 docs-lane holds (no packages/plugins mutation).

However, I found concrete documentation/bookkeeping defects that must be fixed before the
deliverable can pass: an incorrect corpus file count in the RFC itself, a stale context-pack.md
resume point, and a PR body that does not reflect the S10 revision or the re-dispatch. Verdict:
FAIL_FIX (plan valid; docs need small finishing work).

Changes

  • New evaluate.md in the run dir recording this re-dispatch verdict (supersedes the 70d101a PASS,
    preserved as prior record).
  • This summary at the run-scoped path.

Validation

  • deno fmt --check on the RFC: clean.
  • Zod 4.4.3 snippet hand-test: z.int, z.looseObject, z.toJSONSchema (draft-2020-12) all
    functions; EnvelopeV1 mini-model parses.
  • Engine-seam file:line citations verified against source (all accurate).
  • research-sources/ = 32 files; research.md says "32 files" — the RFC's "34-file" is incorrect.
  • Spike figures K1–K6 re-checked against raw; all match the RFC.
  • Open review threads: none unanswered.

Findings (severity-ranked)

  • F1 (medium): RFC "34-file ratified corpus" (Summary + Prior art) and PR body "34 files" are wrong;
    the corpus has 32 files per research-sources/ and research.md. PR body also internally
    inconsistent (32 vs 34). Fix the count in the RFC (both) and align the PR body.
  • F2 (medium): context-pack.md resume point is stale at "S9 complete — awaiting IMPL-EVAL" and does
    not reflect S10 / R5-D-7 / re-dispatch (G3 partial miss; not "updated enough for resume").
  • F3 (low): PR body is stale — Slices list omits S10, Validation/Harness/DoD still present the
    pre-S10 RFC and the 70d101a PASS without noting the owner-mandated re-eval (not a false-done; just
    stale). Update the PR body before closing the DoD.
  • F4 (low): multi-runtime-task-executor.ts:195-205 cited by bare basename; give the full path like
    all other seams.

No FAIL_RESCOPE / FAIL_DEBT / FAIL_PLAN — plan is valid and in-scope; only small doc fixes are owed.

Responses to review comments

None required — no open/outdated review threads were unanswered.

Remaining risks

  • After the F1–F4 fixes, a re-check (or re-dispatch on demand) should confirm the RFC corpus count is
    verified and the resume artifacts/PR body match the S10 head before the DoD is closed and merged.
  • K3/UDS and Docker/Aspire survival remain explicit "not measured in-container" limitations recorded
    in the RFC; they do not block the docs deliverable.

Verdict

OPENHANDS_VERDICT: FAIL_FIX

Run: https://github.com/rickylabs/netscript/actions/runs/32358375889

…text-pack, seam cite)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

Copy link
Copy Markdown
Owner Author

[PHASE: IMPL-EVAL — cycle-2 fixes applied] All four FAIL_FIX findings from run 32358375889 dispositioned in c45d6c1 + this body update; requesting cycle-3 re-dispatch.

  • F1 corpus count corrected 34 → 32 in the RFC (Summary + Prior art) and aligned in the PR body — verified against research-sources/ (ls | wc -l = 32) and research.md.
  • F2 context-pack.md resume point rewritten to the S10 state: records the R5-D-7 revision, the full eval-cycle history (cycle-1 PASS superseded → cycle-2 FAIL_FIX → fixes), and the pending cycle-3.
  • F3 PR body rewritten for the S10 head: S10/S10b in the slice trail, the superseded cycle-1 PASS and cycle-2 FAIL_FIX both recorded honestly in Harness, and the DoD verdict box re-opened pending cycle 3.
  • F4 seam cite expanded to the full path (packages/plugin-workers-core/src/executor/multi-runtime-task-executor.ts:195-205).

Worth noting from the cycle-2 report: the evaluator judged the S10 design itself "substantively sound and reference-complete" — it hand-tested the Zod 4.4.3 normative snippets, verified every engine-seam file:line citation against source, and re-checked the K1–K6 figures; the findings were bookkeeping only. Eval-loop accounting: first FAIL_FIX of the IMPL-EVAL loop (two-failure limit). Mirrored in evaluate.md; re-dispatching via draft→ready now.


Generated by Claude Code

@rickylabs
rickylabs marked this pull request as draft August 20, 2026 10:38
@rickylabs
rickylabs marked this pull request as ready for review August 20, 2026 10:39
@rickylabs

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/deepseek/deepseek-v4-flash-0731 output=pr-comment iterations=800 phase=impl head=c45d6c1cf6cbb7f79d047b9dc694d8d20d406e31

Trusted base SHA: 9634735
Evaluated head SHA: c45d6c1

use harness

SKILL

  • netscript-harness — apply the formal IMPL-EVAL protocol and verdict vocabulary.
  • openhands-handoff — publish one machine-readable OpenHands verdict.
  • netscript-tools — run the smallest decisive repository-native gates without mutating source.
  • netscript-doctrine — apply package/plugin doctrine when the changed surface requires it.

Act as the formal IMPL-EVAL session for this pull request. Do not edit files, create commits, push,
or repair findings. The trigger metadata supplies the trusted base SHA and immutable head SHA: read
the evaluator protocol, verdict definitions, and selected profiles from that base commit, then
evaluate the PR body, linked issues, run artifacts, final diff, review threads, and architecture
debt at the immutable head. Verify the approved plan or recorded PLAN-EVAL: N/A, design
checkpoint, acceptance criteria, static/runtime/consumer gates, public surface, lock hygiene, and
false-done states. For documentation changes, also read every changed document fully and hand-test
representative executable claims.

Return concise, severity-ranked findings with exact evidence and required action. End with exactly
one supported verdict line using OPENHANDS_VERDICT: PASS, OPENHANDS_VERDICT: FAIL_FIX,
OPENHANDS_VERDICT: FAIL_RESCOPE, OPENHANDS_VERDICT: FAIL_DEBT, or
OPENHANDS_VERDICT: FAIL_PLAN. Write the same verdict to OPENHANDS_SUMMARY_PATH.

@rickylabs

rickylabs commented Aug 20, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

OPENHANDS_VERDICT: FAIL_FIX

Model: openrouter/deepseek/deepseek-v4-flash-0731
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=skipped replies=skipped draft-pr=skipped
Trace: .llm/tmp/run/openhands/pr-1687/run-32359981617-1 (in the Actions artifact)
Artifact: openhands-agent-32359981617-1

IMPL-EVAL cycle-3 (OpenHands) — RFC-5 polyglot task protocol

Trusted base: 9634735
Evaluated head: c45d6c1
PR: #1687 (draft)
Profile: ARCHETYPE-3-runtime-behavior + SCOPE-docs overlay. Docs-only run.

Summary

The sole changed deliverable is rfcs/0000-polyglot-task-protocol.md (803-line S10 full
architectural-design RFC) plus the .llm/runs/...polyglot-protocol-rfc/ run artifacts. Cycle-2
FAIL_FIX fixes F1-F4 (corpus count 34->32; context-pack refresh; PR body rewrite; full-path seam
cite) are all confirmed applied at c45d6c1c:

  • F1: RFC Summary + Prior art + PR body all state the corrected "32-file" ratified corpus (verified
    32 files under research-sources/).
  • F2: context-pack resume point now reads "S10 revision cycle — owner review R5-D-7 rejected the
    S9 report-shaped RFC".
  • F3: PR body is non-stale — reflects S10 + 32-file corpus + cycle-1/2 verdict history + DoD with
    the cycle-3 box (this pass) opened.
  • F4: seam cite is full-path
    packages/plugin-workers-core/src/executor/multi-runtime-task-executor.ts:195-205 (verified at
    source, createDefaultRuntimeAdapterMap).

Validation

  • Design checkpoint recorded in worklog.md (L1-L9 pre-registered, verdict criteria L8 spikes,
    PLAN-EVAL cycle-2 PASS at head f4ae089). No process-regression (Plan-Gate satisfied, design
    recorded, per-slice commit trail with named gates, drift R5-D-1..R5-D-7 complete, no SKILL-less
    subagent briefs).
  • Every seam file:line cite in the per-seam table hand-verified against source:
    job-dispatcher.ts:234-240 (env{TASK_ID,TASK_PAYLOAD?}), dax-process-runner.ts:89-98
    (buildEnvironment with Deno.env.toObject()), dax-process-runner.ts:171-189 and definition
    span at :178-189 (parseJsonLastLine), :39-49 (abort), :84/191-197 (timeout/status),
    executor-types.ts:60-73 (TaskExecutionOptions additive fields), :90-104 (TaskResult addenda),
    worker.ts:163-169 (reportProgress console.log), job-execution.ts:24 (executePolyglotTask drops
    correlationId), auth-core ports mod.ts:212-355 (AuthBackendPort). All match.
  • Spike evidence Appendix A reconciled against results/results-spikes.md and raw JSONL presence:
    K1 200/200 frames ~127 MB/s ~1.25 GB/rep (10 reps) fd-3 infeasible R5-D-2, K2 0400 + allowlist,
    K3 0.49/0.67 ms + NotCapable-denied + UDS demoted, K4 +0.41 ms / host 0.06-0.10 ms / e2e negative
    (1920/1920), K5 3.5/30.2 ms p50 60/60, K6 93.9 ms p95 steady 9.5x coalesce 82-84B
    MEASURED-ON-REPLICA (R5-D-3). Numbers match.
  • Docs-overlay gates: source alignment (per-prescriptive claim cites code/RFC/spike result),
    scope separation (RFC is explicitly target-state Draft; baseline/Motivation separates current
    state), link integrity (all cited local code/corpus paths resolve), terminology self-consistent,
    drift log (R5-D-2/3/5/7 carried into Unresolved + Known Limitations as required), no false-done
    states engaged.

Findings (severity-ranked)

F-1 [moderate — doc accuracy, FAIL_FIX] Summary misstates the plan's retired-defect register

  • SVG evidence (line-registered): rfcs/0000-polyglot-task-protocol.md line 32 (Summary) states
    the five-wave plan "structurally retires defect classes D-1..D-10/D-12/D-13".
  • The authoritative in-doc scope (wave table lines 671-675 + per-seam table lines 447-458) retires
    exactly D-1, D-2, D-3, D-4, D-5, D-6, D-7, D-8, D-9, D-12, D-13, D-14 — i.e. D-10 (unbounded
    runner stdout buffering, audit row "cap buffers head/tail, persist overflow") is NOT retired by
    any wave or seam, and D-14 (complete() recomputes measured duration lost) IS retired (W3 /
    seam rows 452, 458).
  • So this "D-1..D-10/D-12/D-13" is wrong two ways: it over-claims D-10 and omits D-10/14.
    The correct ability set per the document's own plan is "D-1..D-9, D-12, D-13, D-14"
    (D-10/D-11 not retired).
  • Required: fix the summary sentence on line 32 to match the wave table (drop "D-10"; append D-14);
    optionally state in the table that D-10/D-11 are intentionally out of the five-wave retirement.
    Same accuracy bar as the cycle-1 F1 corpus-count fix.

Remaining (minor, non-blocking, consistent with R5 records, no action):

  • Loopback Docker/Aspire/Windows survival still unmeasured (R5-D-5, opened in Unresolved) — W2 must measure.
  • K6 progress chain is MEASURED-ON-REPLICA (R5-D-3) — W4 discharges.
  • Line 14/38 "+0.41 ms to the 7.6 ms exec-wall p50" matches measured K4 (7.59->8.00), OK.

Responses to review comments

Blocking per instruction (evaluator session; do not repair). This cycle reports FAIL_FIX with the
precise required action for the current head c45d6d1c and does not self-certify.

Remaining risks

  • Evidence was sampled (seam file:line + spike figure spot-checks), not a full re-run re-derivation
    of every raw figure; independent confirmation of sub-figure (unavoidable in read-only evaluator).
  • The defect-register discrepancy is per in-doc tables; a reviewer might disregard the Summary as
    prose, but for this RFC accuracy of the headline retirement claim is precisely the correctness bar
    the harness has enforced all cycle.

OPENHANDS_VERDICT: FAIL_FIX

Run: https://github.com/rickylabs/netscript/actions/runs/32359981617

…aligned with wave tables

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

Copy link
Copy Markdown
Owner Author

[PHASE: IMPL-EVAL — cycle-3 fix applied; escalating per the two-failure rule]

Cycle 3 (run 32359981617) confirmed all four cycle-2 fixes and re-verified every seam citation and spike figure, then returned FAIL_FIX on one moderate finding: the Summary's headline claim said the five-wave plan retires "D-1..D-10/D-12/D-13", contradicting the RFC's own wave/seam tables (which retire D-1..D-9, D-12, D-13, D-14 — D-10 is not wave-retired). The evaluator was right on both counts.

Fixed in 02b1c6e: the Summary now reads "retires defect classes D-1..D-9, D-12, D-13, and D-14 (D-10 buffer capping and D-11 slot accounting stay supervisor-side engine fixes outside the wave plan)", and an explicit "deliberately outside the five-wave retirement" note for D-10/D-11 now sits under the wave table. FMT re-checked clean; mirrored in evaluate.md.

Escalation: this was the second FAIL_FIX of the IMPL-EVAL loop, which reaches the harness's two-failure limit — so per evaluator/plan-protocol.md I'm escalating to you rather than self-dispatching a fourth cycle. Both cycles' findings were doc-accuracy items (now all applied); no design finding has survived any cycle. Your call:

  1. Final re-dispatch (I'll toggle draft→ready on your word) to get a head-pinned PASS on 02b1c6e, or
  2. Fold verification into your Codex Sol Max adversarial pass — its prompt already covers the full artifact set; a CLEAR there plus the applied-fix trail could stand as the closing record, with the DoD box ticked on your acceptance.

Until you rule, the DoD verdict box stays open and status:impl-eval stands.


Generated by Claude Code

…ia OpenHands (R5-D-8)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/x-ai/grok-4.6 output=pr-comment iterations=800

use harness

SKILL

  • netscript-harness — run-artifact map and evaluator-independence rules; this is an owner-commissioned adversarial pass, distinct from the formal PLAN/IMPL-EVAL lanes.
  • netscript-doctrine — judge the proposed port/adapter package split against package/plugin doctrine.
  • netscript-tools — repository-native read-only evidence commands; do not mutate source or deno.lock.
  • netscript-pr — PR conventions context for reading the phase-comment trail.
  • netscript-deno-toolchain — for any dependency/API-surface verification you need (deno doc, deps wrappers).

ROLE

Act as the OWNER-COMMISSIONED ADVERSARIAL REVIEWER for RFC-5 on PR #1687, at HIGH reasoning effort throughout. This is a third pass ON TOP of the formal gates — do not re-run their checklists, and do not re-discover their findings: PLAN-EVAL cycle-2 PASS (run 32343592955); IMPL-EVAL cycle-1 PASS on the pre-revision head (superseded by the S10 redesign), cycle-2 FAIL_FIX (4 doc fixes, applied in c45d6c1), cycle-3 FAIL_FIX (1 doc fix — defect-retirement claim — applied in 02b1c6e). No design finding has survived any cycle; every applied fix is verifiable in the commit trail. Your job is to BREAK the design and the argument. Read-only: do not edit files, commit, push, or repair findings; never mutate deno.lock. Review head: 3d772d2 (RFC content identical to 02b1c6e; the tip adds only a drift record) on branch claude/harness-profile-rfc-benchmark-shzhgv, base 9634735.

CONTEXT

Run 5 of the polyglot RFC series (#1678 scriptc, #1683 rust, #1685 dotnet, #1686 golang — merged execution-layer runs). RFC-5 specifies the NetScript Task Protocol (NTP): ecosystem citizenship for polyglot tasks — interoperability, observability, communication layer, error & lifecycle management.

Read in order:

  1. rfcs/0000-polyglot-task-protocol.md — THE deliverable (803-line architectural design: Zod wire schemas, TS ports on the auth-core blueprint, package map, per-seam engine integration table, five-wave plan).
  2. .llm/runs/claude-harness-profile-rfc-benchmark-shzhgv--polyglot-protocol-rfc/plan.md (locks L1–L9, spike criteria), research.md (+ UNVERIFIED register §7), research-sources/ (32-file corpus incl. netscript-engine-audit.md, defect register D-1..D-14 at file:line), results/results-spikes.md + results/raw/k*.jsonl, bench/spikes/, drift.md (R5-D-1..8), plan-eval.md, evaluate.md, worklog.md.
  3. PR docs(rfc): RFC-5 polyglot task protocol — ecosystem citizenship for foreign-language tasks #1687 body + phase comments (the commit trail).

Core decisions to attack: three conformance tiers computed by a conformance suite (T0 legacy-forever / T1 structured one-shot / T2 duplex worker); two surfaces (attempt-bound task-channel verbs vs authenticated loopback oRPC citizen surface, TCP 127.0.0.1, per-attempt opaque capability tokens invalidated on retry); sentinel-NDJSON stdout framing (\x00NSF\x00, single-write ≤ PIPE_BUF, byte-stream sentinel-scan demux — never line-anchored); closed versioned Zod envelope in TASK_PAYLOAD; structured errors {errtype, behavior: RETRY|PAUSE|FAIL} + terminal-frame discipline (exit-without-terminal ⇒ synthesized unknown-failure); reserved NETSCRIPT_* env + constructed allowlisted env; withTaskProtocol decorator integration (no adapter rewrite); auth-blueprint package split; five-wave staged plan.

ACCEPTED caveats — do NOT report as discoveries, but DO attack whether accepting them was sound: K6 measured on a replica (R5-D-3); fd-3 infeasible on the Deno host (R5-D-2); UDS demoted (R5-D-4); Docker/Aspire/Windows loopback survival untested (R5-D-5); sentinel-forgery residual (RFC Drawbacks #2); engine defects D-1..D-14 deliberately not fixed in this docs-only PR; D-10/D-11 deliberately outside the five-wave retirement.

ATTACK SURFACE (in priority order)

  1. Wire-protocol soundness: races, ambiguous states, unrepresentable outcomes in the frame grammar, terminal discipline, cancellation ladder, deadline semantics, T2 handshake. Construct CONCRETE failure scenarios (interleavings, crash-mid-frame, duplicate results, redelivery races, clock skew).
  2. Security model: token lifetime/rotation holes, loopback SSRF/confused-deputy angles, the same-uid env-delivery trust argument, capability-scoping gaps, whether per-dispatch T2 tokens actually fence stale attempts under queue redelivery.
  3. Evidence validity: does any spike measure something other than what its conclusion claims (4-core container, short workloads, replica boundaries, throttle-dominated K6, K4 noise-negative deltas)? Would a conclusion plausibly flip at scale or on other hardware/OS?
  4. Spec ambiguity: anything a Rust/Go/C#/Python SDK author could implement two incompatible ways from the RFC text alone.
  5. Migration realism: can the five waves land without breaking T0 guarantees or the fix(workers): documented web-worker/subprocess runner modes are unimplemented — docs truthing or Tier-3 pool implementation #1684 pool plans? Anything load-bearing hidden in "follow-up waves"?
  6. Completeness probe: what did this run treat as out of scope that an owner who asked for tasks to be "actually part of the ecosystem, not just foreign language runners" would consider the point?

OUTPUT

Write the full deliverable to OPENHANDS_SUMMARY_PATH (and the run-dir trace per the handoff contract): severity-ranked findings (BLOCKER / MAJOR / MINOR / OBSERVATION), each with an exact file/section cite, a concrete failure scenario or ambiguity demonstration (not vibes), and a minimal fix direction. End with BOTH verdict lines, exactly once each:
ADVERSARIAL_VERDICT: CLEAR | CONCERNS | BLOCK (your real verdict), and the machine mapping
OPENHANDS_VERDICT: PASS (if CLEAR) or OPENHANDS_VERDICT: FAIL_FIX (if CONCERNS or BLOCK).
Do not resolve threads, post extra comments, or modify the PR.


Generated by Claude Code

claude added 2 commits August 20, 2026 15:33
…R5-D-8 update)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
…wen3.8-max

R5-D-8 resolved: owner chose the allowlisted open-model fallback after the
Grok 4.6 dispatch was rejected by the main-ref workflow allowlist. No
workflow or config change; dispatch proceeds on the refreshed head.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/qwen/qwen3.8-max output=pr-comment iterations=800

use harness

SKILL

  • netscript-harness — run-artifact map and evaluator-independence rules; this is an owner-commissioned adversarial pass, distinct from the formal PLAN/IMPL-EVAL lanes.
  • netscript-doctrine — judge the proposed port/adapter package split against package/plugin doctrine.
  • netscript-tools — repository-native read-only evidence commands; do not mutate source or deno.lock.
  • netscript-pr — PR conventions context for reading the phase-comment trail.
  • netscript-deno-toolchain — for any dependency/API-surface verification you need (deno doc, deps wrappers).

ROLE

Act as the OWNER-COMMISSIONED ADVERSARIAL REVIEWER for RFC-5 on PR #1687, at HIGH reasoning effort throughout. This is a third pass ON TOP of the formal gates — do not re-run their checklists, and do not re-discover their findings: PLAN-EVAL cycle-2 PASS (run 32343592955); IMPL-EVAL cycle-1 PASS on the pre-revision head (superseded by the S10 redesign), cycle-2 FAIL_FIX (4 doc fixes, applied in c45d6c1), cycle-3 FAIL_FIX (1 doc fix — defect-retirement claim — applied in 02b1c6e). No design finding has survived any cycle; every applied fix is verifiable in the commit trail. This pass also serves as the closing verification for the IMPL-EVAL two-failure escalation (owner ruling in drift R5-D-8). Your job is to BREAK the design and the argument. Read-only: do not edit files, commit, push, or repair findings; never mutate deno.lock. Review head: 4242a46 (RFC content identical to 02b1c6e; the tip adds only drift records) on branch claude/harness-profile-rfc-benchmark-shzhgv, base 9634735.

CONTEXT

Run 5 of the polyglot RFC series (#1678 scriptc, #1683 rust, #1685 dotnet, #1686 golang — merged execution-layer runs). RFC-5 specifies the NetScript Task Protocol (NTP): ecosystem citizenship for polyglot tasks — interoperability, observability, communication layer, error & lifecycle management.

Read in order:

  1. rfcs/0000-polyglot-task-protocol.md — THE deliverable (803-line architectural design: Zod wire schemas, TS ports on the auth-core blueprint, package map, per-seam engine integration table, five-wave plan).
  2. .llm/runs/claude-harness-profile-rfc-benchmark-shzhgv--polyglot-protocol-rfc/plan.md (locks L1–L9, spike criteria), research.md (+ UNVERIFIED register §7), research-sources/ (32-file corpus incl. netscript-engine-audit.md, defect register D-1..D-14 at file:line), results/results-spikes.md + results/raw/k*.jsonl, bench/spikes/, drift.md (R5-D-1..8), plan-eval.md, evaluate.md, worklog.md.
  3. PR docs(rfc): RFC-5 polyglot task protocol — ecosystem citizenship for foreign-language tasks #1687 body + phase comments (the commit trail).

Core decisions to attack: three conformance tiers computed by a conformance suite (T0 legacy-forever / T1 structured one-shot / T2 duplex worker); two surfaces (attempt-bound task-channel verbs vs authenticated loopback oRPC citizen surface, TCP 127.0.0.1, per-attempt opaque capability tokens invalidated on retry); sentinel-NDJSON stdout framing (\x00NSF\x00, single-write ≤ PIPE_BUF, byte-stream sentinel-scan demux — never line-anchored); closed versioned Zod envelope in TASK_PAYLOAD; structured errors {errtype, behavior: RETRY|PAUSE|FAIL} + terminal-frame discipline (exit-without-terminal ⇒ synthesized unknown-failure); reserved NETSCRIPT_* env + constructed allowlisted env; withTaskProtocol decorator integration (no adapter rewrite); auth-blueprint package split; five-wave staged plan.

ACCEPTED caveats — do NOT report as discoveries, but DO attack whether accepting them was sound: K6 measured on a replica (R5-D-3); fd-3 infeasible on the Deno host (R5-D-2); UDS demoted (R5-D-4); Docker/Aspire/Windows loopback survival untested (R5-D-5); sentinel-forgery residual (RFC Drawbacks #2); engine defects D-1..D-14 deliberately not fixed in this docs-only PR; D-10/D-11 deliberately outside the five-wave retirement.

ATTACK SURFACE (in priority order)

  1. Wire-protocol soundness: races, ambiguous states, unrepresentable outcomes in the frame grammar, terminal discipline, cancellation ladder, deadline semantics, T2 handshake. Construct CONCRETE failure scenarios (interleavings, crash-mid-frame, duplicate results, redelivery races, clock skew).
  2. Security model: token lifetime/rotation holes, loopback SSRF/confused-deputy angles, the same-uid env-delivery trust argument, capability-scoping gaps, whether per-dispatch T2 tokens actually fence stale attempts under queue redelivery.
  3. Evidence validity: does any spike measure something other than what its conclusion claims (4-core container, short workloads, replica boundaries, throttle-dominated K6, K4 noise-negative deltas)? Would a conclusion plausibly flip at scale or on other hardware/OS?
  4. Spec ambiguity: anything a Rust/Go/C#/Python SDK author could implement two incompatible ways from the RFC text alone.
  5. Migration realism: can the five waves land without breaking T0 guarantees or the fix(workers): documented web-worker/subprocess runner modes are unimplemented — docs truthing or Tier-3 pool implementation #1684 pool plans? Anything load-bearing hidden in "follow-up waves"?
  6. Completeness probe: what did this run treat as out of scope that an owner who asked for tasks to be "actually part of the ecosystem, not just foreign language runners" would consider the point?

OUTPUT

Write the full deliverable to OPENHANDS_SUMMARY_PATH (and the run-dir trace per the handoff contract): severity-ranked findings (BLOCKER / MAJOR / MINOR / OBSERVATION), each with an exact file/section cite, a concrete failure scenario or ambiguity demonstration (not vibes), and a minimal fix direction. End with BOTH verdict lines, exactly once each:
ADVERSARIAL_VERDICT: CLEAR | CONCERNS | BLOCK (your real verdict), and the machine mapping
OPENHANDS_VERDICT: PASS (if CLEAR) or OPENHANDS_VERDICT: FAIL_FIX (if CONCERNS or BLOCK).
Do not resolve threads, post extra comments, or modify the PR.


Generated by Claude Code

@rickylabs

rickylabs commented Aug 20, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

OPENHANDS_VERDICT: FAIL_FIX

Model: openrouter/qwen/qwen3.8-max
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=success replies=skipped draft-pr=skipped
Trace: .llm/tmp/run/openhands/pr-1687/run-32398533310-1 (in the Actions artifact)
Artifact: openhands-agent-32398533310-1

Adversarial review — RFC-5 NetScript Task Protocol (PR #1687)

Pass: owner-commissioned adversarial review (third pass on top of formal gates; closing
verification of the IMPL-EVAL two-failure escalation per drift R5-D-8).
Review head: 4242a46 on claude/harness-profile-rfc-benchmark-shzhgv (verified: RFC content
byte-identical to 02b1c6e; tip adds only drift.md). Base 9634735 (merge-base confirmed).
Method: read-only. All citations were re-derived directly from the RFC text, the run corpus
(plan.md, research.md, results/results-spikes.md, results/raw/k4_*.jsonl,
bench/spikes/k6/run-k6.ts, research-sources/netscript-engine-audit.md), and live source
(packages/plugin-workers-core/src/domain/task.ts, plugins/workers/worker/job-dispatcher.ts).
Where statistics mattered, they were recomputed from raw JSONL (see MINOR-1).

Non-duplication: PLAN-EVAL c2 PASS and IMPL-EVAL c1–c3 findings were read before this pass.
Nothing below repeats cycle-2 F1–F4 (corpus count, context-pack, PR body, seam-cite path) or
cycle-3 F-1 (Summary's defect-retirement enumeration D-10/D-14). MAJOR-2 is adjacent in flavor
to F-1 (a retirement claim checked against wave scope) but targets a different defect (D-5), a
different location (wave/seam tables, not the Summary), and a different mechanism (missing retry
engine vs wrong enumeration) — flagged explicitly so the overlap is visible.

What survived attack (not findings): sentinel-scan demux soundness (K1 corpus, partial-sentinel
carry, crash-mid-frame ⇒ malformed-sentinel/unknown-failure is coherent); the closed-envelope

  • open-world-frame split; first-wins duplicate-result discipline; result-wins-over-exit-code;
    deadline-as-advisory clock-skew resolution; T2 zombie rule; same-uid env trust argument (accepted
    caveat, sound: /proc/*/environ 0400 + sandboxed deno cannot read /proc); and the package split
    (OBSERVATION-1).

Findings (severity-ranked)

MAJOR-1 — Frame grammar cannot represent a >~4 KB task result: terminal result frames have no size ladder, so ordinary T1 results demote to log and flip to unknown-failure

Cites: rfcs/0000-polyglot-task-protocol.md:343 (FRAME_MAX_BYTES = 4096, "a frame MUST be
one write(2) ≤ this budget"), :348-353 (demux: sentinel→newline spans are candidates), :658
(edge table: "Frame > FRAME_MAX_BYTES ⇒ … content demoted to log; writers must chunk via progress
detail or artifacts"), :311-315 (terminal result frame: value: z.unknown()), :594-595
(size ladder exists only for checkpoints), :519 (artifacts route = optional capability).

Failure scenario. A Tier-1 Go task does its job and returns a perfectly ordinary result — an
image metadata blob, a batch summary, an API payload — whose JSON serializes to 6 KB. The shim
writes one sentinel-prefixed result frame (as it must: terminal discipline requires exactly one
result, and no multipart-result frame type exists). The span exceeds FRAME_MAX_BYTES; per the
normative edge table the engine demotes the terminal frame to a log line and logs
Protocol.FrameOverflow. The attempt now has no terminal frame ⇒ engine-synthesized
unknown-failure (:110, :581, :584-586) despite exit code 0 and a successful run. The
suggested escape — "chunk via progress detail" — cannot carry a result: progress frames are not
terminal, first-wins applies only to result frames, and reassembling a chunked terminal value
is not specified anywhere. The other escape — artifact ref — requires the loopback citizen surface
and the optional artifacts capability, which (a) is absent offline and (b) contradicts T1's
structured-one-shot zero-infrastructure promise. Note the inversion: T0's last-JSON-line fallback
(parseJsonLastLine, no 4096 budget) returns arbitrarily large results today, so the RFC's own
"upgrade path" from T0 to T1 shrinks the representable result space. Any Go/Rust/C#/Python SDK
author hits this in week 1 of real workloads.

Fix direction. Give the terminal frame the same ladder the RFC already wrote for checkpoints
(T-5): result ≤ ~4 KB in-frame; larger results as a normative artifactRef convention inside
value (with resultSchema validation defined over the ref shape), or an explicit
terminal-frame-only exemption from FRAME_MAX_BYTES with a bounded two-phase read. State the max
in-frame result size in the constants block so SDK authors can enforce it client-side.


MAJOR-2 — The RETRY lifecycle, attempt fencing, and checkpoint redelivery all presuppose a retry engine that does not exist and that no wave W1–W5 builds; W2's D-5 retirement is therefore over-claimed

Cites: RFC :576 (result{error,behavior:RETRY} → retrying → new attempt), :587-588
("Retry is decided by the engine (budget + per-errtype disposition hook)"), :267-271
(envelope retry.remaining — no producer specified), :315 (checkpoint "persisted for
redelivery when 'cancelled'" — no consumer in v1), :60 (admission: "maxRetries never drives a
retry loop (D-5)"), wave table :670-676 (no wave creates a retry/re-enqueue controller), seam
table :448 and wave row :673 (W2 claims to retire D-5); audit
research-sources/netscript-engine-audit.md:212 (D-5 fix = "Carry attempt …, increment on
retry
, expose ATTEMPT" — three parts).

Failure scenario. This is a migration-realism hole inside the RFC's own staged plan, not a
complaint that implementation is deferred (out-of-scope item 1 is acknowledged). The five-wave
plan is normative, and it is internally inconsistent: behavior: RETRY, nextRetryDelayMs
(:284), retry.remaining, attempt tokens "invalidated on retry" (:108, :377, :558-559),
checkpoint-into-next-envelope (:594), and the retrying state are all load-bearing vocabulary
whose only executor is a retry controller that no wave scopes, builds, or gates. W2 threads
attempt "from the message/record" — but nothing increments it, so W2's acceptance bar passes with
attempt permanently 0 (today's behavior), while the wave table claims D-5 retired. D-5's
increment-on-retry half is retired by nothing. Similarly the edge-case row "Queue redelivery races
a live attempt ⇒ attempt tokens fence" (:659) describes a race that cannot occur in v1 (no
redelivery) and whose fencing (MAJOR-4/MINOR-2) is specified against a surface, not the engine's
own state writes. A Rust/Go SDK author implementing behavior: RETRY from this text will build
retry semantics the engine silently ignores; an operator reading "D-5 retired" will believe
retries work.

Fix direction. Either (a) add the retry/re-enqueue controller (attempt increment, budget,
disposition hooks, checkpoint handoff) to an explicit wave with its own acceptance bar, or (b)
state normatively that in v1 behavior is advisory-and-unconsumed, retry.remaining is a
reserved field, attempt is constant 0, and split the D-5 retirement claim (propagation/exposure
in W2; increment deferred with the retry engine). Either fix is text-only; what breaks today is
the claim set, not the design.


MAJOR-3 — Normative progress throttle contradicts the spike that validates it: min(0.8×timeout, 30 s) vs K6's 100 ms flush, and the W4 acceptance bar is unmeetable under the normative parameter

Cites: RFC :454 ("Latest-wins throttle (min(0.8×timeout, 30 s), Temporal shape) lives in
the sink"), :675 (W4 bar: "K6 chain re-measured in-plugin incl. SSE (replica caveat
discharged)"); plan.md:104 (K6 criterion: "≤500 ms p95 at 10 events/s with Temporal-throttle
coalescing engaged"); bench/spikes/k6/run-k6.ts:66-74 (replica flusher: setInterval(…, 100)
a fixed 100 ms latest-wins flush); results/results-spikes.md K6 table (steady-10evs: 92.0/93.9
ms p50/p95, PASS against the ≤500 ms bar); research.md:114-115 (source of the formula).

Failure scenario. With the default timeout 300 000 ms (task.ts:113), the normative formula
evaluates to min(240 s, 30 s) = 30 s. A latest-wins persistence throttle with a 30 s period
flushes at most once per 30 s; at the criterion's steady 10 ev/s, delivery latency p95 is ~30 s —
60× the ≤500 ms bar K6 is supposed to discharge. K6 "passed" only because the replica measured
a different parameter (100 ms flush) than the one the RFC ships (30 s). So one of the two is
wrong: if the formula is right, the W4 bar is unmeetable and the replica evidence is irrelevant to
it; if the replica is right, :454 (and research.md §6) misstate the shipped constant. The
Temporal analogy is misapplied: min(0.8×timeout, 30 s) is Temporal's workflow-execution update
throttle, not a progress-report cadence — 30 s coarseness also undercuts the observability
citizenship this RFC exists to deliver (a 5-minute task gets ≤10 progress updates). This is the
flagship D-12 retirement whose acceptance evidence does not cover its own normative parameter.

Fix direction. Pick one constant and make the evidence and the norm agree: state the actual
flush period (e.g., 100 ms-class, latest-wins) at :454, keep min(0.8×timeout, …) only if a
re-measured bar matches it, and restate the W4 bar against the same number. Text-only fix; the
replica harness already exists to re-run.


MAJOR-4 — Token lifecycle holes: no refresh path under unbounded task timeouts, and the attempt-fencing guarantee leans on a bootstrap channel that is not specified as attempt-fenced

Cites: RFC :551 (exp: z.int() — "TTL floor 600 s, default 3600 s"), :498-500
(expiresInS), route table :513 (GET /v1/credentials requires bootstrap token only),
:555-561 (verification steps 1–7 apply to capability checks), :566 (T2: env carries bootstrap;
per-dispatch attempt tokens), :536 (env carries NETSCRIPT_TASK_TOKEN (T0/T1) — so the
credentials-fetch consumer for T0/T1 is unspecified), :518 (async completion = detached handle),
packages/plugin-workers-core/src/domain/task.ts:113 (timeout: z.number().int().positive()
no upper bound); grep of the citizen contract surface shows no refresh/renew/rotate verb.

Failure scenario A (long task). Timeouts are any positive int: a 2-hour ETL task is legal.
Its attempt token expires at the 3600 s default exp; every later citizen call — progress, scoped
KV, and critically POST /v1/executions/:id/complete (the Temporal detached-handle path for async
completion, :518) — gets 401 from step 2. The protocol has a deadline-extension delivery
(pong.deadlineMs, :593) but no credential-extension delivery. Long-running T1 tasks lose
citizenship mid-flight with no recovery verb.

Failure scenario B (fencing bypass). The RFC's redelivery-fencing guarantee (:108,
:558-559, :659) rests on attempt-token invalidation. But the bootstrap token in the attempt's
env is not stated to be attempt-bound, single-use, or invalidated with the attempt token. If
GET /v1/credentials (:513) accepts the bootstrap token without verification step 3's
liveness/fencing check, a fenced stale attempt (zombie inside its SIGTERM grace window, or a
redelivered duplicate) can re-fetch credentials and continue mutating — defeating the fence. The
RFC simply never says which.

Fix direction. (A) Bind exp to deadlineMs + grace instead of a fixed TTL, or add a
refresh verb (the natural home is the existing ping/pong control channel — Temporal carries
credentials on heartbeats for exactly this reason), or state a normative max task duration =
token TTL. (B) One sentence: credential fetch is subject to the same liveness+fencing check
(step 3) and the bootstrap token is attempt-bound and invalidated with the attempt token.


MINOR-1 — Evidence presentation overstates precision: the "+0.41 ms" headline is statistically indistinguishable from zero, and a −13.9% noise delta is recorded as passing a ≤5% bar

Cites: RFC :39-40 ("the full T1 contract adds +0.41 ms"), :804;
results/results-spikes.md K4 table; recomputation from results/raw/k4_*-go_*.jsonl (n=300
measured per subject/series): queue_c1 exec-wall delta +0.43 ms, 95% CI ±0.50 (BASE 8.00±3.82,
T1 8.43±2.30) — the CI contains zero; direct_c1 −0.20 ±0.31; queue_c16 exec-wall −1.68 ±1.81;
queue_c16 e2e −20.6 ms ±6.0 (−12.5%) on sd ≈36–40 ms series — recorded as PASS against a
"≤5%" bar; host-side protoHostMs 0.063–0.097 ms p50 (tight, and the genuinely causal number).

Assessment. The conclusion does not flip — protocol cost is bounded by ~1 ms and more likely
~0.1 ms; the pre-registered bars were met. But the headline presents a within-noise point estimate
as the measured cost, and a negative delta larger than the bar itself (scheduling noise on a
300-sample, sd-38 ms e2e series) is treated as evidence of passing an overhead budget. At scale or
on other hardware the sign of these deltas would plausibly flip; only protoHostMs is robust.
Fix direction: restate as "host-side validate+demux 0.06–0.10 ms (tight); exec-wall delta
within noise, ≤ ~1 ms at 95% CI; e2e deltas at c=16 not statistically separable," and let the
tight host-side number carry the headline.

MINOR-2 — Normative edge table asserts redelivery fencing is resolved, while Unresolved Questions lists the same area as open; and tokens never fence the engine's own writes

Cites: RFC :659 ("Queue redelivery races a live attempt — attempt tokens fence … rejected at
verification step 3") vs :771 ("Grace-window semantics for fenced-out attempts (absolute
rejection vs bounded final diagnostic flush)" — unresolved); :555-561 (verification guards the
citizen surface, not stdout frames or engine-side KvExecutionState writes).

Failure scenario. During the SIGTERM grace window of attempt N (deadline ladder, :589-591),
the stale attempt's last frames and the engine's own completion write race attempt N+1's state.
Attempt tokens do not apply: frames on the task channel are unauthenticated by design, and the
engine writes execution state directly. The edge table's normative "resolved" and the unresolved
question contradict each other, and an implementer cannot tell whether a fenced attempt's final
result frame is honored, dropped, or logged. Fix direction: mark :659 resolved-pending-W5
or scope it precisely to citizen-surface mutations; move the grace-window decision out of
Unresolved into the edge table with a normative rule (recommend: absolute rejection + bounded
diagnostic, per the K8s analogue already cited).

MINOR-3 — No crash-window semantics between terminal-frame read and state persistence: results are lossy and dispatch has no idempotency story

Cites: RFC :571-586 (state machine; "Terminal-frame discipline"), :713 (Rationale admits
"Restate's all-frames model needs a durability journal T0/T1 lack"), :506 (idempotencyKey
exists only on citizen enqueue, not dispatch).

Failure scenario. Engine reads and accepts the terminal result frame, crashes before
KvExecutionState.complete() persists; on restart the record is still running/dispatched;
whatever redelivers re-runs the task; the frame (stdout, unjournaled) is gone. The RFC never
states dispatch is at-least-once, never asks tasks to be idempotent, and gives dispatch no
idempotency key. Side effects already performed by the task (citizen enqueues, KV writes — all
authenticated and durable) survive, so a silent double-run is observable downstream.
Fix direction: one normative paragraph: T0/T1 delivery is at-least-once across engine crashes;
the terminal frame is best-effort until persisted; tasks with side effects should use
idempotencyKey on citizen enqueue; consider persisting the accepted terminal frame's hash before
process teardown in W3.

MINOR-4 — Ambiguities a foreign-language SDK author can implement two incompatible ways

  1. Deadline vs buffered result ordering (:571-582): the state machine branches on events but
    gives no ordering rule when a valid result frame is already in the engine's read buffer at the
    instant the deadline fires (task finished in time, engine kills it anyway). Two engines, two
    outcomes (completed vs timeout). Rule needed: a terminal frame read before kill wins.
  2. T2 stale-attempt frame fencing (:598-611, :311-319): T2 multiplexes many attempts over
    one duplex channel; task→engine frames (TaskFrameV1) carry no attempt field — fencing relies
    implicitly on oRPC {i,t,p} correlation ids, which the normative schema section never pins.
    State the correlation-id ⇒ attempt binding explicitly.
  3. "Leases" undefined (:610: missed-ping "re-queues its leases") — the term appears exactly
    once with no definition in Terminology (:100-111).
  4. GET /v1/credentials consumer (:513 vs :536): T0/T1 already carry
    NETSCRIPT_TASK_TOKEN in env and T2 gets per-dispatch tokens — the route's lifecycle consumer
    is never identified (see MAJOR-4A; if it is the refresh path, say so and fence it).

OBSERVATION-1 — Package split judged against doctrine: compliant (attack failed)

The protocol core as pure contract + pure functions inside
packages/plugin-workers-core/src/protocol/ (:207-216), with deferred extraction of
@netscript/task-protocol until a second consumer exists (:725-729), matches
docs/architecture/doctrine/11-plugin-thinness-and-base-seams.md:17-34 (a primitive earns its own
core package only when a second plugin legitimately needs it) and mirrors the plugin-auth-core
ports-in-core precedent (06-archetypes.md:170). Shims as conformance-gated scaffold assets
rather than four published package trains is doctrine-safe. No finding.

OBSERVATION-2 — Evidence scale conditions noted, no flip risk identified beyond accepted caveats

K4/K5 ran on a 4-core container, 300 measured executions per series, concurrency ≤16; K1's
127 MB/s sentinel-scan is CPU-bound and the RFC sets no CPU budget for log-heavy tasks (at
1 GB/s log rates demux becomes a measurable fraction of a core — still cheap, worth one sentence
in Drawbacks). K6-replica, fd-3, UDS, Docker/Aspire/Windows loopback caveats are already accepted
(R5-D-2/3/4/5) and were not re-reported; the protocol-cost conclusion itself rests on the tight
host-side number and would not plausibly flip on other hardware.

OBSERVATION-3 — Completeness probe: the citizenship surface meets the owner's ask for v1

Against "actually part of the ecosystem, not just foreign-language runners": enqueue-from-task,
scoped KV, stream publish, progress-to-SSE convergence with JS jobs, structured errors, trace
continuation through the queue, async completion, and status read are all specified with routes,
capabilities, and schemas. Saga park/child verbs, dynamic capability registration, and signed
tokens are explicitly reserved (:779-792), and the per-language depth is deferred to the
owner-directed citizenship addenda — the right sequencing. The completeness risk is not missing
surface; it is the four MAJOR holes above, each of which would make a real citizen's first week
fail (large result, retry expectation, progress cadence, long-task credentials).


Summary

Adversarial third pass over RFC-5 (head 4242a46, RFC byte-identical to 02b1c6e). Read-only;
no source or deno.lock mutated; no PR mutation. The architecture survives: tiers, two-surface
split, sentinel framing, terminal discipline, error model, and the auth-blueprint port placement
all held under concrete attack, and every formal-gate fix (f4ae089, c45d6c1, 02b1c6e) is
verifiable in the commit trail. Four MAJOR spec-level holes remain: (1) terminal result frames
over ~4 KB are unrepresentable and demote to unknown-failure (common case, T0-inversion); (2)
the RETRY/attempt/checkpoint vocabulary has no executor in the normative five-wave plan and W2's
D-5 retirement is over-claimed; (3) the normative progress throttle min(0.8×timeout, 30 s)
contradicts the K6 spike that validates it (100 ms flush) and makes the W4 bar unmeetable; (4)
token TTL is fixed while task timeouts are unbounded (no refresh verb), and credential-fetch
fencing is unspecified. Four MINOR findings: statistical overstatement of the +0.41 ms headline
(recomputed CIs contain zero), edge-table/unresolved-question contradiction on redelivery fencing,
missing crash-window delivery semantics, and four SDK-author ambiguities. The package split is
doctrine-compliant. This closes the R5-D-8 IMPL-EVAL two-failure escalation: eval-loop mechanics
are sound, but this independent pass found substantive findings the formal cycles did not surface.

Changes

None — read-only adversarial review. Deliverable written to this summary and mirrored to the
run-dir trace (adversarial-review.md under
.llm/tmp/run/openhands/pr-1687/run-32398533310-1/).

Validation

  • Verified head/base: git diff 02b1c6e 4242a46 touches only drift.md (RFC identical at tip);
    merge-base of 9634735 and 4242a46 is 9634735.
  • All line cites verified against the working tree at head (sed -n/grep -n on
    rfcs/0000-polyglot-task-protocol.md).
  • K4 statistics recomputed from raw JSONL (results/raw/k4_*-go_*.jsonl, n=300 measured/series)
    with proper pooled 95% CIs.
  • K6 replica flush constant re-derived from bench/spikes/k6/run-k6.ts:66-74.
  • Retry-loop absence confirmed against live source (job-dispatcher.ts:98 carries maxRetries
    but no loop consumes it; consistent with audit D-5).

Remaining risks

  • If the four MAJOR findings are accepted as text fixes, none alter the architecture; all are
    spec-text repairs. The largest residual risk is MAJOR-1: if large results are routed
    artifacts-only, T1's zero-infrastructure promise needs explicit scoping language.
  • MINOR-1: the protocol-cost headline survives correction but should not be cited more precisely
    than the host-side 0.06–0.10 ms number.
  • No risks to deno.lock or source: nothing was mutated.

ADVERSARIAL_VERDICT: CONCERNS
OPENHANDS_VERDICT: FAIL_FIX

Run: https://github.com/rickylabs/netscript/actions/runs/32398533310

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/qwen/qwen3.8-max output=pr-comment iterations=800

use harness

RE-DISPATCH NOTE: attempt 1 (run 32398533310) stalled mid-analysis with no log output for 100+ minutes and was abandoned; this is a fresh attempt with an identical brief. If that run ever posts a late verdict, THIS run's verdict is the one of record.

SKILL

  • netscript-harness — run-artifact map and evaluator-independence rules; this is an owner-commissioned adversarial pass, distinct from the formal PLAN/IMPL-EVAL lanes.
  • netscript-doctrine — judge the proposed port/adapter package split against package/plugin doctrine.
  • netscript-tools — repository-native read-only evidence commands; do not mutate source or deno.lock.
  • netscript-pr — PR conventions context for reading the phase-comment trail.
  • netscript-deno-toolchain — for any dependency/API-surface verification you need (deno doc, deps wrappers).

ROLE

Act as the OWNER-COMMISSIONED ADVERSARIAL REVIEWER for RFC-5 on PR #1687, at HIGH reasoning effort throughout. This is a third pass ON TOP of the formal gates — do not re-run their checklists, and do not re-discover their findings: PLAN-EVAL cycle-2 PASS (run 32343592955); IMPL-EVAL cycle-1 PASS on the pre-revision head (superseded by the S10 redesign), cycle-2 FAIL_FIX (4 doc fixes, applied in c45d6c1), cycle-3 FAIL_FIX (1 doc fix — defect-retirement claim — applied in 02b1c6e). No design finding has survived any cycle; every applied fix is verifiable in the commit trail. This pass also serves as the closing verification for the IMPL-EVAL two-failure escalation (owner ruling in drift R5-D-8). Your job is to BREAK the design and the argument. Read-only: do not edit files, commit, push, or repair findings; never mutate deno.lock. Review head: 4242a46 (RFC content identical to 02b1c6e; the tip adds only drift records) on branch claude/harness-profile-rfc-benchmark-shzhgv, base 9634735.

CONTEXT

Run 5 of the polyglot RFC series (#1678 scriptc, #1683 rust, #1685 dotnet, #1686 golang — merged execution-layer runs). RFC-5 specifies the NetScript Task Protocol (NTP): ecosystem citizenship for polyglot tasks — interoperability, observability, communication layer, error & lifecycle management.

Read in order:

  1. rfcs/0000-polyglot-task-protocol.md — THE deliverable (803-line architectural design: Zod wire schemas, TS ports on the auth-core blueprint, package map, per-seam engine integration table, five-wave plan).
  2. .llm/runs/claude-harness-profile-rfc-benchmark-shzhgv--polyglot-protocol-rfc/plan.md (locks L1–L9, spike criteria), research.md (+ UNVERIFIED register §7), research-sources/ (32-file corpus incl. netscript-engine-audit.md, defect register D-1..D-14 at file:line), results/results-spikes.md + results/raw/k*.jsonl, bench/spikes/, drift.md (R5-D-1..8), plan-eval.md, evaluate.md, worklog.md.
  3. PR docs(rfc): RFC-5 polyglot task protocol — ecosystem citizenship for foreign-language tasks #1687 body + phase comments (the commit trail).

Core decisions to attack: three conformance tiers computed by a conformance suite (T0 legacy-forever / T1 structured one-shot / T2 duplex worker); two surfaces (attempt-bound task-channel verbs vs authenticated loopback oRPC citizen surface, TCP 127.0.0.1, per-attempt opaque capability tokens invalidated on retry); sentinel-NDJSON stdout framing (\x00NSF\x00, single-write ≤ PIPE_BUF, byte-stream sentinel-scan demux — never line-anchored); closed versioned Zod envelope in TASK_PAYLOAD; structured errors {errtype, behavior: RETRY|PAUSE|FAIL} + terminal-frame discipline (exit-without-terminal ⇒ synthesized unknown-failure); reserved NETSCRIPT_* env + constructed allowlisted env; withTaskProtocol decorator integration (no adapter rewrite); auth-blueprint package split; five-wave staged plan.

ACCEPTED caveats — do NOT report as discoveries, but DO attack whether accepting them was sound: K6 measured on a replica (R5-D-3); fd-3 infeasible on the Deno host (R5-D-2); UDS demoted (R5-D-4); Docker/Aspire/Windows loopback survival untested (R5-D-5); sentinel-forgery residual (RFC Drawbacks #2); engine defects D-1..D-14 deliberately not fixed in this docs-only PR; D-10/D-11 deliberately outside the five-wave retirement.

ATTACK SURFACE (in priority order)

  1. Wire-protocol soundness: races, ambiguous states, unrepresentable outcomes in the frame grammar, terminal discipline, cancellation ladder, deadline semantics, T2 handshake. Construct CONCRETE failure scenarios (interleavings, crash-mid-frame, duplicate results, redelivery races, clock skew).
  2. Security model: token lifetime/rotation holes, loopback SSRF/confused-deputy angles, the same-uid env-delivery trust argument, capability-scoping gaps, whether per-dispatch T2 tokens actually fence stale attempts under queue redelivery.
  3. Evidence validity: does any spike measure something other than what its conclusion claims (4-core container, short workloads, replica boundaries, throttle-dominated K6, K4 noise-negative deltas)? Would a conclusion plausibly flip at scale or on other hardware/OS?
  4. Spec ambiguity: anything a Rust/Go/C#/Python SDK author could implement two incompatible ways from the RFC text alone.
  5. Migration realism: can the five waves land without breaking T0 guarantees or the fix(workers): documented web-worker/subprocess runner modes are unimplemented — docs truthing or Tier-3 pool implementation #1684 pool plans? Anything load-bearing hidden in "follow-up waves"?
  6. Completeness probe: what did this run treat as out of scope that an owner who asked for tasks to be "actually part of the ecosystem, not just foreign language runners" would consider the point?

OUTPUT

Write the full deliverable to OPENHANDS_SUMMARY_PATH (and the run-dir trace per the handoff contract): severity-ranked findings (BLOCKER / MAJOR / MINOR / OBSERVATION), each with an exact file/section cite, a concrete failure scenario or ambiguity demonstration (not vibes), and a minimal fix direction. End with BOTH verdict lines, exactly once each:
ADVERSARIAL_VERDICT: CLEAR | CONCERNS | BLOCK (your real verdict), and the machine mapping
OPENHANDS_VERDICT: PASS (if CLEAR) or OPENHANDS_VERDICT: FAIL_FIX (if CONCERNS or BLOCK).
Do not resolve threads, post extra comments, or modify the PR.


Generated by Claude Code

…h (R5-D-8)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
@rickylabs

rickylabs commented Aug 20, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

OPENHANDS_VERDICT: FAIL_FIX

Model: openrouter/qwen/qwen3.8-max
Provider: OPENROUTER
Job status: success
Verdict source: summary-file
Summary source: agent
Housekeeping: ack=success trace=success commit-artifacts=success replies=skipped draft-pr=skipped
Trace: .llm/tmp/run/openhands/pr-1687/run-32416122194-1 (in the Actions artifact)
Artifact: openhands-agent-32416122194-1

RFC-5 (PR #1687) - Owner-commissioned adversarial pass (attempt 2, run 32416122194)

Role: owner-commissioned adversarial reviewer, third pass ON TOP of the formal gates; closing
verification of the IMPL-EVAL two-failure escalation (drift R5-D-8).
Reviewed head: 4242a46 (RFC content identical to 02b1c6e; tip adds drift records only),
branch claude/harness-profile-rfc-benchmark-shzhgv, base 9634735.
Mode: read-only. No source, RFC, or deno.lock mutation; no commits; no thread resolution.
Per the re-dispatch note, this run's verdict is the one of record (attempt 1, run 32398533310,
stalled mid-analysis).

Bottom line: the architecture survives the attack. Tiers, two surfaces, sentinel framing,
terminal discipline, and the auth-blueprint port split are sound, and the evidence discipline
(pre-registered spikes, file:line defect register, drift ledger) is the strongest part of the run.
But the pass found 0 BLOCKER / 6 MAJOR / 7 MINOR / 6 OBSERVATION issues - normative text that
two independent SDK/host authors would implement incompatibly, one internal contradiction in the
frame grammar, one security-model ambiguity that can void the entire capability scheme, and one
load-bearing component (the retry/redelivery driver) that no implementation wave owns while the
lifecycle section legislates its behavior. All are fixable in RFC text. That is a CONCERNS-tier
result, not CLEAR and not BLOCK.

Not re-reported: PLAN-EVAL cycle-2 PASS (run 32343592955) and all IMPL-EVAL findings (cycle-2
F1-F4, cycle-3 F-1) - those were doc fixes, each verified applied at the reviewed head (see
escalation section). No design finding survived any formal cycle; every finding below is new.


IMPL-EVAL two-failure escalation - closing verification (brief-mandated)

  • Cycle-2 FAIL_FIX (head 45b134a, fixes in c45d6c1): F1 32-file corpus count (RFC lines 37,
    749: "32-file ratified corpus" - matches research-sources/), F2 resume point (context-pack
    records S10 + eval history), F3 PR body rewritten for S10 head, F4 full-path seam cite (RFC line
    462 now cites packages/plugin-workers-core/src/executor/multi-runtime-task-executor.ts:195-205).
    All four verified at head.
  • Cycle-3 FAIL_FIX (fix in 02b1c6e): Summary lines 32-34 now read "D-1..D-9, D-12, D-13, and
    D-14" with the explicit "deliberately outside the five-wave retirement" note for D-10/D-11
    (lines 678-681) - consistent with the wave table (lines 670-676) and seam table. Verified.
  • Closure: the escalation closes with this pass. Residual: finding A1 below - cycle-3 fixed
    the retirement sentence, but the omission it pointed at (D-5's retry loop has no owner) remains.

MAJOR findings

A1 - The retry/redelivery driver is load-bearing and owned by no wave (migration realism)

Cites: RFC lifecycle section lines 571-588 (result{error,behavior:RETRY} -> retrying -> new attempt, "Retry is decided by the engine"); envelope retry block lines 267-271; invalidateAttempt
port line 377; attempt-token rotation lines 108, 499, 559; checkpoint redelivery line 316; wave table
lines 670-676; audit D-5 (research-sources/netscript-engine-audit.md:212: "maxRetries is stored
but no dispatcher retry loop exists").

The hole: the RFC specifies an attempt state machine whose RETRY, PAUSE, checkpoint-redelivery,
and token-rotation semantics all require an engine-side retry/redelivery driver - and none of W1-W5
builds one. W2 "Retires D-5" via "options threading (D-1/D-2/D-5 seams)" (line 673), but threading
attempt fixes only the propagation half of D-5. The audit's defect statement includes the missing
loop itself. Today maxRetries defaults to 1 (packages/plugin-workers-core/src/domain/task.ts:114)
and drives nothing; nothing in the wave plan changes that.

Concrete scenario: W2-W3 land. A Go task emits
result{outcome:'error', error:{errtype:'Net.Transient', behavior:'RETRY', nextRetryDelayMs:5000}}.
Per line 576 the engine must transition retrying -> new attempt. No component exists to: consume the
frame's behavior, decrement retry.remaining, mint attempt N+1's token, call invalidateAttempt,
or re-dispatch after nextRetryDelayMs. behavior:RETRY is observably identical to FAIL;
retry.checkpoint "persisted for redelivery when 'cancelled'" (line 316) is written and never read;
paused executions (line 578, "park for operator") have no resume verb anywhere (the citizen route
table lines 511-519 has none); the attempt token "rotates on retry" (line 499) never rotates, so the
entire fencing story of A2/A5 rests on an event that cannot occur. Two implementations diverge
further: one host wires RETRY to its queue's native redelivery, another invents an in-process loop -
conformance cases cannot reconcile them because the driver is not in the tier authority either.

Fix direction: either (a) add an explicit wave/scope row owning the retry driver (behavior
consumption, attempt increment, invalidateAttempt call sites, nextRetryDelayMs delay, paused->
resume verb, checkpoint consumption), or (b) descope retry semantics: mark the lifecycle RETRY/PAUSE
branches and envelope retry block reserved (extension model item 4) and say D-5 retirement in W2 is
propagation-only. Silence is the only wrong answer.

A2 - Redelivery identity semantics unspecified; token fencing does not cover the case it promises (wire + security)

Cites: RFC edge-case row line 659 ("Queue redelivery races a live attempt | attempt tokens
fence: the stale attempt's mutations are rejected at verification step 3"); verification step 3 lines
558-559; async-completion route line 518; source: plugins/workers/worker/job-dispatcher.ts:192-245
(every delivery calls executionState.create() -> fresh executionId); idempotency claim lines
210-221 (pending claim -> claimed:false -> skip); packages/queue/adapters/kv-polling.adapter.ts:222
(visibilityTimeout default 30 000 ms) and its recovery loop at visibilityTimeout/2;
packages/plugin-workers-core/src/state/execution-state.ts #transition (read-modify-write, no
conditional guard).

The hole: the RFC never says whether a redelivered message reuses the execution record
(bumping attempt) or creates a new one. Today it creates a new one. Fencing step 3 checks that
(executionId, attempt) is the current running attempt of that execution - so if redelivery mints a
new execution, old and new attempts are each "current" in their own record and both tokens verify.

Concrete scenario: task A runs 40 s under the default 300 s timeout. The KV visibility timeout is
30 s (kv-polling.adapter.ts:222); the recovery loop redelivers at ~30 s while A is alive. Worker B
receives the redelivery, creates execution E2, mints token T2. A (execution E1, token T1) is still
running. Both pass verification step 3 (each is the current attempt of its own execution). Both tasks
execute side effects - both POST /v1/enqueue children, both write scoped KV - exactly the duplicate
work the edge-case row promises is fenced. (Today the idempotency claim happens to absorb this -
pending claim -> skip - but the RFC's normative story never mentions the claim, and W2's threading
rewrite of processWorkerTask can break that accident without the spec noticing.)
Second gap, same root: the RFC defines two terminal channels - the in-band result frame
(first-wins per line 655) and POST /v1/executions/:id/complete|fail (line 518) - with no ordering
rule between them. Scenario: task posts complete(ok) at 99% and its buffered result{error} frame
arrives a beat later (or vice versa); step 3 only fences after the state leaves running, and
#transition is an unguarded read-modify-write, so in the TOCTOU window the last writer wins and the
recorded outcome is a coin flip.

Fix direction: normative edge rows: (1) redelivery identity - reuse executionId, increment
attempt, invalidate the prior token; state how the idempotency claim interacts (pending claim ->
skip vs re-attempt); (2) first-terminal-wins across both channels, with the completion API
rejected once any terminal is recorded, and require an atomic/CAS state transition in the
execution-state seam row (line 454) instead of today's merge-write.

A3 - Bootstrap-token lifetime and caller binding unspecified -> confused-deputy (security)

Cites: RFC line 457 ("bound to 127.0.0.1: with a per-boot SSRF bearer (spike K3
pattern)"); reserved env line 535 (NETSCRIPT_BOOT_TOKEN delivered per task); credentials route line
513 ("GET /v1/credentials ... bootstrap token ... env carries the pointer + bootstrap token; real
material stays server-side"); verification algorithm lines 543-563 (note step 6: "mutating verbs
accept the token only" - the credentials GET is not governed by steps 1-7); K3 results
(results-spikes.md K3 table: "bootstrap->attempt token flow: true" - same-task only, no cross-task
probe).

The hole: the RFC contradicts itself on bootstrap scope ("per-boot" line 457 vs per-task env
delivery line 535) and never specifies that the credentials exchange is bound to the presenting
execution
. The exchange is the one verb not covered by the seven-step verification.

Concrete scenario (per-boot reading): task A is declared with capabilities(['kv:reports/']);
task B runs on the same host with enqueue:billing and KV write scopes. A reads the shared per-boot
bootstrap token from its own env, calls GET /v1/credentials, and - since the server has no
specified mechanism to learn which execution is asking - obtains (or negotiates) B-class attempt
tokens. A now enqueues billing work under B's capabilities. The capability model is void.
Even under the per-task reading, the binding must be normative, because the accepted same-uid env
read (Rationale, lines 720-722) means any same-uid process holding one task's bootstrap
pointer must get only that task's record.

Fix direction: one sentence, normative: bootstrap tokens are per-execution, and
GET /v1/credentials resolves exclusively to the presenting execution's (executionId, attempt) -
server-side, no client-supplied identity accepted. Add a conformance case
(t1.citizen.cross_task_credentials_rejected). Reconcile line 457's "per-boot" wording.

A4 - Frame size limits are transport-dependent but legislated universally; envelope size is unbounded (frame grammar)

Cites: RFC line 105 (Frame = single write(2) <= PIPE_BUF), lines 343-344
(FRAME_MAX_BYTES = 4096; "a frame MUST be one write(2) <= this budget"); line 595 ("8-256 KB via
inbound frame (T2)", "> 256 KB artifacts-only (T-5 caps)"); line 607 (T2 dispatch = "an envelope
frame with a fresh attempt token"); envelope payload: z.unknown() line 272 - no size bound
anywhere
(grep for payload-size/env-limit/ARG_MAX across the RFC: zero hits); reserved env delivery
line 534 (NETSCRIPT_PAYLOAD carries the whole envelope).

The hole, in three clauses: (1) T2 checkpoint routing "8-256 KB via inbound frame" contradicts
the 4 096-byte frame MUST - the RFC implies transport-dependent limits but states a universal one.
(2) A T2 dispatch envelope with a > 4 KB payload violates the frame MUST as written. (3) T0/T1
deliver the envelope via env var with no cap: Linux execve caps a single string at
MAX_ARG_STRLEN (128 KiB) and Windows CreateProcess caps the entire environment block at
32 767 chars - the Aspire/Windows targets named in R5-D-5 make the Windows number load-bearing.

Concrete scenario: a C# SDK author targets Windows and caps payloads at 24 KB; a Rust author
rejects at 128 KB on Linux; a Go author streams 1 MB payloads through NETSCRIPT_PAYLOAD on Linux
until exec fails non-deterministically (arg+env total is stack/4 on Linux). Three incompatible SDKs,
all RFC-conformant. Meanwhile a T2 host implementing the 4 096 MUST literally cannot receive a
dispatch envelope carrying the same 64 KB payload that T1 delivered fine - an unrepresentable
outcome in the frame grammar, which the brief flags as a first-class attack.

Fix direction: define (a) an envelope size budget + spill rule (payloads over budget -> artifact
ref in envelope, mirroring the checkpoint tiers at line 595), (b) per-transport frame limits (stdio:
PIPE_BUF/4 096; peer channel: explicit larger bound), and (c) an OS env-bound conformance case. The
checkpoint-tier machinery already exists; extend it to payloads.

A5 - resultSchema validation failure has no semantics (spec ambiguity)

Cites: RFC line 313 (value ... validated against resultSchema when 'ok'); line 451 (T0 fallback
parseJsonLastLine "Zod-checked"); line 662 (the payload analogue is specified: "dispatch
fails before spawn with Protocol.BadEnvelope (behavior: FAIL)"); edge-case table lines 652-663 -
no resultSchema row (verified by inspection of every row).

The hole: the RFC specifies the failure semantics of payload validation (authoring error, FAIL,
pre-spawn) and is silent on result validation - the mirror case at the other boundary.

Concrete scenario: task declares resultSchema = z.object({url: z.string().url()}), emits
result{outcome:'ok', value:{url: 42}}. Host X treats it as a protocol failure and records failed;
host Y retries it (wasted work, same output forever - an infinite retry loop once A1 lands); host Z
passes the raw value through (typed consumers crash downstream); host D synthesizes
Protocol.MalformedFrame. All four are RFC-conformant today. The same hole covers the T0 fallback
path (line 451): a legacy task's last JSON line failing a newly added resultSchema - does a T0 task
that was conformant yesterday fail today when the definition gains a schema? That interacts with
"None at T0" (line 737).

Fix direction: one edge-case row: result-validation failure -> attempt ends failed with
Protocol.BadResult (new errtype), behavior: FAIL (authoring error, never retry-eligible - same
philosophy as line 662), raw value kept in resultRaw like the fallback parse. State explicitly that
adding a resultSchema to an existing definition cannot retroactively fail schema-less legacy
output (validation applies only when declared).

A6 - Normative progress throttle contradicts the spike it cites and the W4 acceptance bar (evidence validity + spec)

Cites: RFC line 454 ("Latest-wins throttle (min(0.8x timeout, 30 s), Temporal shape) lives in
the sink"); K6 spike implementation (.llm/runs/.../bench/spikes/k6/run-k6.ts: "throttle: latest-wins
flush every 100 ms"); K6 results (results-spikes.md: steady 93.9 ms p95 at 10 ev/s, bar <= 500 ms
PASS; burst 9.52x coalescing); wave table W4 acceptance (line 675): "K6 chain re-measured in-plugin
incl. SSE
(replica caveat discharged)".

The hole: min(0.8x timeout, 30 s) evaluates to a 30 s flush interval at the default 300 s
timeout (task.ts:113), 24 s for a 30 s task, 8 s for a 10 s task. K6 measured a 100 ms interval.
Under the normative formula, steady-state progress persistence latency is about the flush interval,
up to 30 s p95 - sixty times over K6's <= 500 ms bar, which W4 promises to reproduce in-plugin. The
W4 acceptance bar is structurally unmeetable as specified, and progress would be useless for every
task shorter than ~37 s. This reads like an inverted formula (a Temporal-style heartbeat cap, or
0.8x elapsed, or simply the measured 100 ms constant was intended).

Concrete scenario: W4 implements line 454 literally; the in-plugin K6 re-measurement reports
~30 000 ms p95 steady; the wave's own acceptance gate fails and the implementer must either amend the
RFC mid-wave or silently deviate from normative text - exactly the drift the harness exists to catch.

Fix direction: correct line 454 to the measured design (state the flush interval constant, e.g.
100 ms latest-wins, with the throttle cap - if that was the intent - stated separately and bounded
well under the 500 ms bar), and cite K6's interval in the same row so spec and evidence are co-located.


MINOR findings

M1 - Cancellation ladder gaps (grace defaults, kill timing, drain-after-kill). Lines 589-591,
327, 580. cancel.graceMs is optional with no default; the ladder "cancel frame -> SIGTERM at
graceMs -> SIGKILL" never states the SIGTERM->SIGKILL interval or whether the deadline path (line
580) reuses it. Drain rule missing: if the engine kills at deadline before draining buffered
stdout, a terminal frame already written to the pipe (atomic, <= PIPE_BUF, but unread) is discarded
and a successful result becomes a synthesized timeout/unknown-failure. Fix: default graceMs,
explicit second interval, and "engine MUST drain the task channel to EOF before finalizing any
kill-initiated outcome".

M2 - Duplicate-terminal vocabulary mismatch. Line 655: duplicate result frames log
Protocol.TerminalMissing - but a duplicate is not a missing terminal; the constant vocabulary
(lines 413-417) has no duplicate member. Errtype constants feed per-errtype disposition hooks
(line 587) and foreign codegen, so two SDKs will diverge (one reuses the wrong constant, one invents
Protocol.TerminalDuplicate). Fix: add the constant or state the reuse is deliberate.

M3 - T2 init silence: demotion vs zombie. Lines 603-606 ("no echo -> T0/T1 detection") vs lines
610-612 (zombie rule: "fail init must exit, be killed, never pooled"). Is a silent peer demoted
or killed? No detection timeout is given, and dispatch semantics during detection are undefined -
on a duplex channel, "T0/T1 detection" cannot even reuse stdin framing without a stated downgrade
path. Two hosts: one waits forever (pool slot wedged), one kills at 5 s (flapping). Fix: pick
one, give the timeout, define in-flight dispatch handling.

M4 - Conformance exclusion authority undefined. Lines 630-631: "Tier N achieved iff all
non-excluded t0..tN.* cases pass" - but nothing defines who excludes, on what evidence. As
written, a testee can self-exclude hard cases and compute itself T2. Fix: exclusions derive only
from observed capability absence in the generated matrix (declaration-vs-probed-vs-observed already
exists at line 630), never testee assertion.

M5 - Invalidation trigger list incomplete (lease requeue). Lines 608-609: missed-ping budget
"re-queues its leases"; lines 108/499/559 say tokens are "invalidated on retry". A lease-expiry
requeue is not a behavior:RETRY - if invalidation keys only on retry, a GC-paused T2 worker that
wakes after 61 s holds a still-valid token while its replacement runs: two completions race (A2's
TOCTOU, T2 edition). Fix: enumerate invalidation triggers normatively: RETRY, lease requeue,
deadline kill, cancel.

M6 - Attempt arithmetic undefined. Lines 262, 550 (z.int().min(0)), audit D-5 (hardcoded 0).
First attempt = 0 or 1? Does redelivery-without-retry bump attempt? Is retry.remaining maxRetries
or maxRetries+1? Fencing math (step 3) and envelope accounting depend on it. Fix: one normative
sentence: attempts are 0-based, bump on every dispatch of the same logical execution.

M7 - "Closed envelope" vs open-world MUST. Summary line 22 ("closed, versioned envelope")
vs lines 245-249 (wire objects MUST be .loose(), unknown members preserved, never fatal) and line
258 ("Closed vocabulary" - the intended meaning). A Rust/C# author implementing strict-object
parsing from the Summary violates the normative MUST. Fix: replace "closed envelope" with
"closed-vocabulary envelope" in the Summary and Guide.


OBSERVATIONS (including accepted-caveat soundness attacks)

O1 - K4 evidence: verdict stands, presentation overclaims at c=16. Negative deltas
(direct -0.35 ms; queue c16 exec-wall 13.29 -> 11.21 ms; e2e c16 -13.9%) mean per-run variance in
that regime exceeds some measured effects - the protocol did not make concurrency 14% faster; that
is noise. The <= 1 ms headline rests on c=1 p50 (n=300, no confidence interval recorded), short
workload (100k), one container, one OS. The verdict survives because the host-side measurement
(0.06-0.10 ms validate+demux) independently bounds the protocol's share and the K4 criterion
(c=1, <= 1.0 ms) passed 2.4x inside bar. Would it flip at scale/hardware? The host-side cost is
CPU-bound parsing - no; the exec-wall delta at high concurrency could exceed 1 ms under different
scheduling, and the RFC should say the bar is a c=1 statement. Recommendation: stop citing the c16
delta as an improvement; note c=1 scope beside the bar.

O2 - Accepted caveats were soundly accepted. K6-on-replica (R5-D-3): sound conditional on A6 -
W4's in-plugin re-measurement discharges it only if the throttle formula is fixed. fd-3 (R5-D-2):
sound - Deno.Command API fact, escape hatch exists. UDS demotion (R5-D-4): sound, two independent
measurements. Docker/Aspire/Windows loopback untested (R5-D-5): acceptable as a design argument
(Aspire injects service URLs via env - the same mechanism NTP uses), but A4's Windows env-block
number sharpens it: loopback survival and envelope delivery both need a Windows conformance case
in W2. Sentinel-forgery residual (Drawbacks #2): acceptance defensible - forgery requires control of
the task's own stdout (same trust domain); worst case is a forged result{ok} flipping a failing
task, and T2/socket is the documented escape hatch. Note it is an integrity residual, not merely
noise - worth saying so in Drawbacks.

O3 - "None at T0" vs the env allowlist. Lines 737-745 admit the allowlist is "the one
behavioral change tasks could observe" with the envPassthrough hatch - acceptance sound (it retires
D-9, a secret-leak defect, behind a deprecation window). The internal tension is presentational:
Goal 2 (line 82, "every existing task remains valid forever") and the conformance case
t0.env.no_supervisor_inheritance (line 621) encode different vintages of T0. Add the
envPassthrough footnote to Goal 2.

O4 - Doctrine judgment on the package split: COMPLIANT. The in-core protocol/ module with
deferred extraction (lines 723-727) follows the plugin-auth-core ports-in-core precedent; exactly
one consumer exists, so a @netscript/task-protocol package would be premature extraction against
doctrine. The import-clean constraint (no I/O, no plugin imports) is the right mechanical guarantee.
contracts/v1 through assemblePluginContractRouter (line 457) is the sanctioned plugin seam. One
watch-item: the conformance suite ships inside packages/plugin-workers-core/src/testing/ - W1 must
confirm arch:check tolerates test infrastructure in a published package (auth-core precedent keeps
testing utilities out of the public surface).

O5 - Completeness probe: no FAIL_RESCOPE. The owner's ask - tasks "actually part of the
ecosystem, not just foreign language runners" - is answered by the citizen surface (enqueue, scoped
KV, streams, status, artifacts, async completion) + trace context + progress + structured errors,
reachable from any language with stdlib HTTP/NDJSON. Declaration stays TS-side (defineTask) by
design, consistent with the series' executor-seam precedent; per-language SDKs are the named
citizenship addenda (RFCs 1-4 follow-up). Deliberate out-of-scope register items 1-10 hold.

O6 - Deadline/clock model is sound. Line 660's resolution (host-authored, host-enforced,
advisory to tasks) removes clock skew as a failure source - single-clock enforcement means skew
cannot extend a lease. No finding.


What I could NOT break

  • Terminal discipline (exactly-one result, exit-without-terminal -> synthesized unknown-failure,
    frame-wins-over-exit-code): coherent, Restate-precedented, crash-mid-frame covered (line 654).
  • Sentinel demux rule: byte-stream scan mandated with measured justification (K1 line-anchored
    loss 8-44/200); PIPE_BUF atomicity closes the interleaving race for compliant writers;
    partial-sentinel EOF handled (line 654).
  • Per-attempt tokens with server-side verification order (steps 1-7): K8s-TokenReview-shaped,
    all-must-pass, no identification fallback - sound modulo A3's binding gap and M5's trigger list.
  • Tier computation by conformance suite (never asserted): the right move; modulo M4's exclusion
    authority it resists tier inflation.
  • The +0.41 ms cost case: bounded by two independent measurements (K4 c=1 delta + in-path
    protoHost 0.06-0.10 ms), through the real dispatch path, 1 920/1 920 executions.

Fix priority

A3 (security hole) > A1 (unimplementable lifecycle) > A4 (unrepresentable payloads) > A2 (races)

A5 (ambiguity) > A6 (self-contradicting constant) > M1-M7 (text fixes). All are RFC-text edits;
none require re-running spikes.


Validation performed this pass (read-only)

  • RFC re-read in full at head 4242a46 (803 lines); all findings cite verified line numbers.
  • Cross-checked every RFC claim against source: job-dispatcher.ts (dispatch/idempotency/execution
    creation), kv-polling.adapter.ts (visibility timeout 30 s default, recovery loop), execution-state.ts
    (#transition read-modify-write; status vocabulary), executor-types.ts (TaskExecutionOptions
    fields), runtime-types.ts (TaskMessage fields - no attempt/executionId/deadlineMs),
    domain/task.ts (timeout default 300 000, maxRetries default 1), dax-process-runner.ts seams.
  • Spike evidence re-read: results-spikes.md K1-K6, k6/run-k6.ts throttle source, drift.md R5-D-1..8,
    plan.md L1-L9 + out-of-scope register, evaluate.md cycles 1-3, netscript-engine-audit.md D-1..D-14.
  • No source, RFC, or deno.lock modified; git status clean except this run's own artifacts.

ADVERSARIAL_VERDICT: CONCERNS
OPENHANDS_VERDICT: FAIL_FIX

Run: https://github.com/rickylabs/netscript/actions/runs/32416122194

claude added 2 commits August 20, 2026 22:01
…-1 stall-diagnosis correction

Attempt 1 (qwen3.8-max, run 32398533310) completed after 4h21m with the
full review on head 4242a46; 4 MAJOR + 4 MINOR findings, all MAJORs
supervisor-verified against RFC/spike sources. Attempt-2 of-record clause
superseded; disposition returned to owner with S11 proposal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
…nsolidated S11 scope

Attempt 2 (run 32416122194) completed uncancelled with CONCERNS/FAIL_FIX;
of-record status unchanged (attempt 1). Novel findings A2-A5, M1-M7 and
three observation notes folded into the proposed S11 scope; A1/A6 converge
with the of-record MAJOR-2/MAJOR-3.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H2FUAx1v6BbP6PgLTNqH5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:docs area:plugins plugins/* and plugin-core packages ci:skip-e2e ci:skip-scaffold Skip the scaffold-static (deno-only) scaffold gate rfc Request for Comments - substantial/breaking design change status:impl type:docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants