Skip to content

fix(mcp): make truncation and receipt evidence honest #1183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rickylabs merged 5 commits into from
Aug 3, 2026
Merged

fix(mcp): make truncation and receipt evidence honest #1183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5192b07
plan(mcp): lock receipt and truncation correctness gates
rickylabs Aug 3, 2026
0ade673
gate(mcp): approve receipt and truncation plan
rickylabs Aug 3, 2026
f00d633
fix(mcp): settle receipts after output validation
rickylabs Aug 3, 2026
8a03149
fix(mcp): report bounded output honestly
rickylabs Aug 3, 2026
3d328c5
gate(mcp): record implementation evaluation
rickylabs Aug 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .llm/runs/fix-mcp-truncation-receipt-ordering--s8/context-pack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Context Pack: OMB S8 existing-machinery correctness fixes

## Run Metadata

| Field | Value |
| --- | --- |
| Run ID | `fix-mcp-truncation-receipt-ordering--s8` |
| Branch | `fix/mcp-truncation-receipt-ordering` |
| Current phase | `implement` |
| Archetype | `2 - Integration` |
| Scope overlays | `none` |

## Current State

Clean worktree baseline equals `origin/main` at `fb75cf6f`. Research and design are locked, and a
separate local open-model PLAN-EVAL returned `PASS`. No package source has changed yet.

## Completed

- Read issue #1134 and RFC #1123 §4 plus seed S-13/S-15 findings.
- Read required harness, PR, doctrine, JSR, tooling, and Archetype-2 authorities.
- Re-baselined actual MCP receipt/truncation code and adjacent debt.
- Ran baseline full-export doc lint: 0 diagnostics across `mod.ts` and `cli.ts`.
- Obtained tracked PLAN-EVAL `PASS` from Claude Code + OpenRouter Qwen.

## In Progress

- Slice 1: receipt settlement after validation and failed-attempt fixture.

## Next Steps

1. Implement and gate receipt settlement after validation.
2. Perform opposite-family slice review, commit, push, and comment.
3. Implement and gate truncation metadata + byte ceiling.

## Key Decisions

| Decision | Source | Notes |
| --- | --- | --- |
| No public export change | user contract / plan D1-D4 | Internal callback and fixed byte ceiling only |
| Full Archetype-2 column | RFC S-20 | quality gate + scoped wrappers + targeted runtime fixtures |
| Adjacent MCP v2 debt untouched | issue/user contract | `MCP-A6-V2-SHAPE` remains open |

## Files Changed

| Path | Status | Notes |
| --- | --- | --- |
| `.llm/runs/fix-mcp-truncation-receipt-ordering--s8/*` | new | Harness bootstrap/plan artifacts only |

## Gates

| Gate family | Current status | Evidence |
| --- | --- | --- |
| Static | baseline PASS | package doc lint 0 diagnostics |
| Fitness | plan-gate PASS | full Archetype-2 matrix selected and approved |
| Runtime | planned | two required fixtures |
| Consumer | N/A unless drift | exports locked unchanged |

## Open Questions

- None blocking PLAN-EVAL.

## Drift and Debt

- Drift: minor file-location clarification logged.
- Debt: adjacent `MCP-A6-V2-SHAPE` unchanged; no new debt expected.

## Commits

- See the draft PR's commit list + per-slice PR comments.
55 changes: 55 additions & 0 deletions .llm/runs/fix-mcp-truncation-receipt-ordering--s8/drift.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Drift Log: OMB S8 existing-machinery correctness fixes

Drift is append-only. Record facts that diverge from the plan, RFC, doctrine, or current-state
documentation.

## 2026-08-03 — Receipt wrapper location clarified

- **What:** The user brief names `mcp-server.ts` `withReceipt` near lines 96–112; current baseline
defines `withReceipt` in `packages/mcp/cli.ts`, while `mcp-server.ts` owns output validation at
the cited lifecycle point.
- **Source:** `packages/mcp/cli.ts:175-211`; `packages/mcp/src/application/runner/mcp-server.ts:96-116`.
- **Expected:** Receipt-ordering change centered on `mcp-server.ts`.
- **Actual:** Correctness requires a narrow wrapper/runner integration across both existing files.
- **Severity:** minor
- **Action:** fix
- **Evidence:** RFC S-15 already calls this a `withReceipt`/runner integration and anticipates both
locations; scope remains ordering only.

## 2026-08-03 — First local PLAN-EVAL transport timed out

- **What:** The canonical guarded local evaluator launched on Claude Code + OpenRouter Qwen but hit
the suite child-process timeout before writing `plan-eval.md`.
- **Source:** local evaluator outcome for command `plan-eval-1134`.
- **Expected:** A tracked `PASS` or `FAIL_PLAN` artifact.
- **Actual:** Transport exit 1 with diagnostic `timeout`; no verdict artifact and no tree changes.
- **Severity:** minor
- **Action:** fix
- **Evidence:** Retry the same route with a concise prompt and a longer bounded process timeout;
this is not counted as a `FAIL_PLAN` cycle because no evaluator verdict was emitted.

## 2026-08-03 — Ordinary review primary route unavailable

- **What:** The installed Claude CLI rejected the policy-configured Fable model id before the
slice-1 ordinary review began.
- **Source:** `review_codex` launch exit 1, API 404 `model_not_found`.
- **Expected:** Claude Fable low ordinary review.
- **Actual:** No review turn and no file changes; policy-declared Claude Opus low fallback available.
- **Severity:** minor
- **Action:** accept
- **Evidence:** Use the `review_codex` `token_limit_fallback` entry from
`workflow/lane-policy.md` / `runtime/routing-policy.ts`; opposite-family invariant remains intact.

## 2026-08-03 — IMPL-EVAL transport retries were guard-clean

- **What:** The first IMPL-EVAL launch omitted the runtime profile's OpenRouter source-to-Claude
credential mapping and received 401 before a turn. The next guarded Qwen attempt tried to spawn a
closed-model lookup child, which the evaluator model guard terminated before a verdict.
- **Source:** local Claude Code evaluator launch transcripts; evaluator policy audit guard.
- **Expected:** One complete open-model Qwen parent evaluation turn.
- **Actual:** Neither failed attempt produced a verdict or source change. A tightened no-delegation
parent-only Qwen retry completed and wrote `evaluate.md` with `PASS`.
- **Severity:** minor
- **Action:** accept
- **Evidence:** Final evaluator session records model `qwen/qwen3.7-max`, guard enabled, full gate
execution, and exact `PASS`; no closed-model evaluator work was admitted.
137 changes: 137 additions & 0 deletions .llm/runs/fix-mcp-truncation-receipt-ordering--s8/evaluate.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# IMPL-EVAL — fix-mcp-truncation-receipt-ordering--s8

- Evaluator session: Qwen parent session (no subagent delegation, no closed-model child)
- Run: `fix-mcp-truncation-receipt-ordering--s8`
- Branch: `fix/mcp-truncation-receipt-ordering`
- Baseline: `fb75cf6fc5ad02130ada0ac42e6f44035ac03a9b` (origin/main at plan lock)
- HEAD: `8a0314931ca000604f49b240f45523d333a7c2b6`
- Commits evaluated: `5192b07a9` (plan), `0ade6736a` (gate), `f00d6338e` (slice 1), `8a0314931` (slice 2)
- Surface / archetype: `packages/mcp` / Archetype 2 — Integration (RFC #1123 S-20)
- Scope overlays: none
- PR: #1183 (OPEN, `status:impl`); referenced issue #1134

## Plan-Gate prerequisite

`plan-eval.md` = `PASS`. Both implementation slices were authorized before any package source
change. The Plan-Gate commit `0ade6736a` precedes the first implementation commit `f00d6338e`.

## Design checkpoint verification

`worklog.md` contains a `## Design` section with all seven required subsections: Public Surface,
Domain Vocabulary, Ports, Constants, Commit Slices (2 slices, both < 30), Deferred Scope, and
Contributor Path. Every file created during implementation traces back to a concept named in the
design: `receipt-lifecycle.ts` (FlowReceiptRecorder + internal callback seam), truncation changes
(TruncationOutcome + ResultByteLimitError).

## Commit slice conformance

| Slice | Design plan | Commit | Gate evidence |
| --- | --- | --- | --- |
| 1 | Receipt-after-validation lifecycle and failed-attempt fixture | `f00d6338e` | targeted `drift-evidence_test.ts` + `doctor_test.ts` 16/16 PASS |
| 2 | Honest truncation metadata and whole-result byte ceiling | `8a0314931` | targeted `truncation_test.ts` + full package tests 66/66 PASS |

Both slices match the design checkpoint ordering and named gates. Supervisor sign-off commit
(`f00d6338e`) names what the slice proves, not what it contains.

## Verification of the five close-gated behaviors

### 1. Invalid/throwing/failed output settles a failed receipt; green settlement occurs only after output validation, central bounds, and revalidation

**PASS.** `mcp-server.ts` diff confirms the lifecycle ordering:

- **Throw path**: `tool.flow(input)` wrapped in try/catch. On throw → `settleFlowReceipt(flow, input, false)` then `rpcError(-32603, 'tool_execution_failed')`.
- **Error result path**: `!execution.ok` → `settleFlowReceipt(flow, input, false)` then `rpcResult` with `isError: true`.
- **Validation/bound path**: `validateSchema(outputSchema, execution.value)` → `truncateResult(execution.value, policy)` → `validateSchema(outputSchema, bounded)`. On any throw → `settleFlowReceipt(flow, input, false)` then `rpcError(-32603, 'invalid_tool_result')` or `tool_result_too_large`.
- **Green path**: only after all three stages succeed → `settleFlowReceipt(flow, input, resultSucceeded(execution.value))` where `resultSucceeded` returns false when `status === 'fail'`.

**Evidence:** `mcp-server.ts:107-148` diff; test `invalid MCP tool output replaces stale green evidence with a failed receipt` (PASS); test `throwing MCP tool flow replaces stale green evidence with a failed receipt` (PASS).

### 2. A 75-row result capped to 50 cannot retain `truncated: false`; nested caps propagate only into already-existing boolean metadata

**PASS.** `truncation.ts` recursive `truncateValue` returns `TruncationOutcome` pairs. For arrays:
`truncated: value.length > policy.maxItems || outcomes.some((o) => o.truncated)`. For objects: if
any descendant was truncated AND the object has an own boolean `truncated` property, it is flipped
to `true`. The rewrite is guarded by `typeof bounded.truncated === 'boolean'` — no new metadata
fields are invented, and non-boolean properties named `truncated` are untouched.

**Evidence:** `truncation.ts:31-55` diff; test `75 rows cannot become 50 rows with false truncation metadata` (PASS: `rows.length === 50`, `truncated === true`); test `nested truncation marks existing metadata on every declaring ancestor` (PASS: both `page.truncated` and root `truncated` flipped).

### 3. A fixed post-truncation UTF-8 byte ceiling rejects irreducibly oversized results before response and receipt success

**PASS.** `truncateResult` computes `JSON.stringify(outcome.value)` byte length via `TextEncoder`.
If `byteLength > MAX_SERIALIZED_RESULT_BYTES` (65,536 bytes, 64 KiB), throws `ResultByteLimitError`.
The runner catches this in the validation/bound try/catch, settles a failed receipt, and returns
`rpcError(-32603, 'tool_result_too_large')` — before any green receipt can be written.

**Evidence:** `truncation.ts:12-14, 23-28` diff; `mcp-server.ts:119-127` diff; test `irreducibly large bounded results fail the UTF-8 byte ceiling` (PASS: `assertThrows` with `ResultByteLimitError` and message `'after truncation; limit is 65536'`).

### 4. No public export-map/type surface, `deno.lock`, MCP v2 registration shape, prohibited ignore, or double-cast movement occurred

**PASS.**

- `git diff fb75cf6fc..HEAD -- packages/mcp/deno.json packages/mcp/mod.ts deno.lock` — zero output (no changes).
- `git diff` prohibited pattern audit — zero occurrences of `deno-lint-ignore`, `@ts-ignore`, `as unknown as`, `as any`.
- `ResultByteLimitError` and `withFlowReceipt` are consumed only by the runner and by test files via internal paths (`../src/application/runner/...`); neither is re-exported from `mod.ts` or `cli.ts`.
- `createMcpServer`, `createMcpCliServer`, `truncateResult`, `TruncationPolicy`, `DEFAULT_TRUNCATION_POLICY` retain their existing source-compatible signatures.
- `MCP-A6-V2-SHAPE` debt (arch-debt.md line 2096) is adjacent and untouched: no folder structure, tool registration, or command tree changes.

**Evidence:** raw `git diff` empty for `deno.json`/`mod.ts`/`deno.lock`; `grep -cE` prohibited patterns returned 0; `deno task doc:lint --root packages/mcp --pretty` — 2 entrypoints, 0 diagnostics (unchanged from baseline).

### 5. Required targeted/package tests, scoped check/lint/fmt, `quality:gate`, close-gate/DoD evidence, commit slices, and debt disposition satisfy the protocol

**PASS.**

| Gate | Command | Result |
| --- | --- | --- |
| Receipt/doctor fixture | `deno test --allow-env --allow-net --allow-run --allow-read --allow-write tests/drift-evidence_test.ts tests/doctor_test.ts` | 16 passed, 0 failed |
| Truncation fixture | `deno test tests/truncation_test.ts` | 4 passed, 0 failed |
| Full package tests | `deno test --allow-env --allow-net --allow-run --allow-read --allow-write tests/` | 66 passed, 0 failed |
| Scoped check | `run-deno-check.ts --root packages/mcp --ext ts,tsx` | 68 files, 0 occurrences |
| Scoped lint | `run-deno-lint.ts --root packages/mcp --ext ts,tsx --config packages/mcp/deno.json` | 68 files, 0 occurrences |
| Scoped format | `run-deno-fmt.ts --root packages/mcp --ext ts,tsx --config packages/mcp/deno.json` | 68 files, 0 findings |
| Quality scan | `deno task quality:scan` | 0 findings, 7 allowances (all pre-existing, none in `packages/mcp`) |
| Architecture check | `deno task arch:check` | exit 0; `@netscript/mcp` 0 FAIL, 0 WARN, 0 INFO |
| Baseline doc lint | `deno task doc:lint --root packages/mcp --pretty` | 2 entrypoints, 0 diagnostics |
| Publish dry-run | conditional | N/A — no export drift occurred per verification point 4 |

`e2e:cli` is explicitly N/A by the orchestrator brief (non-release PR slice).

**Close-gate / Definition-of-Done disposition:** PR #1183 body explicitly states _"the closing keyword will be added only after both close-gated fixtures pass with quoted evidence"_ and the Definition-of-Done checkboxes are intentionally unchecked pending this evaluator verdict. The orchestrator brief names this as intentional lifecycle ordering, not an implementation defect. The evaluator confirms the underlying evidence exists: both close-gated fixtures pass with the exact behaviors named in #1134's acceptance criteria (invalid output → failed receipt; 75 rows → `truncated: true`). The PR's per-slice comments (`[PHASE: RESEARCH]`, `[PHASE: PLAN]`, `[PHASE: PLAN-EVAL] [VERDICT: APPROVED]`) are present; post-implementation slice comments are pending per the orchestrator's post-verdict finalization lifecycle.

**Debt disposition:** `MCP-A6-V2-SHAPE` (arch-debt.md line 2096) is adjacent and unchanged — the PR neither closes nor deepens it. No new debt entries were created. `arch-debt.md` was not modified in this branch.

**Commit slices:** Two implementation commits (`f00d6338e`, `8a0314931`) plus two plan/gate commits (`5192b07a9`, `0ade6736a`), matching the design checkpoint's two-slice structure.

## Archetype-2 full-column verification

| Gate family | Required | Result | Evidence |
| --- | --- | --- | --- |
| Fitness F-1..F-5 | yes | PASS | `quality:gate` (scan + arch:check) exit 0; scoped check/lint/fmt 0 findings |
| Fitness F-6/F-7 | yes | PASS | Baseline doc-lint 0 diagnostics; publish dry-run N/A (no export drift) |
| Fitness F-8..F-12 | yes | PASS | `quality:gate` + targeted fixtures 66/66 |
| Fitness F-14..F-19 | yes | PASS | `quality:gate` + diff audit (0 prohibited patterns) + wrapper-sourced results |
| Static | yes | PASS | scoped check/lint/fmt + doc-lint + prohibited-pattern diff scan |
| Runtime/consumer | touched | PASS | targeted receipt/doctor/truncation fixtures; consumer import gate N/A (exports locked) |

## Findings

No implementation findings. The implementation is complete, correct, and satisfies every locked
decision (D1–D5) and every risk mitigation from the approved plan.

One lifecycle observation: the PR body's Definition-of-Done checkboxes, issue #1134's acceptance
checkboxes, and the closing keyword are all intentionally deferred until after this IMPL-EVAL
verdict, per the orchestrator brief and the PR body's own explicit statement. This is a
post-verdict administrative step, not an implementation or documentation defect.

## Verdict

`PASS`

## Evidence summary

- Source diff: 4 package source files changed (cli.ts, mcp-server.ts, receipt-lifecycle.ts, truncation.ts), 2 test files changed (drift-evidence_test.ts, truncation_test.ts), 7 harness artifacts.
- Test evidence: 66 passed, 0 failed across the full MCP package suite; 3 new close-gated fixtures all pass.
- Static evidence: scoped check/lint/fmt 0 findings on 68 TypeScript files; quality:scan 0 findings; arch:check exit 0; doc-lint 0 diagnostics.
- No export drift, no lock churn, no prohibited patterns, no new debt.
- Plan-Gate passed before implementation; design checkpoint present and followed; commit slices match design.
- Close-gate evidence is complete in substance (all acceptance behaviors verified by passing fixtures); PR body and issue checkbox updates are the documented post-verdict lifecycle step.
Loading
Loading