Skip to content

docs(reports): close the nest_migration campaign with its verification record - #16

Merged
LittleCoinCoin merged 3 commits into
mainfrom
roadmap/nest-migration
Sep 16, 2026
Merged

LittleCoinCoin merged 3 commits into
mainfrom
roadmap/nest-migration

Conversation

@LittleCoinCoin

Copy link
Copy Markdown
Member

Why

Closes the nest_migration campaign. Documentation only — no code, no manifests, no version change. The migration itself landed in #15; this records what verifying it actually proved.

What changed

  • __reports__/nest_migration/00-findings_migration_v0.md — the install check as first measured, when colgrep-mcp could not be installed from Nest.
  • __reports__/nest_migration/01-findings_migration_v1.md — after the Nest fix: all seven entries install.
  • __reports__/nest_migration/README.md — topic index, plus the record of the one gate that closed after both reports were written.
  • __roadmap__/nest_migration/ — all three leaves ✅, set through dirtree-rdm only, validate clean at both levels.

What the verification actually proved

Gate Result
colgrep-mcp installs from Nest as published ✅
All 7 catalogue entries install and enable ✅
No failed to load, no duplicate-hooks failure ✅ 0
MCP server connects ✅ uvx colgrep-mcp==0.5.1 - ✔ Connected
Installed artifact shape ✅ extensions["com.openai"], 7 interface keys, no .codex-plugin/ / marketplace.json / .agents/
Search-policy hook fires in a fresh session ✅

The duplicate-hooks failure this repository hit at 0.4.0 and 0.5.0 did not fire, on a release where the hooks manifest changed shape. That class of failure is only reproducible through a real marketplace install — --plugin-dir, plugin details and plugin validate all accept a tree that fails one.

Two pieces of evidence that were nearly wrong

The first colgrep-mcp install was discarded rather than reused. It succeeded only because of a local marketplace-cache edit carrying an unpublished fix. Re-running the gate without uninstalling first would have re-confirmed the workaround and passed for the wrong reason — the marketplace-level twin of running a suite against the wrong checkout. The plugin was uninstalled and reinstalled from the published catalogue before the gate was called met.

The hook gate was left open rather than inferred. claude -p failed twice with an expired OAuth session, so v0 and v1 both record it unverified, with the installed hook script exercised directly as a partial substitute. After re-authentication a fresh session received the SessionStart context and quoted the denied-tools sentence verbatim. That closure is recorded in the topic README; the reports are left as written, because a finding states what was known when it was made, and editing them to look prescient would destroy the only evidence the gate was ever open.

Findings handed to other repositories

Each was diagnosed here, verified, and fixed upstream rather than worked around locally:

  • Nest's colgrep-mcp entry used the github + repo shorthand, which Claude Code clones over SSH for a plugin source — while resolving the same shorthand over HTTPS for a marketplace source. Ruled out as a credential fault: gh auth reports https as its git protocol with repo scope, osxkeychain is configured, and both were ignored. Any user without a GitHub SSH key could not install. Fixed in Nest 2ebd2bc; the replacement shape was verified locally before being recommended.
  • The upstream end_to_end gate verified five installs plus colgrep-mcp's listing. Listing and installing are different claims, and all five verified entries shared the working shape, so no counter-example existed anywhere in the evidence. The gate now installs every entry.
  • claude plugin validate . validates one manifest, prefers the marketplace, and stops — so while this repository shipped a catalogue, its own gate never checked the plugin manifest. Documented here as stack-traps#validate-picks-one and upstream under the same anchor.
  • The generator advised --force on a file it refuses to overwrite. Fixed upstream; splitting the ledger there nearly reintroduced the class of bug it was fixing, since the guard iterated one list.

Verification

  • uv run pytest — 307 passed, 1 skipped
  • uv run ruff check / ruff format --check — clean
  • uv run cz check --rev-range main..HEAD — clean
  • dirtree-rdm validate — OK at both roadmap levels

Not in scope

The v0.5.2 release is deliberately separate, and left to cz to compute from these commits. Worth recording that refactor has no bump_map entry in this repository, so the campaign's headline change — the manifest reshape — contributed no version bump; the patch comes from incidental fix commits. Since Codex reinstalls only on a version change, a pure refactor + docs campaign would have shipped a manifest no Codex user ever received.

🤖 Generated with Claude Code

LittleCoinCoin and others added 3 commits September 16, 2026 18:53
Findings report for `verify/install_check`, separating what was exercised
live in Claude Code from what rests on the published format alone.

Six of Nest's seven entries install. `colgrep-mcp` — the one this campaign
exists to serve — does not, as published: its Claude-side entry uses the
`github` + `repo` shorthand, which Claude Code clones over SSH, and fails for
anyone without a GitHub SSH key. The defect is in Nest, not here. Confirmed
not to be a credential fault: `gh auth` reports https as its git protocol and
`osxkeychain` is configured, both ignored, while the same shorthand at
*marketplace* level resolved over HTTPS on this machine. The replacement shape
was verified locally before recommending it, rather than pushed as a guess.

Everything this repository owns passes: no plugin fails to load, the server
connects under the published `uvx colgrep-mcp==0.5.1` pin, and the installed
artifact carries `extensions["com.openai"]` with the full interface block and
no `.codex-plugin/`, `marketplace.json` or `.agents/`. The duplicate-hooks
failure this repository hit at 0.4.0 and 0.5.0 did not fire, on a release
where the hooks manifest changed shape.

One gate is recorded as unverified rather than inferred: whether the hook
fires at the start of a fresh session. `claude -p` fails with an expired OAuth
session here (`stack-traps#oauth`), so the installed hook script was exercised
directly instead — it emits the policy on `SessionStart` and denies both
`Grep` and a shell corpus search on `PreToolUse`. Script behaviour and
manifest wiring are proven; harness-level firing is not, and the report says
so in its own section rather than in a footnote.

`install_check` is marked blocked, not done: it cannot pass on its own terms
until Nest's entry is fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nest 2ebd2bc gives colgrep-mcp an explicit HTTPS `url` source; no entry uses
the `github` shorthand any more, and all seven install. The campaign closes.

The v0 pass for this entry had to be discarded as evidence rather than reused.
It came from a local marketplace-cache edit, so re-running the gate without
uninstalling first would have re-confirmed the workaround and passed for the
wrong reason — the marketplace-level twin of running a suite against the wrong
checkout. The plugin was uninstalled and reinstalled from the published
catalogue, and the artifact re-inspected: extensions["com.openai"] with seven
interface keys, hooks split correctly, no .codex-plugin/, marketplace.json or
.agents/.

Also verified for the playbook, whose guard entry was held back on the belief
that this repository's shared checkout was stale: it is already on a75fb36,
clean, with the extensions key present and both marketplace files gone. Their
`test_spec_regenerates_manifests()` was re-run in memory with
ALLOWED_DIVERGENCE reduced to `{"dev/README.md": None}` and passes, so the
entry can come out. Their file was not edited to establish this.

install_check is marked done with one gate recorded as unverified rather than
inferred: whether the hook fires at the start of a fresh session. `claude -p`
fails with an expired OAuth session here, retried twice (stack-traps#oauth).
The installed hook script was exercised directly instead and behaves
correctly, so what is proven is script behaviour and manifest wiring, not
harness-level firing. The report says so in its own section.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both findings reports record this gate as unverified, because `claude -p`
failed twice with an expired OAuth session. After re-authentication it was
exercised for real: a fresh `claude -p` session, running the plugin installed
from the published Nest catalogue, received the SessionStart context and quoted
the denied-tools sentence back verbatim. Harness-level firing is confirmed, not
just script behaviour and manifest wiring.

Recorded in the topic README's status section rather than as a new findings
round. The reports are left as written: a finding states what was known when it
was made, and rewriting v0 and v1 to look prescient would destroy the only
evidence that the gate was ever open.

Every gate in verify/install_check has now been exercised live.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LittleCoinCoin
LittleCoinCoin merged commit 2a3578a into main Sep 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant