docs(reports): close the nest_migration campaign with its verification record - #16
Merged
Merged
Conversation
Findings report for `verify/install_check`, separating what was exercised live in Claude Code from what rests on the published format alone. Six of Nest's seven entries install. `colgrep-mcp` — the one this campaign exists to serve — does not, as published: its Claude-side entry uses the `github` + `repo` shorthand, which Claude Code clones over SSH, and fails for anyone without a GitHub SSH key. The defect is in Nest, not here. Confirmed not to be a credential fault: `gh auth` reports https as its git protocol and `osxkeychain` is configured, both ignored, while the same shorthand at *marketplace* level resolved over HTTPS on this machine. The replacement shape was verified locally before recommending it, rather than pushed as a guess. Everything this repository owns passes: no plugin fails to load, the server connects under the published `uvx colgrep-mcp==0.5.1` pin, and the installed artifact carries `extensions["com.openai"]` with the full interface block and no `.codex-plugin/`, `marketplace.json` or `.agents/`. The duplicate-hooks failure this repository hit at 0.4.0 and 0.5.0 did not fire, on a release where the hooks manifest changed shape. One gate is recorded as unverified rather than inferred: whether the hook fires at the start of a fresh session. `claude -p` fails with an expired OAuth session here (`stack-traps#oauth`), so the installed hook script was exercised directly instead — it emits the policy on `SessionStart` and denies both `Grep` and a shell corpus search on `PreToolUse`. Script behaviour and manifest wiring are proven; harness-level firing is not, and the report says so in its own section rather than in a footnote. `install_check` is marked blocked, not done: it cannot pass on its own terms until Nest's entry is fixed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nest 2ebd2bc gives colgrep-mcp an explicit HTTPS `url` source; no entry uses the `github` shorthand any more, and all seven install. The campaign closes. The v0 pass for this entry had to be discarded as evidence rather than reused. It came from a local marketplace-cache edit, so re-running the gate without uninstalling first would have re-confirmed the workaround and passed for the wrong reason — the marketplace-level twin of running a suite against the wrong checkout. The plugin was uninstalled and reinstalled from the published catalogue, and the artifact re-inspected: extensions["com.openai"] with seven interface keys, hooks split correctly, no .codex-plugin/, marketplace.json or .agents/. Also verified for the playbook, whose guard entry was held back on the belief that this repository's shared checkout was stale: it is already on a75fb36, clean, with the extensions key present and both marketplace files gone. Their `test_spec_regenerates_manifests()` was re-run in memory with ALLOWED_DIVERGENCE reduced to `{"dev/README.md": None}` and passes, so the entry can come out. Their file was not edited to establish this. install_check is marked done with one gate recorded as unverified rather than inferred: whether the hook fires at the start of a fresh session. `claude -p` fails with an expired OAuth session here, retried twice (stack-traps#oauth). The installed hook script was exercised directly instead and behaves correctly, so what is proven is script behaviour and manifest wiring, not harness-level firing. The report says so in its own section. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both findings reports record this gate as unverified, because `claude -p` failed twice with an expired OAuth session. After re-authentication it was exercised for real: a fresh `claude -p` session, running the plugin installed from the published Nest catalogue, received the SessionStart context and quoted the denied-tools sentence back verbatim. Harness-level firing is confirmed, not just script behaviour and manifest wiring. Recorded in the topic README's status section rather than as a new findings round. The reports are left as written: a finding states what was known when it was made, and rewriting v0 and v1 to look prescient would destroy the only evidence that the gate was ever open. Every gate in verify/install_check has now been exercised live. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Closes the
nest_migrationcampaign. Documentation only — no code, no manifests, no version change. The migration itself landed in #15; this records what verifying it actually proved.What changed
__reports__/nest_migration/00-findings_migration_v0.md— the install check as first measured, whencolgrep-mcpcould not be installed from Nest.__reports__/nest_migration/01-findings_migration_v1.md— after the Nest fix: all seven entries install.__reports__/nest_migration/README.md— topic index, plus the record of the one gate that closed after both reports were written.__roadmap__/nest_migration/— all three leaves ✅, set throughdirtree-rdmonly,validateclean at both levels.What the verification actually proved
colgrep-mcpinstalls from Nest as publishedfailed to load, no duplicate-hooks failureuvx colgrep-mcp==0.5.1 - ✔ Connectedextensions["com.openai"], 7 interface keys, no.codex-plugin//marketplace.json/.agents/The duplicate-hooks failure this repository hit at 0.4.0 and 0.5.0 did not fire, on a release where the hooks manifest changed shape. That class of failure is only reproducible through a real marketplace install —
--plugin-dir,plugin detailsandplugin validateall accept a tree that fails one.Two pieces of evidence that were nearly wrong
The first
colgrep-mcpinstall was discarded rather than reused. It succeeded only because of a local marketplace-cache edit carrying an unpublished fix. Re-running the gate without uninstalling first would have re-confirmed the workaround and passed for the wrong reason — the marketplace-level twin of running a suite against the wrong checkout. The plugin was uninstalled and reinstalled from the published catalogue before the gate was called met.The hook gate was left open rather than inferred.
claude -pfailed twice with an expired OAuth session, so v0 and v1 both record it unverified, with the installed hook script exercised directly as a partial substitute. After re-authentication a fresh session received theSessionStartcontext and quoted the denied-tools sentence verbatim. That closure is recorded in the topic README; the reports are left as written, because a finding states what was known when it was made, and editing them to look prescient would destroy the only evidence the gate was ever open.Findings handed to other repositories
Each was diagnosed here, verified, and fixed upstream rather than worked around locally:
colgrep-mcpentry used thegithub+reposhorthand, which Claude Code clones over SSH for a plugin source — while resolving the same shorthand over HTTPS for a marketplace source. Ruled out as a credential fault:gh authreportshttpsas its git protocol withreposcope,osxkeychainis configured, and both were ignored. Any user without a GitHub SSH key could not install. Fixed in Nest2ebd2bc; the replacement shape was verified locally before being recommended.end_to_endgate verified five installs pluscolgrep-mcp's listing. Listing and installing are different claims, and all five verified entries shared the working shape, so no counter-example existed anywhere in the evidence. The gate now installs every entry.claude plugin validate .validates one manifest, prefers the marketplace, and stops — so while this repository shipped a catalogue, its own gate never checked the plugin manifest. Documented here asstack-traps#validate-picks-oneand upstream under the same anchor.--forceon a file it refuses to overwrite. Fixed upstream; splitting the ledger there nearly reintroduced the class of bug it was fixing, since the guard iterated one list.Verification
uv run pytest— 307 passed, 1 skippeduv run ruff check/ruff format --check— cleanuv run cz check --rev-range main..HEAD— cleandirtree-rdm validate— OK at both roadmap levelsNot in scope
The
v0.5.2release is deliberately separate, and left toczto compute from these commits. Worth recording thatrefactorhas nobump_mapentry in this repository, so the campaign's headline change — the manifest reshape — contributed no version bump; the patch comes from incidentalfixcommits. Since Codex reinstalls only on a version change, a purerefactor+docscampaign would have shipped a manifest no Codex user ever received.🤖 Generated with Claude Code