Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,11 +67,12 @@ Opt-in: `COLGREP_MCP_REAL=1 uv run pytest` adds the few tests that need the real
`colgrep`; `uv run python tests/e2e/run_e2e.py --corpus <some other repo>` runs the
end-to-end driver — never against this repository (see `stack-traps`).

## Report ids in docstrings
## Report ids in docstrings and skills

Docstrings cite report ids as pointers to measured evidence, not narration.
The same `R01` label is reused per campaign; the module docstring's topic
disambiguates which one.
Docstrings and the dev skills cite report ids as pointers to measured evidence,
not narration. The same `R01` label is reused per campaign; the module
docstring's topic disambiguates which one. `KT-*` are knowledge-transfer
reports, `OBS-*` observation reports from a reviewer pass.

| Id | Report |
|:--|:--|
Expand All @@ -87,6 +88,16 @@ disambiguates which one.
| `R01` (harness_wiring) | `__reports__/harness_wiring/00-architecture_v0.md` |
| `R01` (index_housekeeping) | `__reports__/index_housekeeping/00-architecture_v0.md` |
| `R02` (index_housekeeping) | `__reports__/index_housekeeping/00-findings_clear_probe_v0.md` |
| `KT-B` | `__reports__/colgrep_mcp/03-knowledge_transfer_v0.md` |
| `KT-H` | `__reports__/repo_health/02-knowledge_transfer_v0.md` |
| `KT-C` | `__reports__/consistency/02-knowledge_transfer_v0.md` |
| `KT-D` | `__reports__/dev_plugin/03-knowledge_transfer_v0.md` |
| `OBS-H` | `__reports__/repo_health/01-observation_review_v0.md` |
| `OBS-C` | `__reports__/consistency/01-observation_review_v0.md` |
| `OBS-D` | `__reports__/dev_plugin/02-observation_review_v0.md` |
| `PH` | `__reports__/repo_health/00-findings_launch_placeholders_v0.md` |
| `CI` | `__reports__/repo_health/03-findings_ci_matrix_v0.md` |
| `RI` | `__reports__/repo_health/04-findings_remote_install_v0.md` |

## Where the rest went

Expand Down
28 changes: 23 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ Semantic and hybrid code search for coding agents, as an MCP server.

[colgrep](https://github.com/lightonai/next-plaid) indexes a repository into *code units* (functions, classes, methods, Markdown sections) and ranks them with a ColBERT late-interaction model fused with keyword search. It is fast and it understands meaning. It is also a CLI, and agents trained on `grep` rarely reach for it unprompted. `colgrep-mcp` puts the same capability in the agent's tool list, where it gets used.

This is an independent project. It is not affiliated with or supported by [LightOn](https://www.lighton.ai), who make colgrep. A problem with the search tools belongs in this repository's issues, a problem with colgrep itself in [next-plaid](https://github.com/lightonai/next-plaid/issues).

One directory installs as a **Claude Code plugin**, an **[Agent Plugins 1.0](https://agent-plugins.org) plugin** (Codex, Cursor, GitHub Copilot, VS Code, Kiro) and a **Codex plugin**.

## Requirements
Expand Down Expand Up @@ -42,11 +44,19 @@ codex plugin marketplace add CrackingShells/colgrep-mcp
codex plugin add colgrep-mcp@colgrep-mcp-marketplace
```

The Codex manifests are `.agents/plugins/marketplace.json` and `.codex-plugin/plugin.json`. These commands follow the Codex plugin documentation and have not yet been exercised end to end; a report of a working (or failing) install is welcome as an issue.
The Codex manifests are `.agents/plugins/marketplace.json` and `.codex-plugin/plugin.json`.

### Agent Plugins 1.0 clients (Cursor, GitHub Copilot, VS Code, Kiro)

The [Agent Plugins 1.0 spec](https://agent-plugins.org/specification) defines the plugin package (`plugin.json`, `mcp.json`) but explicitly leaves installation, distribution and marketplaces to each client — there is no spec-defined command for installing straight from a git URL. Check that client's own plugin or extension docs for how it adds a plugin from a repository; until then, point it at a local clone the way it expects a plugin directory (below).
The [Agent Plugins 1.0 spec](https://agent-plugins.org/specification) defines the plugin package (`plugin.json`, `mcp.json`) and leaves installation, distribution and marketplaces to each client, so the install command is the client's own. Each of these clients can add a plugin straight from this repository; its plugin or extension docs name the command.

For example, in VS Code:

1. Open the Command Palette (`Cmd`+`Shift`+`P` on macOS, `Ctrl`+`Shift`+`P` elsewhere).
2. Run **Chat: Install Plugin from Source**.
3. Choose the git repository option and enter `CrackingShells/colgrep-mcp` (the full URL `https://github.com/CrackingShells/colgrep-mcp` works too).

A client without such a command takes a local clone as its plugin directory ([From a local clone](#from-a-local-clone)).

### Any MCP client

Expand Down Expand Up @@ -140,7 +150,7 @@ The plugin also ships harness hooks (`hooks/`), so the policy the skill teaches
| `PreToolUse` on `Grep` and `Bash` | Denies the built-in Grep tool and shell corpus searches (`grep -r`, `rg`, `find -exec grep`, `xargs grep`) inside a source corpus, with a reason naming `search`, `find_files` and `expand`. Single-file grep, `cmd \| grep`, `grep -c`/`-v`/`-o`, `rg --files` and file-name lookup stay allowed. Targets that are machine state (hidden directories, `~/Library`, temp directories outside a git work tree) are never gated. Prefix `COLGREP_BYPASS=1` to a command colgrep cannot serve. |
| `WorktreeRemove` (Claude Code only) | Clears the colgrep index a removed worktree owned, never one it was folded into. |

`hooks/hooks.json` holds only events that Claude Code, Codex and Cursor all understand; `hooks/claude-code.json` holds the Claude-only event and is named by the Claude Code manifest alone. Codex loads a plugin's `hooks/hooks.json` and sets `CLAUDE_PLUGIN_ROOT` for it, but skips the hooks until you trust them once in `/hooks`. Cursor imports Claude Code hooks from `settings.json` files, not from plugins, so a Cursor project copies the three `hooks.json` entries into its `.claude/settings.json`. Agent Plugins 1.0 defines no hooks component and ignores the directory. Design and measurements: `__reports__/harness_wiring/00-architecture_v0.md`.
`hooks/hooks.json` holds only events that Claude Code, Codex and Cursor all understand; `hooks/claude-code.json` holds the Claude-only event and is named by the Claude Code manifest alone. Codex loads a plugin's `hooks/hooks.json` and sets `CLAUDE_PLUGIN_ROOT` for it, but skips the hooks until you trust them once in `/hooks`. Cursor imports Claude Code hooks from `settings.json` files, not from plugins, so a Cursor project copies the three `hooks.json` entries into its `.claude/settings.json`. Agent Plugins 1.0 defines no hooks component and ignores the directory.

## Configuration

Expand All @@ -166,6 +176,14 @@ Without `COLGREP_MCP_ROOT` the server falls back to the client's first root, if
- **`index_clear` refuses.** colgrep folds a directory into the nearest already-indexed ancestor project. The tool tells you the project root it would clear; pass that root explicitly if that is really intended.
- **Line numbers.** colgrep 1.6 reports wrong `line`/`end_line` for most units. The server re-locates every unit from its source text and flags `location_verified` on each hit.

## For agents, by agents

Everyone who touches this repository is an LLM agent. Users reach it through the MCP tools, and the maintenance itself is handed to a coding agent, at present Claude Fable 5.1 in Claude Code: it reads the architecture reports, plans the work as a roadmap, implements in its own git worktree, writes the tests and the docs, and opens the pull request. The MCP server and the skill are the two layers made for the human and the agent to talk to each other; everything else, from the drift tests to the maintainer skills in `dev/`, is optimised for an agent picking the work up cold.

A change goes through an ordinary pull-request cycle. The agent commits with the vocabulary `cz check` enforces (`CONTRIBUTING.md`), pushes a branch and opens the PR; CI runs ruff, the test suite on Linux, macOS and Windows, the commit check and a build; the maintainer reads the diff and the PR body, then merges; a release is a `cz bump` on `main` and a tag push, which publishes to PyPI. Larger work runs as a campaign: an architecture report under `__reports__/`, a roadmap under `__roadmap__/`, one worktree per leaf, a read-only reviewer pass, and a retrospective whose lessons become the next revision of the `dev/` skills.

This holds because the agent is a frontier model and because the maintainer, who has built MCP servers before, reads every diff. The tests, the drift guards and the reports exist so that the trust placed in the agent is verified at each merge rather than assumed; the same process with a weaker model, or with merges nobody reads, would drift.

## Development

```bash
Expand All @@ -174,7 +192,7 @@ cd server && uv run pytest -q

If you are a coding agent maintaining this repository, start with `AGENTS.md`: repo map, gate commands, conventions and known traps.

Tests run against a fake `colgrep` (`server/tests/fake_colgrep.py`); set `COLGREP_MCP_REAL=1` to include the few that need the real binary. Architecture, measured behaviour and decisions live in `__reports__/colgrep_mcp/`; the execution plan in `__roadmap__/colgrep_mcp/`; commit conventions in `CONTRIBUTING.md`.
Tests run against a fake `colgrep` (`server/tests/fake_colgrep.py`); set `COLGREP_MCP_REAL=1` to include the few that need the real binary. Architecture, findings and retrospectives live under `__reports__/`, one directory per campaign; the roadmaps under `__roadmap__/`; commit conventions in `CONTRIBUTING.md`.

`server/tests/e2e/run_e2e.py` is a separate, non-pytest script (no `test_` prefix, so `pytest` never collects it) that drives the assembled server over stdio against a **real** `colgrep` binary and a real repository, for measured end-to-end validation rather than fixture-driven unit tests:

Expand All @@ -183,7 +201,7 @@ cd server && uv run python tests/e2e/run_e2e.py --corpus /path/to/a/real/repo
cd server && uv run python tests/e2e/run_e2e.py --corpus /path/to/a/real/repo --dry-run # print the call plan, exit 0
```

It refuses to run against this repository/its worktrees or anything under `/private/tmp` (colgrep folds such paths into whichever project already anchors that prefix — see `__reports__/colgrep_mcp/02-architecture_v1.md` D3). Findings from the latest run live in `__reports__/colgrep_mcp/02-findings_e2e_validation_v0.md`.
It refuses to run against this repository, its worktrees or anything under `/private/tmp`: colgrep folds such paths into whichever project already anchors that prefix, and the driver builds and clears indexes.

## Packaging

Expand Down
37 changes: 18 additions & 19 deletions dev/skills/campaign-lead/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ description: "Guides a lead through running a multi-agent campaign on this repos
# Campaign Lead

Composes with the machine-level `writing-reports`, `managing-roadmaps` and `writing-history`
skills for mechanics; this skill is the order of operations and the rules three prior campaigns
(`colgrep_mcp`, `repo_health`, `consistency`) paid to learn.
skills for mechanics; this skill is the order of operations and the rules this repository's
campaigns established, each cited to the retrospective that records it (`AGENTS.md` legend).

## Order of Operations

Expand All @@ -21,14 +21,14 @@ skills for mechanics; this skill is the order of operations and the rules three
KT-C §Wins: helpers before dispatch → zero conflicts).
4. **One worktree per leaf, created by hand**: `git worktree add <path> -b task/<leaf>
<campaign-branch>`. Never rely on the Agent tool's `isolation: worktree` for this — it
branches from the primary checkout's `main`, not the campaign branch, and cost a full
wasted dispatch round in `repo_health` (KT-H §Pain Points, KT-B §Root Causes, lead memory).
branches from the primary checkout's `main`, not the campaign branch, and a whole dispatch
round lands on the wrong base (KT-H §Pain Points, KT-B §Root Causes).
5. **Dispatch** with `references/dispatch-prompt.md`. Every prompt states "stop if the leaf
file is missing from your worktree" — it turned the wrong-base failure above into a
~15-second no-op with zero stray commits instead of wasted work (KT-H §Wins).
file is missing from your worktree" — it turns a wrong-base worktree into a ~15-second
no-op with zero stray commits instead of wasted work (KT-H §Wins).
6. **Merge level by level**, rebase-then-`--no-ff` (mechanics: `writing-history`). Re-run the
gates yourself on every branch first — an implementer's reported count is not evidence; one
branch reported 196 passing tests on a tree that actually collected 206 (KT-C §Pain Points).
gates yourself on every branch first — an implementer's reported test count is not evidence
(KT-C §Pain Points).
7. **Reviewer after any level with more than two parallel branches** — see
`references/reviewer-brief.md` — before merging onward.
8. **Close with a knowledge-transfer report** (`writing-reports`); its Next-cycle Changes seed
Expand All @@ -39,9 +39,8 @@ skills for mechanics; this skill is the order of operations and the rules three
- **Worktrees see only committed files.** Commit every spec and helper before dispatching, or
the implementer's worktree simply will not have them.
- **File-disjoint ownership**, stated in the roadmap README's Gotchas, not assumed: each leaf
edits only its own files and *reports* anything else it notices in its final message.
`repo_health` and `consistency` both closed with zero merge conflicts this way (KT-H, KT-C
§Wins).
edits only its own files and *reports* anything else it notices in its final message. This
is what closes a campaign with zero merge conflicts (KT-H, KT-C §Wins).
- **Hard stop time**, stated in both the leaf and the dispatch prompt: the lead merges whatever
is green at the stop (`consistency` roadmap README §Gotchas).
- **Do lead-sized leaves yourself.** Scaffolding, shared helpers, CI wiring, `AGENTS.md`,
Expand All @@ -57,18 +56,18 @@ skills for mechanics; this skill is the order of operations and the rules three
whenever a leaf is the first to exercise a previously-untested path, and name that path in
the PR body (KT-C §Pain Points, CI).
- **A subagent that says it is "watching CI in the background" has already ended its turn.**
Make it block on `gh run watch --exit-status`, or read the run yourself (lead memory).
Make it block on `gh run watch --exit-status`, or read the run yourself.
- **A drift test that relates two artefacts owned by different leaves is red until both land.**
Order the roadmap so the *naming* side lands first (e.g. the `AGENTS.md` skill table before
the skill leaves it names), or every merge in between is red on a test nobody owns; the
`dev_plugin` campaign had to land its surface leaf ahead of BFS order for this (dev_plugin
KT §Pain Points). Also: git does not track empty directories, so a scaffolded-but-empty
directory never reaches a task worktree — a test that lists it must treat "absent" as "empty".
the skill leaves it names), even ahead of BFS order, or every merge in between is red on a
test nobody owns (KT-D §Pain Points). Also: git does not track empty directories, so a
scaffolded-but-empty directory never reaches a task worktree — a test that lists it must
treat "absent" as "empty".
- **Merge commits are commits too.** Probe every merge subject with `cz check --message` and run
`cz check --rev-range main..HEAD` on the campaign branch before pushing; per-task-branch ranges
never contain the lead's own merges (PR #4 of `dev_plugin` failed CI on one).
- **Note the clock at the start; time-box the cycle.** The three prior cycles closed between
50 minutes and 4 hours (KT-C, KT-H).
never contain the lead's own merges, and CI checks the whole range (KT-D §Pain Points).
- **Note the clock at the start; time-box the cycle.** A cycle here runs between one and four
hours (KT-C, KT-H).

## What This Skill Does Not Restate

Expand Down
6 changes: 3 additions & 3 deletions dev/skills/campaign-lead/references/dirtree-gotchas.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# dirtree-rdm Grammar Gotchas

`managing-roadmaps` owns `dirtree-rdm` itself and its full BNF grammar; this file only lists
the traps this repository's campaigns have actually hit. Before writing any leaf file or
roadmap README by hand, run:
the traps that matter in this repository. Before writing any leaf file or roadmap README by
hand, run:

```bash
bash ~/.claude/skills/managing-roadmaps/scripts/dirtree-rdm.sh grammar leaf
```

Read what it prints — do not guess the shape from a prior campaign's file and hand-edit a new
Read what it prints — do not guess the shape from another campaign's file and hand-edit a new
one to match; the grammar is strict and a mismatch fails validation without necessarily saying
which line is wrong.

Expand Down
33 changes: 16 additions & 17 deletions dev/skills/campaign-lead/references/dispatch-prompt.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,26 @@
# Dispatch-Prompt Template

Copy this template for every implementer you dispatch; fill the `<placeholders>` from the
leaf file and the roadmap README. Every field exists because a prior campaign paid for its
absence, not because the shape looked complete:
leaf file and the roadmap README. Every field answers a recorded failure, not a wish for the
shape to look complete:

- **Worktree/branch, created from the campaign branch** and the **"stop if the leaf file is
missing"** instruction both answer the same failure from two directions: `KT-H` records a
worktree made from `main` instead of the campaign branch (the Agent tool's `isolation:
worktree` does this by default) — the field names the correct base explicitly. When a
worktree still ends up wrong-based, the "stop if missing" line is what turned that mistake
into a ~15-second no-op with zero stray commits, instead of an implementer inventing work
against a repo state the lead never intended (`KT-H` §Wins).
missing"** instruction both answer the same failure from two directions: a worktree made
from `main` instead of the campaign branch (the Agent tool's `isolation: worktree` does this
by default) — the field names the correct base explicitly. When a worktree still ends up
wrong-based, the "stop if missing" line turns that mistake into a ~15-second no-op with zero
stray commits, instead of an implementer inventing work against a repo state the lead never
intended (`KT-H` §Pain Points, §Wins).
- **Read-first list and owned files, plus "commit specs before dispatching"** answer `KT-B`:
specs that existed only in the lead's own working tree were invisible to a worktree that
sees only committed files, and separately an unanchored `.gitignore` rule once hid roadmap
leaf files from git entirely. Naming the exact files to read, and stating that the lead
commits specs first, closes both holes.
a spec that exists only in the lead's own working tree is invisible to a worktree, which
sees only committed files, and an unanchored `.gitignore` rule can hide roadmap leaf files
from git entirely. Naming the exact files to read, and stating that the lead commits specs
first, closes both holes.
- **Gate commands, spelled out verbatim, and the final-report shape asking for gate output
lines** answer `KT-C`: an implementer's own claimed test count is not evidence (one branch
reported 196 passing on a tree that actually collected 206), and a refactor with no named
oracle has nothing for the lead — or a reviewer — to diff against. The template asks for the
exact output line of each gate, not a paraphrase, precisely so the lead re-runs and compares
rather than trusting a summary.
lines** answer `KT-C`: an implementer's own claimed test count is not evidence, and a
refactor with no named oracle has nothing for the lead — or a reviewer — to diff against.
The template asks for the exact output line of each gate, not a paraphrase, precisely so
the lead re-runs and compares rather than trusting a summary.
- **Commit rules and hard-stop time** keep the history and the time-box legible across many
parallel branches the lead did not watch being written in real time.

Expand Down
Loading