feat(index): list, classify and prune stale colgrep indexes with a housekeeping prompt - #9
Merged
Merged
Conversation
…ar probe R01 (index_housekeeping) fixes the contracts before any code: the store root is derived from one `Index:` line, classification is pure and ordered, `index_prune` deletes store directories under a `project.json` guard, and the cycle runs on step commits. R02 records the probe that forced the deletion design: `colgrep clear` on a gone path exits 1 and leaves the index directory in place, so the retired hook's comment was right and a prune cannot delegate to the CLI. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…, age, path and shadowing An agent deciding which of 165 indexes to keep needed one index_status per guess: the facts — does the path exist, how big is the index, when was it last used, does another indexed project already cover it — sit in each index directory's project.json and state.json and never reached the client (index_housekeeping R01 §C2–C4). The new store module reads the store once (40 ms for 165 indexes) and classifies every entry; the store root is the parent of one `Index:` line rather than a platform path (R01 §C1, R02: `status` fails on a gone path, so an existing project is asked). The machine-state roots restate the hook's and a drift test pins them equal (R01 D4). `list_indexes(stale_only=...)`, `IndexInfo` and `colgrep://indexes` carry the new fields; the four legacy tokens stay at the front of each text block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…wed and cold indexes Cleaning the store used to cost one confirmed index_clear per project, dozens of calls on a machine with 65 orphaned indexes (index_housekeeping R01 §Executive Summary). index_prune classifies the whole store once, lists the candidates grouped by class as a dry run by default, and with dry_run=false removes them after confirm=true or an accepted elicitation — the index_clear flow, now one shared helper. It never runs `colgrep clear`: that command exits 1 on a gone path (R02) and clears the folding ancestor on a shadowed one, so the removal is a directory delete guarded by the directory being a direct child of the store and its project.json naming the candidate at deletion time (R01 §C5). `cold` is opt-in because a live but idle project is a judgement call, not dead weight (R01 D8). Both READMEs' tool tables gain the row so test_readme stays green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
doctor is the self-check an agent runs when something looks off, so it is where a stale store gets noticed without a per-session hook — hooks stay at bare-interpreter cost and carry the search rule only (harness_wiring R01 §C3; index_housekeeping R01 §C6, D9). The verdict is a `hints` entry, never a `problems` one: a bloated store does not make the environment not-ok. Shadowed indexes are listed by list_indexes but not nagged about; a machine with no index at all gets no hint; a store whose every project is gone gets INDEX_STORE_UNKNOWN because nothing can be asked for its `Index:` line (R01 §C1). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…prune and check The tools alone still leave an agent to invent the order; the prompt fixes it as list_indexes(stale_only) → index_prune dry run → the confirmed prune with exactly the reviewed classes → list again (index_housekeeping R01 §C7), with `cold` explicitly a judgement to opt into. It takes no `path`, so the shared `complete_path` completion is untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…de and readmes The tools and the prompt exist; the skill is what makes an agent reach for them, and the guide is where it reads the classes and the confirmation flow before a non-trivial call (index_housekeeping R01 §C7). The guide's new section also states why index_prune never runs `colgrep clear` (R02), so an agent does not "help" by clearing a shadowed path and take the ancestor project's index with it. The architecture report's C5 wording now names `store.remove_index_dir`, where the removal actually landed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`gh pr merge --subject "<step subject> (PR #N)"` repeats the step's subject on the merge commit, so v0.3.0 and v0.4.0 each listed one entry twice; in v0.3.0 GitHub had also copied the PR title into the merge body, which commitizen parses as a third line. commitizen applies `changelog_pattern` with `re.match` to the whole message, so the negative lookahead stops at the first newline rather than `$` — a `$`-anchored first attempt passed the subject-only form and changed nothing. Oracle (`maintainer-policy` §Drift tests): `cz changelog --dry-run 0.3.0..0.4.0` with incremental mode off, before and after. Before reproduces the committed CHANGELOG.md byte for byte; the diff after removes exactly the `(PR #7)` line of v0.4.0 and the `(PR #6)` merge's two lines of v0.3.0, nothing else. `probe_cz_check.sh` still meets every expectation; `test_changelog.py` gains a guard that fails against the v0.4.0 pattern and against the `$`-anchored attempt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd the hook gate The read-only dogfood of the classifier on the maintainer's store showed /private/tmp — indexed with 2 434 units, the project every session scratchpad folds into — as a live project: `tempfile.gettempdir()` is `/var/folders/.../T` on macOS, so nothing named /tmp as machine state. `/tmp` is a system temp directory on every POSIX system whatever the per-user one is; it joins the roots in both the hook and the server, so the drift test that pins the two lists equal stays green and the hook's grep gate fails open under /tmp the way it already does under the per-user temp directory. The new test failed against the old roots on macOS (gettempdir and /tmp differ there); on Linux the two coincide and it passed before the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…in its reports index The topic README is where the next cycle reads what landed, what was measured and what was left undone; this fills its Status section before the merge so the branch carries its own record. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seventh campaign,
index_housekeeping(R01/R02 in__reports__/index_housekeeping/). Gives agents the facts and the one tool needed to keep colgrep's index store lean, plus the changelog fix carried from theharness_wiringcycle.What changes for a client
Surface dump-and-diff (
Client.list_tools()/list_resources()/list_resource_templates()/list_prompts()JSON onmainvs this branch):list_indexes:descriptionandinput_schemachanged (newstale_onlyargument). ItsIndexInforows gainpath_exists,size_bytes,last_modified,shadowed_by,stale;IndexListgainsstore_root,total_bytes,total. The four legacy tokens stay at the front of every text block.index_prune(classes, days, max_searches, dry_run=true, confirm=false)— destructive, idempotent, not open-world.doctor: tool listing identical;structured_contentgainshints([INDEX_STORE_STALE] …,[INDEX_STORE_UNKNOWN] …).colgrep://indexes:descriptionchanged; serves the same enrichedIndexList.housekeeping(days="30").search,find_files,expand,index_status,index_build,index_clear, the other resources, the status template and the three existing prompts: byte-identical.INDEX_STORE_UNKNOWN,INDEX_STORE_STALEincolgrep://errors.Why
Measured on the maintainer's machine on 2026-09-13: 165 indexes, 3.3 GiB; 65 orphaned (path gone), 46 shadowed (inside another indexed project), 7 machine-state, 4 cold.
list_indexesshowed none of that, so a clean-up cost oneindex_statusper guess and one confirmedindex_clearper project.The probe that shaped the design (R02)
colgrep clear <gone path>exits 1 withError: No such file or directory (os error 2)and leaves the index directory in place, soindex_prunedeletes store directories itself, guarded by the directory being a direct child of the store root and itsproject.jsonnaming the candidate at deletion time. It never runscolgrep clear(on a shadowed path that would clear the folding ancestor). The store root is the parent of oneIndex:line, never a platform path.Read-only dogfood against the real store
store.index_listover this machine's 165 indexes: 321 ms including the--statsand onestatusspawn; classification 65 orphaned / 8 machine-state / 46 shadowed / 4 cold / 42 live. Nothing was deleted; the realindex_prunewas not run.The first pass classified
/private/tmp(2 434 units, the project every session scratchpad folds into) as live: macOS's per-user temp directory sits under/var/folders, so nothing named/tmpas machine state.fix(index): treat the posix /tmp as machine state in the classifier and the hook gateadds it to the shared roots in both the hook and the server; the regression test failed against the old roots on macOS (on Linux the two directories coincide, so it passed there before the fix).Changelog fix (
build(repo))cz changelog --dry-run 0.3.0..0.4.0with incremental mode off reproduces the committedCHANGELOG.mdbefore the change; after it the diff removes exactly the(PR #7)line of v0.4.0 and the(PR #6)merge's two lines of v0.3.0 (GitHub had copied the PR title into that merge's body). commitizen matcheschangelog_patternagainst the whole message, so the lookahead stops at the first newline; a$-anchored first attempt changed nothing and the new test intest_changelog.pyfails against it.Portability
First exercise of: the hook gate's roots on Windows with the
os.name == "posix"branch skipped, reading a synthetic store undertmp_pathon Windows,Path(index_path).parenton a drive-rootedIndex:line,os.scandirsize walks,shutil.rmtreeof store directories. The pure classifier takeshomeand the machine-state roots explicitly because the Windows runner's temp directory sits under the home directory (R01 risk 2). Read the Windows job, not just the badge.Gates (from
server/)uv run pytest: 306 passed, 1 skippeduv run ruff check: All checks passed!uv run ruff format --check: 46 files already formatteduv run cz check --rev-range main..HEAD: Commit validation: successful!claude plugin validate .and./dev: Validation passeddev/skills/landing-and-release/scripts/probe_cz_check.sh: all expectations metUnfinished
Nothing planned in R01 is left out. Not done, deliberately: the real
index_prunewas not run against the maintainer's store (the PI decides when); nodirtree-rdmroadmap (R01 D10, step commits).🤖 Generated with Claude Code