Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/ai/design/2026-09-07-feature-local-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
phase: design
title: Local Folder Skill Registry Design
description: Explicit file URL sources with read-only preparation and contained discovery
---

# Local Folder Skill Registry Design

## Architecture

flowchart TD
Input[CLI/config/template string] --> Parse[parse and normalize]
Parse -->|non-file| Git[Git cache preparation]
Parse -->|file URL| Local[read-only local preparation]
Git --> Discover[bounded discovery]
Local --> Discover
Discover --> Install[install target]
Discover --> Index[skills.json]
Remove --> Owned[config/index/contained cache only]

The string map remains the storage boundary. A parser returns the local path for file URLs and null for Git sources; SkillRegistry branches preparation/update and consumers receive the actual prepared root.

## Data Model

parseLocalRegistryPath(source): string | null

Local storage is a canonical file:///absolute/path string. No object migration or provider class is introduced.

## Parsing and Normalization

- Any file: prefix is local and validated strictly with fileURLToPath; malformed values throw.
- Every non-file persisted value is Git. Existence is never a discriminator.
- CLI shorthand is absolute, ./, or ../ and uses realpath then pathToFileURL.
- Templates use their directory as base; configs use the containing config directory.
- Hosted file URLs and unsupported Windows syntax are rejected.
- Canonical paths reject a different ID for the same folder.

## Preparation and Freshness

prepareRegistryRepository retains its per-instance promise map. Git keeps clone/pull/stale-cache behavior. Local validates its directory and skills, emits a local-source message, returns the path, calls no Git helper, and never falls back to cache. Reads remain live.

## Discovery and Containment

A shared routine canonicalizes root and skills, streams direct entries with opendir, enforces a candidate limit, validates names, canonicalizes skill and metadata paths, requires strict containment, stats metadata before a bounded read, and never recurses. Explicit install uses the same containment guard.

Production limits are documented constants based on measured repositories, with no user-facing or test-only configuration surface.

## Flow Integration

- Add-registry normalizes and validates before mutation, rejects duplicates, prepares, then indexes the returned root.
- Add/reconcile requires Git only on the Git branch.
- Find refreshes local entries every call; rebuild partitions GitHub and local sources.
- Update pulls Git caches but validates/reports local sources as live.
- Removal deletes config, index entries, and optionally only an ID-derived contained cache.
- Status formats local file URLs and preserves Git credential sanitization.
- Installed listing infers cache provenance only after containment.

## Errors and Security

Errors cover missing/non-directory/missing-skills/empty sources, escaping symlinks, duplicates, hosted file URLs, and limits. Local operations are read-only by construction; destructive APIs accept IDs only. Registry flows never execute skill content.

## Alternatives

| Option | Decision |
|---|---|
| Existing-directory detection | Reject: ambiguous and state-dependent. |
| Raw relative storage | Reject: cwd-dependent. |
| --local only | Reject: config cannot express type. |
| Object config | Defer: broad migration. |
| Provider hierarchy | Defer: no current third source. |

## Rollback

Removing the parser branch restores Git-only behavior without migrating Git users. No local data migration or source mutation exists.
50 changes: 50 additions & 0 deletions docs/ai/implementation/2026-09-07-feature-local-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
phase: implementation
title: Local Folder Skill Registry Implementation
description: Running implementation record
---

# Implementation

## Setup

- Worktree: .worktrees/feature-local-registry
- Branch: feature-local-registry from fetched origin/main at 60c3bc1.
- npm ci completed; initial npm run build built six projects.
- Task tracing unavailable: npx ai-devkit@latest task list --name local-registry --json returned unknown command task.

## Code Structure

- Parsing: packages/cli/src/util/skill-registry.ts
- Config edges: Config.ts, GlobalConfig.ts, InitTemplate.ts
- Preparation/update: SkillRegistry.ts
- Discovery/install/removal: SkillManager.ts
- Search: SkillIndex.ts
- CLI/status: commands/skill.ts and status.service.ts

## Implementation Log

- T1–T2: Parser/canonicalization/duplicate tests drove the explicit source boundary and base-directory normalization.
- T4: Prep-once/no-Git/cache-fallback tests drove separate read-only local and Git preparation.
- T5: Real temp-directory tests drove direct discovery, entry/file limits, and symlink containment.
- T6–T7: Indexing uses actual local roots and update treats them as live; stale same-ID caches are excluded.
- T8–T9: Removal cleans focused index and only ID-derived cache paths; status/provenance are source-aware.
- T10: Built-CLI e2e registers a relative folder, installs, removes registration, and proves the source remains.

Red evidence included missing parser functions and a missing local-registry module. Green evidence includes focused suites, 1,139 CLI tests, the full workspace suite, and e2e.

## Invariants

- file: is the only persisted local discriminator.
- Local preparation is read-only and never falls back to cache.
- Deletion derives only from ID beneath the owned cache.
- Local skill/metadata paths are canonically contained.
- Local index data is live rather than governed by remote TTL.

## Deviations

The pre-merge simplification audit replaced the exported parsed-source union with a local-path-or-null parser, consolidated cross-scope duplicate detection into the existing normalization pass, removed test-only discovery-limit injection, removed unused discovery fields, required the prepared path at the focused-index call site, and deleted a redundant metadata stat/limit check. Limits remain 10,000 direct entries and 1 MiB per SKILL.md, against a measured built-in baseline of 28 entries and a largest SKILL.md of 7,522 bytes.

## Final Review

The simplified implementation matches the requirements and design. All parser, config, preparation, discovery, install, index, update, removal, status, template, and CLI call sites were traced. No local source path reaches Git or deletion operations; removal remains ID-derived and cache-contained. The audit removed 43 net lines from production and tests in commit 5d61f46 without changing the safety contract. No blocking findings remain.
49 changes: 49 additions & 0 deletions docs/ai/planning/2026-09-07-feature-local-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
phase: planning
title: Local Folder Skill Registry Plan
description: TDD plan for explicit read-only local sources
---

# Plan

## Milestones and Tasks

- [x] **M1 Source boundary**
- [x] T1: TDD source parser and explicit file:/Git classification (AC-01, SR-06, SR-12).
- [x] T2: TDD cwd/config/template normalization and canonical duplicate rejection (AC-02–05).
- [x] T3: Document canonical storage and move/re-add semantics.
- [x] **M2 Runtime flows**
- [x] T4: TDD read-only local preparation, no Git calls, and prep-once compatibility (AC-06, SR-01–05).
- [x] T5: TDD fixture discovery/install, missing/empty errors, containment, direct-only and bounded reads (AC-07, SR-07–10).
- [x] T6: TDD focused/full/seed/TTL index behavior using actual source roots (AC-08).
- [x] T7: TDD selected/local-only/mixed update behavior (AC-09).
- [x] **M3 Removal and surfaces**
- [x] T8: TDD config/index cleanup and ID-derived cache-only deletion; snapshot local fixtures (AC-10, SR-01–04).
- [x] T9: TDD status and installed provenance (AC-11–12, SR-11).
- [x] T10: CLI e2e normalization/error/removal journeys and user docs.
- [x] T11: Reconcile docs; implementation check, coverage, build, tests, lint, e2e, final review.
- [x] **M4 Pre-merge simplification**
- [x] T12: Trace every new abstraction, guard, fallback, and test to a current caller or demonstrated safety trigger.
- [x] T13: Remove unused source/discovery surface, duplicate validation, redundant filesystem work, and implementation-detail assertions.
- [x] T14: Reconcile lifecycle docs and rerun build, full tests, lint, and e2e before push.

## Dependencies

T1 precedes all source-aware flows. T4–T5 precede index/update. T8 remains ID-derived. Documentation follows stable CLI behavior. No external API or database migration exists.

## TDD Evidence

Every production change follows focused red, green, refactor commands recorded in implementation/testing docs. Final evidence: npm run build, npm test, npm run lint, npm run test:e2e, and focused coverage.

## Risks

- Local mutation: snapshot fixtures and spy on Git/write/delete boundaries.
- Symlink escape: canonical containment before read/install.
- Stale search: local entries refresh independently of remote TTL/seed.
- Compatibility: non-file values stay on Git branch.
- Oversized sources: streamed direct iteration and metadata-size limits.
- Scope: no provider hierarchy, watcher, object schema, or Windows/UNC support.

## Progress

All tasks are complete. Final review found no blocking issues. The only validation limitation is that the repository's test:coverage script forwards --coverage as an npm config flag and produces no trustworthy percentage.
72 changes: 72 additions & 0 deletions docs/ai/requirements/2026-09-07-feature-local-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
phase: requirements
title: Local Folder Skill Registries
description: Support canonical, read-only local folders as skill registry sources
---

# Local Folder Skill Registries

## Problem

AI DevKit treats every registry string as Git and clones/pulls it into ~/.ai-devkit/skills. Skill authors cannot consume a registry directly from a local development folder.

## Goals

- Accept absolute paths, ./..., ../..., and explicit file: URLs in skill add-registry.
- Resolve shorthand at registration and persist one canonical absolute file: URL.
- Support local roots across add, reconciliation, discovery, find/index, update, remove, status, and templates.
- Preserve Git behavior and give clear errors for missing, moved, empty, or malformed folders.

## Non-goals

- Windows drive/UNC support in phase 1.
- Watchers, local registry caches, provider hierarchies, or execution of SKILL.md.
- Automatic repair after a folder moves.

## Acceptance Criteria

- **AC-01:** Persisted file: values are unambiguously local; non-file strings retain Git semantics. Invalid file: values never fall through to Git.
- **AC-02:** CLI absolute, ./, and ../ inputs resolve against registration cwd, pass through realpath, and store pathToFileURL(realPath).href.
- **AC-03:** Template-relative paths resolve against the template directory. Manually authored project/global relative paths resolve against their config directory.
- **AC-04:** Canonical identity rejects another registry ID for the same directory. Moving a folder requires re-adding it.
- **AC-05:** Hosted file URLs, unsupported Windows paths, missing/non-directory/unusable/empty roots fail clearly.
- **AC-06:** Local preparation memoizes once per SkillRegistry instance, returns the canonical path, and stays read-only.
- **AC-07:** Add and reconciliation discover/install local skills without requiring Git.
- **AC-08:** Focused/full indexing reads local roots directly; local entries supplement seeds and refresh independently of remote TTL.
- **AC-09:** Update validates local availability and reports a live-filesystem no-op without mutation.
- **AC-10:** Removal deletes config and focused index data; global removal may clean only owned cache data and never the source.
- **AC-11:** Status identifies local registries without misclassifying or redacting them.
- **AC-12:** Installed listing never infers ../ registry IDs from symlinks outside the cache.

## Safety Rules

- **SR-01:** Never clone, pull, checkout, clean, create, copy into, write into, or delete a local registry.
- **SR-02:** Local preparation bypasses all Git operations, including Git-installation checks.
- **SR-03:** Never derive a deletion target from a local source path.
- **SR-04:** Removal may delete only an ID-derived target proven strictly beneath SKILL_CACHE_DIR.
- **SR-05:** Never fall back from an unavailable local source to a same-ID cache.
- **SR-06:** Never detect source type by filesystem existence.
- **SR-07:** Skill directories and SKILL.md must canonically remain beneath registry/skills; reject escaping symlinks.
- **SR-08:** Examine direct skills children only; never recursively search.
- **SR-09:** Bound candidate enumeration and SKILL.md reads with documented limits.
- **SR-10:** Never execute SKILL.md in registry flows.
- **SR-11:** Continue sanitizing credential-bearing Git URLs in status.
- **SR-12:** Reject malformed file: sources as local errors, never Git.

## Success Criteria

- Unit tests cover every safety rule and parser branch.
- Temp-directory adapters prove sources remain unchanged through preparation, index, update, and removal.
- CLI e2e covers normalization and errors.
- Six-project build, full tests, lint, and e2e pass.

## Constraints and Assumptions

- Registry IDs keep org/repo validation; config remains Record<string, string>.
- Installs keep symlink-first/copy-fallback after containment validation; writes target install locations only.
- Local sources are live; search re-enumerates them.
- Production limits use measured evidence and tests exercise those limits directly.

## Questions

All phase-1 choices are resolved by the approved design of record. Windows/UNC, watchers/fingerprints, and structured config are deferred.
58 changes: 58 additions & 0 deletions docs/ai/testing/2026-09-07-feature-local-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
phase: testing
title: Local Folder Skill Registry Testing
description: Safety-first testing strategy
---

# Testing

## Goals

Safety-contract coverage for SR-01–12, temp-directory adapter coverage, CLI e2e path/error coverage, and green Git registry regressions. Tests target observable boundaries and demonstrated guard triggers rather than private result shapes or injectable test-only limits.

## Source and Config

- [x] Absolute, ./, and ../ normalize to canonical file URLs against registration cwd. (AC-01–02)
- [x] Template/config relative paths use their containing file. (AC-03)
- [x] Root symlinks and trailing slashes deduplicate; different IDs are rejected. (AC-04)
- [x] Git URL/SCP values remain Git; malformed/hosted file values fail locally. (AC-01, AC-05, SR-06, SR-12)

## Preparation and Install

- [x] Local preparation returns the root with no Git/write call and memoizes once. (AC-06, SR-01–02)
- [x] Missing/moved, non-directory, missing-skills, and empty roots error clearly without cache fallback. (AC-05, SR-05)
- [x] Valid temp fixture installs without modifying source. (AC-07, SR-01)
- [x] Escaping symlinks are rejected before read/install. (SR-07)
- [x] Nested skills are ignored; candidate/metadata limits fail clearly. (SR-08–09)
- [x] Fixture skill content is never executed. (SR-10)

## Index and Update

- [x] Focused/full indexing uses local roots; seed/TTL paths refresh local entries. (AC-08)
- [x] Selected/local-only/mixed updates report live local sources and make no Git call for them. (AC-09, SR-01–02)
- [x] Missing local update errors without fallback. (SR-05)

## Removal and Display

- [x] Project/global removal deletes config/index but not source. (AC-10, SR-01, SR-03)
- [x] Global removal deletes only contained ID-derived cache data. (SR-04)
- [x] Status displays local and redacts Git credentials. (AC-11, SR-11)
- [x] Listing does not infer escaped cache-relative IDs. (AC-12)

## Fixtures

Tests create isolated temp roots with skills/name/SKILL.md, snapshot source content/metadata, and clean only the test-owned outer temp directory. Escape fixtures point to a second temp root.

## Required Validation

- [x] Focused unit/coverage
- [x] npm run build
- [x] npm test
- [x] npm run lint
- [x] npm run test:e2e

## Results

Post-simplification evidence: npm run build built six projects; npm test passed 2,190 tests across six projects, including 1,142 CLI tests; npm run lint passed with zero errors and two unrelated existing warnings; npm run test:e2e passed 42 tests. The focused local-registry suites passed 139 tests before the full gates. Coverage tooling limitation: npm run test:coverage exits successfully but Nx forwards --coverage as an npm config option, so Vitest runs without a coverage report; a selected-file direct coverage run is not representative because unselected files count as zero.

The pre-merge simplification pass kept all named safety tests, changed the enumeration and metadata-limit tests to exercise production thresholds directly, and removed duplicate no-Git and UI-format assertions.
35 changes: 35 additions & 0 deletions e2e/cli.e2e.ts
Original file line number Diff line number Diff line change
Expand Up @@ -377,6 +377,41 @@ describe('install command', () => {
});

describe('skill command', () => {
it('registers, installs, and removes a relative local registry without deleting it', () => {
const projectDir = createTempProject();
const homeDir = join(projectDir, 'home');
const registryDir = join(projectDir, 'local-registry');
mkdirSync(join(registryDir, 'skills', 'local-test'), { recursive: true });
writeFileSync(join(registryDir, 'skills', 'local-test', 'SKILL.md'), '---\ndescription: local fixture\n---\n');
writeConfigFile(projectDir, {
version: '1.0.0', environments: ['claude'], phases: [], createdAt: new Date().toISOString(),
});

try {
const added = run('skill add-registry local/skills ./local-registry', {
cwd: projectDir, env: { HOME: homeDir },
});
expect(added.exitCode).toBe(0);
const config = JSON.parse(readFileSync(join(projectDir, '.ai-devkit.json'), 'utf8'));
expect(config.registries['local/skills']).toBe(`file://${realpathSync(registryDir)}`);

const installed = run('skill add local/skills local-test', {
cwd: projectDir, env: { HOME: homeDir },
});
expect(installed.exitCode).toBe(0);
expect(existsSync(join(projectDir, '.claude', 'skills', 'local-test', 'SKILL.md'))).toBe(true);

const removed = run('skill remove-registry local/skills', {
cwd: projectDir, env: { HOME: homeDir },
});
expect(removed.exitCode).toBe(0);
expect(existsSync(join(registryDir, 'skills', 'local-test', 'SKILL.md'))).toBe(true);
expect(JSON.parse(readFileSync(join(projectDir, '.ai-devkit.json'), 'utf8')).registries).toEqual({});
} finally {
cleanupTempProject(projectDir);
}
});

it('should list skills (empty)', () => {
const projectDir = createTempProject();
run('init -e claude -p requirements', { cwd: projectDir });
Expand Down
Loading
Loading