skillguard

A small CLI that inventories Claude Code skills on a device, correlates each one against a pinned open-source catalog (github.com/anthropics/skills), infers its lineage, and surfaces security-relevant capability drift with redacted, deterministic evidence.

It answers four separable questions per skill and never lets one masquerade as another:

Concern	Question	Output
integrity	did the tree differ from a baseline?	unchanged / changed / no-baseline
provenance	what is the likely relationship to a baseline?	baseline-exact / probable-derived / similar / unknown-origin
findings	does a change introduce dangerous capability?	drift + absolute rules
coverage	what could/could not be semantically inspected?	per-file content_analysis: skipped

Install

Pure standard library plus PyYAML; Python 3.11+.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
# or run in place: PYTHONPATH=. python -m skillguard.cli ...

Usage

# 1. Pin a baseline catalog (the only networked step).
skillguard catalog sync            # defaults: anthropics/skills @ main, 3 skills

# 2. Scan local skills against it (offline, deterministic).
skillguard scan                    # human report
skillguard scan --json             # machine-readable
skillguard scan --fail-on high     # CI gate (see exit codes)

Discovery roots default to the global personal root ~/.claude/skills/ and any project root .claude/skills/ found by walking ancestors of the current directory. Override with a repeatable --root, optionally scoped:

skillguard scan --root project=./.claude/skills --root global=~/.claude/skills

By default, catalog state is stored under ~/.skillguard/:

• catalog.lock records the repository, requested ref, resolved commit SHA, and selected skill paths.
• catalog_cache/<commit_sha>/ contains the corresponding extracted skills.

Set SKILLGUARD_HOME to relocate both, or use --lock and --cache to override them individually. The requested ref is retained for context, but scans bind only to the immutable commit_sha.

Policy can be tuned independently by finding severity and detector confidence:

skillguard scan --fail-on medium --min-confidence medium
skillguard scan --fail-on never  # report findings without failing the command
skillguard scan --fail-on-unknown-origin  # reject skills with no credible baseline

For large inventories, human output can be reduced or filtered without changing policy evaluation:

skillguard scan --succinct       # aggregate counts; omit per-skill blocks
skillguard scan --failed-only    # details only for policy-failing skills
skillguard scan --succinct --failed-only

--failed-only uses the active severity, confidence, and unknown-origin policy. Both display options apply only to human output and cannot be combined with --json, whose complete schema remains stable for machine consumers.

Exit codes

• 0 policy passed
• 1 findings crossed the configured thresholds, or an unknown-origin skill was found with --fail-on-unknown-origin
• 2 operational / configuration failure

Finding zero skills currently passes policy because no finding or unknown-origin skill was discovered. Consumers that require a known inventory must also check summary.skills; there is not yet a built-in expected-inventory policy.

How it works

catalog sync (network) ──> content-addressed cache @ commit sha
discover roots ──> fingerprint (raw sha256 + normalized text)
        └─> candidate select ──> whole-tree compare ──> lineage
                └─> security (drift | absolute) ──> redaction ──> report

• Two representations per file. A raw SHA-256 over exact bytes drives tree-exact equality; normalized text drives similarity and scanning. So baseline-exact means tree-exact (paths, file types, exec bits, symlink targets, and bytes), while a whitespace-only edit still reads as highly similar.
• Conservative correlation. With only a few catalog entries, a false "derived" is more damaging than a missed match, so similarity_score (algorithmic) is kept separate from confidence (a policy bucket), and weak evidence lands on similar / unknown-origin.
• Drift vs absolute security rules. Drift rules require a pinned baseline and analyze only content added relative to it (a dangerous command already present upstream is not a local finding). Absolute rules are baseline-free and run on skills that have no usable baseline (similar / unknown-origin). Both scan scripts and the Markdown body, since the body is agent-executable instruction content.
• Detectors are compound syntactic heuristics, not malware detection or data-flow analysis — e.g. "network retrieval piped into a shell", "credential path read + outbound network in the same file". severity (impact if executed) and finding_confidence (detector certainty) are independent axes.

Security posture

• The scanner will not execute skill code and treats every skill directory as hostile input: no-follow file opens, symlinks recorded (and flagged when they escape the skill root) but never followed, per-file/total-tree size caps, SKILL.md size-capped before YAML parsing, yaml.safe_load only, and content-vs-extension binary sniffing. "Not analyzed" is a visible coverage state, never a silent omission.
• Evidence is redacted. A security scanner must not become a second leakage path. Secrets in evidence are replaced by [REDACTED:<type>] with a {type, span} annotation; we never emit the raw secret and never emit sha256(secret) (an unkeyed hash of a low-entropy secret is recoverable).
• Deterministic output. The JSON payload has sorted keys, sorted arrays, relative paths, and stable root identifiers, with no timestamps, absolute paths, or random ids. Machine-specific data lives only in the human footer / --envelope. Output records tool_version, ruleset_version, and matcher thresholds for reproducibility.

Security findings

Claude app skills are not stored under the default Skillguard roots.

Skillguard discovers ~/.claude/skills/ and project .claude/skills/ by default. In the tested Claude app setup, skills managed by the Claude app are not written to those directories (and may exist only in the app or Anthropic's service). A clean Skillguard scan therefore does not mean every skill available to the user was reviewed. Until those skills are exported to a local directory and passed with --root, or Skillguard gains an authenticated app integration, app-managed skills remain an inventory blind spot.

JSON output contract

skillguard scan --json emits schema version 1. Its top-level fields are:

• version and reproducibility metadata: schema_version, tool_version, ruleset_version, catalog, and thresholds
• evaluated policy and result: policy and summary
• runtime name shadowing: precedence
• one record per discovered skill: skills, containing identity, warnings, integrity, provenance, findings, and coverage

Consumers should reject unsupported schema_version values rather than infer a compatible shape. New tool or ruleset versions may change analysis results without changing the JSON structure.

Trust model (important)

Pinning to a commit gives immutability and reproducibility, not trust. The configured Anthropic repository is a policy-selected trust anchor: TLS protects transport and the resolved commit binds subsequent scans to stable content, but neither proves publisher authenticity. baseline-exact therefore means "byte-for-byte identical to the pinned commit" — not "verified origin." Because skills carry no signed provenance today, lineage is inferred, not proven; the tool detects baseline divergence and capability drift, and labels its confidence honestly.

Proposed production evolution

A production installation could establish a verifiable chain of evidence:

1. The publisher signs a manifest containing the source repository, commit, and canonical tree digest.
2. The installer verifies the signature, checks out that exact commit, and recomputes the digest before installing the skill.
3. The installer stores a receipt containing the verified source and installed digest. This receipt becomes the local integrity baseline.
4. Later scans compare the current skill with that receipt and report the exact files that changed.
5. Policy evaluates only the recorded changes and decides whether to warn, block, or require review.

With that chain intact, Skillguard could say “this is the skill installed from this signed source at this commit,” instead of today’s weaker statement that a local tree merely resembles a catalog entry.

For larger installations, this design also enables several optimizations:

• use an exact tree-digest match as a fast path before running similarity and file-level drift analysis;
• precompute catalog digests and matching features once, indexed by skill name and digest, rather than rebuilding them for every local skill;
• cache unchanged file hashes between scans, while periodically performing a full rehash (filesystem metadata alone must not be treated as integrity proof);
• hash independent skills with bounded parallel workers, then sort results before reporting to preserve deterministic output; and
• use filesystem change notifications for quick incremental checks while retaining scheduled full scans as a backstop.

Demo

PYTHONPATH=. python demo/demo.py

Walks one skill through pristine → benign edit → malicious added script, printing stable output and the changing exit code ([0, 0, 1]). The benign middle state is the point: a useful tool distinguishes modified from malicious rather than alarming on every hash mismatch.

Tests

pytest

Covers the acceptance criteria (the three demo states, unknown-origin, reproducibility), determinism (byte-identical JSON, no machine-specific data), redaction (no raw secret / no hash), and hostile inputs (malformed YAML, symlink escape, oversized + binary files, empty root, name collisions).

Architecture

Skillguard follows a linear pipeline:

discover → fingerprint → match → classify → inspect → report

Input and discovery

• cli.py defines the catalog sync and scan commands.
• discovery.py finds local skills and parses their metadata.
• safety.py reads potentially hostile files with strict safety limits.

Baseline comparison

• catalog.py downloads, pins, and loads the baseline catalog.
• fingerprint.py produces exact and normalized tree fingerprints.
• matching.py finds the most likely catalog match.
• lineage.py classifies the inferred relationship to that match.
• integrity.py reports factual differences from the baseline.

Security analysis

• security/rules.py applies drift or baseline-free security rules.
• security/redaction.py removes secrets from finding evidence.

Output and data model

• model.py defines the pipeline's shared data structures.
• report.py produces deterministic JSON, human output, and policy results.

Limitations

Small by design. It scans only local directories, discovers a limited set of locations by default, uses a three-skill default catalog, and infers rather than cryptographically proves provenance. See LIMITATIONS.md for the operational consequences and current enterprise-integration gaps.

Deferred hardening also includes stale-version tracking against a moving upstream, richer tuple-based policy, keyed-HMAC evidence correlation, semantic/data-flow analysis, and binary content analysis (binaries are still safely hashed and reported, just not inspected).