Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
<!-- Intentionally synchronized with CLAUDE.md. Edit both files together. -->
<!-- Intentionally synchronized with CLAUDE.md. Edit both files together.
v1 docs sweep: AGENTS.md drops session-local spec coordinates that
CLAUDE.md still carries. The substantive guidance is identical. -->

## OpenCodeHub MCP Tools

This repository has been indexed by OpenCodeHub. When you are working in this
Expand All @@ -19,7 +22,7 @@ with the working tree. `codehub status` reports staleness.

## Full MCP surface

The full MCP surface is **28 tools** (see `packages/mcp/src/server.ts`);
The full MCP surface is **29 tools** (see `packages/mcp/src/server.ts`);
the 7 listed above are the high-frequency exploration tools. For the
full inventory, use the `/opencodehub-guide` skill.

Expand All @@ -37,12 +40,11 @@ the list was truncated.

See ADR 0012 (`docs/adr/0012-repo-as-first-class-node.md`) for the
rationale behind `repo_uri` as a first-class node attribute. The
`repo_uri` shape was promoted to a typed graph attribute by AC-M6-1
(`packages/core-types/src/nodes.ts:524-552`). `group_cross_repo_links`
(the AC-M6-3-reframed MCP tool) and the `group_*` family (AC-M6-4) all
emit `repo_uri` in the same canonical form, so a caller can use any of
those tools' `repo_uri` outputs as input to `AMBIGUOUS_REPO.choices`
retries.
`repo_uri` shape is a typed graph attribute on every `Repo` node
(`packages/core-types/src/nodes.ts`). `group_cross_repo_links` and
the `group_*` family of MCP tools all emit `repo_uri` in the same
canonical form, so a caller can use any of those tools' `repo_uri`
outputs as input to `AMBIGUOUS_REPO.choices` retries.

Worked example — error envelope, then retry:

Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,24 @@
# Changelog

## [Unreleased]

### Fixed

- **cli:** `scan` ingests SARIF into the scanned repo, not CWD.
- **cli:** `doctor` resolves native bindings from owner workspaces.
- **smoke-mcp:** asserts 29 tools, matching the v1.0 server surface.

### Docs

- **repo:** README v1.0 status, 29-tool surface, parse-runtime section,
and accurate 17-package list (drops `eval` / `gym`, adds
`cobol-proleap`, `frameworks`, `pack`, `policy`, `wiki`).
- **adr:** cross-link the two concurrently-numbered ADR 0013 files,
flip 0011 + 0013-m7 status to Accepted, and scrub session-local
spec coordinates from ADR text.
- **repo:** sync `CHANGELOG`, `USECASE`, `AGENTS`, and `OBJECTIVES`
with v1 reality (tool count, language count, package set).

## [0.1.1](https://github.com/theagenticguy/opencodehub/compare/root-v0.1.0...root-v0.1.1) (2026-04-22)


Expand Down
19 changes: 10 additions & 9 deletions OBJECTIVES.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ scope.
call.** *Because the README's problem statement is exactly this:
grep is textual, language servers are per-file, embeddings are
lossy; agents need callers, callees, processes, and blast radius
answered before they write a diff, and the 28-tool MCP surface is
answered before they write a diff, and the 29-tool MCP surface is
the primary product.*

2. **Stay Apache-2.0 end-to-end, with every transitive runtime
Expand All @@ -26,21 +26,22 @@ scope.
commit, and `scripts/acceptance.sh` gate 6 gates on exactly that
invariant.*

4. **Cover the 14 GA languages with tree-sitter and upgrade five of
them (TypeScript, Python, Go, Rust, Java) with SCIP indexers.**
4. **Cover the 15 GA languages (14 via tree-sitter plus a regex
provider for fixed-format COBOL) and upgrade five of them
(TypeScript, Python, Go, Rust, Java) with SCIP indexers.**
*Because heuristic call-graph edges miss cross-module resolution,
the `scip-index` phase runs each language's native SCIP indexer
once, the `confidence-demote` phase reconciles heuristic and
compiler-grade edges, and the gym harness gates per-language F1
with SCIP-derived baselines.*
compiler-grade edges, and the gym harness (extracted to a sibling
testbed in M5) gates per-language F1 with SCIP-derived baselines.*

## Quality bar

5. **Hold a three-layer regression gate on every eval and gym run.**
*Because the gym's absolute-F1-floor + relative-F1-delta + per-case
non-regression layering is baked into the harness, and acceptance
gate 9 requires ≥ 40/49 Python-eval cases to pass — soft regressions
are not an option.*
*Because the sibling testbed's absolute-F1-floor + relative-F1-delta
+ per-case non-regression layering is baked into the harness, and
acceptance gate 9 requires ≥ 40/49 Python-eval cases to pass — soft
regressions are not an option.*

6. **Fail CI on any non-zero exit.** *Because `pnpm run check` chains
lint → typecheck → test → banned-strings and exits on first
Expand Down
82 changes: 58 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ flowchart LR
C -->|detect communities + flows| E[Processes / clusters]
D --> F[MCP server]
E --> F
F -->|28 tools| G[AI coding agent]
F -->|29 tools| G[AI coding agent]
```

## Design choices worth knowing
Expand All @@ -71,13 +71,16 @@ flowchart LR
| **Local-first, offline-capable** | `codehub analyze --offline` opens zero sockets. Your code never leaves your machine. No telemetry. |
| **Deterministic indexing** | Identical inputs produce a byte-identical graph hash. Reproducible. Auditable. Cacheable in CI. |
| **MCP-native** | Works out-of-the-box with Claude Code, Cursor, Codex, Windsurf, OpenCode. The MCP server is the primary interface; CLI exists for scripts and CI. |
| **Embedded storage** | DuckDB + `hnsw_acorn` (filter-aware HNSW via ACORN-1 + RaBitQ) + `fts` (BM25). One file. No daemon. No database to operate. |
| **14 languages at GA** | TypeScript, JavaScript, Python, Go, Rust, Java, C#, C, C++, Ruby, Kotlin, Swift, PHP, Dart — via tree-sitter native bindings (WASM fallback for the web surface). |
| **Embedded storage, graph-default** | `@ladybugdb/core` graph engine for the structural store (default at v1) with DuckDB + `hnsw_acorn` (filter-aware HNSW via ACORN-1 + RaBitQ) + `fts` (BM25) for the temporal + retrieval views. Embedded files. No daemon. No database to operate. `CODEHUB_STORE=duck` reverts to the legacy single-file layout. |
| **15 languages at GA** | TypeScript, JavaScript, Python, Go, Rust, Java, C#, C, C++, Ruby, Kotlin, Swift, PHP, Dart, COBOL — tree-sitter for the first 14 plus a regex provider for fixed-format COBOL. |
| **WASM-default parse runtime** | `web-tree-sitter` WASM is the default on Node 22 and Node 24; the native `tree-sitter` N-API addon is opt-in via `OCH_NATIVE_PARSER=1` for Node 22 dev boxes. The complexity phase still uses native where supported and degrades with a one-shot warning otherwise. |

## Quick start

**Requirements:** macOS, Linux, or Windows; Node 20+; pnpm 10+; Python
3.12 (for the eval harness); `mise` recommended to manage them.
**Requirements:** macOS, Linux, or Windows; Node 22 or 24 (Node 22
recommended for the native-parser opt-in); pnpm 10+; Python 3.12 (only
needed when running the SCIP indexers for Python-heavy repos);
`mise` recommended to manage them.

```bash
git clone https://github.com/theagenticguy/opencodehub
Expand Down Expand Up @@ -106,7 +109,7 @@ codehub analyze
# your agent can now call impact, query, context, detect_changes, rename, ...
```

## MCP tool surface (28 tools)
## MCP tool surface (29 tools)

| Tool | Purpose |
|---|---|
Expand All @@ -116,34 +119,43 @@ codehub analyze
| `detect_changes` | Git-diff impact — what do your current changes affect |
| `rename` | Multi-file coordinated rename with confidence-tagged edits |
| `route_map` / `api_impact` / `shape_check` / `tool_map` | HTTP route & MCP tool intelligence |
| `group_query` | BM25-fused search across a group of repos |
| `group_query` / `group_status` / `group_contracts` / `group_cross_repo_links` / `group_sync` / `group_list` | Cross-repo federation — fan out BM25, contracts, and staleness across a named group |
| `list_repos` · `sql` | Registry & escape-hatch SQL (read-only, timeout-guarded) |
| …and 17 more | Communities, processes, SBOM, SARIF, verdict, etc. |
| `pack_codebase` | Deterministic Repomix-compatible code pack export |
| …and the rest | `verdict`, `risk_trends`, `project_profile`, `dependencies`, `license_audit`, `owners`, `list_findings`, `list_findings_delta`, `list_dead_code`, `remove_dead_code`, `scan` |

Architecture decision records live in [`docs/adr/`](./docs/adr/). A
Claude Code plugin at `plugins/opencodehub/` wraps the MCP tools into
slash commands + skills — install via `codehub init`.

## Repository layout

The monorepo is organised as 14 workspace packages under `packages/`:
The monorepo is organised as 17 workspace packages under `packages/`:

| Package | Purpose |
|---|---|
| `analysis` | Heuristic + SCIP call-graph resolution, community + flow detection |
| `cli` | `codehub` command — `init`, `analyze`, `status`, `setup`, scanners |
| `core-types` | Shared TypeScript types, Zod schemas, error codes |
| `embedder` | Embedding backends — local ONNX, HTTP, SageMaker |
| `eval` | Retrieval / graph-quality evaluation harness |
| `gym` | Per-language F1 regression gym with SCIP baselines |
| `ingestion` | Tree-sitter parsers, symbol extraction, import resolution |
| `mcp` | Model Context Protocol server — 28 tools, resources |
| `cli` | `codehub` command — `init`, `analyze`, `status`, `setup`, scanners, group federation |
| `cobol-proleap` | ProLeap-backed deep-parse path for free-format COBOL (regex provider handles fixed-format) |
| `core-types` | Shared TypeScript types, Zod schemas, error codes, canonical `LanguageId` and node/edge kinds |
| `embedder` | Embedding backends — local ONNX, HTTP, SageMaker; deterministic `embedderId` fingerprint |
| `frameworks` | HTTP route + MCP tool detectors used by `route_map` / `api_impact` / `tool_map` |
| `ingestion` | Tree-sitter + WASM parsers, symbol extraction, import resolution, complexity phase |
| `mcp` | Model Context Protocol server — 29 tools, resources, structured error envelopes |
| `pack` | Deterministic Repomix-compatible code-pack generator (M5) |
| `policy` | Allowlist + license-tier policy engine driving `license_audit` and CI gates |
| `sarif` | SARIF schema validation and scanner output normalisation |
| `scanners` | Subprocess wrappers for OSV, Semgrep, hadolint, tflint, etc. |
| `scip-ingest` | SCIP indexer runners (TS, Python, Go, Rust, Java) |
| `scanners` | Subprocess wrappers for 20 scanners — OSV, Semgrep, hadolint, tflint, detect-secrets, and the rest |
| `scip-ingest` | SCIP indexer runners (TS, Python, Go, Rust, Java) — emits CALLS, REFERENCES, IMPLEMENTS, TYPE_OF |
| `search` | Hybrid BM25 + HNSW (ACORN-1 + RaBitQ) query layer |
| `storage` | DuckDB-backed graph store, deterministic `graphHash` |
| `storage` | `IGraphStore` / `ITemporalStore` adapters — `@ladybugdb/core` (default) and DuckDB; deterministic `graphHash` |
| `summarizer` | Process + cluster summaries for MCP responses |
| `wiki` | LLM-narrated module pages emitted by `codehub wiki --llm` |

The retrieval / graph-quality evaluation harness and the per-language F1
regression gym used to live here as `eval` and `gym`; they were
extracted into a sibling testbed in M5 so the production package set
ships free of test-time dependencies.

## Embedding backends

Expand Down Expand Up @@ -199,19 +211,41 @@ for the M3 phase-1 rationale and
[`docs/adr/0013-m7-default-flip-and-abstraction.md`](./docs/adr/0013-m7-default-flip-and-abstraction.md)
for the M7 default-flip + interface segregation.

## Parse runtime — WASM default, native opt-in

`@opencodehub/ingestion` defaults to the `web-tree-sitter` (WASM)
runtime on Node 22 and Node 24. The native `tree-sitter` N-API addon
is opt-in on Node 22 dev boxes via `OCH_NATIVE_PARSER=1` (or
`--native-parser` on the `codehub` CLI). Native is not supported on
Node 24 until `node-tree-sitter@0.25.1` lands on npm
([tree-sitter/node-tree-sitter#276](https://github.com/tree-sitter/node-tree-sitter/issues/276)).

Kotlin, Swift, and Dart use `.wasm` blobs vendored at
`packages/ingestion/vendor/wasms/` and rebuilt via
`bash scripts/build-vendor-wasms.sh` whenever the underlying grammar
versions in `package.json` change. The complexity phase
(cyclomatic-complexity metrics) still uses native tree-sitter where
available; on Node 24 or Node 22 without the opt-in, complexity
extraction degrades with a one-shot stderr warning and all other
parsing continues via WASM.

See [`docs/adr/0013-parse-runtime-wasm-default.md`](./docs/adr/0013-parse-runtime-wasm-default.md)
for the WASM-default rationale and the Node 24 unblock plan.

## Status

**v0.1.0 — initial public release.** The codebase is feature-complete
along the scope described below, but the project is brand-new on
GitHub and the API surface is not yet stable.
**v1 — feature-complete on M1–M7.** Tracks A (M7 graph-DB default + the
`IGraphStore` / `ITemporalStore` interface segregation), B (20-scanner
fleet incl. detect-secrets), C (debt sweep — embedder fingerprint, SCIP
REFERENCES + TYPE_OF), and D (dogfood polish) have all merged. The
current shipped tag remains `0.1.1`; `1.0.0` is cut once schema +
tool-surface stability is signed off.

While on `0.x`, **any release may contain breaking changes** to the
graph schema, MCP tool shapes, CLI flags, or storage layout. Breaking
changes are called out with `!` or a `BREAKING CHANGE:` footer in the
commit log and summarised in each release's generated CHANGELOG.

`1.0.0` will be cut when we commit to schema + tool-surface stability.

## Supply-chain posture

- **CycloneDX SBOM** at [`SBOM.cdx.json`](./SBOM.cdx.json) (regenerated on every release)
Expand Down
5 changes: 4 additions & 1 deletion docs/adr/0011-graph-db-backend.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# ADR 0011 — Graph-DB backend (LadybugDB phase-1)

- Status: **Proposed** — 2026-05-05 (flips to **Accepted** on the M3 merge).
- Status: **Accepted** — 2026-05-05 (Proposed) → flipped on the M3 merge.
- Authors: Laith Al-Saadoon + Claude.
- Branch: `feat/v1-m3-m4`.
- Supersedes nothing. Interacts with ADR 0001 (DuckDB backend stays the
default through M6; this ADR records the opt-in second backend and the
phased plan to flip the default in M7).
- Followed by ADR 0013 (M7 default-flip + interface segregation), which
records the M7 flip from "DuckDB-default + LadybugDB opt-in" to
"LadybugDB-default with auto-fallback to DuckDB".

## Context

Expand Down
29 changes: 17 additions & 12 deletions docs/adr/0013-m7-default-flip-and-abstraction.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
# ADR 0013 — M7 default-flip + storage abstraction (LadybugDB phase-2)

- Status: **Proposed** — 2026-05-09 (flips to **Accepted** on the
`feat/v1-finalize-track-a` merge).
> Note: there is a sibling ADR — `0013-parse-runtime-wasm-default.md` —
> that landed concurrently and shares the same number. Both are kept
> in-tree because they were authored in parallel branches and accepted
> on the same release. The next ADR uses 0014.

- Status: **Accepted** — 2026-05-09 (Proposed) → flipped on the
`feat/v1-finalize-track-a` merge (PR #71).
- Authors: Laith Al-Saadoon + Claude.
- Branch: `feat/v1-finalize-track-a`.
- Supersedes nothing. Extends ADR 0011 (LadybugDB phase-1) by flipping
Expand Down Expand Up @@ -299,11 +304,6 @@ of `@opencodehub/storage` required.

## References

- Spec: `.erpaval/specs/006-v1-finalize/architecture-revised.md`
§AC-A-1 (interface split), §AC-A-2 (column encoders), §AC-A-3
(`ITemporalStore` route), §AC-A-6 (108-SQL migration), §AC-A-7
(parity harness), §AC-A-8 (`describeArtifacts`), §AC-A-9 (this ADR
+ the default flip), §AC-A-11 (conformance suite).
- Code:
- `packages/storage/src/interface.ts` — `IGraphStore` + `ITemporalStore`
type definitions; the typed-finder method surface.
Expand All @@ -328,13 +328,18 @@ of `@opencodehub/storage` required.
- `packages/storage/src/interface.test.ts` — interface-level
contract assertions.
- `packages/storage/src/finders.test.ts` — typed-finder coverage.
- Spec: `.erpaval/specs/006-v1-finalize/architecture-revised.md`
§AC-A-1 (interface split), §AC-A-2 (column encoders), §AC-A-3
(`ITemporalStore` route), §AC-A-6 (108-SQL migration), §AC-A-7
(parity harness), §AC-A-8 (`describeArtifacts`), §AC-A-9 (this ADR
+ the default flip), §AC-A-11 (conformance suite).
- Related ADRs:
- ADR 0001 — DuckDB selection. This ADR keeps DuckDB as the
temporal store and the legacy graph store; no rip-out.
- ADR 0011 — LadybugDB phase-1. This ADR is its M7 follow-up.
- ADR 0012 — Repo as a first-class graph node. The M6 federation
surface routes through the new typed finders via this ADR's
AC-A-6 migration.
108-site SQL migration.

## Provenance

Expand All @@ -356,15 +361,15 @@ checksums because the two artifacts are written by different engines
and have different on-disk representations. mtime is the only stable
signal.

## Empirical evidence — graphHash parity audit (AC-A-10)
## Empirical evidence — graphHash parity audit

The whole-pipeline parity gate is `scripts/m7-parity-audit.sh`. It runs
`codehub analyze --force` against the same corpus under
`CODEHUB_STORE=duck` and `CODEHUB_STORE=lbug`, then compares the
`graph <hash>` summary line emitted by each invocation. This is the
end-to-end companion to the in-memory `assertGraphParity` harness
(AC-A-7); together they pin U1 (graphHash byte-identity) from both
layers — fixtures and a real on-disk analyze.
end-to-end companion to the in-memory `assertGraphParity` harness;
together they pin graphHash byte-identity from both layers — fixtures
and a real on-disk analyze.

The script is wired into `scripts/acceptance.sh` as gate 17 (the final
gate). Sample outputs follow.
Expand Down
5 changes: 5 additions & 0 deletions docs/adr/0013-parse-runtime-wasm-default.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# ADR 0013 — Parse runtime: WASM default, native opt-in

> Note: there is a sibling ADR — `0013-m7-default-flip-and-abstraction.md`
> — that landed concurrently and shares the same number. Both are kept
> in-tree because they were authored in parallel branches and accepted
> on the same release. The next ADR uses 0014.

- Status: **Accepted** — 2026-05-08.
- Authors: Laith Al-Saadoon + Claude.
- Branch: `feat/node24-wasm-default`.
Expand Down
17 changes: 11 additions & 6 deletions docs/adr/0014-scip-references-and-embedder-fingerprint.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,14 @@ For Hole B, every existing store needs a `codehub analyze --force` to pick up th

## References

- `.erpaval/specs/006-v1-finalize/spec.md§AC-C-3, §AC-C-5, §E-C-3, §E-C-4, §W-A-2`
- `.erpaval/sessions/session-33f24f/research-detectsecrets-scip.yaml` (SCIP role enum + Relationship message)
- `.erpaval/solutions/architecture-patterns/scip-callee-definition-site.md`
- `.erpaval/solutions/conventions/scip-0-indexed-vs-graph-1-indexed.md`
- `docs/adr/0011-graph-db-backend.md` (M3+M6 IGraphStore precedent)
- `docs/adr/0013-m7-default-flip-and-abstraction.md` (M7 LadybugDB default flip)
- `packages/embedder/src/fingerprint.ts` — `assertEmbedderCompatible`,
the frozen `EMBEDDER_MISMATCH_HINT` string.
- `packages/scip-ingest/src/derive.ts` — REFERENCES emission and the
`is_implementation`/`is_type_definition` collector.
- `packages/ingestion/src/pipeline/phases/scip-index.ts` — `emitEdges`
and the new `emitRelations` sibling.
- `packages/core-types/src/edges.ts` — append-only `RelationType`
union; `TYPE_OF` lands at position 25.
- `docs/adr/0011-graph-db-backend.md` — `IGraphStore` precedent.
- `docs/adr/0013-m7-default-flip-and-abstraction.md` — M7 LadybugDB
default flip.
Loading
Loading