Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions knowledge/architecture/graph-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ tags:
- components
- graph-runtime
- rust
timestamp: 2026-08-06
timestamp: 2026-08-17
title: Graph Runtime Architecture
type: architecture
---
Expand Down Expand Up @@ -72,7 +72,7 @@ The Materialization API requests these operations but does not publish paths its

## Refresh behavior

The Repository Refresh Service supports continuous and one-shot refresh. It filters and coalesces filesystem events into bounded batches, resolves canonical repository context, normalizes refresh options, and reuses generation-backed materialization. Transient failures are classified and retried with bounded backoff. Runtime entry also gives the janitor an opportunity to recover abandoned work and retry retirement.
The Repository Refresh Service supports continuous and one-shot refresh. Continuous refresh is a cross-process elected role: one nonblocking `refresh.lock` holder performs a manifest catch-up before creating the watcher, while followers remain read-only and retry election with deterministic jitter. Install schema v3 defaults to `refresh.policy = leader`; `off` starts a watcher-free MCP runtime.\n\nThe leader admits supported source and rebuild-triggering configuration events, never admits CodebaseGraph-owned state or storage roots, and collapses churn into one dirty signal plus a bounded path set. Path-count, byte-count, or watcher-channel overflow becomes one full-rescan marker. After the materialization writer lock is held, refresh intent may discard an unchanged candidate without publishing a generation; explicit builds retain their publication semantics. Transient failures are classified and retried with bounded backoff.

## Source evidence

Expand Down
10 changes: 5 additions & 5 deletions knowledge/architecture/graph-storage-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ description: Immutable generation storage, recoverable run workspaces, artifact
resource: repository-architecture
tags:
- architecture
- graph-storage
- generations
- graph-storage
- recovery
- runbook
timestamp: 2026-08-06
timestamp: 2026-08-17
title: Graph Storage Lifecycle and Recovery
type: architecture
---
Expand Down Expand Up @@ -54,7 +54,7 @@ storage_root/
7. **Paths are confined.** Cleanup accepts only expected descendants of the managed root, rejects symlinks and traversal, and never follows links into user data.
8. **Direct paths remain transactional.** Explicit `--db` and `--manifest` targets use adjacent shadow files plus a checksummed recovery journal so the pair cannot remain half-published after a crash.

These rules replace the stale-file write-intent heuristic and all in-place partition deletion or replacement. The compatibility `atomic_rebuild` request field remains accepted but does not re-enable in-place mutation.
These rules replace the stale-file write-intent heuristic and all in-place partition deletion or replacement. The compatibility `atomic_rebuild` request field remains accepted but does not re-enable in-place mutation.\n\n## Refresh ownership\n\n`refresh.lock` is independent from `writer.lock` and `state.lock`. Its nonblocking exclusive holder is the only process allowed to create a repository watcher. Followers do not materialize and retry election every second with up to 250 ms of deterministic jitter; operating-system lock release enables takeover without a persisted leader record. On acquisition, the new leader reconciles the active manifest before it begins watching.\n\nManaged storage places the lock under `storage_root`. Direct storage derives a destination-scoped lock from the explicit database and manifest pair. Lock files reject symlinks. Refresh candidates still acquire the ordinary writer lock for the complete mutation, and an unchanged refresh may close its write session without publishing after comparing against the latest active manifest.

## Run workspace lifecycle

Expand Down Expand Up @@ -90,13 +90,13 @@ Health and materialization output expose:
- active generation;
- reused and rebuilt artifact counts;
- pending run count and cleanup status;
- physical and logical database sizes.
- physical and logical database sizes;\n- refresh role, leader process, pending state, coalesced and overflow counts, deduplicated refreshes, and the latest no-op reason.

A healthy idle managed store reports format v2, one active generation, zero run directories, and `cleanup_pending = false`.

## Recovery runbook

1. Quiesce the repository watcher and any long-lived readers before reinstalling or investigating retirement.
1. Confirm the process reporting refresh role `leader`, then quiesce the repository watcher and any long-lived readers before reinstalling or investigating retirement.
2. Run health and record the storage format, active generation, pending runs, cleanup status, and physical/logical sizes.
3. If a v2 run or publication was interrupted, enter the runtime through health or another repository operation. The janitor will acquire unlocked run journals and recover them before normal work continues.
4. If `cleanup_pending` remains true, confirm no process holds the run or retired-generation lease, then enter the runtime again. Do not manually delete a locked workspace or generation.
Expand Down
6 changes: 3 additions & 3 deletions knowledge/architecture/invariants.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ tags:
- architecture
- constraints
- decisions
- invariants
- graph-storage
timestamp: 2026-08-06
- invariants
timestamp: 2026-08-17
title: Architecture Invariants
type: architecture
---
Expand All @@ -34,7 +34,7 @@ These constraints are the shortest durable test for whether a change still fits
15. **Cleanup is confined and primary errors survive.** Cleanup rejects symlinks and escaping paths, is idempotent, and never masks the failure that caused abort.
16. **Artifacts optimize parsing, not persistence correctness.** Raw partitions are content-addressed across every invalidation dimension; all partitions are assembled deterministically and global semantic enrichment always reruns.
17. **Legacy state is read-only until explicit reinstall.** Schema-v1 reads remain available; mutations return `legacy_storage_requires_reinstall`. Successful reinstall deletes renamed legacy state immediately after validated v2 activation.
18. **Refresh orchestrates rather than reimplements.** Event filtering, batching, recovery, and retry wrap generation-backed materialization instead of duplicating indexing logic.
18. **Refresh orchestrates rather than reimplements.** Event filtering, batching, recovery, and retry wrap generation-backed materialization instead of duplicating indexing logic.\n19. **Refresh ownership and admission are bounded.** One nonblocking refresh lease holder creates the watcher; followers remain read-only. Event churn collapses to one bounded dirty state, overflow forces a full rescan, CodebaseGraph-owned roots are never admitted, and only refresh intent may close an unchanged writer session without publication.

## Knowledge invariants

Expand Down
8 changes: 4 additions & 4 deletions knowledge/architecture/operation-paths.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ tags:
- mcp
- runtime
- storage
timestamp: 2026-08-06
timestamp: 2026-08-17
title: Public Operations and Runtime Paths
type: architecture
---
Expand Down Expand Up @@ -51,17 +51,17 @@ Every repository-scoped operation resolves one `RepoRuntime`: source root, confi

Managed reads resolve `active.json` and lease its generation for the entire operation. Direct reads recover any interrupted paired publication before opening their destinations. Runtime entry also recovers abandoned managed runs and retries pending retirement.

Config schema v2 supplies a managed `storage_root`. Schema-v1 deserialization remains available for reads, but the resolved runtime is not writable until explicit reinstall.
Config schema v3 supplies a managed `storage_root`, refresh policy and backend, and bounded materialization defaults. Schema-v2 remains readable and receives v3 defaults for missing fields. Schema-v1 deserialization remains available for reads, but the resolved runtime is not writable until explicit reinstall.

## Graph read path

Health, schema, helper catalogs, architecture catalogs, search, context, and raw query operations dispatch from the core to the Graph Read Service. Search reads native full-text indexes and applies lexical/entity ranking. Context expands selected relationship profiles. Raw statements are parameterized, single-statement, read-only, and result-bounded.

Health reports storage format, writability, active generation, reused and rebuilt artifacts, pending runs, cleanup status, and physical/logical database sizes.
Health reports storage format, writability, active generation, reused and rebuilt artifacts, pending runs, cleanup status, physical/logical database sizes, and refresh ownership/coalescing/no-op state.

## Lifecycle and refresh paths

Repository installation, reinstallation, client registration, and removal are coordinated by the Repository Lifecycle Service. Continuous or one-shot refresh is coordinated by the Repository Refresh Service, which invokes the same Materialization API used by explicit builds.
Repository installation, reinstallation, client registration, and removal are coordinated by the Repository Lifecycle Service. Continuous or one-shot refresh is coordinated by the Repository Refresh Service, which invokes the same Materialization API used by explicit builds. Under the default `leader` policy, one cross-process lock holder owns the watcher and followers remain read-only standbys; `off` starts MCP without refresh. Refresh-only materialization may return `database_written = false` after the writer lock proves the active generation already consumed the change.

For schema-v1 state, search, context, query, and health remain available. Build, watch, refresh, and install return `legacy_storage_requires_reinstall`. Reinstall moves the legacy state without copying it, restores it after any pre-activation failure, and deletes it immediately after successful v2 activation and validation; there is no grace-period copy.

Expand Down
2 changes: 1 addition & 1 deletion src/adapters/cli/format/help.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ pub(in crate::adapters::cli) fn top_level_help() -> &'static str {
}

pub(crate) fn mcp_help() -> &'static str {
"codebase-graph mcp\n\nUSAGE:\n codebase-graph mcp install [--client <client>] [--scope <scope>] [--config-path <path>] [--client-config-path <path>] [--dry-run] [--json]\n codebase-graph mcp start [--repo-root <path>] [--config <path>] [--db <path>] [--manifest <path>]\n codebase-graph mcp http [--repo-root <path>] [--config <path>] [--db <path>] [--manifest <path>] [--host <host>] [--port <port>] [--path <path>] [--allow-remote] [--auth-token <token>|--auth-token-env <name>]\n\nOPTIONS:\n --repo-root <path> Repository root override; auto-detected when omitted\n --config <path> Setup config path; defaults to .codebaseGraph/config.json\n --db <path> Ladybug database path override\n --manifest <path> Manifest path override\n --host <host> HTTP bind host; defaults to 127.0.0.1\n --port <port> HTTP bind port; defaults to 8765\n --path <path> HTTP endpoint path; defaults to /mcp\n --allow-remote Permit non-local HTTP bind when an auth token is supplied\n --auth-token <token> Bearer token required for HTTP requests\n --auth-token-env <name> Environment variable containing the bearer token"
"codebase-graph mcp\n\nUSAGE:\n codebase-graph mcp install [--client <client>] [--scope <scope>] [--config-path <path>] [--client-config-path <path>] [--dry-run] [--json]\n codebase-graph mcp start [--repo-root <path>] [--config <path>] [--db <path>] [--manifest <path>] [--refresh-policy off|leader] [memory options]\n codebase-graph mcp http [--repo-root <path>] [--config <path>] [--db <path>] [--manifest <path>] [--refresh-policy off|leader] [memory options] [--host <host>] [--port <port>] [--path <path>] [--allow-remote] [--auth-token <token>|--auth-token-env <name>]\n\nOPTIONS:\n --repo-root <path> Repository root override; auto-detected when omitted\n --config <path> Setup config path; defaults to .codebaseGraph/config.json\n --db <path> Ladybug database path override\n --manifest <path> Manifest path override\n --refresh-policy <policy> off or leader; defaults to config or leader\n --worker-memory-mib <n> Worker RSS budget; defaults to config or 768\n --rust-memory-mib <n> Rust working-state budget; defaults to config or 384\n --spill-chunk-mib <n> External-sort chunk budget; defaults to config or 32\n --max-parallelism <n> Materialization worker count; defaults to config or 2\n --host <host> HTTP bind host; defaults to 127.0.0.1\n --port <port> HTTP bind port; defaults to 8765\n --path <path> HTTP endpoint path; defaults to /mcp\n --allow-remote Permit non-local HTTP bind when an auth token is supplied\n --auth-token <token> Bearer token required for HTTP requests\n --auth-token-env <name> Environment variable containing the bearer token"
}

pub(in crate::adapters::cli) fn mcp_install_help() -> &'static str {
Expand Down
10 changes: 9 additions & 1 deletion src/adapters/cli/tests/dispatch_materialize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -485,7 +485,15 @@ fn setup_materializes_graph_and_writes_config() {
&fs::read_to_string(root.join(".codebaseGraph").join("config.json")).unwrap(),
)
.unwrap();
assert_eq!(config["schema_version"], 2);
assert_eq!(config["schema_version"], 3);
assert_eq!(config["refresh"]["policy"], "leader");
assert_eq!(config["refresh"]["backend"], "auto");
assert_eq!(config["materialization"]["include_fts"], true);
assert_eq!(config["materialization"]["semantic_enrichment"], true);
assert_eq!(config["materialization"]["worker_memory_mib"], 768);
assert_eq!(config["materialization"]["rust_memory_mib"], 384);
assert_eq!(config["materialization"]["spill_chunk_mib"], 32);
assert_eq!(config["materialization"]["max_parallelism"], 2);
assert_eq!(config["mcp"]["server_name"], "codebase_graph");
let instructions = fs::read_to_string(root.join("AGENTS.md")).unwrap();
for expected in [
Expand Down
5 changes: 5 additions & 0 deletions src/adapters/cli/tests/fixtures.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,11 @@ pub(super) fn test_http_options(root: PathBuf, auth_token: Option<&str>) -> McpH
db: None,
manifest: None,
api: None,
refresh_policy: None,
worker_memory_mib: None,
rust_memory_mib: None,
spill_chunk_mib: None,
max_parallelism: None,
},
host: "127.0.0.1".to_string(),
port: 8765,
Expand Down
12 changes: 9 additions & 3 deletions src/adapters/cli/tests/install.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ fn install_skips_materialization_when_graph_state_already_exists() {
}

#[test]
fn install_writes_managed_v2_config_without_static_database_or_manifest_paths() {
let root = unique_temp_dir("codebase-graph-rust-install-managed-v2");
fn install_writes_schema_v3_managed_config_without_static_database_or_manifest_paths() {
let root = unique_temp_dir("codebase-graph-rust-install-managed-v3");
fs::create_dir_all(&root).unwrap();
fs::write(root.join("service.py"), "def helper():\n return 1\n").unwrap();

Expand Down Expand Up @@ -87,7 +87,13 @@ fn install_writes_managed_v2_config_without_static_database_or_manifest_paths()
&fs::read_to_string(root.join(".codebaseGraph").join("config.json")).unwrap(),
)
.unwrap();
assert_eq!(config["schema_version"], 2);
assert_eq!(config["schema_version"], 3);
assert_eq!(config["refresh"]["policy"], "leader");
assert_eq!(config["refresh"]["backend"], "auto");
assert_eq!(config["materialization"]["worker_memory_mib"], 768);
assert_eq!(config["materialization"]["rust_memory_mib"], 384);
assert_eq!(config["materialization"]["spill_chunk_mib"], 32);
assert_eq!(config["materialization"]["max_parallelism"], 2);
assert!(config.get("database_path").is_none());
assert!(config.get("manifest_path").is_none());
let expected_storage_root =
Expand Down
Loading