From 4f3a8d40b10954c9ce2252a11956cf6021c6571c Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 12:07:53 -0700 Subject: [PATCH 01/11] Document grant-tag access and quiet public vendor names Align PRODUCT, README, ARCHITECTURE, IMPLEMENTATION, MIGRATION, CHANGELOG, and AUTHZ-DOCUMENT-ACCESS with access_tags + creator and share sugars. Public product surface speaks only of DocumentStore and host stores; third-party memory vendor names stay in packages/* adapter plugins only. --- AGENTS.md | 2 +- ARCHITECTURE.md | 12 +-- CHANGELOG.md | 9 +- IMPLEMENTATION.md | 63 +++++++------- MIGRATION.md | 10 ++- PRODUCT.md | 87 +++++++++---------- README.md | 28 +++--- docs/AUTHZ-DOCUMENT-ACCESS.md | 157 ++++++++++++++++++++++++++++++++++ 8 files changed, 270 insertions(+), 98 deletions(-) create mode 100644 docs/AUTHZ-DOCUMENT-ACCESS.md diff --git a/AGENTS.md b/AGENTS.md index e941598..e39a319 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ CI runs `typecheck` + `test` — both must pass before any push. - `src/ports/` — `DocumentStore` / `SourceProvider` / `MemoryProvider` + fakes - `src/core/` — embed/rerank clients, merge, arktype schemas - `src/db/` + `migrations/` — Drizzle schema + SQL migrations (pgvector, `knowledge.*`) -- `packages/` — optional DocumentStore adapters (`knowledge-adapter-mem0`, `knowledge-adapter-supermemory`); pure fetch, no vendor SDKs in core. Linear tools live in sibling `@corbits/linear`. +- `packages/` — optional DocumentStore adapters (pure fetch; no vendor SDKs in core). Linear tools live in sibling `@corbits/linear`. ## Non-negotiable invariants diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f974b81..53db269 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -43,15 +43,15 @@ HTTP hop. 1. **Who is calling** is the request principal, read off the Interchange context. Clients never send `tenant_id`/`principal_id` — the handlers read - only content fields (title/text/query/k/acl) and take identity from context. + only content fields (title/text/query/limit/access_tags/share) and take identity from context. 2. **What is stored** is opaque data on every record: `tenant_id`, `principal_id`, `created_by_kind` (human/agent/system), `source_class`, and - relations (the edge graph). Every query is scoped by `tenant_id` first, then - the document ACL is matched against the caller's subject. + relations (the edge graph). Every query is scoped by `tenant_id` first; then + document access uses Interchange grant tags (`accessTags` + creator). Cross-tenant isolation is enforced at query time by `tenant_id`; document-level -visibility (allow/block) is enforced on top. This is the trust -model. +access is grant tags via `@intx/authz` (creator always allowed). This is the +trust model. ## Layers @@ -74,7 +74,7 @@ model. the principal, grounds a prompt from hit snippets, calls host-injected `generate`. Optional memory recall when `includeMemory` is true. - `GET /api/knowledge/recent` — recent documents for the caller's scope, - filtered with the same document ACL as local find (visibility + block list). + filtered with the same grant-tag access as local find (`canAccessDocument`). It also returns an in-process `KnowledgePlane` (`add`, `find`, `ask`, `recent`, optional `remember` / `recall`). `ask()` is grant-checked in-process diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a49eb3..ffc4d4a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,11 +19,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `ask` and `recent` use the `find` grant. - **Breaking:** `add` returns `{ documentId }`; find body uses `limit` (not `k`); find wire uses `items` (not `hits`). +- **Breaking:** document access is Interchange **grant tags** (`accessTags` + + creator-always + host `GrantStore`), not the visibility-mode / block-list + mini-ACL. Share sugar only mints tags. See `docs/AUTHZ-DOCUMENT-ACCESS.md`. +- **Breaking:** Postgres baseline is two files (`0001_extensions` + + `0002_knowledge_baseline`) with `access_tags` and no `visibility_*` columns. + Fresh installs only — drop/recreate the knowledge schema on existing DBs. ### Added - Optional `TextExtractor` + `file` XOR `content` on `add` -- `share` sugar on `add` (maps to existing visibility / block ACL) +- `share` sugar on `add` (maps to access tags only: owner, tenant, peers) +- `access_tags` on `knowledge.document` (baseline schema) - `POST /api/knowledge/ask` HTTP route - `MIGRATION.md` hard-cutover notes for in-repo consumers diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 2a43c3d..a81ba46 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -12,7 +12,7 @@ src/ mount-config.ts # KnowledgeConfig + loadKnowledgeConfig() — the mount config config.ts # EngineConfig — the core vector-plane config (db + embed + rerank) knowledge.ts # createKnowledgePlane — add/find/ask/recent against store or pgvector - acl.ts # parseAcl + shared acl_block read-path helpers + acl.ts # resolveAccessTags + canAccessDocument (grant tags) log.ts # getLogger(["knowledge-engine"]) from @intx/log migrations.ts # runKnowledgeMigrations(url) ports/ # DocumentStore / SourceProvider / MemoryProvider + fakes @@ -26,12 +26,11 @@ src/ services/ capture.ts # captureDocument, deriveFromRawCapture — the write path search.ts # hybridSearch and every retrieval-candidate query - timeline.ts # listTimelineEvents — durable recent docs + ACL filter + timeline.ts # listTimelineEvents — durable recent docs + grant-tag filter transform.ts # transform_config CRUD + runTransform (replay) core/ # framework-agnostic (chunking, embed/rerank, merge, schemas) packages/ - knowledge-adapter-mem0/ # DocumentStore backend (Mem0) - knowledge-adapter-supermemory/ # DocumentStore backend (Supermemory) + # optional DocumentStore adapters (not part of public product surface) migrations/ # pgvector schema, applied in filename order by scripts/db-setup.ts scripts/db-setup.ts # idempotent migration runner, tracked in `_migrations` compose.yml # pgvector + Ollama + reranker for local dev @@ -110,17 +109,16 @@ control-plane table — `tenant_id`/`principal_id`/source refs are plain `text`. ### `knowledge_document` The stable logical row for a captured source. Unique on `(tenant_id, adapter, external_ref)` — this triple is the dedupe/identity key -every capture upserts against. Visibility/ACL lives directly on the row: -`visibility_mode` (`'tenant'|'principals'|'source_acl'|'private'`), -`visibility_principal_ids` (jsonb array), `visibility_source_acl` (jsonb -array). `attributes` is a flat jsonb bag of scalars. `last_seen_at` bumps on -every re-capture, even a content-hash NOOP. +every capture upserts against. Document access is **grant tags**: +`access_tags text[]` (resource strings in grant-pattern space; see +`docs/AUTHZ-DOCUMENT-ACCESS.md`). `attributes` is a flat jsonb bag of scalars. +`last_seen_at` bumps on every re-capture, even a content-hash NOOP. ### `knowledge_version` The versioned body of a document. `version` is a monotonic integer scoped to `(document_id, generation)` — **not** globally per-document — per the -`knowledge_version_document_generation_version_uniq` unique index (migration -0009). `status` tracks `'active'|'superseded'|'deprecated'|'archived'|'tombstoned'`; +`knowledge_version_document_generation_version_uniq` unique index (baseline +schema). `status` tracks `'active'|'superseded'|'deprecated'|'archived'|'tombstoned'`; only one `active` row exists per `(document_id, generation)` at a time (the capture path enforces this by flipping the prior active row to `superseded` before inserting a new one). `content_hash` is the NOOP-check key. Attribution @@ -129,8 +127,8 @@ columns: `created_by_principal_id`, `created_by_kind` columns (`authority`, `actor_count`, `has_social_signal`, `source_class`) are a **snapshot computed once at capture time** (`computeAuthority`, never recomputed retroactively). `raw_capture_id` points at the immutable source row -this version was derived from. `generation` (added by migration 0009, -default `'live'`) is the replay-generation tag: the normal add path always writes +this version was derived from. `generation` (default `'live'`) is the +replay-generation tag: the normal add path always writes `'live'`; a replay (`runTransform`) writes its own `transform_run.id` instead, so a replayed corpus's versions never collide with, or even become visible alongside, the live ones unless a caller explicitly searches that generation. @@ -306,7 +304,7 @@ returns the run summary either way. - Existing document, content changed → flip the prior active version to `status: "superseded"`, insert a new version at `version + 1` with `supersedesVersionId` pointing at it, update the document's mutable - fields (`title`, `visibility*`, `attributes`, `last_seen_at`). + fields (`title`, `access_tags`, `attributes`, `last_seen_at`). 4. **`insertChunksAndGraph`**: inserts every plan chunk fresh (chunks are never reused across versions), then best-effort upserts entity hints (`upsertEntity`) and edge hints (`upsertEdge`) — these are independent of @@ -362,7 +360,8 @@ search); otherwise it throws `KnowledgeSearchInputError` (400). bound as a `regconfig` parameter, over `knowledge_chunk.text_fts`), joined to `knowledge_version` (filtered to `status = 'active'` and the resolved `generation`) and `knowledge_document` - (filtered by `visibilityPredicateSql`), optionally further filtered by + (tenant-scoped only — document access is grant-tag post-filter in the plane), + optionally further filtered by `kinds` and/or `entityIds` (via a sub-select against `knowledge_edge`). Overfetches up to `overfetchLimit` rows, non-deduped, per-chunk. 3. **Dense channel** — `fetchDenseCandidates`: embeds the query @@ -374,9 +373,8 @@ search); otherwise it throws `KnowledgeSearchInputError` (400). so the halfvec HNSW index is used) against that table joined back to `knowledge_chunk`/`knowledge_version`/`knowledge_document` with the - **exact same visibility predicate**, hand-mirrored as - `VISIBILITY_PREDICATE_RAW_SQL` (there is no third ACL implementation - anywhere). Returns `null` (not an error) when there's no active embed + **same tenant-only scope** as the lexical channel (no mini-ACL in SQL). + Returns `null` (not an error) when there's no active embed model yet or the query is empty; a thrown error from the embed call or the SQL itself is caught by the caller and also folds into `null`/degraded — the dense channel never fails the whole search. @@ -474,16 +472,16 @@ so knowing it will flip the tenant's live dense channel too. `mountKnowledgeEngine` mounts these onto the host app. Identity is the request principal read off the Interchange context (`caller(c)` → `{ scopeId: principal.tenantId, subjectId: principal.id }`); clients never send -`tenant_id`/`principal_id` — the handlers only read title/text/query/limit/acl. +`tenant_id`/`principal_id` — the handlers only read title/text/query/limit/access_tags/share. Each route is guarded with `grantGuard(deps, action)`, which applies the host's `requireGrant("knowledge", action)` when provided (else a pass-through). | Method + path | Grant action | Request body | Response | |---|---|---|---| -| `POST /api/knowledge/add` | `add` | `{ title, text, acl? }` | `200 { documentId }`; `400` on validation | +| `POST /api/knowledge/add` | `add` | `{ title, text, access_tags?, share? }` | `200 { documentId }`; `400` on validation | | `POST /api/knowledge/find` | `find` | `{ query, limit?, kinds?, entity_ids? }` (limit 1–50; `kinds`/`entity_ids` narrow every retrieval channel — lexical and dense — to a document `kind` or linked entity id before fusion; unset or `[]` = unfiltered) | `200 { items[], evidence?, degraded? }`; `400` on bad input | | `POST /api/knowledge/ask` | `find` | `{ query, limit? }` (1–50) | `200 { text, citations[], evidence }`; `403` / `501` as plane errors | -| `GET /api/knowledge/recent` | `find` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope (`last_seen_at` DESC), filtered with the same visibility SQL + `acl_block` post-filter as find. One event per document (active live version). | +| `GET /api/knowledge/recent` | `find` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope, filtered with grant-tag access (`canAccessDocument`). One event per document (active live version). | `mountKnowledgeRoutes` and `mountKnowledgeEngine` mount the four HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). @@ -501,17 +499,22 @@ timeline maps different columns: | `tenantId` | `knowledge_document.tenant_id` | | `principalId` | `knowledge_version.created_by_principal_id` of the active live version (empty string when null) — the capturing actor stored on the version, not the request principal of a later timeline read | -### ACL validators +### Document access (grant tags) -The document ACL (`acl` on capture) is validated by `parseAcl` — mode -`scope|tenant|private|allowlist`, `subjects` only (groups/grants rejected until -membership lands). `parseAcl` is the single write-path ACL validator. +Document access is Interchange authz — **not** a mini-ACL. -Read-path block lists use one gate: **`readBlockList`** (search via -`blockedDocumentIds`, timeline via `timelineRowBlock` / -`filterTimelineRowsForPrincipal`). Both paths share fail-closed semantics — -native jsonb arrays, JSON strings, unreadable shapes, and missing search rows -are withheld. There is no second, weaker interpreter of membership. +- Write path: `resolveAccessTags` always writes `knowledge.owner:` and + merges optional `accessTags` / share sugar (`tenant`, peer `principals`, + explicit `tags`). Stored on `knowledge.document.access_tags`. +- Read path (find + recent): `canAccessDocument` — creator always allowed; + otherwise `authorize(grantStore, principal, tenant, tag, "find")` for any + tag on the document. +- SQL retrieval is **tenant-scoped only**. Document access is grant-tag + post-filter in the plane (`canAccessDocument`); there is no SQL mini-ACL. +- HTTP `POST /add` accepts `access_tags` and/or `share` — not product `acl` + modes or block lists. + +See `docs/AUTHZ-DOCUMENT-ACCESS.md`. ## Observability diff --git a/MIGRATION.md b/MIGRATION.md index 3c1939e..3b9c1a6 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -24,7 +24,8 @@ Identity fields on every call: - Returns `{ documentId }` only (no `status` / `versionId` / `chunks` on the public result). - Exactly one of `content: { title, text }` or `file: { bytes, mimeType?, filename? }`. - File ingest requires a host-supplied `textExtractor` on the plane options. -- Optional `share` sugar (`private` / `tenant` / `principals`) maps onto the existing ACL path. Do not pass `share` and `visibility` together. +- Optional `share` sugar (`private` / `tenant` / `principals`) maps to **access tags** + only (see `docs/AUTHZ-DOCUMENT-ACCESS.md`). There is no separate `visibility` field. ### `find` @@ -50,7 +51,7 @@ Old paths return **404**. No redirect, no dual mount. ### Wire body / response deltas -- **add** request: still `{ title, text, acl? }`. Response: `{ documentId }` (dropped `status: "captured"`). +- **add** request: `{ title, text, access_tags?, share? }`. Response: `{ documentId }` (dropped `status: "captured"`). - **find** request: `{ query, limit? }` (`k` is no longer accepted). Response: `{ items, evidence?, degraded? }` (was `{ hits, evidence, degraded? }`). - **recent** response: unchanged `{ events: [...] }`. - **ask** request: `{ query, limit? }`. Response: `{ text, citations, evidence }`. @@ -74,3 +75,8 @@ In-process `ask()` also checks `knowledge` / `find` (was `search`). 3. Rewrite grant rules: `capture`→`add`, `search`→`find`. 4. Drop any reliance on `status: "captured"` or `hits` / `k` on the wire. 5. If you use file capture, pass `textExtractor` into `createKnowledgePlane` / mount options. +6. Document access is grant tags (`accessTags` + creator + host `GrantStore`), not + visibility modes or block lists. Update any host code that wrote `visibility`. +7. Fresh Postgres: baseline migrations are `0001_extensions.sql` + + `0002_knowledge_baseline.sql` (schema `knowledge`, `access_tags` on document). + Existing DBs: drop/recreate the knowledge schema (no in-place dual-write migration). diff --git a/PRODUCT.md b/PRODUCT.md index e94d9b2..55d56d3 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -38,14 +38,15 @@ Identity on the plane is always **`principalId` + `tenantId`** (never | Port | Purpose | | --- | --- | -| `DocumentStore` | **The** durable backend for add/find/recent (default: engine pgvector, wrapped as a DocumentStore). Replace with Mem0, Supermemory, or fakes — no Postgres required when overridden. The plane is always store-backed; there is no second engine-only path. | +| `DocumentStore` | **The** durable backend for add/find/recent (default: engine pgvector, wrapped as a DocumentStore). Replace with any host `DocumentStore` or in-package fakes — no Postgres required when overridden. The plane is always store-backed; there is no second engine-only path. | | `SourceProvider` | Optional **tools-shaped** live search (`searchLive`); merge is fail-soft. Not a store replacement. | | `MemoryProvider` | Optional ask side-channel only (`includeMemory`); **not** how you swap backends. | Mount options accept `documentStore`, `sources[]`, `memory`, plus in-package **fakes** so a host can mount with fakes only and exercise add/find/ask/recent -without Postgres. Hosts that want Mem0 or Supermemory as the sole durable store -pass that adapter as `documentStore` and omit `KnowledgeConfig`. +without Postgres. Hosts that want a third-party durable backend implement +`DocumentStore` (or use an optional adapter package) and pass it as +`documentStore`, omitting `KnowledgeConfig` when Postgres is not needed. **MergeLocalLiveV1** merges local DocumentStore + live SourceProviders: fail-soft per provider (timeout/error → degrade flags, never fail the request), dedupe by @@ -54,27 +55,22 @@ per provider (timeout/error → degrade flags, never fail the request), dedupe b **Memory side-channel:** `includeMemory` on `ask` defaults **false**. When true and a `MemoryProvider` is mounted, recall injects uncited personal context; failures degrade with `memory_unavailable` (docs-only). This is unrelated to -using Mem0/Supermemory as the DocumentStore. Writes via `plane.remember` are -host-owned — ask never auto-writes. +replacing the DocumentStore. Writes via `plane.remember` are host-owned — ask +never auto-writes. -### Adapter packages (same monorepo tree) +### Optional adapter packages -| Package | Role | -| --- | --- | -| `@corbits/knowledge-adapter-mem0` | **DocumentStore** via Mem0 Platform HTTP; tenant key `mapUser` length-prefixed | -| `@corbits/knowledge-adapter-supermemory` | **DocumentStore** via Supermemory HTTP; tenant key `containerTag` length-prefixed | - -Linear tools live in a **sibling repo** ([`@corbits/linear`](https://github.com/corbitsdev/corbits-linear)) — same tools shape as Granola, not a DocumentStore. +Optional `DocumentStore` implementations live under `packages/` in this tree +(or as separate packages). Core never imports vendor SDKs. Hosts that need a +store beyond default pgvector mount their own `documentStore`. -Core never imports vendor SDKs. Adapters are pure-fetch; tenant-safe keys only. -`MemoryProvider` factories in the mem0/supermemory packages are back-compat only. - -**Vendor store honesty:** Mem0/Supermemory adapters isolate by **principal -bucket** (one Mem0 `user_id` / Supermemory `containerTag` per tenant+principal). -They do **not** implement the multi-principal / tenant visibility ladder or -block lists — those need the default pgvector store (or a store that enforces -them). `recent` is empty on both adapters. Never mount them as -`options.memory`. +**Third-party store honesty:** not every `DocumentStore` evaluates host grant +tags. Some isolate by **principal bucket** only (one private namespace per +tenant+principal) and do **not** multi-share via `accessTags` + host +`GrantStore`. For full grant-tag ACL, use the default pgvector store (or a store +that implements the contract in `docs/AUTHZ-DOCUMENT-ACCESS.md`). Adapter +packages must document their isolation model in their own README. Never mount a +durable store as `options.memory`. ### What is not in scope @@ -97,8 +93,8 @@ Claude Code / Codex / Workbench (clients) │ Host Interchange createApp │ │ + mountKnowledgeEngine(app, opts) │ │ grants: knowledge:add | knowledge:find │ -│ documentStore: pgvector | Mem0 | SM | │ -│ fake │ +│ documentStore: pgvector | host store │ +│ | fake │ │ optional: sources, memory, │ │ textExtractor │ │ │ in-process │ @@ -108,16 +104,26 @@ Claude Code / Codex / Workbench (clients) └──────────────────────────────────────────────┘ ``` -## Identity and ACL ladder (honest) +## Identity and access (Interchange authz — one system) + +Knowledge does **not** ship a second ACL. Document access uses the host’s +`@intx/authz` grant store — the same grants/roles as the rest of Interchange. -1. **Capability** — host grant store: may this principal `knowledge:add` or - `knowledge:find` at all? -2. **Document visibility** — modes `private` | `principals` | `tenant` - (optional block list). Self-contained on the document row. -3. **Share sugars on add** — map to existing visibility; no new ACL system. +1. **Capability** — may this principal use knowledge at all? + `authorize(…, resource: "knowledge", action: "add" | "find")`. +2. **Document access** — each document carries **`accessTags`** (resource strings + in grant-pattern space). A principal sees a document if they are the creator + **or** `authorize(…, resource: , action: "find")` allows for any tag on + the document. Patterns (`knowledge.space:*`) work via `@intx/authz`. +3. **Share sugars on add** — only mint tags (owner / tenant / peer owner tags / + explicit tags). They do **not** invent visibility modes or block lists. -There is **no** dual grant path and **no** second secret ACL. If a connector -cannot prove a principal set, it must not write `tenant` visibility. +Default add is **owner-only** (`knowledge.owner:` + creator rule). +Deny is absence of allow (or a more specific host deny grant) — not a document +block list. Full design: `docs/AUTHZ-DOCUMENT-ACCESS.md`. + +Third-party DocumentStores may be **principal-bucket** only and not evaluate +host grants; that limit belongs in the adapter's own docs, not hidden here. ### On the wire @@ -125,7 +131,7 @@ HTTP bodies are a thin subset of the in-process plane (identity always comes from the host principal context, never the body): ```http -POST /api/knowledge/add { "title", "text", "acl"? } +POST /api/knowledge/add { "title", "text", "access_tags"?, "share"? } POST /api/knowledge/find { "query", "limit?", "kinds?", "entity_ids?", "sources?", "includeEvidence?" } POST /api/knowledge/ask { "query", "limit?", "sources?", "includeMemory?" } GET /api/knowledge/recent ?limit= @@ -134,22 +140,17 @@ GET /api/knowledge/recent ?limit= `kinds` / `entity_ids` on find narrow both lexical and dense channels before fusion (unset or `[]` = no filter). -Plane-only shapes (`content`/`file` XOR, `share` sugars, full `visibility`) -are available via `createKnowledgePlane` / `plane.add`. HTTP `acl` maps to the -document visibility ladder; `share` is plane sugar only. - ### Live sources and memory (trust) -- **Local documents** are the durable ACL plane (visibility + block). Engine - path enforces this; a host-supplied `DocumentStore` **owns** ACL for that - mount — the engine does not re-filter store results. -- **Live `SourceProvider` hits** merge into find/ask without document-row ACL. - Auth is the host token / connector scope, not Interchange principal - visibility. Treat live as enrichment; fail-soft on timeout/error. +- **Local documents** are grant-tagged; default engine evaluates tags via the + host `GrantStore`. An injected `DocumentStore` owns enforcement for that mount. +- **Live `SourceProvider` hits** merge into find/ask without grant tags. Auth is + the host token / connector scope. Treat live as enrichment; fail-soft. - **Memory** is opt-in recall (`includeMemory`, default false). Adapters must key by injective tenant+principal encodings; ask never auto-writes memory. ## Out of scope forever here -Auth, OAuth for Linear, Mem0/Supermemory account management, embedding models +Auth, OAuth for Linear, third-party memory/account management, embedding models in-process, and any standalone process entrypoint. + diff --git a/README.md b/README.md index 0489713..976aa1e 100644 --- a/README.md +++ b/README.md @@ -92,22 +92,20 @@ const answer = await knowledge.ask({ failure → `memory_unavailable`, docs-only. `plane.remember` / `plane.recall` for host-owned writes (ask never auto-remembers). -### Adapter packages +### Optional DocumentStore adapters -```ts -// packages/knowledge-adapter-mem0 — DocumentStore (not MemoryProvider) -import { createMem0DocumentStore } from "@corbits/knowledge-adapter-mem0"; - -// packages/knowledge-adapter-supermemory — DocumentStore -import { createSupermemoryDocumentStore } from "@corbits/knowledge-adapter-supermemory"; +Optional adapter packages under `packages/` implement `DocumentStore` for hosts +that want a non-pgvector backend. Core never imports vendor SDKs. See each +package's README for mount examples and isolation limits. +```ts // Linear SourceProvider lives in sibling repo @corbits/linear // (https://github.com/corbitsdev/corbits-linear), not this monorepo. import { createLinearSourceProvider, mapLinearWebhook, } from "@corbits/linear"; -// host owns OAuth + webhook verify; private issues never map to tenant visibility +// host owns OAuth + webhook verify; private issues never map into tenant docs ``` ## Migrations @@ -122,11 +120,13 @@ All tables live under Postgres schema **`knowledge`** (`knowledge.document`, `knowledge.version`, `knowledge.chunk`, …). Hard cutover pre-1.0: re-run migrations on a fresh knowledge DB. -## ACL ladder (honest) +## Document access (grant tags) + +1. Host capability grants (`knowledge:add` / `knowledge:find`) +2. Per-document **access tags** + creator rule (Interchange `@intx/authz`) +3. Share sugars on `add` only mint tags — no visibility modes or block lists -1. Host grants (`knowledge:add` / `knowledge:find`) -2. Per-document visibility (`private` | `principals` | `tenant` + optional blocks) -3. Share sugars on `add` map onto (2) — no second ACL system +Full design: `docs/AUTHZ-DOCUMENT-ACCESS.md`. ## Docs @@ -141,9 +141,7 @@ pre-1.0: re-run migrations on a fresh knowledge DB. bun install bun run typecheck bun run test -# adapter packages (DocumentStore backends): -bun test packages/knowledge-adapter-mem0 -bun test packages/knowledge-adapter-supermemory +# optional DocumentStore adapters under packages/ (each package has its own tests) # Linear tools: sibling repo @corbits/linear ``` diff --git a/docs/AUTHZ-DOCUMENT-ACCESS.md b/docs/AUTHZ-DOCUMENT-ACCESS.md new file mode 100644 index 0000000..f6e5871 --- /dev/null +++ b/docs/AUTHZ-DOCUMENT-ACCESS.md @@ -0,0 +1,157 @@ +# Document access = Interchange authz (not a second ACL) + +**Status:** shipped hard cutover (grant tags + creator; baseline migrations) +**Problem (historical):** the green plane shipped a **mini-ACL** (`visibility` mode + principal list + block list) that was parallel to Interchange grants. That path is removed. + + +## Source of truth + +| Layer | Owner | Mechanism | +| --- | --- | --- | +| Who is the caller? | Host (Interchange) | `principal` + `tenant` on context / plane args | +| May they use knowledge at all? | Host grant store | `authorize(…, resource: "knowledge", action: "add" \| "find")` | +| Which **documents** may they see? | Host grant store + tags on the document | `authorize(…, resource: , action: "find")` for any tag on the doc | + +There is **one** authorization system: `@intx/authz` + host `GrantStore`. The knowledge engine does not invent modes, allowlists, or block lists as a security boundary. + +## Document model + +Each document stores: + +```ts +accessTags: string[] // resource strings in grant-pattern space +createdByPrincipalId: string // audit + default owner tag +``` + +### Default tags on `add` + +When the caller does not pass tags/share: + +1. Always tag: `knowledge.owner:` +2. No other tags → **owner-only** (only principals granted `find` on that owner resource can see it; typically the owner has that grant, or the plane treats owner as implicit allow for the creating principal) + +**Implicit owner rule (engine convenience, not a second ACL):** the creating principal may always find/recent their own documents without a matching grant tag. Everyone else must match a tag via `authorize`. This is equivalent to auto-issuing a creator grant without writing one. + +### Explicit tags + +```ts +plane.add({ + tenantId, principalId, title, text, + accessTags: ["knowledge.space:eng", "knowledge.project:ke"], +}) +``` + +Host issues grants such as: + +```ts +{ + principalId: "alice", + resource: "knowledge.space:eng", + action: "find", + effect: "allow", + // …origin, etc. +} +``` + +Alice then sees any document tagged `knowledge.space:eng` (capability `knowledge`/`find` still required). + +Patterns work: a grant on `knowledge.space:*` matches `knowledge.space:eng` via `@intx/authz` `matchPattern`. + +### Share sugars (map to tags only) + +Share helpers **must not** reintroduce visibility modes. They only mint tags: + +| Sugar | Tags written | +| --- | --- | +| (default / private) | `knowledge.owner:` | +| `share: { tenant: true }` | `knowledge.owner:`, `knowledge.tenant:` | +| `share: { principals: ["p2","p3"] }` | `knowledge.owner:`, `knowledge.owner:p2`, `knowledge.owner:p3` | +| `share: { tags: ["knowledge.space:eng"] }` | `knowledge.owner:`, plus those tags | + +**Removed:** `visibility: { mode: private|principals|tenant }`, `blockPrincipalIds`, product `acl.mode` / `acl.allow` / `acl.block` as security. + +Deny is expressed as **absence of allow** (or an explicit deny grant in the host store with higher specificity) — not a document-row block list. + +## Evaluation algorithm + +### Capability (unchanged) + +```ts +authorize(grantStore, principalId, tenantId, "knowledge", "find"|"add") +// effect must be "allow" +``` + +### Document access (new) + +```ts +function canSeeDocument(doc, principalId, grantStore, tenantId): + if doc.createdByPrincipalId === principalId: + return true // creator + for tag of doc.accessTags: + r = authorize(grantStore, principalId, tenantId, tag, "find") + if r.effect === "allow": + return true + return false +``` + +**SQL / store path:** prefer expand-then-filter: + +1. `collectGrants(principalId, tenantId)` once per request. +2. Keep allow-grants whose `action` matches `find` (exact or pattern). +3. Document is visible if creator **or** any `accessTags[i]` is matched by any allow grant resource pattern (`matchPattern(grant.resource, tag)`), and not denied by a more specific deny. + +This keeps evaluation inside Interchange authz semantics (specificity, conditions, deny). + +Injected `DocumentStore` backends still own enforcement for their mount; the **contract** is grant-tags + creator, not visibility modes. Some third-party stores are principal-bucket only and must document that they do not evaluate host grants. + +## Live sources + +Unchanged intentional tradeoff: live `SourceProvider` hits are **enrichment under host tokens**, not grant-tagged documents. Hosts that need local-only retrieval pass `sources: ["local"]`. Do not invent a second live ACL. + +## Wire (HTTP) + +`POST …/add` body (thin): + +```json +{ + "title": "…", + "text": "…", + "access_tags": ["knowledge.space:eng"], + "share": { "tenant": true, "principals": ["alice"], "tags": ["knowledge.space:eng"] } +} +``` + +Identity never in body. `access_tags` and `share` are optional; default owner-only. + +`find` / `ask` / `recent` need no ACL body — principal from context + grant store. + +## Schema + +Document access is stored as: + +| Column | Role | +| --- | --- | +| `access_tags text[]` | Resource strings in grant-pattern space (+ creator always allowed) | + +There is no `visibility_mode`, principal-id array, block list, or dual-write ACL +column. Share sugar only mints tags via `resolveAccessTags`. + +Fresh databases apply the baseline migrations (`0001_extensions.sql` + +`0002_knowledge_baseline.sql`) with `access_tags` from day one. + +## Non-goals + +- Group membership resolution inside the knowledge engine (host/roles issue grants). +- Per-chunk ACL. +- Live channel grant tags (host policy). +- Re-implementing roles inside this package. + +## Acceptance + +1. No public plane/HTTP API accepts `visibility` mode or block list as security. +2. Default add is owner-visible only (creator + owner tag). +3. Principal B sees A’s doc only when host grant allows `find` on a tag present on the doc (or B is creator). +4. Capability `knowledge`/`find` still required for find/ask/recent. +5. PRODUCT.md / MIGRATION.md / README describe grant tags, not mini-ACL. +6. Engine + fakes enforce the algorithm; vendor adapters document principal-bucket limit. +7. `bun run typecheck && bun run test` green. From 8912b646c0521f891feab049fa303c723faeb743 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 14:53:33 -0700 Subject: [PATCH 02/11] Document host contract for peer share via grant tags MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fleet review (grant-tag tip): make multi-user share honest — tag minting is not grant minting; hosts must bootstrap find on knowledge.owner:. Remove invented share.private sugar from MIGRATION; drop stale "visibility logic" wording in IMPLEMENTATION. --- IMPLEMENTATION.md | 2 +- MIGRATION.md | 5 +++-- PRODUCT.md | 4 ++++ docs/AUTHZ-DOCUMENT-ACCESS.md | 28 +++++++++++++++++++++++++--- 4 files changed, 33 insertions(+), 6 deletions(-) diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index a81ba46..c74ef0c 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -422,7 +422,7 @@ search); otherwise it throws `KnowledgeSearchInputError` (400). AUTHORITY_STRONG_FLOOR (0.3)`; else `"weak"`. Tenant isolation is unconditional and first in every query (`tenant_id` -filtered before any visibility logic); every table/channel is scoped that +filtered before grant-tag / creator document access); every table/channel is scoped that way, with no exception. ## Raw + replay (the replay pipeline — `services/transform.ts`) diff --git a/MIGRATION.md b/MIGRATION.md index 3b9c1a6..6df1093 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -24,8 +24,9 @@ Identity fields on every call: - Returns `{ documentId }` only (no `status` / `versionId` / `chunks` on the public result). - Exactly one of `content: { title, text }` or `file: { bytes, mimeType?, filename? }`. - File ingest requires a host-supplied `textExtractor` on the plane options. -- Optional `share` sugar (`private` / `tenant` / `principals`) maps to **access tags** - only (see `docs/AUTHZ-DOCUMENT-ACCESS.md`). There is no separate `visibility` field. +- Optional `share` sugar (`tenant` / `principals` / `tags`) maps to **access tags** + only (see `docs/AUTHZ-DOCUMENT-ACCESS.md`). Omit `share` for owner-only. There is + no separate `visibility` field and no `share.private` key. ### `find` diff --git a/PRODUCT.md b/PRODUCT.md index 55d56d3..f74e809 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -117,6 +117,10 @@ Knowledge does **not** ship a second ACL. Document access uses the host’s the document. Patterns (`knowledge.space:*`) work via `@intx/authz`. 3. **Share sugars on add** — only mint tags (owner / tenant / peer owner tags / explicit tags). They do **not** invent visibility modes or block lists. + **Host contract:** peers named in `share.principals` only see the doc if the + host has granted them `find` on their owner tag (or matching pattern) — + typically bootstrap every principal with `find` on `knowledge.owner:`. + Tag minting is not grant minting. See `docs/AUTHZ-DOCUMENT-ACCESS.md`. Default add is **owner-only** (`knowledge.owner:` + creator rule). Deny is absence of allow (or a more specific host deny grant) — not a document diff --git a/docs/AUTHZ-DOCUMENT-ACCESS.md b/docs/AUTHZ-DOCUMENT-ACCESS.md index f6e5871..707077c 100644 --- a/docs/AUTHZ-DOCUMENT-ACCESS.md +++ b/docs/AUTHZ-DOCUMENT-ACCESS.md @@ -28,9 +28,11 @@ createdByPrincipalId: string // audit + default owner tag When the caller does not pass tags/share: 1. Always tag: `knowledge.owner:` -2. No other tags → **owner-only** (only principals granted `find` on that owner resource can see it; typically the owner has that grant, or the plane treats owner as implicit allow for the creating principal) +2. No other tags → **owner-only by default**. The **creating** principal always + sees their own docs (engine convenience). **Peers** need an explicit host + grant of `find` on that owner resource (or a matching pattern) — the engine + never auto-grants tags to anyone. -**Implicit owner rule (engine convenience, not a second ACL):** the creating principal may always find/recent their own documents without a matching grant tag. Everyone else must match a tag via `authorize`. This is equivalent to auto-issuing a creator grant without writing one. ### Explicit tags @@ -63,11 +65,31 @@ Share helpers **must not** reintroduce visibility modes. They only mint tags: | Sugar | Tags written | | --- | --- | -| (default / private) | `knowledge.owner:` | +| (omit `share` — owner-only default) | `knowledge.owner:` | | `share: { tenant: true }` | `knowledge.owner:`, `knowledge.tenant:` | | `share: { principals: ["p2","p3"] }` | `knowledge.owner:`, `knowledge.owner:p2`, `knowledge.owner:p3` | | `share: { tags: ["knowledge.space:eng"] }` | `knowledge.owner:`, plus those tags | +There is **no** `share.private` key. Owner-only is the default when `share` is omitted. + +### Host contract for multi-user share (required) + +Tag minting is **not** grant minting. For peer share to work in product: + +1. When Alice adds with `share: { principals: ["bob"] }`, the document is tagged + `knowledge.owner:alice` and `knowledge.owner:bob`. +2. Bob sees it only if the host has granted Bob `find` on `knowledge.owner:bob` + (or a pattern that matches). **Recommended host bootstrap:** every principal + receives `find` (and optionally `add` side-effects as you prefer) on + `knowledge.owner:` at signup, or a single pattern grant such as + `knowledge.owner:*` only if that matches your tenancy model. +3. Space/tenant tags work the same way: host must issue grants on + `knowledge.space:eng` / `knowledge.tenant:` for non-creators to match. + +Without (2), `share.principals` is a silent no-op for peers (fail-closed; looks +like empty search). Document this in host mount guides — do not reintroduce a +document mini-ACL in this package. + **Removed:** `visibility: { mode: private|principals|tenant }`, `blockPrincipalIds`, product `acl.mode` / `acl.allow` / `acl.block` as security. Deny is expressed as **absence of allow** (or an explicit deny grant in the host store with higher specificity) — not a document-row block list. From 338a50e656c9d8fddae3c548ece2687920c4a0fc Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 15:15:51 -0700 Subject: [PATCH 03/11] Rename acl module to grant-tags MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit File and imports only — same resolveAccessTags / canAccessDocument API. Matches grant-tag product language; drops the mini-ACL filename. --- IMPLEMENTATION.md | 3 ++- src/{acl.test.ts => grant-tags.test.ts} | 3 ++- src/{acl.ts => grant-tags.ts} | 12 +++++------- src/knowledge.ts | 6 ++++-- src/ports/fakes.test.ts | 3 ++- src/ports/fakes.ts | 3 ++- src/routes/add.ts | 3 ++- src/services/timeline.ts | 3 ++- 8 files changed, 21 insertions(+), 15 deletions(-) rename src/{acl.test.ts => grant-tags.test.ts} (99%) rename src/{acl.ts => grant-tags.ts} (92%) diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index c74ef0c..58d4ef7 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -12,7 +12,8 @@ src/ mount-config.ts # KnowledgeConfig + loadKnowledgeConfig() — the mount config config.ts # EngineConfig — the core vector-plane config (db + embed + rerank) knowledge.ts # createKnowledgePlane — add/find/ask/recent against store or pgvector - acl.ts # resolveAccessTags + canAccessDocument (grant tags) + grant-tags.ts # resolveAccessTags + canAccessDocument (host grants) + log.ts # getLogger(["knowledge-engine"]) from @intx/log migrations.ts # runKnowledgeMigrations(url) ports/ # DocumentStore / SourceProvider / MemoryProvider + fakes diff --git a/src/acl.test.ts b/src/grant-tags.test.ts similarity index 99% rename from src/acl.test.ts rename to src/grant-tags.test.ts index a85dac4..1ce444d 100644 --- a/src/acl.test.ts +++ b/src/grant-tags.test.ts @@ -7,7 +7,8 @@ import { ownerTag, resolveAccessTags, tenantTag, -} from "./acl.ts"; +} from "./grant-tags.ts"; + describe("resolveAccessTags", () => { test("always includes owner tag", () => { diff --git a/src/acl.ts b/src/grant-tags.ts similarity index 92% rename from src/acl.ts rename to src/grant-tags.ts index 3563bf9..cb0d286 100644 --- a/src/acl.ts +++ b/src/grant-tags.ts @@ -1,14 +1,12 @@ /** - * Document access via Interchange authz grant tags — not a mini-ACL. + * Grant-tag helpers for document access (Interchange `@intx/authz`). * * Spec: docs/AUTHZ-DOCUMENT-ACCESS.md * - * - Capability (knowledge:add / knowledge:find) is checked elsewhere. - * - Document access: creator always sees own docs; otherwise any accessTag - * that authorize(…, tag, "find") allows. - * - Share sugars only mint tags. - * - * Hard cutover: no visibility modes, no block lists, no dual-read ACL path. + * - Capability checks (add/find on `knowledge`) live on the HTTP mount. + * - Document access: creator always sees own docs; otherwise any `accessTag` + * that `authorize(…, tag, "find")` allows. + * - Share sugars only mint tags — they never write grants. */ import { authorize } from "@intx/authz"; import type { ConditionRegistry, GrantStore } from "@intx/authz"; diff --git a/src/knowledge.ts b/src/knowledge.ts index c087c1b..4037ce2 100644 --- a/src/knowledge.ts +++ b/src/knowledge.ts @@ -5,7 +5,8 @@ import { resolveAccessTags, ownerTag, type ShareSugar, -} from "./acl.ts"; +} from "./grant-tags.ts"; + import type { EngineConfig } from "./config.ts"; import { log } from "./log.ts"; import { createDb, type Db, type RawSql } from "./db/client.ts"; @@ -60,7 +61,8 @@ export { tenantTag, canAccessDocument, type ShareSugar, -} from "./acl.ts"; +} from "./grant-tags.ts"; + export type ChatMessage = { role: "system" | "user" | "assistant"; diff --git a/src/ports/fakes.test.ts b/src/ports/fakes.test.ts index 310c35c..e6dc299 100644 --- a/src/ports/fakes.test.ts +++ b/src/ports/fakes.test.ts @@ -1,7 +1,8 @@ import { describe, expect, it } from "bun:test"; import { createInMemoryGrantStore } from "@intx/authz"; -import { ownerTag, tenantTag } from "../acl.ts"; +import { ownerTag, tenantTag } from "../grant-tags.ts"; + import { createFakeDocumentStore, createFakeSourceProvider, diff --git a/src/ports/fakes.ts b/src/ports/fakes.ts index 207d356..6fea14f 100644 --- a/src/ports/fakes.ts +++ b/src/ports/fakes.ts @@ -6,7 +6,8 @@ * authorize(grants, …, tag, "find") allows when grants are provided. Without * grants, only creator access (safe default for unit tests). */ -import { canAccessDocument } from "../acl.ts"; +import { canAccessDocument } from "../grant-tags.ts"; + import type { DocumentStore, DocumentStoreAddParams, diff --git a/src/routes/add.ts b/src/routes/add.ts index e59cd6b..88e48be 100644 --- a/src/routes/add.ts +++ b/src/routes/add.ts @@ -4,7 +4,8 @@ import { describeRoute, resolver, validator } from "hono-openapi"; import { type } from "arktype"; import { formatCaughtError, log } from "../log.ts"; -import { resolveAccessTags, type ShareSugar } from "../acl.ts"; +import { resolveAccessTags, type ShareSugar } from "../grant-tags.ts"; + import { KnowledgeError } from "../knowledge.ts"; import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; diff --git a/src/services/timeline.ts b/src/services/timeline.ts index 853c51b..16a14a6 100644 --- a/src/services/timeline.ts +++ b/src/services/timeline.ts @@ -9,7 +9,8 @@ */ import { and, desc, eq, sql } from "drizzle-orm"; import type { ConditionRegistry, GrantStore } from "@intx/authz"; -import { canAccessDocument } from "../acl.ts"; +import { canAccessDocument } from "../grant-tags.ts"; + import type { Db } from "../db/client.ts"; import { knowledgeDocument, knowledgeVersion } from "../db/schema.ts"; import { log } from "../log.ts"; From 44b3d3278935b15a5b6f761a5a3f80c0fc639c15 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 15:37:39 -0700 Subject: [PATCH 04/11] Rename public package surface to @corbits/memory Hard cutover: createMemory / mountMemory, Memory types, /api/memory routes, memory:add and memory:find grants, memory.* access tags. Side-channel is memoryProvider. Postgres schema knowledge stays for stability. --- AGENTS.md | 6 +- ARCHITECTURE.md | 30 ++-- CHANGELOG.md | 36 +++-- CONTRIBUTING.md | 6 +- IMPLEMENTATION.md | 54 +++---- MIGRATION.md | 51 ++++--- PRODUCT.md | 56 +++---- README.md | 48 +++--- docs/AUTHZ-DOCUMENT-ACCESS.md | 42 ++--- package.json | 10 +- packages/knowledge-adapter-mem0/README.md | 16 +- packages/knowledge-adapter-mem0/package.json | 8 +- .../src/create-mem0-document-store.test.ts | 4 +- .../src/create-mem0-document-store.ts | 2 +- packages/knowledge-adapter-mem0/src/types.ts | 4 +- .../knowledge-adapter-supermemory/README.md | 12 +- .../package.json | 6 +- .../src/index.test.ts | 8 +- .../src/index.ts | 6 +- scripts/db-setup.ts | 4 +- src/config.ts | 8 +- src/core/adapt-and-plan.test.ts | 2 +- src/core/fts-language.test.ts | 6 +- src/core/schemas/adapted-document.test.ts | 6 +- src/core/schemas/document.test.ts | 4 +- src/grant-tags.test.ts | 8 +- src/grant-tags.ts | 12 +- src/index.ts | 92 +++++------ src/log.ts | 2 +- src/{knowledge.test.ts => memory.test.ts} | 144 +++++++++--------- src/{knowledge.ts => memory.ts} | 144 +++++++++--------- src/migrations.ts | 6 +- src/mount-config.ts | 16 +- src/ports/memory-plane.test.ts | 30 ++-- src/ports/merge-plane.test.ts | 18 +-- src/ports/mount-fakes.test.ts | 20 +-- src/routes/add.ts | 16 +- src/routes/ask.ts | 22 +-- src/routes/deps.test.ts | 8 +- src/routes/deps.ts | 20 +-- src/routes/find.ts | 16 +- src/routes/mount.ts | 4 +- src/routes/recent.ts | 16 +- src/routes/routes.test.ts | 70 ++++----- src/services/search.ts | 10 +- src/services/timeline.test.ts | 8 +- 46 files changed, 572 insertions(+), 545 deletions(-) rename src/{knowledge.test.ts => memory.test.ts} (85%) rename src/{knowledge.ts => memory.ts} (90%) diff --git a/AGENTS.md b/AGENTS.md index e39a319..e799b16 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ -# Agent guide — @corbits/knowledge-engine +# Agent guide — @corbits/memory -A library, not a service. `src/` is the whole product: a knowledge add / find / +A library, not a service. `src/` is the whole product: a memory add / find / ask / recent SDK that **mounts onto a host Interchange app**. There is no server, port, or process entrypoint here, and there never should be. @@ -18,7 +18,7 @@ CI runs `typecheck` + `test` — both must pass before any push. ## Layout -- `src/index.ts` — public surface: `mountKnowledgeEngine`, `mountKnowledgeRoutes`, `createKnowledgePlane` +- `src/index.ts` — public surface: `mountMemory`, `mountMemoryRoutes`, `createMemory` - `src/mount-config.ts` / `src/config.ts` — mount config + engine config - `src/routes/` — Hono routes (`add`, `find`, `ask`, `recent`) - `src/services/` — capture / search / transform internals (not public verbs) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 53db269..465c943 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,24 +1,24 @@ -# Knowledge Engine — Architecture +# Corbits Memory — Architecture -A knowledge add / find / ask / recent SDK that mounts onto an Interchange hub. The -host owns auth, tenancy, and the process; this library owns the knowledge / +A memory add / find / ask / recent SDK that mounts onto an Interchange hub. The +host owns auth, tenancy, and the process; this library owns the memory / vector plane and the routes that read and write it. ## Why an SDK, not a service -The knowledge store was originally built inside a larger backend. It turned out +The memory store was originally built inside a larger backend. It turned out to be cleanly detachable, and then cleanly *mountable*: -- No knowledge table has a foreign key into any control-plane table — every +- No memory table has a foreign key into any control-plane table — every cross-reference (`tenant_id`, `principal_id`, source refs) is plain `text`. - Embedding and reranking go out as plain HTTP to configured model endpoints, not through any agent runtime. - The ACL rule is a self-contained scope stored on the row, not a join against a grant engine. -So the engine needs nothing but a pgvector Postgres and an embed/rerank -endpoint. It ships as `mountKnowledgeEngine(app, opts)`: the host passes its -Hono app and its grant store; the engine mounts its routes, reads identity from +So the library needs nothing but a pgvector Postgres and an embed/rerank +endpoint. It ships as `mountMemory(app, opts)`: the host passes its +Hono app and its grant store; the library mounts its routes, reads identity from the request context, and talks to its own vector store. No second server, no HTTP hop. @@ -64,21 +64,21 @@ trust model. ## Mounted surface -`mountKnowledgeEngine` adds, under the host app: +`mountMemory` adds, under the host app: -- `POST /api/knowledge/add` — ingest a note (raw + derive). -- `POST /api/knowledge/find` — hybrid retrieval: FTS + dense (pgvector) → RRF +- `POST /api/memory/add` — ingest a note (raw + derive). +- `POST /api/memory/find` — hybrid retrieval: FTS + dense (pgvector) → RRF fusion → cross-encoder rerank → bounded authority/recency boosts → MMR; optional live `SourceProvider` merge (fail-soft). -- `POST /api/knowledge/ask` — grant-checked as `knowledge:find`; retrieves as +- `POST /api/memory/ask` — grant-checked as `memory:find`; retrieves as the principal, grounds a prompt from hit snippets, calls host-injected `generate`. Optional memory recall when `includeMemory` is true. -- `GET /api/knowledge/recent` — recent documents for the caller's scope, +- `GET /api/memory/recent` — recent documents for the caller's scope, filtered with the same grant-tag access as local find (`canAccessDocument`). -It also returns an in-process `KnowledgePlane` (`add`, `find`, `ask`, +It also returns an in-process `Memory` (`add`, `find`, `ask`, `recent`, optional `remember` / `recall`). `ask()` is grant-checked in-process -(callers bypass the HTTP `requireGrant` guard). The engine owns no generation +(callers bypass the HTTP `requireGrant` guard). The library owns no generation client; hosts wire `generate` to their inference layer. MCP is not part of this package — mount `@corbitsdev/hono-openapi-mcp` to expose diff --git a/CHANGELOG.md b/CHANGELOG.md index ffc4d4a..2791b78 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -All notable changes to `@corbits/knowledge-engine` are documented in this file. +All notable changes to `@corbits/memory` are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). @@ -9,14 +9,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed -- **Breaking:** knowledge plane surface is `add` / `find` / `ask` / `recent` with +- **Breaking:** package and public surface renamed from `@corbits/knowledge-engine` + to `@corbits/memory`. Public APIs: `mountMemory`, `mountMemoryRoutes`, + `createMemory`, `loadMemoryConfig`, `runMemoryMigrations`, `Memory`, + `MemoryConfig`, `MemoryError`. HTTP paths are under `/api/memory/`; grants are + `memory:add` / `memory:find`; access tags use `memory.owner:` / `memory.tenant:` + / `memory.space:`. Postgres schema name remains `knowledge`. See `MIGRATION.md`. +- **Breaking:** memory plane surface is `add` / `find` / `ask` / `recent` with `principalId` + `tenantId` only (`capture` / `search` / `timeline` and `subjectId` / `scopeId` removed). See `MIGRATION.md`. -- **Breaking:** HTTP routes are `POST /api/knowledge/add`, - `POST /api/knowledge/find`, `POST /api/knowledge/ask`, - `GET /api/knowledge/recent`. Old paths are not mounted. +- **Breaking:** HTTP routes are `POST /api/memory/add`, + `POST /api/memory/find`, `POST /api/memory/ask`, + `GET /api/memory/recent`. Old paths are not mounted. - **Breaking:** grant actions are `add` and `find` (was `capture` / `search`). - `ask` and `recent` use the `find` grant. + `ask` and `recent` use the `find` grant. Capability resource is `memory`. - **Breaking:** `add` returns `{ documentId }`; find body uses `limit` (not `k`); find wire uses `items` (not `hits`). - **Breaking:** document access is Interchange **grant tags** (`accessTags` + @@ -25,19 +31,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Breaking:** Postgres baseline is two files (`0001_extensions` + `0002_knowledge_baseline`) with `access_tags` and no `visibility_*` columns. Fresh installs only — drop/recreate the knowledge schema on existing DBs. +- **Breaking:** MemoryProvider side-channel option is `options.memoryProvider` + (was `options.memory`). ### Added - Optional `TextExtractor` + `file` XOR `content` on `add` - `share` sugar on `add` (maps to access tags only: owner, tenant, peers) -- `access_tags` on `knowledge.document` (baseline schema) -- `POST /api/knowledge/ask` HTTP route +- `access_tags` on `knowledge.document` (baseline schema; Postgres schema name unchanged) +- `POST /api/memory/ask` HTTP route - `MIGRATION.md` hard-cutover notes for in-repo consumers ## [0.1.2] — 2026-07-31 ### Added -- Public `createKnowledgePlane` export for out-of-band capture and search (CLI seeders, batch ingesters, tests) without mounting HTTP routes (`#8`) +- Public `createMemory` export for out-of-band capture and search (CLI seeders, batch ingesters, tests) without mounting HTTP routes (`#8`) ### Fixed @@ -57,7 +65,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - Dense search scales `hnsw.ef_search` to the overfetch limit (floor 40) and probes `hnsw.iterative_scan = relaxed_order` once per process when available (`#2`) -- Package install docs point at `@corbits/knowledge-engine` (`#6`) +- Package install docs point at `@corbits/memory` (`#6`) ### Fixed @@ -68,8 +76,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- Initial cut: mountable knowledge capture + search SDK for Interchange hubs +- Initial cut: mountable memory capture + search SDK for Interchange hubs -[0.1.2]: https://github.com/corbitsdev/corbits-knowledge-engine/compare/v0.1.1...v0.1.2 -[0.1.1]: https://github.com/corbitsdev/corbits-knowledge-engine/compare/v0.1.0...v0.1.1 -[0.1.0]: https://github.com/corbitsdev/corbits-knowledge-engine/releases/tag/v0.1.0 +[0.1.2]: https://github.com/corbitsdev/corbits-memory/compare/v0.1.1...v0.1.2 +[0.1.1]: https://github.com/corbitsdev/corbits-memory/compare/v0.1.0...v0.1.1 +[0.1.0]: https://github.com/corbitsdev/corbits-memory/releases/tag/v0.1.0 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 941fb48..8a40ad4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,12 +1,12 @@ # Contributing -Thanks for considering a contribution to Knowledge Engine. +Thanks for considering a contribution to Corbits Memory. ## Running it locally ```bash -git clone https://github.com/corbitsdev/corbits-knowledge-engine.git -cd corbits-knowledge-engine +git clone https://github.com/corbitsdev/corbits-memory.git +cd corbits-memory docker compose up -d # pgvector Postgres on localhost:5434 cp .env.example .env # edit as needed — see README.md's quickstart bun install diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 58d4ef7..2201de9 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -1,4 +1,4 @@ -# Knowledge Engine — Implementation Reference +# Corbits Memory — Implementation Reference This is the detailed implementation reference: concrete files, tables, functions, and wire shapes. For the "why standalone" / boundaries story, read @@ -8,17 +8,17 @@ and wire shapes. For the "why standalone" / boundaries story, read ``` src/ - index.ts # mountKnowledgeEngine / mountKnowledgeRoutes - mount-config.ts # KnowledgeConfig + loadKnowledgeConfig() — the mount config + index.ts # mountMemory / mountMemoryRoutes + mount-config.ts # MemoryConfig + loadMemoryConfig() — the mount config config.ts # EngineConfig — the core vector-plane config (db + embed + rerank) - knowledge.ts # createKnowledgePlane — add/find/ask/recent against store or pgvector + memory.ts # createMemory — add/find/ask/recent against store or pgvector grant-tags.ts # resolveAccessTags + canAccessDocument (host grants) - log.ts # getLogger(["knowledge-engine"]) from @intx/log - migrations.ts # runKnowledgeMigrations(url) + log.ts # getLogger(["memory"]) from @intx/log + migrations.ts # runMemoryMigrations(url) ports/ # DocumentStore / SourceProvider / MemoryProvider + fakes routes/ # the mounted routes - mount.ts # mountKnowledgeRoutes (HTTP) + mount.ts # mountMemoryRoutes (HTTP) deps.ts # RouteDeps, caller(c) (context identity), grantGuard add.ts, find.ts, ask.ts, recent.ts db/ @@ -38,7 +38,7 @@ compose.yml # pgvector + Ollama + reranker for local dev ``` -The SDK has no server and no process entrypoint. `mountKnowledgeEngine` takes +The SDK has no server and no process entrypoint. `mountMemory` takes the host's `Hono` app plus `{ config, grants? }` and mounts the routes; each reads identity from the context (`caller(c)`) and guards via `grantGuard`. Services take `{ db, sql, config }` explicitly (no module-level @@ -47,7 +47,7 @@ tests exercise the routes and services directly without a listening server. Mounting does **not** verify the FTS language against the database at boot — see the `knowledge_chunk` section below. A host is expected to either run -`runKnowledgeMigrations` itself (which verifies) or wire its own readiness +`runMemoryMigrations` itself (which verifies) or wire its own readiness probe to call `verifyFtsLanguage`; without one of those, a language mismatch surfaces as a runtime failure on the plane's first query, not at mount time. @@ -58,12 +58,12 @@ There are two config types, both in the SDK: - **`EngineConfig`** (`src/config.ts`) — the core vector-plane config the DB client and capture/search/transform services consume: `databaseUrl`, `dbPoolMax`, `embed`, `rerank`. -- **`KnowledgeConfig`** (`src/mount-config.ts`) — what `mountKnowledgeEngine` - takes: just `{ knowledge: EngineConfig }`. `loadKnowledgeConfig()` builds one +- **`MemoryConfig`** (`src/mount-config.ts`) — what `mountMemory` + takes: just `{ memory: EngineConfig }`. `loadMemoryConfig()` builds one from the environment; hosts may also construct it programmatically. Auth, tenancy, and grants are the host's — none of that is config here. -`loadKnowledgeConfig()` uses the same fail-loud helpers: `requireEnv(name)` +`loadMemoryConfig()` uses the same fail-loud helpers: `requireEnv(name)` throws if unset/empty, `optionalEnv(name)` returns `undefined`, `intEnv(name, fallback)` parses a positive integer or throws. @@ -71,7 +71,7 @@ fallback)` parses a positive integer or throws. |---|---|---|---| | `KNOWLEDGE_DATABASE_URL` | **yes** | — | the engine's own pgvector Postgres | | `DB_POOL_MAX` | no | `8` | postgres-js pool size | -| `FTS_LANGUAGE` | no | `english` | text search config for the lexical channel; fixed into the generated column at migration time — changing it later requires rebuilding the column (recipe below), and `runKnowledgeMigrations` fails loudly if config and column disagree. Unqualified `pg_catalog` config names only — a schema-qualified config (`myschema.mycfg`) is rejected explicitly, both when configuring and when read back from an already-migrated column. | +| `FTS_LANGUAGE` | no | `english` | text search config for the lexical channel; fixed into the generated column at migration time — changing it later requires rebuilding the column (recipe below), and `runMemoryMigrations` fails loudly if config and column disagree. Unqualified `pg_catalog` config names only — a schema-qualified config (`myschema.mycfg`) is rejected explicitly, both when configuring and when read back from an already-migrated column. | | `EMBED_BASE_URL` | **yes** | — | embed endpoint root, no path suffix | | `EMBED_MODEL` | **yes** | — | model id/name passed to the embed endpoint | | `EMBED_API_STYLE` | no | `"openai"` | `"openai" \| "tei" \| "ollama"` | @@ -141,14 +141,14 @@ that powers the lexical search channel — this is the only place FTS is computed; no separate FTS table exists. Its language comes from `FTS_LANGUAGE` at migration time; the query side binds the same configured language as a `regconfig` parameter. The invariant is verified twice, both -read-only against the catalog: `runKnowledgeMigrations` checks after -applying (the deploy step), and the knowledge plane runs the same check +read-only against the catalog: `runMemoryMigrations` checks after +applying (the deploy step), and the memory plane runs the same check once, memoized, before its first query (the serving path) — so a mismatch or unmigrated schema fails loudly on first use regardless of who ran the migrations. **The serving-path check only runs when something actually calls it** — `search()`/`capture()` invoke it lazily and memoize the result, but nothing forces that first call to happen at boot. A host that mounts the -engine without running `runKnowledgeMigrations` itself and without wiring a +engine without running `runMemoryMigrations` itself and without wiring a readiness probe will not learn about a language mismatch until the first real query or capture fails — not at startup. Hosts that want a boot-time guarantee **must** call the exported `verifyFtsLanguage` from their own @@ -159,7 +159,7 @@ fresh full insert of its own chunks. **Changing `FTS_LANGUAGE` on an already-migrated database** (the mismatch `verifyFtsLanguage` throws on) requires rebuilding the generated column — -`runKnowledgeMigrations` only applies new files and will not retroactively +`runMemoryMigrations` only applies new files and will not retroactively alter an existing one. One-time recipe (verified against a live `postgres:16` instance): @@ -323,7 +323,7 @@ returns the run summary either way. this repo; chunks left unembedded simply never populate the dense channel for the query — they're still found by lexical/FTS). Any of these failure modes sets `degraded: true` on the `CaptureResult`, surfaced by - `POST /api/knowledge/add` as a `degraded` field in its response — the add + `POST /api/memory/add` as a `degraded` field in its response — the add still succeeded (chunks are durable and lexically searchable), only the dense/vector channel for those chunks is incomplete. @@ -348,7 +348,7 @@ the same core function called directly by a replay with an **existing** Entry point, one query in, one ranked/citable hit list out. `k` is clamped to `[1, MAX_K=100]`, default `DEFAULT_HYBRID_TOP_K = 8`. An empty `query` string is only accepted if `kinds` or `entityIds` is provided (structured-filter-only -search); otherwise it throws `KnowledgeSearchInputError` (400). +search); otherwise it throws `MemorySearchInputError` (400). 1. **Generation resolution** — `generation` defaults to `LIVE_GENERATION`. For any non-live generation, `resolveGenerationSearchParams` (transform.ts) @@ -470,21 +470,21 @@ so knowing it will flip the tenant's live dense channel too. ## Mounted routes -`mountKnowledgeEngine` mounts these onto the host app. Identity is the request +`mountMemory` mounts these onto the host app. Identity is the request principal read off the Interchange context (`caller(c)` → `{ scopeId: principal.tenantId, subjectId: principal.id }`); clients never send `tenant_id`/`principal_id` — the handlers only read title/text/query/limit/access_tags/share. Each route is guarded with `grantGuard(deps, action)`, which applies the host's -`requireGrant("knowledge", action)` when provided (else a pass-through). +`requireGrant("memory", action)` when provided (else a pass-through). | Method + path | Grant action | Request body | Response | |---|---|---|---| -| `POST /api/knowledge/add` | `add` | `{ title, text, access_tags?, share? }` | `200 { documentId }`; `400` on validation | -| `POST /api/knowledge/find` | `find` | `{ query, limit?, kinds?, entity_ids? }` (limit 1–50; `kinds`/`entity_ids` narrow every retrieval channel — lexical and dense — to a document `kind` or linked entity id before fusion; unset or `[]` = unfiltered) | `200 { items[], evidence?, degraded? }`; `400` on bad input | -| `POST /api/knowledge/ask` | `find` | `{ query, limit? }` (1–50) | `200 { text, citations[], evidence }`; `403` / `501` as plane errors | -| `GET /api/knowledge/recent` | `find` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope, filtered with grant-tag access (`canAccessDocument`). One event per document (active live version). | +| `POST /api/memory/add` | `add` | `{ title, text, access_tags?, share? }` | `200 { documentId }`; `400` on validation | +| `POST /api/memory/find` | `find` | `{ query, limit?, kinds?, entity_ids? }` (limit 1–50; `kinds`/`entity_ids` narrow every retrieval channel — lexical and dense — to a document `kind` or linked entity id before fusion; unset or `[]` = unfiltered) | `200 { items[], evidence?, degraded? }`; `400` on bad input | +| `POST /api/memory/ask` | `find` | `{ query, limit? }` (1–50) | `200 { text, citations[], evidence }`; `403` / `501` as plane errors | +| `GET /api/memory/recent` | `find` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope, filtered with grant-tag access (`canAccessDocument`). One event per document (active live version). | -`mountKnowledgeRoutes` and `mountKnowledgeEngine` mount the four HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). +`mountMemoryRoutes` and `mountMemory` mount the four HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). ### Timeline wire fields (vs the old CaptureLog ring) @@ -504,7 +504,7 @@ timeline maps different columns: Document access is Interchange authz — **not** a mini-ACL. -- Write path: `resolveAccessTags` always writes `knowledge.owner:` and +- Write path: `resolveAccessTags` always writes `memory.owner:` and merges optional `accessTags` / share sugar (`tenant`, peer `principals`, explicit `tags`). Stored on `knowledge.document.access_tags`. - Read path (find + recent): `canAccessDocument` — creator always allowed; diff --git a/MIGRATION.md b/MIGRATION.md index 6df1093..02b196f 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -7,10 +7,10 @@ move to the new names in the same release. | Was | Now | | --- | --- | -| `knowledge.capture(params)` | `knowledge.add(params)` | -| `knowledge.search(params)` | `knowledge.find(params)` | -| `knowledge.timeline(params)` | `knowledge.recent(params)` | -| `knowledge.ask(params)` | unchanged verb; grant action changed (below) | +| `knowledge.capture(params)` | `memory.add(params)` | +| `knowledge.search(params)` | `memory.find(params)` | +| `knowledge.timeline(params)` | `memory.recent(params)` | +| `knowledge.ask(params)` | `memory.ask(params)`; grant action changed (below) | Identity fields on every call: @@ -43,10 +43,10 @@ Identity fields on every call: | Was | Now | | --- | --- | -| `POST /api/knowledge/capture` | `POST /api/knowledge/add` | -| `POST /api/knowledge/search` | `POST /api/knowledge/find` | -| `GET /api/knowledge/timeline` | `GET /api/knowledge/recent` | -| — | `POST /api/knowledge/ask` (new) | +| `POST /api/knowledge/capture` | `POST /api/memory/add` | +| `POST /api/knowledge/search` | `POST /api/memory/find` | +| `GET /api/knowledge/timeline` | `GET /api/memory/recent` | +| — | `POST /api/memory/ask` (new) | Old paths return **404**. No redirect, no dual mount. @@ -61,21 +61,38 @@ Old paths return **404**. No redirect, no dual mount. | Was | Now | | --- | --- | -| `requireGrant("knowledge", "capture")` | `requireGrant("knowledge", "add")` | -| `requireGrant("knowledge", "search")` | `requireGrant("knowledge", "find")` | +| `requireGrant("knowledge", "capture")` | `requireGrant("memory", "add")` | +| `requireGrant("knowledge", "search")` | `requireGrant("memory", "find")` | -`find`, `ask`, and `recent` all require the **`find`** action. Old action names -are not accepted — update grant rows in the host grant store before deploy. +`find`, `ask`, and `recent` all require the **`find`** action on resource +`memory`. Old resource/action names are not accepted — update grant rows in the +host grant store before deploy. -In-process `ask()` also checks `knowledge` / `find` (was `search`). +In-process `ask()` also checks `memory` / `find` (was `knowledge` / `search`). + +## Package / public API rename (`@corbits/memory`) + +| Was | Now | +| --- | --- | +| `@corbits/knowledge-engine` | `@corbits/memory` | +| `mountKnowledgeEngine` | `mountMemory` | +| `mountKnowledgeRoutes` | `mountMemoryRoutes` | +| `createKnowledgePlane` | `createMemory` | +| `loadKnowledgeConfig` | `loadMemoryConfig` | +| `runKnowledgeMigrations` | `runMemoryMigrations` | +| `KnowledgePlane` / `KnowledgeConfig` / `KnowledgeError` | `Memory` / `MemoryConfig` / `MemoryError` | +| `options.memory` (MemoryProvider side-channel) | `options.memoryProvider` | +| Access tags `knowledge.owner:` / `knowledge.tenant:` / `knowledge.space:` | `memory.owner:` / `memory.tenant:` / `memory.space:` | + +Postgres schema name remains **`knowledge`** (tables such as `knowledge.document`). ## Host checklist (this package's consumers) -1. Rename plane method calls and identity fields. -2. Point HTTP clients at the new paths and bodies. -3. Rewrite grant rules: `capture`→`add`, `search`→`find`. +1. Rename plane method calls and identity fields; switch package import to `@corbits/memory`. +2. Point HTTP clients at the new `/api/memory/*` paths and bodies. +3. Rewrite grant rules: resource `knowledge`→`memory`, `capture`→`add`, `search`→`find`. 4. Drop any reliance on `status: "captured"` or `hits` / `k` on the wire. -5. If you use file capture, pass `textExtractor` into `createKnowledgePlane` / mount options. +5. If you use file capture, pass `textExtractor` into `createMemory` / mount options. 6. Document access is grant tags (`accessTags` + creator + host `GrantStore`), not visibility modes or block lists. Update any host code that wrote `visibility`. 7. Fresh Postgres: baseline migrations are `0001_extensions.sql` + diff --git a/PRODUCT.md b/PRODUCT.md index f74e809..ead7840 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -1,20 +1,20 @@ -# Corbits Knowledge Engine — Product shape +# Corbits Memory — Product shape -A **mountable knowledge plane** for Interchange hubs: durable documents, hybrid -search, grounded ask, and optional live sources / personal memory. Workbench and -coding agents are clients — not owners of ingestion or auth. +A **mountable memory plane** for Interchange hubs: durable documents, hybrid +search, grounded ask, and optional live sources / personal memory side-channel. +Workbench and coding agents are clients — not owners of ingestion or auth. ## Shape (locked) -**`src/` is the `@corbits/knowledge-engine` SDK.** Interchange is the hub — the +**`src/` is the `@corbits/memory` SDK.** Interchange is the hub — the SDK never creates one; it mounts onto yours. | Surface | Role | | --- | --- | -| `mountKnowledgeEngine(app, opts)` | Plane + HTTP on an Interchange `createApp` | -| `createKnowledgePlane(config, grants?, options?)` | Same plane without HTTP | -| `runKnowledgeMigrations(url)` | Apply pgvector schema under Postgres `knowledge` | -| `loadKnowledgeConfig()` | Mount config from env | +| `mountMemory(app, opts)` | Plane + HTTP on an Interchange `createApp` | +| `createMemory(config, grants?, options?)` | Same plane without HTTP | +| `runMemoryMigrations(url)` | Apply pgvector schema under Postgres `knowledge` | +| `loadMemoryConfig()` | Mount config from env | ### Green public plane (only these verbs) @@ -26,8 +26,8 @@ SDK never creates one; it mounts onto yours. | `recent` | Recent documents for the principal | Hard cutover: there is no `capture` / `search` / `timeline` export. HTTP paths -and grants match the verbs: `POST /api/knowledge/add|find|ask`, -`GET /api/knowledge/recent`; grants `knowledge:add` and `knowledge:find` +and grants match the verbs: `POST /api/memory/add|find|ask`, +`GET /api/memory/recent`; grants `memory:add` and `memory:find` (ask/recent share `find`). Identity on the plane is always **`principalId` + `tenantId`** (never @@ -42,11 +42,11 @@ Identity on the plane is always **`principalId` + `tenantId`** (never | `SourceProvider` | Optional **tools-shaped** live search (`searchLive`); merge is fail-soft. Not a store replacement. | | `MemoryProvider` | Optional ask side-channel only (`includeMemory`); **not** how you swap backends. | -Mount options accept `documentStore`, `sources[]`, `memory`, plus in-package +Mount options accept `documentStore`, `sources[]`, `memoryProvider`, plus in-package **fakes** so a host can mount with fakes only and exercise add/find/ask/recent without Postgres. Hosts that want a third-party durable backend implement `DocumentStore` (or use an optional adapter package) and pass it as -`documentStore`, omitting `KnowledgeConfig` when Postgres is not needed. +`documentStore`, omitting `MemoryConfig` when Postgres is not needed. **MergeLocalLiveV1** merges local DocumentStore + live SourceProviders: fail-soft per provider (timeout/error → degrade flags, never fail the request), dedupe by @@ -70,7 +70,7 @@ tenant+principal) and do **not** multi-share via `accessTags` + host `GrantStore`. For full grant-tag ACL, use the default pgvector store (or a store that implements the contract in `docs/AUTHZ-DOCUMENT-ACCESS.md`). Adapter packages must document their isolation model in their own README. Never mount a -durable store as `options.memory`. +durable store as `options.memoryProvider`. ### What is not in scope @@ -91,38 +91,38 @@ Claude Code / Codex / Workbench (clients) ▼ ┌──────────────────────────────────────────────┐ │ Host Interchange createApp │ -│ + mountKnowledgeEngine(app, opts) │ -│ grants: knowledge:add | knowledge:find │ +│ + mountMemory(app, opts) │ +│ grants: memory:add | memory:find │ │ documentStore: pgvector | host store │ │ | fake │ -│ optional: sources, memory, │ +│ optional: sources, memoryProvider, │ │ textExtractor │ │ │ in-process │ │ ▼ │ -│ Knowledge plane: add / find / ask / recent │ +│ Memory plane: add / find / ask / recent │ │ → DocumentStore (sole durable backend) │ └──────────────────────────────────────────────┘ ``` ## Identity and access (Interchange authz — one system) -Knowledge does **not** ship a second ACL. Document access uses the host’s +Memory does **not** ship a second ACL. Document access uses the host’s `@intx/authz` grant store — the same grants/roles as the rest of Interchange. -1. **Capability** — may this principal use knowledge at all? - `authorize(…, resource: "knowledge", action: "add" | "find")`. +1. **Capability** — may this principal use memory at all? + `authorize(…, resource: "memory", action: "add" | "find")`. 2. **Document access** — each document carries **`accessTags`** (resource strings in grant-pattern space). A principal sees a document if they are the creator **or** `authorize(…, resource: , action: "find")` allows for any tag on - the document. Patterns (`knowledge.space:*`) work via `@intx/authz`. + the document. Patterns (`memory.space:*`) work via `@intx/authz`. 3. **Share sugars on add** — only mint tags (owner / tenant / peer owner tags / explicit tags). They do **not** invent visibility modes or block lists. **Host contract:** peers named in `share.principals` only see the doc if the host has granted them `find` on their owner tag (or matching pattern) — - typically bootstrap every principal with `find` on `knowledge.owner:`. + typically bootstrap every principal with `find` on `memory.owner:`. Tag minting is not grant minting. See `docs/AUTHZ-DOCUMENT-ACCESS.md`. -Default add is **owner-only** (`knowledge.owner:` + creator rule). +Default add is **owner-only** (`memory.owner:` + creator rule). Deny is absence of allow (or a more specific host deny grant) — not a document block list. Full design: `docs/AUTHZ-DOCUMENT-ACCESS.md`. @@ -135,10 +135,10 @@ HTTP bodies are a thin subset of the in-process plane (identity always comes from the host principal context, never the body): ```http -POST /api/knowledge/add { "title", "text", "access_tags"?, "share"? } -POST /api/knowledge/find { "query", "limit?", "kinds?", "entity_ids?", "sources?", "includeEvidence?" } -POST /api/knowledge/ask { "query", "limit?", "sources?", "includeMemory?" } -GET /api/knowledge/recent ?limit= +POST /api/memory/add { "title", "text", "access_tags"?, "share"? } +POST /api/memory/find { "query", "limit?", "kinds?", "entity_ids?", "sources?", "includeEvidence?" } +POST /api/memory/ask { "query", "limit?", "sources?", "includeMemory?" } +GET /api/memory/recent ?limit= ``` `kinds` / `entity_ids` on find narrow both lexical and dense channels before diff --git a/README.md b/README.md index 976aa1e..b8267ea 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# @corbits/knowledge-engine +# @corbits/memory Mountable knowledge plane for [Interchange](https://github.com/corbitsdev) hubs: **add** documents, **find** with hybrid search, **ask** grounded answers, **recent** @@ -6,7 +6,7 @@ timeline — with optional live sources and personal memory. **Authenticates nothing.** Identity is `c.get("principal")` on HTTP; in-process callers pass `principalId` + `tenantId`. Authorization is the host grant store -(`knowledge:add` / `knowledge:find`). Never embeds in-process — embedding and +(`memory:add` / `memory:find`). Never embeds in-process — embedding and rerank are outbound HTTP to configured endpoints. Requires Bun 1.2+. @@ -14,20 +14,20 @@ Requires Bun 1.2+. ## Install ```bash -bun add @corbits/knowledge-engine +bun add @corbits/memory ``` ## Mount (green path) ```ts -import { mountKnowledgeEngine } from "@corbits/knowledge-engine"; -import { loadKnowledgeConfig } from "@corbits/knowledge-engine/config"; +import { mountMemory } from "@corbits/memory"; +import { loadMemoryConfig } from "@corbits/memory/config"; -mountKnowledgeEngine(app, { - config: loadKnowledgeConfig(), +mountMemory(app, { + config: loadMemoryConfig(), grants: { grantStore, conditionRegistry }, // optional ports: - // documentStore, sources, memory, textExtractor, generate + // documentStore, sources, memoryProvider, textExtractor, generate }); ``` @@ -35,41 +35,41 @@ Routes (each grant-checked): | Method | Path | Grant | | --- | --- | --- | -| POST | `/api/knowledge/add` | `knowledge:add` | -| POST | `/api/knowledge/find` | `knowledge:find` | -| POST | `/api/knowledge/ask` | `knowledge:find` | -| GET | `/api/knowledge/recent` | `knowledge:find` | +| POST | `/api/memory/add` | `memory:add` | +| POST | `/api/memory/find` | `memory:find` | +| POST | `/api/memory/ask` | `memory:find` | +| GET | `/api/memory/recent` | `memory:find` | Clients never send tenant/principal in the body. -### Host must set principal on `/api/knowledge/*` +### Host must set principal on `/api/memory/*` These routes sit outside `/api/tenants/:tenantId/*`. Mount middleware **before** -`mountKnowledgeEngine` that sets `c.set("principal", …)` and `c.set("tenant", …)`. +`mountMemory` that sets `c.set("principal", …)` and `c.set("tenant", …)`. Without it: **401 `principal_required`**. ### Plane without HTTP ```ts import { - createKnowledgePlane, + createMemory, createFakeDocumentStore, createFakeMemoryProvider, -} from "@corbits/knowledge-engine"; +} from "@corbits/memory"; -const knowledge = createKnowledgePlane(undefined, grants, { +const memory = createMemory(undefined, grants, { documentStore: createFakeDocumentStore(), - memory: createFakeMemoryProvider(), + memoryProvider: createFakeMemoryProvider(), generate: async (messages) => "…", // wire your inference layer }); -await knowledge.add({ +await memory.add({ tenantId, principalId, content: { title: "Note", text: "…" }, }); -const hits = await knowledge.find({ tenantId, principalId, query: "…" }); -const answer = await knowledge.ask({ +const hits = await memory.find({ tenantId, principalId, query: "…" }); +const answer = await memory.ask({ tenantId, principalId, query: "…", @@ -83,7 +83,7 @@ const answer = await knowledge.ask({ | --- | --- | --- | | `DocumentStore` | Engine pgvector | `options.documentStore` / `createFakeDocumentStore()` | | `SourceProvider[]` | none | `options.sources` — live merge is fail-soft | -| `MemoryProvider` | none | `options.memory` / `createFakeMemoryProvider()` | +| `MemoryProvider` | none | `options.memoryProvider` / `createFakeMemoryProvider()` | **Live merge (MergeLocalLiveV1):** per-provider timeout/error → `live_timeout` / `live_error` degrade; dedupe `adapter:externalRef`; optional `sources` filter. @@ -113,7 +113,7 @@ import { ```bash # KNOWLEDGE_DATABASE_URL required — no DATABASE_URL fallback bun run db:setup -# or: runKnowledgeMigrations(process.env.KNOWLEDGE_DATABASE_URL) +# or: runMemoryMigrations(process.env.KNOWLEDGE_DATABASE_URL) ``` All tables live under Postgres schema **`knowledge`** @@ -122,7 +122,7 @@ pre-1.0: re-run migrations on a fresh knowledge DB. ## Document access (grant tags) -1. Host capability grants (`knowledge:add` / `knowledge:find`) +1. Host capability grants (`memory:add` / `memory:find`) 2. Per-document **access tags** + creator rule (Interchange `@intx/authz`) 3. Share sugars on `add` only mint tags — no visibility modes or block lists diff --git a/docs/AUTHZ-DOCUMENT-ACCESS.md b/docs/AUTHZ-DOCUMENT-ACCESS.md index 707077c..1e18cbf 100644 --- a/docs/AUTHZ-DOCUMENT-ACCESS.md +++ b/docs/AUTHZ-DOCUMENT-ACCESS.md @@ -9,10 +9,10 @@ | Layer | Owner | Mechanism | | --- | --- | --- | | Who is the caller? | Host (Interchange) | `principal` + `tenant` on context / plane args | -| May they use knowledge at all? | Host grant store | `authorize(…, resource: "knowledge", action: "add" \| "find")` | +| May they use memory at all? | Host grant store | `authorize(…, resource: "memory", action: "add" \| "find")` | | Which **documents** may they see? | Host grant store + tags on the document | `authorize(…, resource: , action: "find")` for any tag on the doc | -There is **one** authorization system: `@intx/authz` + host `GrantStore`. The knowledge engine does not invent modes, allowlists, or block lists as a security boundary. +There is **one** authorization system: `@intx/authz` + host `GrantStore`. Corbits Memory does not invent modes, allowlists, or block lists as a security boundary. ## Document model @@ -27,7 +27,7 @@ createdByPrincipalId: string // audit + default owner tag When the caller does not pass tags/share: -1. Always tag: `knowledge.owner:` +1. Always tag: `memory.owner:` 2. No other tags → **owner-only by default**. The **creating** principal always sees their own docs (engine convenience). **Peers** need an explicit host grant of `find` on that owner resource (or a matching pattern) — the engine @@ -39,7 +39,7 @@ When the caller does not pass tags/share: ```ts plane.add({ tenantId, principalId, title, text, - accessTags: ["knowledge.space:eng", "knowledge.project:ke"], + accessTags: ["memory.space:eng", "memory.project:ke"], }) ``` @@ -48,16 +48,16 @@ Host issues grants such as: ```ts { principalId: "alice", - resource: "knowledge.space:eng", + resource: "memory.space:eng", action: "find", effect: "allow", // …origin, etc. } ``` -Alice then sees any document tagged `knowledge.space:eng` (capability `knowledge`/`find` still required). +Alice then sees any document tagged `memory.space:eng` (capability `memory`/`find` still required). -Patterns work: a grant on `knowledge.space:*` matches `knowledge.space:eng` via `@intx/authz` `matchPattern`. +Patterns work: a grant on `memory.space:*` matches `memory.space:eng` via `@intx/authz` `matchPattern`. ### Share sugars (map to tags only) @@ -65,10 +65,10 @@ Share helpers **must not** reintroduce visibility modes. They only mint tags: | Sugar | Tags written | | --- | --- | -| (omit `share` — owner-only default) | `knowledge.owner:` | -| `share: { tenant: true }` | `knowledge.owner:`, `knowledge.tenant:` | -| `share: { principals: ["p2","p3"] }` | `knowledge.owner:`, `knowledge.owner:p2`, `knowledge.owner:p3` | -| `share: { tags: ["knowledge.space:eng"] }` | `knowledge.owner:`, plus those tags | +| (omit `share` — owner-only default) | `memory.owner:` | +| `share: { tenant: true }` | `memory.owner:`, `memory.tenant:` | +| `share: { principals: ["p2","p3"] }` | `memory.owner:`, `memory.owner:p2`, `memory.owner:p3` | +| `share: { tags: ["memory.space:eng"] }` | `memory.owner:`, plus those tags | There is **no** `share.private` key. Owner-only is the default when `share` is omitted. @@ -77,14 +77,14 @@ There is **no** `share.private` key. Owner-only is the default when `share` is o Tag minting is **not** grant minting. For peer share to work in product: 1. When Alice adds with `share: { principals: ["bob"] }`, the document is tagged - `knowledge.owner:alice` and `knowledge.owner:bob`. -2. Bob sees it only if the host has granted Bob `find` on `knowledge.owner:bob` + `memory.owner:alice` and `memory.owner:bob`. +2. Bob sees it only if the host has granted Bob `find` on `memory.owner:bob` (or a pattern that matches). **Recommended host bootstrap:** every principal receives `find` (and optionally `add` side-effects as you prefer) on - `knowledge.owner:` at signup, or a single pattern grant such as - `knowledge.owner:*` only if that matches your tenancy model. + `memory.owner:` at signup, or a single pattern grant such as + `memory.owner:*` only if that matches your tenancy model. 3. Space/tenant tags work the same way: host must issue grants on - `knowledge.space:eng` / `knowledge.tenant:` for non-creators to match. + `memory.space:eng` / `memory.tenant:` for non-creators to match. Without (2), `share.principals` is a silent no-op for peers (fail-closed; looks like empty search). Document this in host mount guides — do not reintroduce a @@ -99,7 +99,7 @@ Deny is expressed as **absence of allow** (or an explicit deny grant in the host ### Capability (unchanged) ```ts -authorize(grantStore, principalId, tenantId, "knowledge", "find"|"add") +authorize(grantStore, principalId, tenantId, "memory", "find"|"add") // effect must be "allow" ``` @@ -138,8 +138,8 @@ Unchanged intentional tradeoff: live `SourceProvider` hits are **enrichment unde { "title": "…", "text": "…", - "access_tags": ["knowledge.space:eng"], - "share": { "tenant": true, "principals": ["alice"], "tags": ["knowledge.space:eng"] } + "access_tags": ["memory.space:eng"], + "share": { "tenant": true, "principals": ["alice"], "tags": ["memory.space:eng"] } } ``` @@ -163,7 +163,7 @@ Fresh databases apply the baseline migrations (`0001_extensions.sql` + ## Non-goals -- Group membership resolution inside the knowledge engine (host/roles issue grants). +- Group membership resolution inside Corbits Memory (host/roles issue grants). - Per-chunk ACL. - Live channel grant tags (host policy). - Re-implementing roles inside this package. @@ -173,7 +173,7 @@ Fresh databases apply the baseline migrations (`0001_extensions.sql` + 1. No public plane/HTTP API accepts `visibility` mode or block list as security. 2. Default add is owner-visible only (creator + owner tag). 3. Principal B sees A’s doc only when host grant allows `find` on a tag present on the doc (or B is creator). -4. Capability `knowledge`/`find` still required for find/ask/recent. +4. Capability `memory`/`find` still required for find/ask/recent. 5. PRODUCT.md / MIGRATION.md / README describe grant tags, not mini-ACL. 6. Engine + fakes enforce the algorithm; vendor adapters document principal-bucket limit. 7. `bun run typecheck && bun run test` green. diff --git a/package.json b/package.json index 241e641..4b9e662 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { - "name": "@corbits/knowledge-engine", + "name": "@corbits/memory", "version": "0.1.2", - "description": "Mountable knowledge add/find/ask/recent SDK for Interchange hubs", + "description": "Mountable memory add/find/ask/recent SDK for Interchange hubs", "exports": { ".": "./src/index.ts", "./migrations": "./src/migrations.ts", @@ -36,11 +36,11 @@ "author": "Sawyer Cutler ", "repository": { "type": "git", - "url": "git+https://github.com/corbitsdev/corbits-knowledge-engine.git" + "url": "git+https://github.com/corbitsdev/corbits-memory.git" }, - "homepage": "https://github.com/corbitsdev/corbits-knowledge-engine#readme", + "homepage": "https://github.com/corbitsdev/corbits-memory#readme", "bugs": { - "url": "https://github.com/corbitsdev/corbits-knowledge-engine/issues" + "url": "https://github.com/corbitsdev/corbits-memory/issues" }, "keywords": [ "knowledge", diff --git a/packages/knowledge-adapter-mem0/README.md b/packages/knowledge-adapter-mem0/README.md index af0f188..3c4202f 100644 --- a/packages/knowledge-adapter-mem0/README.md +++ b/packages/knowledge-adapter-mem0/README.md @@ -1,6 +1,6 @@ # @corbits/knowledge-adapter-mem0 -**Replaceable DocumentStore** for [@corbits/knowledge-engine](https://github.com/corbitsdev/corbits-knowledge-engine) +**Replaceable DocumentStore** for [@corbits/memory](https://github.com/corbitsdev/corbits-memory) backed by the [Mem0 Platform](https://docs.mem0.ai/) HTTP API. Pure `fetch` — **no** `mem0ai` / vendor SDK. Tenancy is enforced with a @@ -14,10 +14,10 @@ required. This is the product integration path (not `MemoryProvider` / `includeMemory`). ```ts -import { createKnowledgePlane } from "@corbits/knowledge-engine"; +import { createMemory } from "@corbits/memory"; import { createMem0DocumentStore } from "@corbits/knowledge-adapter-mem0"; -const knowledge = createKnowledgePlane(undefined, grants, { +const memory = createMemory(undefined, grants, { documentStore: createMem0DocumentStore({ apiKey: process.env.MEM0_API_KEY!, // baseUrl: "https://api.mem0.ai", // optional @@ -25,13 +25,13 @@ const knowledge = createKnowledgePlane(undefined, grants, { generate: myGenerate, // required only for ask() }); -await knowledge.add({ +await memory.add({ tenantId, principalId, content: { title: "Prefs", text: "Prefers dark mode" }, }); -const { items } = await knowledge.find({ +const { items } = await memory.find({ tenantId, principalId, query: "preferences", @@ -41,7 +41,7 @@ const { items } = await knowledge.find({ Or via mount: ```ts -mountKnowledgeEngine(app, { +mountMemory(app, { documentStore: createMem0DocumentStore({ apiKey }), grants, generate, @@ -55,7 +55,7 @@ mountKnowledgeEngine(app, { | Isolation | **Principal-bucket only** via `mapUser(tenantId, principalId)`. Each principal has a private Mem0 `user_id`; docs are not shared across principals. | | Document access | This adapter is **principal-bucket only**. Host grant tags (`accessTags`) are stored as metadata at best and are **not** evaluated. For multi-principal grant-tag ACL, use the default pgvector store. | | `recent` | Always `[]` — Mem0 has no recent-feed API here. | -| `options.memory` | **Never** mount this package as `options.memory`. That port is an ask side-channel; Mem0 as product backend is `documentStore` only. | +| `options.memoryProvider` | **Never** mount this package as `options.memoryProvider`. That port is an ask side-channel; Mem0 as product backend is `documentStore` only. | ## What is out of scope @@ -86,4 +86,4 @@ Auth: `Authorization: Token `. ## License -LGPL-2.1-only (same as the knowledge engine). +LGPL-2.1-only (same as Corbits Memory). diff --git a/packages/knowledge-adapter-mem0/package.json b/packages/knowledge-adapter-mem0/package.json index 11dacec..2914788 100644 --- a/packages/knowledge-adapter-mem0/package.json +++ b/packages/knowledge-adapter-mem0/package.json @@ -1,7 +1,7 @@ { "name": "@corbits/knowledge-adapter-mem0", "version": "0.1.0", - "description": "Mem0 DocumentStore adapter for @corbits/knowledge-engine (pure fetch, no SDK)", + "description": "Mem0 DocumentStore adapter for @corbits/memory (pure fetch, no SDK)", "type": "module", "module": "src/index.ts", @@ -23,12 +23,12 @@ "author": "Sawyer Cutler ", "repository": { "type": "git", - "url": "git+https://github.com/corbitsdev/corbits-knowledge-engine.git", + "url": "git+https://github.com/corbitsdev/corbits-memory.git", "directory": "packages/knowledge-adapter-mem0" }, - "homepage": "https://github.com/corbitsdev/corbits-knowledge-engine#readme", + "homepage": "https://github.com/corbitsdev/corbits-memory#readme", "bugs": { - "url": "https://github.com/corbitsdev/corbits-knowledge-engine/issues" + "url": "https://github.com/corbitsdev/corbits-memory/issues" }, "keywords": [ "mem0", diff --git a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts b/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts index 9c9bd3e..8017d84 100644 --- a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts +++ b/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts @@ -73,7 +73,7 @@ describe("createMem0DocumentStore", () => { principalId: "p1", title: "Prefs", text: "Prefers dark mode", - accessTags: ["knowledge.owner:p1"], + accessTags: ["memory.owner:p1"], }); expect(documentId).toMatch( @@ -150,7 +150,7 @@ describe("createMem0DocumentStore", () => { principalId: "p", title: "t", text: "x", - accessTags: ["knowledge.owner:p"], + accessTags: ["memory.owner:p"], }), ).rejects.toThrow(/tenantId/); diff --git a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts b/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts index 3afcd93..76a0e1d 100644 --- a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts +++ b/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts @@ -148,7 +148,7 @@ export function parseFindResults(raw: unknown): DocumentStoreFindItem[] { * Pure fetch — no mem0 SDK. Mount as the plane's durable backend: * * ```ts - * createKnowledgePlane(undefined, grants, { + * createMemory(undefined, grants, { * documentStore: createMem0DocumentStore({ apiKey }), * }) * ``` diff --git a/packages/knowledge-adapter-mem0/src/types.ts b/packages/knowledge-adapter-mem0/src/types.ts index ad52995..1e2f44a 100644 --- a/packages/knowledge-adapter-mem0/src/types.ts +++ b/packages/knowledge-adapter-mem0/src/types.ts @@ -1,10 +1,10 @@ /** * Port shapes — defined locally so this adapter never imports runtime from - * @corbits/knowledge-engine. DocumentStore is the product plug (replaceable + * @corbits/memory. DocumentStore is the product plug (replaceable * durable backend). MemoryProvider is a thin legacy shape kept for back-compat. */ -/** Minimal citation open shape (matches knowledge-engine SearchHitCitation). */ +/** Minimal citation open shape (matches @corbits/memory SearchHitCitation). */ export type DocumentStoreCitation = { adapter: string; external_ref: string; diff --git a/packages/knowledge-adapter-supermemory/README.md b/packages/knowledge-adapter-supermemory/README.md index 1f52a8d..f7d369e 100644 --- a/packages/knowledge-adapter-supermemory/README.md +++ b/packages/knowledge-adapter-supermemory/README.md @@ -1,6 +1,6 @@ # @corbits/knowledge-adapter-supermemory -**Replaceable DocumentStore** for [@corbits/knowledge-engine](https://github.com/corbitsdev/corbits-knowledge-engine) +**Replaceable DocumentStore** for [@corbits/memory](https://github.com/corbitsdev/corbits-memory) backed by the [Supermemory](https://supermemory.ai/) HTTP API. Pure `fetch` — **no** vendor SDK. Tenancy is enforced with a length-prefixed @@ -14,10 +14,10 @@ endpoints required. This is the product integration path (not `MemoryProvider` / `includeMemory`). ```ts -import { createKnowledgePlane } from "@corbits/knowledge-engine"; +import { createMemory } from "@corbits/memory"; import { createSupermemoryDocumentStore } from "@corbits/knowledge-adapter-supermemory"; -const knowledge = createKnowledgePlane(undefined, grants, { +const memory = createMemory(undefined, grants, { documentStore: createSupermemoryDocumentStore({ apiKey: process.env.SUPERMEMORY_API_KEY!, }), @@ -28,7 +28,7 @@ const knowledge = createKnowledgePlane(undefined, grants, { Or via mount: ```ts -mountKnowledgeEngine(app, { +mountMemory(app, { documentStore: createSupermemoryDocumentStore({ apiKey }), grants, generate, @@ -45,7 +45,7 @@ plane (add/find/ask), not memories-only personal facts. | Isolation | **Principal-bucket only** via `containerTag(tenantId, principalId)`. Each principal has a private container; docs are not shared across principals. | | Document access | This adapter is **principal-bucket only**. Host grant tags (`accessTags`) are stored as metadata at best and are **not** evaluated. For multi-principal grant-tag ACL, use the default pgvector store. | | `recent` | Always `[]` — no recent-feed API in this adapter. | -| `options.memory` | **Never** mount this package as `options.memory`. Product backend is `documentStore` only. | +| `options.memoryProvider` | **Never** mount this package as `options.memoryProvider`. Product backend is `documentStore` only. | ## What is out of scope @@ -76,4 +76,4 @@ Auth: `Authorization: Bearer `. ## License -LGPL-2.1-only (same as the knowledge engine). +LGPL-2.1-only (same as Corbits Memory). diff --git a/packages/knowledge-adapter-supermemory/package.json b/packages/knowledge-adapter-supermemory/package.json index 4fc7275..28d951b 100644 --- a/packages/knowledge-adapter-supermemory/package.json +++ b/packages/knowledge-adapter-supermemory/package.json @@ -23,12 +23,12 @@ "author": "Sawyer Cutler ", "repository": { "type": "git", - "url": "git+https://github.com/corbitsdev/corbits-knowledge-engine.git", + "url": "git+https://github.com/corbitsdev/corbits-memory.git", "directory": "packages/knowledge-adapter-supermemory" }, - "homepage": "https://github.com/corbitsdev/corbits-knowledge-engine#readme", + "homepage": "https://github.com/corbitsdev/corbits-memory#readme", "bugs": { - "url": "https://github.com/corbitsdev/corbits-knowledge-engine/issues" + "url": "https://github.com/corbitsdev/corbits-memory/issues" }, "keywords": [ "knowledge", diff --git a/packages/knowledge-adapter-supermemory/src/index.test.ts b/packages/knowledge-adapter-supermemory/src/index.test.ts index 52a0bf8..2233a0a 100644 --- a/packages/knowledge-adapter-supermemory/src/index.test.ts +++ b/packages/knowledge-adapter-supermemory/src/index.test.ts @@ -57,7 +57,7 @@ describe("createSupermemoryDocumentStore", () => { principalId: "alice", title: "t", text: "hello", - accessTags: ["knowledge.owner:alice"], + accessTags: ["memory.owner:alice"], }), ).rejects.toThrow(/non-empty/); @@ -92,7 +92,7 @@ describe("createSupermemoryDocumentStore", () => { principalId: "alice", title: "Prefs", text: "prefers dark mode", - accessTags: ["knowledge.owner:alice"], + accessTags: ["memory.owner:alice"], }); expect(documentId).toMatch( @@ -167,14 +167,14 @@ describe("createSupermemoryDocumentStore", () => { principalId: "user-1", title: "a", text: "fact a", - accessTags: ["knowledge.owner:user-1"], + accessTags: ["memory.owner:user-1"], }); await store.add({ tenantId: "tenant-b", principalId: "user-1", title: "b", text: "fact b", - accessTags: ["knowledge.owner:user-1"], + accessTags: ["memory.owner:user-1"], }); expect(tags).toEqual(["t8_tenant-a_u6_user-1", "t8_tenant-b_u6_user-1"]); diff --git a/packages/knowledge-adapter-supermemory/src/index.ts b/packages/knowledge-adapter-supermemory/src/index.ts index 9b14775..6a22534 100644 --- a/packages/knowledge-adapter-supermemory/src/index.ts +++ b/packages/knowledge-adapter-supermemory/src/index.ts @@ -2,13 +2,13 @@ * Supermemory DocumentStore adapter (replaceable durable backend). * * Pure fetch HTTP — no vendor SDK. Port shapes are defined locally so this - * package never imports the knowledge-engine runtime. + * package never imports the @corbits/memory runtime. * * Product path: createSupermemoryDocumentStore → mount as options.documentStore. * MemoryProvider factory is back-compat only. */ -/** Minimal citation open shape (matches knowledge-engine SearchHitCitation). */ +/** Minimal citation open shape (matches @corbits/memory SearchHitCitation). */ export type DocumentStoreCitation = { adapter: string; external_ref: string; @@ -209,7 +209,7 @@ function parseTitleAndSnippet(text: string): { title: string; snippet: string } * Pure fetch — no vendor SDK. Mount as the plane's durable backend: * * ```ts - * createKnowledgePlane(undefined, grants, { + * createMemory(undefined, grants, { * documentStore: createSupermemoryDocumentStore({ apiKey }), * }) * ``` diff --git a/scripts/db-setup.ts b/scripts/db-setup.ts index f6d9d3e..0e2487e 100644 --- a/scripts/db-setup.ts +++ b/scripts/db-setup.ts @@ -1,10 +1,10 @@ -import { runKnowledgeMigrations } from "../src/migrations.ts"; +import { runMemoryMigrations } from "../src/migrations.ts"; const url = process.env["KNOWLEDGE_DATABASE_URL"]; if (!url) throw new Error("KNOWLEDGE_DATABASE_URL is required"); -await runKnowledgeMigrations(url, { +await runMemoryMigrations(url, { log: (line) => console.log(` ${line}`), }); console.log("Migrations complete."); diff --git a/src/config.ts b/src/config.ts index 8a3f468..917899a 100644 --- a/src/config.ts +++ b/src/config.ts @@ -1,8 +1,8 @@ /** - * Knowledge / vector plane config — the core engine behind add/find/recent. + * Memory / vector plane config — the core engine behind add/find/recent. * * This is the low-level engine config consumed by the DB client and internal - * services. The SDK's mount-level config (`KnowledgeConfig`, see + * services. The SDK's mount-level config (`MemoryConfig`, see * mount-config.ts) carries this as its `knowledge` sub-object. There is no * standalone server here — the SDK mounts onto a host Interchange app, so there * is no port, service token, or process entrypoint. @@ -11,8 +11,8 @@ export type EngineConfig = { databaseUrl: string; dbPoolMax: number; // Must match the language the knowledge_chunk.text_fts column was built - // with (runKnowledgeMigrations verifies this against the catalog). - // Required and concrete: loadKnowledgeConfig / createKnowledgePlane resolve + // with (runMemoryMigrations verifies this against the catalog). + // Required and concrete: loadMemoryConfig / createMemory resolve // the default (DEFAULT_FTS_LANGUAGE) once via parseFtsLanguage so services // never re-default. Constructing EngineConfig by hand — pass // DEFAULT_FTS_LANGUAGE (or parseFtsLanguage(undefined)) explicitly. diff --git a/src/core/adapt-and-plan.test.ts b/src/core/adapt-and-plan.test.ts index 6911740..56def50 100644 --- a/src/core/adapt-and-plan.test.ts +++ b/src/core/adapt-and-plan.test.ts @@ -9,7 +9,7 @@ function validAdaptedDocument( kind: "artifact", title: "Q3 renewal brief", externalRef: "artifact:art_1", - accessTags: ["knowledge.owner:principal_1"], + accessTags: ["memory.owner:principal_1"], entityHints: [], chunks: [ { ordinal: 0, text: "The account renews in Q3 with a 12% expansion." }, diff --git a/src/core/fts-language.test.ts b/src/core/fts-language.test.ts index 81242e5..a997f9e 100644 --- a/src/core/fts-language.test.ts +++ b/src/core/fts-language.test.ts @@ -42,14 +42,14 @@ describe("the baseline migration language token", () => { }); }); -describe("runKnowledgeMigrations language boundary", () => { +describe("runMemoryMigrations language boundary", () => { it("falls back to the FTS_LANGUAGE env var when no option is passed", async () => { // Pin the boundary contract without a live database: the runner must // resolve exactly like the config loader, from the same env var. - const { runKnowledgeMigrations } = await import("../migrations.ts"); + const { runMemoryMigrations } = await import("../migrations.ts"); process.env["FTS_LANGUAGE"] = "not a valid name"; try { - await expect(runKnowledgeMigrations("postgres://unused")).rejects.toThrow( + await expect(runMemoryMigrations("postgres://unused")).rejects.toThrow( "not a valid text search config name", ); } finally { diff --git a/src/core/schemas/adapted-document.test.ts b/src/core/schemas/adapted-document.test.ts index fa8498c..7c75b01 100644 --- a/src/core/schemas/adapted-document.test.ts +++ b/src/core/schemas/adapted-document.test.ts @@ -10,7 +10,7 @@ describe("AdaptedDocumentSchema", () => { kind: "call_transcript", title: "Q3 renewal call", externalRef: "granola:note_123", - accessTags: ["knowledge.tenant:t1"], + accessTags: ["memory.tenant:t1"], attributes: { durationSec: 1800 }, entityHints: [{ kind: "person", identifier: "jane@example.com" }], edges: [ @@ -34,7 +34,7 @@ describe("AdaptedDocumentSchema", () => { kind: "task", title: "Follow up with Acme", externalRef: "task:1", - accessTags: ["knowledge.owner:u1"], + accessTags: ["memory.owner:u1"], entityHints: [], chunks: [{ ordinal: 0, text: "Follow up with Acme on pricing." }], contentHash: "sha256:def456", @@ -47,7 +47,7 @@ describe("AdaptedDocumentSchema", () => { kind: "task", title: "Follow up with Acme", externalRef: "task:1", - accessTags: ["knowledge.owner:u1"], + accessTags: ["memory.owner:u1"], entityHints: [], chunks: [{ ordinal: 0, text: "Follow up with Acme on pricing." }], }); diff --git a/src/core/schemas/document.test.ts b/src/core/schemas/document.test.ts index 290f6bb..dde5a25 100644 --- a/src/core/schemas/document.test.ts +++ b/src/core/schemas/document.test.ts @@ -15,7 +15,7 @@ describe("KnowledgeDocumentSchema", () => { title: "Q3 renewal call", adapter: "granola", external_ref: "granola:note_123", - access_tags: ["knowledge.tenant:tenant_1"], + access_tags: ["memory.tenant:tenant_1"], attributes: { channel: "call", pinned: true, score: null }, created_at: "2026-07-19T00:00:00.000Z", last_seen_at: "2026-07-19T00:00:00.000Z", @@ -31,7 +31,7 @@ describe("KnowledgeDocumentSchema", () => { kind: "call_transcript", title: "Q3 renewal call", adapter: "granola", - access_tags: ["knowledge.tenant:tenant_1"], + access_tags: ["memory.tenant:tenant_1"], attributes: {}, created_at: "2026-07-19T00:00:00.000Z", last_seen_at: "2026-07-19T00:00:00.000Z", diff --git a/src/grant-tags.test.ts b/src/grant-tags.test.ts index 1ce444d..d1f5ae9 100644 --- a/src/grant-tags.test.ts +++ b/src/grant-tags.test.ts @@ -42,13 +42,13 @@ describe("resolveAccessTags", () => { const tags = resolveAccessTags({ principalId: "u1", tenantId: "t1", - accessTags: ["knowledge.space:eng"], + accessTags: ["memory.space:eng"], share: { tags: ["knowledge.project:ke"] }, }); expect(tags).toEqual( expect.arrayContaining([ ownerTag("u1"), - "knowledge.space:eng", + "memory.space:eng", "knowledge.project:ke", ]), ); @@ -108,7 +108,7 @@ describe("filterAccessibleDocuments", () => { { id: "g1", principalId: "viewer", - resource: "knowledge.space:eng", + resource: "memory.space:eng", action: "find", effect: "allow", origin: "role", @@ -126,7 +126,7 @@ describe("filterAccessibleDocuments", () => { { id: "b", createdByPrincipalId: "other", - accessTags: ["knowledge.space:eng"], + accessTags: ["memory.space:eng"], }, { id: "c", diff --git a/src/grant-tags.ts b/src/grant-tags.ts index cb0d286..65bfb60 100644 --- a/src/grant-tags.ts +++ b/src/grant-tags.ts @@ -3,7 +3,7 @@ * * Spec: docs/AUTHZ-DOCUMENT-ACCESS.md * - * - Capability checks (add/find on `knowledge`) live on the HTTP mount. + * - Capability checks (add/find on `memory`) live on the HTTP mount. * - Document access: creator always sees own docs; otherwise any `accessTag` * that `authorize(…, tag, "find")` allows. * - Share sugars only mint tags — they never write grants. @@ -12,18 +12,18 @@ import { authorize } from "@intx/authz"; import type { ConditionRegistry, GrantStore } from "@intx/authz"; export function ownerTag(principalId: string): string { - return `knowledge.owner:${principalId}`; + return `memory.owner:${principalId}`; } export function tenantTag(tenantId: string): string { - return `knowledge.tenant:${tenantId}`; + return `memory.tenant:${tenantId}`; } /** Share sugar — maps only to tags (no visibility modes / block lists). */ export type ShareSugar = { - /** Include knowledge.tenant: */ + /** Include memory.tenant: */ tenant?: boolean; - /** Include knowledge.owner: for each peer */ + /** Include memory.owner: for each peer */ principals?: string[]; /** Explicit resource tags (host grant space) */ tags?: string[]; @@ -39,7 +39,7 @@ export type ResolveAccessTagsParams = { /** * Resolve the tag set written on add. - * Always includes knowledge.owner:. Never invents visibility modes. + * Always includes memory.owner:. Never invents visibility modes. */ export function resolveAccessTags(params: ResolveAccessTagsParams): string[] { const tags = new Set(); diff --git a/src/index.ts b/src/index.ts index 2ec2e7b..06f5bc8 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,5 +1,5 @@ /** - * @corbits/knowledge-engine — a knowledge add / find / ask / recent engine you + * @corbits/memory — a memory add / find / ask / recent engine you * mount onto an Interchange hub. * * The host owns auth, tenancy, grants, and the process. This SDK reads the @@ -9,39 +9,39 @@ import type { Hono } from "hono"; import { createRequireGrant, type TenantEnv } from "@intx/hub-api"; -import type { KnowledgeConfig } from "./mount-config.ts"; +import type { MemoryConfig } from "./mount-config.ts"; import { - createKnowledgePlane, + createMemory, type Generate, - type KnowledgePlane, - type KnowledgePlaneOptions, + type Memory, + type MemoryOptions, type TextExtractor, -} from "./knowledge.ts"; +} from "./memory.ts"; import type { DocumentStore, MemoryProvider, SourceProvider, } from "./ports/types.ts"; import { - mountKnowledgeRoutes, + mountMemoryRoutes, type GrantConfig, type RouteDeps, } from "./routes/mount.ts"; // Config -export type { KnowledgeConfig } from "./mount-config.ts"; -export { loadKnowledgeConfig } from "./mount-config.ts"; +export type { MemoryConfig } from "./mount-config.ts"; +export { loadMemoryConfig } from "./mount-config.ts"; export type { EngineConfig } from "./config.ts"; export { RerankConfigError } from "./core/rerank-client.ts"; -// Knowledge plane +// Memory plane // -// `createKnowledgePlane` is exported so a host can add or find outside a +// `createMemory` is exported so a host can add or find outside a // request — a CLI seeder, a batch ingester, or a test — without standing up a // Hono app just to get a plane. Callers acting on behalf of a user are // responsible for the capability check `requireGrant` would have performed; see // the README. Rerank config is validated at construction (same as mount). // Pass `grants` + optional `generate` when the host will call `ask()`. -export { createKnowledgePlane } from "./knowledge.ts"; +export { createMemory } from "./memory.ts"; export type { AskCitation, AskResult, @@ -50,23 +50,23 @@ export type { FindResult, Generate, HybridSearchResult, - KnowledgeAddParams, - KnowledgeAddResult, - KnowledgeAskParams, - KnowledgeFindParams, - KnowledgeIdentity, - KnowledgePlane, - KnowledgePlaneOptions, - KnowledgeRecentParams, - KnowledgeRecallItem, - KnowledgeRecallParams, - KnowledgeRememberParams, - KnowledgeShare, + MemoryAddParams, + MemoryAddResult, + MemoryAskParams, + MemoryFindParams, + MemoryIdentity, + Memory, + MemoryOptions, + MemoryRecentParams, + MemoryRecallItem, + MemoryRecallParams, + MemoryRememberParams, + MemoryShare, SearchHit, TextExtractor, TimelineEvent, -} from "./knowledge.ts"; -export { KnowledgeError, KnowledgeNotPermittedError } from "./knowledge.ts"; +} from "./memory.ts"; +export { MemoryError, MemoryNotPermittedError } from "./memory.ts"; // Ports — pluggable storage, live sources, and optional memory export type { DocumentStore, @@ -87,7 +87,7 @@ export { createFakeSourceProvider, } from "./ports/fakes.ts"; // Migrations -export { runKnowledgeMigrations } from "./migrations.ts"; +export { runMemoryMigrations } from "./migrations.ts"; // Degrade metrics — no metrics dependency exists in this package (see // core/degrade-metrics.ts); a host with its own metrics backend polls this // snapshot and forwards it, rather than the engine owning a /metrics port. @@ -105,18 +105,18 @@ export { verifyFtsLanguage, } from "./core/fts-language.ts"; // Granular mount (compose your own) -export { mountKnowledgeRoutes, type GrantConfig } from "./routes/mount.ts"; +export { mountMemoryRoutes, type GrantConfig } from "./routes/mount.ts"; -export type MountKnowledgeEngineOptions = { +export type MountMemoryOptions = { /** * Engine config (DB + model endpoints). Optional when `documentStore` is * provided — a host can mount with fakes only. */ - config?: KnowledgeConfig; + config?: MemoryConfig; /** * The host's grant store + condition registry — the same pair it passes to * `createApp`/`createRequireGrant`. Required: HTTP routes are guarded with - * `requireGrant("knowledge", )`. The SDK never leaves a route + * `requireGrant("memory", )`. The SDK never leaves a route * unguarded. Also required for in-process `ask()`. */ grants: GrantConfig; @@ -139,38 +139,40 @@ export type MountKnowledgeEngineOptions = { * Optional ask side-channel only (`includeMemory`). Not a DocumentStore * replacement — vendor backends mount as `documentStore`. */ - memory?: MemoryProvider; + memoryProvider?: MemoryProvider; }; -export type MountedKnowledgeEngine = { - knowledge: KnowledgePlane; +export type MountedMemory = { + memory: Memory; }; -/** Mount the knowledge HTTP routes over one knowledge plane. */ -export function mountKnowledgeEngine( +/** Mount the memory HTTP routes over one memory plane. */ +export function mountMemory( app: Hono, - options: MountKnowledgeEngineOptions, -): MountedKnowledgeEngine { - // Rerank config validation runs inside createKnowledgePlane so standalone + options: MountMemoryOptions, +): MountedMemory { + // Rerank config validation runs inside createMemory so standalone // construction and the mount path share one check. Pass grants + generate so // the returned plane's ask() is grant-checked and can synthesize answers. - const planeOptions: KnowledgePlaneOptions = { + const planeOptions: MemoryOptions = { ...(options.generate ? { generate: options.generate } : {}), ...(options.textExtractor ? { textExtractor: options.textExtractor } : {}), ...(options.documentStore ? { documentStore: options.documentStore } : {}), ...(options.sources ? { sources: options.sources } : {}), - ...(options.memory ? { memory: options.memory } : {}), + ...(options.memoryProvider + ? { memoryProvider: options.memoryProvider } + : {}), }; - const knowledge = createKnowledgePlane( + const memory = createMemory( options.config, options.grants, planeOptions, ); const deps: RouteDeps = { - knowledge, + memory, grants: options.grants, requireGrant: createRequireGrant(options.grants), }; - mountKnowledgeRoutes(app, deps); - return { knowledge }; + mountMemoryRoutes(app, deps); + return { memory }; } diff --git a/src/log.ts b/src/log.ts index 7aa64ea..cc1ac12 100644 --- a/src/log.ts +++ b/src/log.ts @@ -1,7 +1,7 @@ import { getLogger } from "@intx/log"; /** Category-bound logger for the knowledge engine (uses the host's sinks). */ -export const log = getLogger(["knowledge-engine"]); +export const log = getLogger(["memory"]); // Some `@intx/log` sinks do not render a call's context object into the // terminal/aggregator output — only the message string is guaranteed to diff --git a/src/knowledge.test.ts b/src/memory.test.ts similarity index 85% rename from src/knowledge.test.ts rename to src/memory.test.ts index 977b5de..c8ef664 100644 --- a/src/knowledge.test.ts +++ b/src/memory.test.ts @@ -1,7 +1,7 @@ /** - * Plane construction, grant-tag ACL wiring, and ask() coverage for knowledge.ts. + * Plane construction, grant-tag ACL wiring, and ask() coverage for memory.ts. * - * - Construction: rerank maxDocChars validation runs in createKnowledgePlane. + * - Construction: rerank maxDocChars validation runs in createMemory. * - Find wiring: grant-tag post-filter (creator-only without grants). * - ask(): grant check, missing generate (501 before find), allow path, * synthesizeAnswer grounding. @@ -22,16 +22,16 @@ import type { GrantRule } from "@intx/authz"; import { RerankConfigError } from "./core/rerank-client.ts"; import type { SearchHit } from "./core/schemas/search.ts"; import { - createKnowledgePlane, - KnowledgeError, - KnowledgeNotPermittedError, + createMemory, + MemoryError, + MemoryNotPermittedError, synthesizeAnswer, type ChatMessage, type FindItem, - type KnowledgeAddParams, + type MemoryAddParams, type TextExtractor, -} from "./knowledge.ts"; -import type { KnowledgeConfig } from "./mount-config.ts"; +} from "./memory.ts"; +import type { MemoryConfig } from "./mount-config.ts"; import * as realDb from "./db/client.ts"; import * as realSearch from "./services/search.ts"; import * as realCapture from "./services/capture.ts"; @@ -52,7 +52,7 @@ const ENGLISH_FTS_EXPR = "to_tsvector('english'::regconfig, text)"; function grant(action: string): GrantRule { return { id: `g-${action}`, - resource: "knowledge", + resource: "memory", action, effect: "allow", origin: "role", @@ -119,8 +119,8 @@ function findItemFromHit(h: SearchHit): FindItem { }; } -const wiringConfig: KnowledgeConfig = { - knowledge: { +const wiringConfig: MemoryConfig = { + memory: { databaseUrl: "postgres://localhost:5432/nonexistent-test-db", dbPoolMax: 1, ftsLanguage: "english", @@ -139,7 +139,7 @@ const wiringConfig: KnowledgeConfig = { }, }; -const askConfig: KnowledgeConfig = wiringConfig; +const askConfig: MemoryConfig = wiringConfig; function ftsUnsafe(sqlText: string): Promise>> { if (sqlText.includes("pg_ts_config")) { @@ -149,10 +149,10 @@ function ftsUnsafe(sqlText: string): Promise>> { } function baseConfig( - rerank: KnowledgeConfig["knowledge"]["rerank"], -): KnowledgeConfig { + rerank: MemoryConfig["memory"]["rerank"], +): MemoryConfig { return { - knowledge: { + memory: { // Validation runs before createDb — a bad URL is fine as long as we throw // first and never open a connection. databaseUrl: "postgres://localhost:5432/nonexistent-test-db", @@ -169,13 +169,13 @@ function baseConfig( }; } -describe("createKnowledgePlane — construction validation", () => { +describe("createMemory — construction validation", () => { it("throws RerankConfigError when maxDocChars overflows a known TEI model", () => { - // Proves validateRerankConfig runs inside createKnowledgePlane (not only - // mountKnowledgeEngine): a standalone plane with a bad override must fail + // Proves validateRerankConfig runs inside createMemory (not only + // mountMemory): a standalone plane with a bad override must fail // construction, not silently degrade on every later find. expect(() => - createKnowledgePlane( + createMemory( baseConfig({ baseUrl: "https://tei.example.com", model: "bge-reranker-base", @@ -187,7 +187,7 @@ describe("createKnowledgePlane — construction validation", () => { }); }); -describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { +describe("createMemory.find — grant-tag post-filter wiring", () => { const hybridSearch = mock((): Promise => Promise.resolve({ hits: [hit("d-other"), hit("d-mine")], @@ -200,12 +200,12 @@ describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { Promise.resolve([ { id: "d-other", - access_tags: ["knowledge.owner:other"], + access_tags: ["memory.owner:other"], created_by: "other", }, { id: "d-mine", - access_tags: [`knowledge.owner:${PRINCIPAL}`], + access_tags: [`memory.owner:${PRINCIPAL}`], created_by: PRINCIPAL, }, ]), @@ -246,19 +246,19 @@ describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { Promise.resolve([ { id: "d-other", - access_tags: ["knowledge.owner:other"], + access_tags: ["memory.owner:other"], created_by: "other", }, { id: "d-mine", - access_tags: [`knowledge.owner:${PRINCIPAL}`], + access_tags: [`memory.owner:${PRINCIPAL}`], created_by: PRINCIPAL, }, ]), ); - const { createKnowledgePlane: makePlane } = await import( - `./knowledge.ts?wiring-blocked=${Date.now()}` + const { createMemory: makePlane } = await import( + `./memory.ts?wiring-blocked=${Date.now()}` ); const plane = makePlane(wiringConfig); const result = await plane.find({ @@ -287,8 +287,8 @@ describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { ); sql.mockClear(); - const { createKnowledgePlane: makePlane } = await import( - `./knowledge.ts?wiring-kinds=${Date.now()}` + const { createMemory: makePlane } = await import( + `./memory.ts?wiring-kinds=${Date.now()}` ); const plane = makePlane(wiringConfig); await plane.find({ @@ -321,8 +321,8 @@ describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { sql.mockClear(); sql.mockImplementation(() => Promise.resolve([])); - const { createKnowledgePlane: makePlane } = await import( - `./knowledge.ts?wiring-unreadable=${Date.now()}` + const { createMemory: makePlane } = await import( + `./memory.ts?wiring-unreadable=${Date.now()}` ); const plane = makePlane(wiringConfig); const result = await plane.find({ @@ -352,14 +352,14 @@ describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { Promise.resolve([ { id: "d-open", - access_tags: [`knowledge.owner:${PRINCIPAL}`], + access_tags: [`memory.owner:${PRINCIPAL}`], created_by: PRINCIPAL, }, ]), ); - const { createKnowledgePlane: makePlane } = await import( - `./knowledge.ts?wiring-evidence=${Date.now()}` + const { createMemory: makePlane } = await import( + `./memory.ts?wiring-evidence=${Date.now()}` ); const plane = makePlane(wiringConfig); @@ -389,7 +389,7 @@ describe("createKnowledgePlane.find — grant-tag post-filter wiring", () => { describe("find/recent — limit bounds", () => { // These throw before any DB work, so a nonexistent URL is fine. it("find rejects limit below 1", async () => { - const plane = createKnowledgePlane(wiringConfig); + const plane = createMemory(wiringConfig); try { await plane.find({ tenantId: TENANT, @@ -399,14 +399,14 @@ describe("find/recent — limit bounds", () => { }); throw new Error("expected find() to reject"); } catch (err) { - expect(err).toBeInstanceOf(KnowledgeError); - expect((err as KnowledgeError).status).toBe(400); - expect((err as KnowledgeError).message).toContain("limit"); + expect(err).toBeInstanceOf(MemoryError); + expect((err as MemoryError).status).toBe(400); + expect((err as MemoryError).message).toContain("limit"); } }); it("find rejects limit above 50", async () => { - const plane = createKnowledgePlane(wiringConfig); + const plane = createMemory(wiringConfig); try { await plane.find({ tenantId: TENANT, @@ -416,13 +416,13 @@ describe("find/recent — limit bounds", () => { }); throw new Error("expected find() to reject"); } catch (err) { - expect(err).toBeInstanceOf(KnowledgeError); - expect((err as KnowledgeError).status).toBe(400); + expect(err).toBeInstanceOf(MemoryError); + expect((err as MemoryError).status).toBe(400); } }); it("recent rejects limit above 100", async () => { - const plane = createKnowledgePlane(wiringConfig); + const plane = createMemory(wiringConfig); try { await plane.recent({ tenantId: TENANT, @@ -431,13 +431,13 @@ describe("find/recent — limit bounds", () => { }); throw new Error("expected recent() to reject"); } catch (err) { - expect(err).toBeInstanceOf(KnowledgeError); - expect((err as KnowledgeError).status).toBe(400); + expect(err).toBeInstanceOf(MemoryError); + expect((err as MemoryError).status).toBe(400); } }); it("recent rejects limit below 1", async () => { - const plane = createKnowledgePlane(wiringConfig); + const plane = createMemory(wiringConfig); try { await plane.recent({ tenantId: TENANT, @@ -446,8 +446,8 @@ describe("find/recent — limit bounds", () => { }); throw new Error("expected recent() to reject"); } catch (err) { - expect(err).toBeInstanceOf(KnowledgeError); - expect((err as KnowledgeError).status).toBe(400); + expect(err).toBeInstanceOf(MemoryError); + expect((err as MemoryError).status).toBe(400); } }); }); @@ -493,16 +493,16 @@ describe("add() — documentId, content/file XOR, share", () => { async function freshPlane(opts?: { textExtractor?: TextExtractor; }) { - const { createKnowledgePlane: makePlane } = await import( - `./knowledge.ts?add-${Date.now()}-${Math.random()}` + const { createMemory: makePlane } = await import( + `./memory.ts?add-${Date.now()}-${Math.random()}` ); return makePlane(wiringConfig, undefined, opts ?? {}); } - /** Dynamic re-import yields a distinct KnowledgeError class; match by shape. */ - function expectKnowledgeError400(err: unknown, messagePart: string) { + /** Dynamic re-import yields a distinct MemoryError class; match by shape. */ + function expectMemoryError400(err: unknown, messagePart: string) { expect(err).toBeInstanceOf(Error); - expect((err as Error).name).toBe("KnowledgeError"); + expect((err as Error).name).toBe("MemoryError"); expect((err as { status: number }).status).toBe(400); expect((err as Error).message).toContain(messagePart); } @@ -553,10 +553,10 @@ async function freshPlane(opts?: { await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, - } as KnowledgeAddParams); + } as MemoryAddParams); throw new Error("expected add() to reject"); } catch (err) { - expectKnowledgeError400(err, "content or file"); + expectMemoryError400(err, "content or file"); } await plane.close(); }); @@ -572,7 +572,7 @@ async function freshPlane(opts?: { }); throw new Error("expected add() to reject"); } catch (err) { - expectKnowledgeError400(err, "content or file"); + expectMemoryError400(err, "content or file"); } await plane.close(); }); @@ -587,7 +587,7 @@ async function freshPlane(opts?: { }); throw new Error("expected add() to reject"); } catch (err) { - expectKnowledgeError400(err, "textExtractor"); + expectMemoryError400(err, "textExtractor"); } await plane.close(); }); @@ -655,10 +655,10 @@ async function freshPlane(opts?: { }, ]; expect(call[1].document.accessTags).toContain( - `knowledge.owner:${PRINCIPAL}`, + `memory.owner:${PRINCIPAL}`, ); expect(call[1].document.accessTags).toContain( - `knowledge.tenant:${TENANT}`, + `memory.tenant:${TENANT}`, ); await plane.close(); }); @@ -689,10 +689,10 @@ async function freshPlane(opts?: { }, ]; expect(call[1].document.accessTags).toContain( - `knowledge.owner:${PRINCIPAL}`, + `memory.owner:${PRINCIPAL}`, ); - expect(call[1].document.accessTags).toContain("knowledge.owner:alice"); - expect(call[1].document.accessTags).toContain("knowledge.owner:bob"); + expect(call[1].document.accessTags).toContain("memory.owner:alice"); + expect(call[1].document.accessTags).toContain("memory.owner:bob"); await plane.close(); }); @@ -721,22 +721,22 @@ async function freshPlane(opts?: { }, ]; expect(call[1].document.accessTags).toEqual([ - `knowledge.owner:${PRINCIPAL}`, + `memory.owner:${PRINCIPAL}`, ]); await plane.close(); }); }); describe("ask() — grant check", () => { - it("denies with KnowledgeNotPermittedError when no grant matches (effect: null)", async () => { + it("denies with MemoryNotPermittedError when no grant matches (effect: null)", async () => { const grants = { grantStore: createInMemoryGrantStore([]), conditionRegistry: {}, }; - const plane = createKnowledgePlane(askConfig, grants); + const plane = createMemory(askConfig, grants); await expect( plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), - ).rejects.toBeInstanceOf(KnowledgeNotPermittedError); + ).rejects.toBeInstanceOf(MemoryNotPermittedError); }); it("denies when the only matching grant is an explicit deny", async () => { @@ -745,29 +745,29 @@ const denyGrant: GrantRule = { ...grant("find"), effect: "deny" }; grantStore: createInMemoryGrantStore([denyGrant]), conditionRegistry: {}, }; - const plane = createKnowledgePlane(askConfig, grants); + const plane = createMemory(askConfig, grants); await expect( plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), - ).rejects.toBeInstanceOf(KnowledgeNotPermittedError); + ).rejects.toBeInstanceOf(MemoryNotPermittedError); }); }); describe("ask() — missing generate", () => { - it("throws KnowledgeError 501 before find when generate is not wired", async () => { + it("throws MemoryError 501 before find when generate is not wired", async () => { // Pointed at a nonexistent DB: if find ran first this would surface a // connection/driver error instead of the promised 501. const grants = { grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, }; - const plane = createKnowledgePlane(askConfig, grants); + const plane = createMemory(askConfig, grants); try { await plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }); throw new Error("expected ask() to reject"); } catch (err) { - expect(err).toBeInstanceOf(KnowledgeError); - expect((err as KnowledgeError).status).toBe(501); - expect((err as KnowledgeError).message).toContain("generate"); + expect(err).toBeInstanceOf(MemoryError); + expect((err as MemoryError).status).toBe(501); + expect((err as MemoryError).message).toContain("generate"); } }); }); @@ -785,7 +785,7 @@ describe("ask() — allow path", () => { expect(messages[1]?.content).toContain("the relevant snippet"); return Promise.resolve("Answer from context [1]."); }); - const plane = createKnowledgePlane(askConfig, grants, { generate }); + const plane = createMemory(askConfig, grants, { generate }); // Stub find so this unit test never needs a live Postgres. ask() looks // up plane.find at call time, so reassignment is the wiring under test. plane.find = mock(() => diff --git a/src/knowledge.ts b/src/memory.ts similarity index 90% rename from src/knowledge.ts rename to src/memory.ts index 4037ce2..7789a00 100644 --- a/src/knowledge.ts +++ b/src/memory.ts @@ -17,7 +17,7 @@ import { validateRerankConfig } from "./core/rerank-client.ts"; import { captureDocument } from "./services/capture.ts"; import { hybridSearch, - KnowledgeSearchInputError, + MemorySearchInputError, toRerankClientConfig, type HybridSearchResult, DEFAULT_HYBRID_TOP_K, @@ -34,7 +34,7 @@ import { type MergeDegradeFlag, } from "./core/merge-local-live.ts"; import type { DegradeFlag } from "./core/hybrid-search.ts"; -import type { KnowledgeConfig } from "./mount-config.ts"; +import type { MemoryConfig } from "./mount-config.ts"; import type { GrantConfig } from "./routes/deps.ts"; import type { DocumentStore, @@ -95,7 +95,7 @@ export type TextExtractor = { }): Promise<{ text: string; title?: string }>; }; -export type KnowledgeIdentity = { +export type MemoryIdentity = { principalId: string; tenantId: string; }; @@ -108,7 +108,7 @@ export const FIND_LIMIT_MAX = 50; export const RECENT_LIMIT_MIN = 1; export const RECENT_LIMIT_MAX = 100; -export type KnowledgeFindParams = KnowledgeIdentity & { +export type MemoryFindParams = MemoryIdentity & { query: string; /** Max items to return (1–50). Default 8. */ limit?: number; @@ -133,7 +133,7 @@ export type KnowledgeFindParams = KnowledgeIdentity & { sources?: string[]; }; -export type KnowledgeAskParams = KnowledgeIdentity & { +export type MemoryAskParams = MemoryIdentity & { query: string; limit?: number; /** Same channel filter as find (passed through). */ @@ -162,17 +162,17 @@ export type AskResult = { degraded?: HybridSearchResult["degraded"]; }; -/** Thrown when the asking principal lacks the knowledge:find capability. */ -export class KnowledgeNotPermittedError extends Error { +/** Thrown when the asking principal lacks the memory:find capability. */ +export class MemoryNotPermittedError extends Error { constructor() { - super("principal lacks the knowledge:find grant"); - this.name = "KnowledgeNotPermittedError"; + super("principal lacks the memory:find grant"); + this.name = "MemoryNotPermittedError"; } } -export type KnowledgeShare = ShareSugar; +export type MemoryShare = ShareSugar; -export type KnowledgeAddParams = KnowledgeIdentity & { +export type MemoryAddParams = MemoryIdentity & { /** Exactly one of `content` or `file` is required. */ content?: { title: string; text: string }; file?: { @@ -196,7 +196,7 @@ export type KnowledgeAddParams = KnowledgeIdentity & { attributes?: Record; }; -export type KnowledgeAddResult = { documentId: string }; +export type MemoryAddResult = { documentId: string }; export type FindItem = { documentId: string; @@ -216,49 +216,49 @@ export type FindResult = { degraded?: HybridSearchResult["degraded"]; }; -export type KnowledgeRecentParams = KnowledgeIdentity & { +export type MemoryRecentParams = MemoryIdentity & { limit?: number; }; -export class KnowledgeError extends Error { +export class MemoryError extends Error { constructor( public readonly status: number, message: string, ) { super(message); - this.name = "KnowledgeError"; + this.name = "MemoryError"; } } -export type KnowledgePlane = { - find(params: KnowledgeFindParams): Promise; - ask(params: KnowledgeAskParams): Promise; - add(params: KnowledgeAddParams): Promise; - recent(params: KnowledgeRecentParams): Promise; +export type Memory = { + find(params: MemoryFindParams): Promise; + ask(params: MemoryAskParams): Promise; + add(params: MemoryAddParams): Promise; + recent(params: MemoryRecentParams): Promise; /** * Write a memory fact for a principal. Requires a mounted MemoryProvider; * throws 501 when memory is not configured. Never called implicitly by ask. */ - remember(params: KnowledgeRememberParams): Promise; + remember(params: MemoryRememberParams): Promise; /** * Recall memory facts for a principal. Empty array when memory is not * configured or nothing matches. */ - recall(params: KnowledgeRecallParams): Promise; + recall(params: MemoryRecallParams): Promise; close(): Promise; }; -export type KnowledgeRememberParams = KnowledgeIdentity & { +export type MemoryRememberParams = MemoryIdentity & { text: string; metadata?: Record; }; -export type KnowledgeRecallParams = KnowledgeIdentity & { +export type MemoryRecallParams = MemoryIdentity & { query: string; limit?: number; }; -export type KnowledgeRecallItem = { +export type MemoryRecallItem = { text: string; score?: number; }; @@ -373,7 +373,7 @@ export async function synthesizeAnswer( }; } -export type KnowledgePlaneOptions = { +export type MemoryOptions = { /** Required for `ask()`; omit if the host only adds and finds. */ generate?: Generate; /** Required for `add({ file })`; omit if the host only adds text content. */ @@ -393,7 +393,7 @@ export type KnowledgePlaneOptions = { * Optional personal-memory side channel for ask(includeMemory). * Not how you swap durable backends — use documentStore for that. */ - memory?: MemoryProvider; + memoryProvider?: MemoryProvider; }; function resolveFindLimit(limit: number | undefined): number { @@ -404,7 +404,7 @@ function resolveFindLimit(limit: number | undefined): number { limit < FIND_LIMIT_MIN || limit > FIND_LIMIT_MAX ) { - throw new KnowledgeError( + throw new MemoryError( 400, `limit must be an integer between ${FIND_LIMIT_MIN} and ${FIND_LIMIT_MAX}`, ); @@ -420,7 +420,7 @@ function resolveRecentLimit(limit: number | undefined): number | undefined { limit < RECENT_LIMIT_MIN || limit > RECENT_LIMIT_MAX ) { - throw new KnowledgeError( + throw new MemoryError( 400, `limit must be an integer between ${RECENT_LIMIT_MIN} and ${RECENT_LIMIT_MAX}`, ); @@ -461,7 +461,7 @@ function findItemsToHits(items: readonly FindItem[]): SearchHit[] { /** * Resolve access tags for add — share sugar + explicit tags only. */ -function resolveAddAccessTags(params: KnowledgeAddParams): string[] { +function resolveAddAccessTags(params: MemoryAddParams): string[] { return resolveAccessTags({ principalId: params.principalId, tenantId: params.tenantId, @@ -484,18 +484,18 @@ function resolveAddAccessTags(params: KnowledgeAddParams): string[] { * MergeLocalLiveV1 (fail-soft, 800ms timeout, prefer-local dedupe). * - Document access uses grant tags via the host GrantStore (not mini-ACL). */ -export function createKnowledgePlane( - config: KnowledgeConfig | undefined, +export function createMemory( + config: MemoryConfig | undefined, grants?: GrantConfig, - options: KnowledgePlaneOptions = {}, -): KnowledgePlane { + options: MemoryOptions = {}, +): Memory { const store = options.documentStore ?? (() => { if (!config) { - throw new KnowledgeError( + throw new MemoryError( 500, - "KnowledgeConfig is required when documentStore is not provided", + "MemoryConfig is required when documentStore is not provided", ); } return createEngineDocumentStore(config); @@ -684,20 +684,20 @@ async function recallForAsk(params: { } } -function makeRememberRecall(options: KnowledgePlaneOptions): { - remember: KnowledgePlane["remember"]; - recall: KnowledgePlane["recall"]; +function makeRememberRecall(options: MemoryOptions): { + remember: Memory["remember"]; + recall: Memory["recall"]; } { return { async remember(params) { - if (!options.memory) { - throw new KnowledgeError( + if (!options.memoryProvider) { + throw new MemoryError( 501, - "remember() requires a MemoryProvider. Pass memory to " + - "createKnowledgePlane/mountKnowledgeEngine.", + "remember() requires a MemoryProvider. Pass memoryProvider to " + + "createMemory/mountMemory.", ); } - await options.memory.remember({ + await options.memoryProvider.remember({ tenantId: params.tenantId, principalId: params.principalId, text: params.text, @@ -705,8 +705,8 @@ function makeRememberRecall(options: KnowledgePlaneOptions): { }); }, async recall(params) { - if (!options.memory) return []; - return options.memory.recall({ + if (!options.memoryProvider) return []; + return options.memoryProvider.recall({ tenantId: params.tenantId, principalId: params.principalId, query: params.query, @@ -720,12 +720,12 @@ function makeRememberRecall(options: KnowledgePlaneOptions): { function createPlaneFromStore( store: DocumentStore, grants: GrantConfig | undefined, - options: KnowledgePlaneOptions, -): KnowledgePlane { + options: MemoryOptions, +): Memory { const memoryApi = makeRememberRecall(options); async function findMerged( - params: KnowledgeFindParams, + params: MemoryFindParams, ): Promise { const limit = resolveFindLimit(params.limit); let localItems: FindItem[] = []; @@ -800,30 +800,30 @@ function createPlaneFromStore( }); } - const plane: KnowledgePlane = { + const plane: Memory = { async find(params) { return findMerged(params); }, async ask(params) { // Capability layer. Callers reaching the plane in-process bypass the - // HTTP surface's `requireGrant("knowledge", ...)` route guard, so the + // HTTP surface's `requireGrant("memory", ...)` route guard, so the // check has to live here — AUTH.md is explicit that the capability and // data layers are independent and BOTH must allow. Per-document // grant-tag access (enforced inside the store) is not a substitute for "may // this principal search at all". if (!grants) { - throw new KnowledgeError( + throw new MemoryError( 501, "ask() requires a GrantConfig. Pass grants to " + - "createKnowledgePlane/mountKnowledgeEngine.", + "createMemory/mountMemory.", ); } const decision = await authorize( grants.grantStore, params.principalId, params.tenantId, - "knowledge", + "memory", "find", grants.conditionRegistry, ); @@ -832,21 +832,21 @@ function createPlaneFromStore( if (decision.effect !== "allow") { const effect = decision.effect ?? "no-matching-grant"; log.info( - `ask: denied knowledge:find for ${params.principalId} (effect=${effect})`, + `ask: denied memory:find for ${params.principalId} (effect=${effect})`, { principalId: params.principalId, effect, }, ); - throw new KnowledgeNotPermittedError(); + throw new MemoryNotPermittedError(); } // Fail closed on missing generate *before* retrieval so a misconfigured // host gets the promised 501 instead of paying for search. if (!options.generate) { - throw new KnowledgeError( + throw new MemoryError( 501, "ask() requires a `generate` function. Pass one to " + - "createKnowledgePlane/mountKnowledgeEngine, wired to your " + + "createMemory/mountMemory, wired to your " + "inference layer.", ); } @@ -859,7 +859,7 @@ function createPlaneFromStore( ...(params.sources !== undefined ? { sources: params.sources } : {}), }); const mem = await recallForAsk({ - memory: options.memory, + memory: options.memoryProvider, includeMemory: params.includeMemory, tenantId: params.tenantId, principalId: params.principalId, @@ -888,7 +888,7 @@ function createPlaneFromStore( const hasContent = params.content !== undefined; const hasFile = params.file !== undefined; if (hasContent === hasFile) { - throw new KnowledgeError( + throw new MemoryError( 400, "provide exactly one of content or file", ); @@ -902,7 +902,7 @@ function createPlaneFromStore( } else { const file = params.file!; if (!options.textExtractor) { - throw new KnowledgeError( + throw new MemoryError( 400, "file requires a textExtractor on the knowledge plane", ); @@ -921,7 +921,7 @@ function createPlaneFromStore( const externalRef = params.externalRef ?? - `knowledge:${params.tenantId}:${crypto.randomUUID()}`; + `memory:${params.tenantId}:${crypto.randomUUID()}`; return store.add({ tenantId: params.tenantId, @@ -968,7 +968,7 @@ function createPlaneFromStore( * Owns construction-time rerank validation, FTS verification, and grant-tag * post-filter for document access. The plane never opens Postgres itself. */ -function createEngineDocumentStore(config: KnowledgeConfig): DocumentStore { +function createEngineDocumentStore(config: MemoryConfig): DocumentStore { // Catch a chunk-size / reranker-limit mismatch at construction time, rather // than silently on every find once the reranker starts rejecting batches. // Throws instead of warning: a mismatch means every rerank call for this @@ -978,18 +978,18 @@ function createEngineDocumentStore(config: KnowledgeConfig): DocumentStore { // (`defaultMaxDocCharsForModel`) is self-consistent by construction — // validation can only fire on an operator's own `maxDocChars` override, // never spuriously on an unmodified config. - // Lives here (not only in mountKnowledgeEngine) so standalone construction + // Lives here (not only in mountMemory) so standalone construction // cannot silently degrade on a bad override. - const rerankConfig = toRerankClientConfig(config.knowledge.rerank); + const rerankConfig = toRerankClientConfig(config.memory.rerank); if (rerankConfig) validateRerankConfig(rerankConfig); // Resolve once here so EngineConfig.ftsLanguage is concrete for every - // service — loadKnowledgeConfig already runs parseFtsLanguage, but a + // service — loadMemoryConfig already runs parseFtsLanguage, but a // hand-built EngineConfig may still carry an empty/absent value; this is // the single defaulting site services rely on. const engineConfig: EngineConfig = { - ...config.knowledge, - ftsLanguage: parseFtsLanguage(config.knowledge.ftsLanguage), + ...config.memory, + ftsLanguage: parseFtsLanguage(config.memory.ftsLanguage), }; const { db, sql }: { db: Db; sql: RawSql } = createDb(engineConfig); const deps = { db, sql, config: engineConfig }; @@ -1000,7 +1000,7 @@ function createEngineDocumentStore(config: KnowledgeConfig): DocumentStore { // awaited by the first query. Read-only; migration stays a deploy step. // NOTE this is a lazy check, not a boot-time one: nothing forces it to run // until the first real find()/add() call, so a host that neither - // runs runKnowledgeMigrations itself nor wires a readiness probe will not + // runs runMemoryMigrations itself nor wires a readiness probe will not // learn about a language mismatch until that first call fails. A host // that wants a real boot-time guarantee MUST call the exported // verifyFtsLanguage from its own readiness probe — this memo then @@ -1103,8 +1103,8 @@ function createEngineDocumentStore(config: KnowledgeConfig): DocumentStore { evidence: hits.length === 0 ? "none" : result.evidence, }; } catch (err) { - if (err instanceof KnowledgeSearchInputError) { - throw new KnowledgeError(400, err.message); + if (err instanceof MemorySearchInputError) { + throw new MemoryError(400, err.message); } throw err; } @@ -1117,7 +1117,7 @@ function createEngineDocumentStore(config: KnowledgeConfig): DocumentStore { const adapter = params.adapter ?? "http"; const externalRef = params.externalRef ?? - `knowledge:${params.tenantId}:${crypto.randomUUID()}`; + `memory:${params.tenantId}:${crypto.randomUUID()}`; const accessTags = params.accessTags ?? [ownerTag(params.principalId)]; const captureResult = await captureDocument(deps, { diff --git a/src/migrations.ts b/src/migrations.ts index b77b7db..bb51496 100644 --- a/src/migrations.ts +++ b/src/migrations.ts @@ -1,5 +1,5 @@ /** - * Knowledge-plane (pgvector) schema migrations, callable by host apps. + * Memory-plane (pgvector) schema migrations, callable by host apps. * Applies every migrations/*.sql in filename order, each in its own * transaction, tracked in knowledge._migrations so re-runs are idempotent * and the ledger never collides with a host's public migration bookkeeping. @@ -17,12 +17,12 @@ import { KNOWLEDGE_SCHEMA } from "./db/schema.ts"; const MIGRATIONS_DIR = join(import.meta.dir, "..", "migrations"); -export async function runKnowledgeMigrations( +export async function runMemoryMigrations( databaseUrl: string, opts: { log?: (line: string) => void; ftsLanguage?: string } = {}, ): Promise { const log = opts.log ?? (() => {}); - // This runner is an env-driven boundary like loadKnowledgeConfig: when the + // This runner is an env-driven boundary like loadMemoryConfig: when the // caller does not pass a language it reads the same FTS_LANGUAGE the query // side will, so the two cannot diverge by defaulting differently. const ftsLanguage = parseFtsLanguage( diff --git a/src/mount-config.ts b/src/mount-config.ts index ec72f44..9eb72f8 100644 --- a/src/mount-config.ts +++ b/src/mount-config.ts @@ -2,15 +2,15 @@ import type { EngineConfig } from "./config.ts"; import { parseFtsLanguage } from "./core/fts-language.ts"; /** - * SDK mount config — what `mountKnowledgeEngine` consumes. + * SDK mount config — what `mountMemory` consumes. * * The host Interchange app owns auth, tenancy, grants, and the process. This - * config carries only what the knowledge engine itself needs: its vector DB + + * config carries only what the memory engine itself needs: its vector DB + * model endpoints. */ -export type KnowledgeConfig = { - /** Knowledge / vector plane (the engine's own DB + model endpoints). */ - knowledge: EngineConfig; +export type MemoryConfig = { + /** Memory / vector plane (the engine's own DB + model endpoints). */ + memory: EngineConfig; }; function requireEnv(name: string): string { @@ -49,11 +49,11 @@ function optionalIntEnv(name: string): number | undefined { /** * Build a config from environment variables — a convenience for env-driven - * deploys. Hosts may also construct `KnowledgeConfig` programmatically. + * deploys. Hosts may also construct `MemoryConfig` programmatically. */ -export function loadKnowledgeConfig(): KnowledgeConfig { +export function loadMemoryConfig(): MemoryConfig { return { - knowledge: { + memory: { // Deliberately no DATABASE_URL fallback: the host app's own database // must never be mistaken for the engine's vector plane. databaseUrl: requireEnv("KNOWLEDGE_DATABASE_URL"), diff --git a/src/ports/memory-plane.test.ts b/src/ports/memory-plane.test.ts index e435adc..630372f 100644 --- a/src/ports/memory-plane.test.ts +++ b/src/ports/memory-plane.test.ts @@ -11,8 +11,8 @@ import { import { createFakeDocumentStore, createFakeMemoryProvider, - createKnowledgePlane, - KnowledgeError, + createMemory, + MemoryError, } from "../index.ts"; import type { MemoryProvider } from "./types.ts"; @@ -22,7 +22,7 @@ const PRINCIPAL = "p_mem"; function grant(action: string): GrantRule { return { id: `g-${action}`, - resource: "knowledge", + resource: "memory", action, effect: "allow", origin: "role", @@ -44,7 +44,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, }; const store = createFakeDocumentStore(); - const plane = createKnowledgePlane( + const plane = createMemory( undefined, { grantStore: createInMemoryGrantStore([grant("find")]), @@ -52,7 +52,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, { documentStore: store, - memory, + memoryProvider: memory, generate: async (msgs) => { const last = msgs[msgs.length - 1]?.content ?? ""; return last.includes("Personal memory") ? "HAS_MEM" : "NO_MEM"; @@ -82,7 +82,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { text: "user prefers dark mode", }); const store = createFakeDocumentStore(); - const plane = createKnowledgePlane( + const plane = createMemory( undefined, { grantStore: createInMemoryGrantStore([grant("find")]), @@ -90,7 +90,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, { documentStore: store, - memory, + memoryProvider: memory, generate: async (msgs) => { const last = msgs[msgs.length - 1]?.content ?? ""; return last.includes("user prefers dark mode") @@ -122,7 +122,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, }; const store = createFakeDocumentStore(); - const plane = createKnowledgePlane( + const plane = createMemory( undefined, { grantStore: createInMemoryGrantStore([grant("find")]), @@ -130,7 +130,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, { documentStore: store, - memory, + memoryProvider: memory, generate: async () => "docs-only [1]", }, ); @@ -152,9 +152,9 @@ describe("MemoryProvider product wire (CL-5228)", () => { it("plane.remember writes; plane.recall reads", async () => { const memory = createFakeMemoryProvider(); - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: createFakeDocumentStore(), - memory, + memoryProvider: memory, }); await plane.remember({ tenantId: TENANT, @@ -171,7 +171,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }); it("plane.remember without memory throws 501", async () => { - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: createFakeDocumentStore(), }); try { @@ -182,14 +182,14 @@ describe("MemoryProvider product wire (CL-5228)", () => { }); expect.unreachable("should throw"); } catch (err) { - expect(err).toBeInstanceOf(KnowledgeError); - expect((err as KnowledgeError).status).toBe(501); + expect(err).toBeInstanceOf(MemoryError); + expect((err as MemoryError).status).toBe(501); } await plane.close(); }); it("plane.recall without memory returns empty", async () => { - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: createFakeDocumentStore(), }); const items = await plane.recall({ diff --git a/src/ports/merge-plane.test.ts b/src/ports/merge-plane.test.ts index e5c62c2..34acf73 100644 --- a/src/ports/merge-plane.test.ts +++ b/src/ports/merge-plane.test.ts @@ -10,7 +10,7 @@ import { import { createFakeDocumentStore, createFakeSourceProvider, - createKnowledgePlane, + createMemory, } from "../index.ts"; import type { LiveSearchItem } from "./types.ts"; @@ -20,7 +20,7 @@ const PRINCIPAL = "p_merge"; function grant(action: string): GrantRule { return { id: `g-${action}`, - resource: "knowledge", + resource: "memory", action, effect: "allow", origin: "role", @@ -54,7 +54,7 @@ function liveHit( describe("plane merge (MergeLocalLiveV1)", () => { it("merges local store hits with live source hits", async () => { const store = createFakeDocumentStore(); - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: store, sources: [ createFakeSourceProvider("linear", [ @@ -83,7 +83,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("includes live-only hits when query matches catalog", async () => { const store = createFakeDocumentStore(); - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: store, sources: [ createFakeSourceProvider("linear", [ @@ -105,7 +105,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("source filter local-only excludes live hits", async () => { const store = createFakeDocumentStore(); - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: store, sources: [ createFakeSourceProvider("linear", [ @@ -152,7 +152,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { }, }; - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: store, sources: [brokenSource, slowSource], }); @@ -181,13 +181,13 @@ describe("plane merge (MergeLocalLiveV1)", () => { principalId: PRINCIPAL, title: "local CL-7 body", text: "collision payload local", - accessTags: [`knowledge.owner:${PRINCIPAL}`], + accessTags: [`memory.owner:${PRINCIPAL}`], externalRef: "CL-7", }); // Fake store citation uses adapter "fake" not linear — force collision by // using a custom store find isn't possible; instead use live adapter // "fake" so keys match fake store's citation adapter. - const plane = createKnowledgePlane(undefined, undefined, { + const plane = createMemory(undefined, undefined, { documentStore: store, sources: [ { @@ -231,7 +231,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("ask still works when live source errors", async () => { const store = createFakeDocumentStore(); - const plane = createKnowledgePlane( + const plane = createMemory( undefined, { grantStore: createInMemoryGrantStore([grant("find")]), diff --git a/src/ports/mount-fakes.test.ts b/src/ports/mount-fakes.test.ts index cd9b289..fe40e78 100644 --- a/src/ports/mount-fakes.test.ts +++ b/src/ports/mount-fakes.test.ts @@ -13,7 +13,7 @@ import { import { createFakeDocumentStore, createFakeSourceProvider, - mountKnowledgeEngine, + mountMemory, } from "../index.ts"; const TENANT = "tenant_fake"; @@ -22,7 +22,7 @@ const PRINCIPAL = "principal_fake"; function grant(action: string): GrantRule { return { id: `g-${action}`, - resource: "knowledge", + resource: "memory", action, effect: "allow", origin: "role", @@ -87,7 +87,7 @@ describe("mount with fakes only", () => { ]), ]; const app = appWithPrincipal(); - const { knowledge } = mountKnowledgeEngine(app, { + const { memory } = mountMemory(app, { grants: { grantStore: createInMemoryGrantStore([grant("add"), grant("find")]), conditionRegistry: {}, @@ -97,7 +97,7 @@ describe("mount with fakes only", () => { generate: async () => "Answer from local store [1].", }); - const { documentId } = await knowledge.add({ + const { documentId } = await memory.add({ tenantId: TENANT, principalId: PRINCIPAL, content: { @@ -107,7 +107,7 @@ describe("mount with fakes only", () => { }); expect(documentId).toMatch(/^fake_doc_/); - const found = await knowledge.find({ + const found = await memory.find({ tenantId: TENANT, principalId: PRINCIPAL, query: "DocumentStore override", @@ -116,13 +116,13 @@ describe("mount with fakes only", () => { expect(found.items).toHaveLength(1); expect(found.items[0]?.documentId).toBe(documentId); - const recent = await knowledge.recent({ + const recent = await memory.recent({ tenantId: TENANT, principalId: PRINCIPAL, }); expect(recent.some((e) => e.title === "ports note")).toBe(true); - const asked = await knowledge.ask({ + const asked = await memory.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "DocumentStore override", @@ -131,7 +131,7 @@ describe("mount with fakes only", () => { expect(asked.citations.length).toBeGreaterThan(0); // HTTP surface also works without engine config - const addRes = await app.request("/api/knowledge/add", { + const addRes = await app.request("/api/memory/add", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ @@ -143,7 +143,7 @@ describe("mount with fakes only", () => { const addBody = (await addRes.json()) as { documentId: string }; expect(addBody.documentId).toMatch(/^fake_doc_/); - const findRes = await app.request("/api/knowledge/find", { + const findRes = await app.request("/api/memory/find", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ query: "http path" }), @@ -156,6 +156,6 @@ describe("mount with fakes only", () => { findBody.items.some((i) => i.documentId === addBody.documentId), ).toBe(true); - await knowledge.close(); + await memory.close(); }); }); diff --git a/src/routes/add.ts b/src/routes/add.ts index 88e48be..aeb5cd1 100644 --- a/src/routes/add.ts +++ b/src/routes/add.ts @@ -6,7 +6,7 @@ import { type } from "arktype"; import { formatCaughtError, log } from "../log.ts"; import { resolveAccessTags, type ShareSugar } from "../grant-tags.ts"; -import { KnowledgeError } from "../knowledge.ts"; +import { MemoryError } from "../memory.ts"; import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; @@ -31,10 +31,10 @@ const AddResponse = type({ export function mountAddRoute(app: Hono, deps: RouteDeps): void { app.post( - "/api/knowledge/add", + "/api/memory/add", describeRoute({ - tags: ["knowledge"], - summary: "Add a note into the knowledge base", + tags: ["memory"], + summary: "Add a note into memory", responses: { 200: { description: "Added", @@ -44,7 +44,7 @@ export function mountAddRoute(app: Hono, deps: RouteDeps): void { }, 400: { description: "Invalid request or access tags" }, 401: { description: "No principal on the request context" }, - 403: { description: "Missing the knowledge:add grant" }, + 403: { description: "Missing the memory:add grant" }, 502: { description: "add failed" }, }, }), @@ -70,7 +70,7 @@ export function mountAddRoute(app: Hono, deps: RouteDeps): void { } try { - const { documentId } = await deps.knowledge.add({ + const { documentId } = await deps.memory.add({ content: { title, text }, tenantId: scopeId, principalId: subjectId, @@ -79,14 +79,14 @@ export function mountAddRoute(app: Hono, deps: RouteDeps): void { }); return c.json({ documentId }); } catch (err) { - if (err instanceof KnowledgeError) { + if (err instanceof MemoryError) { return c.json( { error: err.message }, err.status as 400 | 501, ); } const errMessage = formatCaughtError(err); - log.error(`knowledge add failed: ${errMessage}`, { error: errMessage }); + log.error(`memory add failed: ${errMessage}`, { error: errMessage }); return c.json({ error: "add failed" }, 502); } }, diff --git a/src/routes/ask.ts b/src/routes/ask.ts index 36d6017..b1b136e 100644 --- a/src/routes/ask.ts +++ b/src/routes/ask.ts @@ -5,9 +5,9 @@ import { type } from "arktype"; import { formatCaughtError, log } from "../log.ts"; import { - KnowledgeError, - KnowledgeNotPermittedError, -} from "../knowledge.ts"; + MemoryError, + MemoryNotPermittedError, +} from "../memory.ts"; import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; @@ -32,10 +32,10 @@ const AskResponse = type({ export function mountAskRoute(app: Hono, deps: RouteDeps): void { app.post( - "/api/knowledge/ask", + "/api/memory/ask", describeRoute({ - tags: ["knowledge"], - summary: "Answer a question from retrieved knowledge", + tags: ["memory"], + summary: "Answer a question from retrieved memory", responses: { 200: { description: "Grounded answer with citations", @@ -45,7 +45,7 @@ export function mountAskRoute(app: Hono, deps: RouteDeps): void { }, 400: { description: "Invalid query" }, 401: { description: "No principal on the request context" }, - 403: { description: "Missing the knowledge:find grant" }, + 403: { description: "Missing the memory:find grant" }, 501: { description: "ask is not configured (no generate)" }, 502: { description: "ask failed" }, }, @@ -58,7 +58,7 @@ export function mountAskRoute(app: Hono, deps: RouteDeps): void { const { query, limit, sources, includeMemory } = c.req.valid("json"); const { scopeId, subjectId } = caller(c); try { - const result = await deps.knowledge.ask({ + const result = await deps.memory.ask({ query, tenantId: scopeId, principalId: subjectId, @@ -68,17 +68,17 @@ export function mountAskRoute(app: Hono, deps: RouteDeps): void { }); return c.json(result); } catch (err) { - if (err instanceof KnowledgeNotPermittedError) { + if (err instanceof MemoryNotPermittedError) { return c.json({ error: err.message }, 403); } - if (err instanceof KnowledgeError) { + if (err instanceof MemoryError) { return c.json( { error: err.message }, err.status as 400 | 501, ); } const errMessage = formatCaughtError(err); - log.error(`knowledge ask failed: ${errMessage}`, { err }); + log.error(`memory ask failed: ${errMessage}`, { err }); return c.json({ error: "ask failed" }, 502); } }, diff --git a/src/routes/deps.test.ts b/src/routes/deps.test.ts index b9fd41d..0f4e2c2 100644 --- a/src/routes/deps.test.ts +++ b/src/routes/deps.test.ts @@ -14,7 +14,7 @@ import { function grant(principalId: string, action: string): GrantRule { return { id: `g-${action}`, - resource: "knowledge", + resource: "memory", action, effect: "allow", origin: "role", @@ -30,7 +30,7 @@ const noopRequireGrant: RequireGrant = () => (async () => {}) as never; // Minimal RouteDeps for unit tests — routes here only touch grants/requireGrant. function deps(grants: RouteDeps["grants"], requireGrant = noopRequireGrant): RouteDeps { return { - knowledge: {} as RouteDeps["knowledge"], + memory: {} as RouteDeps["memory"], grants, requireGrant, }; @@ -105,13 +105,13 @@ describe("requirePrincipal", () => { }); describe("grantGuard", () => { - test("delegates to the host requireGrant('knowledge', action)", () => { + test("delegates to the host requireGrant('memory', action)", () => { let called: { resource: string; action: string } | undefined; const requireGrant: RequireGrant = (resource, action) => { called = { resource: String(resource), action }; return (async () => {}) as never; }; grantGuard(deps(grantsWith(), requireGrant), "add"); - expect(called).toEqual({ resource: "knowledge", action: "add" }); + expect(called).toEqual({ resource: "memory", action: "add" }); }); }); diff --git a/src/routes/deps.ts b/src/routes/deps.ts index 708a782..74acf55 100644 --- a/src/routes/deps.ts +++ b/src/routes/deps.ts @@ -2,7 +2,7 @@ import type { Context, MiddlewareHandler } from "hono"; import type { RequireGrant, TenantEnv } from "@intx/hub-api"; import type { ConditionRegistry, GrantStore } from "@intx/authz"; -import type { KnowledgePlane } from "../knowledge.ts"; +import type { Memory } from "../memory.ts"; /** * The host's grant store + condition registry — the same pair it feeds @@ -14,7 +14,7 @@ export type GrantConfig = { }; export type RouteDeps = { - knowledge: KnowledgePlane; + memory: Memory; /** Route-guard middleware factory (Interchange `createRequireGrant`). */ requireGrant: RequireGrant; /** The grant store, kept for callers that need imperative checks. */ @@ -29,7 +29,7 @@ export function caller(c: Context): { const principal = c.get("principal"); if (!principal) { throw new Error( - "knowledge-engine: no principal on the request context. Mount below the " + + "memory: no principal on the request context. Mount below the " + "host's auth + tenant middleware (the routes require TenantEnv).", ); } @@ -46,10 +46,10 @@ export function caller(c: Context): { * rather than as the host missing middleware. `caller()` below has a perfectly * good error message for exactly this case, but it never gets to run. * - * These routes mount at `/api/knowledge/*`, outside the + * These routes mount at `/api/memory/*`, outside the * `/api/tenants/:tenantId/*` prefix that Interchange's `createResolveTenant` * covers, so an unresolved context is the DEFAULT for a host that just calls - * `mountKnowledgeEngine`. See the README for the middleware the host supplies. + * `mountMemory`. See the README for the middleware the host supplies. */ export function requirePrincipal(): MiddlewareHandler { return async (c, next) => { @@ -59,11 +59,11 @@ export function requirePrincipal(): MiddlewareHandler { error: { code: "principal_required", message: - "No principal on the request context. The knowledge routes mount " + - "at /api/knowledge/*, which is outside Interchange's " + + "No principal on the request context. The memory routes mount " + + "at /api/memory/*, which is outside Interchange's " + "/api/tenants/:tenantId/* tenant middleware — the host must " + "resolve tenant + principal for this prefix. See the " + - "@corbits/knowledge-engine README.", + "@corbits/memory README.", }, }, 401, @@ -73,10 +73,10 @@ export function requirePrincipal(): MiddlewareHandler { }; } -/** Route-guard middleware for a knowledge action (Interchange `requireGrant`). */ +/** Route-guard middleware for a memory action (Interchange `requireGrant`). */ export function grantGuard( deps: RouteDeps, action: string, ): MiddlewareHandler { - return deps.requireGrant("knowledge", action); + return deps.requireGrant("memory", action); } diff --git a/src/routes/find.ts b/src/routes/find.ts index c8ec46f..70278df 100644 --- a/src/routes/find.ts +++ b/src/routes/find.ts @@ -4,12 +4,12 @@ import { describeRoute, resolver, validator } from "hono-openapi"; import { type } from "arktype"; import { formatCaughtError, log } from "../log.ts"; -import { KnowledgeError } from "../knowledge.ts"; +import { MemoryError } from "../memory.ts"; import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; // `kinds`/`entity_ids` scope every retrieval channel — see the -// `kinds`/`entityIds` doc comments on KnowledgeFindParams (knowledge.ts) +// `kinds`/`entityIds` doc comments on MemoryFindParams (memory.ts) // for the full explanation. // // An empty array on either field is equivalent to omitting it — "no filter" @@ -39,9 +39,9 @@ const FindResponse = type({ export function mountFindRoute(app: Hono, deps: RouteDeps): void { app.post( - "/api/knowledge/find", + "/api/memory/find", describeRoute({ - tags: ["knowledge"], + tags: ["memory"], summary: "Hybrid semantic + keyword find", description: "`kinds`/`entity_ids` scope every retrieval channel (lexical and " + @@ -57,7 +57,7 @@ export function mountFindRoute(app: Hono, deps: RouteDeps): void { }, 400: { description: "Invalid query" }, 401: { description: "No principal on the request context" }, - 403: { description: "Missing the knowledge:find grant" }, + 403: { description: "Missing the memory:find grant" }, 502: { description: "find failed" }, }, }), @@ -69,7 +69,7 @@ export function mountFindRoute(app: Hono, deps: RouteDeps): void { c.req.valid("json"); const { scopeId, subjectId } = caller(c); try { - const result = await deps.knowledge.find({ + const result = await deps.memory.find({ query, tenantId: scopeId, principalId: subjectId, @@ -84,8 +84,8 @@ export function mountFindRoute(app: Hono, deps: RouteDeps): void { return c.json(result); } catch (err) { const errMessage = formatCaughtError(err); - log.error(`knowledge find failed: ${errMessage}`, { err }); - if (err instanceof KnowledgeError) { + log.error(`memory find failed: ${errMessage}`, { err }); + if (err instanceof MemoryError) { return c.json({ error: err.message }, err.status as 400); } return c.json({ error: "find failed" }, 502); diff --git a/src/routes/mount.ts b/src/routes/mount.ts index 2534f50..9472d89 100644 --- a/src/routes/mount.ts +++ b/src/routes/mount.ts @@ -1,5 +1,5 @@ /** - * Mount the knowledge engine HTTP routes onto a host Interchange app. + * Mount the memory engine HTTP routes onto a host Interchange app. * (MCP moved out to the standalone @corbitsdev/hono-openapi-mcp bridge.) */ import type { Hono } from "hono"; @@ -14,7 +14,7 @@ import { mountRecentRoute } from "./recent.ts"; export type { GrantConfig, RouteDeps } from "./deps.ts"; /** HTTP JSON routes: add, find, ask, recent. */ -export function mountKnowledgeRoutes( +export function mountMemoryRoutes( app: Hono, deps: RouteDeps, ): void { diff --git a/src/routes/recent.ts b/src/routes/recent.ts index 9ff01d5..1f1fd28 100644 --- a/src/routes/recent.ts +++ b/src/routes/recent.ts @@ -5,10 +5,10 @@ import { type } from "arktype"; import { formatCaughtError, log } from "../log.ts"; import { - KnowledgeError, + MemoryError, RECENT_LIMIT_MAX, RECENT_LIMIT_MIN, -} from "../knowledge.ts"; +} from "../memory.ts"; import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; @@ -41,9 +41,9 @@ function parseLimit(raw: string | undefined): number | undefined { export function mountRecentRoute(app: Hono, deps: RouteDeps): void { app.get( - "/api/knowledge/recent", + "/api/memory/recent", describeRoute({ - tags: ["knowledge"], + tags: ["memory"], summary: "Recent documents for the caller's scope", responses: { 200: { @@ -54,7 +54,7 @@ export function mountRecentRoute(app: Hono, deps: RouteDeps): void { }, 400: { description: "Invalid limit query param" }, 401: { description: "No principal on the request context" }, - 403: { description: "Missing the knowledge:find grant" }, + 403: { description: "Missing the memory:find grant" }, 502: { description: "Recent query failed" }, }, }), @@ -78,18 +78,18 @@ export function mountRecentRoute(app: Hono, deps: RouteDeps): void { } const limit = parseLimit(rawLimit); try { - const events = await deps.knowledge.recent({ + const events = await deps.memory.recent({ tenantId: scopeId, principalId: subjectId, ...(limit !== undefined ? { limit } : {}), }); return c.json({ events }); } catch (err) { - if (err instanceof KnowledgeError) { + if (err instanceof MemoryError) { return c.json({ error: err.message }, err.status as 400); } const errMessage = formatCaughtError(err); - log.error(`knowledge recent failed: ${errMessage}`, { + log.error(`memory recent failed: ${errMessage}`, { error: errMessage, }); return c.json({ error: "recent failed" }, 502); diff --git a/src/routes/routes.test.ts b/src/routes/routes.test.ts index 8334fe2..deb1f4f 100644 --- a/src/routes/routes.test.ts +++ b/src/routes/routes.test.ts @@ -4,14 +4,14 @@ import { createInMemoryGrantStore } from "@intx/authz"; import type { GrantRule } from "@intx/authz"; import { createRequireGrant, type TenantEnv } from "@intx/hub-api"; -import type { KnowledgePlane, TimelineEvent } from "../knowledge.ts"; -import { mountKnowledgeRoutes } from "./mount.ts"; +import type { Memory, TimelineEvent } from "../memory.ts"; +import { mountMemoryRoutes } from "./mount.ts"; import type { RouteDeps } from "./deps.ts"; function grant(principalId: string, action: string): GrantRule { return { id: `g-${principalId}-${action}`, - resource: "knowledge", + resource: "memory", action, effect: "allow", origin: "role", @@ -36,7 +36,7 @@ function stubPlane(opts?: { timelineCatalog?: Array< TimelineEvent & { visibleTo: readonly string[] | "tenant" } >; - askImpl?: KnowledgePlane["ask"]; + askImpl?: Memory["ask"]; }) { const added: { title: string; tenantId: string; principalId: string }[] = []; const searched: Array<{ @@ -45,7 +45,7 @@ function stubPlane(opts?: { limit: number | undefined; }> = []; const catalog = opts?.timelineCatalog ?? []; - const plane: KnowledgePlane = { + const plane: Memory = { find: async (p) => { searched.push({ kinds: p.kinds, entityIds: p.entityIds, limit: p.limit }); return { items: [], evidence: "none" }; @@ -84,7 +84,7 @@ function buildApp( TimelineEvent & { visibleTo: readonly string[] | "tenant" } >; principalId?: string; - askImpl?: KnowledgePlane["ask"]; + askImpl?: Memory["ask"]; }, ) { const { plane, added, searched } = stubPlane(opts); @@ -93,7 +93,7 @@ function buildApp( conditionRegistry: {}, }; const deps: RouteDeps = { - knowledge: plane, + memory: plane, grants: grantConfig, requireGrant: createRequireGrant(grantConfig), }; @@ -124,7 +124,7 @@ function buildApp( }); await next(); }); - mountKnowledgeRoutes(app, deps); + mountMemoryRoutes(app, deps); return { app, added, searched }; } @@ -137,12 +137,12 @@ function buildAppWithoutPrincipal() { conditionRegistry: {}, }; const deps: RouteDeps = { - knowledge: plane, + memory: plane, grants: grantConfig, requireGrant: createRequireGrant(grantConfig), }; const app = new Hono(); - mountKnowledgeRoutes(app, deps); + mountMemoryRoutes(app, deps); return app; } @@ -181,7 +181,7 @@ describe("knowledge HTTP routes", () => { grant(PRINCIPAL, "find"), ]); const res = await app.request( - "/api/knowledge/add", + "/api/memory/add", jsonPost({ title: "t", text: "body" }), ); expect(res.status).toBe(200); @@ -195,7 +195,7 @@ describe("knowledge HTTP routes", () => { test("add without the add grant is 403", async () => { const { app, added } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/add", + "/api/memory/add", jsonPost({ title: "t", text: "body" }), ); expect(res.status).toBe(403); @@ -205,7 +205,7 @@ describe("knowledge HTTP routes", () => { test("legacy capture grant does not authorize add", async () => { const { app, added } = buildApp([grant(PRINCIPAL, "capture")]); const res = await app.request( - "/api/knowledge/add", + "/api/memory/add", jsonPost({ title: "t", text: "body" }), ); expect(res.status).toBe(403); @@ -215,7 +215,7 @@ describe("knowledge HTTP routes", () => { test("add validates the body (400 on missing text)", async () => { const { app } = buildApp([grant(PRINCIPAL, "add")]); const res = await app.request( - "/api/knowledge/add", + "/api/memory/add", jsonPost({ title: "t" }), ); expect(res.status).toBe(400); @@ -224,7 +224,7 @@ describe("knowledge HTTP routes", () => { test("find with the find grant returns a result", async () => { const { app } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hello" }), ); expect(res.status).toBe(200); @@ -239,7 +239,7 @@ describe("knowledge HTTP routes", () => { test("find requires the find grant", async () => { const { app } = buildApp([grant(PRINCIPAL, "add")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hi" }), ); expect(res.status).toBe(403); @@ -248,7 +248,7 @@ describe("knowledge HTTP routes", () => { test("legacy search grant does not authorize find", async () => { const { app } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hi" }), ); expect(res.status).toBe(403); @@ -257,7 +257,7 @@ describe("knowledge HTTP routes", () => { test("find rejects out-of-range limit (400)", async () => { const { app } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hi", limit: 999 }), ); expect(res.status).toBe(400); @@ -266,7 +266,7 @@ describe("knowledge HTTP routes", () => { test("find threads kinds and entity_ids through to the plane", async () => { const { app, searched } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hello", kinds: ["artifact", "task"], @@ -286,7 +286,7 @@ describe("knowledge HTTP routes", () => { test("find with no kinds/entity_ids leaves them unset on the plane call", async () => { const { app, searched } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hello" }), ); expect(res.status).toBe(200); @@ -298,7 +298,7 @@ describe("knowledge HTTP routes", () => { test("find rejects a non-string-array kinds (400)", async () => { const { app } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hi", kinds: [1, 2] }), ); expect(res.status).toBe(400); @@ -309,7 +309,7 @@ describe("knowledge HTTP routes", () => { test("find passes an empty kinds/entity_ids array through unchanged", async () => { const { app, searched } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hello", kinds: [], entity_ids: [] }), ); expect(res.status).toBe(200); @@ -321,7 +321,7 @@ describe("knowledge HTTP routes", () => { test("ask requires the find grant (same as find)", async () => { const { app } = buildApp([grant(PRINCIPAL, "add")]); const res = await app.request( - "/api/knowledge/ask", + "/api/memory/ask", jsonPost({ query: "what?" }), ); expect(res.status).toBe(403); @@ -330,7 +330,7 @@ describe("knowledge HTTP routes", () => { test("ask with the find grant returns a grounded answer", async () => { const { app } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/knowledge/ask", + "/api/memory/ask", jsonPost({ query: "what?" }), ); expect(res.status).toBe(200); @@ -346,7 +346,7 @@ describe("knowledge HTTP routes", () => { test("recent requires the find grant", async () => { const { app } = buildApp([grant(PRINCIPAL, "add")]); - const res = await app.request("/api/knowledge/recent"); + const res = await app.request("/api/memory/recent"); expect(res.status).toBe(403); }); @@ -355,7 +355,7 @@ describe("knowledge HTTP routes", () => { timelineCatalog: RECENT_CATALOG, principalId: PRINCIPAL, }); - const res = await app.request("/api/knowledge/recent"); + const res = await app.request("/api/memory/recent"); expect(res.status).toBe(200); const body = (await res.json()) as { events: TimelineEvent[] }; expect(body.events.map((e) => e.title)).toEqual([PUBLIC_TITLE]); @@ -367,7 +367,7 @@ describe("knowledge HTTP routes", () => { timelineCatalog: RECENT_CATALOG, principalId: "alice", }); - const res = await app.request("/api/knowledge/recent"); + const res = await app.request("/api/memory/recent"); expect(res.status).toBe(200); const body = (await res.json()) as { events: TimelineEvent[] }; expect(body.events.map((e) => e.title).sort()).toEqual( @@ -385,7 +385,7 @@ describe("knowledge HTTP routes", () => { principalId: OTHER, }); const titles = async (app: Hono) => { - const res = await app.request("/api/knowledge/recent"); + const res = await app.request("/api/memory/recent"); expect(res.status).toBe(200); const body = (await res.json()) as { events: TimelineEvent[] }; return body.events.map((e) => e.title); @@ -404,9 +404,9 @@ describe("knowledge HTTP routes", () => { grant(PRINCIPAL, "search"), ]); for (const path of [ - "/api/knowledge/capture", - "/api/knowledge/search", - "/api/knowledge/timeline", + "/api/memory/capture", + "/api/memory/search", + "/api/memory/timeline", ]) { const method = path.endsWith("timeline") ? "GET" : "POST"; const res = await app.request( @@ -426,7 +426,7 @@ describe("knowledge HTTP routes", () => { test("add is rejected (401) when no principal is on the context", async () => { const app = buildAppWithoutPrincipal(); const res = await app.request( - "/api/knowledge/add", + "/api/memory/add", jsonPost({ title: "t", text: "body" }), ); expect(res.status).toBe(401); @@ -435,7 +435,7 @@ describe("knowledge HTTP routes", () => { test("find is rejected (401) when no principal is on the context", async () => { const app = buildAppWithoutPrincipal(); const res = await app.request( - "/api/knowledge/find", + "/api/memory/find", jsonPost({ query: "hello" }), ); expect(res.status).toBe(401); @@ -444,7 +444,7 @@ describe("knowledge HTTP routes", () => { test("ask is rejected (401) when no principal is on the context", async () => { const app = buildAppWithoutPrincipal(); const res = await app.request( - "/api/knowledge/ask", + "/api/memory/ask", jsonPost({ query: "hello" }), ); expect(res.status).toBe(401); @@ -452,7 +452,7 @@ describe("knowledge HTTP routes", () => { test("recent is rejected (401) when no principal is on the context", async () => { const app = buildAppWithoutPrincipal(); - const res = await app.request("/api/knowledge/recent"); + const res = await app.request("/api/memory/recent"); expect(res.status).toBe(401); }); }); diff --git a/src/services/search.ts b/src/services/search.ts index 1df27a9..737faac 100644 --- a/src/services/search.ts +++ b/src/services/search.ts @@ -98,11 +98,11 @@ const AUTHORITY_STRONG_FLOOR = 0.3; // equally valid "strong" signal alongside the lexical one. const RERANK_STRONG_FLOOR = 0.6; -export class KnowledgeSearchInputError extends Error { +export class MemorySearchInputError extends Error { readonly status = 400 as const; constructor(message: string) { super(message); - this.name = "KnowledgeSearchInputError"; + this.name = "MemorySearchInputError"; } } @@ -356,7 +356,7 @@ export async function fetchLexicalCandidates( if (query !== "") { // Bound as a parameter and cast to regconfig — never spliced — and // required to match the language the generated column was built with - // (verified against the catalog by runKnowledgeMigrations). + // (verified against the catalog by runMemoryMigrations). rankExpr = sql`ts_rank("knowledge"."chunk"."text_fts", plainto_tsquery(${ftsLanguage}::regconfig, ${query}))`; conditions.push( sql`"knowledge"."chunk"."text_fts" @@ plainto_tsquery(${ftsLanguage}::regconfig, ${query})`, @@ -767,7 +767,7 @@ export async function hybridSearch( const hasStructuredFilter = (kinds && kinds.length > 0) || (entityIds && entityIds.length > 0); if (query === "" && !hasStructuredFilter) { - throw new KnowledgeSearchInputError( + throw new MemorySearchInputError( "query must be non-empty unless kinds or entityIds is provided", ); } @@ -881,7 +881,7 @@ export async function hybridSearch( try { // A replay's transform_config can supply its own rerank endpoint/model // (resolvedTuning?.rerank, above) that never passes through - // mountKnowledgeEngine's startup validation — validate it here, on the + // mountMemory's startup validation — validate it here, on the // same terms as the mounted config, so a replay-authored mismatch // degrades to fused ranking (caught below) instead of silently // 413-ing every rerank call for that generation. diff --git a/src/services/timeline.test.ts b/src/services/timeline.test.ts index d7b22ec..8be2e80 100644 --- a/src/services/timeline.test.ts +++ b/src/services/timeline.test.ts @@ -18,7 +18,7 @@ function row(overrides: Partial = {}): TimelineRow { externalRef: "note:1", occurredAt: new Date("2026-01-01T00:00:00Z"), createdByPrincipalId: "owner", - accessTags: ["knowledge.owner:owner"], + accessTags: ["memory.owner:owner"], ...overrides, }; } @@ -36,7 +36,7 @@ describe("filterTimelineRows (grant-tag access)", () => { it("denies a peer without a matching grant", async () => { const { events, withheld } = await filterTimelineRows( - [row({ createdByPrincipalId: "owner", accessTags: ["knowledge.tenant:t1"] })], + [row({ createdByPrincipalId: "owner", accessTags: ["memory.tenant:t1"] })], { principalId: "peer", tenantId: "t1", @@ -52,7 +52,7 @@ describe("filterTimelineRows (grant-tag access)", () => { { id: "g1", principalId: "peer", - resource: "knowledge.tenant:t1", + resource: "memory.tenant:t1", action: "find", effect: "allow", origin: "role", @@ -65,7 +65,7 @@ describe("filterTimelineRows (grant-tag access)", () => { [ row({ createdByPrincipalId: "owner", - accessTags: ["knowledge.tenant:t1"], + accessTags: ["memory.tenant:t1"], }), ], { principalId: "peer", tenantId: "t1", grants }, From cdec20c0bee82565c4001624ae39b069281c35f3 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 15:55:29 -0700 Subject: [PATCH 05/11] Extract mem0/supermemory adapters to sibling packages Remove packages/knowledge-adapter-mem0 and packages/knowledge-adapter-supermemory from this monorepo. Point PRODUCT/README/AGENTS/IMPLEMENTATION at sibling @corbits/mem0 and @corbits/supermemory (granola/linear layout). Staging dirs ignored (.staging-*/). Local extracts live at abklabs/corbits-mem0 and abklabs/corbits-supermemory. --- .gitignore | 3 + AGENTS.md | 3 +- IMPLEMENTATION.md | 6 +- PRODUCT.md | 11 +- README.md | 13 +- packages/knowledge-adapter-mem0/README.md | 89 ---- packages/knowledge-adapter-mem0/package.json | 48 -- .../src/create-mem0-document-store.test.ts | 230 --------- .../src/create-mem0-document-store.ts | 268 ---------- .../src/create-mem0-memory-provider.test.ts | 215 -------- .../src/create-mem0-memory-provider.ts | 138 ------ packages/knowledge-adapter-mem0/src/index.ts | 16 - .../src/map-user.test.ts | 37 -- .../knowledge-adapter-mem0/src/map-user.ts | 21 - packages/knowledge-adapter-mem0/src/types.ts | 110 ----- packages/knowledge-adapter-mem0/tsconfig.json | 20 - .../knowledge-adapter-supermemory/README.md | 79 --- .../package.json | 47 -- .../src/index.test.ts | 236 --------- .../src/index.ts | 458 ------------------ .../tsconfig.json | 20 - 21 files changed, 25 insertions(+), 2043 deletions(-) delete mode 100644 packages/knowledge-adapter-mem0/README.md delete mode 100644 packages/knowledge-adapter-mem0/package.json delete mode 100644 packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts delete mode 100644 packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts delete mode 100644 packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.test.ts delete mode 100644 packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.ts delete mode 100644 packages/knowledge-adapter-mem0/src/index.ts delete mode 100644 packages/knowledge-adapter-mem0/src/map-user.test.ts delete mode 100644 packages/knowledge-adapter-mem0/src/map-user.ts delete mode 100644 packages/knowledge-adapter-mem0/src/types.ts delete mode 100644 packages/knowledge-adapter-mem0/tsconfig.json delete mode 100644 packages/knowledge-adapter-supermemory/README.md delete mode 100644 packages/knowledge-adapter-supermemory/package.json delete mode 100644 packages/knowledge-adapter-supermemory/src/index.test.ts delete mode 100644 packages/knowledge-adapter-supermemory/src/index.ts delete mode 100644 packages/knowledge-adapter-supermemory/tsconfig.json diff --git a/.gitignore b/.gitignore index 98a4ba4..d7f81b8 100644 --- a/.gitignore +++ b/.gitignore @@ -15,3 +15,6 @@ tmp/ # Dispatch orchestration scratch (local only) dispatch/ +# Staging dirs for sibling package extracts (copy out with cp only) +.staging-*/ + diff --git a/AGENTS.md b/AGENTS.md index e799b16..39716d5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,8 @@ CI runs `typecheck` + `test` — both must pass before any push. - `src/ports/` — `DocumentStore` / `SourceProvider` / `MemoryProvider` + fakes - `src/core/` — embed/rerank clients, merge, arktype schemas - `src/db/` + `migrations/` — Drizzle schema + SQL migrations (pgvector, `knowledge.*`) -- `packages/` — optional DocumentStore adapters (pure fetch; no vendor SDKs in core). Linear tools live in sibling `@corbits/linear`. +- `packages/` — removed; DocumentStore adapters are sibling packages + (`@corbits/mem0`, `@corbits/supermemory`). Linear tools: `@corbits/linear`. ## Non-negotiable invariants diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 2201de9..0d81ff3 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -30,8 +30,10 @@ src/ timeline.ts # listTimelineEvents — durable recent docs + grant-tag filter transform.ts # transform_config CRUD + runTransform (replay) core/ # framework-agnostic (chunking, embed/rerank, merge, schemas) -packages/ - # optional DocumentStore adapters (not part of public product surface) +# DocumentStore adapters live as sibling packages (not in this tree): +# @corbits/mem0 → github.com/corbitsdev/corbits-mem0 +# @corbits/supermemory → github.com/corbitsdev/corbits-supermemory +# Linear SourceProvider: sibling @corbits/linear migrations/ # pgvector schema, applied in filename order by scripts/db-setup.ts scripts/db-setup.ts # idempotent migration runner, tracked in `_migrations` compose.yml # pgvector + Ollama + reranker for local dev diff --git a/PRODUCT.md b/PRODUCT.md index ead7840..dd8ce7b 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -60,9 +60,14 @@ never auto-writes. ### Optional adapter packages -Optional `DocumentStore` implementations live under `packages/` in this tree -(or as separate packages). Core never imports vendor SDKs. Hosts that need a -store beyond default pgvector mount their own `documentStore`. +Optional `DocumentStore` implementations live as **sibling packages** (not in +this tree), same pattern as `@corbits/linear` / `@corbits/granola`: + +- [`@corbits/mem0`](https://github.com/corbitsdev/corbits-mem0) — Mem0 backend +- [`@corbits/supermemory`](https://github.com/corbitsdev/corbits-supermemory) — Supermemory backend + +Core never imports vendor SDKs. Hosts that need a store beyond default pgvector +mount their own `documentStore`. **Third-party store honesty:** not every `DocumentStore` evaluates host grant tags. Some isolate by **principal bucket** only (one private namespace per diff --git a/README.md b/README.md index b8267ea..cb20878 100644 --- a/README.md +++ b/README.md @@ -94,9 +94,12 @@ host-owned writes (ask never auto-remembers). ### Optional DocumentStore adapters -Optional adapter packages under `packages/` implement `DocumentStore` for hosts -that want a non-pgvector backend. Core never imports vendor SDKs. See each -package's README for mount examples and isolation limits. +Optional adapters are **sibling packages** (not vendored in this repo). Core +never imports vendor SDKs. See each package README for mount examples and +isolation limits: + +- [`@corbits/mem0`](https://github.com/corbitsdev/corbits-mem0) +- [`@corbits/supermemory`](https://github.com/corbitsdev/corbits-supermemory) ```ts // Linear SourceProvider lives in sibling repo @corbits/linear @@ -141,8 +144,8 @@ Full design: `docs/AUTHZ-DOCUMENT-ACCESS.md`. bun install bun run typecheck bun run test -# optional DocumentStore adapters under packages/ (each package has its own tests) -# Linear tools: sibling repo @corbits/linear +# DocumentStore adapters: sibling @corbits/mem0, @corbits/supermemory +# Linear tools: sibling @corbits/linear ``` License: LGPL-2.1 (`LICENSE`). Contributions: `CLA.md`. diff --git a/packages/knowledge-adapter-mem0/README.md b/packages/knowledge-adapter-mem0/README.md deleted file mode 100644 index 3c4202f..0000000 --- a/packages/knowledge-adapter-mem0/README.md +++ /dev/null @@ -1,89 +0,0 @@ -# @corbits/knowledge-adapter-mem0 - -**Replaceable DocumentStore** for [@corbits/memory](https://github.com/corbitsdev/corbits-memory) -backed by the [Mem0 Platform](https://docs.mem0.ai/) HTTP API. - -Pure `fetch` — **no** `mem0ai` / vendor SDK. Tenancy is enforced with a -length-prefixed `user_id` (`mapUser`) so free-form ids cannot collide. - -## Product path: DocumentStore - -Mount Mem0 as `documentStore` so the plane routes `add` / `find` / `recent` -(and `ask` via find) through this store — no Postgres / embed endpoints -required. This is the product integration path (not `MemoryProvider` / -`includeMemory`). - -```ts -import { createMemory } from "@corbits/memory"; -import { createMem0DocumentStore } from "@corbits/knowledge-adapter-mem0"; - -const memory = createMemory(undefined, grants, { - documentStore: createMem0DocumentStore({ - apiKey: process.env.MEM0_API_KEY!, - // baseUrl: "https://api.mem0.ai", // optional - }), - generate: myGenerate, // required only for ask() -}); - -await memory.add({ - tenantId, - principalId, - content: { title: "Prefs", text: "Prefers dark mode" }, -}); - -const { items } = await memory.find({ - tenantId, - principalId, - query: "preferences", -}); -``` - -Or via mount: - -```ts -mountMemory(app, { - documentStore: createMem0DocumentStore({ apiKey }), - grants, - generate, -}); -``` - -## Limitations (honest) - -| Area | Behavior | -| --- | --- | -| Isolation | **Principal-bucket only** via `mapUser(tenantId, principalId)`. Each principal has a private Mem0 `user_id`; docs are not shared across principals. | -| Document access | This adapter is **principal-bucket only**. Host grant tags (`accessTags`) are stored as metadata at best and are **not** evaluated. For multi-principal grant-tag ACL, use the default pgvector store. | -| `recent` | Always `[]` — Mem0 has no recent-feed API here. | -| `options.memoryProvider` | **Never** mount this package as `options.memoryProvider`. That port is an ask side-channel; Mem0 as product backend is `documentStore` only. | - -## What is out of scope - -- **Not** a tools-shaped source (Linear-style live connectors stay separate). -- **Not** the product path for `MemoryProvider` / `includeMemory`. -- `createMem0MemoryProvider` remains exported for back-compat only; prefer - `createMem0DocumentStore`. - -## Tenant mapping - -```ts -import { mapUser } from "@corbits/knowledge-adapter-mem0"; - -mapUser("acme", "alice"); // "4:acme:5:alice" -``` - -Never pass bare `principalId` as Mem0 `user_id`. - -## HTTP surface (thin) - -| Verb | Method | Path | -| ------ | ------ | ------------------------ | -| add | POST | `/v3/memories/add/` | -| find | POST | `/v3/memories/search/` | -| recent | — | empty (API has no feed) | - -Auth: `Authorization: Token `. - -## License - -LGPL-2.1-only (same as Corbits Memory). diff --git a/packages/knowledge-adapter-mem0/package.json b/packages/knowledge-adapter-mem0/package.json deleted file mode 100644 index 2914788..0000000 --- a/packages/knowledge-adapter-mem0/package.json +++ /dev/null @@ -1,48 +0,0 @@ -{ - "name": "@corbits/knowledge-adapter-mem0", - "version": "0.1.0", - "description": "Mem0 DocumentStore adapter for @corbits/memory (pure fetch, no SDK)", - - "type": "module", - "module": "src/index.ts", - "exports": { - ".": "./src/index.ts" - }, - "engines": { - "bun": ">=1.2.0" - }, - "scripts": { - "typecheck": "tsc --noEmit", - "test": "bun test" - }, - "devDependencies": { - "@types/bun": "latest", - "typescript": "^5.9.0" - }, - "license": "LGPL-2.1-only", - "author": "Sawyer Cutler ", - "repository": { - "type": "git", - "url": "git+https://github.com/corbitsdev/corbits-memory.git", - "directory": "packages/knowledge-adapter-mem0" - }, - "homepage": "https://github.com/corbitsdev/corbits-memory#readme", - "bugs": { - "url": "https://github.com/corbitsdev/corbits-memory/issues" - }, - "keywords": [ - "mem0", - "memory", - "knowledge", - "corbits" - ], - "files": [ - "src", - "!src/**/*.test.ts", - "README.md", - "LICENSE" - ], - "publishConfig": { - "access": "public" - } -} diff --git a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts b/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts deleted file mode 100644 index 8017d84..0000000 --- a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.test.ts +++ /dev/null @@ -1,230 +0,0 @@ -import { describe, expect, it } from "bun:test"; - -import { - createMem0DocumentStore, - parseFindResults, -} from "./create-mem0-document-store.ts"; - -type Captured = { - url: string; - method: string; - headers: Record; - body: unknown; -}; - -function mockFetch( - handler: (req: Captured) => { status?: number; json?: unknown }, -): { fetch: typeof fetch; calls: Captured[] } { - const calls: Captured[] = []; - const fetchImpl = (async ( - input: string | URL | Request, - init?: RequestInit, - ): Promise => { - const url = - typeof input === "string" - ? input - : input instanceof URL - ? input.href - : input.url; - const headers: Record = {}; - if (init?.headers) { - const h = new Headers(init.headers); - h.forEach((v, k) => { - headers[k] = v; - }); - } - let body: unknown; - if (typeof init?.body === "string") { - body = JSON.parse(init.body); - } - const cap: Captured = { - url, - method: init?.method ?? "GET", - headers, - body, - }; - calls.push(cap); - const result = handler(cap); - const status = result.status ?? 200; - const payload = - result.json === undefined ? "" : JSON.stringify(result.json); - return new Response(payload, { - status, - headers: { "Content-Type": "application/json" }, - }); - }) as typeof fetch; - return { fetch: fetchImpl, calls }; -} - -describe("createMem0DocumentStore", () => { - it("rejects missing apiKey", () => { - expect(() => createMem0DocumentStore({ apiKey: "" })).toThrow(/apiKey/); - }); - - it("add posts mapped user_id and returns documentId", async () => { - const { fetch, calls } = mockFetch(() => ({ - status: 200, - json: { event_id: "e1", status: "PENDING" }, - })); - const store = createMem0DocumentStore({ apiKey: "test-key", fetch }); - - const { documentId } = await store.add({ - tenantId: "t1", - principalId: "p1", - title: "Prefs", - text: "Prefers dark mode", - accessTags: ["memory.owner:p1"], - }); - - expect(documentId).toMatch( - /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i, - ); - expect(calls).toHaveLength(1); - const call = calls[0]!; - expect(call.method).toBe("POST"); - expect(call.url).toBe("https://api.mem0.ai/v3/memories/add/"); - const body = call.body as Record; - expect(body.user_id).toBe("2:t1:2:p1"); - expect(body.user_id).not.toBe("p1"); - expect(body.infer).toBe(false); - const messages = body.messages as Array<{ content: string }>; - expect(messages[0]!.content).toContain("# Prefs"); - expect(messages[0]!.content).toContain("Prefers dark mode"); - expect(messages[0]!.content).toContain(documentId); - const meta = body.metadata as Record; - expect(meta.documentId).toBe(documentId); - expect(meta.title).toBe("Prefs"); - }); - - it("find scopes search by mapped user_id and maps hits", async () => { - const { fetch, calls } = mockFetch(() => ({ - status: 200, - json: { - results: [ - { - id: "m1", - memory: "# Home\n\nLives in SF", - score: 0.91, - metadata: { documentId: "doc-1", externalRef: "ref-1" }, - }, - ], - }, - })); - const store = createMem0DocumentStore({ - apiKey: "k", - baseUrl: "https://mem0.example.com/", - fetch, - }); - - const result = await store.find({ - tenantId: "acme", - principalId: "bob", - query: "where do I live?", - limit: 3, - includeEvidence: true, - }); - - expect(calls).toHaveLength(1); - const call = calls[0]!; - expect(call.url).toBe("https://mem0.example.com/v3/memories/search/"); - const body = call.body as Record; - expect(body.filters).toEqual({ user_id: "4:acme:3:bob" }); - expect(body.top_k).toBe(3); - - expect(result.evidence).toBe("weak"); - expect(result.items).toHaveLength(1); - expect(result.items[0]!.documentId).toBe("doc-1"); - expect(result.items[0]!.title).toBe("Home"); - expect(result.items[0]!.snippet).toContain("Lives in SF"); - expect(result.items[0]!.citation.adapter).toBe("mem0"); - expect(result.items[0]!.citation.external_ref).toBe("ref-1"); - }); - - it("add/find reject empty identity", async () => { - const { fetch, calls } = mockFetch(() => ({ status: 200, json: {} })); - const store = createMem0DocumentStore({ apiKey: "k", fetch }); - - await expect( - store.add({ - tenantId: "", - principalId: "p", - title: "t", - text: "x", - accessTags: ["memory.owner:p"], - }), - ).rejects.toThrow(/tenantId/); - - await expect( - store.find({ - tenantId: "t", - principalId: "", - query: "q", - }), - ).rejects.toThrow(/principalId/); - - expect(calls).toHaveLength(0); - }); - - it("tenant isolation: same principal different tenants → distinct user_id", async () => { - const { fetch, calls } = mockFetch(() => ({ - status: 200, - json: { results: [] }, - })); - const store = createMem0DocumentStore({ apiKey: "k", fetch }); - - await store.find({ - tenantId: "tenant-a", - principalId: "alice", - query: "prefs", - }); - await store.find({ - tenantId: "tenant-b", - principalId: "alice", - query: "prefs", - }); - - const userIds = calls.map( - (c) => (c.body as { filters: { user_id: string } }).filters.user_id, - ); - expect(userIds).toEqual(["8:tenant-a:5:alice", "8:tenant-b:5:alice"]); - expect(userIds[0]).not.toBe(userIds[1]); - }); - - it("recent returns empty list; close is a no-op", async () => { - const store = createMem0DocumentStore({ - apiKey: "k", - fetch: (async () => new Response("{}")) as unknown as typeof fetch, - }); - - expect( - await store.recent({ - tenantId: "t", - principalId: "p", - }), - ).toEqual([]); - await store.close(); - }); -}); - -describe("parseFindResults", () => { - it("reads results[].memory with metadata documentId", () => { - const items = parseFindResults({ - results: [ - { - memory: "# Note\n\nbody", - score: 0.5, - metadata: { documentId: "d1" }, - }, - ], - }); - expect(items).toHaveLength(1); - expect(items[0]!.documentId).toBe("d1"); - expect(items[0]!.title).toBe("Note"); - }); - - it("handles empty / null", () => { - expect(parseFindResults(null)).toEqual([]); - expect(parseFindResults(undefined)).toEqual([]); - expect(parseFindResults({})).toEqual([]); - }); -}); diff --git a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts b/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts deleted file mode 100644 index 76a0e1d..0000000 --- a/packages/knowledge-adapter-mem0/src/create-mem0-document-store.ts +++ /dev/null @@ -1,268 +0,0 @@ -import { mapUser } from "./map-user.ts"; -import type { - DocumentStore, - DocumentStoreFindItem, - Mem0ClientOptions, -} from "./types.ts"; - -const DEFAULT_BASE_URL = "https://api.mem0.ai"; -const ADAPTER = "mem0"; - -function normalizeBaseUrl(baseUrl: string): string { - return baseUrl.replace(/\/+$/, ""); -} - -async function readErrorBody(res: Response): Promise { - try { - const text = await res.text(); - return text.length > 500 ? `${text.slice(0, 500)}…` : text; - } catch { - return ""; - } -} - -function requireIdentity(tenantId: string, principalId: string): void { - if (typeof tenantId !== "string" || tenantId.trim() === "") { - throw new Error("Mem0 DocumentStore requires non-empty tenantId"); - } - if (typeof principalId !== "string" || principalId.trim() === "") { - throw new Error("Mem0 DocumentStore requires non-empty principalId"); - } -} - -function encodeDocumentBody(params: { - title: string; - text: string; - documentId: string; - externalRef?: string; - accessTags: string[]; -}): string { - const header = `# ${params.title}`; - const meta = [ - `documentId: ${params.documentId}`, - params.externalRef ? `externalRef: ${params.externalRef}` : null, - params.accessTags.length > 0 - ? `accessTags: ${params.accessTags.join(",")}` - : null, - ] - .filter(Boolean) - .join("\n"); - return `${header}\n\n${params.text}\n\n---\n${meta}`; -} - -function parseTitleAndSnippet(text: string): { title: string; snippet: string } { - const lines = text.split("\n"); - if (lines[0]?.startsWith("# ")) { - const title = lines[0].slice(2).trim() || "untitled"; - const rest = lines - .slice(1) - .join("\n") - .replace(/\n---\n[\s\S]*$/, "") - .trim(); - return { - title, - snippet: rest.slice(0, 240) || title, - }; - } - return { - title: text.slice(0, 80) || "untitled", - snippet: text.slice(0, 240), - }; -} - -/** Normalize Mem0 search JSON into DocumentStore find items. */ -export function parseFindResults(raw: unknown): DocumentStoreFindItem[] { - if (raw == null) return []; - - let items: unknown[] = []; - if (Array.isArray(raw)) { - items = raw; - } else if (typeof raw === "object" && raw !== null) { - const obj = raw as Record; - if (Array.isArray(obj.results)) { - items = obj.results; - } else if (Array.isArray(obj.memories)) { - items = obj.memories; - } - } - - const out: DocumentStoreFindItem[] = []; - for (const item of items) { - if (item == null || typeof item !== "object") continue; - const row = item as Record; - const text = - typeof row.memory === "string" - ? row.memory - : typeof row.text === "string" - ? row.text - : null; - if (text == null) continue; - - const idFromRow = - typeof row.id === "string" - ? row.id - : typeof row.memory_id === "string" - ? row.memory_id - : undefined; - const meta = - row.metadata && typeof row.metadata === "object" - ? (row.metadata as Record) - : {}; - const documentId = - (typeof meta.documentId === "string" && meta.documentId) || - idFromRow || - crypto.randomUUID(); - const externalRef = - (typeof meta.externalRef === "string" && meta.externalRef) || documentId; - const { title, snippet } = parseTitleAndSnippet(text); - const score = - typeof row.score === "number" && Number.isFinite(row.score) - ? row.score - : 0.5; - - out.push({ - documentId, - title, - snippet, - score, - kind: "note", - adapter: ADAPTER, - externalRef, - citation: { - adapter: ADAPTER, - external_ref: externalRef, - open: { - type: "document", - id: documentId, - url: `mem0://${documentId}`, - }, - }, - }); - } - return out; -} - -/** - * Create a DocumentStore backed by the Mem0 Platform HTTP API (v3). - * - * Pure fetch — no mem0 SDK. Mount as the plane's durable backend: - * - * ```ts - * createMemory(undefined, grants, { - * documentStore: createMem0DocumentStore({ apiKey }), - * }) - * ``` - * - * No local Postgres required. Tenancy is enforced via length-prefixed `user_id` - * (`mapUser`); never pass bare principalId. - */ -export function createMem0DocumentStore( - opts: Mem0ClientOptions, -): DocumentStore { - if (typeof opts.apiKey !== "string" || opts.apiKey.trim() === "") { - throw new Error( - "createMem0DocumentStore: apiKey is required and must be a non-empty string", - ); - } - - const baseUrl = normalizeBaseUrl(opts.baseUrl ?? DEFAULT_BASE_URL); - const doFetch = opts.fetch ?? globalThis.fetch.bind(globalThis); - const authHeader = `Token ${opts.apiKey}`; - - async function mem0Post( - path: string, - body: Record, - ): Promise { - const url = `${baseUrl}${path}`; - const res = await doFetch(url, { - method: "POST", - headers: { - Authorization: authHeader, - "Content-Type": "application/json", - Accept: "application/json", - }, - body: JSON.stringify(body), - }); - - if (!res.ok) { - const detail = await readErrorBody(res); - throw new Error( - `Mem0 API ${path} failed: HTTP ${res.status}${detail ? ` — ${detail}` : ""}`, - ); - } - - if (res.status === 204) return undefined; - const text = await res.text(); - if (!text) return undefined; - try { - return JSON.parse(text) as unknown; - } catch { - return undefined; - } - } - - return { - async add(params) { - requireIdentity(params.tenantId, params.principalId); - const userId = mapUser(params.tenantId, params.principalId); - const documentId = crypto.randomUUID(); - const content = encodeDocumentBody({ - title: params.title, - text: params.text, - documentId, - ...(params.externalRef !== undefined - ? { externalRef: params.externalRef } - : {}), - accessTags: params.accessTags ?? [], - }); - const metadata: Record = { - documentId, - title: params.title, - }; - if (params.accessTags?.length) { - metadata.accessTags = params.accessTags.join(","); - } - if (params.externalRef !== undefined) { - metadata.externalRef = params.externalRef; - } - - // Platform v3 add path (same as legacy MemoryProvider). - await mem0Post("/v3/memories/add/", { - messages: [{ role: "user", content }], - user_id: userId, - infer: false, - metadata, - }); - - return { documentId }; - }, - - async find(params) { - requireIdentity(params.tenantId, params.principalId); - const userId = mapUser(params.tenantId, params.principalId); - const topK = params.limit ?? 8; - const raw = await mem0Post("/v3/memories/search/", { - query: params.query, - filters: { user_id: userId }, - top_k: topK, - }); - const items = parseFindResults(raw); - if (params.includeEvidence) { - return { - items, - evidence: items.length === 0 ? "none" : "weak", - }; - } - return { items }; - }, - - async recent() { - // Query-oriented API; no stable recent timeline in this thin adapter. - return []; - }, - - async close() { - // Stateless HTTP client. - }, - }; -} diff --git a/packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.test.ts b/packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.test.ts deleted file mode 100644 index b01aca7..0000000 --- a/packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.test.ts +++ /dev/null @@ -1,215 +0,0 @@ -import { describe, expect, it } from "bun:test"; - -import { - createMem0MemoryProvider, - parseSearchResults, -} from "./create-mem0-memory-provider.ts"; - -type Captured = { - url: string; - method: string; - headers: Record; - body: unknown; -}; - -function mockFetch( - handler: (req: Captured) => { status?: number; json?: unknown }, -): { fetch: typeof fetch; calls: Captured[] } { - const calls: Captured[] = []; - const fetchImpl = (async ( - input: string | URL | Request, - init?: RequestInit, - ): Promise => { - const url = - typeof input === "string" - ? input - : input instanceof URL - ? input.href - : input.url; - const headers: Record = {}; - if (init?.headers) { - const h = new Headers(init.headers); - h.forEach((v, k) => { - headers[k] = v; - }); - } - let body: unknown; - if (typeof init?.body === "string") { - body = JSON.parse(init.body); - } - const cap: Captured = { - url, - method: init?.method ?? "GET", - headers, - body, - }; - calls.push(cap); - const result = handler(cap); - const status = result.status ?? 200; - const payload = - result.json === undefined ? "" : JSON.stringify(result.json); - return new Response(payload, { - status, - headers: { "Content-Type": "application/json" }, - }); - }) as typeof fetch; - return { fetch: fetchImpl, calls }; -} - -describe("createMem0MemoryProvider", () => { - it("rejects missing apiKey", () => { - expect(() => createMem0MemoryProvider({ apiKey: "" })).toThrow(/apiKey/); - }); - - it("remember sends mapped user_id and never bare principal", async () => { - const { fetch, calls } = mockFetch(() => ({ - status: 200, - json: { event_id: "e1", status: "PENDING" }, - })); - const provider = createMem0MemoryProvider({ - apiKey: "test-key", - fetch, - }); - - await provider.remember({ - tenantId: "t1", - principalId: "p1", - text: "Prefers dark mode", - metadata: { source: "settings" }, - }); - - expect(calls).toHaveLength(1); - const call = calls[0]!; - expect(call.method).toBe("POST"); - expect(call.url).toBe("https://api.mem0.ai/v3/memories/add/"); - expect(call.headers["authorization"] ?? call.headers["Authorization"]).toBe( - "Token test-key", - ); - const body = call.body as Record; - expect(body.user_id).toBe("2:t1:2:p1"); - - expect(body.user_id).not.toBe("p1"); - expect(body.messages).toEqual([ - { role: "user", content: "Prefers dark mode" }, - ]); - expect(body.metadata).toEqual({ source: "settings" }); - expect(body.infer).toBe(false); - }); - - it("recall scopes search filters by mapped user_id", async () => { - const { fetch, calls } = mockFetch(() => ({ - status: 200, - json: { - results: [ - { id: "m1", memory: "Lives in SF", score: 0.91 }, - { id: "m2", memory: "Works remote", score: 0.7 }, - ], - }, - })); - const provider = createMem0MemoryProvider({ - apiKey: "k", - baseUrl: "https://mem0.example.com/", - fetch, - }); - - const hits = await provider.recall({ - tenantId: "acme", - principalId: "bob", - query: "where do I live?", - limit: 3, - }); - - expect(calls).toHaveLength(1); - const call = calls[0]!; - expect(call.url).toBe("https://mem0.example.com/v3/memories/search/"); - const body = call.body as Record; - expect(body.query).toBe("where do I live?"); - expect(body.top_k).toBe(3); - expect(body.filters).toEqual({ user_id: "4:acme:3:bob" }); - - expect(hits).toEqual([ - { text: "Lives in SF", score: 0.91 }, - { text: "Works remote", score: 0.7 }, - ]); - }); - - it("remember/recall reject empty identity (no silent default)", async () => { - const { fetch, calls } = mockFetch(() => ({ status: 200, json: {} })); - const provider = createMem0MemoryProvider({ apiKey: "k", fetch }); - - await expect( - provider.remember({ - tenantId: "", - principalId: "p", - text: "x", - }), - ).rejects.toThrow(/tenantId/); - - await expect( - provider.recall({ - tenantId: "t", - principalId: "", - query: "q", - }), - ).rejects.toThrow(/principalId/); - - expect(calls).toHaveLength(0); - }); - - it("throws on non-OK HTTP from Mem0", async () => { - const { fetch } = mockFetch(() => ({ - status: 401, - json: { detail: "Unauthorized" }, - })); - const provider = createMem0MemoryProvider({ apiKey: "bad", fetch }); - await expect( - provider.remember({ - tenantId: "t", - principalId: "p", - text: "x", - }), - ).rejects.toThrow(/HTTP 401/); - }); - - it("tenant isolation: same principal different tenants → distinct user_id", async () => { - const { fetch, calls } = mockFetch(() => ({ - status: 200, - json: { results: [] }, - })); - const provider = createMem0MemoryProvider({ apiKey: "k", fetch }); - - await provider.recall({ - tenantId: "tenant-a", - principalId: "alice", - query: "prefs", - }); - await provider.recall({ - tenantId: "tenant-b", - principalId: "alice", - query: "prefs", - }); - - const userIds = calls.map( - (c) => (c.body as { filters: { user_id: string } }).filters.user_id, - ); - expect(userIds).toEqual(["8:tenant-a:5:alice", "8:tenant-b:5:alice"]); - - expect(userIds[0]).not.toBe(userIds[1]); - }); -}); - -describe("parseSearchResults", () => { - it("reads results[].memory", () => { - expect( - parseSearchResults({ - results: [{ memory: "a", score: 0.5 }], - }), - ).toEqual([{ text: "a", score: 0.5 }]); - }); - - it("handles empty / null", () => { - expect(parseSearchResults(null)).toEqual([]); - expect(parseSearchResults(undefined)).toEqual([]); - expect(parseSearchResults({})).toEqual([]); - }); -}); diff --git a/packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.ts b/packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.ts deleted file mode 100644 index 739295c..0000000 --- a/packages/knowledge-adapter-mem0/src/create-mem0-memory-provider.ts +++ /dev/null @@ -1,138 +0,0 @@ -/** - * @deprecated Prefer createMem0DocumentStore and mount as documentStore. - * Thin remember/recall wrapper kept for back-compat only — not the product path. - */ -import { mapUser } from "./map-user.ts"; -import type { Mem0ClientOptions, MemoryProvider } from "./types.ts"; - -const DEFAULT_BASE_URL = "https://api.mem0.ai"; - -function normalizeBaseUrl(baseUrl: string): string { - return baseUrl.replace(/\/+$/, ""); -} - -async function readErrorBody(res: Response): Promise { - try { - const text = await res.text(); - return text.length > 500 ? `${text.slice(0, 500)}…` : text; - } catch { - return ""; - } -} - -/** - * @deprecated Use createMem0DocumentStore({ apiKey }) as options.documentStore. - */ -export function createMem0MemoryProvider( - opts: Mem0ClientOptions, -): MemoryProvider { - if (typeof opts.apiKey !== "string" || opts.apiKey.trim() === "") { - throw new Error( - "createMem0MemoryProvider: apiKey is required and must be a non-empty string", - ); - } - - const baseUrl = normalizeBaseUrl(opts.baseUrl ?? DEFAULT_BASE_URL); - const doFetch = opts.fetch ?? globalThis.fetch.bind(globalThis); - const authHeader = `Token ${opts.apiKey}`; - - async function mem0Post( - path: string, - body: Record, - ): Promise { - const url = `${baseUrl}${path}`; - const res = await doFetch(url, { - method: "POST", - headers: { - Authorization: authHeader, - "Content-Type": "application/json", - Accept: "application/json", - }, - body: JSON.stringify(body), - }); - - if (!res.ok) { - const detail = await readErrorBody(res); - throw new Error( - `Mem0 API ${path} failed: HTTP ${res.status}${detail ? ` — ${detail}` : ""}`, - ); - } - - if (res.status === 204) return undefined; - const text = await res.text(); - if (!text) return undefined; - try { - return JSON.parse(text) as unknown; - } catch { - return undefined; - } - } - - return { - async remember(params) { - const userId = mapUser(params.tenantId, params.principalId); - const body: Record = { - messages: [{ role: "user", content: params.text }], - user_id: userId, - infer: false, - }; - if (params.metadata !== undefined) { - body.metadata = params.metadata; - } - await mem0Post("/v3/memories/add/", body); - }, - - async recall(params) { - const userId = mapUser(params.tenantId, params.principalId); - const topK = params.limit ?? 5; - const raw = await mem0Post("/v3/memories/search/", { - query: params.query, - filters: { user_id: userId }, - top_k: topK, - }); - return parseSearchResults(raw); - }, - }; -} - -/** Normalize Mem0 search JSON into MemoryProvider recall hits. */ -export function parseSearchResults( - raw: unknown, -): Array<{ text: string; score?: number }> { - if (raw == null) return []; - - let items: unknown[] = []; - if (Array.isArray(raw)) { - items = raw; - } else if (typeof raw === "object" && raw !== null) { - const obj = raw as Record; - if (Array.isArray(obj.results)) { - items = obj.results; - } else if (Array.isArray(obj.memories)) { - items = obj.memories; - } - } - - const out: Array<{ text: string; score?: number }> = []; - for (const item of items) { - if (item == null || typeof item !== "object") continue; - const row = item as Record; - const text = - typeof row.memory === "string" - ? row.memory - : typeof row.text === "string" - ? row.text - : null; - if (text == null) continue; - const score = - typeof row.score === "number" && Number.isFinite(row.score) - ? row.score - : undefined; - if (score === undefined) { - out.push({ text }); - } else { - out.push({ text, score }); - } - } - return out; -} diff --git a/packages/knowledge-adapter-mem0/src/index.ts b/packages/knowledge-adapter-mem0/src/index.ts deleted file mode 100644 index 643fddd..0000000 --- a/packages/knowledge-adapter-mem0/src/index.ts +++ /dev/null @@ -1,16 +0,0 @@ -export type { - DocumentStore, - DocumentStoreAddParams, - DocumentStoreFindItem, - DocumentStoreFindParams, - DocumentStoreFindResult, - DocumentStoreRecentEvent, - DocumentStoreRecentParams, - Mem0ClientOptions, - Mem0MemoryProviderOptions, - MemoryProvider, -} from "./types.ts"; -export { mapUser } from "./map-user.ts"; -export { createMem0DocumentStore } from "./create-mem0-document-store.ts"; -/** @deprecated Prefer createMem0DocumentStore as options.documentStore. */ -export { createMem0MemoryProvider } from "./create-mem0-memory-provider.ts"; diff --git a/packages/knowledge-adapter-mem0/src/map-user.test.ts b/packages/knowledge-adapter-mem0/src/map-user.test.ts deleted file mode 100644 index 6deff8b..0000000 --- a/packages/knowledge-adapter-mem0/src/map-user.test.ts +++ /dev/null @@ -1,37 +0,0 @@ -import { describe, expect, it } from "bun:test"; - -import { mapUser } from "./map-user.ts"; - -describe("mapUser", () => { - it("length-prefixes tenant and principal", () => { - expect(mapUser("tenant-a", "user-1")).toBe("8:tenant-a:6:user-1"); - }); - - it("isolates the same principal across tenants", () => { - const a = mapUser("tenant-a", "alice"); - const b = mapUser("tenant-b", "alice"); - expect(a).not.toBe(b); - }); - - it("rejects empty tenantId", () => { - expect(() => mapUser("", "alice")).toThrow(/tenantId/); - }); - - it("rejects empty principalId", () => { - expect(() => mapUser("t", "")).toThrow(/principalId/); - }); - - it("rejects whitespace-only ids", () => { - expect(() => mapUser(" ", "alice")).toThrow(/tenantId/); - expect(() => mapUser("t", " ")).toThrow(/principalId/); - }); - - it("is injective when ids contain delimiter sequences", () => { - // Old `tenant::principal` encoding collided on these pairs. - const a = mapUser("a::b", "c"); - const b = mapUser("a", "b::c"); - expect(a).not.toBe(b); - expect(a).toBe("4:a::b:1:c"); - expect(b).toBe("1:a:4:b::c"); - }); -}); diff --git a/packages/knowledge-adapter-mem0/src/map-user.ts b/packages/knowledge-adapter-mem0/src/map-user.ts deleted file mode 100644 index 683644a..0000000 --- a/packages/knowledge-adapter-mem0/src/map-user.ts +++ /dev/null @@ -1,21 +0,0 @@ -/** - * Map Corbits (tenantId, principalId) → Mem0 user_id. - * - * Length-prefixed encoding is injective for any free-form ids that do not - * contain only digits-before-colon collisions: distinct pairs never share a - * user_id even when ids contain `::` or `_`. - */ -export function mapUser(tenantId: string, principalId: string): string { - if (typeof tenantId !== "string" || tenantId.trim() === "") { - throw new Error( - "mapUser: tenantId is required and must be a non-empty string", - ); - } - if (typeof principalId !== "string" || principalId.trim() === "") { - throw new Error( - "mapUser: principalId is required and must be a non-empty string", - ); - } - // `${len}:${id}` twice — cannot collide across delimiter injection. - return `${tenantId.length}:${tenantId}:${principalId.length}:${principalId}`; -} diff --git a/packages/knowledge-adapter-mem0/src/types.ts b/packages/knowledge-adapter-mem0/src/types.ts deleted file mode 100644 index 1e2f44a..0000000 --- a/packages/knowledge-adapter-mem0/src/types.ts +++ /dev/null @@ -1,110 +0,0 @@ -/** - * Port shapes — defined locally so this adapter never imports runtime from - * @corbits/memory. DocumentStore is the product plug (replaceable - * durable backend). MemoryProvider is a thin legacy shape kept for back-compat. - */ - -/** Minimal citation open shape (matches @corbits/memory SearchHitCitation). */ -export type DocumentStoreCitation = { - adapter: string; - external_ref: string; - open: { - type: string; - id: string; - url?: string; - }; -}; - -export type DocumentStoreAddParams = { - tenantId: string; - principalId: string; - title: string; - text: string; - /** Grant-tag resource strings (ignored for enforcement; principal-bucket only). */ - accessTags: string[]; - attributes?: Record; - externalRef?: string; - adapter?: string; - kind?: string; -}; - -export type DocumentStoreFindParams = { - tenantId: string; - principalId: string; - query: string; - limit?: number; - includeEvidence?: boolean; -}; - -export type DocumentStoreFindItem = { - documentId: string; - title: string; - snippet: string; - score: number; - kind: string; - citation: DocumentStoreCitation; - adapter?: string; - externalRef?: string; - updatedAt?: string; -}; - -export type DocumentStoreFindResult = { - items: DocumentStoreFindItem[]; - evidence?: "strong" | "weak" | "none"; - degraded?: string[]; -}; - -export type DocumentStoreRecentParams = { - tenantId: string; - principalId: string; - limit?: number; -}; - -export type DocumentStoreRecentEvent = { - at: string; - title: string; - source: string; - tenantId: string; - principalId: string; -}; - -/** - * Replaceable durable backend for the knowledge plane (add / find / recent). - * Mount as `options.documentStore` — no local Postgres required. - */ -export type DocumentStore = { - add(params: DocumentStoreAddParams): Promise<{ documentId: string }>; - find(params: DocumentStoreFindParams): Promise; - recent( - params: DocumentStoreRecentParams, - ): Promise; - close(): Promise; -}; - -/** @deprecated Prefer DocumentStore. Thin remember/recall only. */ -export type MemoryProvider = { - remember(params: { - tenantId: string; - principalId: string; - text: string; - metadata?: Record; - }): Promise; - recall(params: { - tenantId: string; - principalId: string; - query: string; - limit?: number; - }): Promise>; -}; - -export type Mem0ClientOptions = { - /** Mem0 platform API key (sent as `Authorization: Token …`). */ - apiKey: string; - /** API origin; default `https://api.mem0.ai`. */ - baseUrl?: string; - /** Injectable fetch for tests; defaults to global fetch. */ - fetch?: typeof fetch; -}; - -/** @deprecated Use Mem0ClientOptions */ -export type Mem0MemoryProviderOptions = Mem0ClientOptions; diff --git a/packages/knowledge-adapter-mem0/tsconfig.json b/packages/knowledge-adapter-mem0/tsconfig.json deleted file mode 100644 index 20b1834..0000000 --- a/packages/knowledge-adapter-mem0/tsconfig.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "compilerOptions": { - "lib": ["ESNext"], - "target": "ESNext", - "module": "ESNext", - "moduleResolution": "bundler", - "moduleDetection": "force", - "allowImportingTsExtensions": true, - "verbatimModuleSyntax": true, - "noEmit": true, - "strict": true, - "skipLibCheck": true, - "noUncheckedIndexedAccess": true, - "exactOptionalPropertyTypes": true, - "noFallthroughCasesInSwitch": true, - "forceConsistentCasingInFileNames": true, - "types": ["bun"] - }, - "include": ["src"] -} diff --git a/packages/knowledge-adapter-supermemory/README.md b/packages/knowledge-adapter-supermemory/README.md deleted file mode 100644 index f7d369e..0000000 --- a/packages/knowledge-adapter-supermemory/README.md +++ /dev/null @@ -1,79 +0,0 @@ -# @corbits/knowledge-adapter-supermemory - -**Replaceable DocumentStore** for [@corbits/memory](https://github.com/corbitsdev/corbits-memory) -backed by the [Supermemory](https://supermemory.ai/) HTTP API. - -Pure `fetch` — **no** vendor SDK. Tenancy is enforced with a length-prefixed -`containerTag` so free-form ids cannot collide. - -## Product path: DocumentStore - -Mount Supermemory as `documentStore` so the plane routes `add` / `find` / -`recent` (and `ask` via find) through this store — no Postgres / embed -endpoints required. This is the product integration path (not -`MemoryProvider` / `includeMemory`). - -```ts -import { createMemory } from "@corbits/memory"; -import { createSupermemoryDocumentStore } from "@corbits/knowledge-adapter-supermemory"; - -const memory = createMemory(undefined, grants, { - documentStore: createSupermemoryDocumentStore({ - apiKey: process.env.SUPERMEMORY_API_KEY!, - }), - generate: myGenerate, -}); -``` - -Or via mount: - -```ts -mountMemory(app, { - documentStore: createSupermemoryDocumentStore({ apiKey }), - grants, - generate, -}); -``` - -Find uses `searchMode: "hybrid"` so document retrieval works for the green -plane (add/find/ask), not memories-only personal facts. - -## Limitations (honest) - -| Area | Behavior | -| --- | --- | -| Isolation | **Principal-bucket only** via `containerTag(tenantId, principalId)`. Each principal has a private container; docs are not shared across principals. | -| Document access | This adapter is **principal-bucket only**. Host grant tags (`accessTags`) are stored as metadata at best and are **not** evaluated. For multi-principal grant-tag ACL, use the default pgvector store. | -| `recent` | Always `[]` — no recent-feed API in this adapter. | -| `options.memoryProvider` | **Never** mount this package as `options.memoryProvider`. Product backend is `documentStore` only. | - -## What is out of scope - -- **Not** a tools-shaped source (Linear-style live connectors stay separate). -- **Not** the product path for `MemoryProvider` / `includeMemory`. -- `createSupermemoryMemoryProvider` remains exported for back-compat only - (memories-only recall); prefer `createSupermemoryDocumentStore`. - -## Tenant mapping - -```ts -import { containerTag } from "@corbits/knowledge-adapter-supermemory"; - -containerTag("acme", "alice"); // "t4_acme_u5_alice" -``` - -Never pass bare `principalId` as `containerTag`. - -## HTTP surface (thin) - -| Verb | Method | Path | Notes | -| ------ | ------ | ----------------- | ------------------------------ | -| add | POST | `/v3/documents` | content + containerTag | -| find | POST | `/v4/search` | `searchMode: "hybrid"` | -| recent | — | empty | API has no recent feed here | - -Auth: `Authorization: Bearer `. - -## License - -LGPL-2.1-only (same as Corbits Memory). diff --git a/packages/knowledge-adapter-supermemory/package.json b/packages/knowledge-adapter-supermemory/package.json deleted file mode 100644 index 28d951b..0000000 --- a/packages/knowledge-adapter-supermemory/package.json +++ /dev/null @@ -1,47 +0,0 @@ -{ - "name": "@corbits/knowledge-adapter-supermemory", - "version": "0.1.0", - "description": "Supermemory DocumentStore adapter for Corbits Knowledge Engine (pure fetch, no vendor SDK)", - - "exports": { - ".": "./src/index.ts" - }, - "license": "LGPL-2.1-only", - "type": "module", - "module": "src/index.ts", - "engines": { - "bun": ">=1.2.0" - }, - "scripts": { - "typecheck": "tsc --noEmit", - "test": "bun test ./src" - }, - "devDependencies": { - "@types/bun": "latest", - "typescript": "^5.9.0" - }, - "author": "Sawyer Cutler ", - "repository": { - "type": "git", - "url": "git+https://github.com/corbitsdev/corbits-memory.git", - "directory": "packages/knowledge-adapter-supermemory" - }, - "homepage": "https://github.com/corbitsdev/corbits-memory#readme", - "bugs": { - "url": "https://github.com/corbitsdev/corbits-memory/issues" - }, - "keywords": [ - "knowledge", - "memory", - "supermemory", - "corbits" - ], - "files": [ - "src", - "!src/**/*.test.ts", - "README.md" - ], - "publishConfig": { - "access": "public" - } -} diff --git a/packages/knowledge-adapter-supermemory/src/index.test.ts b/packages/knowledge-adapter-supermemory/src/index.test.ts deleted file mode 100644 index 2233a0a..0000000 --- a/packages/knowledge-adapter-supermemory/src/index.test.ts +++ /dev/null @@ -1,236 +0,0 @@ -import { describe, expect, it, mock } from "bun:test"; - -import { - containerTag, - createSupermemoryDocumentStore, - createSupermemoryMemoryProvider, -} from "./index.ts"; - -function jsonResponse(body: unknown, status = 200): Response { - return new Response(JSON.stringify(body), { - status, - headers: { "content-type": "application/json" }, - }); -} - -describe("containerTag", () => { - it("maps tenant + principal with length prefixes", () => { - expect(containerTag("acme", "alice")).toBe("t4_acme_u5_alice"); - expect(containerTag("org-1", "user-42")).toBe("t5_org-1_u7_user-42"); - }); - - it("produces distinct tags per tenant for the same principal", () => { - const a = containerTag("tenant-a", "user-1"); - const b = containerTag("tenant-b", "user-1"); - expect(a).toBe("t8_tenant-a_u6_user-1"); - expect(b).toBe("t8_tenant-b_u6_user-1"); - expect(a).not.toBe(b); - }); - - it("is injective when ids contain delimiter sequences", () => { - const a = containerTag("x_u_y", "z"); - const b = containerTag("x", "y_u_z"); - expect(a).not.toBe(b); - expect(a).toBe("t5_x_u_y_u1_z"); - expect(b).toBe("t1_x_u5_y_u_z"); - }); - - it("rejects empty tenantId or principalId", () => { - expect(() => containerTag("", "alice")).toThrow(/non-empty/); - expect(() => containerTag("acme", "")).toThrow(/non-empty/); - expect(() => containerTag("", "")).toThrow(/non-empty/); - expect(() => containerTag(" ", "alice")).toThrow(/non-empty/); - }); -}); - -describe("createSupermemoryDocumentStore", () => { - it("rejects empty identity on add", async () => { - const fetchImpl = mock(() => Promise.resolve(jsonResponse({ id: "x" }))); - const store = createSupermemoryDocumentStore({ - apiKey: "test-key", - fetch: fetchImpl as unknown as typeof fetch, - }); - - await expect( - store.add({ - tenantId: "", - principalId: "alice", - title: "t", - text: "hello", - accessTags: ["memory.owner:alice"], - }), - ).rejects.toThrow(/non-empty/); - - expect(fetchImpl).not.toHaveBeenCalled(); - }); - - it("add posts to /v3/documents with containerTag and returns documentId", async () => { - const fetchImpl = mock((url: string, init?: RequestInit) => { - expect(url).toBe("https://api.supermemory.ai/v3/documents"); - expect(init?.method).toBe("POST"); - const headers = init?.headers as Record; - expect(headers.authorization).toBe("Bearer test-key"); - const body = JSON.parse(init?.body as string) as { - content: string; - containerTag: string; - metadata?: Record; - }; - expect(body.containerTag).toBe("t4_acme_u5_alice"); - expect(body.content).toContain("# Prefs"); - expect(body.content).toContain("prefers dark mode"); - expect(body.metadata?.title).toBe("Prefs"); - return Promise.resolve(jsonResponse({ id: "doc_1", status: "queued" })); - }); - - const store = createSupermemoryDocumentStore({ - apiKey: "test-key", - fetch: fetchImpl as unknown as typeof fetch, - }); - - const { documentId } = await store.add({ - tenantId: "acme", - principalId: "alice", - title: "Prefs", - text: "prefers dark mode", - accessTags: ["memory.owner:alice"], - }); - - expect(documentId).toMatch( - /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i, - ); - expect(fetchImpl).toHaveBeenCalledTimes(1); - }); - - it("find uses hybrid searchMode and maps hits", async () => { - const fetchImpl = mock((url: string, init?: RequestInit) => { - expect(url).toBe("https://api.supermemory.ai/v4/search"); - const body = JSON.parse(init?.body as string) as { - q: string; - containerTag: string; - searchMode: string; - limit?: number; - }; - expect(body.q).toBe("preferences"); - expect(body.containerTag).toBe("t4_acme_u5_alice"); - expect(body.searchMode).toBe("hybrid"); - expect(body.limit).toBe(3); - return Promise.resolve( - jsonResponse({ - results: [ - { - id: "mem_1", - chunk: "# Theme\n\nUser prefers dark mode", - similarity: 0.92, - metadata: { documentId: "d1" }, - }, - ], - }), - ); - }); - - const store = createSupermemoryDocumentStore({ - apiKey: "test-key", - fetch: fetchImpl as unknown as typeof fetch, - }); - - const result = await store.find({ - tenantId: "acme", - principalId: "alice", - query: "preferences", - limit: 3, - includeEvidence: true, - }); - - expect(result.evidence).toBe("weak"); - expect(result.items).toHaveLength(1); - expect(result.items[0]!.documentId).toBe("d1"); - expect(result.items[0]!.title).toBe("Theme"); - expect(result.items[0]!.citation.adapter).toBe("supermemory"); - expect(fetchImpl).toHaveBeenCalledTimes(1); - }); - - it("scopes container tags distinctly per tenant on add", async () => { - const tags: string[] = []; - const fetchImpl = mock((_url: string, init?: RequestInit) => { - const body = JSON.parse(init?.body as string) as { containerTag: string }; - tags.push(body.containerTag); - return Promise.resolve(jsonResponse({ id: "x" })); - }); - - const store = createSupermemoryDocumentStore({ - apiKey: "test-key", - fetch: fetchImpl as unknown as typeof fetch, - }); - - await store.add({ - tenantId: "tenant-a", - principalId: "user-1", - title: "a", - text: "fact a", - accessTags: ["memory.owner:user-1"], - }); - await store.add({ - tenantId: "tenant-b", - principalId: "user-1", - title: "b", - text: "fact b", - accessTags: ["memory.owner:user-1"], - }); - - expect(tags).toEqual(["t8_tenant-a_u6_user-1", "t8_tenant-b_u6_user-1"]); - }); - - it("recent returns empty; close is a no-op", async () => { - const store = createSupermemoryDocumentStore({ - apiKey: "test-key", - fetch: mock(() => Promise.resolve(jsonResponse({}))) as unknown as typeof fetch, - }); - expect( - await store.recent({ tenantId: "t", principalId: "u" }), - ).toEqual([]); - await store.close(); - }); - - it("throws when apiKey is empty", () => { - expect(() => createSupermemoryDocumentStore({ apiKey: "" })).toThrow( - /apiKey/, - ); - }); -}); - -describe("createSupermemoryMemoryProvider (legacy)", () => { - it("recall always sends searchMode: memories", async () => { - const fetchImpl = mock((url: string, init?: RequestInit) => { - expect(url).toBe("https://api.supermemory.ai/v4/search"); - const body = JSON.parse(init?.body as string) as { - searchMode: string; - }; - expect(body.searchMode).toBe("memories"); - return Promise.resolve( - jsonResponse({ - results: [ - { - id: "mem_1", - memory: "User prefers dark mode", - similarity: 0.92, - }, - ], - }), - ); - }); - - const provider = createSupermemoryMemoryProvider({ - apiKey: "test-key", - fetch: fetchImpl as unknown as typeof fetch, - }); - - const hits = await provider.recall({ - tenantId: "acme", - principalId: "alice", - query: "preferences", - limit: 3, - }); - - expect(hits).toEqual([{ text: "User prefers dark mode", score: 0.92 }]); - }); -}); diff --git a/packages/knowledge-adapter-supermemory/src/index.ts b/packages/knowledge-adapter-supermemory/src/index.ts deleted file mode 100644 index 6a22534..0000000 --- a/packages/knowledge-adapter-supermemory/src/index.ts +++ /dev/null @@ -1,458 +0,0 @@ -/** - * Supermemory DocumentStore adapter (replaceable durable backend). - * - * Pure fetch HTTP — no vendor SDK. Port shapes are defined locally so this - * package never imports the @corbits/memory runtime. - * - * Product path: createSupermemoryDocumentStore → mount as options.documentStore. - * MemoryProvider factory is back-compat only. - */ - -/** Minimal citation open shape (matches @corbits/memory SearchHitCitation). */ -export type DocumentStoreCitation = { - adapter: string; - external_ref: string; - open: { - type: string; - id: string; - url?: string; - }; -}; - -export type DocumentStoreAddParams = { - tenantId: string; - principalId: string; - title: string; - text: string; - /** Grant-tag resource strings (ignored for enforcement; principal-bucket only). */ - accessTags: string[]; - attributes?: Record; - externalRef?: string; - adapter?: string; - kind?: string; -}; - -export type DocumentStoreFindParams = { - tenantId: string; - principalId: string; - query: string; - limit?: number; - includeEvidence?: boolean; -}; - -export type DocumentStoreFindItem = { - documentId: string; - title: string; - snippet: string; - score: number; - kind: string; - citation: DocumentStoreCitation; - adapter?: string; - externalRef?: string; - updatedAt?: string; -}; - -export type DocumentStoreFindResult = { - items: DocumentStoreFindItem[]; - evidence?: "strong" | "weak" | "none"; - degraded?: string[]; -}; - -export type DocumentStoreRecentParams = { - tenantId: string; - principalId: string; - limit?: number; -}; - -export type DocumentStoreRecentEvent = { - at: string; - title: string; - source: string; - tenantId: string; - principalId: string; -}; - -/** - * Replaceable durable backend for the knowledge plane (add / find / recent). - * Mount as `options.documentStore` — no local Postgres required. - */ -export type DocumentStore = { - add(params: DocumentStoreAddParams): Promise<{ documentId: string }>; - find(params: DocumentStoreFindParams): Promise; - recent( - params: DocumentStoreRecentParams, - ): Promise; - close(): Promise; -}; - -/** @deprecated Prefer DocumentStore. Thin remember/recall only. */ -export type MemoryProvider = { - remember(params: { - tenantId: string; - principalId: string; - text: string; - metadata?: Record; - }): Promise; - recall(params: { - tenantId: string; - principalId: string; - query: string; - limit?: number; - }): Promise>; -}; - -const DEFAULT_BASE_URL = "https://api.supermemory.ai"; -const ADAPTER = "supermemory"; - -/** - * Map tenant + principal to a Supermemory containerTag. - * Length-prefixed so free-form ids cannot collide across delimiter injection: - * `t{len}_{tenant}_u{len}_{principal}` - */ -export function containerTag(tenantId: string, principalId: string): string { - if (typeof tenantId !== "string" || tenantId.trim() === "") { - throw new Error( - "containerTag requires non-empty tenantId and principalId", - ); - } - if (typeof principalId !== "string" || principalId.trim() === "") { - throw new Error( - "containerTag requires non-empty tenantId and principalId", - ); - } - return `t${tenantId.length}_${tenantId}_u${principalId.length}_${principalId}`; -} - -export type SupermemoryClientOpts = { - apiKey: string; - /** API root (no trailing slash). Default: https://api.supermemory.ai */ - baseUrl?: string; - /** Injectable fetch for tests. Default: globalThis.fetch */ - fetch?: typeof fetch; -}; - -/** @deprecated Use SupermemoryClientOpts */ -export type SupermemoryMemoryProviderOpts = SupermemoryClientOpts; - -function requireIdentity(tenantId: string, principalId: string): void { - if (typeof tenantId !== "string" || tenantId.trim() === "") { - throw new Error( - "Supermemory DocumentStore requires non-empty tenantId and principalId", - ); - } - if (typeof principalId !== "string" || principalId.trim() === "") { - throw new Error( - "Supermemory DocumentStore requires non-empty tenantId and principalId", - ); - } -} - -function jsonHeaders(apiKey: string): Record { - return { - authorization: `Bearer ${apiKey}`, - "content-type": "application/json", - }; -} - -async function readErrorBody(res: Response): Promise { - try { - const text = await res.text(); - return text.length > 200 ? `${text.slice(0, 200)}…` : text; - } catch { - return ""; - } -} - -function encodeContent(params: { - title: string; - text: string; - documentId: string; - externalRef?: string; - accessTags: string[]; -}): string { - const header = `# ${params.title}`; - const meta = [ - `documentId: ${params.documentId}`, - params.externalRef ? `externalRef: ${params.externalRef}` : null, - params.accessTags.length > 0 - ? `accessTags: ${params.accessTags.join(",")}` - : null, - ] - .filter(Boolean) - .join("\n"); - return `${header}\n\n${params.text}\n\n---\n${meta}`; -} - -function parseTitleAndSnippet(text: string): { title: string; snippet: string } { - const lines = text.split("\n"); - if (lines[0]?.startsWith("# ")) { - const title = lines[0].slice(2).trim() || "untitled"; - const rest = lines - .slice(1) - .join("\n") - .replace(/\n---\n[\s\S]*$/, "") - .trim(); - return { - title, - snippet: rest.slice(0, 240) || title, - }; - } - return { - title: text.slice(0, 80) || "untitled", - snippet: text.slice(0, 240), - }; -} - -/** - * Create a DocumentStore backed by Supermemory (v3 documents + v4 search). - * - * Pure fetch — no vendor SDK. Mount as the plane's durable backend: - * - * ```ts - * createMemory(undefined, grants, { - * documentStore: createSupermemoryDocumentStore({ apiKey }), - * }) - * ``` - * - * Find uses hybrid search (documents + chunks) so the store can answer - * retrieval for add/find/ask — not memories-only personal facts. - */ -export function createSupermemoryDocumentStore( - opts: SupermemoryClientOpts, -): DocumentStore { - const baseUrl = (opts.baseUrl ?? DEFAULT_BASE_URL).replace(/\/$/, ""); - const fetchImpl = opts.fetch ?? globalThis.fetch; - const { apiKey } = opts; - - if (typeof apiKey !== "string" || apiKey.trim() === "") { - throw new Error( - "createSupermemoryDocumentStore requires a non-empty apiKey", - ); - } - - return { - async add(params) { - requireIdentity(params.tenantId, params.principalId); - const tag = containerTag(params.tenantId, params.principalId); - const documentId = crypto.randomUUID(); - const content = encodeContent({ - title: params.title, - text: params.text, - documentId, - ...(params.externalRef !== undefined - ? { externalRef: params.externalRef } - : {}), - accessTags: params.accessTags ?? [], - }); - const metadata: Record = { - documentId, - title: params.title, - }; - if (params.accessTags?.length) { - metadata.accessTags = params.accessTags.join(","); - } - if (params.externalRef !== undefined) { - metadata.externalRef = params.externalRef; - } - - const res = await fetchImpl(`${baseUrl}/v3/documents`, { - method: "POST", - headers: jsonHeaders(apiKey), - body: JSON.stringify({ - content, - containerTag: tag, - metadata, - }), - }); - if (!res.ok) { - const snippet = await readErrorBody(res); - throw new Error( - `Supermemory add failed HTTP ${res.status}: ${snippet}`, - ); - } - return { documentId }; - }, - - async find(params) { - requireIdentity(params.tenantId, params.principalId); - const tag = containerTag(params.tenantId, params.principalId); - const body: Record = { - q: params.query, - containerTag: tag, - // Hybrid retrieval for store replacement (not memories-only facts). - searchMode: "hybrid", - }; - if (params.limit !== undefined) { - body.limit = params.limit; - } - - const res = await fetchImpl(`${baseUrl}/v4/search`, { - method: "POST", - headers: jsonHeaders(apiKey), - body: JSON.stringify(body), - }); - if (!res.ok) { - const snippet = await readErrorBody(res); - throw new Error( - `Supermemory find failed HTTP ${res.status}: ${snippet}`, - ); - } - - const data = (await res.json()) as { - results?: Array<{ - id?: string; - memory?: string; - chunk?: string; - content?: string; - similarity?: number; - metadata?: Record; - }>; - }; - - const results = data.results ?? []; - const items: DocumentStoreFindItem[] = []; - for (const r of results) { - const text = r.memory ?? r.chunk ?? r.content ?? ""; - if (text === "") continue; - const meta = r.metadata ?? {}; - const documentId = - (typeof meta.documentId === "string" && meta.documentId) || - (typeof r.id === "string" && r.id) || - crypto.randomUUID(); - const externalRef = - (typeof meta.externalRef === "string" && meta.externalRef) || - documentId; - const { title, snippet } = parseTitleAndSnippet(text); - const score = - typeof r.similarity === "number" && Number.isFinite(r.similarity) - ? r.similarity - : 0.5; - items.push({ - documentId, - title, - snippet, - score, - kind: "note", - adapter: ADAPTER, - externalRef, - citation: { - adapter: ADAPTER, - external_ref: externalRef, - open: { - type: "document", - id: documentId, - url: `supermemory://${documentId}`, - }, - }, - }); - } - - if (params.includeEvidence) { - return { - items, - evidence: items.length === 0 ? "none" : "weak", - }; - } - return { items }; - }, - - async recent() { - return []; - }, - - async close() { - // Stateless HTTP client. - }, - }; -} - -/** - * @deprecated Prefer createSupermemoryDocumentStore as options.documentStore. - * Thin remember/recall kept for back-compat; not the product path. - */ -export function createSupermemoryMemoryProvider( - opts: SupermemoryClientOpts, -): MemoryProvider { - const baseUrl = (opts.baseUrl ?? DEFAULT_BASE_URL).replace(/\/$/, ""); - const fetchImpl = opts.fetch ?? globalThis.fetch; - const { apiKey } = opts; - - if (typeof apiKey !== "string" || apiKey.trim() === "") { - throw new Error( - "createSupermemoryMemoryProvider requires a non-empty apiKey", - ); - } - - return { - async remember(params) { - requireIdentity(params.tenantId, params.principalId); - const tag = containerTag(params.tenantId, params.principalId); - const body: Record = { - content: params.text, - containerTag: tag, - }; - if (params.metadata !== undefined) { - body.metadata = params.metadata; - } - - const res = await fetchImpl(`${baseUrl}/v3/documents`, { - method: "POST", - headers: jsonHeaders(apiKey), - body: JSON.stringify(body), - }); - if (!res.ok) { - const snippet = await readErrorBody(res); - throw new Error( - `Supermemory remember failed HTTP ${res.status}: ${snippet}`, - ); - } - }, - - async recall(params) { - requireIdentity(params.tenantId, params.principalId); - const tag = containerTag(params.tenantId, params.principalId); - const body: Record = { - q: params.query, - containerTag: tag, - // Legacy memories-only path for personal-fact recall. - searchMode: "memories", - }; - if (params.limit !== undefined) { - body.limit = params.limit; - } - - const res = await fetchImpl(`${baseUrl}/v4/search`, { - method: "POST", - headers: jsonHeaders(apiKey), - body: JSON.stringify(body), - }); - if (!res.ok) { - const snippet = await readErrorBody(res); - throw new Error( - `Supermemory recall failed HTTP ${res.status}: ${snippet}`, - ); - } - - const data = (await res.json()) as { - results?: Array<{ - memory?: string; - chunk?: string; - similarity?: number; - }>; - }; - - const results = data.results ?? []; - return results - .map((r) => { - const text = r.memory ?? r.chunk ?? ""; - if (text === "") return null; - const item: { text: string; score?: number } = { text }; - if (typeof r.similarity === "number") { - item.score = r.similarity; - } - return item; - }) - .filter((x): x is { text: string; score?: number } => x !== null); - }, - }; -} diff --git a/packages/knowledge-adapter-supermemory/tsconfig.json b/packages/knowledge-adapter-supermemory/tsconfig.json deleted file mode 100644 index 20b1834..0000000 --- a/packages/knowledge-adapter-supermemory/tsconfig.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "compilerOptions": { - "lib": ["ESNext"], - "target": "ESNext", - "module": "ESNext", - "moduleResolution": "bundler", - "moduleDetection": "force", - "allowImportingTsExtensions": true, - "verbatimModuleSyntax": true, - "noEmit": true, - "strict": true, - "skipLibCheck": true, - "noUncheckedIndexedAccess": true, - "exactOptionalPropertyTypes": true, - "noFallthroughCasesInSwitch": true, - "forceConsistentCasingInFileNames": true, - "types": ["bun"] - }, - "include": ["src"] -} From c0196165995ff3d9e110c0ce7b4cb8976c00f250 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 16:42:09 -0700 Subject: [PATCH 06/11] Lock createMemory to a single options bag; rename sibling packages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit createMemory({ grants, config?, documentStore?, ... }) only — no undefined config slot. Docs/package links point at @corbits/mem0-memory-adapter, @corbits/supermemory-memory-adapter, and @corbits/linear-tools. --- AGENTS.md | 5 ++- IMPLEMENTATION.md | 8 ++-- PRODUCT.md | 24 +++++----- README.md | 82 ++++++++++++++++++++++------------ src/index.ts | 11 ++--- src/memory.test.ts | 37 ++++++++------- src/memory.ts | 59 +++++++++++++++++++----- src/ports/memory-plane.test.ts | 63 +++++++++++--------------- src/ports/merge-plane.test.ts | 37 +++++++-------- 9 files changed, 192 insertions(+), 134 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 39716d5..3df9319 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,8 +25,9 @@ CI runs `typecheck` + `test` — both must pass before any push. - `src/ports/` — `DocumentStore` / `SourceProvider` / `MemoryProvider` + fakes - `src/core/` — embed/rerank clients, merge, arktype schemas - `src/db/` + `migrations/` — Drizzle schema + SQL migrations (pgvector, `knowledge.*`) -- `packages/` — removed; DocumentStore adapters are sibling packages - (`@corbits/mem0`, `@corbits/supermemory`). Linear tools: `@corbits/linear`. +- `packages/` — removed; DocumentStore adapters and Linear tools are sibling packages + (`@corbits/mem0-memory-adapter`, `@corbits/supermemory-memory-adapter`, + `@corbits/linear-tools`). ## Non-negotiable invariants diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 0d81ff3..5c4c78e 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -30,10 +30,10 @@ src/ timeline.ts # listTimelineEvents — durable recent docs + grant-tag filter transform.ts # transform_config CRUD + runTransform (replay) core/ # framework-agnostic (chunking, embed/rerank, merge, schemas) -# DocumentStore adapters live as sibling packages (not in this tree): -# @corbits/mem0 → github.com/corbitsdev/corbits-mem0 -# @corbits/supermemory → github.com/corbitsdev/corbits-supermemory -# Linear SourceProvider: sibling @corbits/linear +# DocumentStore adapters / tools live as sibling packages (not in this tree): +# @corbits/mem0-memory-adapter → github.com/corbitsdev/corbits-mem0-memory-adapter +# @corbits/supermemory-memory-adapter → github.com/corbitsdev/corbits-supermemory-memory-adapter +# @corbits/linear-tools → github.com/corbitsdev/corbits-linear-tools migrations/ # pgvector schema, applied in filename order by scripts/db-setup.ts scripts/db-setup.ts # idempotent migration runner, tracked in `_migrations` compose.yml # pgvector + Ollama + reranker for local dev diff --git a/PRODUCT.md b/PRODUCT.md index dd8ce7b..86b6b9f 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -12,7 +12,7 @@ SDK never creates one; it mounts onto yours. | Surface | Role | | --- | --- | | `mountMemory(app, opts)` | Plane + HTTP on an Interchange `createApp` | -| `createMemory(config, grants?, options?)` | Same plane without HTTP | +| `createMemory(options?: MemoryOptions)` | Same plane without HTTP | | `runMemoryMigrations(url)` | Apply pgvector schema under Postgres `knowledge` | | `loadMemoryConfig()` | Mount config from env | @@ -60,22 +60,22 @@ never auto-writes. ### Optional adapter packages -Optional `DocumentStore` implementations live as **sibling packages** (not in -this tree), same pattern as `@corbits/linear` / `@corbits/granola`: +Optional `DocumentStore` implementations and tools live as **sibling packages** +(not in this tree), same pattern as `@corbits/granola`: -- [`@corbits/mem0`](https://github.com/corbitsdev/corbits-mem0) — Mem0 backend -- [`@corbits/supermemory`](https://github.com/corbitsdev/corbits-supermemory) — Supermemory backend +- [`@corbits/mem0-memory-adapter`](https://github.com/corbitsdev/corbits-mem0-memory-adapter) — Mem0 backend +- [`@corbits/supermemory-memory-adapter`](https://github.com/corbitsdev/corbits-supermemory-memory-adapter) — Supermemory backend +- [`@corbits/linear-tools`](https://github.com/corbitsdev/corbits-linear-tools) — Linear SourceProvider + webhook tools (not a store) Core never imports vendor SDKs. Hosts that need a store beyond default pgvector mount their own `documentStore`. -**Third-party store honesty:** not every `DocumentStore` evaluates host grant -tags. Some isolate by **principal bucket** only (one private namespace per -tenant+principal) and do **not** multi-share via `accessTags` + host -`GrantStore`. For full grant-tag ACL, use the default pgvector store (or a store -that implements the contract in `docs/AUTHZ-DOCUMENT-ACCESS.md`). Adapter -packages must document their isolation model in their own README. Never mount a -durable store as `options.memoryProvider`. +Not every `DocumentStore` evaluates host grant tags the same way. Some isolate by +**principal bucket** only (one private namespace per tenant+principal) and do not +multi-share via `accessTags` + host `GrantStore`. For full grant-tag ACL, use the +default pgvector store (or a store that implements +`docs/AUTHZ-DOCUMENT-ACCESS.md`). Adapter packages document their isolation model +in their own README. Never mount a durable store as `options.memoryProvider`. ### What is not in scope diff --git a/README.md b/README.md index cb20878..52dac07 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # @corbits/memory -Mountable knowledge plane for [Interchange](https://github.com/corbitsdev) hubs: +Mountable memory plane for [Interchange](https://github.com/corbitsdev) hubs: **add** documents, **find** with hybrid search, **ask** grounded answers, **recent** timeline — with optional live sources and personal memory. @@ -13,21 +13,28 @@ Requires Bun 1.2+. ## Install +Not published to npm yet. Install from git: + ```bash -bun add @corbits/memory +bun add git+https://github.com/corbitsdev/corbits-memory.git ``` -## Mount (green path) +## Mount ```ts -import { mountMemory } from "@corbits/memory"; -import { loadMemoryConfig } from "@corbits/memory/config"; +import { mountMemory, loadMemoryConfig } from "@corbits/memory"; +import type { GrantStore, ConditionRegistry } from "@intx/authz"; + +// Same pair the host passes to createApp / createRequireGrant. +const grants = { + grantStore: hostGrantStore as GrantStore, + conditionRegistry: hostConditionRegistry as ConditionRegistry, +}; mountMemory(app, { config: loadMemoryConfig(), - grants: { grantStore, conditionRegistry }, - // optional ports: - // documentStore, sources, memoryProvider, textExtractor, generate + grants, + // optional: documentStore, sources, memoryProvider, textExtractor, generate }); ``` @@ -50,17 +57,26 @@ Without it: **401 `principal_required`**. ### Plane without HTTP +One options bag — never `createMemory(undefined, …)`. + ```ts import { createMemory, createFakeDocumentStore, createFakeMemoryProvider, } from "@corbits/memory"; +import type { GrantStore, ConditionRegistry } from "@intx/authz"; + +const grants = { + grantStore: hostGrantStore as GrantStore, + conditionRegistry: hostConditionRegistry as ConditionRegistry, +}; -const memory = createMemory(undefined, grants, { +const memory = createMemory({ + grants, documentStore: createFakeDocumentStore(), memoryProvider: createFakeMemoryProvider(), - generate: async (messages) => "…", // wire your inference layer + generate: async (messages) => "…", // wire your inference layer for ask() }); await memory.add({ @@ -77,6 +93,15 @@ const answer = await memory.ask({ }); ``` +With the default Postgres store: + +```ts +const memory = createMemory({ + config: loadMemoryConfig(), + grants, +}); +``` + ## Ports | Port | Default | Override | @@ -88,27 +113,30 @@ const answer = await memory.ask({ **Live merge (MergeLocalLiveV1):** per-provider timeout/error → `live_timeout` / `live_error` degrade; dedupe `adapter:externalRef`; optional `sources` filter. -**Memory:** `ask({ includeMemory: true })` recalls when a provider is mounted; -failure → `memory_unavailable`, docs-only. `plane.remember` / `plane.recall` for -host-owned writes (ask never auto-remembers). +**Memory side-channel:** `ask({ includeMemory: true })` recalls when a provider +is mounted; failure → `memory_unavailable`, docs-only. `plane.remember` / +`plane.recall` for host-owned writes (ask never auto-remembers). -### Optional DocumentStore adapters +### Optional DocumentStore adapters (sibling packages) -Optional adapters are **sibling packages** (not vendored in this repo). Core -never imports vendor SDKs. See each package README for mount examples and -isolation limits: +Optional backends are **sibling packages**, not vendored here. Core never imports +vendor SDKs. Install each from git; each README only documents itself + this core. -- [`@corbits/mem0`](https://github.com/corbitsdev/corbits-mem0) -- [`@corbits/supermemory`](https://github.com/corbitsdev/corbits-supermemory) +| Package | Role | +| --- | --- | +| [`@corbits/mem0-memory-adapter`](https://github.com/corbitsdev/corbits-mem0-memory-adapter) | Mem0 as `documentStore` | +| [`@corbits/supermemory-memory-adapter`](https://github.com/corbitsdev/corbits-supermemory-memory-adapter) | Supermemory as `documentStore` | +| [`@corbits/linear-tools`](https://github.com/corbitsdev/corbits-linear-tools) | Linear **tools** (`SourceProvider` + webhook map) — not a store | ```ts -// Linear SourceProvider lives in sibling repo @corbits/linear -// (https://github.com/corbitsdev/corbits-linear), not this monorepo. -import { - createLinearSourceProvider, - mapLinearWebhook, -} from "@corbits/linear"; -// host owns OAuth + webhook verify; private issues never map into tenant docs +import { createMem0DocumentStore } from "@corbits/mem0-memory-adapter"; +// or: createSupermemoryDocumentStore from @corbits/supermemory-memory-adapter +// or: createLinearSourceProvider from @corbits/linear-tools + +const memory = createMemory({ + documentStore: createMem0DocumentStore({ apiKey: process.env.MEM0_API_KEY! }), + grants, +}); ``` ## Migrations @@ -144,8 +172,6 @@ Full design: `docs/AUTHZ-DOCUMENT-ACCESS.md`. bun install bun run typecheck bun run test -# DocumentStore adapters: sibling @corbits/mem0, @corbits/supermemory -# Linear tools: sibling @corbits/linear ``` License: LGPL-2.1 (`LICENSE`). Contributions: `CLA.md`. diff --git a/src/index.ts b/src/index.ts index 06f5bc8..bcde12c 100644 --- a/src/index.ts +++ b/src/index.ts @@ -41,6 +41,7 @@ export { RerankConfigError } from "./core/rerank-client.ts"; // responsible for the capability check `requireGrant` would have performed; see // the README. Rerank config is validated at construction (same as mount). // Pass `grants` + optional `generate` when the host will call `ask()`. +// One options bag — never createMemory(undefined, …). export { createMemory } from "./memory.ts"; export type { AskCitation, @@ -163,11 +164,11 @@ export function mountMemory( ? { memoryProvider: options.memoryProvider } : {}), }; - const memory = createMemory( - options.config, - options.grants, - planeOptions, - ); + const memory = createMemory({ + ...(options.config ? { config: options.config } : {}), + grants: options.grants, + ...planeOptions, + }); const deps: RouteDeps = { memory, grants: options.grants, diff --git a/src/memory.test.ts b/src/memory.test.ts index c8ef664..937ffb4 100644 --- a/src/memory.test.ts +++ b/src/memory.test.ts @@ -175,14 +175,14 @@ describe("createMemory — construction validation", () => { // mountMemory): a standalone plane with a bad override must fail // construction, not silently degrade on every later find. expect(() => - createMemory( - baseConfig({ + createMemory({ + config: baseConfig({ baseUrl: "https://tei.example.com", model: "bge-reranker-base", apiKey: undefined, maxDocChars: 5_000, }), - ), + }), ).toThrow(RerankConfigError); }); }); @@ -260,7 +260,8 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { const { createMemory: makePlane } = await import( `./memory.ts?wiring-blocked=${Date.now()}` ); - const plane = makePlane(wiringConfig); + const plane = makePlane({ config: wiringConfig }); + const result = await plane.find({ tenantId: TENANT, principalId: PRINCIPAL, @@ -290,7 +291,8 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { const { createMemory: makePlane } = await import( `./memory.ts?wiring-kinds=${Date.now()}` ); - const plane = makePlane(wiringConfig); + const plane = makePlane({ config: wiringConfig }); + await plane.find({ tenantId: TENANT, principalId: PRINCIPAL, @@ -324,7 +326,8 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { const { createMemory: makePlane } = await import( `./memory.ts?wiring-unreadable=${Date.now()}` ); - const plane = makePlane(wiringConfig); + const plane = makePlane({ config: wiringConfig }); + const result = await plane.find({ tenantId: TENANT, principalId: PRINCIPAL, @@ -361,7 +364,8 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { const { createMemory: makePlane } = await import( `./memory.ts?wiring-evidence=${Date.now()}` ); - const plane = makePlane(wiringConfig); + const plane = makePlane({ config: wiringConfig }); + const without = await plane.find({ tenantId: TENANT, @@ -389,7 +393,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { describe("find/recent — limit bounds", () => { // These throw before any DB work, so a nonexistent URL is fine. it("find rejects limit below 1", async () => { - const plane = createMemory(wiringConfig); + const plane = createMemory({ config: wiringConfig }); try { await plane.find({ tenantId: TENANT, @@ -406,7 +410,7 @@ describe("find/recent — limit bounds", () => { }); it("find rejects limit above 50", async () => { - const plane = createMemory(wiringConfig); + const plane = createMemory({ config: wiringConfig }); try { await plane.find({ tenantId: TENANT, @@ -422,7 +426,7 @@ describe("find/recent — limit bounds", () => { }); it("recent rejects limit above 100", async () => { - const plane = createMemory(wiringConfig); + const plane = createMemory({ config: wiringConfig }); try { await plane.recent({ tenantId: TENANT, @@ -437,7 +441,7 @@ describe("find/recent — limit bounds", () => { }); it("recent rejects limit below 1", async () => { - const plane = createMemory(wiringConfig); + const plane = createMemory({ config: wiringConfig }); try { await plane.recent({ tenantId: TENANT, @@ -496,7 +500,8 @@ async function freshPlane(opts?: { const { createMemory: makePlane } = await import( `./memory.ts?add-${Date.now()}-${Math.random()}` ); - return makePlane(wiringConfig, undefined, opts ?? {}); + return makePlane({ config: wiringConfig, ...(opts ?? {}) }); + } /** Dynamic re-import yields a distinct MemoryError class; match by shape. */ @@ -733,7 +738,7 @@ describe("ask() — grant check", () => { grantStore: createInMemoryGrantStore([]), conditionRegistry: {}, }; - const plane = createMemory(askConfig, grants); + const plane = createMemory({ config: askConfig, grants }); await expect( plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), ).rejects.toBeInstanceOf(MemoryNotPermittedError); @@ -745,7 +750,7 @@ const denyGrant: GrantRule = { ...grant("find"), effect: "deny" }; grantStore: createInMemoryGrantStore([denyGrant]), conditionRegistry: {}, }; - const plane = createMemory(askConfig, grants); + const plane = createMemory({ config: askConfig, grants }); await expect( plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), ).rejects.toBeInstanceOf(MemoryNotPermittedError); @@ -760,7 +765,7 @@ describe("ask() — missing generate", () => { grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, }; - const plane = createMemory(askConfig, grants); + const plane = createMemory({ config: askConfig, grants }); try { await plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }); throw new Error("expected ask() to reject"); @@ -785,7 +790,7 @@ describe("ask() — allow path", () => { expect(messages[1]?.content).toContain("the relevant snippet"); return Promise.resolve("Answer from context [1]."); }); - const plane = createMemory(askConfig, grants, { generate }); + const plane = createMemory({ config: askConfig, grants, generate }); // Stub find so this unit test never needs a live Postgres. ask() looks // up plane.find at call time, so reassignment is the wiring under test. plane.find = mock(() => diff --git a/src/memory.ts b/src/memory.ts index 7789a00..e6541de 100644 --- a/src/memory.ts +++ b/src/memory.ts @@ -374,6 +374,13 @@ export async function synthesizeAnswer( } export type MemoryOptions = { + /** Engine config (DB + model endpoints). Required when `documentStore` is omitted. */ + config?: MemoryConfig; + /** + * Host grant store + condition registry. Required for `ask()` (in-process + * capability check). Standalone add/find callers may omit it. + */ + grants?: GrantConfig; /** Required for `ask()`; omit if the host only adds and finds. */ generate?: Generate; /** Required for `add({ file })`; omit if the host only adds text content. */ @@ -471,36 +478,66 @@ function resolveAddAccessTags(params: MemoryAddParams): string[] { } /** - * Build a knowledge plane. + * Build a memory plane. * * One product path: every plane is store-backed. When `options.documentStore` * is omitted, the default pgvector engine is wrapped as that store. Hosts * inject a DocumentStore or fakes the same way — no second plane implementation. * - * - `grants` is required for `ask()` (in-process capability check). Standalone - * add/find callers may omit it. + * - `options.grants` is required for `ask()` (in-process capability check). + * Standalone add/find callers may omit it. * - Rerank config is validated at construction when using the default store. * - Pass `options.sources` for live SourceProviders; find/ask merge via * MergeLocalLiveV1 (fail-soft, 800ms timeout, prefer-local dedupe). * - Document access uses grant tags via the host GrantStore (not mini-ACL). + * + * @example With default pgvector store + * ```ts + * const memory = createMemory({ + * config: loadMemoryConfig(), + * grants: { grantStore, conditionRegistry }, + * }); + * ``` + * + * @example With a host DocumentStore (no Postgres) + * ```ts + * const memory = createMemory({ + * documentStore: myStore, + * grants: { grantStore, conditionRegistry }, + * }); + * ``` */ -export function createMemory( - config: MemoryConfig | undefined, - grants?: GrantConfig, - options: MemoryOptions = {}, -): Memory { +export function createMemory(options: MemoryOptions = {}): Memory { + const { + config, + grants, + documentStore, + generate, + textExtractor, + sources, + memoryProvider, + } = options; const store = - options.documentStore ?? + documentStore ?? (() => { if (!config) { throw new MemoryError( 500, - "MemoryConfig is required when documentStore is not provided", + "config is required when documentStore is not provided", ); } return createEngineDocumentStore(config); })(); - return createPlaneFromStore(store, grants, options); + return createPlaneFromStore( + store, + grants, + { + ...(generate ? { generate } : {}), + ...(textExtractor ? { textExtractor } : {}), + ...(sources ? { sources } : {}), + ...(memoryProvider ? { memoryProvider } : {}), + }, + ); } function wantsLocalChannel(sources: string[] | undefined): boolean { diff --git a/src/ports/memory-plane.test.ts b/src/ports/memory-plane.test.ts index 630372f..1084f2f 100644 --- a/src/ports/memory-plane.test.ts +++ b/src/ports/memory-plane.test.ts @@ -44,21 +44,18 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, }; const store = createFakeDocumentStore(); - const plane = createMemory( - undefined, - { + const plane = createMemory({ + grants: { grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, }, - { - documentStore: store, - memoryProvider: memory, - generate: async (msgs) => { - const last = msgs[msgs.length - 1]?.content ?? ""; - return last.includes("Personal memory") ? "HAS_MEM" : "NO_MEM"; - }, + documentStore: store, + memoryProvider: memory, + generate: async (msgs) => { + const last = msgs[msgs.length - 1]?.content ?? ""; + return last.includes("Personal memory") ? "HAS_MEM" : "NO_MEM"; }, - ); + }); await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, @@ -82,23 +79,20 @@ describe("MemoryProvider product wire (CL-5228)", () => { text: "user prefers dark mode", }); const store = createFakeDocumentStore(); - const plane = createMemory( - undefined, - { + const plane = createMemory({ + grants: { grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, }, - { - documentStore: store, - memoryProvider: memory, - generate: async (msgs) => { - const last = msgs[msgs.length - 1]?.content ?? ""; - return last.includes("user prefers dark mode") - ? "saw-memory" - : "missed"; - }, + documentStore: store, + memoryProvider: memory, + generate: async (msgs) => { + const last = msgs[msgs.length - 1]?.content ?? ""; + return last.includes("user prefers dark mode") + ? "saw-memory" + : "missed"; }, - ); + }); await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, @@ -122,18 +116,15 @@ describe("MemoryProvider product wire (CL-5228)", () => { }, }; const store = createFakeDocumentStore(); - const plane = createMemory( - undefined, - { + const plane = createMemory({ + grants: { grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, }, - { - documentStore: store, - memoryProvider: memory, - generate: async () => "docs-only [1]", - }, - ); + documentStore: store, + memoryProvider: memory, + generate: async () => "docs-only [1]", + }); await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, @@ -152,7 +143,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { it("plane.remember writes; plane.recall reads", async () => { const memory = createFakeMemoryProvider(); - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: createFakeDocumentStore(), memoryProvider: memory, }); @@ -171,7 +162,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }); it("plane.remember without memory throws 501", async () => { - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: createFakeDocumentStore(), }); try { @@ -189,7 +180,7 @@ describe("MemoryProvider product wire (CL-5228)", () => { }); it("plane.recall without memory returns empty", async () => { - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: createFakeDocumentStore(), }); const items = await plane.recall({ diff --git a/src/ports/merge-plane.test.ts b/src/ports/merge-plane.test.ts index 34acf73..33eb7dc 100644 --- a/src/ports/merge-plane.test.ts +++ b/src/ports/merge-plane.test.ts @@ -54,7 +54,7 @@ function liveHit( describe("plane merge (MergeLocalLiveV1)", () => { it("merges local store hits with live source hits", async () => { const store = createFakeDocumentStore(); - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: store, sources: [ createFakeSourceProvider("linear", [ @@ -83,7 +83,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("includes live-only hits when query matches catalog", async () => { const store = createFakeDocumentStore(); - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: store, sources: [ createFakeSourceProvider("linear", [ @@ -105,7 +105,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("source filter local-only excludes live hits", async () => { const store = createFakeDocumentStore(); - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: store, sources: [ createFakeSourceProvider("linear", [ @@ -152,7 +152,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { }, }; - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: store, sources: [brokenSource, slowSource], }); @@ -187,7 +187,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { // Fake store citation uses adapter "fake" not linear — force collision by // using a custom store find isn't possible; instead use live adapter // "fake" so keys match fake store's citation adapter. - const plane = createMemory(undefined, undefined, { + const plane = createMemory({ documentStore: store, sources: [ { @@ -231,25 +231,22 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("ask still works when live source errors", async () => { const store = createFakeDocumentStore(); - const plane = createMemory( - undefined, - { + const plane = createMemory({ + grants: { grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, }, - { - documentStore: store, - sources: [ - { - id: "broken", - searchLive: async () => { - throw new Error("boom"); - }, + documentStore: store, + sources: [ + { + id: "broken", + searchLive: async () => { + throw new Error("boom"); }, - ], - generate: async () => "ok [1]", - }, - ); + }, + ], + generate: async () => "ok [1]", + }); await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, From a095db52ff769d00c7d7b9ace37897c466b57844 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 18:24:27 -0700 Subject: [PATCH 07/11] =?UTF-8?q?Document=20full=200=E2=86=921=20mini=20ho?= =?UTF-8?q?st=20in=20README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace mountMemory/grants fragments with a complete Hono + @intx/authz in-memory grant store + fakes example (in-process and HTTP), plus curl and a short production-host note. --- README.md | 256 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 181 insertions(+), 75 deletions(-) diff --git a/README.md b/README.md index 52dac07..ff64761 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,8 @@ timeline — with optional live sources and personal memory. **Authenticates nothing.** Identity is `c.get("principal")` on HTTP; in-process callers pass `principalId` + `tenantId`. Authorization is the host grant store -(`memory:add` / `memory:find`). Never embeds in-process — embedding and -rerank are outbound HTTP to configured endpoints. +(`memory` resource + `add` / `find` actions via `@intx/authz`). Never embeds +in-process — embedding and rerank are outbound HTTP to configured endpoints. Requires Bun 1.2+. @@ -17,91 +17,211 @@ Not published to npm yet. Install from git: ```bash bun add git+https://github.com/corbitsdev/corbits-memory.git +bun add @intx/authz@0.2.2 @intx/hub-api@0.2.2 hono ``` -## Mount +## Quick start — complete mini app (no Postgres) + +This is a full host you can run. It uses: + +- Hono as the HTTP app +- `@intx/authz` in-memory grant store (same type as a real Interchange hub) +- in-package fakes for storage (no DB, no embed endpoints) +- principal + tenant middleware on `/api/memory/*` (required — these routes sit + outside `/api/tenants/:tenantId/*`) + +Save as `server.ts` and run with `bun run server.ts`. ```ts -import { mountMemory, loadMemoryConfig } from "@corbits/memory"; -import type { GrantStore, ConditionRegistry } from "@intx/authz"; +import { Hono } from "hono"; +import type { TenantEnv } from "@intx/hub-api"; +import { createInMemoryGrantStore, type GrantRule } from "@intx/authz"; +import { + createFakeDocumentStore, + createMemory, + mountMemory, +} from "@corbits/memory"; + +const TENANT = "tenant_demo"; +const PRINCIPAL = "principal_demo"; + +// 1. Capability grants the host would normally load from Interchange. +// Routes call requireGrant("memory", "add" | "find"). +const grantRules: GrantRule[] = [ + { + id: "g-add", + principalId: PRINCIPAL, + resource: "memory", + action: "add", + effect: "allow", + origin: "role", + conditions: null, + expiresAt: null, + roleId: null, + }, + { + id: "g-find", + principalId: PRINCIPAL, + resource: "memory", + action: "find", + effect: "allow", + origin: "role", + conditions: null, + expiresAt: null, + roleId: null, + }, +]; -// Same pair the host passes to createApp / createRequireGrant. const grants = { - grantStore: hostGrantStore as GrantStore, - conditionRegistry: hostConditionRegistry as ConditionRegistry, + grantStore: createInMemoryGrantStore(grantRules), + conditionRegistry: {}, // empty registry is fine when grants have no conditions }; -mountMemory(app, { - config: loadMemoryConfig(), +// 2. Durable store. Fakes prove the port boundary; swap for Postgres or a +// sibling DocumentStore adapter later. +const documentStore = createFakeDocumentStore(); + +// 3. Hono app with principal + tenant on every request (Interchange shape). +const app = new Hono(); + +app.use("/api/memory/*", async (c, next) => { + // In a real hub, session + tenant middleware set these. + // Memory routes do NOT sit under /api/tenants/:tenantId/* — you must set them. + c.set("principal", { + id: PRINCIPAL, + tenantId: TENANT, + kind: "user", + refId: "demo-user", + status: "active", + createdAt: new Date(0), + updatedAt: new Date(0), + }); + c.set("tenant", { + id: TENANT, + name: "Demo", + slug: "demo", + domain: "demo.local", + parentId: null, + config: {}, + createdAt: new Date(0), + updatedAt: new Date(0), + }); + await next(); +}); + +// 4. Mount HTTP plane. Returns the same Memory object for in-process use. +const { memory } = mountMemory(app, { grants, - // optional: documentStore, sources, memoryProvider, textExtractor, generate + documentStore, + // optional: generate for ask(); textExtractor for add({ file }) + generate: async (messages) => { + const last = messages.at(-1)?.content ?? ""; + return `demo answer for: ${last}`; + }, }); -``` -Routes (each grant-checked): +// 5. In-process path (CLI, worker, tests) — same plane, no HTTP. +// One options bag only. Never createMemory(undefined, …). +const plane = createMemory({ + grants, + documentStore, + generate: async () => "in-process answer", +}); -| Method | Path | Grant | -| --- | --- | --- | -| POST | `/api/memory/add` | `memory:add` | -| POST | `/api/memory/find` | `memory:find` | -| POST | `/api/memory/ask` | `memory:find` | -| GET | `/api/memory/recent` | `memory:find` | +await plane.add({ + tenantId: TENANT, + principalId: PRINCIPAL, + content: { title: "Kickoff", text: "Ship memory 0→1 docs" }, +}); -Clients never send tenant/principal in the body. +const hits = await plane.find({ + tenantId: TENANT, + principalId: PRINCIPAL, + query: "memory docs", +}); +console.log("find hits:", hits.items.length); -### Host must set principal on `/api/memory/*` +const answer = await plane.ask({ + tenantId: TENANT, + principalId: PRINCIPAL, + query: "what should we ship?", +}); +console.log("ask:", answer.answer); -These routes sit outside `/api/tenants/:tenantId/*`. Mount middleware **before** -`mountMemory` that sets `c.set("principal", …)` and `c.set("tenant", …)`. -Without it: **401 `principal_required`**. +// 6. HTTP +export default { + port: 8787, + fetch: app.fetch, +}; -### Plane without HTTP +// Or: bun.serve({ port: 8787, fetch: app.fetch }) +console.log("listening on http://127.0.0.1:8787"); +``` -One options bag — never `createMemory(undefined, …)`. +Try the HTTP routes (principal is fixed by middleware above): -```ts -import { - createMemory, - createFakeDocumentStore, - createFakeMemoryProvider, -} from "@corbits/memory"; -import type { GrantStore, ConditionRegistry } from "@intx/authz"; +```bash +curl -sS -X POST http://127.0.0.1:8787/api/memory/add \ + -H 'content-type: application/json' \ + -d '{"content":{"title":"Note","text":"hello from curl"}}' -const grants = { - grantStore: hostGrantStore as GrantStore, - conditionRegistry: hostConditionRegistry as ConditionRegistry, -}; +curl -sS -X POST http://127.0.0.1:8787/api/memory/find \ + -H 'content-type: application/json' \ + -d '{"query":"hello"}' -const memory = createMemory({ - grants, - documentStore: createFakeDocumentStore(), - memoryProvider: createFakeMemoryProvider(), - generate: async (messages) => "…", // wire your inference layer for ask() -}); +curl -sS -X POST http://127.0.0.1:8787/api/memory/ask \ + -H 'content-type: application/json' \ + -d '{"query":"what do we know?"}' -await memory.add({ - tenantId, - principalId, - content: { title: "Note", text: "…" }, -}); -const hits = await memory.find({ tenantId, principalId, query: "…" }); -const answer = await memory.ask({ - tenantId, - principalId, - query: "…", - includeMemory: false, // default -}); +curl -sS 'http://127.0.0.1:8787/api/memory/recent' ``` -With the default Postgres store: +Clients never send tenant/principal in the body. Without principal middleware: +**401 `principal_required`**. Without the grant: **403**. + +| Method | Path | Grant (`requireGrant`) | +| --- | --- | --- | +| POST | `/api/memory/add` | `("memory", "add")` | +| POST | `/api/memory/find` | `("memory", "find")` | +| POST | `/api/memory/ask` | `("memory", "find")` | +| GET | `/api/memory/recent` | `("memory", "find")` | + +## Production host (Postgres + real Interchange) + +Same `mountMemory` call. Replace fakes and the demo grant store with the hub’s +real wiring: ```ts -const memory = createMemory({ - config: loadMemoryConfig(), - grants, +import { mountMemory, loadMemoryConfig } from "@corbits/memory"; + +// Same grantStore + conditionRegistry you already pass to createApp / +// createRequireGrant from @intx/hub-api. +mountMemory(app, { + config: loadMemoryConfig(), // needs KNOWLEDGE_DATABASE_URL + embed env + grants: { + grantStore: hubGrantStore, + conditionRegistry: hubConditionRegistry, + }, + // generate: wire to @intx/inference (or your own) for ask() }); ``` +Env / migrations for the default pgvector store: + +```bash +# KNOWLEDGE_DATABASE_URL required — no DATABASE_URL fallback +export KNOWLEDGE_DATABASE_URL=postgres://… +bun run db:setup +# or: runMemoryMigrations(process.env.KNOWLEDGE_DATABASE_URL) +``` + +Tables live under Postgres schema **`knowledge`** +(`knowledge.document`, `knowledge.version`, `knowledge.chunk`, …). Hard cutover +pre-1.0: re-run migrations on a fresh knowledge DB. + +Mount middleware **before** `mountMemory` that sets `c.set("principal", …)` and +`c.set("tenant", …)` for `/api/memory/*` (session auth is still the host’s job). + ## Ports | Port | Default | Override | @@ -120,7 +240,7 @@ is mounted; failure → `memory_unavailable`, docs-only. `plane.remember` / ### Optional DocumentStore adapters (sibling packages) Optional backends are **sibling packages**, not vendored here. Core never imports -vendor SDKs. Install each from git; each README only documents itself + this core. +vendor SDKs. Install each from git; each package README is a full 0→1 example. | Package | Role | | --- | --- | @@ -130,8 +250,6 @@ vendor SDKs. Install each from git; each README only documents itself + this cor ```ts import { createMem0DocumentStore } from "@corbits/mem0-memory-adapter"; -// or: createSupermemoryDocumentStore from @corbits/supermemory-memory-adapter -// or: createLinearSourceProvider from @corbits/linear-tools const memory = createMemory({ documentStore: createMem0DocumentStore({ apiKey: process.env.MEM0_API_KEY! }), @@ -139,21 +257,9 @@ const memory = createMemory({ }); ``` -## Migrations - -```bash -# KNOWLEDGE_DATABASE_URL required — no DATABASE_URL fallback -bun run db:setup -# or: runMemoryMigrations(process.env.KNOWLEDGE_DATABASE_URL) -``` - -All tables live under Postgres schema **`knowledge`** -(`knowledge.document`, `knowledge.version`, `knowledge.chunk`, …). Hard cutover -pre-1.0: re-run migrations on a fresh knowledge DB. - ## Document access (grant tags) -1. Host capability grants (`memory:add` / `memory:find`) +1. Host capability grants (`memory` + `add` / `find`) 2. Per-document **access tags** + creator rule (Interchange `@intx/authz`) 3. Share sugars on `add` only mint tags — no visibility modes or block lists From 40576f13155377b3283deb6f30b73e0e156fc65e Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 20:55:39 -0700 Subject: [PATCH 08/11] Hoist grantStore and conditionRegistry onto createMemory options Drop the nested grants bag so hosts pass the same top-level fields Interchange hubs already have. resolveGrantConfig remains for route composition. Docs and tests updated; typecheck + src tests green. --- AGENTS.md | 3 +- ARCHITECTURE.md | 6 +- CHANGELOG.md | 4 +- IMPLEMENTATION.md | 18 ++-- MIGRATION.md | 6 +- PRODUCT.md | 10 +- README.md | 99 ++++++++++---------- src/index.ts | 161 ++++++++++++++------------------- src/memory.test.ts | 38 ++++---- src/memory.ts | 51 ++++++++--- src/mount-config.ts | 2 +- src/ports/memory-plane.test.ts | 18 ++-- src/ports/merge-plane.test.ts | 6 +- src/ports/mount-fakes.test.ts | 16 ++-- src/routes/deps.ts | 2 +- src/routes/mount.ts | 7 +- src/routes/routes.test.ts | 8 +- src/services/search.ts | 2 +- 18 files changed, 235 insertions(+), 222 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3df9319..9562ac5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,7 +18,8 @@ CI runs `typecheck` + `test` — both must pass before any push. ## Layout -- `src/index.ts` — public surface: `mountMemory`, `mountMemoryRoutes`, `createMemory` +- `src/index.ts` — public surface: `createMemory` (optional `app` registers HTTP), `registerMemoryRoutes` + - `src/mount-config.ts` / `src/config.ts` — mount config + engine config - `src/routes/` — Hono routes (`add`, `find`, `ask`, `recent`) - `src/services/` — capture / search / transform internals (not public verbs) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 465c943..d5b60a8 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -17,7 +17,8 @@ to be cleanly detachable, and then cleanly *mountable*: a grant engine. So the library needs nothing but a pgvector Postgres and an embed/rerank -endpoint. It ships as `mountMemory(app, opts)`: the host passes its +endpoint. It ships as `createMemory(opts)` — pass `app` to register HTTP. The host passes its + Hono app and its grant store; the library mounts its routes, reads identity from the request context, and talks to its own vector store. No second server, no HTTP hop. @@ -64,7 +65,8 @@ trust model. ## Mounted surface -`mountMemory` adds, under the host app: +`createMemory({ app })` adds, under the host app: + - `POST /api/memory/add` — ingest a note (raw + derive). - `POST /api/memory/find` — hybrid retrieval: FTS + dense (pgvector) → RRF diff --git a/CHANGELOG.md b/CHANGELOG.md index 2791b78..451bacd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - **Breaking:** package and public surface renamed from `@corbits/knowledge-engine` - to `@corbits/memory`. Public APIs: `mountMemory`, `mountMemoryRoutes`, + to `@corbits/memory`. Public APIs: `createMemory` (optional `app` registers HTTP), + `registerMemoryRoutes`. + `createMemory`, `loadMemoryConfig`, `runMemoryMigrations`, `Memory`, `MemoryConfig`, `MemoryError`. HTTP paths are under `/api/memory/`; grants are `memory:add` / `memory:find`; access tags use `memory.owner:` / `memory.tenant:` diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 5c4c78e..bc59a68 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -8,7 +8,8 @@ and wire shapes. For the "why standalone" / boundaries story, read ``` src/ - index.ts # mountMemory / mountMemoryRoutes + index.ts # createMemory / registerMemoryRoutes + mount-config.ts # MemoryConfig + loadMemoryConfig() — the mount config config.ts # EngineConfig — the core vector-plane config (db + embed + rerank) memory.ts # createMemory — add/find/ask/recent against store or pgvector @@ -18,7 +19,8 @@ src/ migrations.ts # runMemoryMigrations(url) ports/ # DocumentStore / SourceProvider / MemoryProvider + fakes routes/ # the mounted routes - mount.ts # mountMemoryRoutes (HTTP) + mount.ts # registerMemoryRoutes (HTTP) + deps.ts # RouteDeps, caller(c) (context identity), grantGuard add.ts, find.ts, ask.ts, recent.ts db/ @@ -40,7 +42,8 @@ compose.yml # pgvector + Ollama + reranker for local dev ``` -The SDK has no server and no process entrypoint. `mountMemory` takes +The SDK has no server and no process entrypoint. `createMemory` takes + the host's `Hono` app plus `{ config, grants? }` and mounts the routes; each reads identity from the context (`caller(c)`) and guards via `grantGuard`. Services take `{ db, sql, config }` explicitly (no module-level @@ -60,7 +63,8 @@ There are two config types, both in the SDK: - **`EngineConfig`** (`src/config.ts`) — the core vector-plane config the DB client and capture/search/transform services consume: `databaseUrl`, `dbPoolMax`, `embed`, `rerank`. -- **`MemoryConfig`** (`src/mount-config.ts`) — what `mountMemory` +- **`MemoryConfig`** (`src/mount-config.ts`) — what `createMemory` + takes: just `{ memory: EngineConfig }`. `loadMemoryConfig()` builds one from the environment; hosts may also construct it programmatically. Auth, tenancy, and grants are the host's — none of that is config here. @@ -472,7 +476,8 @@ so knowing it will flip the tenant's live dense channel too. ## Mounted routes -`mountMemory` mounts these onto the host app. Identity is the request +`createMemory({ app })` registers these onto the host app. Identity is the request + principal read off the Interchange context (`caller(c)` → `{ scopeId: principal.tenantId, subjectId: principal.id }`); clients never send `tenant_id`/`principal_id` — the handlers only read title/text/query/limit/access_tags/share. @@ -486,7 +491,8 @@ Each route is guarded with `grantGuard(deps, action)`, which applies the host's | `POST /api/memory/ask` | `find` | `{ query, limit? }` (1–50) | `200 { text, citations[], evidence }`; `403` / `501` as plane errors | | `GET /api/memory/recent` | `find` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope, filtered with grant-tag access (`canAccessDocument`). One event per document (active live version). | -`mountMemoryRoutes` and `mountMemory` mount the four HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). +`registerMemoryRoutes` and `createMemory({ app })` register the four HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). + ### Timeline wire fields (vs the old CaptureLog ring) diff --git a/MIGRATION.md b/MIGRATION.md index 02b196f..8dc97a9 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -75,8 +75,10 @@ In-process `ask()` also checks `memory` / `find` (was `knowledge` / `search`). | Was | Now | | --- | --- | | `@corbits/knowledge-engine` | `@corbits/memory` | -| `mountKnowledgeEngine` | `mountMemory` | -| `mountKnowledgeRoutes` | `mountMemoryRoutes` | +| `mountKnowledgeEngine` | `createMemory({ app, … })` | +| `mountKnowledgeRoutes` | `registerMemoryRoutes` (or `createMemory({ app })`) | +| `mountMemory` / `mountMemoryRoutes` | `createMemory({ app })` / `registerMemoryRoutes` | + | `createKnowledgePlane` | `createMemory` | | `loadKnowledgeConfig` | `loadMemoryConfig` | | `runKnowledgeMigrations` | `runMemoryMigrations` | diff --git a/PRODUCT.md b/PRODUCT.md index 86b6b9f..913a1cb 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -11,10 +11,11 @@ SDK never creates one; it mounts onto yours. | Surface | Role | | --- | --- | -| `mountMemory(app, opts)` | Plane + HTTP on an Interchange `createApp` | -| `createMemory(options?: MemoryOptions)` | Same plane without HTTP | +| `createMemory(opts)` | Plane; pass `app` to register HTTP on an Interchange host | | `runMemoryMigrations(url)` | Apply pgvector schema under Postgres `knowledge` | -| `loadMemoryConfig()` | Mount config from env | +| `loadMemoryConfig()` | Config from env | +| `registerMemoryRoutes` | Optional low-level HTTP registration | + ### Green public plane (only these verbs) @@ -96,7 +97,8 @@ Claude Code / Codex / Workbench (clients) ▼ ┌──────────────────────────────────────────────┐ │ Host Interchange createApp │ -│ + mountMemory(app, opts) │ +│ + createMemory({ app, … }) │ + │ grants: memory:add | memory:find │ │ documentStore: pgvector | host store │ │ | fake │ diff --git a/README.md b/README.md index ff64761..d87f44d 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,13 @@ # @corbits/memory -Mountable memory plane for [Interchange](https://github.com/corbitsdev) hubs: +Memory plane for [Interchange](https://github.com/corbitsdev) hubs: **add** documents, **find** with hybrid search, **ask** grounded answers, **recent** timeline — with optional live sources and personal memory. +**One entry point:** `createMemory(options)`. Pass `app` to register +`/api/memory/*` on your Hono host. Without `app`, you get an in-process plane +only (CLI, worker, tests). + **Authenticates nothing.** Identity is `c.get("principal")` on HTTP; in-process callers pass `principalId` + `tenantId`. Authorization is the host grant store (`memory` resource + `add` / `find` actions via `@intx/authz`). Never embeds @@ -39,7 +43,6 @@ import { createInMemoryGrantStore, type GrantRule } from "@intx/authz"; import { createFakeDocumentStore, createMemory, - mountMemory, } from "@corbits/memory"; const TENANT = "tenant_demo"; @@ -72,10 +75,9 @@ const grantRules: GrantRule[] = [ }, ]; -const grants = { - grantStore: createInMemoryGrantStore(grantRules), - conditionRegistry: {}, // empty registry is fine when grants have no conditions -}; +const grantStore = createInMemoryGrantStore(grantRules); +// Empty condition registry is fine when grants have no conditions. +const conditionRegistry = {}; // 2. Durable store. Fakes prove the port boundary; swap for Postgres or a // sibling DocumentStore adapter later. @@ -109,9 +111,11 @@ app.use("/api/memory/*", async (c, next) => { await next(); }); -// 4. Mount HTTP plane. Returns the same Memory object for in-process use. -const { memory } = mountMemory(app, { - grants, +// 4. One call: plane + HTTP routes. Returns Memory for in-process use too. +const memory = createMemory({ + app, + grantStore, + conditionRegistry, documentStore, // optional: generate for ask(); textExtractor for add({ file }) generate: async (messages) => { @@ -120,28 +124,21 @@ const { memory } = mountMemory(app, { }, }); -// 5. In-process path (CLI, worker, tests) — same plane, no HTTP. -// One options bag only. Never createMemory(undefined, …). -const plane = createMemory({ - grants, - documentStore, - generate: async () => "in-process answer", -}); - -await plane.add({ +// 5. In-process path (CLI, worker, tests) — same plane, no second factory. +await memory.add({ tenantId: TENANT, principalId: PRINCIPAL, content: { title: "Kickoff", text: "Ship memory 0→1 docs" }, }); -const hits = await plane.find({ +const hits = await memory.find({ tenantId: TENANT, principalId: PRINCIPAL, query: "memory docs", }); console.log("find hits:", hits.items.length); -const answer = await plane.ask({ +const answer = await memory.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "what should we ship?", @@ -154,7 +151,6 @@ export default { fetch: app.fetch, }; -// Or: bun.serve({ port: 8787, fetch: app.fetch }) console.log("listening on http://127.0.0.1:8787"); ``` @@ -163,7 +159,8 @@ Try the HTTP routes (principal is fixed by middleware above): ```bash curl -sS -X POST http://127.0.0.1:8787/api/memory/add \ -H 'content-type: application/json' \ - -d '{"content":{"title":"Note","text":"hello from curl"}}' + -d '{"title":"Note","text":"hello from curl"}' + curl -sS -X POST http://127.0.0.1:8787/api/memory/find \ -H 'content-type: application/json' \ @@ -188,20 +185,19 @@ Clients never send tenant/principal in the body. Without principal middleware: ## Production host (Postgres + real Interchange) -Same `mountMemory` call. Replace fakes and the demo grant store with the hub’s -real wiring: +Same `createMemory` call. Replace fakes and the demo grant store with the hub’s +real wiring. Register principal/tenant middleware **before** `createMemory({ app })`. ```ts -import { mountMemory, loadMemoryConfig } from "@corbits/memory"; +import { createMemory, loadMemoryConfig } from "@corbits/memory"; // Same grantStore + conditionRegistry you already pass to createApp / // createRequireGrant from @intx/hub-api. -mountMemory(app, { +const memory = createMemory({ + app, config: loadMemoryConfig(), // needs KNOWLEDGE_DATABASE_URL + embed env - grants: { - grantStore: hubGrantStore, - conditionRegistry: hubConditionRegistry, - }, + grantStore: hubGrantStore, + conditionRegistry: hubConditionRegistry, // generate: wire to @intx/inference (or your own) for ask() }); ``` @@ -219,9 +215,6 @@ Tables live under Postgres schema **`knowledge`** (`knowledge.document`, `knowledge.version`, `knowledge.chunk`, …). Hard cutover pre-1.0: re-run migrations on a fresh knowledge DB. -Mount middleware **before** `mountMemory` that sets `c.set("principal", …)` and -`c.set("tenant", …)` for `/api/memory/*` (session auth is still the host’s job). - ## Ports | Port | Default | Override | @@ -234,8 +227,8 @@ Mount middleware **before** `mountMemory` that sets `c.set("principal", …)` an `live_error` degrade; dedupe `adapter:externalRef`; optional `sources` filter. **Memory side-channel:** `ask({ includeMemory: true })` recalls when a provider -is mounted; failure → `memory_unavailable`, docs-only. `plane.remember` / -`plane.recall` for host-owned writes (ask never auto-remembers). +is set; failure → `memory_unavailable`, docs-only. `memory.remember` / +`memory.recall` for host-owned writes (ask never auto-remembers). ### Optional DocumentStore adapters (sibling packages) @@ -253,7 +246,8 @@ import { createMem0DocumentStore } from "@corbits/mem0-memory-adapter"; const memory = createMemory({ documentStore: createMem0DocumentStore({ apiKey: process.env.MEM0_API_KEY! }), - grants, + grantStore, + conditionRegistry, }); ``` @@ -261,23 +255,30 @@ const memory = createMemory({ 1. Host capability grants (`memory` + `add` / `find`) 2. Per-document **access tags** + creator rule (Interchange `@intx/authz`) -3. Share sugars on `add` only mint tags — no visibility modes or block lists +3. Share mints tags only — no visibility modes or block lists -Full design: `docs/AUTHZ-DOCUMENT-ACCESS.md`. +See [docs/AUTHZ-DOCUMENT-ACCESS.md](./docs/AUTHZ-DOCUMENT-ACCESS.md). -## Docs +## Compose routes yourself -- `PRODUCT.md` — product shape and out-of-scope -- `ARCHITECTURE.md` — design decisions -- `IMPLEMENTATION.md` — env vars, data model, services -- `MIGRATION.md` — hard cutover from capture/search/timeline +Most hosts use `createMemory({ app, … })`. For custom route composition: -## Develop +```ts +import { createMemory, registerMemoryRoutes, resolveGrantConfig } from "@corbits/memory"; +import { createRequireGrant } from "@intx/hub-api"; -```bash -bun install -bun run typecheck -bun run test +const grantStore = hubGrantStore; +const conditionRegistry = hubConditionRegistry; +const grants = resolveGrantConfig({ grantStore, conditionRegistry })!; + +const memory = createMemory({ grantStore, conditionRegistry, documentStore }); // no app +registerMemoryRoutes(app, { + memory, + grants, + requireGrant: createRequireGrant(grants), +}); ``` -License: LGPL-2.1 (`LICENSE`). Contributions: `CLA.md`. +## License + +LGPL-2.1-only diff --git a/src/index.ts b/src/index.ts index bcde12c..c0db7de 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,30 +1,22 @@ /** - * @corbits/memory — a memory add / find / ask / recent engine you - * mount onto an Interchange hub. + * @corbits/memory — add / find / ask / recent for Interchange hubs. * - * The host owns auth, tenancy, grants, and the process. This SDK reads the - * request principal off the Interchange context and (optionally) taps the - * host's grant middleware; it authenticates nothing itself. + * One entry: `createMemory(options)`. Pass `app` to register HTTP routes on + * a Hono host. Identity is `c.get("principal")` on HTTP, or `principalId` + + * `tenantId` in-process. Authz is the host grant store — this package + * authenticates nothing itself. */ import type { Hono } from "hono"; import { createRequireGrant, type TenantEnv } from "@intx/hub-api"; -import type { MemoryConfig } from "./mount-config.ts"; import { - createMemory, - type Generate, + createMemory as createMemoryPlane, + resolveGrantConfig, type Memory, type MemoryOptions, - type TextExtractor, } from "./memory.ts"; -import type { - DocumentStore, - MemoryProvider, - SourceProvider, -} from "./ports/types.ts"; import { - mountMemoryRoutes, - type GrantConfig, + registerMemoryRoutes, type RouteDeps, } from "./routes/mount.ts"; @@ -33,16 +25,9 @@ export type { MemoryConfig } from "./mount-config.ts"; export { loadMemoryConfig } from "./mount-config.ts"; export type { EngineConfig } from "./config.ts"; export { RerankConfigError } from "./core/rerank-client.ts"; -// Memory plane -// -// `createMemory` is exported so a host can add or find outside a -// request — a CLI seeder, a batch ingester, or a test — without standing up a -// Hono app just to get a plane. Callers acting on behalf of a user are -// responsible for the capability check `requireGrant` would have performed; see -// the README. Rerank config is validated at construction (same as mount). -// Pass `grants` + optional `generate` when the host will call `ask()`. -// One options bag — never createMemory(undefined, …). -export { createMemory } from "./memory.ts"; + +// Memory plane — types from memory.ts; createMemory is defined below so it +// can optionally register HTTP routes when `app` is passed. export type { AskCitation, AskResult, @@ -67,7 +52,12 @@ export type { TextExtractor, TimelineEvent, } from "./memory.ts"; -export { MemoryError, MemoryNotPermittedError } from "./memory.ts"; +export { + MemoryError, + MemoryNotPermittedError, + resolveGrantConfig, +} from "./memory.ts"; + // Ports — pluggable storage, live sources, and optional memory export type { DocumentStore, @@ -87,8 +77,10 @@ export { createFakeMemoryProvider, createFakeSourceProvider, } from "./ports/fakes.ts"; + // Migrations export { runMemoryMigrations } from "./migrations.ts"; + // Degrade metrics — no metrics dependency exists in this package (see // core/degrade-metrics.ts); a host with its own metrics backend polls this // snapshot and forwards it, rather than the engine owning a /metrics port. @@ -105,75 +97,58 @@ export { parseFtsLanguage, verifyFtsLanguage, } from "./core/fts-language.ts"; -// Granular mount (compose your own) -export { mountMemoryRoutes, type GrantConfig } from "./routes/mount.ts"; -export type MountMemoryOptions = { - /** - * Engine config (DB + model endpoints). Optional when `documentStore` is - * provided — a host can mount with fakes only. - */ - config?: MemoryConfig; - /** - * The host's grant store + condition registry — the same pair it passes to - * `createApp`/`createRequireGrant`. Required: HTTP routes are guarded with - * `requireGrant("memory", )`. The SDK never leaves a route - * unguarded. Also required for in-process `ask()`. - */ - grants: GrantConfig; - /** - * How `ask()` reaches a model. Omit if this host only adds and finds; - * `ask()` then fails with a 501 naming what is missing. - * - * The engine owns no generation client on purpose — Interchange's - * `@intx/inference` already has provider adapters, tenant-scoped credentials, - * retry, audit and authz gates. Wire this to that rather than to a bare fetch. - */ - generate?: Generate; - /** Required for `add({ file })` via HTTP or plane. */ - textExtractor?: TextExtractor; - /** Override durable storage (default: engine pgvector store). */ - documentStore?: DocumentStore; - /** Live source connectors merged into find/ask (fail-soft). */ - sources?: SourceProvider[]; +// Granular HTTP composition (most hosts use createMemory({ app, … }) instead) +export { registerMemoryRoutes, type GrantConfig } from "./routes/mount.ts"; + +export type CreateMemoryOptions = MemoryOptions & { /** - * Optional ask side-channel only (`includeMemory`). Not a DocumentStore - * replacement — vendor backends mount as `documentStore`. + * When set, register `/api/memory/*` on this Hono app. Requires `grantStore` + * (routes are guarded with `requireGrant("memory", …)`). */ - memoryProvider?: MemoryProvider; + app?: Hono; }; -export type MountedMemory = { - memory: Memory; -}; - -/** Mount the memory HTTP routes over one memory plane. */ -export function mountMemory( - app: Hono, - options: MountMemoryOptions, -): MountedMemory { - // Rerank config validation runs inside createMemory so standalone - // construction and the mount path share one check. Pass grants + generate so - // the returned plane's ask() is grant-checked and can synthesize answers. - const planeOptions: MemoryOptions = { - ...(options.generate ? { generate: options.generate } : {}), - ...(options.textExtractor ? { textExtractor: options.textExtractor } : {}), - ...(options.documentStore ? { documentStore: options.documentStore } : {}), - ...(options.sources ? { sources: options.sources } : {}), - ...(options.memoryProvider - ? { memoryProvider: options.memoryProvider } - : {}), - }; - const memory = createMemory({ - ...(options.config ? { config: options.config } : {}), - grants: options.grants, - ...planeOptions, - }); - const deps: RouteDeps = { - memory, - grants: options.grants, - requireGrant: createRequireGrant(options.grants), - }; - mountMemoryRoutes(app, deps); - return { memory }; +/** + * Build a memory plane. Optionally register HTTP routes when `app` is set. + * + * @example In-process only + * ```ts + * const memory = createMemory({ + * documentStore: createFakeDocumentStore(), + * grantStore, + * conditionRegistry, + * }); + * await memory.add({ tenantId, principalId, content: { title, text } }); + * ``` + * + * @example With HTTP + * ```ts + * const memory = createMemory({ + * app, + * grantStore, + * conditionRegistry, + * config: loadMemoryConfig(), + * generate, + * }); + * ``` + */ +export function createMemory(options: CreateMemoryOptions = {}): Memory { + const { app, ...planeOptions } = options; + const memory = createMemoryPlane(planeOptions); + if (app) { + const grants = resolveGrantConfig(options); + if (!grants) { + throw new Error( + "createMemory({ app }): grantStore is required to register HTTP routes", + ); + } + const deps: RouteDeps = { + memory, + grants, + requireGrant: createRequireGrant(grants), + }; + registerMemoryRoutes(app, deps); + } + return memory; } diff --git a/src/memory.test.ts b/src/memory.test.ts index 937ffb4..6ae9a1c 100644 --- a/src/memory.test.ts +++ b/src/memory.test.ts @@ -172,7 +172,7 @@ function baseConfig( describe("createMemory — construction validation", () => { it("throws RerankConfigError when maxDocChars overflows a known TEI model", () => { // Proves validateRerankConfig runs inside createMemory (not only - // mountMemory): a standalone plane with a bad override must fail + // createMemory): a standalone plane with a bad override must fail // construction, not silently degrade on every later find. expect(() => createMemory({ @@ -734,23 +734,24 @@ async function freshPlane(opts?: { describe("ask() — grant check", () => { it("denies with MemoryNotPermittedError when no grant matches (effect: null)", async () => { - const grants = { - grantStore: createInMemoryGrantStore([]), + const grantStore = createInMemoryGrantStore([]); + const plane = createMemory({ + config: askConfig, + grantStore, conditionRegistry: {}, - }; - const plane = createMemory({ config: askConfig, grants }); + }); await expect( plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), ).rejects.toBeInstanceOf(MemoryNotPermittedError); }); it("denies when the only matching grant is an explicit deny", async () => { -const denyGrant: GrantRule = { ...grant("find"), effect: "deny" }; - const grants = { + const denyGrant: GrantRule = { ...grant("find"), effect: "deny" }; + const plane = createMemory({ + config: askConfig, grantStore: createInMemoryGrantStore([denyGrant]), conditionRegistry: {}, - }; - const plane = createMemory({ config: askConfig, grants }); + }); await expect( plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), ).rejects.toBeInstanceOf(MemoryNotPermittedError); @@ -761,11 +762,11 @@ describe("ask() — missing generate", () => { it("throws MemoryError 501 before find when generate is not wired", async () => { // Pointed at a nonexistent DB: if find ran first this would surface a // connection/driver error instead of the promised 501. - const grants = { -grantStore: createInMemoryGrantStore([grant("find")]), + const plane = createMemory({ + config: askConfig, + grantStore: createInMemoryGrantStore([grant("find")]), conditionRegistry: {}, - }; - const plane = createMemory({ config: askConfig, grants }); + }); try { await plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }); throw new Error("expected ask() to reject"); @@ -779,10 +780,6 @@ grantStore: createInMemoryGrantStore([grant("find")]), describe("ask() — allow path", () => { it("finds as the principal and synthesizes when grant allows and generate is wired", async () => { - const grants = { - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - }; const generate = mock((messages: readonly ChatMessage[]) => { expect(messages[0]?.role).toBe("system"); expect(messages[1]?.content).toContain("what is the answer?"); @@ -790,7 +787,12 @@ describe("ask() — allow path", () => { expect(messages[1]?.content).toContain("the relevant snippet"); return Promise.resolve("Answer from context [1]."); }); - const plane = createMemory({ config: askConfig, grants, generate }); + const plane = createMemory({ + config: askConfig, + grantStore: createInMemoryGrantStore([grant("find")]), + conditionRegistry: {}, + generate, + }); // Stub find so this unit test never needs a live Postgres. ask() looks // up plane.find at call time, so reassignment is the wiring under test. plane.find = mock(() => diff --git a/src/memory.ts b/src/memory.ts index e6541de..839efff 100644 --- a/src/memory.ts +++ b/src/memory.ts @@ -377,10 +377,17 @@ export type MemoryOptions = { /** Engine config (DB + model endpoints). Required when `documentStore` is omitted. */ config?: MemoryConfig; /** - * Host grant store + condition registry. Required for `ask()` (in-process - * capability check). Standalone add/find callers may omit it. + * Host Interchange `GrantStore`. Required for `ask()` (in-process capability + * check) and for document access filtering on find/recent. Standalone + * add-only callers may omit it. */ - grants?: GrantConfig; + grantStore?: GrantConfig["grantStore"]; + /** + * Host Interchange condition evaluators keyed by name. Optional — default + * `{}` when `grantStore` is set. Same object the hub passes to + * `createRequireGrant` / `createApp`. + */ + conditionRegistry?: GrantConfig["conditionRegistry"]; /** Required for `ask()`; omit if the host only adds and finds. */ generate?: Generate; /** Required for `add({ file })`; omit if the host only adds text content. */ @@ -403,6 +410,17 @@ export type MemoryOptions = { memoryProvider?: MemoryProvider; }; +/** Bundle host authz pieces for route guards and document access. */ +export function resolveGrantConfig( + options: Pick, +): GrantConfig | undefined { + if (!options.grantStore) return undefined; + return { + grantStore: options.grantStore, + conditionRegistry: options.conditionRegistry ?? {}, + }; +} + function resolveFindLimit(limit: number | undefined): number { if (limit === undefined) return DEFAULT_HYBRID_TOP_K; if ( @@ -484,8 +502,9 @@ function resolveAddAccessTags(params: MemoryAddParams): string[] { * is omitted, the default pgvector engine is wrapped as that store. Hosts * inject a DocumentStore or fakes the same way — no second plane implementation. * - * - `options.grants` is required for `ask()` (in-process capability check). - * Standalone add/find callers may omit it. + * - `options.grantStore` is required for `ask()` (in-process capability check). + * Standalone add/find callers may omit it. `conditionRegistry` is optional + * (defaults to `{}`). * - Rerank config is validated at construction when using the default store. * - Pass `options.sources` for live SourceProviders; find/ask merge via * MergeLocalLiveV1 (fail-soft, 800ms timeout, prefer-local dedupe). @@ -495,7 +514,8 @@ function resolveAddAccessTags(params: MemoryAddParams): string[] { * ```ts * const memory = createMemory({ * config: loadMemoryConfig(), - * grants: { grantStore, conditionRegistry }, + * grantStore, + * conditionRegistry, * }); * ``` * @@ -503,20 +523,25 @@ function resolveAddAccessTags(params: MemoryAddParams): string[] { * ```ts * const memory = createMemory({ * documentStore: myStore, - * grants: { grantStore, conditionRegistry }, + * grantStore, * }); * ``` */ export function createMemory(options: MemoryOptions = {}): Memory { const { config, - grants, + grantStore, + conditionRegistry, documentStore, generate, textExtractor, sources, memoryProvider, } = options; + const grants = resolveGrantConfig({ + ...(grantStore !== undefined ? { grantStore } : {}), + ...(conditionRegistry !== undefined ? { conditionRegistry } : {}), + }); const store = documentStore ?? (() => { @@ -731,7 +756,7 @@ function makeRememberRecall(options: MemoryOptions): { throw new MemoryError( 501, "remember() requires a MemoryProvider. Pass memoryProvider to " + - "createMemory/mountMemory.", + "createMemory.", ); } await options.memoryProvider.remember({ @@ -852,8 +877,8 @@ function createPlaneFromStore( if (!grants) { throw new MemoryError( 501, - "ask() requires a GrantConfig. Pass grants to " + - "createMemory/mountMemory.", + "ask() requires a grantStore. Pass grantStore to " + + "createMemory({ grantStore, … }) (conditionRegistry optional).", ); } const decision = await authorize( @@ -883,7 +908,7 @@ function createPlaneFromStore( throw new MemoryError( 501, "ask() requires a `generate` function. Pass one to " + - "createMemory/mountMemory, wired to your " + + "createMemory, wired to your " + "inference layer.", ); } @@ -1015,7 +1040,7 @@ function createEngineDocumentStore(config: MemoryConfig): DocumentStore { // (`defaultMaxDocCharsForModel`) is self-consistent by construction — // validation can only fire on an operator's own `maxDocChars` override, // never spuriously on an unmodified config. - // Lives here (not only in mountMemory) so standalone construction + // Lives here (not only in createMemory with app) so standalone construction // cannot silently degrade on a bad override. const rerankConfig = toRerankClientConfig(config.memory.rerank); if (rerankConfig) validateRerankConfig(rerankConfig); diff --git a/src/mount-config.ts b/src/mount-config.ts index 9eb72f8..8ee7d17 100644 --- a/src/mount-config.ts +++ b/src/mount-config.ts @@ -2,7 +2,7 @@ import type { EngineConfig } from "./config.ts"; import { parseFtsLanguage } from "./core/fts-language.ts"; /** - * SDK mount config — what `mountMemory` consumes. + * SDK config — what `createMemory` consumes. * * The host Interchange app owns auth, tenancy, grants, and the process. This * config carries only what the memory engine itself needs: its vector DB + diff --git a/src/ports/memory-plane.test.ts b/src/ports/memory-plane.test.ts index 1084f2f..7daa78d 100644 --- a/src/ports/memory-plane.test.ts +++ b/src/ports/memory-plane.test.ts @@ -45,10 +45,8 @@ describe("MemoryProvider product wire (CL-5228)", () => { }; const store = createFakeDocumentStore(); const plane = createMemory({ - grants: { - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - }, + grantStore: createInMemoryGrantStore([grant("find")]), + conditionRegistry: {}, documentStore: store, memoryProvider: memory, generate: async (msgs) => { @@ -80,10 +78,8 @@ describe("MemoryProvider product wire (CL-5228)", () => { }); const store = createFakeDocumentStore(); const plane = createMemory({ - grants: { - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - }, + grantStore: createInMemoryGrantStore([grant("find")]), + conditionRegistry: {}, documentStore: store, memoryProvider: memory, generate: async (msgs) => { @@ -117,10 +113,8 @@ describe("MemoryProvider product wire (CL-5228)", () => { }; const store = createFakeDocumentStore(); const plane = createMemory({ - grants: { - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - }, + grantStore: createInMemoryGrantStore([grant("find")]), + conditionRegistry: {}, documentStore: store, memoryProvider: memory, generate: async () => "docs-only [1]", diff --git a/src/ports/merge-plane.test.ts b/src/ports/merge-plane.test.ts index 33eb7dc..d71724b 100644 --- a/src/ports/merge-plane.test.ts +++ b/src/ports/merge-plane.test.ts @@ -232,10 +232,8 @@ describe("plane merge (MergeLocalLiveV1)", () => { it("ask still works when live source errors", async () => { const store = createFakeDocumentStore(); const plane = createMemory({ - grants: { - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - }, + grantStore: createInMemoryGrantStore([grant("find")]), + conditionRegistry: {}, documentStore: store, sources: [ { diff --git a/src/ports/mount-fakes.test.ts b/src/ports/mount-fakes.test.ts index fe40e78..8a24f20 100644 --- a/src/ports/mount-fakes.test.ts +++ b/src/ports/mount-fakes.test.ts @@ -1,5 +1,5 @@ /** - * Acceptance: a host can mount with only fakes and get working + * Acceptance: a host can createMemory with only fakes and get working * add / find / ask / recent — proves the port boundary is real. */ import { describe, expect, it } from "bun:test"; @@ -13,7 +13,7 @@ import { import { createFakeDocumentStore, createFakeSourceProvider, - mountMemory, + createMemory, } from "../index.ts"; const TENANT = "tenant_fake"; @@ -62,7 +62,7 @@ function appWithPrincipal() { return app; } -describe("mount with fakes only", () => { +describe("createMemory with fakes only", () => { it("add → find → recent → ask without Postgres or embed config", async () => { const store = createFakeDocumentStore(); const sources = [ @@ -87,11 +87,10 @@ describe("mount with fakes only", () => { ]), ]; const app = appWithPrincipal(); - const { memory } = mountMemory(app, { - grants: { - grantStore: createInMemoryGrantStore([grant("add"), grant("find")]), - conditionRegistry: {}, - }, + const memory = createMemory({ + app, + grantStore: createInMemoryGrantStore([grant("add"), grant("find")]), + conditionRegistry: {}, documentStore: store, sources, generate: async () => "Answer from local store [1].", @@ -138,6 +137,7 @@ describe("mount with fakes only", () => { title: "via http", text: "http path uses the same store", }), + }); expect(addRes.status).toBe(200); const addBody = (await addRes.json()) as { documentId: string }; diff --git a/src/routes/deps.ts b/src/routes/deps.ts index 74acf55..ec945f8 100644 --- a/src/routes/deps.ts +++ b/src/routes/deps.ts @@ -49,7 +49,7 @@ export function caller(c: Context): { * These routes mount at `/api/memory/*`, outside the * `/api/tenants/:tenantId/*` prefix that Interchange's `createResolveTenant` * covers, so an unresolved context is the DEFAULT for a host that just calls - * `mountMemory`. See the README for the middleware the host supplies. + * `createMemory({ app })`. See the README for the middleware the host supplies. */ export function requirePrincipal(): MiddlewareHandler { return async (c, next) => { diff --git a/src/routes/mount.ts b/src/routes/mount.ts index 9472d89..4c8897e 100644 --- a/src/routes/mount.ts +++ b/src/routes/mount.ts @@ -1,6 +1,7 @@ /** - * Mount the memory engine HTTP routes onto a host Interchange app. - * (MCP moved out to the standalone @corbitsdev/hono-openapi-mcp bridge.) + * Register memory HTTP routes on a host Interchange app. + * Prefer `createMemory({ app, … })` unless you need to compose routes yourself. + * (MCP lives in the standalone @corbitsdev/hono-openapi-mcp bridge.) */ import type { Hono } from "hono"; import type { TenantEnv } from "@intx/hub-api"; @@ -14,7 +15,7 @@ import { mountRecentRoute } from "./recent.ts"; export type { GrantConfig, RouteDeps } from "./deps.ts"; /** HTTP JSON routes: add, find, ask, recent. */ -export function mountMemoryRoutes( +export function registerMemoryRoutes( app: Hono, deps: RouteDeps, ): void { diff --git a/src/routes/routes.test.ts b/src/routes/routes.test.ts index deb1f4f..d9b3cfb 100644 --- a/src/routes/routes.test.ts +++ b/src/routes/routes.test.ts @@ -5,9 +5,11 @@ import type { GrantRule } from "@intx/authz"; import { createRequireGrant, type TenantEnv } from "@intx/hub-api"; import type { Memory, TimelineEvent } from "../memory.ts"; -import { mountMemoryRoutes } from "./mount.ts"; +import { registerMemoryRoutes } from "./mount.ts"; + import type { RouteDeps } from "./deps.ts"; + function grant(principalId: string, action: string): GrantRule { return { id: `g-${principalId}-${action}`, @@ -124,7 +126,7 @@ function buildApp( }); await next(); }); - mountMemoryRoutes(app, deps); + registerMemoryRoutes(app, deps); return { app, added, searched }; } @@ -142,7 +144,7 @@ function buildAppWithoutPrincipal() { requireGrant: createRequireGrant(grantConfig), }; const app = new Hono(); - mountMemoryRoutes(app, deps); + registerMemoryRoutes(app, deps); return app; } diff --git a/src/services/search.ts b/src/services/search.ts index 737faac..4d76d9e 100644 --- a/src/services/search.ts +++ b/src/services/search.ts @@ -881,7 +881,7 @@ export async function hybridSearch( try { // A replay's transform_config can supply its own rerank endpoint/model // (resolvedTuning?.rerank, above) that never passes through - // mountMemory's startup validation — validate it here, on the + // createMemory's startup validation — validate it here, on the // same terms as the mounted config, so a replay-authored mismatch // degrades to fused ranking (caught below) instead of silently // 413-ing every rerank call for that generation. From b2b2d27e4bda7d30890f04f84a6179a862836d55 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 21:22:59 -0700 Subject: [PATCH 09/11] Cut public surface to add/search/list; drop ask and side-channel MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hard cutover for CL-5349: find→search, recent→list; remove ask, remember, recall, MemoryProvider, and generate from the product plane. HTTP grants are memory:add | memory:search. README/PRODUCT updated. --- PRODUCT.md | 34 +-- README.md | 141 +++------ package.json | 2 +- src/grant-tags.test.ts | 4 +- src/grant-tags.ts | 8 +- src/index.ts | 74 ++--- src/memory.test.ts | 242 +-------------- src/memory.ts | 473 ++++-------------------------- src/ports/fakes.test.ts | 12 +- src/ports/fakes.ts | 55 +--- src/ports/index.ts | 2 - src/ports/memory-plane.test.ts | 156 ++-------- src/ports/merge-plane.test.ts | 58 +--- src/ports/mount-fakes.test.ts | 36 +-- src/ports/types.ts | 49 +--- src/routes/ask.ts | 86 ------ src/routes/{recent.ts => list.ts} | 38 +-- src/routes/mount.ts | 12 +- src/routes/routes.test.ts | 207 +++---------- src/routes/{find.ts => search.ts} | 28 +- src/services/timeline.test.ts | 2 +- src/services/timeline.ts | 4 +- 22 files changed, 334 insertions(+), 1389 deletions(-) delete mode 100644 src/routes/ask.ts rename src/routes/{recent.ts => list.ts} (68%) rename src/routes/{find.ts => search.ts} (79%) diff --git a/PRODUCT.md b/PRODUCT.md index 913a1cb..16b3ed0 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -1,8 +1,9 @@ # Corbits Memory — Product shape A **mountable memory plane** for Interchange hubs: durable documents, hybrid -search, grounded ask, and optional live sources / personal memory side-channel. -Workbench and coding agents are clients — not owners of ingestion or auth. +search, and recent list. Workbench and coding agents are clients — not owners +of ingestion or auth. Inference is **host-owned and ephemeral** (call your +model, then `add` / `search`); core does not ship `ask` or an ingest agent. ## Shape (locked) @@ -22,14 +23,14 @@ SDK never creates one; it mounts onto yours. | Method | Meaning | | --- | --- | | `add` | Capture a document (`content` **xor** `file` + TextExtractor) | -| `find` | Hybrid retrieval (+ optional live sources) | -| `ask` | Grounded answer from find (+ optional memory) | -| `recent` | Recent documents for the principal | +| `search` | Hybrid retrieval (+ optional live sources) | +| `list` | Recent documents for the principal | -Hard cutover: there is no `capture` / `search` / `timeline` export. HTTP paths -and grants match the verbs: `POST /api/memory/add|find|ask`, -`GET /api/memory/recent`; grants `memory:add` and `memory:find` -(ask/recent share `find`). +Hard cutover from older names: `find` → `search`, `recent` → `list`. +**Removed from product:** `ask`, `remember`, `recall`, and any +`MemoryProvider` side-channel. HTTP paths and grants match the verbs: +`POST /api/memory/add|search`, `GET /api/memory/list`; grants `memory:add` and +`memory:search` (`list` shares `search`). Identity on the plane is always **`principalId` + `tenantId`** (never `scopeId` / `subjectId`). HTTP routes never take body identity — they read @@ -39,12 +40,11 @@ Identity on the plane is always **`principalId` + `tenantId`** (never | Port | Purpose | | --- | --- | -| `DocumentStore` | **The** durable backend for add/find/recent (default: engine pgvector, wrapped as a DocumentStore). Replace with any host `DocumentStore` or in-package fakes — no Postgres required when overridden. The plane is always store-backed; there is no second engine-only path. | +| `DocumentStore` | **The** durable backend for add/search/list (default: engine pgvector, wrapped as a DocumentStore). Replace with any host `DocumentStore` or in-package fakes — no Postgres required when overridden. The plane is always store-backed; there is no second engine-only path. | | `SourceProvider` | Optional **tools-shaped** live search (`searchLive`); merge is fail-soft. Not a store replacement. | -| `MemoryProvider` | Optional ask side-channel only (`includeMemory`); **not** how you swap backends. | -Mount options accept `documentStore`, `sources[]`, `memoryProvider`, plus in-package -**fakes** so a host can mount with fakes only and exercise add/find/ask/recent +Mount options accept `documentStore`, `sources[]`, plus in-package +**fakes** so a host can mount with fakes only and exercise add/search/list without Postgres. Hosts that want a third-party durable backend implement `DocumentStore` (or use an optional adapter package) and pass it as `documentStore`, omitting `MemoryConfig` when Postgres is not needed. @@ -53,11 +53,9 @@ without Postgres. Hosts that want a third-party durable backend implement per provider (timeout/error → degrade flags, never fail the request), dedupe by `adapter:externalRef`, optional `sources` filter (`local` + provider ids). -**Memory side-channel:** `includeMemory` on `ask` defaults **false**. When true -and a `MemoryProvider` is mounted, recall injects uncited personal context; -failures degrade with `memory_unavailable` (docs-only). This is unrelated to -replacing the DocumentStore. Writes via `plane.remember` are host-owned — ask -never auto-writes. +**Inference:** host-owned. Extract durable facts with the host model before +`add`, or answer with `search` + host model. No `generate` option on +`createMemory`; no auto-write on search. ### Optional adapter packages diff --git a/README.md b/README.md index d87f44d..b81f331 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,7 @@ # @corbits/memory Memory plane for [Interchange](https://github.com/corbitsdev) hubs: -**add** documents, **find** with hybrid search, **ask** grounded answers, **recent** -timeline — with optional live sources and personal memory. +**add** documents, **search** with hybrid retrieval, **list** recent events. **One entry point:** `createMemory(options)`. Pass `app` to register `/api/memory/*` on your Hono host. Without `app`, you get an in-process plane @@ -10,8 +9,11 @@ only (CLI, worker, tests). **Authenticates nothing.** Identity is `c.get("principal")` on HTTP; in-process callers pass `principalId` + `tenantId`. Authorization is the host grant store -(`memory` resource + `add` / `find` actions via `@intx/authz`). Never embeds -in-process — embedding and rerank are outbound HTTP to configured endpoints. +(`memory` resource + `add` / `search` actions via `@intx/authz`). Document access +uses grant tags on each row (`access_tags`); creator always sees their own docs. + +**No baked-in LLM.** Inference is host-owned and ephemeral: call your model, then +`add` / `search`. Core does not mount an ingest agent or require `generate`. Requires Bun 1.2+. @@ -49,7 +51,7 @@ const TENANT = "tenant_demo"; const PRINCIPAL = "principal_demo"; // 1. Capability grants the host would normally load from Interchange. -// Routes call requireGrant("memory", "add" | "find"). +// Routes call requireGrant("memory", "add" | "search"). const grantRules: GrantRule[] = [ { id: "g-add", @@ -63,10 +65,10 @@ const grantRules: GrantRule[] = [ roleId: null, }, { - id: "g-find", + id: "g-search", principalId: PRINCIPAL, resource: "memory", - action: "find", + action: "search", effect: "allow", origin: "role", conditions: null, @@ -117,11 +119,6 @@ const memory = createMemory({ grantStore, conditionRegistry, documentStore, - // optional: generate for ask(); textExtractor for add({ file }) - generate: async (messages) => { - const last = messages.at(-1)?.content ?? ""; - return `demo answer for: ${last}`; - }, }); // 5. In-process path (CLI, worker, tests) — same plane, no second factory. @@ -131,19 +128,18 @@ await memory.add({ content: { title: "Kickoff", text: "Ship memory 0→1 docs" }, }); -const hits = await memory.find({ +const hits = await memory.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "memory docs", }); -console.log("find hits:", hits.items.length); +console.log("search hits:", hits.items.length); -const answer = await memory.ask({ +const events = await memory.list({ tenantId: TENANT, principalId: PRINCIPAL, - query: "what should we ship?", }); -console.log("ask:", answer.answer); +console.log("list events:", events.length); // 6. HTTP export default { @@ -161,16 +157,11 @@ curl -sS -X POST http://127.0.0.1:8787/api/memory/add \ -H 'content-type: application/json' \ -d '{"title":"Note","text":"hello from curl"}' - -curl -sS -X POST http://127.0.0.1:8787/api/memory/find \ +curl -sS -X POST http://127.0.0.1:8787/api/memory/search \ -H 'content-type: application/json' \ -d '{"query":"hello"}' -curl -sS -X POST http://127.0.0.1:8787/api/memory/ask \ - -H 'content-type: application/json' \ - -d '{"query":"what do we know?"}' - -curl -sS 'http://127.0.0.1:8787/api/memory/recent' +curl -sS 'http://127.0.0.1:8787/api/memory/list' ``` Clients never send tenant/principal in the body. Without principal middleware: @@ -179,9 +170,8 @@ Clients never send tenant/principal in the body. Without principal middleware: | Method | Path | Grant (`requireGrant`) | | --- | --- | --- | | POST | `/api/memory/add` | `("memory", "add")` | -| POST | `/api/memory/find` | `("memory", "find")` | -| POST | `/api/memory/ask` | `("memory", "find")` | -| GET | `/api/memory/recent` | `("memory", "find")` | +| POST | `/api/memory/search` | `("memory", "search")` | +| GET | `/api/memory/list` | `("memory", "search")` | ## Production host (Postgres + real Interchange) @@ -198,87 +188,40 @@ const memory = createMemory({ config: loadMemoryConfig(), // needs KNOWLEDGE_DATABASE_URL + embed env grantStore: hubGrantStore, conditionRegistry: hubConditionRegistry, - // generate: wire to @intx/inference (or your own) for ask() }); ``` -Env / migrations for the default pgvector store: - -```bash -# KNOWLEDGE_DATABASE_URL required — no DATABASE_URL fallback -export KNOWLEDGE_DATABASE_URL=postgres://… -bun run db:setup -# or: runMemoryMigrations(process.env.KNOWLEDGE_DATABASE_URL) -``` - -Tables live under Postgres schema **`knowledge`** -(`knowledge.document`, `knowledge.version`, `knowledge.chunk`, …). Hard cutover -pre-1.0: re-run migrations on a fresh knowledge DB. - -## Ports - -| Port | Default | Override | -| --- | --- | --- | -| `DocumentStore` | Engine pgvector | `options.documentStore` / `createFakeDocumentStore()` | -| `SourceProvider[]` | none | `options.sources` — live merge is fail-soft | -| `MemoryProvider` | none | `options.memoryProvider` / `createFakeMemoryProvider()` | - -**Live merge (MergeLocalLiveV1):** per-provider timeout/error → `live_timeout` / -`live_error` degrade; dedupe `adapter:externalRef`; optional `sources` filter. - -**Memory side-channel:** `ask({ includeMemory: true })` recalls when a provider -is set; failure → `memory_unavailable`, docs-only. `memory.remember` / -`memory.recall` for host-owned writes (ask never auto-remembers). - -### Optional DocumentStore adapters (sibling packages) - -Optional backends are **sibling packages**, not vendored here. Core never imports -vendor SDKs. Install each from git; each package README is a full 0→1 example. - -| Package | Role | -| --- | --- | -| [`@corbits/mem0-memory-adapter`](https://github.com/corbitsdev/corbits-mem0-memory-adapter) | Mem0 as `documentStore` | -| [`@corbits/supermemory-memory-adapter`](https://github.com/corbitsdev/corbits-supermemory-memory-adapter) | Supermemory as `documentStore` | -| [`@corbits/linear-tools`](https://github.com/corbitsdev/corbits-linear-tools) | Linear **tools** (`SourceProvider` + webhook map) — not a store | +### Inference (host-owned, ephemeral) ```ts -import { createMem0DocumentStore } from "@corbits/mem0-memory-adapter"; - -const memory = createMemory({ - documentStore: createMem0DocumentStore({ apiKey: process.env.MEM0_API_KEY! }), - grantStore, - conditionRegistry, -}); +// Host extracts durable facts with its own model, then writes: +const facts = await hostGenerate(transcript); +for (const fact of facts) { + await memory.add({ + tenantId, + principalId, + content: { title: fact.title, text: fact.text }, + }); +} +// Host answers with retrieval + its own model: +const { items } = await memory.search({ tenantId, principalId, query }); +const answer = await hostGenerate(buildPrompt(query, items)); ``` -## Document access (grant tags) +There is no `ask` / `remember` / `recall` product path and no ingest agent in core. -1. Host capability grants (`memory` + `add` / `find`) -2. Per-document **access tags** + creator rule (Interchange `@intx/authz`) -3. Share mints tags only — no visibility modes or block lists +## Hard cutover notes -See [docs/AUTHZ-DOCUMENT-ACCESS.md](./docs/AUTHZ-DOCUMENT-ACCESS.md). - -## Compose routes yourself - -Most hosts use `createMemory({ app, … })`. For custom route composition: - -```ts -import { createMemory, registerMemoryRoutes, resolveGrantConfig } from "@corbits/memory"; -import { createRequireGrant } from "@intx/hub-api"; - -const grantStore = hubGrantStore; -const conditionRegistry = hubConditionRegistry; -const grants = resolveGrantConfig({ grantStore, conditionRegistry })!; - -const memory = createMemory({ grantStore, conditionRegistry, documentStore }); // no app -registerMemoryRoutes(app, { - memory, - grants, - requireGrant: createRequireGrant(grants), -}); -``` +| Old | New | +| --- | --- | +| `find` | `search` | +| `recent` | `list` | +| `ask` / `remember` / `recall` | removed (host-owned inference) | +| grant action `find` | `search` | +| `/api/memory/find` | `/api/memory/search` | +| `/api/memory/recent` | `/api/memory/list` | +| `/api/memory/ask` | removed | ## License -LGPL-2.1-only +See repository. diff --git a/package.json b/package.json index 4b9e662..0633878 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@corbits/memory", "version": "0.1.2", - "description": "Mountable memory add/find/ask/recent SDK for Interchange hubs", + "description": "Mountable memory add/search/list SDK for Interchange hubs", "exports": { ".": "./src/index.ts", "./migrations": "./src/migrations.ts", diff --git a/src/grant-tags.test.ts b/src/grant-tags.test.ts index d1f5ae9..d593655 100644 --- a/src/grant-tags.test.ts +++ b/src/grant-tags.test.ts @@ -74,7 +74,7 @@ describe("canAccessDocument", () => { id: "g1", principalId: "peer", resource: tenantTag("t1"), - action: "find", + action: "search", effect: "allow", origin: "role", conditions: null, @@ -109,7 +109,7 @@ describe("filterAccessibleDocuments", () => { id: "g1", principalId: "viewer", resource: "memory.space:eng", - action: "find", + action: "search", effect: "allow", origin: "role", conditions: null, diff --git a/src/grant-tags.ts b/src/grant-tags.ts index 65bfb60..3affe4b 100644 --- a/src/grant-tags.ts +++ b/src/grant-tags.ts @@ -3,9 +3,9 @@ * * Spec: docs/AUTHZ-DOCUMENT-ACCESS.md * - * - Capability checks (add/find on `memory`) live on the HTTP mount. + * - Capability checks (add/search on `memory`) live on the HTTP mount. * - Document access: creator always sees own docs; otherwise any `accessTag` - * that `authorize(…, tag, "find")` allows. + * that `authorize(…, tag, "search")` allows. * - Share sugars only mint tags — they never write grants. */ import { authorize } from "@intx/authz"; @@ -83,7 +83,7 @@ export type CanAccessDocumentParams = { /** * True when the principal may see this document under grant-tag rules. * Creator is always allowed (implicit owner). Everyone else needs an allow - * on at least one accessTag for action "find". + * on at least one accessTag for action "search". */ export async function canAccessDocument( params: CanAccessDocumentParams, @@ -103,7 +103,7 @@ export async function canAccessDocument( params.principalId, params.tenantId, tag, - "find", + "search", params.conditionRegistry, ); if (decision.effect === "allow") return true; diff --git a/src/index.ts b/src/index.ts index c0db7de..1dd42e3 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,10 +1,13 @@ /** - * @corbits/memory — add / find / ask / recent for Interchange hubs. + * @corbits/memory — add / search / list for Interchange hubs. * * One entry: `createMemory(options)`. Pass `app` to register HTTP routes on * a Hono host. Identity is `c.get("principal")` on HTTP, or `principalId` + * `tenantId` in-process. Authz is the host grant store — this package * authenticates nothing itself. + * + * Inference is host-owned and ephemeral (call your model, then add/search). + * Core does not mount an ingest agent or bake LLM into the write path. */ import type { Hono } from "hono"; import { createRequireGrant, type TenantEnv } from "@intx/hub-api"; @@ -29,52 +32,45 @@ export { RerankConfigError } from "./core/rerank-client.ts"; // Memory plane — types from memory.ts; createMemory is defined below so it // can optionally register HTTP routes when `app` is passed. export type { - AskCitation, - AskResult, - ChatMessage, - FindItem, - FindResult, - Generate, HybridSearchResult, MemoryAddParams, MemoryAddResult, - MemoryAskParams, - MemoryFindParams, + MemorySearchParams, MemoryIdentity, Memory, MemoryOptions, - MemoryRecentParams, - MemoryRecallItem, - MemoryRecallParams, - MemoryRememberParams, + MemoryListParams, MemoryShare, SearchHit, + SearchItem, + SearchResult, TextExtractor, TimelineEvent, } from "./memory.ts"; export { MemoryError, - MemoryNotPermittedError, resolveGrantConfig, + SEARCH_LIMIT_MIN, + SEARCH_LIMIT_MAX, + LIST_LIMIT_MIN, + LIST_LIMIT_MAX, } from "./memory.ts"; -// Ports — pluggable storage, live sources, and optional memory +// Ports — pluggable storage and live sources export type { DocumentStore, DocumentStoreAddParams, - DocumentStoreFindItem, - DocumentStoreFindParams, - DocumentStoreFindResult, - DocumentStoreRecentEvent, - DocumentStoreRecentParams, + DocumentStoreSearchItem, + DocumentStoreSearchParams, + DocumentStoreSearchResult, + DocumentStoreListEvent, + DocumentStoreListParams, LiveSearchItem, - MemoryProvider, SourceProvider, } from "./ports/types.ts"; export { createFakeDocumentStore, - createFakeMemoryProvider, createFakeSourceProvider, } from "./ports/fakes.ts"; @@ -119,36 +115,48 @@ export type CreateMemoryOptions = MemoryOptions & { * grantStore, * conditionRegistry, * }); - * await memory.add({ tenantId, principalId, content: { title, text } }); + * await memory.add({ principalId, tenantId, content: { title, text } }); + * const { items } = await memory.search({ principalId, tenantId, query }); * ``` * - * @example With HTTP + * @example HTTP on a Hono host * ```ts - * const memory = createMemory({ + * createMemory({ * app, + * documentStore: createFakeDocumentStore(), * grantStore, * conditionRegistry, - * config: loadMemoryConfig(), - * generate, * }); + * // POST /api/memory/add | search · GET /api/memory/list * ``` */ -export function createMemory(options: CreateMemoryOptions = {}): Memory { - const { app, ...planeOptions } = options; - const memory = createMemoryPlane(planeOptions); +export function createMemory(options: CreateMemoryOptions): Memory { + const { app, grantStore, conditionRegistry, ...planeOpts } = options; + const grants = resolveGrantConfig({ + ...(grantStore !== undefined ? { grantStore } : {}), + ...(conditionRegistry !== undefined ? { conditionRegistry } : {}), + }); + const memory = createMemoryPlane({ + ...planeOpts, + ...(grantStore !== undefined ? { grantStore } : {}), + ...(conditionRegistry !== undefined ? { conditionRegistry } : {}), + }); + if (app) { - const grants = resolveGrantConfig(options); if (!grants) { throw new Error( - "createMemory({ app }): grantStore is required to register HTTP routes", + "createMemory({ app }): grantStore is required when registering HTTP routes " + + "(pass grantStore; conditionRegistry is optional)", ); } + const requireGrant = createRequireGrant(grants); const deps: RouteDeps = { memory, + requireGrant, grants, - requireGrant: createRequireGrant(grants), }; registerMemoryRoutes(app, deps); } + return memory; } diff --git a/src/memory.test.ts b/src/memory.test.ts index 6ae9a1c..a693996 100644 --- a/src/memory.test.ts +++ b/src/memory.test.ts @@ -1,12 +1,10 @@ /** - * Plane construction, grant-tag ACL wiring, and ask() coverage for memory.ts. + * Plane construction, grant-tag ACL wiring, and surface coverage for memory.ts. * * - Construction: rerank maxDocChars validation runs in createMemory. - * - Find wiring: grant-tag post-filter (creator-only without grants). - * - ask(): grant check, missing generate (501 before find), allow path, - * synthesizeAnswer grounding. + * - Search wiring: grant-tag post-filter (creator-only without grants). * - add(): documentId return, content/file XOR, share → access tags. - * - find/recent: limit bounds, evidence default omit. + * - search/list: limit bounds, evidence default omit. */ import { afterAll, @@ -24,11 +22,8 @@ import type { SearchHit } from "./core/schemas/search.ts"; import { createMemory, MemoryError, - MemoryNotPermittedError, - synthesizeAnswer, - type ChatMessage, - type FindItem, type MemoryAddParams, + type SearchItem, type TextExtractor, } from "./memory.ts"; import type { MemoryConfig } from "./mount-config.ts"; @@ -108,17 +103,6 @@ function hit(overrides: Partial | string = {}): SearchHit { }; } -function findItemFromHit(h: SearchHit): FindItem { - return { - documentId: h.document_id, - title: h.title, - snippet: h.snippet, - score: h.score, - kind: h.kind, - citation: h.citation, - }; -} - const wiringConfig: MemoryConfig = { memory: { databaseUrl: "postgres://localhost:5432/nonexistent-test-db", @@ -139,8 +123,6 @@ const wiringConfig: MemoryConfig = { }, }; -const askConfig: MemoryConfig = wiringConfig; - function ftsUnsafe(sqlText: string): Promise>> { if (sqlText.includes("pg_ts_config")) { return Promise.resolve([{ ok: 1 }]); @@ -262,7 +244,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { ); const plane = makePlane({ config: wiringConfig }); - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -270,7 +252,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { }); expect(hybridSearch).toHaveBeenCalled(); - expect(result.items.map((i: FindItem) => i.documentId)).toEqual([ + expect(result.items.map((i: SearchItem) => i.documentId)).toEqual([ "d-mine", ]); expect(result.evidence).toBe("strong"); @@ -293,7 +275,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { ); const plane = makePlane({ config: wiringConfig }); - await plane.find({ + await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -328,7 +310,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { ); const plane = makePlane({ config: wiringConfig }); - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -367,7 +349,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { const plane = makePlane({ config: wiringConfig }); - const without = await plane.find({ + const without = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -376,7 +358,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { expect(without.evidence).toBeUndefined(); expect(without.degraded).toBeUndefined(); - const withEv = await plane.find({ + const withEv = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -395,7 +377,7 @@ describe("find/recent — limit bounds", () => { it("find rejects limit below 1", async () => { const plane = createMemory({ config: wiringConfig }); try { - await plane.find({ + await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -412,7 +394,7 @@ describe("find/recent — limit bounds", () => { it("find rejects limit above 50", async () => { const plane = createMemory({ config: wiringConfig }); try { - await plane.find({ + await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "q", @@ -428,7 +410,7 @@ describe("find/recent — limit bounds", () => { it("recent rejects limit above 100", async () => { const plane = createMemory({ config: wiringConfig }); try { - await plane.recent({ + await plane.list({ tenantId: TENANT, principalId: PRINCIPAL, limit: 101, @@ -443,7 +425,7 @@ describe("find/recent — limit bounds", () => { it("recent rejects limit below 1", async () => { const plane = createMemory({ config: wiringConfig }); try { - await plane.recent({ + await plane.list({ tenantId: TENANT, principalId: PRINCIPAL, limit: 0, @@ -731,199 +713,3 @@ async function freshPlane(opts?: { await plane.close(); }); }); - -describe("ask() — grant check", () => { - it("denies with MemoryNotPermittedError when no grant matches (effect: null)", async () => { - const grantStore = createInMemoryGrantStore([]); - const plane = createMemory({ - config: askConfig, - grantStore, - conditionRegistry: {}, - }); - await expect( - plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), - ).rejects.toBeInstanceOf(MemoryNotPermittedError); - }); - - it("denies when the only matching grant is an explicit deny", async () => { - const denyGrant: GrantRule = { ...grant("find"), effect: "deny" }; - const plane = createMemory({ - config: askConfig, - grantStore: createInMemoryGrantStore([denyGrant]), - conditionRegistry: {}, - }); - await expect( - plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }), - ).rejects.toBeInstanceOf(MemoryNotPermittedError); - }); -}); - -describe("ask() — missing generate", () => { - it("throws MemoryError 501 before find when generate is not wired", async () => { - // Pointed at a nonexistent DB: if find ran first this would surface a - // connection/driver error instead of the promised 501. - const plane = createMemory({ - config: askConfig, - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - }); - try { - await plane.ask({ tenantId: TENANT, principalId: PRINCIPAL, query: "q" }); - throw new Error("expected ask() to reject"); - } catch (err) { - expect(err).toBeInstanceOf(MemoryError); - expect((err as MemoryError).status).toBe(501); - expect((err as MemoryError).message).toContain("generate"); - } - }); -}); - -describe("ask() — allow path", () => { - it("finds as the principal and synthesizes when grant allows and generate is wired", async () => { - const generate = mock((messages: readonly ChatMessage[]) => { - expect(messages[0]?.role).toBe("system"); - expect(messages[1]?.content).toContain("what is the answer?"); - expect(messages[1]?.content).toContain("[1] Doc One"); - expect(messages[1]?.content).toContain("the relevant snippet"); - return Promise.resolve("Answer from context [1]."); - }); - const plane = createMemory({ - config: askConfig, - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - generate, - }); - // Stub find so this unit test never needs a live Postgres. ask() looks - // up plane.find at call time, so reassignment is the wiring under test. - plane.find = mock(() => - Promise.resolve({ - items: [findItemFromHit(hit())], - evidence: "strong" as const, - }), - ); - - const result = await plane.ask({ - tenantId: TENANT, - principalId: PRINCIPAL, - query: "what is the answer?", - }); - - expect(plane.find).toHaveBeenCalledWith({ - tenantId: TENANT, - principalId: PRINCIPAL, - query: "what is the answer?", - includeEvidence: true, - }); - expect(generate).toHaveBeenCalledTimes(1); - expect(result.text).toBe("Answer from context [1]."); - expect(result.evidence).toBe("strong"); - expect(result.citations).toEqual([ - { - index: 1, - documentId: "doc-1", - title: "Doc One", - citation: hit().citation, - }, - ]); - }); -}); - -describe("synthesizeAnswer", () => { - const neverCalled = mock(() => - Promise.reject(new Error("generate must not be called")), - ); - - it("refuses with no citations when there are no hits", async () => { - const result = await synthesizeAnswer( - "q", - { hits: [], evidence: "none" }, - neverCalled, - ); - expect(result.citations).toEqual([]); - expect(result.evidence).toBe("none"); - expect(neverCalled).not.toHaveBeenCalled(); - }); - - it("refuses with no citations when hits have no readable snippet text", async () => { - const result = await synthesizeAnswer( - "q", - { hits: [hit({ snippet: " " })], evidence: "weak" }, - neverCalled, - ); - expect(result.text).toContain("couldn't read any text"); - expect(result.citations).toEqual([]); - expect(neverCalled).not.toHaveBeenCalled(); - }); - - it("grounds the prompt in the retrieved snippets and returns their citations", async () => { - const generate = mock((messages: readonly ChatMessage[]) => { - // The grounding contract: the model sees the numbered context, and only - // the numbered context, for the hits that fit the budget. - expect(messages[1]?.content).toContain("[1] Doc One"); - expect(messages[1]?.content).toContain("the relevant snippet"); - return Promise.resolve("Grounded answer [1]."); - }); - - const result = await synthesizeAnswer( - "what is the answer?", - { hits: [hit()], evidence: "strong" }, - generate, - ); - - expect(result.text).toBe("Grounded answer [1]."); - expect(result.evidence).toBe("strong"); - expect(result.citations).toEqual([ - { - index: 1, - documentId: "doc-1", - title: "Doc One", - citation: hit().citation, - }, - ]); - }); - - it("numbers citations sequentially among included entries, skipping empty snippets", async () => { - const generate = mock((messages: readonly ChatMessage[]) => { - const content = messages[1]?.content ?? ""; - // First hit is empty → skipped; second becomes [1], not [2]. - expect(content).toContain("[1] Real Doc"); - expect(content).toContain("actual content"); - expect(content).not.toContain("[2]"); - expect(content).not.toContain("Empty Doc"); - return Promise.resolve("From [1]."); - }); - - const result = await synthesizeAnswer( - "q", - { - hits: [ - hit({ title: "Empty Doc", snippet: " " }), - hit({ - title: "Real Doc", - snippet: "actual content", - document_id: "doc-2", - chunk_id: "chunk-2", - }), - ], - evidence: "weak", - }, - generate, - ); - - expect(result.citations).toEqual([ - { - index: 1, - documentId: "doc-2", - title: "Real Doc", - citation: hit().citation, - }, - ]); - }); - - it("propagates a generate failure rather than inventing an answer", async () => { - const generate = mock(() => Promise.reject(new Error("model unreachable"))); - await expect( - synthesizeAnswer("q", { hits: [hit()], evidence: "weak" }, generate), - ).rejects.toThrow("model unreachable"); - }); -}); diff --git a/src/memory.ts b/src/memory.ts index 839efff..54e5e58 100644 --- a/src/memory.ts +++ b/src/memory.ts @@ -38,8 +38,7 @@ import type { MemoryConfig } from "./mount-config.ts"; import type { GrantConfig } from "./routes/deps.ts"; import type { DocumentStore, - DocumentStoreFindParams, - MemoryProvider, + DocumentStoreSearchParams, SourceProvider, } from "./ports/types.ts"; // (drizzle select was used briefly for grant-tag load; raw sql keeps unit-test @@ -52,7 +51,6 @@ export type { DocumentStore, DocumentStoreAddParams, LiveSearchItem, - MemoryProvider, SourceProvider, } from "./ports/types.ts"; export { @@ -63,26 +61,6 @@ export { type ShareSugar, } from "./grant-tags.ts"; - -export type ChatMessage = { - role: "system" | "user" | "assistant"; - content: string; -}; - -/** - * How `ask()` reaches a model. Supplied by the host, not owned here. - * - * The engine deliberately has no generation client. Interchange already has an - * inference layer (`@intx/inference`) with provider adapters, tenant-scoped - * credentials, retry policy, audit and authz gates — hand-rolling a `fetch` here - * would bypass all of it and take an API key from a raw env var. Hosts wire - * this to that layer; tests pass a stub. - * - * Same posture the engine already takes on embedding: never in-process, always - * an endpoint the owner plugs in. - */ -export type Generate = (messages: readonly ChatMessage[]) => Promise; - /** * Optional host-supplied extractor for `add({ file })`. The engine never * ships a PDF/OCR/vendor SDK — the host plugs one in when file ingest is needed. @@ -101,14 +79,14 @@ export type MemoryIdentity = { }; /** Green find limit bounds (stricter than hybridSearch's internal MAX_K). */ -export const FIND_LIMIT_MIN = 1; -export const FIND_LIMIT_MAX = 50; +export const SEARCH_LIMIT_MIN = 1; +export const SEARCH_LIMIT_MAX = 50; /** Green recent limit bounds (matches timeline service default/cap). */ -export const RECENT_LIMIT_MIN = 1; -export const RECENT_LIMIT_MAX = 100; +export const LIST_LIMIT_MIN = 1; +export const LIST_LIMIT_MAX = 100; -export type MemoryFindParams = MemoryIdentity & { +export type MemorySearchParams = MemoryIdentity & { query: string; /** Max items to return (1–50). Default 8. */ limit?: number; @@ -133,43 +111,6 @@ export type MemoryFindParams = MemoryIdentity & { sources?: string[]; }; -export type MemoryAskParams = MemoryIdentity & { - query: string; - limit?: number; - /** Same channel filter as find (passed through). */ - sources?: string[]; - /** - * When true and a MemoryProvider is mounted, recall personal memory into - * the ask context. Default false — memory is opt-in per call. - */ - includeMemory?: boolean; -}; - -/** One source cited in an `ask()` answer, matched to its bracket in the text. */ -export type AskCitation = { - /** The `[N]` marker the grounding prompt asked the model to cite. */ - index: number; - documentId: string; - title: string; - citation: SearchHit["citation"]; -}; - -export type AskResult = { - text: string; - citations: AskCitation[]; - evidence: HybridSearchResult["evidence"]; - /** Present when memory/live stages degraded (ask still answered). */ - degraded?: HybridSearchResult["degraded"]; -}; - -/** Thrown when the asking principal lacks the memory:find capability. */ -export class MemoryNotPermittedError extends Error { - constructor() { - super("principal lacks the memory:find grant"); - this.name = "MemoryNotPermittedError"; - } -} - export type MemoryShare = ShareSugar; export type MemoryAddParams = MemoryIdentity & { @@ -198,7 +139,7 @@ export type MemoryAddParams = MemoryIdentity & { export type MemoryAddResult = { documentId: string }; -export type FindItem = { +export type SearchItem = { documentId: string; title: string; snippet: string; @@ -209,14 +150,14 @@ export type FindItem = { updatedAt?: string; }; -export type FindResult = { - items: FindItem[]; +export type SearchResult = { + items: SearchItem[]; /** Only present when includeEvidence: true */ evidence?: "strong" | "weak" | "none"; degraded?: HybridSearchResult["degraded"]; }; -export type MemoryRecentParams = MemoryIdentity & { +export type MemoryListParams = MemoryIdentity & { limit?: number; }; @@ -231,155 +172,20 @@ export class MemoryError extends Error { } export type Memory = { - find(params: MemoryFindParams): Promise; - ask(params: MemoryAskParams): Promise; + search(params: MemorySearchParams): Promise; add(params: MemoryAddParams): Promise; - recent(params: MemoryRecentParams): Promise; - /** - * Write a memory fact for a principal. Requires a mounted MemoryProvider; - * throws 501 when memory is not configured. Never called implicitly by ask. - */ - remember(params: MemoryRememberParams): Promise; - /** - * Recall memory facts for a principal. Empty array when memory is not - * configured or nothing matches. - */ - recall(params: MemoryRecallParams): Promise; + list(params: MemoryListParams): Promise; close(): Promise; }; -export type MemoryRememberParams = MemoryIdentity & { - text: string; - metadata?: Record; -}; - -export type MemoryRecallParams = MemoryIdentity & { - query: string; - limit?: number; -}; - -export type MemoryRecallItem = { - text: string; - score?: number; -}; - export type { TimelineEvent }; -// Character budget for the grounded context block handed to the generation -// endpoint. Deliberately conservative: it bounds prompt size regardless of -// how many/large the retrieved hits are. -const MAX_CONTEXT_CHARS = 8_000; - -const SYSTEM_PROMPT = [ - "You are a knowledge assistant answering questions from retrieved context.", - "", - "Answer ONLY from the numbered context provided. The context has already", - "been filtered to what this specific principal is permitted to read, so", - "never speculate beyond it or fill gaps from your own knowledge.", - "", - "If the context does not contain the answer, say so plainly in one sentence", - "and stop — do not guess.", - "", - "Cite the sources you used as bracketed numbers, e.g. [1] or [2]. Be", - "concise: a few sentences.", -].join("\n"); - -/** Build the grounded context block, truncated to a sane prompt budget. */ -function buildContext(hits: readonly SearchHit[]): { - block: string; - citations: AskCitation[]; -} { - const citations: AskCitation[] = []; - const parts: string[] = []; - let budget = MAX_CONTEXT_CHARS; - - // Number only among entries that actually land in the prompt. Skipping an - // empty snippet (or stopping on budget) must not leave gaps in [N] markers. - let nextIndex = 1; - for (const hit of hits) { - const text = hit.snippet.trim(); - if (!text) continue; - const index = nextIndex; - const entry = `[${index}] ${hit.title}\n${text}`; - if (entry.length > budget) break; - budget -= entry.length; - parts.push(entry); - citations.push({ - index, - documentId: hit.document_id, - title: hit.title, - citation: hit.citation, - }); - nextIndex += 1; - } - - return { block: parts.join("\n\n"), citations }; -} - -/** - * Turn a search result into an answer: assemble grounded context, call the - * configured generation endpoint, and return the citations actually used. - * Factored out of `ask()` so it is unit-testable against a mocked generation - * endpoint without a real search result / database. - * - * Optional `memoryTexts` are prepended as uncited personal context when the - * host opted into includeMemory. They never produce citations. - */ -export async function synthesizeAnswer( - query: string, - result: Pick, - generate: Generate, - memoryTexts: readonly string[] = [], -): Promise { - if (result.hits.length === 0 && memoryTexts.length === 0) { - return { - text: "I couldn't find anything you have access to that answers that.", - citations: [], - evidence: "none", - }; - } - - const { block, citations } = buildContext(result.hits); - const memoryBlock = - memoryTexts.length > 0 - ? "Personal memory:\n" + - memoryTexts.map((t, i) => `- (m${i + 1}) ${t}`).join("\n") - : ""; - - if (!block && !memoryBlock) { - return { - text: "I found matching documents but couldn't read any text out of them.", - citations: [], - evidence: "none", - }; - } - - const contextParts = [memoryBlock, block].filter(Boolean); - const text = await generate([ - { role: "system", content: SYSTEM_PROMPT }, - { - role: "user", - content: `Question: ${query}\n\nContext:\n${contextParts.join("\n\n")}`, - }, - ]); - - return { - text, - citations, - evidence: - result.hits.length === 0 - ? "weak" - : result.evidence, - }; -} - export type MemoryOptions = { /** Engine config (DB + model endpoints). Required when `documentStore` is omitted. */ config?: MemoryConfig; /** - * Host Interchange `GrantStore`. Required for `ask()` (in-process capability - * check) and for document access filtering on find/recent. Standalone - * add-only callers may omit it. + * Host Interchange `GrantStore`. Used for document access filtering on + * search/list. Standalone add-only callers may omit it. */ grantStore?: GrantConfig["grantStore"]; /** @@ -388,8 +194,6 @@ export type MemoryOptions = { * `createRequireGrant` / `createApp`. */ conditionRegistry?: GrantConfig["conditionRegistry"]; - /** Required for `ask()`; omit if the host only adds and finds. */ - generate?: Generate; /** Required for `add({ file })`; omit if the host only adds text content. */ textExtractor?: TextExtractor; /** @@ -399,15 +203,10 @@ export type MemoryOptions = { */ documentStore?: DocumentStore; /** - * Live source connectors (tools-shaped). Merged into find/ask via + * Live source connectors (tools-shaped). Merged into search via * MergeLocalLiveV1; not a DocumentStore replacement. */ sources?: SourceProvider[]; - /** - * Optional personal-memory side channel for ask(includeMemory). - * Not how you swap durable backends — use documentStore for that. - */ - memoryProvider?: MemoryProvider; }; /** Bundle host authz pieces for route guards and document access. */ @@ -421,39 +220,39 @@ export function resolveGrantConfig( }; } -function resolveFindLimit(limit: number | undefined): number { +function resolveSearchLimit(limit: number | undefined): number { if (limit === undefined) return DEFAULT_HYBRID_TOP_K; if ( typeof limit !== "number" || !Number.isInteger(limit) || - limit < FIND_LIMIT_MIN || - limit > FIND_LIMIT_MAX + limit < SEARCH_LIMIT_MIN || + limit > SEARCH_LIMIT_MAX ) { throw new MemoryError( 400, - `limit must be an integer between ${FIND_LIMIT_MIN} and ${FIND_LIMIT_MAX}`, + `limit must be an integer between ${SEARCH_LIMIT_MIN} and ${SEARCH_LIMIT_MAX}`, ); } return limit; } -function resolveRecentLimit(limit: number | undefined): number | undefined { +function resolveListLimit(limit: number | undefined): number | undefined { if (limit === undefined) return undefined; if ( typeof limit !== "number" || !Number.isInteger(limit) || - limit < RECENT_LIMIT_MIN || - limit > RECENT_LIMIT_MAX + limit < LIST_LIMIT_MIN || + limit > LIST_LIMIT_MAX ) { throw new MemoryError( 400, - `limit must be an integer between ${RECENT_LIMIT_MIN} and ${RECENT_LIMIT_MAX}`, + `limit must be an integer between ${LIST_LIMIT_MIN} and ${LIST_LIMIT_MAX}`, ); } return limit; } -function hitsToFindItems(hits: readonly SearchHit[]): FindItem[] { +function hitsToSearchItems(hits: readonly SearchHit[]): SearchItem[] { return hits.map((h) => ({ documentId: h.document_id, title: h.title, @@ -464,25 +263,6 @@ function hitsToFindItems(hits: readonly SearchHit[]): FindItem[] { })); } -/** Map FindItems back to the minimal SearchHit shape synthesizeAnswer needs. */ -function findItemsToHits(items: readonly FindItem[]): SearchHit[] { - return items.map((item) => ({ - chunk_id: "", - document_id: item.documentId, - version: 0, - version_id: "", - status: "active" as const, - score: item.score, - title: item.title, - snippet: item.snippet, - kind: item.kind, - created_by_kind: "human" as const, - citation: item.citation, - entity_ids: [], - channels_matched: [], - })); -} - /** * Resolve access tags for add — share sugar + explicit tags only. */ @@ -502,13 +282,14 @@ function resolveAddAccessTags(params: MemoryAddParams): string[] { * is omitted, the default pgvector engine is wrapped as that store. Hosts * inject a DocumentStore or fakes the same way — no second plane implementation. * - * - `options.grantStore` is required for `ask()` (in-process capability check). - * Standalone add/find callers may omit it. `conditionRegistry` is optional - * (defaults to `{}`). + * - Pass `options.grantStore` for document access filtering on search/list. + * `conditionRegistry` is optional (defaults to `{}`). * - Rerank config is validated at construction when using the default store. - * - Pass `options.sources` for live SourceProviders; find/ask merge via + * - Pass `options.sources` for live SourceProviders; search merges via * MergeLocalLiveV1 (fail-soft, 800ms timeout, prefer-local dedupe). * - Document access uses grant tags via the host GrantStore (not mini-ACL). + * - Inference is host-owned and ephemeral: call your model, then `add` / + * `search`. Core does not run an ingest agent or bake LLM into writes. * * @example With default pgvector store * ```ts @@ -533,10 +314,8 @@ export function createMemory(options: MemoryOptions = {}): Memory { grantStore, conditionRegistry, documentStore, - generate, textExtractor, sources, - memoryProvider, } = options; const grants = resolveGrantConfig({ ...(grantStore !== undefined ? { grantStore } : {}), @@ -557,10 +336,8 @@ export function createMemory(options: MemoryOptions = {}): Memory { store, grants, { - ...(generate ? { generate } : {}), ...(textExtractor ? { textExtractor } : {}), ...(sources ? { sources } : {}), - ...(memoryProvider ? { memoryProvider } : {}), }, ); } @@ -570,7 +347,7 @@ function wantsLocalChannel(sources: string[] | undefined): boolean { } function findItemsToMergeChannel( - items: FindItem[], + items: SearchItem[], channel: "local" | "live", ): MergeChannelItem[] { return items.map((item) => ({ @@ -651,8 +428,8 @@ async function collectLiveItems(params: { return { items, degraded }; } -function mergeToFindResult(params: { - localItems: FindItem[]; +function mergeToSearchResult(params: { + localItems: SearchItem[]; localDegraded?: DegradeFlag[]; /** Hybrid evidence from the local channel when no live items were active. */ localEvidence?: HybridSearchResult["evidence"]; @@ -661,7 +438,7 @@ function mergeToFindResult(params: { limit: number; sources?: string[]; includeEvidence?: boolean; -}): FindResult { +}): SearchResult { const merged = mergeLocalLiveV1({ local: findItemsToMergeChannel(params.localItems, "local"), live: params.liveItems, @@ -669,7 +446,7 @@ function mergeToFindResult(params: { ...(params.sources !== undefined ? { sources: params.sources } : {}), }); - const items: FindItem[] = merged.items.map((it) => ({ + const items: SearchItem[] = merged.items.map((it) => ({ documentId: it.documentId, title: it.title, snippet: it.snippet, @@ -715,68 +492,9 @@ function mergeToFindResult(params: { } /** - * Optional memory recall for ask. Never throws — failures become - * memory_unavailable degrade. Does not call remember (host-owned writes only). + * Optional personal-memory path removed from product surface. + * Hosts that need LLM extract-on-add call their own model, then `add`. */ -async function recallForAsk(params: { - memory: MemoryProvider | undefined; - includeMemory: boolean | undefined; - tenantId: string; - principalId: string; - query: string; -}): Promise<{ texts: string[]; degraded: DegradeFlag[] }> { - if (!params.includeMemory || !params.memory) { - return { texts: [], degraded: [] }; - } - try { - const items = await params.memory.recall({ - tenantId: params.tenantId, - principalId: params.principalId, - query: params.query, - }); - return { - texts: items.map((i) => i.text).filter((t) => t.trim().length > 0), - degraded: [], - }; - } catch (err) { - log.warn("ask: memory recall failed; continuing docs-only", { - error: err instanceof Error ? err.message : String(err), - }); - return { texts: [], degraded: ["memory_unavailable"] }; - } -} - -function makeRememberRecall(options: MemoryOptions): { - remember: Memory["remember"]; - recall: Memory["recall"]; -} { - return { - async remember(params) { - if (!options.memoryProvider) { - throw new MemoryError( - 501, - "remember() requires a MemoryProvider. Pass memoryProvider to " + - "createMemory.", - ); - } - await options.memoryProvider.remember({ - tenantId: params.tenantId, - principalId: params.principalId, - text: params.text, - ...(params.metadata !== undefined ? { metadata: params.metadata } : {}), - }); - }, - async recall(params) { - if (!options.memoryProvider) return []; - return options.memoryProvider.recall({ - tenantId: params.tenantId, - principalId: params.principalId, - query: params.query, - ...(params.limit !== undefined ? { limit: params.limit } : {}), - }); - }, - }; -} /** Plane backed by a DocumentStore. Store owns tenancy and document ACL. */ function createPlaneFromStore( @@ -784,18 +502,16 @@ function createPlaneFromStore( grants: GrantConfig | undefined, options: MemoryOptions, ): Memory { - const memoryApi = makeRememberRecall(options); - - async function findMerged( - params: MemoryFindParams, - ): Promise { - const limit = resolveFindLimit(params.limit); - let localItems: FindItem[] = []; + async function searchMerged( + params: MemorySearchParams, + ): Promise { + const limit = resolveSearchLimit(params.limit); + let localItems: SearchItem[] = []; let localDegraded: DegradeFlag[] | undefined; let localEvidence: HybridSearchResult["evidence"] | undefined; if (wantsLocalChannel(params.sources)) { - const local = await store.find({ + const local = await store.search({ tenantId: params.tenantId, principalId: params.principalId, query: params.query, @@ -848,7 +564,7 @@ function createPlaneFromStore( return { items: localItems }; } - return mergeToFindResult({ + return mergeToSearchResult({ localItems, ...(localDegraded !== undefined ? { localDegraded } : {}), ...(localEvidence !== undefined ? { localEvidence } : {}), @@ -863,87 +579,8 @@ function createPlaneFromStore( } const plane: Memory = { - async find(params) { - return findMerged(params); - }, - - async ask(params) { - // Capability layer. Callers reaching the plane in-process bypass the - // HTTP surface's `requireGrant("memory", ...)` route guard, so the - // check has to live here — AUTH.md is explicit that the capability and - // data layers are independent and BOTH must allow. Per-document - // grant-tag access (enforced inside the store) is not a substitute for "may - // this principal search at all". - if (!grants) { - throw new MemoryError( - 501, - "ask() requires a grantStore. Pass grantStore to " + - "createMemory({ grantStore, … }) (conditionRegistry optional).", - ); - } - const decision = await authorize( - grants.grantStore, - params.principalId, - params.tenantId, - "memory", - "find", - grants.conditionRegistry, - ); - // `effect: null` means no grant matched at all — deny by default, same - // as an explicit deny. Only an explicit allow proceeds. - if (decision.effect !== "allow") { - const effect = decision.effect ?? "no-matching-grant"; - log.info( - `ask: denied memory:find for ${params.principalId} (effect=${effect})`, - { - principalId: params.principalId, - effect, - }, - ); - throw new MemoryNotPermittedError(); - } - // Fail closed on missing generate *before* retrieval so a misconfigured - // host gets the promised 501 instead of paying for search. - if (!options.generate) { - throw new MemoryError( - 501, - "ask() requires a `generate` function. Pass one to " + - "createMemory, wired to your " + - "inference layer.", - ); - } - const findResult = await plane.find({ - tenantId: params.tenantId, - principalId: params.principalId, - query: params.query, - includeEvidence: true, - ...(params.limit !== undefined ? { limit: params.limit } : {}), - ...(params.sources !== undefined ? { sources: params.sources } : {}), - }); - const mem = await recallForAsk({ - memory: options.memoryProvider, - includeMemory: params.includeMemory, - tenantId: params.tenantId, - principalId: params.principalId, - query: params.query, - }); - const answer = await synthesizeAnswer( - params.query, - { - hits: findItemsToHits(findResult.items), - evidence: findResult.evidence ?? "none", - }, - options.generate, - mem.texts, - ); - const degraded: DegradeFlag[] = [ - ...(findResult.degraded ?? []), - ...mem.degraded, - ]; - return { - ...answer, - ...(degraded.length > 0 ? { degraded } : {}), - }; + async search(params) { + return searchMerged(params); }, async add(params) { @@ -1000,9 +637,9 @@ function createPlaneFromStore( }); }, - async recent(params) { - const limit = resolveRecentLimit(params.limit); - return store.recent({ + async list(params) { + const limit = resolveListLimit(params.limit); + return store.list({ tenantId: params.tenantId, principalId: params.principalId, ...(limit !== undefined ? { limit } : {}), @@ -1013,10 +650,6 @@ function createPlaneFromStore( }); }, - remember: memoryApi.remember, - recall: memoryApi.recall, - - async close() { await store.close(); }, @@ -1083,8 +716,8 @@ function createEngineDocumentStore(config: MemoryConfig): DocumentStore { k?: number; kinds?: string[]; entityIds?: string[]; - grants?: DocumentStoreFindParams["grants"]; - conditionRegistry?: DocumentStoreFindParams["conditionRegistry"]; + grants?: DocumentStoreSearchParams["grants"]; + conditionRegistry?: DocumentStoreSearchParams["conditionRegistry"]; }): Promise { try { await ensureVerified(); @@ -1203,7 +836,7 @@ function createEngineDocumentStore(config: MemoryConfig): DocumentStore { return { documentId: captureResult.documentId }; }, - async find(params) { + async search(params) { const result = await retrieve({ tenantId: params.tenantId, principalId: params.principalId, @@ -1218,7 +851,7 @@ function createEngineDocumentStore(config: MemoryConfig): DocumentStore { ? { conditionRegistry: params.conditionRegistry } : {}), }); - const items = hitsToFindItems(result.hits); + const items = hitsToSearchItems(result.hits); if (params.includeEvidence) { return { items, @@ -1232,7 +865,7 @@ function createEngineDocumentStore(config: MemoryConfig): DocumentStore { }; }, - async recent(params) { + async list(params) { return listTimelineEvents({ db, tenantId: params.tenantId, diff --git a/src/ports/fakes.test.ts b/src/ports/fakes.test.ts index e6dc299..b3e15ed 100644 --- a/src/ports/fakes.test.ts +++ b/src/ports/fakes.test.ts @@ -24,7 +24,7 @@ describe("createFakeDocumentStore", () => { }); expect(documentId).toMatch(/^fake_doc_/); - const found = await store.find({ + const found = await store.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "ports foundation", @@ -34,7 +34,7 @@ describe("createFakeDocumentStore", () => { expect(found.items[0]?.documentId).toBe(documentId); expect(found.evidence).toBe("weak"); - const events = await store.recent({ + const events = await store.list({ tenantId: TENANT, principalId: PRINCIPAL, }); @@ -51,12 +51,12 @@ describe("createFakeDocumentStore", () => { text: "classified payload", accessTags: [ownerTag(PRINCIPAL)], }); - const asOwner = await store.find({ + const asOwner = await store.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "classified", }); - const asOther = await store.find({ + const asOther = await store.search({ tenantId: TENANT, principalId: OTHER, query: "classified", @@ -80,7 +80,7 @@ describe("createFakeDocumentStore", () => { id: "g1", principalId: OTHER, resource: ownerTag(OTHER), - action: "find", + action: "search", effect: "allow", origin: "role", conditions: null, @@ -88,7 +88,7 @@ describe("createFakeDocumentStore", () => { roleId: null, }, ]); - const asOther = await store.find({ + const asOther = await store.search({ tenantId: TENANT, principalId: OTHER, query: "visible", diff --git a/src/ports/fakes.ts b/src/ports/fakes.ts index 6fea14f..8d63414 100644 --- a/src/ports/fakes.ts +++ b/src/ports/fakes.ts @@ -3,7 +3,7 @@ * Enough for hosts/tests to mount without Postgres or embed endpoints. * * Document access: creator always sees own docs; otherwise any accessTag that - * authorize(grants, …, tag, "find") allows when grants are provided. Without + * authorize(grants, …, tag, "search") allows when grants are provided. Without * grants, only creator access (safe default for unit tests). */ import { canAccessDocument } from "../grant-tags.ts"; @@ -11,12 +11,11 @@ import { canAccessDocument } from "../grant-tags.ts"; import type { DocumentStore, DocumentStoreAddParams, - DocumentStoreFindParams, - DocumentStoreFindResult, - DocumentStoreRecentEvent, - DocumentStoreRecentParams, + DocumentStoreSearchParams, + DocumentStoreSearchResult, + DocumentStoreListEvent, + DocumentStoreListParams, LiveSearchItem, - MemoryProvider, SourceProvider, } from "./types.ts"; @@ -36,8 +35,8 @@ async function visibleTo( params: { principalId: string; tenantId: string; - grants?: DocumentStoreFindParams["grants"]; - conditionRegistry?: DocumentStoreFindParams["conditionRegistry"]; + grants?: DocumentStoreSearchParams["grants"]; + conditionRegistry?: DocumentStoreSearchParams["conditionRegistry"]; }, ): Promise { if (!params.grants) { @@ -90,9 +89,9 @@ export function createFakeDocumentStore(): DocumentStore { return { documentId }; }, - async find( - params: DocumentStoreFindParams, - ): Promise { + async search( + params: DocumentStoreSearchParams, + ): Promise { const limit = params.limit ?? 8; const scored: { d: StoredDoc; score: number }[] = []; for (const d of docs) { @@ -131,11 +130,11 @@ export function createFakeDocumentStore(): DocumentStore { return { items }; }, - async recent( - params: DocumentStoreRecentParams, - ): Promise { + async list( + params: DocumentStoreListParams, + ): Promise { const limit = params.limit ?? 50; - const out: DocumentStoreRecentEvent[] = []; + const out: DocumentStoreListEvent[] = []; const sorted = [...docs] .filter((d) => d.tenantId === params.tenantId) .sort((a, b) => (a.createdAt < b.createdAt ? 1 : -1)); @@ -183,29 +182,3 @@ export function createFakeSourceProvider( }, }; } - -/** - * In-memory MemoryProvider for tests. - */ -export function createFakeMemoryProvider(): MemoryProvider { - const byKey = new Map(); - const key = (tenantId: string, principalId: string) => - `${tenantId}::${principalId}`; - return { - async remember(params) { - const k = key(params.tenantId, params.principalId); - const list = byKey.get(k) ?? []; - list.push(params.text); - byKey.set(k, list); - }, - async recall(params) { - const list = byKey.get(key(params.tenantId, params.principalId)) ?? []; - const q = params.query.toLowerCase(); - const limit = params.limit ?? 5; - return list - .filter((t) => !q || t.toLowerCase().includes(q)) - .slice(0, limit) - .map((text) => ({ text, score: 1 })); - }, - }; -} diff --git a/src/ports/index.ts b/src/ports/index.ts index e07a1b3..c8addc8 100644 --- a/src/ports/index.ts +++ b/src/ports/index.ts @@ -2,11 +2,9 @@ export type { DocumentStore, DocumentStoreAddParams, LiveSearchItem, - MemoryProvider, SourceProvider, } from "./types.ts"; export { createFakeDocumentStore, - createFakeMemoryProvider, createFakeSourceProvider, } from "./fakes.ts"; diff --git a/src/ports/memory-plane.test.ts b/src/ports/memory-plane.test.ts index 7daa78d..f945311 100644 --- a/src/ports/memory-plane.test.ts +++ b/src/ports/memory-plane.test.ts @@ -1,6 +1,6 @@ /** - * M3 MemoryProvider product wire: remember/recall, includeMemory on ask, - * degrade on failure, default includeMemory=false. + * Product path: DocumentStore-backed plane (add / search / list). + * MemoryProvider / ask / remember / recall are not on the product surface. */ import { describe, expect, it } from "bun:test"; import { @@ -10,11 +10,8 @@ import { import { createFakeDocumentStore, - createFakeMemoryProvider, createMemory, - MemoryError, } from "../index.ts"; -import type { MemoryProvider } from "./types.ts"; const TENANT = "t_mem"; const PRINCIPAL = "p_mem"; @@ -33,156 +30,51 @@ function grant(action: string): GrantRule { }; } -describe("MemoryProvider product wire (CL-5228)", () => { - it("includeMemory defaults false — ask does not call recall", async () => { - let recallCalls = 0; - const memory: MemoryProvider = { - async remember() {}, - async recall() { - recallCalls += 1; - return [{ text: "should not appear" }]; - }, - }; - const store = createFakeDocumentStore(); +describe("DocumentStore product plane", () => { + it("add + search + list round-trip on fakes", async () => { const plane = createMemory({ - grantStore: createInMemoryGrantStore([grant("find")]), + grantStore: createInMemoryGrantStore([ + grant("add"), + grant("search"), + ]), conditionRegistry: {}, - documentStore: store, - memoryProvider: memory, - generate: async (msgs) => { - const last = msgs[msgs.length - 1]?.content ?? ""; - return last.includes("Personal memory") ? "HAS_MEM" : "NO_MEM"; - }, - }); - await plane.add({ - tenantId: TENANT, - principalId: PRINCIPAL, - content: { title: "doc", text: "document body" }, + documentStore: createFakeDocumentStore(), }); - const ans = await plane.ask({ + const { documentId } = await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, - query: "document", + content: { title: "doc", text: "document body about widgets" }, }); - expect(recallCalls).toBe(0); - expect(ans.text).toBe("NO_MEM"); - await plane.close(); - }); - - it("includeMemory true injects recalled texts into generate context", async () => { - const memory = createFakeMemoryProvider(); - await memory.remember({ + const found = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, - text: "user prefers dark mode", - }); - const store = createFakeDocumentStore(); - const plane = createMemory({ - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - documentStore: store, - memoryProvider: memory, - generate: async (msgs) => { - const last = msgs[msgs.length - 1]?.content ?? ""; - return last.includes("user prefers dark mode") - ? "saw-memory" - : "missed"; - }, + query: "widgets", + includeEvidence: true, }); - await plane.add({ + expect(found.items.some((i) => i.documentId === documentId)).toBe(true); + const listed = await plane.list({ tenantId: TENANT, principalId: PRINCIPAL, - content: { title: "prefs", text: "settings doc" }, }); - const ans = await plane.ask({ - tenantId: TENANT, - principalId: PRINCIPAL, - query: "dark", - includeMemory: true, - }); - expect(ans.text).toBe("saw-memory"); + expect(listed.some((e) => e.title === "doc")).toBe(true); await plane.close(); }); - it("memory recall failure degrades with memory_unavailable", async () => { - const memory: MemoryProvider = { - async remember() {}, - async recall() { - throw new Error("vendor down"); - }, - }; - const store = createFakeDocumentStore(); - const plane = createMemory({ - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - documentStore: store, - memoryProvider: memory, - generate: async () => "docs-only [1]", - }); - await plane.add({ - tenantId: TENANT, - principalId: PRINCIPAL, - content: { title: "d", text: "still answerable" }, - }); - const ans = await plane.ask({ - tenantId: TENANT, - principalId: PRINCIPAL, - query: "still answerable", - includeMemory: true, - }); - expect(ans.text).toContain("docs-only"); - expect(ans.degraded).toContain("memory_unavailable"); - await plane.close(); - }); - - it("plane.remember writes; plane.recall reads", async () => { - const memory = createFakeMemoryProvider(); + it("search does not invent hits for other principals without grants", async () => { const plane = createMemory({ documentStore: createFakeDocumentStore(), - memoryProvider: memory, }); - await plane.remember({ - tenantId: TENANT, - principalId: PRINCIPAL, - text: "favorite color is blue", - }); - const items = await plane.recall({ + await plane.add({ tenantId: TENANT, principalId: PRINCIPAL, - query: "favorite color", - }); - expect(items.some((i) => i.text.includes("blue"))).toBe(true); - await plane.close(); - }); - - it("plane.remember without memory throws 501", async () => { - const plane = createMemory({ - documentStore: createFakeDocumentStore(), + content: { title: "secret", text: "private note" }, }); - try { - await plane.remember({ - tenantId: TENANT, - principalId: PRINCIPAL, - text: "x", - }); - expect.unreachable("should throw"); - } catch (err) { - expect(err).toBeInstanceOf(MemoryError); - expect((err as MemoryError).status).toBe(501); - } - await plane.close(); - }); - - it("plane.recall without memory returns empty", async () => { - const plane = createMemory({ - documentStore: createFakeDocumentStore(), - }); - const items = await plane.recall({ + const other = await plane.search({ tenantId: TENANT, - principalId: PRINCIPAL, - query: "anything", + principalId: "someone-else", + query: "private", }); - expect(items).toEqual([]); + expect(other.items).toHaveLength(0); await plane.close(); }); }); diff --git a/src/ports/merge-plane.test.ts b/src/ports/merge-plane.test.ts index d71724b..6b86e4b 100644 --- a/src/ports/merge-plane.test.ts +++ b/src/ports/merge-plane.test.ts @@ -2,10 +2,6 @@ * Plane-level merge: live fail-soft, source filter, prefer-local via store path. */ import { describe, expect, it } from "bun:test"; -import { - createInMemoryGrantStore, - type GrantRule, -} from "@intx/authz"; import { createFakeDocumentStore, @@ -17,20 +13,6 @@ import type { LiveSearchItem } from "./types.ts"; const TENANT = "t_merge"; const PRINCIPAL = "p_merge"; -function grant(action: string): GrantRule { - return { - id: `g-${action}`, - resource: "memory", - action, - effect: "allow", - origin: "role", - conditions: null, - expiresAt: null, - roleId: null, - principalId: PRINCIPAL, - }; -} - function liveHit( ref: string, title: string, @@ -69,7 +51,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { content: { title: "local note", text: "ports and merge together" }, }); - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "ports", @@ -92,7 +74,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { ], }); - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "ports foundation", @@ -119,7 +101,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { content: { title: "local ports", text: "ports foundation local" }, }); - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "ports foundation", @@ -162,7 +144,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { content: { title: "stable local", text: "always available body" }, }); - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "always available", @@ -213,7 +195,7 @@ describe("plane merge (MergeLocalLiveV1)", () => { // Re-add via plane so local is searchable with text match // (store already has the doc; find via store path uses substring) - const result = await plane.find({ + const result = await plane.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "collision payload", @@ -228,34 +210,4 @@ describe("plane merge (MergeLocalLiveV1)", () => { expect(hit?.snippet).toContain("local"); await plane.close(); }); - - it("ask still works when live source errors", async () => { - const store = createFakeDocumentStore(); - const plane = createMemory({ - grantStore: createInMemoryGrantStore([grant("find")]), - conditionRegistry: {}, - documentStore: store, - sources: [ - { - id: "broken", - searchLive: async () => { - throw new Error("boom"); - }, - }, - ], - generate: async () => "ok [1]", - }); - await plane.add({ - tenantId: TENANT, - principalId: PRINCIPAL, - content: { title: "q", text: "answer material" }, - }); - const ans = await plane.ask({ - tenantId: TENANT, - principalId: PRINCIPAL, - query: "answer material", - }); - expect(ans.text).toContain("ok"); - await plane.close(); - }); }); diff --git a/src/ports/mount-fakes.test.ts b/src/ports/mount-fakes.test.ts index 8a24f20..d0cf1ac 100644 --- a/src/ports/mount-fakes.test.ts +++ b/src/ports/mount-fakes.test.ts @@ -1,6 +1,6 @@ /** * Acceptance: a host can createMemory with only fakes and get working - * add / find / ask / recent — proves the port boundary is real. + * add / search / list — proves the port boundary is real. */ import { describe, expect, it } from "bun:test"; import { Hono } from "hono"; @@ -36,8 +36,6 @@ function grant(action: string): GrantRule { function appWithPrincipal() { const app = new Hono(); app.use("*", async (c, next) => { - // Interchange's tenant middleware puts both principal + tenant on the - // context; requireGrant reads tenant.id, our caller() reads principal. c.set("principal", { id: PRINCIPAL, tenantId: TENANT, @@ -63,7 +61,7 @@ function appWithPrincipal() { } describe("createMemory with fakes only", () => { - it("add → find → recent → ask without Postgres or embed config", async () => { + it("add → search → list without Postgres or embed config", async () => { const store = createFakeDocumentStore(); const sources = [ createFakeSourceProvider("linear", [ @@ -89,11 +87,13 @@ describe("createMemory with fakes only", () => { const app = appWithPrincipal(); const memory = createMemory({ app, - grantStore: createInMemoryGrantStore([grant("add"), grant("find")]), + grantStore: createInMemoryGrantStore([ + grant("add"), + grant("search"), + ]), conditionRegistry: {}, documentStore: store, sources, - generate: async () => "Answer from local store [1].", }); const { documentId } = await memory.add({ @@ -106,7 +106,7 @@ describe("createMemory with fakes only", () => { }); expect(documentId).toMatch(/^fake_doc_/); - const found = await memory.find({ + const found = await memory.search({ tenantId: TENANT, principalId: PRINCIPAL, query: "DocumentStore override", @@ -115,21 +115,12 @@ describe("createMemory with fakes only", () => { expect(found.items).toHaveLength(1); expect(found.items[0]?.documentId).toBe(documentId); - const recent = await memory.recent({ + const listed = await memory.list({ tenantId: TENANT, principalId: PRINCIPAL, }); - expect(recent.some((e) => e.title === "ports note")).toBe(true); + expect(listed.some((e) => e.title === "ports note")).toBe(true); - const asked = await memory.ask({ - tenantId: TENANT, - principalId: PRINCIPAL, - query: "DocumentStore override", - }); - expect(asked.text).toContain("Answer from local store"); - expect(asked.citations.length).toBeGreaterThan(0); - - // HTTP surface also works without engine config const addRes = await app.request("/api/memory/add", { method: "POST", headers: { "content-type": "application/json" }, @@ -137,23 +128,22 @@ describe("createMemory with fakes only", () => { title: "via http", text: "http path uses the same store", }), - }); expect(addRes.status).toBe(200); const addBody = (await addRes.json()) as { documentId: string }; expect(addBody.documentId).toMatch(/^fake_doc_/); - const findRes = await app.request("/api/memory/find", { + const searchRes = await app.request("/api/memory/search", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ query: "http path" }), }); - expect(findRes.status).toBe(200); - const findBody = (await findRes.json()) as { + expect(searchRes.status).toBe(200); + const searchBody = (await searchRes.json()) as { items: Array<{ documentId: string }>; }; expect( - findBody.items.some((i) => i.documentId === addBody.documentId), + searchBody.items.some((i) => i.documentId === addBody.documentId), ).toBe(true); await memory.close(); diff --git a/src/ports/types.ts b/src/ports/types.ts index dc1888f..8c3f785 100644 --- a/src/ports/types.ts +++ b/src/ports/types.ts @@ -1,14 +1,17 @@ /** - * Port contracts for pluggable storage, live sources, and optional personal memory. + * Port contracts for pluggable storage and live sources. * - * DocumentStore is the durable backend for add/find/recent (default: local + * DocumentStore is the durable backend for add/search/list (default: local * pgvector). Hosts replace it with any DocumentStore implementation or fakes — * no dual store. SourceProvider is tools-shaped live connectors (e.g. Linear), - * not a store. MemoryProvider is an optional ask side-channel (includeMemory); - * not how you swap backends. + * not a store. * * Document access uses Interchange grant tags (accessTags), not a mini-ACL. * See docs/AUTHZ-DOCUMENT-ACCESS.md. + * + * Inference (extract-on-add, synthesize answers) is host-owned and ephemeral — + * call your own model, then `add` / `search`. Core does not mount an agent or + * bake LLM into the write path. */ import type { GrantStore, ConditionRegistry } from "@intx/authz"; import type { @@ -34,7 +37,7 @@ export type DocumentStoreAddParams = { kind?: string; }; -export type DocumentStoreFindParams = { +export type DocumentStoreSearchParams = { tenantId: string; principalId: string; query: string; @@ -52,7 +55,7 @@ export type DocumentStoreFindParams = { conditionRegistry?: ConditionRegistry; }; -export type DocumentStoreFindItem = { +export type DocumentStoreSearchItem = { documentId: string; title: string; snippet: string; @@ -65,13 +68,13 @@ export type DocumentStoreFindItem = { updatedAt?: string; }; -export type DocumentStoreFindResult = { - items: DocumentStoreFindItem[]; +export type DocumentStoreSearchResult = { + items: DocumentStoreSearchItem[]; evidence?: SearchEvidence; degraded?: DegradeFlag[]; }; -export type DocumentStoreRecentParams = { +export type DocumentStoreListParams = { tenantId: string; principalId: string; limit?: number; @@ -79,7 +82,7 @@ export type DocumentStoreRecentParams = { conditionRegistry?: ConditionRegistry; }; -export type DocumentStoreRecentEvent = { +export type DocumentStoreListEvent = { at: string; title: string; source: string; @@ -95,10 +98,8 @@ export type DocumentStoreRecentEvent = { */ export type DocumentStore = { add(params: DocumentStoreAddParams): Promise<{ documentId: string }>; - find(params: DocumentStoreFindParams): Promise; - recent( - params: DocumentStoreRecentParams, - ): Promise; + search(params: DocumentStoreSearchParams): Promise; + list(params: DocumentStoreListParams): Promise; close(): Promise; }; @@ -133,24 +134,4 @@ export type SourceProvider = { }): Promise; }; -/** - * Optional personal-memory side channel for ask(includeMemory). Not a - * DocumentStore replacement — product backends for durable knowledge implement - * DocumentStore, not this port. - */ -export type MemoryProvider = { - remember(params: { - tenantId: string; - principalId: string; - text: string; - metadata?: Record; - }): Promise; - recall(params: { - tenantId: string; - principalId: string; - query: string; - limit?: number; - }): Promise>; -}; - export type { SearchHit }; diff --git a/src/routes/ask.ts b/src/routes/ask.ts deleted file mode 100644 index b1b136e..0000000 --- a/src/routes/ask.ts +++ /dev/null @@ -1,86 +0,0 @@ -import type { Hono } from "hono"; -import type { TenantEnv } from "@intx/hub-api"; -import { describeRoute, resolver, validator } from "hono-openapi"; -import { type } from "arktype"; - -import { formatCaughtError, log } from "../log.ts"; -import { - MemoryError, - MemoryNotPermittedError, -} from "../memory.ts"; -import type { RouteDeps } from "./deps.ts"; -import { caller, grantGuard, requirePrincipal } from "./deps.ts"; - -const AskRequest = type({ - query: "string >= 1", - "limit?": "1 <= number.integer <= 50", - "sources?": "string[]", - "includeMemory?": "boolean", -}); - -const AskResponse = type({ - text: "string", - citations: type({ - index: "number", - documentId: "string", - title: "string", - citation: "unknown", - }).array(), - evidence: "'strong'|'weak'|'none'", - "degraded?": "string[]", -}); - -export function mountAskRoute(app: Hono, deps: RouteDeps): void { - app.post( - "/api/memory/ask", - describeRoute({ - tags: ["memory"], - summary: "Answer a question from retrieved memory", - responses: { - 200: { - description: "Grounded answer with citations", - content: { - "application/json": { schema: resolver(AskResponse) }, - }, - }, - 400: { description: "Invalid query" }, - 401: { description: "No principal on the request context" }, - 403: { description: "Missing the memory:find grant" }, - 501: { description: "ask is not configured (no generate)" }, - 502: { description: "ask failed" }, - }, - }), - requirePrincipal(), - // Same capability as find — ask retrieves as the principal then synthesizes. - grantGuard(deps, "find"), - validator("json", AskRequest), - async (c) => { - const { query, limit, sources, includeMemory } = c.req.valid("json"); - const { scopeId, subjectId } = caller(c); - try { - const result = await deps.memory.ask({ - query, - tenantId: scopeId, - principalId: subjectId, - ...(limit !== undefined ? { limit } : {}), - ...(sources !== undefined ? { sources } : {}), - ...(includeMemory !== undefined ? { includeMemory } : {}), - }); - return c.json(result); - } catch (err) { - if (err instanceof MemoryNotPermittedError) { - return c.json({ error: err.message }, 403); - } - if (err instanceof MemoryError) { - return c.json( - { error: err.message }, - err.status as 400 | 501, - ); - } - const errMessage = formatCaughtError(err); - log.error(`memory ask failed: ${errMessage}`, { err }); - return c.json({ error: "ask failed" }, 502); - } - }, - ); -} diff --git a/src/routes/recent.ts b/src/routes/list.ts similarity index 68% rename from src/routes/recent.ts rename to src/routes/list.ts index 1f1fd28..e6674c3 100644 --- a/src/routes/recent.ts +++ b/src/routes/list.ts @@ -6,17 +6,17 @@ import { type } from "arktype"; import { formatCaughtError, log } from "../log.ts"; import { MemoryError, - RECENT_LIMIT_MAX, - RECENT_LIMIT_MIN, + LIST_LIMIT_MAX, + LIST_LIMIT_MIN, } from "../memory.ts"; import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; -const RecentQuery = type({ +const ListQuery = type({ "limit?": "string", }); -const RecentResponse = type({ +const ListResponse = type({ events: type({ at: "string", title: "string", @@ -31,36 +31,36 @@ function parseLimit(raw: string | undefined): number | undefined { const n = Number(raw); if ( !Number.isInteger(n) || - n < RECENT_LIMIT_MIN || - n > RECENT_LIMIT_MAX + n < LIST_LIMIT_MIN || + n > LIST_LIMIT_MAX ) { return undefined; } return n; } -export function mountRecentRoute(app: Hono, deps: RouteDeps): void { +export function mountListRoute(app: Hono, deps: RouteDeps): void { app.get( - "/api/memory/recent", + "/api/memory/list", describeRoute({ tags: ["memory"], - summary: "Recent documents for the caller's scope", + summary: "List recent documents for the caller's scope", responses: { 200: { - description: "Recent events visible to the caller", + description: "Events visible to the caller", content: { - "application/json": { schema: resolver(RecentResponse) }, + "application/json": { schema: resolver(ListResponse) }, }, }, 400: { description: "Invalid limit query param" }, 401: { description: "No principal on the request context" }, - 403: { description: "Missing the memory:find grant" }, - 502: { description: "Recent query failed" }, + 403: { description: "Missing the memory:search grant" }, + 502: { description: "List query failed" }, }, }), requirePrincipal(), - grantGuard(deps, "find"), - validator("query", RecentQuery), + grantGuard(deps, "search"), + validator("query", ListQuery), async (c) => { const { scopeId, subjectId } = caller(c); const rawLimit = c.req.valid("query").limit; @@ -71,14 +71,14 @@ export function mountRecentRoute(app: Hono, deps: RouteDeps): void { ) { return c.json( { - error: `limit must be an integer from ${RECENT_LIMIT_MIN} to ${RECENT_LIMIT_MAX}`, + error: `limit must be an integer from ${LIST_LIMIT_MIN} to ${LIST_LIMIT_MAX}`, }, 400, ); } const limit = parseLimit(rawLimit); try { - const events = await deps.memory.recent({ + const events = await deps.memory.list({ tenantId: scopeId, principalId: subjectId, ...(limit !== undefined ? { limit } : {}), @@ -89,10 +89,10 @@ export function mountRecentRoute(app: Hono, deps: RouteDeps): void { return c.json({ error: err.message }, err.status as 400); } const errMessage = formatCaughtError(err); - log.error(`memory recent failed: ${errMessage}`, { + log.error(`memory list failed: ${errMessage}`, { error: errMessage, }); - return c.json({ error: "recent failed" }, 502); + return c.json({ error: "list failed" }, 502); } }, ); diff --git a/src/routes/mount.ts b/src/routes/mount.ts index 4c8897e..34d38ae 100644 --- a/src/routes/mount.ts +++ b/src/routes/mount.ts @@ -8,19 +8,17 @@ import type { TenantEnv } from "@intx/hub-api"; import type { RouteDeps } from "./deps.ts"; import { mountAddRoute } from "./add.ts"; -import { mountFindRoute } from "./find.ts"; -import { mountAskRoute } from "./ask.ts"; -import { mountRecentRoute } from "./recent.ts"; +import { mountSearchRoute } from "./search.ts"; +import { mountListRoute } from "./list.ts"; export type { GrantConfig, RouteDeps } from "./deps.ts"; -/** HTTP JSON routes: add, find, ask, recent. */ +/** HTTP JSON routes: add, search, list. */ export function registerMemoryRoutes( app: Hono, deps: RouteDeps, ): void { mountAddRoute(app, deps); - mountFindRoute(app, deps); - mountAskRoute(app, deps); - mountRecentRoute(app, deps); + mountSearchRoute(app, deps); + mountListRoute(app, deps); } diff --git a/src/routes/routes.test.ts b/src/routes/routes.test.ts index d9b3cfb..51d055c 100644 --- a/src/routes/routes.test.ts +++ b/src/routes/routes.test.ts @@ -6,10 +6,8 @@ import { createRequireGrant, type TenantEnv } from "@intx/hub-api"; import type { Memory, TimelineEvent } from "../memory.ts"; import { registerMemoryRoutes } from "./mount.ts"; - import type { RouteDeps } from "./deps.ts"; - function grant(principalId: string, action: string): GrantRule { return { id: `g-${principalId}-${action}`, @@ -25,20 +23,14 @@ function grant(principalId: string, action: string): GrantRule { } const PRINCIPAL = "p1"; -const OTHER = "p2"; const TENANT = "t1"; - const SECRET_TITLE = "Q3 layoffs — draft list"; const PUBLIC_TITLE = "team standup notes"; -// A knowledge plane stub that records adds and returns fixed results. -// Recent applies a simple ACL model so route tests can prove the route -// never invents titles and always scopes by the caller's principal. function stubPlane(opts?: { timelineCatalog?: Array< TimelineEvent & { visibleTo: readonly string[] | "tenant" } >; - askImpl?: Memory["ask"]; }) { const added: { title: string; tenantId: string; principalId: string }[] = []; const searched: Array<{ @@ -48,13 +40,10 @@ function stubPlane(opts?: { }> = []; const catalog = opts?.timelineCatalog ?? []; const plane: Memory = { - find: async (p) => { + search: async (p) => { searched.push({ kinds: p.kinds, entityIds: p.entityIds, limit: p.limit }); return { items: [], evidence: "none" }; }, - ask: - opts?.askImpl ?? - (async () => ({ text: "stub answer", citations: [], evidence: "none" })), add: async (p) => { added.push({ title: p.content?.title ?? "", @@ -63,7 +52,7 @@ function stubPlane(opts?: { }); return { documentId: "doc-stub" }; }, - recent: async (p) => { + list: async (p) => { return catalog .filter( (e) => @@ -72,8 +61,6 @@ function stubPlane(opts?: { ) .map(({ visibleTo: _v, ...event }) => event); }, - remember: async () => {}, - recall: async () => [], close: async () => {}, }; return { plane, added, searched }; @@ -86,7 +73,6 @@ function buildApp( TimelineEvent & { visibleTo: readonly string[] | "tenant" } >; principalId?: string; - askImpl?: Memory["ask"]; }, ) { const { plane, added, searched } = stubPlane(opts); @@ -103,8 +89,6 @@ function buildApp( const principalId = opts?.principalId ?? PRINCIPAL; const app = new Hono(); app.use("*", async (c, next) => { - // Interchange's tenant middleware puts both principal + tenant on the - // context; requireGrant reads tenant.id, our caller() reads principal. c.set("principal", { id: principalId, tenantId: TENANT, @@ -130,8 +114,6 @@ function buildApp( return { app, added, searched }; } -// Mirrors a host that mounts the knowledge routes outside the tenant prefix -// Interchange's middleware covers — no principal ever lands on the context. function buildAppWithoutPrincipal() { const { plane } = stubPlane(); const grantConfig = { @@ -154,7 +136,7 @@ const jsonPost = (body: unknown) => ({ body: JSON.stringify(body), }); -const RECENT_CATALOG: Array< +const LIST_CATALOG: Array< TimelineEvent & { visibleTo: readonly string[] | "tenant" } > = [ { @@ -171,16 +153,15 @@ const RECENT_CATALOG: Array< source: "mcp", tenantId: TENANT, principalId: "alice", - // Private to alice only — p1 must never see this title on the wire. visibleTo: ["alice"], }, ]; -describe("knowledge HTTP routes", () => { +describe("memory HTTP routes", () => { test("add with the add grant writes under the caller's scope", async () => { const { app, added } = buildApp([ grant(PRINCIPAL, "add"), - grant(PRINCIPAL, "find"), + grant(PRINCIPAL, "search"), ]); const res = await app.request( "/api/memory/add", @@ -195,7 +176,7 @@ describe("knowledge HTTP routes", () => { }); test("add without the add grant is 403", async () => { - const { app, added } = buildApp([grant(PRINCIPAL, "find")]); + const { app, added } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( "/api/memory/add", jsonPost({ title: "t", text: "body" }), @@ -223,10 +204,10 @@ describe("knowledge HTTP routes", () => { expect(res.status).toBe(400); }); - test("find with the find grant returns a result", async () => { - const { app } = buildApp([grant(PRINCIPAL, "find")]); + test("search with the search grant returns a result", async () => { + const { app } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hello" }), ); expect(res.status).toBe(200); @@ -238,37 +219,37 @@ describe("knowledge HTTP routes", () => { expect(body.evidence).toBe("none"); }); - test("find requires the find grant", async () => { + test("search requires the search grant", async () => { const { app } = buildApp([grant(PRINCIPAL, "add")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hi" }), ); expect(res.status).toBe(403); }); - test("legacy search grant does not authorize find", async () => { - const { app } = buildApp([grant(PRINCIPAL, "search")]); + test("legacy find grant does not authorize search", async () => { + const { app } = buildApp([grant(PRINCIPAL, "find")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hi" }), ); expect(res.status).toBe(403); }); - test("find rejects out-of-range limit (400)", async () => { - const { app } = buildApp([grant(PRINCIPAL, "find")]); + test("search rejects out-of-range limit (400)", async () => { + const { app } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hi", limit: 999 }), ); expect(res.status).toBe(400); }); - test("find threads kinds and entity_ids through to the plane", async () => { - const { app, searched } = buildApp([grant(PRINCIPAL, "find")]); + test("search threads kinds and entity_ids through to the plane", async () => { + const { app, searched } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hello", kinds: ["artifact", "task"], @@ -285,10 +266,10 @@ describe("knowledge HTTP routes", () => { ]); }); - test("find with no kinds/entity_ids leaves them unset on the plane call", async () => { - const { app, searched } = buildApp([grant(PRINCIPAL, "find")]); + test("search with no kinds/entity_ids leaves them unset on the plane call", async () => { + const { app, searched } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hello" }), ); expect(res.status).toBe(200); @@ -297,21 +278,19 @@ describe("knowledge HTTP routes", () => { ]); }); - test("find rejects a non-string-array kinds (400)", async () => { - const { app } = buildApp([grant(PRINCIPAL, "find")]); + test("search rejects a non-string-array kinds (400)", async () => { + const { app } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hi", kinds: [1, 2] }), ); expect(res.status).toBe(400); }); - // The route does not collapse [] to absent — hybridSearch treats an empty - // array and an absent field the same way (see services/search.ts). - test("find passes an empty kinds/entity_ids array through unchanged", async () => { - const { app, searched } = buildApp([grant(PRINCIPAL, "find")]); + test("search passes an empty kinds/entity_ids array through unchanged", async () => { + const { app, searched } = buildApp([grant(PRINCIPAL, "search")]); const res = await app.request( - "/api/memory/find", + "/api/memory/search", jsonPost({ query: "hello", kinds: [], entity_ids: [] }), ); expect(res.status).toBe(200); @@ -320,141 +299,41 @@ describe("knowledge HTTP routes", () => { ]); }); - test("ask requires the find grant (same as find)", async () => { - const { app } = buildApp([grant(PRINCIPAL, "add")]); - const res = await app.request( - "/api/memory/ask", - jsonPost({ query: "what?" }), - ); - expect(res.status).toBe(403); - }); - - test("ask with the find grant returns a grounded answer", async () => { - const { app } = buildApp([grant(PRINCIPAL, "find")]); - const res = await app.request( - "/api/memory/ask", - jsonPost({ query: "what?" }), - ); - expect(res.status).toBe(200); - const body = (await res.json()) as { - text: string; - citations: unknown[]; - evidence: string; - }; - expect(body.text).toBe("stub answer"); - expect(body.citations).toEqual([]); - expect(body.evidence).toBe("none"); - }); - - test("recent requires the find grant", async () => { + test("list requires the search grant", async () => { const { app } = buildApp([grant(PRINCIPAL, "add")]); - const res = await app.request("/api/memory/recent"); + const res = await app.request("/api/memory/list"); expect(res.status).toBe(403); }); - test("recent never returns a title private to another principal", async () => { - const { app } = buildApp([grant(PRINCIPAL, "find")], { - timelineCatalog: RECENT_CATALOG, + test("list never returns a title private to another principal", async () => { + const { app } = buildApp([grant(PRINCIPAL, "search")], { + timelineCatalog: LIST_CATALOG, principalId: PRINCIPAL, }); - const res = await app.request("/api/memory/recent"); + const res = await app.request("/api/memory/list"); expect(res.status).toBe(200); const body = (await res.json()) as { events: TimelineEvent[] }; expect(body.events.map((e) => e.title)).toEqual([PUBLIC_TITLE]); expect(body.events.map((e) => e.title)).not.toContain(SECRET_TITLE); }); - test("recent returns a private title only to the allowed principal", async () => { - const { app } = buildApp([grant("alice", "find")], { - timelineCatalog: RECENT_CATALOG, + test("list returns a private title only to the allowed principal", async () => { + const { app } = buildApp([grant("alice", "search")], { + timelineCatalog: LIST_CATALOG, principalId: "alice", }); - const res = await app.request("/api/memory/recent"); + const res = await app.request("/api/memory/list"); expect(res.status).toBe(200); const body = (await res.json()) as { events: TimelineEvent[] }; - expect(body.events.map((e) => e.title).sort()).toEqual( - [PUBLIC_TITLE, SECRET_TITLE].sort(), - ); - }); - - test("recent scopes by the caller's principal (different principal → different events)", async () => { - const { app: appP1 } = buildApp([grant(PRINCIPAL, "find")], { - timelineCatalog: RECENT_CATALOG, - principalId: PRINCIPAL, - }); - const { app: appOther } = buildApp([grant(OTHER, "find")], { - timelineCatalog: RECENT_CATALOG, - principalId: OTHER, - }); - const titles = async (app: Hono) => { - const res = await app.request("/api/memory/recent"); - expect(res.status).toBe(200); - const body = (await res.json()) as { events: TimelineEvent[] }; - return body.events.map((e) => e.title); - }; - expect(await titles(appP1)).toEqual([PUBLIC_TITLE]); - expect(await titles(appOther)).toEqual([PUBLIC_TITLE]); - expect(await titles(appP1)).not.toContain(SECRET_TITLE); - expect(await titles(appOther)).not.toContain(SECRET_TITLE); - }); - - test("old paths are not mounted (hard cutover, no fallback)", async () => { - const { app } = buildApp([ - grant(PRINCIPAL, "add"), - grant(PRINCIPAL, "find"), - grant(PRINCIPAL, "capture"), - grant(PRINCIPAL, "search"), - ]); - for (const path of [ - "/api/memory/capture", - "/api/memory/search", - "/api/memory/timeline", - ]) { - const method = path.endsWith("timeline") ? "GET" : "POST"; - const res = await app.request( - path, - method === "GET" - ? undefined - : jsonPost( - path.includes("capture") - ? { title: "t", text: "body" } - : { query: "hi" }, - ), - ); - expect(res.status).toBe(404); - } + expect(body.events.map((e) => e.title)).toContain(SECRET_TITLE); }); - test("add is rejected (401) when no principal is on the context", async () => { + test("missing principal is 401", async () => { const app = buildAppWithoutPrincipal(); const res = await app.request( - "/api/memory/add", - jsonPost({ title: "t", text: "body" }), - ); - expect(res.status).toBe(401); - }); - - test("find is rejected (401) when no principal is on the context", async () => { - const app = buildAppWithoutPrincipal(); - const res = await app.request( - "/api/memory/find", - jsonPost({ query: "hello" }), - ); - expect(res.status).toBe(401); - }); - - test("ask is rejected (401) when no principal is on the context", async () => { - const app = buildAppWithoutPrincipal(); - const res = await app.request( - "/api/memory/ask", - jsonPost({ query: "hello" }), + "/api/memory/search", + jsonPost({ query: "hi" }), ); expect(res.status).toBe(401); }); - - test("recent is rejected (401) when no principal is on the context", async () => { - const app = buildAppWithoutPrincipal(); - const res = await app.request("/api/memory/recent"); - expect(res.status).toBe(401); - }); }); diff --git a/src/routes/find.ts b/src/routes/search.ts similarity index 79% rename from src/routes/find.ts rename to src/routes/search.ts index 70278df..df0b36f 100644 --- a/src/routes/find.ts +++ b/src/routes/search.ts @@ -9,13 +9,13 @@ import type { RouteDeps } from "./deps.ts"; import { caller, grantGuard, requirePrincipal } from "./deps.ts"; // `kinds`/`entity_ids` scope every retrieval channel — see the -// `kinds`/`entityIds` doc comments on MemoryFindParams (memory.ts) +// `kinds`/`entityIds` doc comments on MemorySearchParams (memory.ts) // for the full explanation. // // An empty array on either field is equivalent to omitting it — "no filter" // — not "match nothing", and does not satisfy the requirement that an empty // `query` be paired with a non-empty structured filter. -const FindRequest = type({ +const SearchRequest = type({ query: "string >= 1", "limit?": "1 <= number.integer <= 50", "kinds?": "string[]", @@ -24,7 +24,7 @@ const FindRequest = type({ "includeEvidence?": "boolean", }); -const FindResponse = type({ +const SearchResponse = type({ items: type({ documentId: "string", title: "string", @@ -37,12 +37,12 @@ const FindResponse = type({ "degraded?": "string[]", }); -export function mountFindRoute(app: Hono, deps: RouteDeps): void { +export function mountSearchRoute(app: Hono, deps: RouteDeps): void { app.post( - "/api/memory/find", + "/api/memory/search", describeRoute({ tags: ["memory"], - summary: "Hybrid semantic + keyword find", + summary: "Hybrid semantic + keyword search", description: "`kinds`/`entity_ids` scope every retrieval channel (lexical and " + "dense) before results are fused, so every hit matches the " + @@ -52,24 +52,24 @@ export function mountFindRoute(app: Hono, deps: RouteDeps): void { 200: { description: "Ranked items with evidence", content: { - "application/json": { schema: resolver(FindResponse) }, + "application/json": { schema: resolver(SearchResponse) }, }, }, 400: { description: "Invalid query" }, 401: { description: "No principal on the request context" }, - 403: { description: "Missing the memory:find grant" }, - 502: { description: "find failed" }, + 403: { description: "Missing the memory:search grant" }, + 502: { description: "search failed" }, }, }), requirePrincipal(), - grantGuard(deps, "find"), - validator("json", FindRequest), + grantGuard(deps, "search"), + validator("json", SearchRequest), async (c) => { const { query, limit, kinds, entity_ids, sources, includeEvidence } = c.req.valid("json"); const { scopeId, subjectId } = caller(c); try { - const result = await deps.memory.find({ + const result = await deps.memory.search({ query, tenantId: scopeId, principalId: subjectId, @@ -84,11 +84,11 @@ export function mountFindRoute(app: Hono, deps: RouteDeps): void { return c.json(result); } catch (err) { const errMessage = formatCaughtError(err); - log.error(`memory find failed: ${errMessage}`, { err }); + log.error(`memory search failed: ${errMessage}`, { err }); if (err instanceof MemoryError) { return c.json({ error: err.message }, err.status as 400); } - return c.json({ error: "find failed" }, 502); + return c.json({ error: "search failed" }, 502); } }, ); diff --git a/src/services/timeline.test.ts b/src/services/timeline.test.ts index 8be2e80..b623560 100644 --- a/src/services/timeline.test.ts +++ b/src/services/timeline.test.ts @@ -53,7 +53,7 @@ describe("filterTimelineRows (grant-tag access)", () => { id: "g1", principalId: "peer", resource: "memory.tenant:t1", - action: "find", + action: "search", effect: "allow", origin: "role", conditions: null, diff --git a/src/services/timeline.ts b/src/services/timeline.ts index 16a14a6..d88c2c7 100644 --- a/src/services/timeline.ts +++ b/src/services/timeline.ts @@ -1,8 +1,8 @@ /** * Timeline / recent — tenant-scoped document history. * - * Document access is Interchange grant tags (same as find): creator always - * sees own docs; otherwise any accessTag that authorize(..., tag, "find") + * Document access is Interchange grant tags (same as search): creator always + * sees own docs; otherwise any accessTag that authorize(..., tag, "search") * allows. No visibility SQL, no acl_block post-filter. * * See docs/AUTHZ-DOCUMENT-ACCESS.md. From f29d62130faf7849634361c71e00c81e052c60bb Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 21:27:15 -0700 Subject: [PATCH 10/11] Align docs with add/search/list public surface Scrub PRODUCT, MIGRATION, ARCHITECTURE, CHANGELOG, IMPLEMENTATION, AGENTS, and AUTHZ-DOCUMENT-ACCESS so grants, HTTP paths, and plane verbs match the hard cutover (no ask/find/recent product path). --- AGENTS.md | 4 ++-- ARCHITECTURE.md | 21 +++++++++------------ CHANGELOG.md | 32 +++++++++++++++++++------------- IMPLEMENTATION.md | 14 +++++++------- MIGRATION.md | 30 ++++++++++++++++-------------- PRODUCT.md | 31 +++++++++++++++---------------- docs/AUTHZ-DOCUMENT-ACCESS.md | 10 +++++----- src/config.ts | 2 +- src/memory.test.ts | 2 +- 9 files changed, 75 insertions(+), 71 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9562ac5..f22d6b6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -21,9 +21,9 @@ CI runs `typecheck` + `test` — both must pass before any push. - `src/index.ts` — public surface: `createMemory` (optional `app` registers HTTP), `registerMemoryRoutes` - `src/mount-config.ts` / `src/config.ts` — mount config + engine config -- `src/routes/` — Hono routes (`add`, `find`, `ask`, `recent`) +- `src/routes/` — Hono routes (`add`, `search`, `list`) - `src/services/` — capture / search / transform internals (not public verbs) -- `src/ports/` — `DocumentStore` / `SourceProvider` / `MemoryProvider` + fakes +- `src/ports/` — `DocumentStore` / `SourceProvider` + fakes - `src/core/` — embed/rerank clients, merge, arktype schemas - `src/db/` + `migrations/` — Drizzle schema + SQL migrations (pgvector, `knowledge.*`) - `packages/` — removed; DocumentStore adapters and Linear tools are sibling packages diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d5b60a8..c5ffcb7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -67,21 +67,17 @@ trust model. `createMemory({ app })` adds, under the host app: - - `POST /api/memory/add` — ingest a note (raw + derive). -- `POST /api/memory/find` — hybrid retrieval: FTS + dense (pgvector) → RRF +- `POST /api/memory/search` — hybrid retrieval: FTS + dense (pgvector) → RRF fusion → cross-encoder rerank → bounded authority/recency boosts → MMR; optional live `SourceProvider` merge (fail-soft). -- `POST /api/memory/ask` — grant-checked as `memory:find`; retrieves as - the principal, grounds a prompt from hit snippets, calls host-injected - `generate`. Optional memory recall when `includeMemory` is true. -- `GET /api/memory/recent` — recent documents for the caller's scope, - filtered with the same grant-tag access as local find (`canAccessDocument`). +- `GET /api/memory/list` — recent documents for the caller's scope, + filtered with the same grant-tag access as local search (`canAccessDocument`). -It also returns an in-process `Memory` (`add`, `find`, `ask`, -`recent`, optional `remember` / `recall`). `ask()` is grant-checked in-process -(callers bypass the HTTP `requireGrant` guard). The library owns no generation -client; hosts wire `generate` to their inference layer. +It also returns an in-process `Memory` (`add`, `search`, `list`, `close`). +There is no product `ask` / `remember` / `recall` and no host-injected +`generate` on the plane — inference is host-owned and ephemeral (call your +model, then `add` / `search`). MCP is not part of this package — mount `@corbitsdev/hono-openapi-mcp` to expose these routes as MCP tools. @@ -90,7 +86,8 @@ External ingestion (Linear, GitHub, …) is not a route here — the host authenticates the forwarder to Interchange and calls `plane.add` / a `SourceProvider` mapper, or mounts HTTP add after its own auth. -Legacy paths `/capture`, `/search`, `/timeline` are not mounted (hard cutover). +Legacy paths `/capture`, `/search` (old knowledge), `/timeline`, `/find`, +`/ask`, `/recent` are not mounted (hard cutover). ## Provenance diff --git a/CHANGELOG.md b/CHANGELOG.md index 451bacd..060cc4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,35 +15,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `createMemory`, `loadMemoryConfig`, `runMemoryMigrations`, `Memory`, `MemoryConfig`, `MemoryError`. HTTP paths are under `/api/memory/`; grants are - `memory:add` / `memory:find`; access tags use `memory.owner:` / `memory.tenant:` + `memory:add` / `memory:search`; access tags use `memory.owner:` / `memory.tenant:` / `memory.space:`. Postgres schema name remains `knowledge`. See `MIGRATION.md`. -- **Breaking:** memory plane surface is `add` / `find` / `ask` / `recent` with - `principalId` + `tenantId` only (`capture` / `search` / `timeline` and - `subjectId` / `scopeId` removed). See `MIGRATION.md`. +- **Breaking:** memory plane surface is `add` / `search` / `list` with + `principalId` + `tenantId` only. Removed product verbs: `find` (→`search`), + `recent` (→`list`), `ask`, `remember`, `recall`, and any `MemoryProvider` / + `generate` path. Inference is host-owned. See `MIGRATION.md`. - **Breaking:** HTTP routes are `POST /api/memory/add`, - `POST /api/memory/find`, `POST /api/memory/ask`, - `GET /api/memory/recent`. Old paths are not mounted. -- **Breaking:** grant actions are `add` and `find` (was `capture` / `search`). - `ask` and `recent` use the `find` grant. Capability resource is `memory`. -- **Breaking:** `add` returns `{ documentId }`; find body uses `limit` (not `k`); - find wire uses `items` (not `hits`). + `POST /api/memory/search`, `GET /api/memory/list`. Old paths are not mounted. +- **Breaking:** grant actions are `add` and `search` (was `capture` / `find` / + knowledge `search`). `list` uses the `search` grant. Capability resource is + `memory`. Document-tag checks use action `search`. +- **Breaking:** `add` returns `{ documentId }`; search body uses `limit` (not `k`); + search wire uses `items` (not `hits`). - **Breaking:** document access is Interchange **grant tags** (`accessTags` + creator-always + host `GrantStore`), not the visibility-mode / block-list mini-ACL. Share sugar only mints tags. See `docs/AUTHZ-DOCUMENT-ACCESS.md`. - **Breaking:** Postgres baseline is two files (`0001_extensions` + `0002_knowledge_baseline`) with `access_tags` and no `visibility_*` columns. Fresh installs only — drop/recreate the knowledge schema on existing DBs. -- **Breaking:** MemoryProvider side-channel option is `options.memoryProvider` - (was `options.memory`). +- **Breaking:** `grantStore` + `conditionRegistry` are top-level `createMemory` + options (no nested `grants: { … }`). ### Added - Optional `TextExtractor` + `file` XOR `content` on `add` - `share` sugar on `add` (maps to access tags only: owner, tenant, peers) - `access_tags` on `knowledge.document` (baseline schema; Postgres schema name unchanged) -- `POST /api/memory/ask` HTTP route - `MIGRATION.md` hard-cutover notes for in-repo consumers +### Removed + +- Product `ask` / `remember` / `recall` and `MemoryProvider` side-channel +- Host-injected `generate` on the plane (use host inference + `add` / `search`) +- HTTP `POST /api/memory/ask`, `POST /api/memory/find`, `GET /api/memory/recent` + ## [0.1.2] — 2026-07-31 ### Added diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index bc59a68..93da8b7 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -12,17 +12,17 @@ src/ mount-config.ts # MemoryConfig + loadMemoryConfig() — the mount config config.ts # EngineConfig — the core vector-plane config (db + embed + rerank) - memory.ts # createMemory — add/find/ask/recent against store or pgvector + memory.ts # createMemory — add/search/list against store or pgvector grant-tags.ts # resolveAccessTags + canAccessDocument (host grants) log.ts # getLogger(["memory"]) from @intx/log migrations.ts # runMemoryMigrations(url) - ports/ # DocumentStore / SourceProvider / MemoryProvider + fakes + ports/ # DocumentStore / SourceProvider + fakes routes/ # the mounted routes mount.ts # registerMemoryRoutes (HTTP) deps.ts # RouteDeps, caller(c) (context identity), grantGuard - add.ts, find.ts, ask.ts, recent.ts + add.ts, search.ts, list.ts db/ schema.ts # Drizzle table defs (knowledge.* schema) client.ts # createDb(config) -> { db (drizzle), sql (raw postgres-js) } @@ -487,11 +487,11 @@ Each route is guarded with `grantGuard(deps, action)`, which applies the host's | Method + path | Grant action | Request body | Response | |---|---|---|---| | `POST /api/memory/add` | `add` | `{ title, text, access_tags?, share? }` | `200 { documentId }`; `400` on validation | -| `POST /api/memory/find` | `find` | `{ query, limit?, kinds?, entity_ids? }` (limit 1–50; `kinds`/`entity_ids` narrow every retrieval channel — lexical and dense — to a document `kind` or linked entity id before fusion; unset or `[]` = unfiltered) | `200 { items[], evidence?, degraded? }`; `400` on bad input | -| `POST /api/memory/ask` | `find` | `{ query, limit? }` (1–50) | `200 { text, citations[], evidence }`; `403` / `501` as plane errors | -| `GET /api/memory/recent` | `find` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope, filtered with grant-tag access (`canAccessDocument`). One event per document (active live version). | +| `POST /api/memory/search` | `search` | `{ query, limit?, kinds?, entity_ids? }` (limit 1–50; `kinds`/`entity_ids` narrow every retrieval channel — lexical and dense — to a document `kind` or linked entity id before fusion; unset or `[]` = unfiltered) | `200 { items[], evidence?, degraded? }`; `400` on bad input | +| `GET /api/memory/list` | `search` | — | `200 { events: [{ at, title, source, tenantId, principalId }] }` — durable recent documents for the caller's scope, filtered with grant-tag access (`canAccessDocument`). One event per document (active live version). | -`registerMemoryRoutes` and `createMemory({ app })` register the four HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). +`registerMemoryRoutes` and `createMemory({ app })` register the three HTTP routes. MCP is a separate package (`@corbitsdev/hono-openapi-mcp`). +There is no product `ask` / `remember` / `recall` HTTP or plane surface. ### Timeline wire fields (vs the old CaptureLog ring) diff --git a/MIGRATION.md b/MIGRATION.md index 8dc97a9..e7857f0 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -44,32 +44,33 @@ Identity fields on every call: | Was | Now | | --- | --- | | `POST /api/knowledge/capture` | `POST /api/memory/add` | -| `POST /api/knowledge/search` | `POST /api/memory/find` | -| `GET /api/knowledge/timeline` | `GET /api/memory/recent` | -| — | `POST /api/memory/ask` (new) | +| `POST /api/knowledge/search` | `POST /api/memory/search` | +| `GET /api/knowledge/timeline` | `GET /api/memory/list` | +| `POST /api/memory/find` | `POST /api/memory/search` | +| `GET /api/memory/recent` | `GET /api/memory/list` | +| `POST /api/memory/ask` | **removed** (host-owned inference) | Old paths return **404**. No redirect, no dual mount. ### Wire body / response deltas - **add** request: `{ title, text, access_tags?, share? }`. Response: `{ documentId }` (dropped `status: "captured"`). -- **find** request: `{ query, limit? }` (`k` is no longer accepted). Response: `{ items, evidence?, degraded? }` (was `{ hits, evidence, degraded? }`). -- **recent** response: unchanged `{ events: [...] }`. -- **ask** request: `{ query, limit? }`. Response: `{ text, citations, evidence }`. +- **search** request: `{ query, limit? }` (`k` is no longer accepted). Response: `{ items, evidence?, degraded? }` (was `{ hits, evidence, degraded? }`). +- **list** response: `{ events: [...] }` (same shape as old `recent`). +- **ask** / **remember** / **recall**: not product surface — host calls its model, then `add` / `search`. ## Grants | Was | Now | | --- | --- | | `requireGrant("knowledge", "capture")` | `requireGrant("memory", "add")` | -| `requireGrant("knowledge", "search")` | `requireGrant("memory", "find")` | +| `requireGrant("knowledge", "search")` | `requireGrant("memory", "search")` | +| `requireGrant("memory", "find")` | `requireGrant("memory", "search")` | -`find`, `ask`, and `recent` all require the **`find`** action on resource +`search` and `list` both require the **`search`** action on resource `memory`. Old resource/action names are not accepted — update grant rows in the host grant store before deploy. -In-process `ask()` also checks `memory` / `find` (was `knowledge` / `search`). - ## Package / public API rename (`@corbits/memory`) | Was | Now | @@ -78,12 +79,12 @@ In-process `ask()` also checks `memory` / `find` (was `knowledge` / `search`). | `mountKnowledgeEngine` | `createMemory({ app, … })` | | `mountKnowledgeRoutes` | `registerMemoryRoutes` (or `createMemory({ app })`) | | `mountMemory` / `mountMemoryRoutes` | `createMemory({ app })` / `registerMemoryRoutes` | - | `createKnowledgePlane` | `createMemory` | | `loadKnowledgeConfig` | `loadMemoryConfig` | | `runKnowledgeMigrations` | `runMemoryMigrations` | | `KnowledgePlane` / `KnowledgeConfig` / `KnowledgeError` | `Memory` / `MemoryConfig` / `MemoryError` | -| `options.memory` (MemoryProvider side-channel) | `options.memoryProvider` | +| plane verbs `find` / `recent` / `ask` | `search` / `list` / *(removed)* | +| `options.memory` / `options.memoryProvider` | **removed** (no MemoryProvider product path) | | Access tags `knowledge.owner:` / `knowledge.tenant:` / `knowledge.space:` | `memory.owner:` / `memory.tenant:` / `memory.space:` | Postgres schema name remains **`knowledge`** (tables such as `knowledge.document`). @@ -91,12 +92,13 @@ Postgres schema name remains **`knowledge`** (tables such as `knowledge.document ## Host checklist (this package's consumers) 1. Rename plane method calls and identity fields; switch package import to `@corbits/memory`. -2. Point HTTP clients at the new `/api/memory/*` paths and bodies. -3. Rewrite grant rules: resource `knowledge`→`memory`, `capture`→`add`, `search`→`find`. +2. Point HTTP clients at `/api/memory/add|search|list` (not find/ask/recent). +3. Rewrite grant rules: resource `knowledge`→`memory`, `capture`→`add`, `find`/`search`→`search`. 4. Drop any reliance on `status: "captured"` or `hits` / `k` on the wire. 5. If you use file capture, pass `textExtractor` into `createMemory` / mount options. 6. Document access is grant tags (`accessTags` + creator + host `GrantStore`), not visibility modes or block lists. Update any host code that wrote `visibility`. +7. Drop `ask` / `remember` / `recall` / `generate` / `memoryProvider` usage — host owns inference. 7. Fresh Postgres: baseline migrations are `0001_extensions.sql` + `0002_knowledge_baseline.sql` (schema `knowledge`, `access_tags` on document). Existing DBs: drop/recreate the knowledge schema (no in-place dual-write migration). diff --git a/PRODUCT.md b/PRODUCT.md index 16b3ed0..c6d8b45 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -97,14 +97,13 @@ Claude Code / Codex / Workbench (clients) │ Host Interchange createApp │ │ + createMemory({ app, … }) │ -│ grants: memory:add | memory:find │ +│ grants: memory:add | memory:search │ │ documentStore: pgvector | host store │ │ | fake │ -│ optional: sources, memoryProvider, │ -│ textExtractor │ +│ optional: sources, textExtractor │ │ │ in-process │ │ ▼ │ -│ Memory plane: add / find / ask / recent │ +│ Memory plane: add / search / list │ │ → DocumentStore (sole durable backend) │ └──────────────────────────────────────────────┘ ``` @@ -115,16 +114,16 @@ Memory does **not** ship a second ACL. Document access uses the host’s `@intx/authz` grant store — the same grants/roles as the rest of Interchange. 1. **Capability** — may this principal use memory at all? - `authorize(…, resource: "memory", action: "add" | "find")`. + `authorize(…, resource: "memory", action: "add" | "search")`. 2. **Document access** — each document carries **`accessTags`** (resource strings in grant-pattern space). A principal sees a document if they are the creator - **or** `authorize(…, resource: , action: "find")` allows for any tag on + **or** `authorize(…, resource: , action: "search")` allows for any tag on the document. Patterns (`memory.space:*`) work via `@intx/authz`. 3. **Share sugars on add** — only mint tags (owner / tenant / peer owner tags / explicit tags). They do **not** invent visibility modes or block lists. **Host contract:** peers named in `share.principals` only see the doc if the - host has granted them `find` on their owner tag (or matching pattern) — - typically bootstrap every principal with `find` on `memory.owner:`. + host has granted them `search` on their owner tag (or matching pattern) — + typically bootstrap every principal with `search` on `memory.owner:`. Tag minting is not grant minting. See `docs/AUTHZ-DOCUMENT-ACCESS.md`. Default add is **owner-only** (`memory.owner:` + creator rule). @@ -141,22 +140,22 @@ from the host principal context, never the body): ```http POST /api/memory/add { "title", "text", "access_tags"?, "share"? } -POST /api/memory/find { "query", "limit?", "kinds?", "entity_ids?", "sources?", "includeEvidence?" } -POST /api/memory/ask { "query", "limit?", "sources?", "includeMemory?" } -GET /api/memory/recent ?limit= +POST /api/memory/search { "query", "limit?", "kinds?", "entity_ids?", "sources?", "includeEvidence?" } +GET /api/memory/list ?limit= ``` -`kinds` / `entity_ids` on find narrow both lexical and dense channels before +`kinds` / `entity_ids` on search narrow both lexical and dense channels before fusion (unset or `[]` = no filter). -### Live sources and memory (trust) +### Live sources (trust) - **Local documents** are grant-tagged; default engine evaluates tags via the host `GrantStore`. An injected `DocumentStore` owns enforcement for that mount. -- **Live `SourceProvider` hits** merge into find/ask without grant tags. Auth is +- **Live `SourceProvider` hits** merge into search without grant tags. Auth is the host token / connector scope. Treat live as enrichment; fail-soft. -- **Memory** is opt-in recall (`includeMemory`, default false). Adapters must - key by injective tenant+principal encodings; ask never auto-writes memory. +- **Inference is host-owned.** Core does not ship `ask` / `remember` / `recall` + or a `MemoryProvider` product path. Hosts call their model, then `add` / + `search`. ## Out of scope forever here diff --git a/docs/AUTHZ-DOCUMENT-ACCESS.md b/docs/AUTHZ-DOCUMENT-ACCESS.md index 1e18cbf..d93d7a3 100644 --- a/docs/AUTHZ-DOCUMENT-ACCESS.md +++ b/docs/AUTHZ-DOCUMENT-ACCESS.md @@ -9,8 +9,8 @@ | Layer | Owner | Mechanism | | --- | --- | --- | | Who is the caller? | Host (Interchange) | `principal` + `tenant` on context / plane args | -| May they use memory at all? | Host grant store | `authorize(…, resource: "memory", action: "add" \| "find")` | -| Which **documents** may they see? | Host grant store + tags on the document | `authorize(…, resource: , action: "find")` for any tag on the doc | +| May they use memory at all? | Host grant store | `authorize(…, resource: "memory", action: "add" \| "search")` | +| Which **documents** may they see? | Host grant store + tags on the document | `authorize(…, resource: , action: "search")` for any tag on the doc | There is **one** authorization system: `@intx/authz` + host `GrantStore`. Corbits Memory does not invent modes, allowlists, or block lists as a security boundary. @@ -30,7 +30,7 @@ When the caller does not pass tags/share: 1. Always tag: `memory.owner:` 2. No other tags → **owner-only by default**. The **creating** principal always sees their own docs (engine convenience). **Peers** need an explicit host - grant of `find` on that owner resource (or a matching pattern) — the engine + grant of `search` on that owner resource (or a matching pattern) — the engine never auto-grants tags to anyone. @@ -49,13 +49,13 @@ Host issues grants such as: { principalId: "alice", resource: "memory.space:eng", - action: "find", + action: "search", effect: "allow", // …origin, etc. } ``` -Alice then sees any document tagged `memory.space:eng` (capability `memory`/`find` still required). +Alice then sees any document tagged `memory.space:eng` (capability `memory`/`search` still required). Patterns work: a grant on `memory.space:*` matches `memory.space:eng` via `@intx/authz` `matchPattern`. diff --git a/src/config.ts b/src/config.ts index 917899a..b99a11e 100644 --- a/src/config.ts +++ b/src/config.ts @@ -1,5 +1,5 @@ /** - * Memory / vector plane config — the core engine behind add/find/recent. + * Memory / vector plane config — the core engine behind add/search/list. * * This is the low-level engine config consumed by the DB client and internal * services. The SDK's mount-level config (`MemoryConfig`, see diff --git a/src/memory.test.ts b/src/memory.test.ts index a693996..c4957cd 100644 --- a/src/memory.test.ts +++ b/src/memory.test.ts @@ -372,7 +372,7 @@ describe("createMemory.find — grant-tag post-filter wiring", () => { }); }); -describe("find/recent — limit bounds", () => { +describe("search/list — limit bounds", () => { // These throw before any DB work, so a nonexistent URL is fine. it("find rejects limit below 1", async () => { const plane = createMemory({ config: wiringConfig }); From 366f872bebd1064e550883d05459644be76bd399 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Tue, 4 Aug 2026 21:27:40 -0700 Subject: [PATCH 11/11] Finish AUTHZ doc grant action rename to search Document-tag and capability checks use action search, not find. --- docs/AUTHZ-DOCUMENT-ACCESS.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/AUTHZ-DOCUMENT-ACCESS.md b/docs/AUTHZ-DOCUMENT-ACCESS.md index d93d7a3..cb8ae8d 100644 --- a/docs/AUTHZ-DOCUMENT-ACCESS.md +++ b/docs/AUTHZ-DOCUMENT-ACCESS.md @@ -172,8 +172,8 @@ Fresh databases apply the baseline migrations (`0001_extensions.sql` + 1. No public plane/HTTP API accepts `visibility` mode or block list as security. 2. Default add is owner-visible only (creator + owner tag). -3. Principal B sees A’s doc only when host grant allows `find` on a tag present on the doc (or B is creator). -4. Capability `memory`/`find` still required for find/ask/recent. +3. Principal B sees A’s doc only when host grant allows `search` on a tag present on the doc (or B is creator). +4. Capability `memory`/`search` still required for search/list. 5. PRODUCT.md / MIGRATION.md / README describe grant tags, not mini-ACL. 6. Engine + fakes enforce the algorithm; vendor adapters document principal-bucket limit. 7. `bun run typecheck && bun run test` green.