Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@ tmp/
# Dispatch orchestration scratch (local only)
dispatch/

# Staging dirs for sibling package extracts (copy out with cp only)
.staging-*/

15 changes: 9 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Agent guide — @corbits/knowledge-engine
# Agent guide — @corbits/memory

A library, not a service. `src/` is the whole product: a knowledge add / find /
A library, not a service. `src/` is the whole product: a memory add / find /
ask / recent SDK that **mounts onto a host Interchange app**. There is no server,
port, or process entrypoint here, and there never should be.

Expand All @@ -18,14 +18,17 @@ CI runs `typecheck` + `test` — both must pass before any push.

## Layout

- `src/index.ts` — public surface: `mountKnowledgeEngine`, `mountKnowledgeRoutes`, `createKnowledgePlane`
- `src/index.ts` — public surface: `createMemory` (optional `app` registers HTTP), `registerMemoryRoutes`

- `src/mount-config.ts` / `src/config.ts` — mount config + engine config
- `src/routes/` — Hono routes (`add`, `find`, `ask`, `recent`)
- `src/routes/` — Hono routes (`add`, `search`, `list`)
- `src/services/` — capture / search / transform internals (not public verbs)
- `src/ports/` — `DocumentStore` / `SourceProvider` / `MemoryProvider` + fakes
- `src/ports/` — `DocumentStore` / `SourceProvider` + fakes
- `src/core/` — embed/rerank clients, merge, arktype schemas
- `src/db/` + `migrations/` — Drizzle schema + SQL migrations (pgvector, `knowledge.*`)
- `packages/` — optional DocumentStore adapters (`knowledge-adapter-mem0`, `knowledge-adapter-supermemory`); pure fetch, no vendor SDKs in core. Linear tools live in sibling `@corbits/linear`.
- `packages/` — removed; DocumentStore adapters and Linear tools are sibling packages
(`@corbits/mem0-memory-adapter`, `@corbits/supermemory-memory-adapter`,
`@corbits/linear-tools`).

## Non-negotiable invariants

Expand Down
51 changes: 25 additions & 26 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,25 @@
# Knowledge Engine — Architecture
# Corbits Memory — Architecture

A knowledge add / find / ask / recent SDK that mounts onto an Interchange hub. The
host owns auth, tenancy, and the process; this library owns the knowledge /
A memory add / find / ask / recent SDK that mounts onto an Interchange hub. The
host owns auth, tenancy, and the process; this library owns the memory /
vector plane and the routes that read and write it.

## Why an SDK, not a service

The knowledge store was originally built inside a larger backend. It turned out
The memory store was originally built inside a larger backend. It turned out
to be cleanly detachable, and then cleanly *mountable*:

- No knowledge table has a foreign key into any control-plane table — every
- No memory table has a foreign key into any control-plane table — every
cross-reference (`tenant_id`, `principal_id`, source refs) is plain `text`.
- Embedding and reranking go out as plain HTTP to configured model endpoints,
not through any agent runtime.
- The ACL rule is a self-contained scope stored on the row, not a join against
a grant engine.

So the engine needs nothing but a pgvector Postgres and an embed/rerank
endpoint. It ships as `mountKnowledgeEngine(app, opts)`: the host passes its
Hono app and its grant store; the engine mounts its routes, reads identity from
So the library needs nothing but a pgvector Postgres and an embed/rerank
endpoint. It ships as `createMemory(opts)` — pass `app` to register HTTP. The host passes its

Hono app and its grant store; the library mounts its routes, reads identity from
the request context, and talks to its own vector store. No second server, no
HTTP hop.

Expand All @@ -43,15 +44,15 @@ HTTP hop.

1. **Who is calling** is the request principal, read off the Interchange
context. Clients never send `tenant_id`/`principal_id` — the handlers read
only content fields (title/text/query/k/acl) and take identity from context.
only content fields (title/text/query/limit/access_tags/share) and take identity from context.
2. **What is stored** is opaque data on every record: `tenant_id`,
`principal_id`, `created_by_kind` (human/agent/system), `source_class`, and
relations (the edge graph). Every query is scoped by `tenant_id` first, then
the document ACL is matched against the caller's subject.
relations (the edge graph). Every query is scoped by `tenant_id` first; then
document access uses Interchange grant tags (`accessTags` + creator).

Cross-tenant isolation is enforced at query time by `tenant_id`; document-level
visibility (allow/block) is enforced on top. This is the trust
model.
access is grant tags via `@intx/authz` (creator always allowed). This is the
trust model.

## Layers

Expand All @@ -64,22 +65,19 @@ model.

## Mounted surface

`mountKnowledgeEngine` adds, under the host app:
`createMemory({ app })` adds, under the host app:

- `POST /api/knowledge/add` — ingest a note (raw + derive).
- `POST /api/knowledge/find` — hybrid retrieval: FTS + dense (pgvector) → RRF
- `POST /api/memory/add` — ingest a note (raw + derive).
- `POST /api/memory/search` — hybrid retrieval: FTS + dense (pgvector) → RRF
fusion → cross-encoder rerank → bounded authority/recency boosts → MMR;
optional live `SourceProvider` merge (fail-soft).
- `POST /api/knowledge/ask` — grant-checked as `knowledge:find`; retrieves as
the principal, grounds a prompt from hit snippets, calls host-injected
`generate`. Optional memory recall when `includeMemory` is true.
- `GET /api/knowledge/recent` — recent documents for the caller's scope,
filtered with the same document ACL as local find (visibility + block list).
- `GET /api/memory/list` — recent documents for the caller's scope,
filtered with the same grant-tag access as local search (`canAccessDocument`).

It also returns an in-process `KnowledgePlane` (`add`, `find`, `ask`,
`recent`, optional `remember` / `recall`). `ask()` is grant-checked in-process
(callers bypass the HTTP `requireGrant` guard). The engine owns no generation
client; hosts wire `generate` to their inference layer.
It also returns an in-process `Memory` (`add`, `search`, `list`, `close`).
There is no product `ask` / `remember` / `recall` and no host-injected
`generate` on the plane — inference is host-owned and ephemeral (call your
model, then `add` / `search`).

MCP is not part of this package — mount `@corbitsdev/hono-openapi-mcp` to expose
these routes as MCP tools.
Expand All @@ -88,7 +86,8 @@ External ingestion (Linear, GitHub, …) is not a route here — the host
authenticates the forwarder to Interchange and calls `plane.add` / a
`SourceProvider` mapper, or mounts HTTP add after its own auth.

Legacy paths `/capture`, `/search`, `/timeline` are not mounted (hard cutover).
Legacy paths `/capture`, `/search` (old knowledge), `/timeline`, `/find`,
`/ask`, `/recent` are not mounted (hard cutover).

## Provenance

Expand Down
61 changes: 42 additions & 19 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Changelog

All notable changes to `@corbits/knowledge-engine` are documented in this file.
All notable changes to `@corbits/memory` are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
Expand All @@ -9,28 +9,51 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- **Breaking:** knowledge plane surface is `add` / `find` / `ask` / `recent` with
`principalId` + `tenantId` only (`capture` / `search` / `timeline` and
`subjectId` / `scopeId` removed). See `MIGRATION.md`.
- **Breaking:** HTTP routes are `POST /api/knowledge/add`,
`POST /api/knowledge/find`, `POST /api/knowledge/ask`,
`GET /api/knowledge/recent`. Old paths are not mounted.
- **Breaking:** grant actions are `add` and `find` (was `capture` / `search`).
`ask` and `recent` use the `find` grant.
- **Breaking:** `add` returns `{ documentId }`; find body uses `limit` (not `k`);
find wire uses `items` (not `hits`).
- **Breaking:** package and public surface renamed from `@corbits/knowledge-engine`
to `@corbits/memory`. Public APIs: `createMemory` (optional `app` registers HTTP),
`registerMemoryRoutes`.

`createMemory`, `loadMemoryConfig`, `runMemoryMigrations`, `Memory`,
`MemoryConfig`, `MemoryError`. HTTP paths are under `/api/memory/`; grants are
`memory:add` / `memory:search`; access tags use `memory.owner:` / `memory.tenant:`
/ `memory.space:`. Postgres schema name remains `knowledge`. See `MIGRATION.md`.
- **Breaking:** memory plane surface is `add` / `search` / `list` with
`principalId` + `tenantId` only. Removed product verbs: `find` (→`search`),
`recent` (→`list`), `ask`, `remember`, `recall`, and any `MemoryProvider` /
`generate` path. Inference is host-owned. See `MIGRATION.md`.
- **Breaking:** HTTP routes are `POST /api/memory/add`,
`POST /api/memory/search`, `GET /api/memory/list`. Old paths are not mounted.
- **Breaking:** grant actions are `add` and `search` (was `capture` / `find` /
knowledge `search`). `list` uses the `search` grant. Capability resource is
`memory`. Document-tag checks use action `search`.
- **Breaking:** `add` returns `{ documentId }`; search body uses `limit` (not `k`);
search wire uses `items` (not `hits`).
- **Breaking:** document access is Interchange **grant tags** (`accessTags` +
creator-always + host `GrantStore`), not the visibility-mode / block-list
mini-ACL. Share sugar only mints tags. See `docs/AUTHZ-DOCUMENT-ACCESS.md`.
- **Breaking:** Postgres baseline is two files (`0001_extensions` +
`0002_knowledge_baseline`) with `access_tags` and no `visibility_*` columns.
Fresh installs only — drop/recreate the knowledge schema on existing DBs.
- **Breaking:** `grantStore` + `conditionRegistry` are top-level `createMemory`
options (no nested `grants: { … }`).

### Added

- Optional `TextExtractor` + `file` XOR `content` on `add`
- `share` sugar on `add` (maps to existing visibility / block ACL)
- `POST /api/knowledge/ask` HTTP route
- `share` sugar on `add` (maps to access tags only: owner, tenant, peers)
- `access_tags` on `knowledge.document` (baseline schema; Postgres schema name unchanged)
- `MIGRATION.md` hard-cutover notes for in-repo consumers

### Removed

- Product `ask` / `remember` / `recall` and `MemoryProvider` side-channel
- Host-injected `generate` on the plane (use host inference + `add` / `search`)
- HTTP `POST /api/memory/ask`, `POST /api/memory/find`, `GET /api/memory/recent`

## [0.1.2] — 2026-07-31

### Added
- Public `createKnowledgePlane` export for out-of-band capture and search (CLI seeders, batch ingesters, tests) without mounting HTTP routes (`#8`)
- Public `createMemory` export for out-of-band capture and search (CLI seeders, batch ingesters, tests) without mounting HTTP routes (`#8`)

### Fixed

Expand All @@ -50,7 +73,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed

- Dense search scales `hnsw.ef_search` to the overfetch limit (floor 40) and probes `hnsw.iterative_scan = relaxed_order` once per process when available (`#2`)
- Package install docs point at `@corbits/knowledge-engine` (`#6`)
- Package install docs point at `@corbits/memory` (`#6`)

### Fixed

Expand All @@ -61,8 +84,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Initial cut: mountable knowledge capture + search SDK for Interchange hubs
- Initial cut: mountable memory capture + search SDK for Interchange hubs

[0.1.2]: https://github.com/corbitsdev/corbits-knowledge-engine/compare/v0.1.1...v0.1.2
[0.1.1]: https://github.com/corbitsdev/corbits-knowledge-engine/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/corbitsdev/corbits-knowledge-engine/releases/tag/v0.1.0
[0.1.2]: https://github.com/corbitsdev/corbits-memory/compare/v0.1.1...v0.1.2
[0.1.1]: https://github.com/corbitsdev/corbits-memory/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/corbitsdev/corbits-memory/releases/tag/v0.1.0
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Contributing

Thanks for considering a contribution to Knowledge Engine.
Thanks for considering a contribution to Corbits Memory.

## Running it locally

```bash
git clone https://github.com/corbitsdev/corbits-knowledge-engine.git
cd corbits-knowledge-engine
git clone https://github.com/corbitsdev/corbits-memory.git
cd corbits-memory
docker compose up -d # pgvector Postgres on localhost:5434
cp .env.example .env # edit as needed — see README.md's quickstart
bun install
Expand Down
Loading
Loading