Self-hosted apply client for Entity Enricher schema databases.
Entity Enricher can mirror every enrichment into your own database as plain relational tables. This CLI runs next to that database, connects outward over WSS (no inbound access, no credentials shared with Entity Enricher), receives delta batches, applies them and acknowledges:
Entity Enricher ──WSS──▶ ee-database ──SQL──▶ your PostgreSQL / MySQL / SQLite
◀──────ack───────┘
- Bootstrap included — on first run it fetches the
.sqlsnapshot (tables + current data) and applies it before consuming deltas. - Replay-safe — every delta is an idempotent, revision-guarded upsert; if the client dies mid-batch, the batch is redelivered after its lease expires and re-applying converges to the same rows.
- Your DSN stays local — the connection string is passed on the command line or stored mode-600 on the machine; it is never sent anywhere.
- Quarantine on failure — a SQL error rolls back the batch and reports the failing delta. The server then sets that enrichment's whole batch aside and re-sends the queue without it, so one bad row can no longer stall everything behind it; the client keeps applying the rest. Nothing is silently skipped: the quarantined work stays listed in the Database Sync page's Quarantine tab until you reinject it (re-projected from current state) or drop it. A failure that names no delta still exits, since there would be nothing to quarantine and the same window would come straight back.
- Several databases per machine — each pairing lives in its own local
profile;
run --allsyncs every paired database concurrently from one process.
curl -fsSL https://entityenricher.ai/install-eedatabase.sh | sh(Windows: iwr -useb https://entityenricher.ai/install-eedatabase.ps1 | iex.
Both URLs are 302 aliases of install.sh /
install.ps1 at the root of this repo.)
The installer prints what it's about to do, pauses 5 seconds, and verifies the
binary's Sigstore keyless signature before making it executable: releases
are signed in CI by this repo's release.yml GitHub OIDC identity and logged
in the public Rekor transparency log — there is
no long-lived signing key anywhere.
Alternatively, download a signed binary from Releases yourself, or build from source (Go ≥ 1.23):
go build -o ee-database .For an editable install while hacking on the CLI, symlink the build output
onto your PATH so every rebuild is instantly live:
mkdir -p ~/.local/bin
go build -o tmp/ee-database .
ln -sfn "$PWD/tmp/ee-database" ~/.local/bin/ee-database # ensure ~/.local/bin is on PATH# 1. Pair with your Entity Enricher organization (opens the browser to pick the database)
ee-database pair --server https://entityenricher.ai
# 2. Run it next to your database
ee-database run --dsn "postgres://user:pass@localhost:5432/mydb" --save-dsnAlternatively, issue a token on the Databases page (Sync client → Pair a client)
and paste it: ee-database pair --server https://entityenricher.ai <refresh-token>.
If the target database does not exist yet, add --create-missing to the run
command: the client creates it with the DSN's own credentials before syncing
(postgres: needs CREATEDB, goes through the standard postgres maintenance
database; mysql: needs the CREATE privilege; sqlite: unnecessary — the file
is created automatically).
When even the DSN's role/user does not exist yet, add --admin-dsn with an
administrative connection: the client then bootstraps everything the target
DSN names — the role (with the target DSN's password, postgres) or user
(mysql, plus a grant on the database) when missing, and the database owned by
it. The target DSN needs no create rights of its own in this mode:
ee-database run \
--dsn "postgres://ee_replica:secret@localhost:5432/my_replica" \
--create-missing --admin-dsn "postgres://postgres:admin@localhost:5432/postgres"The admin DSN is used only for this bootstrap — it is never stored.
On every run the client self-checks its login's provisioning rights — create database (from the catalog), DDL and DML (a create/insert/delete/drop probe table) — prints the result and reports it to the Databases page's Sync client card, so a missing grant is visible there before deltas fail to apply. The check is advisory: it never blocks the run.
Assuming the target database already exists, the login needs only:
CONNECTon the database,USAGEandCREATEon the target schema (note: since PostgreSQL 15,publicno longer grantsCREATEto everyone by default).
Everything else follows from ownership: the client creates the replica
tables itself, so it owns them, and ownership implies the INSERT / UPDATE
/ DELETE / SELECT the data deltas need. Ownership is not optional — the
feed also ships migration statements (ALTER TABLE …, CREATE INDEX …) that
PostgreSQL restricts to the table owner; no combination of GRANTs
substitutes for it, and IF NOT EXISTS does not help (ownership is checked
before the duplicate-name check, so even re-applying an existing index fails
for a non-owner). If the replica tables pre-exist under a different owner, the
preflight passes (the login can create new tables) but the first migration
delta will fail — transfer them with ALTER TABLE … OWNER TO <login> (or
grant the login membership in the owning role) rather than adding grants.
Schemas can flag text properties for substring search; on PostgreSQL those ship
as trigram indexes, which need the pg_trgm extension. On connect the client
checks for it and, when the login is allowed to, installs it itself
(CREATE EXTENSION IF NOT EXISTS pg_trgm). When it cannot, the sync still runs
in full — the search-index statements are self-guarded and simply no-op — and a
warning explains the one-time fix: have a database owner run
CREATE EXTENSION pg_trgm; on the replica. The indexes then materialize on the
next publish or reconcile. MySQL and SQLite replicas never receive search-index
statements.
If none of the database's linked schemas is published yet, the first run has nothing to bootstrap: the client stays connected and waits, and the first publish starts the feed on its own — no restart needed.
Each pushed window is applied in one transaction — the server never ends a window inside an enrichment — and is reported with the shape it wrote, per table:
applying 12 delta(s) (10831 .. 10842) in one transaction
applied 12 delta(s) in 84ms — 38 statement(s): mushroom 4 upserts, mushroom_common_names 12 upserts + 4 prunes, mushroom_human_uses 14 upserts + 4 prunes
acked up to delta 10842
Read it as the write shape of the feed, which is what sizing a nightly re-enrichment needs:
- upsert — one
INSERT … ON CONFLICT DO UPDATEper projected row, so the count is the row count. Children are reconciled in place; they are never wiped and re-inserted. - prune — a revision-guarded
DELETEremoving the child or junction rows the new payload no longer claims. One statement per relationship, however many rows it clears. - delete — a tombstone (an entity deleted upstream) or its owned cascade.
- wipe — an unscoped
DELETE; only a keyless parent's children are replaced wholesale. - ddl — schema deltas: table, column and index statements, including the
ones the feed guards inside a
DOblock.
--verbose adds one line per delta, with its id, kind, entity type, wall time
and its own shape — the per-entity cost, which is otherwise unrecoverable once
a delta is acked and purged server-side:
applying 12 delta(s) (10831 .. 10842) in one transaction
delta 10831 data upsert Mushroom (7ms) — 3 statement(s): mushroom 1 upsert, mushroom_common_names 3 upserts + 1 prune
…
It also reports the bootstrap snapshot's table inventory and apply time.
| Command | Purpose |
|---|---|
pair --server URL |
Browser-confirmed device-code pairing |
pair --server URL <token> |
Pair with a token from the Databases page |
run --dsn DSN [--save-dsn] [--skip-bootstrap] |
Connect and apply deltas |
run … --create-missing |
Create the target database first when it does not exist (postgres/mysql; sqlite files are created anyway) |
run … --create-missing --admin-dsn DSN |
Also create the target DSN's missing role/user via an admin connection (never stored) |
run --all |
Sync every paired database concurrently (each needs a saved DSN) |
run … --verbose / host run … --verbose |
Log every delta's write shape and timing, not just the per-window summary |
host pair --server URL --dsn BASE_DSN [--admin-dsn DSN] <token> |
Pair this machine as a managed sync host (token from the Sync hosts dialog — the "Sync hosts" toolbar button on the Database Sync page); one pairing per server, so several backends can be paired side by side |
host run [--server URL] |
Managed mode: auto-claim, create and sync every database sync assigned to this host — every paired server at once, or one with --server |
host status / host disconnect [--server URL] |
Show / forget host pairings (--server picks one when several servers are paired) |
status |
Show pairing state (all paired databases) |
disconnect |
Forget one pairing's local credentials (revoke server-side in the UI) |
version |
Print version |
Pairing is per database; managed mode moves the ceremony one level up. Pair the
machine once per server — --dsn is a base server DSN with no database name
(it never leaves the machine) — then host run holds one control-plane WebSocket
per paired server (pair against several backends — say staging and production —
and one host run syncs them all; --server URL restricts it to one) and
reacts to assignments made in the Entity Enricher UI (or API): it claims the
credential, derives the database's DSN from the base DSN plus the
server-suggested snake_cased name (override per registration via
database_names in the host config.json), creates the database if missing,
runs the rights preflight, and starts the ordinary sync loop. A database
already paired with another client is reported and skipped — never hijacked.
Revoking the host in the UI cuts the machine off entirely, including every
credential it claimed.
# 1. Register a host on the Database Sync page ("Sync hosts" toolbar button →
# Add host) — the one-time token is shown embedded in this command:
ee-database host pair --server https://entityenricher.ai \
--dsn "postgres://ee_replica:secret@dbserver:5432/" \
--admin-dsn "postgres://postgres:admin@dbserver:5432/postgres" <token>
# 2. Keep it running; assign database syncs to the host in the UI
ee-database host run--admin-dsn plays the same role as on run --create-missing: provisioning
creates the target role and each claimed database through the admin
connection, so the base DSN's login needs no create rights of its own. It is
used at provision time only — never stored. Without it, pairing preflights the
base DSN's own rights and fails fast on a missing CREATEDB
(--skip-preflight to override; host run re-checks and reports to the Sync
hosts card either way).
Pair once per schema database — every pairing gets its own profile directory
(~/.config/ee-database/profiles/<database-id>/ on Linux, the OS config dir
elsewhere), so pairing a second database never disturbs the first.
When several databases are paired, run, status and disconnect accept
--database NAME|ID to pick one. Without the flag, a terminal prompts with a
numbered list; a non-interactive run (systemd, Docker, cron) fails with the
list instead of hanging — pass --database there, or use run --all:
# Save each DSN once…
ee-database run --database analytics --dsn "postgres://…/analytics" --save-dsn
ee-database run --database staging --dsn "postgres://…/staging" --save-dsn
# …then one process syncs both (log lines are prefixed per database)
ee-database run --allA per-profile lock prevents two processes from running the same pairing at the same time (they would evict each other's WebSocket session in a loop).
Pairings made with ee-database ≤ 0.1.x are migrated into a profile automatically on first use.
The target dialect is fixed by the schema database registration in Entity Enricher. PostgreSQL is the launch dialect; the MySQL and SQLite drivers are already bundled for when their SQL renderers ship.
When the target database rejects the snapshot or a delta, ee-database prints
everything the database reported — SQLSTATE, detail, hint, and (when the
server provides an error cursor) the offending script line. A failed snapshot
is additionally saved to the pairing's profile directory as
snapshot-failed.sql (mode 0600, replaced on each attempt, removed on the
next success), so the failing statement can be inspected or replayed with
psql -f. Nothing is partially applied: the snapshot is one transaction, and
a failed delta batch rolls back and is redelivered once the cause is fixed.
- One credential = one schema database = one client. Pairing again rotates the credential; the old token stops working immediately.
- The refresh token (365 days) is stored mode-600 in the pairing's profile directory; it is exchanged for 15-minute access tokens that authenticate the WebSocket.
- The host pairing key (
eeh_…) is a short opaque secret, not a JWT: the server keeps only its hash, it never expires, and revoking the host in the UI is what ends it. - Revoking the credential in the UI disconnects a live client instantly.
- Recommended: run against a dedicated database role scoped to the synced schema.
MIT — see LICENSE.