You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
commitScrap writes ingredients + a source link and nothing else (server/services/catalogDB/commit.js:16-34) — no catalog_ingredient_refs row, no catalog_ingredient_relations edge. So HAS_ANY_HOMING_REF (server/services/catalogDB/shared.js:258, HOMING_REF_KINDS = ['universe','series','creative-director']) is false for every freshly ingested row and they all bucket as unlinked (Raw). That is the "isolated and random" symptom in #7609.
Goal: one user-visible control at ingest — the universe this material belongs to — that links every committed ingredient to that universe and connects a single scrap's extractions to each other.
The infrastructure already exists and already federates. /api/catalog/bulk-import is the prior art in the same file: it already takes defaults.universeRef / seriesRef / workRef / issueRef / role (server/lib/catalogValidation.js:449-462, applied at server/routes/catalog.js:569). The scrap→commit path is the odd one out.
Approach
Add universeRef (+ optional role) to catalogScrapCommitSchema (server/lib/catalogValidation.js:407), mirroring catalogBulkImportSchema.defaults (:449-462) — same field name, same z.string().trim().min(1).max(120).optional() shape, so the two ingest paths cannot drift. Forward both from the commit route (server/routes/catalog.js:163-182) into commitScrap.
linkIngredientToRef and linkIngredientRelation currently run pool-level query(...) with no { client } option (server/services/catalogDB/refs.js:46, :180), so they cannot join commitScrap's withTransaction. Add the { client } = {} option to both, exactly as linkIngredientToSource already has it (refs.js:20). Without this, a mid-batch failure leaves ingredients committed with refs half-written.
Add universeRefRoleForType(type) to refs.js beside the existing seriesRefRoleForType (:69) and cdRefRoleForType (:95): character → canon-character, place → canon-place, object → canon-object, everything else (including idea / scene / concept and user-defined types) → reference. The canon-<kind> convention is what catalogCanonProjection.js and catalogUniverseTags.js already expect.
Inside commitScrap's existing transaction, after each createIngredient, call linkIngredientToRef(ingredient.id, 'universe', universeRef, role || universeRefRoleForType(draft.type), { client }).
Mint related-to edges between the ingredients committed from one scrap in one batch, so a single source's extractions form a connected cluster instead of N isolated nodes. Deterministic direction: one edge per unordered pair, from_id = the lexicographically smaller id — the PK is (from_id, to_id, kind), so an arbitrary direction would let a re-commit create a reciprocal duplicate. Bounded: skip edge minting entirely when a batch exceeds 25 accepted rows (the schema allows 200, which would be 19,900 edges).
Client: a "Catalogue into" <select> on the Ingest form covering all source modes (client/src/pages/CatalogIngest.jsx), listing live universes plus an explicit Unassigned option that preserves today's behavior exactly. Default: Reality for brain-bridge and voice-memo (captured thought is factual by default), otherwise the last-used universe read through safeReadStorage (client/src/lib/safeStorage.js) falling back to Reality. When the selected universe is factual, show a one-line hint that real people and places will be extracted as themselves. Follow client/src/AGENTS.md for htmlFor/id pairing and mobile layout.
Once the universe carries a factual flag (ship a Reality universe and give the universe record a factual flag #7616), feed it into the extraction lens landed in #PHASE1 — catalogExtraction.js derives factual from the scrap's sourceKind today and the universe flag becomes the primary signal, with the source kind as fallback.
Acceptance criteria
Committing a scrap with universeRef set creates one live catalog_ingredient_refs row per ingredient with the role universeRefRoleForType returns; HAS_ANY_HOMING_REF is true for all of them and none bucket as unlinked/Raw.
server/routes/catalogParsedBodies.test.js (which already covers POST /scraps/:id/commit with a mocked commitScrap) asserts universeRef / role are forwarded, and a DB-backed test in server/routes/catalog.test.js asserts the refs and relation edges actually land — plus that a mid-batch failure rolls back ingredients, refs, and edges together.
Omitting universeRef reproduces today's behavior exactly (source link only, no refs, no relations) — a regression test pins this.
Same-batch related-to edges are one per unordered pair with from_id lexicographically smaller; re-committing the same batch creates no duplicate or reciprocal edge; a >25-row batch mints none.
Ingest UI: the "Catalogue into" select is labelled (htmlFor/id), keyboard-reachable, defaults as specified, persists the last choice through safeReadStorage/safeWriteStorage, and renders correctly at phone width.
docs/STORAGE.md and the catalog docs describe the universe binding.
Out of scope
Resolving Mom / Dad to brain people records, and resolving scene.actors[] strings to ingredient ids — both are entity resolution, filed separately.
Reading catalog tables from universeGraph.js / brainGraph.js, and including relations in exportSliceForRef (the [catalog-ingredient-relations] TODO at server/services/catalogDB/facets.js:60).
Problem / Goal
commitScrapwrites ingredients + a source link and nothing else (server/services/catalogDB/commit.js:16-34) — nocatalog_ingredient_refsrow, nocatalog_ingredient_relationsedge. SoHAS_ANY_HOMING_REF(server/services/catalogDB/shared.js:258,HOMING_REF_KINDS = ['universe','series','creative-director']) is false for every freshly ingested row and they all bucket as unlinked (Raw). That is the "isolated and random" symptom in #7609.Goal: one user-visible control at ingest — the universe this material belongs to — that links every committed ingredient to that universe and connects a single scrap's extractions to each other.
The infrastructure already exists and already federates.
/api/catalog/bulk-importis the prior art in the same file: it already takesdefaults.universeRef / seriesRef / workRef / issueRef / role(server/lib/catalogValidation.js:449-462, applied atserver/routes/catalog.js:569). The scrap→commit path is the odd one out.Approach
universeRef(+ optionalrole) tocatalogScrapCommitSchema(server/lib/catalogValidation.js:407), mirroringcatalogBulkImportSchema.defaults(:449-462) — same field name, samez.string().trim().min(1).max(120).optional()shape, so the two ingest paths cannot drift. Forward both from the commit route (server/routes/catalog.js:163-182) intocommitScrap.linkIngredientToRefandlinkIngredientRelationcurrently run pool-levelquery(...)with no{ client }option (server/services/catalogDB/refs.js:46,:180), so they cannot joincommitScrap'swithTransaction. Add the{ client } = {}option to both, exactly aslinkIngredientToSourcealready has it (refs.js:20). Without this, a mid-batch failure leaves ingredients committed with refs half-written.universeRefRoleForType(type)torefs.jsbeside the existingseriesRefRoleForType(:69) andcdRefRoleForType(:95):character→canon-character,place→canon-place,object→canon-object, everything else (includingidea/scene/conceptand user-defined types) →reference. Thecanon-<kind>convention is whatcatalogCanonProjection.jsandcatalogUniverseTags.jsalready expect.commitScrap's existing transaction, after eachcreateIngredient, calllinkIngredientToRef(ingredient.id, 'universe', universeRef, role || universeRefRoleForType(draft.type), { client }).related-toedges between the ingredients committed from one scrap in one batch, so a single source's extractions form a connected cluster instead of N isolated nodes. Deterministic direction: one edge per unordered pair,from_id= the lexicographically smaller id — the PK is(from_id, to_id, kind), so an arbitrary direction would let a re-commit create a reciprocal duplicate. Bounded: skip edge minting entirely when a batch exceeds 25 accepted rows (the schema allows 200, which would be 19,900 edges).<select>on the Ingest form covering all source modes (client/src/pages/CatalogIngest.jsx), listing live universes plus an explicit Unassigned option that preserves today's behavior exactly. Default: Reality forbrain-bridgeandvoice-memo(captured thought is factual by default), otherwise the last-used universe read throughsafeReadStorage(client/src/lib/safeStorage.js) falling back to Reality. When the selected universe isfactual, show a one-line hint that real people and places will be extracted as themselves. Followclient/src/AGENTS.mdforhtmlFor/idpairing and mobile layout.factualflag (ship a Reality universe and give the universe record a factual flag #7616), feed it into the extraction lens landed in #PHASE1 —catalogExtraction.jsderivesfactualfrom the scrap'ssourceKindtoday and the universe flag becomes the primary signal, with the source kind as fallback.Acceptance criteria
universeRefset creates one livecatalog_ingredient_refsrow per ingredient with the roleuniverseRefRoleForTypereturns;HAS_ANY_HOMING_REFis true for all of them and none bucket as unlinked/Raw.server/routes/catalogParsedBodies.test.js(which already coversPOST /scraps/:id/commitwith a mockedcommitScrap) assertsuniverseRef/roleare forwarded, and a DB-backed test inserver/routes/catalog.test.jsasserts the refs and relation edges actually land — plus that a mid-batch failure rolls back ingredients, refs, and edges together.universeRefreproduces today's behavior exactly (source link only, no refs, no relations) — a regression test pins this.related-toedges are one per unordered pair withfrom_idlexicographically smaller; re-committing the same batch creates no duplicate or reciprocal edge; a >25-row batch mints none.htmlFor/id), keyboard-reachable, defaults as specified, persists the last choice throughsafeReadStorage/safeWriteStorage, and renders correctly at phone width.docs/STORAGE.mdand the catalog docs describe the universe binding.Out of scope
Mom/Dadto brainpeoplerecords, and resolvingscene.actors[]strings to ingredient ids — both are entity resolution, filed separately.universeGraph.js/brainGraph.js, and including relations inexportSliceForRef(the[catalog-ingredient-relations]TODO atserver/services/catalogDB/facets.js:60).