diff --git a/.docs/ai-core.md b/.docs/ai-core.md new file mode 100644 index 0000000..b4e9ed4 --- /dev/null +++ b/.docs/ai-core.md @@ -0,0 +1,87 @@ +# SAP AI Core integration + +The plugin exposes SAP AI Core resource groups, deployments, and configurations through the `AICore` CAP service. It also manages the resource groups and SAP-RPT-1 deployments used by recommendations. + +> [!IMPORTANT] +> In multitenant applications with an MTX sidecar, include `@cap-js/ai` in the sidecar so tenant lifecycle events can manage SAP AI Core resources. + +## Service binding + +Production use requires an [SAP AI Core](https://help.sap.com/docs/sap-ai-core) service binding. A Cloud Foundry deployment can declare it like this: + +```yaml +modules: + - name: incidents-srv + type: nodejs + path: gen/srv + requires: + - name: incidents-ai-core + +resources: + - name: incidents-ai-core + type: org.cloudfoundry.managed-service +``` + +A resource group is SAP AI Core's isolation boundary: it scopes deployments, configurations, and executions so that tenants cannot access each other's resources. +The plugin provisions one resource group per tenant in multitenant applications, and uses a single resource group otherwise. + +Single-tenant applications use the `default` resource group unless configured otherwise: + +```json +{ + "cds": { + "requires": { + "AICore": { + "resourceGroup": "CUSTOM_RESOURCE_GROUP" + } + } + } +} +``` + +## Query API + +```js +const aiCore = await cds.connect.to('AICore'); +const { resourceGroups, deployments, configurations } = aiCore.entities; + +await aiCore.run(SELECT.from(resourceGroups)); +await aiCore.run(SELECT.from(resourceGroups).where({ tenantId: cds.context.tenant })); +await aiCore.run( + SELECT.from(deployments).where({ + 'resourceGroup.resourceGroupId': resourceGroups[0].resourceGroupId + }) +); +``` + +Supported `cds.ql` operations: + +| Operation | `resourceGroups` | `deployments` | `configurations` | +| ---------------------- | ---------------- | ------------- | ---------------- | +| `READ` list and single | yes | yes | yes | +| `CREATE` | yes | yes | yes | +| `UPDATE` | yes | yes | no | +| `UPSERT` | yes | yes | no | +| `DELETE` | yes | yes | no | +| `limit` | yes | yes | yes | +| `search` | no | no | yes | + +Filters are limited to simple equality checks: + +- `resourceGroups`: `tenantId`, `resourceGroupId` +- `deployments`: `id`, `resourceGroup.resourceGroupId` +- `configurations`: `resourceGroup.resourceGroupId` + +## Helper methods + +```js +const aiCore = await cds.connect.to('AICore'); +const { resourceGroups, deployments } = aiCore.entities; + +const resourceGroupId = await aiCore.resourceGroupForTenant(cds.context.tenant); +const predictions = await aiCore.predictRowColumns(/* SAP-RPT-1 payload */); +const deploymentId = await aiCore.rpt1DeploymentId(resourceGroups, { resourceGroupId }); +await aiCore.stop(deployments, { id: deploymentId }); +``` + +`rpt1DeploymentId` creates an SAP-RPT-1 deployment when the resource group does not already have one. diff --git a/.docs/knowledge-graph.md b/.docs/knowledge-graph.md new file mode 100644 index 0000000..c76adab --- /dev/null +++ b/.docs/knowledge-graph.md @@ -0,0 +1,57 @@ +# Local knowledge graph + +> [!WARNING] +> The local knowledge graph is experimental and intended only for local development. Its API and storage model may change incompatibly. + +Install the optional peer dependency: + +```sh +npm add -D oxigraph +``` + +With `@cap-js/ai` installed, both `sqlite` and `sqlite:memory` expose a process-local Oxigraph store through `SPARQL_EXECUTE` and `sparql_table`. + +## Load RDF + +Use the HANA-compatible procedure shape: + +```sql +CALL SPARQL_EXECUTE( + 'LOAD INTO GRAPH ', + '', + ?, + ? +) +``` + +The final two `?` tokens are required output placeholders, not input bindings. The local implementation accepts literal SPARQL and header strings only. `LOAD` returns no result. + +Files must be inside the CAP project. Turtle and compressed Turtle inputs are supported; unsafe paths, symlinks escaping the project, malformed RDF, and unsupported formats are rejected. + +## Query RDF + +Procedure-style queries return a serialized result in `RESPONSE`: + +```sql +CALL SPARQL_EXECUTE( + 'SELECT ?subject WHERE { ?subject ?predicate ?object }', + 'accept:application/sparql-results+json', + ?, + ? +) +``` + +Use `sparql_table` from CQN when rows should be projected into a query result: + +```js +await db.run({ + SELECT: { + from: cds.ql.func( + 'sparql_table', + 'SELECT ?subject ?predicate WHERE { ?subject ?predicate ?object }' + ) + } +}); +``` + +The RDF store lives in memory and is tied to the database service connection. Its contents are lost on disconnect or process restart even with file-based `sqlite`, and RDF updates are not transactionally coupled to SQLite changes. diff --git a/.docs/model-selection.md b/.docs/model-selection.md new file mode 100644 index 0000000..cc0372d --- /dev/null +++ b/.docs/model-selection.md @@ -0,0 +1,50 @@ +# Choosing a local embedding model + +> [!WARNING] +> Local model execution and provisioning are experimental development features. A model that installs successfully is not automatically appropriate for an application's data, languages, or quality requirements. + +## Start with a focused list + +Use the [trending Apache-2.0 sentence-similarity models with ONNX artifacts](https://huggingface.co/models?pipeline_tag=sentence-similarity&library=onnx&license=license:apache-2.0&sort=trending) as a discovery starting point: + +- `sentence-similarity` favors sentence-level semantic embeddings rather than text generation or token classification. +- `onnx` indicates that the repository advertises an ONNX export that can potentially run locally. +- `apache-2.0` narrows the list to a permissive license commonly suitable for experimentation. Always review the model card and license for your own use. +- `trending` makes active, commonly used candidates easier to find; it is not a quality ranking. + +The Hub filters are not a compatibility guarantee. Repositories can contain several ambiguous exports, unsupported processing stages, or incomplete metadata. + +## As big as necessary, as small as possible + +For local development, start with the smallest model that meets the application's language, domain, and retrieval-quality needs. Smaller models download and start faster, use less memory, and block SQLite for less time. Move to a larger model only when measurements on representative data show that the smaller one is insufficient. + +The current default and Bookshop sample use [`sentence-transformers/all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It was selected only because, at the time of selection, it was the most-downloaded reasonably small candidate matching the sentence-similarity, ONNX, and Apache-2.0 filters. This is not a recommendation for any application or for production, and the default may change at any time while local embeddings remain experimental. Configure the model explicitly when that choice must stay stable. + +Ideally, identical embedding-models should be employed during development and in production. However, this is not technically required and hard to realize, due to model availability: I.e. local ONNX models and SAP HANA native models will differ. For reference, SAP HANA Cloud's [`SAP_GXY.20250407` is based on RoBERTa base](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-vector-engine-guide/vector-embedding-function-vector#available-models-without-remote-source). A local MiniLM vector has different dimensions and semantics and is not interchangeable with a HANA-generated vector. Never mix vectors from different model setups; regenerate embeddings after a change. + +## Check before installing + +```sh +npx @cap-js/ai check-model owner/model +npx @cap-js/ai install-model owner/model +``` + +`check-model` examines repository metadata and reports likely compatibility without downloading the model weights. `install-model` performs the definitive check by downloading, loading, and probing the selected ONNX graph. + +## Supported model contract + +Discovery currently requires: + +- a public Hugging Face repository whose declared task is absent, `sentence-similarity`, or `feature-extraction` +- an immutable repository revision +- an unambiguously selectable ONNX graph, preferring `onnx/model.onnx` and then `model.onnx` +- `tokenizer.json`, `tokenizer_config.json`, and `config.json` beside the graph or at repository root +- an embedding dimension in a common Transformers field such as `hidden_size`, `n_embd`, `d_model`, or `dim` +- a determinable input limit +- an unambiguous Sentence Transformers pipeline of Transformer, Pooling, and optional Normalize stages +- mean or CLS pooling +- when a model declares [prompts it was trained with](https://sbert.net/examples/sentence_transformer/training/prompts/README.html) (typically `query` and `document`): Metadata that maps these prompts to [SAP HANA `VECTOR_EMBEDDING`s `text-type`](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/vector-embedding-function-vector) + +The ONNX graph must accept rank-2 `int64` `input_ids`; it may also accept `attention_mask` and `token_type_ids`. A token-level output used for pooling must be a floating-point rank-3 tensor whose final dimension matches the discovered model dimension. + +Nested exports are supported when the model and its companion files are unambiguous. Conventional adjacent external-data names are supported. Other module chains, ambiguous pooling, decoder outputs, incompatible task tags, missing metadata, or arbitrary external-data paths are rejected instead of guessed. diff --git a/.docs/recommendations.md b/.docs/recommendations.md new file mode 100644 index 0000000..71806fb --- /dev/null +++ b/.docs/recommendations.md @@ -0,0 +1,72 @@ +# Recommendations + +`@cap-js/ai` uses [SAP-RPT-1](https://help.sap.com/docs/sap-ai-core/generative-ai/sap-rpt-1) through SAP AI Core to add recommendations to CAP draft entities. + +## Selecting fields + +Fields with `@Common.ValueList` or associations whose targets have `@cds.odata.valuelist` are included automatically. Disable recommendations for an individual field with `@UI.RecommendationState: 0`; dynamic expressions are supported as well. + +```cds +annotate Books with { + genre @UI.RecommendationState: (price > 200 ? 0 : 1); +} +``` + +Scalar fields without a value help can opt in with `@UI.RecommendationState`: + +```cds +entity CalibrationData : cuid { + measuringRangeMin : Decimal(16, 6) @UI.RecommendationState; + measuringRangeMax : Decimal(16, 6) @UI.RecommendationState; + description : String @UI.RecommendationState; +} +``` + +Numeric scalar fields without value helps use the `regression` task type. Other fields use `classification`. A numeric field with a value help remains a classification target. + +> [!NOTE] +> SAP Fiori Elements does not yet render recommendations for scalar fields without a value help. The backend provides them, but the client currently requests and displays recommendation fields only when they have `@Common.ValueList` or `@Common.ValueListWithFixedValues`. + +## Generated service shape + +For each draft-enabled entity with recommendable fields, the plugin adds: + +- `@UI.Recommendations: { '=': 'SAP_Recommendations' }` +- a virtual `_Recommendations` companion entity +- one recommendation array per included field + +Each recommendation contains `RecommendedFieldValue`, `RecommendedFieldDescription`, `RecommendedFieldScoreValue`, and `RecommendedFieldIsSuggestion`. Fiori Elements uses the first suggestion as the soft-fill default. + +Recommendations are calculated when a draft-entity `READ` expands `SAP_Recommendations`. Active-entity reads return no recommendations, and reads during `draftActivate` are skipped. + +## Prediction context and data handling + +The plugin sends up to 2,000 active rows of the same entity to SAP-RPT-1. Only rows for which every recommendation target is non-null are included. The active version of the current draft is replaced by the draft row containing `[PREDICT]` placeholders. + +The following elements are removed from the context: + +- `createdAt`, `createdBy`, `modifiedAt`, and `modifiedBy` +- `cds.LargeBinary` and `cds.Vector` elements +- fields excluded by `@UI.RecommendationState: 0` or a matching dynamic expression + +> [!IMPORTANT] +> All other selected columns are forwarded to SAP AI Core. Review the entity model and exclude sensitive fields explicitly. + +There is no sampling or `ORDER BY`; for entities with more than 2,000 qualifying rows, the database determines which rows are used. If `@Common.Text` is configured, the plugin performs an additional lookup to populate the recommendation description. + +## SAP-RPT-1 lifecycle + +The first prediction for a resource group creates an `sap-rpt-1-small` deployment in scenario `foundation-models` when none exists. The plugin waits for the deployment to reach `RUNNING` and reuses it afterward. + +Single-tenant applications use the configured `AICore.resourceGroup`, which defaults to `default`. Multitenant applications create a resource group per tenant during subscription and delete it during unsubscription. + +## Local development + +Without an SAP AI Core binding, the plugin uses `MockAICoreService`. It returns the first non-null value for each target column. This is useful for UI smoke tests but is not a quality signal. + +To use a real deployment locally, bind the application and start it with the `hybrid` profile: + +```sh +cds bind +cds watch --profile hybrid +``` diff --git a/.docs/vector-embeddings.md b/.docs/vector-embeddings.md new file mode 100644 index 0000000..ca4321d --- /dev/null +++ b/.docs/vector-embeddings.md @@ -0,0 +1,179 @@ +# Local vector embeddings + +> [!WARNING] +> Local vector embeddings, the SQLite extensions, local model management, and their CLI tooling are experimental and intended to improve local development. Breaking changes are expected. For production vector search and embeddings, use SAP HANA's vector engine. + +`@cap-js/ai` does not add vector functionality to SAP HANA Cloud. HANA already provides vector storage, [`VECTOR_EMBEDDING`](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-vector-engine-guide/vector-embedding-function-vector), and vector search natively. The SQLite implementation provides a similar development-time SQL shape, but it does not reproduce a HANA model or make locally generated vectors interchangeable with HANA-generated vectors. + +## Database kinds and dependencies + +`@cap-js/ai` redirects CAP's standard `sqlite` implementation instead of adding a separate database kind. With `@sap/cds` `^10.1`, the standard `sqlite:memory` preset inherits that implementation: + +- `sqlite` uses a file-based SQLite database. +- `sqlite:memory` uses an in-memory SQLite database. + +The redirect and synchronous embedding function require `@sap/cds` `^10.1` and `@cap-js/sqlite` `^3.1`. The package's other capabilities continue to support `@sap/cds` 9. + +This applies to every SQLite service in an application that installs `@cap-js/ai`. Its embedding model is provisioned and initialized when the service starts, even if the application does not call `VECTOR_EMBEDDING`. SAP HANA services are unaffected. + +Install the optional peers as development dependencies: + +```sh +npm add -D @cap-js/sqlite@^3.1 @huggingface/hub@^2.15.0 \ + @huggingface/tokenizers@0.1.3 onnxruntime-node@1.20.1 +``` + +`@huggingface/hub` is needed for model discovery and provisioning. `@huggingface/tokenizers` and `onnxruntime-node` are needed for inference. The exact ONNX Runtime version is currently required because synchronous SQLite functions use a version-specific native runtime interface. + +## Configuration + +Use the standard SQLite configuration: + +```json +{ + "cds": { + "requires": { + "db": { + "kind": "sqlite" + } + } + } +} +``` + +The current default model is `sentence-transformers/all-MiniLM-L6-v2`. It was selected only because, at the time of selection, it was the most-downloaded reasonably small model matching the sentence-similarity task, ONNX format, and Apache-2.0 license filters used for the sample. This is not a model recommendation. The default may change at any time while the feature is experimental, so configure `embedding.model` explicitly when the model choice must remain stable: + +```json +{ + "cds": { + "requires": { + "db": { + "kind": "sqlite", + "embedding": { + "model": "owner/model" + } + } + } + } +} +``` + +The built-in runtime reads `model` and the optional `directory`; additional properties are allowed for extensions. + +Without `directory`, the configured or default model is stored below `/.cds/models//`. If a valid installation is absent, startup prints a warning, downloads the model, generates `embedding.lock.json`, and reuses it on later starts. + +Use `sqlite:memory` when the application data itself need not survive a restart: + +```json +{ + "cds": { + "requires": { + "db": { + "kind": "sqlite:memory" + } + } + } +} +``` + +## Explicit provisioning + +Provision the project-local model before startup: + +```sh +npx @cap-js/ai install-model owner/model +``` + +The command finds the enclosing CAP project even when run from a subdirectory. It installs into `.cds/models` at the project root. + +To reuse a model across projects, select a shared cache root: + +```sh +npx @cap-js/ai install-model owner/model --directory ~/.cds/models +``` + +```json +{ + "cds": { + "requires": { + "db": { + "kind": "sqlite", + "embedding": { + "model": "owner/model", + "directory": "~/.cds/models" + } + } + } + } +} +``` + +`directory` is the cache root, so this example installs the artifacts below `~/.cds/models/owner/model`. Relative paths resolve from `cds.root`, absolute paths remain absolute, and `~/` resolves from the user's home directory. + +When `directory` is configured, startup treats it as a pre-installed cache. It validates the lock and files but does not download or modify them. Provision shared models in a controlled environment and consider making the directory read-only at runtime. + +## Compatibility check + +Inspect repository metadata without downloading model weights: + +```sh +npx @cap-js/ai check-model owner/model +``` + +This reports likely compatibility. `install-model` is definitive because it also downloads the artifacts, loads the ONNX model, verifies its inputs and output, and runs a probe inference. + +Prompt metadata is part of `embedding.lock.json`. Locks from earlier experimental versions that do not describe prompt semantics are rejected. Remove the affected model directory, then run `install-model` again to regenerate it. + +See [Choosing a model](model-selection.md) for discovery filters and the supported model contract. + +## SQL function + +Use the HANA-shaped function from CQL or SQL: + +```js +SELECT.from('Books').columns` + VECTOR_EMBEDDING(title, 'DOCUMENT', 'local') as embedding +`; +``` + +The service accepts both the three-argument form and a four-argument form with `remote_source`: + +```sql +VECTOR_EMBEDDING(text, text_type, model_and_version) +VECTOR_EMBEDDING(text, text_type, model_and_version, remote_source) +``` + +`text` is embedded. For models trained with query/document prompts, `text_type` applies the compatible prefix discovered from the model metadata. No additional configuration is normally needed, and `check-model` displays the detected mapping. + +If required prompts are not available in model metadata, configure them explicitly: + +```json +{ + "embedding": { + "model": "owner/model", + "prompts": { + "query": "search_query: ", + "document": "search_document: " + } + } +} +``` + +Configured `embedding.prompts` entries override the corresponding discovered entry; omitted entries continue using discovered metadata. For models with `include_prompt=false`, this runtime cannot apply discovered or configured prompts; For models that were not trained with prompts, prompt-free use is supported. +`model_and_version` and `remote_source` preserve the SQL shape for development compatibility but do not affect local inference. +SQL `NULL` returns `NULL`, and empty or whitespace-only text is treated as no value and returns a zero vector. +The result is a JSON string containing the model's vector dimensions. + +## Runtime behavior + +SQLite user-defined functions cannot await. Tokenization, inference, pooling, and normalization therefore run synchronously and block the Node.js event loop for each call. This tradeoff is acceptable only for local development and low-volume experiments. + +Each invocation embeds the first model input window. Longer input is truncated. Split long documents before persistence and store one vector per chunk when retrieval must cover the full text. + +## Provisioning and trust boundary + +Discovery resolves the repository to an immutable commit and generates an `embedding.lock.json` containing the selected artifacts, dimensions, input limit, pooling, normalization, sizes, and SHA-256 checksums. Downloads use bounded responses, timeouts, retries for transient failures, and size/checksum validation. Existing installations are checked for file changes before use. + +This is trust on first use, not publisher authentication. The initial installation trusts the selected public Hugging Face repository and its metadata. A self-consistent lock does not make an untrusted model safe. Installation loads the tokenizer and ONNX graph into native libraries and executes a probe in the current process. Use repositories you trust and prefer explicit provisioning in a controlled environment. + +Provisioning rejects symlinked model paths and unsafe artifact names. Conventional ONNX external-data sidecars adjacent to the selected graph are supported; arbitrary paths encoded in the ONNX protobuf are not. diff --git a/.gitignore b/.gitignore index 4104bfe..2ce6852 100644 --- a/.gitignore +++ b/.gitignore @@ -4,4 +4,5 @@ gen/ package-lock.json .env .cdsrc-private.json -resources/ \ No newline at end of file +resources/ +.cds/models/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 55fc7eb..5d697a3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ - The format is based on [Keep a Changelog](https://keepachangelog.com/). - This project adheres to [Semantic Versioning](https://semver.org/). +## Version 1.2.0 - tbd + +### Added + +- **Experimental:** Extend the standard `sqlite` service and its `sqlite:memory` preset for local CAP development with `VECTOR_EMBEDDING`, model provisioning tooling, and `sentence-transformers/all-MiniLM-L6-v2` as a replaceable default that may change while the feature remains experimental. +- **Experimental:** Add local `SPARQL_EXECUTE` and `sparql_table` support through the optional `oxigraph` peer dependency. + +Local vector embeddings require `@sap/cds` `^10.1` and `@cap-js/sqlite` `^3.1`; the package's other capabilities continue to support `@sap/cds` 9. ## Version 1.1.0 - 2026-07-20 diff --git a/README.md b/README.md index 7a2de53..e7777c1 100644 --- a/README.md +++ b/README.md @@ -1,244 +1,153 @@ [![REUSE status](https://api.reuse.software/badge/github.com/cap-js/ai)](https://api.reuse.software/info/github.com/cap-js/ai) -# SAP Cloud Application Programming Model, AI plugin for Node.js +# CAP AI plugin for Node.js -## About this project +`@cap-js/ai` adds UI recommendations powered by SAP AI Core, simplified access to SAP AI Core resources, and vector embedding support for CAP applications. -The SAP Cloud Application Programming Model, AI plugin for Node.js bundles two AI capabilities to infuse into your CAP applications: -1. UI Recommendations -2. Simplified AI Core usage +## Recommendations -> [!IMPORTANT] -> In multi tenancy scenarios with a sidecar the plugin must be included in the sidecar for SAP AI Core handling. +The plugin adds SAP-RPT-1 recommendations to draft-enabled entities. Fields with a value help are included automatically: -### 1. Use case: Recommendations - -Recommendations are implemented leveraging [SAP-RPT-1](https://help.sap.com/docs/sap-ai-core/generative-ai/sap-rpt-1) and AI Core. This plugin generically hooks into any entity which has properties with a value help (detected via `@Common.ValueList` on the property or `@cds.odata.valuelist` on the association target). - -```cds +```cds +@odata.draft.enabled entity Books { - key ID : Integer; - title : String(111); - descr : String(1111); - genre : Association to one Genres; - status : Association to one Status; + key ID : Integer; + title : String; + genre : Association to Genres; + price : Decimal; } + annotate Genres with @cds.odata.valuelist; -annotate Books with { - status @Common.ValueList : { - CollectionPath : 'Status', - Parameters: [ - { - $Type: 'Common.ValueListParameterInOut' - ValueListProperty : 'code', - LocalDataProperty : status_code - } - ] - } -} ``` ![Recommendations as default values](./_assets/recommendation-default.png) -![Recommendation in Value Help](./_assets/recommendation-value-help.png) -![Accept recommendations](./_assets/accept-recommendations.png) -The genre field on the UI now automatically has recommendations. If you do not want recommendations for a specific field, it can be annotated with `@UI.RecommendationState`. +Use `@UI.RecommendationState` to opt individual fields in or out: ```cds annotate Books with { - genre @UI.RecommendationState : 0; + genre @UI.RecommendationState: 0; + price @UI.RecommendationState; } ``` -Dynamic expressions as values for `@UI.RecommendationState`, work as well! +A production deployment requires an [SAP AI Core](https://help.sap.com/docs/sap-ai-core) service binding. Without one, local development uses a mock implementation for UI smoke tests. See [Recommendations](.docs/recommendations.md) for regression targets, request behavior, data handling, and deployment lifecycle. -```cds -annotate Books with { - genre @UI.RecommendationState : (price > 200 ? 0 : 1); -} -``` +## SAP AI Core -#### Regression Recommendations on fields without a value help +The plugin provides an `AICore` CAP service for managing resource groups, deployments, and configurations: -By default, the plugin only enhances fields that have a value help list since these columns are good prediction targets for classification. However, some fields are good targets but have no value list: free-form numerics like measurement ranges, calibration values, or planning estimates. Annotate these with `@UI.RecommendationState` to opt in: +```js +const aiCore = await cds.connect.to('AICore'); +const { resourceGroups, deployments } = aiCore.entities; -```cds -entity CalibrationData : cuid { - measuringRangeMin : Decimal(16, 6) @UI.RecommendationState; - measuringRangeMax : Decimal(16, 6) @UI.RecommendationState; - operatingPoint : Decimal(16, 6) @UI.RecommendationState; - description : String @UI.RecommendationState; -} +const groups = await aiCore.run(SELECT.from(resourceGroups)); +await aiCore.stop(deployments, { id: '' }); ``` -The annotation only takes effect on **scalar** elements (no associations / compositions / unmanaged elements; for those, attach a value help instead). Annotated fields are added to the entity's `_Recommendations` companion just like value-helped fields, and Fiori Elements' soft-fill placeholder renders the prediction in the empty input. - -`task_type` is chosen automatically per column: -- numeric scalar (`Integer*`, `Decimal`, `Double`) annotated with `@UI.RecommendationState` → **`regression`** so RPT-1 can interpolate continuous values, -- everything else → **`classification`**. +See [SAP AI Core integration](.docs/ai-core.md) for setup, supported queries, helper methods, and multitenancy. -> [!NOTE] -> Numeric fields that have a value help (e.g. a fixed price-point list) stay on classification — `@UI.RecommendationState` is only needed when there is *no* value help. Combining both is unnecessary. +## Local vector embeddings with SQLite (experimental) > [!WARNING] -> SAP Fiori Elements does not yet support rendering recommendations for scalar fields without a value help. The backend correctly provides predictions for these fields, but the Fiori Elements client currently only requests and displays recommendations for fields annotated with `@Common.ValueList` or `@Common.ValueListWithFixedValues`. - -
-How recommendations work under the hood - -A short FAQ for integrators, so you don't have to read the source. - - -**What does the plugin emit on the OData service?** -On every draft-enabled entity that has at least one value-helped field, it adds an entity-level annotation `@UI.Recommendations: { '=': 'SAP_Recommendations' }` plus a synthetic companion entity (`_Recommendations`, `@cds.persistence.skip`) with one virtual array per recommendable field. Each item carries `RecommendedFieldValue`, `RecommendedFieldDescription`, `RecommendedFieldScoreValue` and `RecommendedFieldIsSuggestion` — the shape Fiori Elements expects for `UI.RecommendationListType`. The first entry per field has `RecommendedFieldIsSuggestion: true` and is rendered as the soft-fill default. - -**When does it run?** -On READ requests to a draft entity that expand `SAP_Recommendations`. Reads against the active entity return nothing in that field. Reads during `draftActivate` are skipped. +> The SQLite extensions, local vector embeddings, local model management, and the related tooling are experimental facilities for local development. Breaking changes are expected, including changes caused by SQLite's synchronous function interface and by local model management. Use SAP HANA's vector engine for production vector workloads. -**What data is sent to RPT-1 as context?** -Up to 2000 rows from the **active** version of the same entity, restricted to rows where every recommendable field is non-null. The columns `createdAt`, `createdBy`, `modifiedAt`, `modifiedBy` plus any `cds.LargeBinary` / `cds.Vector` elements are stripped. The active row corresponding to the draft (if any) is removed and replaced by the draft row carrying `[PREDICT]` placeholders in the columns to predict. There is no sampling or `ORDER BY` — for tables larger than 2000 rows, which rows make the cut is determined by the database. +Here is a complete Bookshop example. -> [!IMPORTANT] -> Everything in the remaining columns is forwarded to AI Core. Annotate sensitive fields with `@UI.RecommendationState : 0` (or a dynamic expression) to keep them out of both the predictions and the context payload. +1. Install the local-development dependencies: -**How are descriptions populated?** -For each predicted value, the plugin issues an extra SELECT against the field's `@Common.Text` association (if set) to fetch the human-readable label. Fields without `@Common.Text` get an empty `RecommendedFieldDescription`. + ```sh + npm add -D @cap-js/ai @cap-js/sqlite@^3.1 @huggingface/hub@^2.15.0 \ + @huggingface/tokenizers@0.1.3 onnxruntime-node@1.20.1 + ``` -**RPT-1 deployment lifecycle** -First prediction call against a resource group provisions an `sap-rpt-1-small` deployment in scenario `foundation-models` (executable `aicore-sap`) and polls up to 10× with exponential backoff until it reaches `RUNNING`. Subsequent calls reuse the cached deployment. Single-tenant uses the configured `resourceGroup` (default `'default'`); multi-tenant creates one resource group per tenant on subscribe (label `ext.ai.sap.com/CDS_TENANT_ID`) and deletes it on unsubscribe. + Local vector embeddings require `@sap/cds` `^10.1` and `@cap-js/sqlite` `^3.1`; the package's other capabilities continue to support `@sap/cds` 9. -**Local development** -Without an AI Core binding the plugin uses `MockAICoreService`, which returns the first non-null value of each target column from the context as the "prediction" — useful for UI smoke tests, useless as a quality signal. Run `cds bind ` and start with profile `hybrid` to talk to a real AI Core deployment locally. +2. Use the standard SQLite service. Most CAP projects already do this in development; an explicit configuration looks like this: -
+ ```json + { + "cds": { + "requires": { + "db": { + "kind": "sqlite" + } + } + } + } + ``` -### 2. Use case: Simplified AI Core usage +3. Add an embedding preview to the Bookshop service. In its service implementation, which imports `cds` from `@sap/cds`: -The plugin introduces an `AICore` CAP service that automatically performs some administrative tasks and offers simplified access to AI Core. + ```cds + function embedding(text : String) returns LargeString; + ``` -#### Automatic operations + ```js + this.on('embedding', async (req) => { + const [row] = await cds.db.run( + `SELECT VECTOR_EMBEDDING(?, 'DOCUMENT', 'local') AS embedding`, + [req.data.text] + ); + return row.embedding; + }); + ``` -- The plugin automatically creates a new SAP AI Core resource group per tenant during tenant onboarding and deletes it during offboarding. -- The plugin automatically creates an RPT-1 deployment per resource group for the recommendations feature. +4. Start the application and call the function: -#### Simplified AI Core API access + ```sh + cds w + ``` -```js -const aiCore = await cds.connect.to('AICore'); -const {resourceGroups, deployments, configurations} = aiCore.entities; -await aiCore.run(SELECT.from(resourceGroups)); -await aiCore.run(SELECT.from(resourceGroups).where({tenantId: cds.context.tenant})); -await aiCore.run(SELECT.from(deployments).where({'resourceGroup.resourceGroupId': resourceGroups[0].resourceGroupId})); -await aiCore.run(SELECT.from(configurations).where({'resourceGroup.resourceGroupId': resourceGroups[0].resourceGroupId})); -``` - -Currently, the following `cds.ql` operations are supported: - -| Operation | resourceGroups | deployments | configurations | -|-----------|---------------|-------------|----------------| -| **READ (list)** | ✓ | ✓ | ✓ | -| - limit | ✓ | ✓ | ✓ | -| - where* | `tenantId`, `resourceGroupId` | `resourceGroup.resourceGroupId` | `resourceGroup.resourceGroupId` | -| - search | - | - | ✓ | -| **READ (single)** | ✓ | ✓ | ✓ | -| **CREATE** | ✓ | ✓ | ✓ | -| **UPDATE** | ✓ | ✓ | - | -| - where* | `tenantId`, `resourceGroupId` | `id`, `resourceGroup.resourceGroupId` | - | -| **UPSERT** | ✓ | ✓ | - | -| - where* | - | `id`, `resourceGroup.resourceGroupId` | - | -| **DELETE** | ✓ | ✓ | - | -| - where* | `tenantId`, `resourceGroupId` | `id`, `resourceGroup.resourceGroupId` | - | - -\* Only simple equality checks against the listed properties are supported - -Next to CRUD operations the following helper functions can be used: - -```js -const aiCore = await cds.connect.to('AICore'); -const {resourceGroups, deployments, configurations} = aiCore.entities; + In another terminal: -// Fetch a resource group for a CDS tenant ID -const resourceGroupId = await aiCore.resourceGroupForTenant(cds.context.tenant) + ```sh + curl 'http://localhost:4004/odata/v4/catalog/embedding(text=%27A%20book%20about%20travel%27)' + ``` -// Call the RPT-1 API to fetch predictions - see AICoreService.cds for the schema -const predictions = await aiCore.predictRowColumns(/** RPT-1 payload */) +`@cap-js/ai` redirects the standard `sqlite` implementation to add the local capabilities. With `@sap/cds` `^10.1`, the standard `sqlite:memory` preset inherits that implementation. The first start warns that the default model is missing, downloads it to `.cds/models`, and then initializes it. The response's `value` contains a JSON-encoded vector with 384 numbers. Later starts reuse the installed model. -/** - * Returns the deployment ID for RPT-1. If no RPT-1 deployment exists, creates one for the - * resource group -*/ -const rpt1DeploymentId = await aiCore.rpt1DeploymentId(resourceGroups, {resourceGroupId}) +The current default is [`sentence-transformers/all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It was selected only because, at the time of selection, it was the most-downloaded reasonably small model matching the sentence-similarity task, ONNX format, and Apache-2.0 license filters below. This is not a recommendation, and the default may change at any time while this feature is experimental. Configure `cds.requires.db.embedding.model` explicitly if the choice must remain stable. -// Stops an AI Core deployment -await aiCore.stop(deployments, {id: ''}) -``` - -## Requirements and Setup - -To use the plugin in production scenarios you need an [SAP AI Core](https://help.sap.com/docs/sap-ai-core) service binding. The plugin will automatically create resource groups per tenant in multi-tenancy scenarios and create an RPT-1 deployment in each for the recommendations feature. In single-tenant setups the plugin uses the 'default' resource group and creates an RPT-1 deployment as well if none exists. +Start model discovery with [trending Apache-2.0 sentence-similarity models that provide ONNX artifacts](https://huggingface.co/models?pipeline_tag=sentence-similarity&library=onnx&license=license:apache-2.0&sort=trending). These filters select a relevant task, a locally runnable format, and a permissive license, but they do not guarantee compatibility. Choose a model as big as necessary and as small as possible, then validate it with the provided tooling. -For single-tenant deployments you can change the resource group as follows: - -```json -{ - "cds": { - "requires": { - "AICore": { - "resourceGroup": "CUSTOM_SINGLE_TENANT_RESOURCE_GROUP" - } - } - } -} -``` +See [Choosing a model](.docs/model-selection.md) and [Local vector embeddings](.docs/vector-embeddings.md) for compatibility checks, explicit or shared provisioning, runtime behavior, and limitations. -For Cloud Foundry apps an example config could look like this: - -```yaml -modules: - - name: incidents-srv - type: nodejs - path: gen/srv - requires: - - name: incidents-ai-core -resources: - - name: incidents-ai-core - type: org.cloudfoundry.managed-service -``` +## Advanced +- [Recommendations](.docs/recommendations.md) — generated service shape, prediction context, regression targets, and lifecycle +- [SAP AI Core integration](.docs/ai-core.md) — bindings, multitenancy, supported operations, and helper methods +- [Local vector embeddings](.docs/vector-embeddings.md) — SQLite kinds, model provisioning, SQL function behavior, and trust boundaries +- [Choosing a model](.docs/model-selection.md) — Hugging Face filters, compatibility requirements, and size tradeoffs +- [Local knowledge graph](.docs/knowledge-graph.md) — experimental `SPARQL_EXECUTE` and `sparql_table` support ## Test the plugin locally -In `tests/bookshop-app/` you can find a sample application that is used to demonstrate how to use the plugin and to run tests against it. - -### Local Testing +The sample application is in `tests/bookshop`. -To execute local tests, simply run: - -```bash -npm run test +```sh +npm test ``` -For tests, the `cds-test` Plugin is used to spin up the application. More information about `cds-test` can be found [here](https://cap.cloud.sap/docs/node.js/cds-test). - -For integration tests you need an AI Core binding. +Integration tests require an SAP AI Core binding: -```bash +```sh cds bind ai-core -2 npm run test:hybrid ``` -## Support, Feedback, Contributing +## Support, feedback, and contributing -This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/cap-js/ai/issues). Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](CONTRIBUTING.md). +This project welcomes feature requests, bug reports, and contributions through [GitHub issues](https://github.com/cap-js/ai/issues). See the [Contribution Guidelines](CONTRIBUTING.md) for development information. -## Security / Disclosure +## Security -If you find any bug that may be a security problem, please follow our instructions [in our security policy](https://github.com/cap-js/ai/security/policy) on how to report it. Please do not create GitHub issues for security-related doubts or problems. +Report potential security issues through the project's [security policy](https://github.com/cap-js/ai/security/policy), not through public issues. ## Code of Conduct -We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone. By participating in this project, you agree to abide by its [Code of Conduct](https://github.com/cap-js/.github/blob/main/CODE_OF_CONDUCT.md) at all times. +Participation in this project is governed by the [Code of Conduct](https://github.com/cap-js/.github/blob/main/CODE_OF_CONDUCT.md). ## Licensing -Copyright 2026 SAP SE or an SAP affiliate company and ai contributors. Please see our [LICENSE](LICENSE) for copyright and license information. Detailed information including third-party components and their licensing/copyright information is available [via the REUSE tool](https://api.reuse.software/info/github.com/cap-js/ai). +Copyright 2026 SAP SE or an SAP affiliate company and ai contributors. See [LICENSE](LICENSE) and the [REUSE report](https://api.reuse.software/info/github.com/cap-js/ai). diff --git a/bin/cds-ai.js b/bin/cds-ai.js new file mode 100755 index 0000000..788d63f --- /dev/null +++ b/bin/cds-ai.js @@ -0,0 +1,10 @@ +#!/usr/bin/env node + +import { runModelCommand } from '../lib/vector_embedding/cli.js'; + +try { + await runModelCommand(process.argv.slice(2)); +} catch (error) { + process.stderr.write(`${error.message}\n`); + process.exitCode = 1; +} diff --git a/embeddings.md b/embeddings.md deleted file mode 100644 index 41c0b7b..0000000 --- a/embeddings.md +++ /dev/null @@ -1,66 +0,0 @@ -## Simplified embeddings - -For natural language processing it is crucial to embed text data into a Vector. HANA Cloud offers a `VECTOR_EMBEDDING` function via which an embedding can be generated. The model which can be specified can either be an SAP model, when [HANA Cloud NLP](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-vector-engine-guide/creating-text-embeddings-with-nlp-51eb170d038d4099a9bbb85c08fda888?locale=en-US) is enabled or a [model provided in SAP AI Core](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-vector-engine-guide/creating-text-embeddings-with-sap-ai-core?locale=en-US), like the ones from OpenAI or AWS. - -You can add embeddings columns like: - -```cds -entity Books { - key ID : Integer; - title : String(111); - descr : String(1111); - @cds.api.ignore - embedding : Vector = (VECTOR_EMBEDDING(descr, 'DOCUMENT', 'amazon--titan-embed-text."1.2"')) stored; -} -``` - -HANA Cloud has native models for text embedding when their Natural Language Processing feature is enabled: `SAP_GXY.20250407` and `SAP_NEB.20240715`. However HANA Cloud can also be connected to AI Core via a remote source, and then embedding models from OpenAI and AWS can be used as well. The remote source defaults to 'AI_CORE' and can be customized via `cds.env.ai.embeddings.remoteSource`. - -> [!INFO] -> The fourth parameter is the remote source in HANA Cloud which is mandatory for models provided by SAP AI Core. The plugin will automatically fill it with the default remote source `cds.env.ai.embeddings.remoteSource` if the parameter is not provided. - -### Using non SAP models for embeddings with SAP HANA Cloud - -Currently the setup is not ideal when models provided by SAP AI Core shall be used. You have to complete the following steps to get it to work: - -1. Follow the [Creating Text Embeddings with SAP AI Core](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-vector-engine-guide/creating-text-embeddings-with-sap-ai-core?locale=en-US) documentation in SAP Help. -2. After creating the PSE and the remote source in HANA Cloud, you need to grant the privileges for referencing the remote source to a user, which in turn can grant it to the HDI user for your CAP application. The following SQL creates a user group which can send requests to the remote source, creates a user and grants the user permissions to grant other users permissions to send requests to the remote source. - - ```sql - CREATE ROLEGROUP HDI_GRANTOR_GROUP; - - CREATE ROLE HC_REMOTESOURCE_GRANTOR SET ROLEGROUP HDI_GRANTOR_GROUP; - - GRANT EXECUTE ON REMOTE SOURCE TO HC_REMOTESOURCE_GRANTOR WITH GRANT OPTION; - - -- Choose a unique password - ALTER USER HDI_GRANT_USER PASSWORD NO FORCE_FIRST_PASSWORD_CHANGE; - GRANT HC_REMOTESOURCE_GRANTOR TO HDI_GRANT_USER WITH GRANT OPTION; - ``` - -> [!NOTE] -> If all HDI containers are allowed to access this remote source, you can run `GRANT EXECUTE ON REMOTE SOURCE TO _SYS_DI#BROKER_CG._SYS_DI_OO_DEFAULTS` instead of doing steps 2-4, because this grants the remote execute privileges to the role which is granted to all HDI containers. - -3. Create a user provided service on BTP with the credentials for the user: - - ```ssh - cf cups hana_ai -p '{"username":"HDI_GRANT_USER","password":"", "tags": ["hana"]}' - ``` - -4. Create an `.hdbgrants` file in `db/src`. HDI will pick this up during deployment and use the permissions of the user to grant its permissions to the HDI users. - - ```json - { - "hana_ai": { - "object_owner": { - "roles": ["HC_REMOTESOURCE_GRANTOR"] - }, - "application_user": { - "roles": ["HC_REMOTESOURCE_GRANTOR"] - } - } - } - ``` - -> [!WARNING] -> In Multi-Tenancy scenarios you would have to create a remote source per tenant and assign the reference privilege to the respective tenant binding. The remote source per tenant should be done because in AI Core each tenant should have a different resource group for isolation. diff --git a/lib/knowledge-graph/triplestore.js b/lib/knowledge-graph/triplestore.js new file mode 100644 index 0000000..8a48f6d --- /dev/null +++ b/lib/knowledge-graph/triplestore.js @@ -0,0 +1,88 @@ +let oxigraph; +try { + oxigraph = await import('oxigraph'); +} catch (err) { + if (err.code !== 'ERR_MODULE_NOT_FOUND') throw err; +} + +import { pipeline } from 'node:stream/promises'; +import { text } from 'node:stream/consumers'; +import { createReadStream } from 'node:fs'; +import { realpath } from 'node:fs/promises'; +import { createGunzip } from 'node:zlib'; + +import cds from '@sap/cds'; +const { path } = cds.utils; + +const formats = { + '.jsonld': 'application/ld+json', + '.nq': 'application/n-quads', + '.nt': 'application/n-triples', + '.rdf': 'application/rdf+xml', + '.trig': 'application/trig', + '.ttl': 'text/turtle' +}; + +export default class TripleStore extends (oxigraph?.Store || class Store {}) { + async load(file, graph) { + this._ready(); + + const root = await realpath(path.resolve(cds.root)); + const resolved = path.resolve(root, file); + if (!resolved.startsWith(`${root}${path.sep}`)) { + throw new Error(`Cannot load RDF data from outside the project: ${file}`); + } + + let ext = path.extname(resolved).toLowerCase(); + if (ext.endsWith('.gz')) { + ext = path.extname(resolved.slice(0, -3)).toLowerCase(); + } + const format = formats[ext]; + if (!format) throw new Error(`Unsupported RDF file format: ${ext || '(none)'}`); + + // Resolve the target before opening it: a project-local symlink must not make + // files outside of cds.root available through SPARQL LOAD. + const target = await realpath(resolved); + if (!target.startsWith(`${root}${path.sep}`)) { + throw new Error(`Cannot load RDF data from outside the project: ${file}`); + } + + const graphNode = graph == null ? oxigraph.defaultGraph() : oxigraph.namedNode(graph); + const steps = [createReadStream(target)]; + if (path.extname(resolved).toLowerCase() === '.gz') steps.push(createGunzip()); + steps.push(text); + return super.load(await pipeline(...steps), { format, to_graph_name: graphNode }); + } + + query(query, headers) { + this._ready(); + + const accept = ( + headers?.split('\r\n').find((header) => /accept:/i.test(header)) ?? + 'accept:application/sparql-results+json' + ) + .replace(/accept:/i, '') + .trim(); // strip HTTP header formatting + + const RESPONSE = super.query(query, { + use_default_graph_as_union: true, + results_format: accept + }); + return { RESPONSE }; + } + + async execute(query, headers) { + this._ready(); + + if (!/^\s*LOAD\b/i.test(query)) return this.query(query, headers); + + const match = /^\s*LOAD\s+<([^>]*)>(?:\s+INTO\s+GRAPH\s+<([^>]*)>)?\s*$/i.exec(query); + if (!match) throw new Error(`Unsupported LOAD syntax: ${query}`); + return this.load(match[1], match[2]); + } + + _ready() { + if (!oxigraph) + throw new Error(`Cannot find 'oxigraph'. Make sure to install it with 'npm i oxigraph'`); + } +} diff --git a/lib/sqlite/AISQLiteService.js b/lib/sqlite/AISQLiteService.js new file mode 100644 index 0000000..f8bda8b --- /dev/null +++ b/lib/sqlite/AISQLiteService.js @@ -0,0 +1,112 @@ +import cds from '@sap/cds'; +import { createEmbeddingRuntime } from '../vector_embedding/embedding.js'; +import TripleStore from '../knowledge-graph/triplestore.js'; +import { loadSQLiteService } from './load-sqlite.js'; + +const SQLiteService = loadSQLiteService(); + +const LOG = cds.log('@cap-js/ai'); +const $tripleStore = Symbol('tripleStore'); + +export default class AISQLiteService extends SQLiteService { + _tripleStores = new Map(); + + async init() { + this._embeddingRuntime = await createEmbeddingRuntime(this.options.embedding, { + root: cds.root, + warn: (message) => LOG.warn(message) + }); + try { + const service = await super.init(); + LOG.info('Vector embedding ONNX model initialized'); + return service; + } catch (error) { + await this._embeddingRuntime.dispose().catch(() => {}); + this._embeddingRuntime = undefined; + throw error; + } + } + + async disconnect(tenant) { + try { + return await super.disconnect(tenant); + } finally { + if (tenant == null) { + this._tripleStores.clear(); + await this._embeddingRuntime?.dispose(); + this._embeddingRuntime = undefined; + } else this._tripleStores.delete(tenant); + } + } + + get factory() { + const factory = super.factory; + const create = factory.create; + factory.create = async (tenant) => { + const dbc = await create(tenant); + const embedding = (input, textType, modelAndVersion) => + input == null + ? null + : this._embeddingRuntime.vectorEmbedding(String(input), textType, modelAndVersion); + const deterministic = { deterministic: true }; + dbc.function('VECTOR_EMBEDDING', { ...deterministic, varargs: true }, embedding); + dbc.function('VECTOR_EMBEDDING', deterministic, embedding); + + const key = tenant ?? ''; + const store = this._tripleStores.get(key) ?? new TripleStore(); + this._tripleStores.set(key, store); + dbc[$tripleStore] = store; + dbc.function('sparql_table', (query) => store.query(query).RESPONSE); + return dbc; + }; + return factory; + } + + onPlainSQL(req, next) { + const { query } = req; + if (!/^\s*CALL\s+SPARQL_EXECUTE\b/i.test(query)) return super.onPlainSQL(req, next); + + const match = + /^\s*CALL\s+SPARQL_EXECUTE\s*\(\s*'((?:[^']|'')*)'\s*,\s*'((?:[^']|'')*)'\s*,\s*\?\s*,\s*\?\s*\)\s*;?\s*$/i.exec( + query + ); + if (!match) throw new Error(`Unsupported SPARQL_EXECUTE syntax: ${query}`); + + const store = this.dbc?.[$tripleStore]; + if (!store) throw new Error('SPARQL_EXECUTE requires an active database connection'); + const unescape = (value) => value.replace(/''/g, "'"); + return store.execute(unescape(match[1]), unescape(match[2])); + } + + static CQN2SQL = class CQN2AISQLite extends SQLiteService.CQN2SQL { + static Functions = { + ...SQLiteService.CQN2SQL.Functions, + sparql_table(query) { + if (typeof query.val !== 'string') { + throw new Error('sparql_table expects a literal SPARQL SELECT query'); + } + + // Keep the SQL projection deliberately narrow, but accept the SPARQL + // prologue and the optional WHERE keyword (both are valid SPARQL). + const iri = '<(?:[^>\\\\]|\\\\.)*>'; + const prologue = `(?:(?:BASE\\s+${iri}|PREFIX\\s+(?:[A-Za-z][A-Za-z0-9._-]*)?:\\s*${iri})\\s*)*`; + const match = new RegExp( + `^\\s*${prologue}SELECT\\s+(?:(?:DISTINCT|REDUCED)\\s+)?((?:[?$][A-Za-z_][A-Za-z0-9_]*\\s*)+)(?:WHERE\\s*)?\\{`, + 'is' + ).exec(query.val); + if (!match) { + throw new Error('sparql_table only supports explicitly projected SPARQL variables'); + } + + const projection = match[1]; + const variables = projection.match(/[?$][A-Za-z_][A-Za-z0-9_]*/g); + + const columns = variables.map((variable) => variable.slice(1)); + const select = columns.map( + (column) => `value->>'$.${column}.value' as ${this.quote(column)}` + ); + return `(SELECT ${select} FROM json_each(sparql_table(${query})->'$.results.bindings'))`; + } + }; + }; +} diff --git a/lib/sqlite/load-sqlite.js b/lib/sqlite/load-sqlite.js new file mode 100644 index 0000000..d0ed2f5 --- /dev/null +++ b/lib/sqlite/load-sqlite.js @@ -0,0 +1,23 @@ +import { createRequire } from 'node:module'; + +const require = createRequire(import.meta.url); + +function loadSQLiteService(requireModule = require) { + try { + const module = requireModule('@cap-js/sqlite'); + return module.default ?? module; + } catch (error) { + if ( + (error?.code === 'ERR_MODULE_NOT_FOUND' || error?.code === 'MODULE_NOT_FOUND') && + /['"]@cap-js\/sqlite['"]/.test(error.message) + ) { + throw new Error( + "Using @cap-js/ai with SQLite requires @cap-js/sqlite. Install it with 'npm add -D @cap-js/sqlite'.", + { cause: error } + ); + } + throw error; + } +} + +export { loadSQLiteService }; diff --git a/lib/vector_embedding/SynchronousInferenceSession.js b/lib/vector_embedding/SynchronousInferenceSession.js new file mode 100644 index 0000000..ad2c5ce --- /dev/null +++ b/lib/vector_embedding/SynchronousInferenceSession.js @@ -0,0 +1,97 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. +// +// Synchronous counterpart to onnxruntime-node's public InferenceSession. SQLite +// user-defined functions cannot await its asynchronous run API. +import { createRequire } from 'module'; +import { loadOnnxRuntime } from './load-onnx-runtime.js'; + +const require = createRequire(import.meta.url); +const { ort, binding } = loadOnnxRuntime(require); + +class SynchronousInferenceSession { + #session; + #inputNames; + #outputNames; + + constructor(pathOrBuffer) { + if (typeof pathOrBuffer !== 'string' && !(pathOrBuffer instanceof Uint8Array)) { + throw new TypeError('Expected an ONNX model path or Uint8Array'); + } + + const session = new binding.InferenceSession(); + try { + if (typeof pathOrBuffer === 'string') { + session.loadModel(pathOrBuffer, {}); + } else { + session.loadModel( + pathOrBuffer.buffer, + pathOrBuffer.byteOffset, + pathOrBuffer.byteLength, + {} + ); + } + this.#inputNames = Object.freeze([...session.inputNames]); + this.#outputNames = Object.freeze([...session.outputNames]); + this.#session = session; + } catch (error) { + try { + session.dispose(); + } catch { + // Preserve the model loading error. + } + throw error; + } + } + + get inputNames() { + return this.#inputNames; + } + + get outputNames() { + return this.#outputNames; + } + + run(feeds) { + const session = this.#session; + if (!session) throw new Error('Inference session has been disposed'); + if (typeof feeds !== 'object' || feeds === null || Array.isArray(feeds)) { + throw new TypeError("'feeds' must be an object that uses input names as keys."); + } + + const nativeFeeds = Object.fromEntries( + this.#inputNames.map((name) => { + const feed = feeds[name]; + if (feed === undefined) throw new Error(`input '${name}' is missing in 'feeds'.`); + return [ + name, + feed instanceof ort.Tensor ? feed : new ort.Tensor(feed.type, feed.data, feed.dims) + ]; + }) + ); + const fetches = Object.fromEntries(this.#outputNames.map((name) => [name, null])); + const results = session.run(nativeFeeds, fetches, {}); + + return Object.fromEntries( + Object.entries(results).map(([name, result]) => [ + name, + result instanceof ort.Tensor + ? result + : new ort.Tensor(result.type, result.data, result.dims) + ]) + ); + } + + dispose() { + const session = this.#session; + if (!session) return; + this.#session = undefined; + session.dispose(); + } + + static async create(pathOrBuffer) { + return new SynchronousInferenceSession(pathOrBuffer); + } +} + +export { SynchronousInferenceSession }; diff --git a/lib/vector_embedding/cli.js b/lib/vector_embedding/cli.js new file mode 100644 index 0000000..9e133f8 --- /dev/null +++ b/lib/vector_embedding/cli.js @@ -0,0 +1,121 @@ +import fs from 'node:fs'; +import path from 'node:path'; + +import { installModel } from './model-install.js'; +import { validateEmbeddingModel } from './embedding.js'; +import { checkModel } from './model-discovery.js'; + +const HELP = `Usage: + npx @cap-js/ai check-model + npx @cap-js/ai install-model [--directory ] + +Options: + --directory Use this model-cache root (relative to the CAP project root) + --help Show this help +`; + +async function runModelCommand(argv, options = {}) { + const cwd = options.cwd ?? process.cwd(); + const root = options.root ?? findProjectRoot(cwd); + const { stdout = process.stdout } = options; + const command = parseArguments(argv); + if (command.help) { + stdout.write(HELP); + return; + } + + if (command.name === 'check-model') { + const check = options.check ?? checkModel; + const result = await check(command.model, { + fetchImpl: options.fetchImpl, + hubClient: options.hubClient, + hubUrl: options.hubUrl + }); + stdout.write(formatModelCheck(result)); + return result; + } + + const install = options.install ?? installModel; + const { modelDir } = await install(command.model, { + root, + directory: command.directory, + home: options.home, + fetchImpl: options.fetchImpl, + hubUrl: options.hubUrl, + discover: options.discover, + validate: options.validate ?? validateEmbeddingModel, + timeoutMs: options.timeoutMs, + retryMs: options.retryMs + }); + stdout.write(`Installed ${command.model} in ${modelDir}\n`); +} + +function findProjectRoot(start = process.cwd()) { + let directory = path.resolve(start); + while (true) { + try { + const pkg = JSON.parse(fs.readFileSync(path.join(directory, 'package.json'), 'utf8')); + const dependencies = { ...pkg.dependencies, ...pkg.devDependencies, ...pkg.peerDependencies }; + if (pkg.cds !== undefined || dependencies['@sap/cds']) return directory; + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } + const parent = path.dirname(directory); + if (parent === directory) return path.resolve(start); + directory = parent; + } +} + +function parseArguments(argv) { + if (argv.length === 0 || argv.includes('--help') || argv.includes('-h')) return { help: true }; + const name = argv[0]; + if (name !== 'check-model' && name !== 'install-model') { + throw new Error(`Unsupported command.\n\n${HELP}`); + } + + let model; + let directory; + for (let index = 1; index < argv.length; index++) { + const argument = argv[index]; + if (argument === '--directory') { + if (name !== 'install-model') { + throw new Error("Unknown option '--directory' for check-model"); + } + const value = argv[++index]; + if (!value || value.startsWith('--')) throw new Error(`${argument} requires a value`); + directory = value; + continue; + } + if (argument.startsWith('-')) throw new Error(`Unknown option '${argument}'`); + if (model) throw new Error(`Unexpected argument '${argument}'`); + model = argument; + } + + if (!model) throw new Error('Specify a model name'); + return { name, directory, model }; +} + +function formatModelCheck(model) { + const modelFile = model.files.find(({ role }) => role === 'model'); + const prompts = model.prompts + ? `\nText-type prompts:\n${['query', 'document'] + .filter((name) => model.prompts[name] !== undefined) + .map((name) => ` ${name.toUpperCase()}: ${JSON.stringify(model.prompts[name])}`) + .join('\n')}` + : '\nText-type prompts: none'; + return `Likely compatible: ${model.repository} +Revision: ${model.revision} +Task: ${model.task ?? 'not declared'} +ONNX: ${modelFile.path} +Dimensions: ${model.dimensions} +Maximum input length: ${model.maxLength} +Expected ONNX output: ${model.output.name} +Pooling: ${model.output.pooling} +Normalization: ${model.output.normalize ? 'enabled' : 'disabled'} +Prompt tokens in pooling: ${model.output.includePrompt ? 'included' : 'excluded'}${prompts} + +Run 'npx @cap-js/ai install-model ${model.repository}' for definitive ONNX Runtime validation. +`; +} + +export { HELP, findProjectRoot, formatModelCheck, parseArguments, runModelCommand }; diff --git a/lib/vector_embedding/embedding.js b/lib/vector_embedding/embedding.js new file mode 100644 index 0000000..5a7d9c1 --- /dev/null +++ b/lib/vector_embedding/embedding.js @@ -0,0 +1,419 @@ +import { installModel } from './model-install.js'; +import { + getModelDirectory, + getModelRoot, + loadModelAndTokenizer, + readModelLock, + verifyModelDirectory +} from './model-utils.js'; + +const STANDARD_INPUT_NAMES = new Set(['input_ids', 'attention_mask', 'token_type_ids']); +const DEFAULT_EMBEDDING_MODEL = 'sentence-transformers/all-MiniLM-L6-v2'; + +// Map HANA's text_type argument onto the model's discovered Sentence Transformers prompts. +// Prompt-less models (no `model.prompts`) always resolve to '' and stay byte-identical. +function promptFor(prompts, textType) { + if (!prompts) return ''; + const key = typeof textType === 'string' ? textType.toLowerCase() : ''; + if (key === 'query') return prompts.query ?? ''; + if (key === 'document') return prompts.document ?? ''; + return ''; +} + +async function createEmbeddingRuntime(configuration, options = {}) { + const { model, modelDir } = await resolveEmbeddingModel(configuration, options); + return createEmbeddingRuntimeFromModel(modelDir, model); +} + +async function createEmbeddingRuntimeFromModel(modelDir, model) { + const { session, tokenizer } = await loadModelAndTokenizer(modelDir, model); + let disposed = false; + const dispose = async () => { + if (disposed) return; + disposed = true; + await session.dispose(); + }; + + try { + const tokenizerState = createTokenizerState(tokenizer, model.maxLength); + + validateSession(session, model); + + const runtime = { + dimensions: model.dimensions, + embedding(text) { + const input = tokenizeToWindow(String(text), tokenizer, tokenizerState); + return processEmbedding(input, session, model); + }, + vectorEmbedding(text, textType) { + // Treat empty and whitespace-only input as "no value": embedding blank text wastes an + // inference and yields a semantically meaningless vector, so return the zero vector. + if (!text || !String(text).trim()) + return JSON.stringify(new Array(model.dimensions).fill(0)); + const prefix = promptFor(model.prompts, textType); + return JSON.stringify(Array.from(this.embedding(prefix ? `${prefix}${text}` : text))); + }, + dispose + }; + + const probe = runtime.embedding('embedding model startup probe'); + if (probe.length !== model.dimensions) { + throw new Error( + `Embedding model produced ${probe.length} dimensions; configured ${model.dimensions}` + ); + } + return runtime; + } catch (error) { + await dispose().catch(() => {}); + throw error; + } +} + +async function validateEmbeddingModel(modelDir, model) { + const runtime = await createEmbeddingRuntimeFromModel(modelDir, model); + await runtime.dispose(); +} + +async function resolveEmbeddingModel(configuration, options = {}) { + const { + root = process.cwd(), + warn = (message) => console.warn(message), + fetchImpl, + discover, + validate + } = options; + const { model: modelName, directory, prompts } = normalizeEmbeddingConfiguration(configuration); + const modelRoot = getModelRoot(directory, root, options.home); + const modelDir = getModelDirectory(modelRoot, modelName); + const installOptions = { + root, + directory: modelRoot, + home: options.home, + fetchImpl, + hubUrl: options.hubUrl, + discover, + validate: validate ?? validateEmbeddingModel, + timeoutMs: options.provisionTimeoutMs, + retryMs: options.provisionRetryMs + }; + + let model; + try { + model = await readModelLock(modelDir); + } catch (error) { + if (directory !== undefined || !/Embedding model lock not found/.test(error.message)) { + const recovery = /Embedding model lock not found/.test(error.message) + ? modelInstallHint(modelName, directory) + : `Remove or replace the invalid lock explicitly, then ${lowercaseFirst( + modelInstallHint(modelName, directory) + )}`; + throw new Error(`${error.message}. ${recovery}`, { cause: error }); + } + const installed = await installModelOnDemand(modelName, modelDir, installOptions, warn); + return { ...installed, model: applyPromptConfiguration(installed.model, prompts) }; + } + + if (model.repository !== modelName) { + throw new Error( + `Embedding model directory ${modelDir} contains ${model.repository}, not ${modelName}. Choose another directory or provision the configured model there.` + ); + } + try { + await verifyModelDirectory(modelDir, model); + } catch (error) { + if (directory !== undefined) { + throw new Error(`${error.message}. ${modelInstallHint(modelName, directory)}`, { + cause: error + }); + } + const installed = await installModelOnDemand(modelName, modelDir, installOptions, warn); + return { ...installed, model: applyPromptConfiguration(installed.model, prompts) }; + } + + return { model: applyPromptConfiguration(model, prompts), modelDir }; +} + +function applyPromptConfiguration(model, configuredPrompts) { + if (!configuredPrompts) return model; + if (!model.output.includePrompt) { + throw new Error( + 'embedding.prompts cannot be used because the model excludes prompt tokens from pooling' + ); + } + return { ...model, prompts: { ...model.prompts, ...configuredPrompts } }; +} + +async function installModelOnDemand(modelName, modelDir, options, warn) { + warn( + `Embedding model '${modelName}' is not available in '${modelDir}'. Downloading it now; application startup may be delayed. Only use models from repositories you trust. ${modelInstallHint(modelName)}` + ); + try { + return await installModel(modelName, options); + } catch (error) { + throw new Error( + `Failed to install embedding model '${modelName}': ${error.message}. ${modelInstallHint(modelName)}`, + { cause: error } + ); + } +} + +function normalizeEmbeddingConfiguration(configuration) { + if (configuration == null) configuration = {}; + if (typeof configuration !== 'object' || Array.isArray(configuration)) { + throw new TypeError('embedding must be an object'); + } + const model = configuration.model === undefined ? DEFAULT_EMBEDDING_MODEL : configuration.model; + if (typeof model !== 'string' || !model.trim()) { + throw new Error('cds.env.requires.db.embedding.model must be a non-empty string'); + } + if ( + configuration.directory !== undefined && + (typeof configuration.directory !== 'string' || !configuration.directory.trim()) + ) { + throw new Error('embedding.directory must be a non-empty string'); + } + + return { + model, + directory: configuration.directory, + prompts: promptsFromConfig(configuration.prompts) + }; +} + +// Turn the user-facing `embedding.prompts.{query,document}` config into a runtime prompts +// object. When set, it takes precedence over the discovered prompts: the returned prompts +// override whatever the lock carries, letting users supply prefixes for models whose prompts +// are not discoverable (e.g. nomic's README-only `search_query: ` / `search_document: `). +// Returns undefined when no prompts are configured, leaving discovered prompts in place. +function promptsFromConfig(configured) { + if (configured === undefined) return undefined; + if (typeof configured !== 'object' || configured === null || Array.isArray(configured)) { + throw new TypeError('embedding.prompts must be an object with query and/or document strings'); + } + + const prompts = {}; + for (const key of ['query', 'document']) { + if (configured[key] === undefined) continue; + if (typeof configured[key] !== 'string' || !configured[key]) { + throw new TypeError(`embedding.prompts.${key} must be a non-empty string`); + } + prompts[key] = configured[key]; + } + + return Object.keys(prompts).length > 0 ? prompts : undefined; +} + +function modelInstallHint(repository, directory) { + const directoryArgument = directory === undefined ? '' : ` --directory ${directory}`; + return `Run 'npx @cap-js/ai install-model ${repository}${directoryArgument}'.`; +} + +function lowercaseFirst(value) { + return `${value[0].toLowerCase()}${value.slice(1)}`; +} + +function createTokenizerState(tokenizer, maxLength) { + const probeText = 'embedding tokenizer boundary probe'; + const content = normalizeEncoding( + tokenizer.encode(probeText, { + add_special_tokens: false, + return_token_type_ids: true + }) + ); + const wrapped = normalizeEncoding( + tokenizer.encode(probeText, { + add_special_tokens: true, + return_token_type_ids: true + }) + ); + + const contentOffset = findSubarray(wrapped.ids, content.ids); + if (content.ids.length === 0 || contentOffset < 0) { + throw new Error('Tokenizer special-token layout is incompatible with windowed encoding'); + } + + const prefix = sliceEncoding(wrapped, 0, contentOffset); + const suffix = sliceEncoding(wrapped, contentOffset + content.ids.length); + if (prefix.ids.length + suffix.ids.length >= maxLength) { + throw new Error('embedding.maxLength leaves no room for tokenizer content'); + } + return { maxLength, prefix, suffix }; +} + +function tokenizeToWindow(text, tokenizer, { maxLength, prefix, suffix }) { + // tokenizers.js intentionally ignores tokenizer.json truncation. Encode the complete + // content without special tokens, truncate it, then add the tokenizer-derived boundaries. + const encoded = normalizeEncoding( + tokenizer.encode(text, { + add_special_tokens: false, + return_token_type_ids: true + }) + ); + const maxContentLength = maxLength - prefix.ids.length - suffix.ids.length; + return concatenateEncodings(prefix, sliceEncoding(encoded, 0, maxContentLength), suffix); +} + +function normalizeEncoding(encoding) { + if (!encoding || typeof encoding !== 'object') { + throw new Error('Tokenizer did not return an encoding'); + } + validateTokenIds(encoding.ids); + const attentionMask = encoding.attention_mask ?? new Array(encoding.ids.length).fill(1); + const tokenTypeIds = encoding.token_type_ids ?? new Array(encoding.ids.length).fill(0); + validateAttentionMask(attentionMask); + validateTokenIds(tokenTypeIds); + if (attentionMask.length !== encoding.ids.length || tokenTypeIds.length !== encoding.ids.length) { + throw new Error('Tokenizer metadata length does not match its token IDs'); + } + return { ids: encoding.ids, attention_mask: attentionMask, token_type_ids: tokenTypeIds }; +} + +function sliceEncoding(encoding, start, end) { + return Object.fromEntries( + Object.entries(encoding).map(([name, values]) => [name, values.slice(start, end)]) + ); +} + +function concatenateEncodings(...encodings) { + return { + ids: encodings.flatMap(({ ids }) => ids), + attention_mask: encodings.flatMap(({ attention_mask: attentionMask }) => attentionMask), + token_type_ids: encodings.flatMap(({ token_type_ids: tokenTypeIds }) => tokenTypeIds) + }; +} + +function findSubarray(values, expected) { + outer: for (let offset = 0; offset <= values.length - expected.length; offset++) { + for (let index = 0; index < expected.length; index++) { + if (values[offset + index] !== expected[index]) continue outer; + } + return offset; + } + return -1; +} + +function validateSession(session, model) { + const inputNames = session.inputNames; + const outputNames = session.outputNames; + if (!Array.isArray(inputNames) || !inputNames.includes('input_ids')) { + throw new Error("Embedding model must expose the standard int64 input 'input_ids'"); + } + const unsupportedInputs = inputNames.filter((name) => !STANDARD_INPUT_NAMES.has(name)); + if (unsupportedInputs.length > 0) { + throw new Error(`Embedding model has unsupported inputs: ${unsupportedInputs.join(', ')}`); + } + if (!Array.isArray(outputNames) || !outputNames.includes(model.output.name)) { + throw new Error( + `Embedding model output '${model.output.name}' not found. Available outputs: ${outputNames?.join(', ') || 'none'}` + ); + } +} + +function processEmbedding(input, session, model) { + const results = session.run(createFeeds(input, session.inputNames)); + const output = results[model.output.name]; + if (!output) { + throw new Error( + `Embedding model output '${model.output.name}' not found. Available outputs: ${Object.keys(results).join(', ')}` + ); + } + const embedding = poolOutput(output, model.output.pooling); + if (embedding.length !== model.dimensions) { + throw new Error( + `Embedding model produced ${embedding.length} dimensions; configured ${model.dimensions}` + ); + } + return model.output.normalize ? normalizeEmbedding(embedding) : embedding; +} + +function createFeeds(encoding, inputNames = STANDARD_INPUT_NAMES) { + const { + ids, + attention_mask: attentionMask, + token_type_ids: tokenTypeIds + } = normalizeEncoding(encoding); + const dimensions = [1, ids.length]; + const values = { + input_ids: new BigInt64Array(ids.map((id) => BigInt(id))), + attention_mask: new BigInt64Array(attentionMask.map((value) => BigInt(value))), + token_type_ids: new BigInt64Array(tokenTypeIds.map((value) => BigInt(value))) + }; + const supportedInputs = new Set(inputNames); + return Object.fromEntries( + Object.entries(values) + .filter(([name]) => supportedInputs.has(name)) + .map(([name, data]) => [name, { type: 'int64', data, dims: dimensions }]) + ); +} + +function poolOutput(output, pooling) { + const { data, dims, type } = output; + if (!data || !Array.isArray(dims)) throw new Error('Embedding model returned an invalid tensor'); + if (type !== 'float32' && type !== 'float64') { + throw new Error(`Embedding model output must be float32 or float64, received '${type}'`); + } + + if (pooling === 'none') { + if (dims.length === 1) return Float32Array.from(data); + if (dims.length === 2 && dims[0] === 1) return Float32Array.from(data); + throw new Error("Pooling 'none' requires an output shaped [dimensions] or [1, dimensions]"); + } + + if (dims.length !== 3 || dims[0] !== 1 || data.length !== dims[1] * dims[2]) { + throw new Error(`Pooling '${pooling}' requires an output shaped [1, sequence, dimensions]`); + } + const [, sequenceLength, dimensions] = dims; + if (sequenceLength < 1) throw new Error('Embedding model returned an empty sequence'); + if (pooling === 'cls') return Float32Array.from(data.slice(0, dimensions)); + + const embedding = new Float32Array(dimensions); + for (let token = 0; token < sequenceLength; token++) { + for (let dimension = 0; dimension < dimensions; dimension++) { + embedding[dimension] += data[token * dimensions + dimension]; + } + } + for (let dimension = 0; dimension < dimensions; dimension++) { + embedding[dimension] /= sequenceLength; + } + return embedding; +} + +function normalizeEmbedding(embedding) { + let squaredNorm = 0; + for (const value of embedding) squaredNorm += value * value; + const norm = Math.sqrt(squaredNorm); + if (norm === 0) return embedding; + for (let index = 0; index < embedding.length; index++) embedding[index] /= norm; + return embedding; +} + +function validateTokenIds(ids) { + if (!Array.isArray(ids)) throw new Error('Tokenizer did not return an ID array'); + for (const id of ids) { + if (!Number.isSafeInteger(id) || id < 0) { + throw new Error(`Invalid token ID detected: ${id} (type: ${typeof id})`); + } + } + return ids; +} + +function validateAttentionMask(mask) { + if (!Array.isArray(mask) || mask.some((value) => value !== 0 && value !== 1)) { + throw new Error('Tokenizer attention mask must contain only zeros and ones'); + } + return mask; +} + +export { + DEFAULT_EMBEDDING_MODEL, + createEmbeddingRuntime, + createEmbeddingRuntimeFromModel, + createFeeds, + createTokenizerState, + poolOutput, + processEmbedding, + resolveEmbeddingModel, + tokenizeToWindow, + validateSession, + validateEmbeddingModel +}; diff --git a/lib/vector_embedding/huggingface-hub.js b/lib/vector_embedding/huggingface-hub.js new file mode 100644 index 0000000..2759f19 --- /dev/null +++ b/lib/vector_embedding/huggingface-hub.js @@ -0,0 +1,294 @@ +import { setTimeout as delay } from 'node:timers/promises'; +import { TransformStream } from 'node:stream/web'; + +import { normalizeHubUrl } from './model-utils.js'; + +const MODEL_REPOSITORY = 'model'; +const HUB_REQUEST_TIMEOUT_MS = 30_000; +const HUB_REQUEST_RETRIES = 2; +const HUB_RETRY_DELAY_MS = 250; +const HUB_RESPONSE_MAX_BYTES = 64 * 1024 * 1024; +const MODEL_INFO_FIELDS = [ + 'cardData', + 'config', + 'filePaths', + 'library_name', + 'sha', + 'tags', + 'transformersInfo' +]; + +function createHuggingFaceClient(options = {}) { + const { fetchImpl = globalThis.fetch, hubApi } = options; + const hubUrl = options.hubUrl === undefined ? undefined : normalizeHubUrl(options.hubUrl); + const requestOptions = { + timeoutMs: options.requestTimeoutMs ?? HUB_REQUEST_TIMEOUT_MS, + retries: options.requestRetries ?? HUB_REQUEST_RETRIES, + retryDelayMs: options.requestRetryMs ?? HUB_RETRY_DELAY_MS, + maxResponseBytes: positiveByteLimit( + options.maxResponseBytes ?? HUB_RESPONSE_MAX_BYTES, + 'maxResponseBytes' + ) + }; + const loadedHubApi = resolveHubApi(hubApi); + + return { + async getModelInfo(repository) { + const { modelInfo } = await loadedHubApi; + return runHubOperation( + `reading model metadata for '${repository}'`, + (fetch) => + modelInfo({ + name: repository, + additionalFields: MODEL_INFO_FIELDS, + fetch, + hubUrl + }), + fetchImpl, + requestOptions + ); + }, + + async getFiles(repository, revision) { + const { listFiles } = await loadedHubApi; + return runHubOperation( + `listing files for '${repository}'`, + async (fetch) => { + const files = []; + for await (const file of listFiles({ + repo: { type: MODEL_REPOSITORY, name: repository }, + revision, + recursive: true, + fetch, + hubUrl + })) { + if (file.type === 'file') files.push(file); + } + return files; + }, + fetchImpl, + requestOptions + ); + }, + + async getFile(repository, revision, remotePath) { + const { downloadFile } = await loadedHubApi; + return runHubOperation( + `downloading '${repository}/${remotePath}'`, + async (fetch) => { + const file = await downloadFile({ + repo: { type: MODEL_REPOSITORY, name: repository }, + path: remotePath, + revision, + xet: false, + fetch, + hubUrl + }); + if (!file) { + throw new Error( + `Hugging Face model '${repository}' does not contain '${remotePath}' at ${revision}` + ); + } + return readBlob(file, requestOptions.maxResponseBytes, repository, remotePath); + }, + fetchImpl, + requestOptions + ); + } + }; +} + +async function runHubOperation(description, operation, fetchImpl, options) { + const { timeoutMs, retries, retryDelayMs, maxResponseBytes } = options; + return attemptOperation(0); + + async function attemptOperation(attempt) { + const controller = new AbortController(); + let timeout; + const timedOperation = Promise.race([ + operation(createOperationFetch(fetchImpl, controller.signal, maxResponseBytes)), + new Promise((_, reject) => { + timeout = setTimeout(() => { + reject(new HubTimeoutError(description, timeoutMs)); + controller.abort(); + }, timeoutMs); + }) + ]); + + try { + return await timedOperation; + } catch (error) { + if (!isRetryable(error) || attempt >= retries) throw error; + await delay(retryDelayMs * 2 ** attempt); + return attemptOperation(attempt + 1); + } finally { + clearTimeout(timeout); + } + } +} + +function createOperationFetch(fetchImpl, signal, maxResponseBytes) { + return async (input, init = {}) => { + try { + const response = await fetchImpl(input, { + ...init, + signal: init.signal ? AbortSignal.any([init.signal, signal]) : signal + }); + if (response.status === 408 || response.status === 429 || response.status >= 500) { + await response.body?.cancel().catch(() => {}); + throw new RetryableHubResponseError(response.status, input); + } + return await limitResponseSize(response, maxResponseBytes, input); + } catch (error) { + if (error instanceof RetryableHubResponseError || error instanceof HubResponseTooLargeError) { + throw error; + } + throw new HubTransportError(input, { cause: error }); + } + }; +} + +async function limitResponseSize(response, maxBytes, input) { + const declaredSize = declaredResponseSize(response); + if (declaredSize !== undefined && declaredSize > maxBytes) { + await response.body?.cancel().catch(() => {}); + throw new HubResponseTooLargeError(input, maxBytes); + } + if (!response.body) return response; + + let received = 0; + const body = response.body.pipeThrough( + new TransformStream({ + transform(chunk, controller) { + received += chunk.byteLength; + if (received > maxBytes) { + throw new HubResponseTooLargeError(input, maxBytes); + } + controller.enqueue(chunk); + } + }) + ); + const limited = new Response(body, { + status: response.status, + statusText: response.statusText, + headers: response.headers + }); + Object.defineProperties(limited, { + redirected: { value: response.redirected }, + type: { value: response.type }, + url: { value: response.url } + }); + return limited; +} + +function declaredResponseSize(response) { + const contentLengthHeader = response.headers?.get?.('content-length'); + const contentLength = contentLengthHeader === null ? undefined : Number(contentLengthHeader); + const contentRange = response.headers?.get?.('content-range'); + const match = typeof contentRange === 'string' && /^bytes\s+\d+-\d+\/(\d+)$/iu.exec(contentRange); + const rangeTotal = match ? Number(match[1]) : undefined; + const candidates = [contentLength, rangeTotal].filter( + (value) => Number.isSafeInteger(value) && value >= 0 + ); + return candidates.length > 0 ? Math.max(...candidates) : undefined; +} + +async function readBlob(file, maxBytes, repository, remotePath) { + const description = `${repository}/${remotePath}`; + if (!Number.isSafeInteger(file.size) || file.size < 0 || typeof file.stream !== 'function') { + throw new Error(`Hugging Face returned an invalid file response for ${description}`); + } + if (file.size > maxBytes) { + throw new HubResponseTooLargeError(description, maxBytes); + } + + const chunks = []; + let received = 0; + for await (const value of file.stream()) { + const chunk = Buffer.from(value); + received += chunk.byteLength; + if (received > maxBytes) throw new HubResponseTooLargeError(description, maxBytes); + chunks.push(chunk); + } + return Buffer.concat(chunks, received); +} + +function positiveByteLimit(value, name) { + if (!Number.isSafeInteger(value) || value < 1) { + throw new TypeError(`${name} must be a positive integer`); + } + return value; +} + +function isRetryable(error) { + return ( + error instanceof HubTimeoutError || + error instanceof HubTransportError || + error instanceof RetryableHubResponseError + ); +} + +class HubTimeoutError extends Error { + constructor(description, timeoutMs) { + super(`Timed out after ${timeoutMs} ms while ${description}`); + this.name = 'HubTimeoutError'; + } +} + +class HubTransportError extends Error { + constructor(input, options) { + super(`Hugging Face request failed for ${String(input)}`, options); + this.name = 'HubTransportError'; + } +} + +class RetryableHubResponseError extends Error { + constructor(status, input) { + super(`Hugging Face request failed with status ${status} for ${String(input)}`); + this.name = 'RetryableHubResponseError'; + this.status = status; + } +} + +class HubResponseTooLargeError extends Error { + constructor(input, maxBytes) { + super(`Refusing Hugging Face response for ${String(input)}: exceeds ${maxBytes} bytes`); + this.name = 'HubResponseTooLargeError'; + } +} + +async function resolveHubApi(hubApi) { + if ( + typeof hubApi?.modelInfo === 'function' && + typeof hubApi?.listFiles === 'function' && + typeof hubApi?.downloadFile === 'function' + ) { + return hubApi; + } + return { ...(await loadHuggingFaceHub()), ...hubApi }; +} + +async function loadHuggingFaceHub(importModule = (specifier) => import(specifier)) { + try { + return await importModule('@huggingface/hub'); + } catch (error) { + if ( + error?.code === 'ERR_MODULE_NOT_FOUND' && + /Cannot find package ['"]@huggingface\/hub['"]/.test(error.message) + ) { + throw new Error( + "Automatic Hugging Face model discovery requires @huggingface/hub. Install it with 'npm add -D @huggingface/hub'.", + { cause: error } + ); + } + throw error; + } +} + +export { + HUB_REQUEST_RETRIES, + HUB_REQUEST_TIMEOUT_MS, + HUB_RESPONSE_MAX_BYTES, + createHuggingFaceClient, + loadHuggingFaceHub +}; diff --git a/lib/vector_embedding/load-onnx-runtime.js b/lib/vector_embedding/load-onnx-runtime.js new file mode 100644 index 0000000..b0eddc4 --- /dev/null +++ b/lib/vector_embedding/load-onnx-runtime.js @@ -0,0 +1,29 @@ +const SUPPORTED_ONNX_RUNTIME_VERSION = '1.20.1'; + +function loadOnnxRuntime(requireModule) { + try { + const runtimeVersion = requireModule('onnxruntime-node/package.json').version; + if (runtimeVersion !== SUPPORTED_ONNX_RUNTIME_VERSION) { + throw new Error( + `Unsupported onnxruntime-node version ${runtimeVersion}; @cap-js/ai requires ${SUPPORTED_ONNX_RUNTIME_VERSION} because its synchronous SQLite integration uses the runtime's private native API.` + ); + } + return { + ort: requireModule('onnxruntime-node'), + binding: requireModule('onnxruntime-node/dist/binding.js').binding + }; + } catch (error) { + if ( + (error?.code === 'ERR_MODULE_NOT_FOUND' || error?.code === 'MODULE_NOT_FOUND') && + /['"]onnxruntime-node(?:\/[^'"]*)?['"]/.test(error.message) + ) { + throw new Error( + "Using local SQLite embeddings requires onnxruntime-node@1.20.1. Install it with 'npm add -D onnxruntime-node@1.20.1'.", + { cause: error } + ); + } + throw error; + } +} + +export { SUPPORTED_ONNX_RUNTIME_VERSION, loadOnnxRuntime }; diff --git a/lib/vector_embedding/model-discovery.js b/lib/vector_embedding/model-discovery.js new file mode 100644 index 0000000..c5c71eb --- /dev/null +++ b/lib/vector_embedding/model-discovery.js @@ -0,0 +1,540 @@ +import { createHash } from 'node:crypto'; +import path from 'node:path'; + +import { createHuggingFaceClient } from './huggingface-hub.js'; +import { assertSafeRepository, validateModelDescriptor } from './model-utils.js'; + +const MODULES_FILE = 'modules.json'; +const SENTENCE_CONFIG_FILE = 'sentence_bert_config.json'; +const CONFIG_ST_FILE = 'config_sentence_transformers.json'; +const EMBEDDING_TASKS = new Set(['feature-extraction', 'sentence-similarity']); +const PIPELINE_MODULES = new Set([ + 'sentence_transformers.models.Transformer', + 'sentence_transformers.models.Pooling', + 'sentence_transformers.models.Normalize' +]); + +async function discoverModel(repository, options = {}) { + const { candidate, context, filesByPath, knownFiles } = await discoverModelMetadata( + repository, + options + ); + const descriptor = { ...candidate }; + delete descriptor.task; + const descriptorFiles = await Promise.all( + candidate.files.map(async (entry) => ({ + ...entry, + ...(await discoverFileIntegrity( + context, + repository, + candidate.revision, + filesByPath.get(entry.path), + knownFiles.get(entry.path) + )) + })) + ); + + return validateModelDescriptor({ ...descriptor, files: descriptorFiles }); +} + +async function checkModel(repository, options = {}) { + const { candidate } = await discoverModelMetadata(repository, options); + return candidate; +} + +async function discoverModelMetadata(repository, options) { + assertSafeRepository(repository); + const normalizedOptions = typeof options === 'function' ? { fetchImpl: options } : options; + const hubClient = normalizedOptions.hubClient ?? createHuggingFaceClient(normalizedOptions); + const context = { hubClient, fileLists: new Map(), jsonFiles: new Map() }; + const modelInfo = await hubClient.getModelInfo(repository); + rejectNonEmbeddingTask(modelInfo, repository); + const task = modelTask(modelInfo); + + const revision = immutableRevision(modelInfo, repository); + const filesByPath = await repositoryFiles(context, repository, revision); + const modelPath = selectOnnxModel(filesByPath, repository); + const tokenizerPath = selectCompanionFile(filesByPath, modelPath, 'tokenizer.json', repository); + const tokenizerConfigPath = selectCompanionFile( + filesByPath, + modelPath, + 'tokenizer_config.json', + repository + ); + const configPath = selectCompanionFile(filesByPath, modelPath, 'config.json', repository); + + const [tokenizer, tokenizerConfig, config] = await Promise.all([ + fetchJsonFile(context, repository, revision, tokenizerPath), + fetchJsonFile(context, repository, revision, tokenizerConfigPath), + fetchJsonFile(context, repository, revision, configPath) + ]); + const dimensions = uniquePositiveInteger( + [config.value.hidden_size, config.value.n_embd, config.value.d_model, config.value.dim], + repository, + configPath + ); + if (!dimensions) { + throw new Error( + `Cannot determine embedding dimensions from '${repository}/${configPath}' (expected hidden_size, n_embd, d_model, or dim)` + ); + } + + const semantics = await discoverSentenceTransformerSemantics( + context, + repository, + modelInfo, + new Set() + ); + const maxLength = minimumPositiveInteger([ + tokenizer.value?.truncation?.max_length, + semantics.maxLength, + tokenizerConfig.value.model_max_length, + config.value.max_position_embeddings, + config.value.n_positions, + config.value.n_ctx + ]); + if (!maxLength) { + throw new Error(`Cannot determine the maximum input length for '${repository}'`); + } + + const selected = [ + artifact('model', modelPath, modelPath), + artifact('tokenizer', tokenizerPath, modelPath), + artifact('tokenizerConfig', tokenizerConfigPath, modelPath), + artifact('auxiliary', configPath, modelPath) + ]; + const externalData = [...filesByPath.values()] + .filter((file) => isExternalDataFile(file.path, modelPath)) + .map((file) => artifact('auxiliary', file.path, modelPath)); + if (usesExternalData(config.value, modelPath) && externalData.length === 0) { + throw new Error( + `Hugging Face model '${repository}' declares external ONNX data but no data file exists next to '${modelPath}'` + ); + } + selected.push(...externalData); + + const knownFiles = new Map([ + [tokenizerPath, tokenizer], + [tokenizerConfigPath, tokenizerConfig], + [configPath, config] + ]); + const candidate = validateModelCandidate({ + repository, + revision, + task, + dimensions, + maxLength, + files: selected, + output: { + name: 'last_hidden_state', + pooling: semantics.pooling, + normalize: semantics.normalize, + includePrompt: semantics.includePrompt + }, + ...(semantics.prompts ? { prompts: semantics.prompts } : {}) + }); + return { + context, + filesByPath, + knownFiles, + candidate + }; +} + +function validateModelCandidate(candidate) { + validateModelDescriptor({ + ...candidate, + files: candidate.files.map((file) => ({ + ...file, + size: 1, + sha256: '0'.repeat(64) + })) + }); + return candidate; +} + +function rejectNonEmbeddingTask(modelInfo, repository) { + const task = modelTask(modelInfo); + if (typeof task === 'string' && task && !EMBEDDING_TASKS.has(task)) { + throw new Error( + `Hugging Face model '${repository}' declares task '${task}', not an embedding task` + ); + } +} + +function modelTask(modelInfo) { + return modelInfo?.task; +} + +function immutableRevision(modelInfo, repository) { + if (typeof modelInfo?.sha !== 'string' || !/^[a-fA-F0-9]{40,64}$/.test(modelInfo.sha)) { + throw new Error(`Hugging Face did not return an immutable revision for '${repository}'`); + } + return modelInfo.sha.toLowerCase(); +} + +function fileMap(files, repository) { + if (!Array.isArray(files)) { + throw new Error(`Hugging Face did not return a file list for '${repository}'`); + } + const result = new Map(); + for (const file of files) { + const remotePath = file?.path ?? file?.rfilename; + if (typeof remotePath !== 'string') continue; + if (result.has(remotePath)) { + throw new Error(`Hugging Face returned duplicate file '${remotePath}'`); + } + result.set(remotePath, { ...file, path: remotePath }); + } + return result; +} + +function selectOnnxModel(filesByPath, repository) { + const paths = [...filesByPath.keys()].filter((remotePath) => + remotePath.toLowerCase().endsWith('.onnx') + ); + if (paths.includes('onnx/model.onnx')) return 'onnx/model.onnx'; + if (paths.includes('model.onnx')) return 'model.onnx'; + + const conventional = paths.filter( + (remotePath) => path.posix.basename(remotePath).toLowerCase() === 'model.onnx' + ); + if (conventional.length === 1) return conventional[0]; + if (conventional.length > 1 || paths.length > 1) { + throw new Error( + `Hugging Face model '${repository}' contains ambiguous ONNX exports: ${paths.join(', ')}` + ); + } + if (paths.length === 1) return paths[0]; + throw new Error(`Hugging Face model '${repository}' does not contain an ONNX model`); +} + +function selectCompanionFile(filesByPath, modelPath, filename, repository) { + const modelDirectory = path.posix.dirname(modelPath); + const adjacent = modelDirectory === '.' ? filename : `${modelDirectory}/${filename}`; + if (filesByPath.has(adjacent)) return adjacent; + if (filesByPath.has(filename)) return filename; + throw new Error( + `Hugging Face model '${repository}' must contain '${filename}' at the repository root or next to '${modelPath}'` + ); +} + +function artifact(role, remotePath, modelPath) { + const modelDirectory = path.posix.dirname(modelPath); + const name = + role === 'auxiliary' && isExternalDataFile(remotePath, modelPath) + ? path.posix.relative(modelDirectory, remotePath) + : path.posix.basename(remotePath); + return { role, name, path: remotePath }; +} + +function isExternalDataFile(remotePath, modelPath) { + if (path.posix.dirname(remotePath) !== path.posix.dirname(modelPath)) return false; + const name = path.posix.basename(remotePath); + const modelName = path.posix.basename(modelPath); + return ( + name.startsWith(`${modelName}_data`) || + name === `${modelName}.data` || + name.startsWith(`${modelName}.data.`) + ); +} + +function usesExternalData(config, modelPath) { + const value = config?.['transformers.js_config']?.use_external_data_format; + if (value === true) return true; + if (!value || typeof value !== 'object' || Array.isArray(value)) return false; + const names = new Set([modelPath, path.posix.basename(modelPath)]); + return Object.entries(value).some( + ([name, enabled]) => names.has(name) && (enabled === true || enabled === 1) + ); +} + +async function discoverSentenceTransformerSemantics(context, repository, modelInfo, visited) { + if (visited.has(repository)) { + throw new Error(`Circular Hugging Face base_model chain involving '${repository}'`); + } + visited.add(repository); + + const revision = immutableRevision(modelInfo, repository); + const filesByPath = await repositoryFiles(context, repository, revision); + if (filesByPath.has(MODULES_FILE)) { + return readSentenceTransformerSemantics(context, repository, revision, filesByPath); + } + + const baseModel = baseModelRepository(modelInfo.cardData?.base_model); + if (!baseModel) { + throw new Error( + `Cannot determine pooling and normalization for '${repository}': no Sentence Transformers modules or unambiguous base_model metadata` + ); + } + assertSafeRepository(baseModel); + const baseInfo = await context.hubClient.getModelInfo(baseModel); + return discoverSentenceTransformerSemantics(context, baseModel, baseInfo, visited); +} + +async function readSentenceTransformerSemantics(context, repository, revision, filesByPath) { + const modules = (await fetchJsonFile(context, repository, revision, MODULES_FILE)).value; + if (!Array.isArray(modules)) { + throw new Error(`Invalid Sentence Transformers modules in '${repository}/${MODULES_FILE}'`); + } + + // modules.json defines the ordered execution pipeline. Accept only stages that this runtime + // actually executes; skipping a custom or future module would silently change the embeddings. + for (const module of modules) { + if (!module || typeof module.type !== 'string') + throw new Error(`Invalid Sentence Transformers module in '${repository}/${MODULES_FILE}'`); + if (!PIPELINE_MODULES.has(module.type)) { + throw new Error( + `Unsupported Sentence Transformers module '${module.type}' in '${repository}'` + ); + } + } + + const expectedTypes = [ + 'sentence_transformers.models.Transformer', + 'sentence_transformers.models.Pooling' + ]; + if (modules.length === 3) expectedTypes.push('sentence_transformers.models.Normalize'); + if ( + modules.length < 2 || + modules.length > 3 || + modules.some((module, index) => module.type !== expectedTypes[index]) + ) { + throw new Error(`Cannot determine an unambiguous pooling pipeline for '${repository}'`); + } + + const poolingPath = moduleConfigPath(modules[1], repository); + if (!filesByPath.has(poolingPath)) { + throw new Error(`Sentence Transformers pooling configuration '${poolingPath}' is missing`); + } + const poolingConfig = (await fetchJsonFile(context, repository, revision, poolingPath)).value; + const pooling = determinePooling(poolingConfig, repository); + if ( + poolingConfig.include_prompt !== undefined && + typeof poolingConfig.include_prompt !== 'boolean' + ) { + throw new Error( + `Invalid include_prompt in Sentence Transformers pooling configuration for '${repository}'` + ); + } + const includePrompt = poolingConfig.include_prompt !== false; + + // Check if the sentence-transformer was trained with registered prompts + // See: https://sbert.net/examples/sentence_transformer/training/prompts/README.html + // A model trained with prompts is likely to be asymmetric: + // > It would break our assumption that query and document can be embedded equally. + let prompts; + if (filesByPath.has(CONFIG_ST_FILE)) { + const stConfig = (await fetchJsonFile(context, repository, revision, CONFIG_ST_FILE)).value; + prompts = interpretPrompts(stConfig, repository); + // Intentionally checked post-normalization: empty-string prompts produce no prefix tokens, + // so include_prompt=false is a no-op for those and should not cause rejection. + if (prompts && !includePrompt) { + throw new Error( + `Sentence Transformers model '${repository}' excludes prompt tokens from pooling (include_prompt=false); prefixing cannot be emulated` + ); + } + } + + let maxLength; + const sentenceConfigPaths = [SENTENCE_CONFIG_FILE]; + if (modules[0]?.path) { + sentenceConfigPaths.unshift(`${normalizedModulePath(modules[0].path)}/${SENTENCE_CONFIG_FILE}`); + } + const sentenceConfigPath = sentenceConfigPaths.find((candidate) => filesByPath.has(candidate)); + if (sentenceConfigPath) { + const sentenceConfig = await fetchJsonFile(context, repository, revision, sentenceConfigPath); + maxLength = positiveInteger(sentenceConfig.value.max_seq_length); + } + + return { + pooling, + normalize: modules.length === 3, + includePrompt, + maxLength, + ...(prompts ? { prompts } : {}) + }; +} + +// Map the conventional retrieval prompts onto HANA text types. Sentence Transformers permits +// arbitrary additional prompt names, which do not affect QUERY/DOCUMENT and are ignored here. +function interpretPrompts(config, repository) { + if (config == null) return undefined; + if (typeof config !== 'object' || Array.isArray(config)) { + throw new Error( + `Invalid Sentence Transformers configuration in '${repository}/${CONFIG_ST_FILE}'` + ); + } + + const { prompts } = config; + + if (prompts === undefined || prompts === null) return undefined; + if (typeof prompts !== 'object' || Array.isArray(prompts)) { + throw new Error( + `Unsupported Sentence Transformers prompts in '${repository}': expected an object` + ); + } + + for (const [name, value] of Object.entries(prompts)) { + if (typeof value !== 'string') { + throw new Error(`Sentence Transformers prompt '${name}' in '${repository}' must be a string`); + } + } + + const normalized = {}; + if (prompts.query) normalized.query = prompts.query; + if (prompts.document) normalized.document = prompts.document; + + return Object.keys(normalized).length > 0 ? normalized : undefined; +} + +function moduleConfigPath(module, repository) { + const modulePath = normalizedModulePath(module.path); + if (!modulePath) { + throw new Error(`Sentence Transformers pooling module in '${repository}' has no path`); + } + return `${modulePath}/config.json`; +} + +function normalizedModulePath(value) { + if ( + typeof value !== 'string' || + !value || + value.includes('\\') || + value.startsWith('/') || + value.split('/').some((part) => !part || part === '.' || part === '..') + ) { + return undefined; + } + return value; +} + +function determinePooling(config, repository) { + if (!config || typeof config !== 'object' || Array.isArray(config)) { + throw new Error(`Invalid Sentence Transformers pooling configuration for '${repository}'`); + } + // We only implement 'mean' and 'cls' pooling, so those are the only values this returns. The + // unsupported modes are still listed here on purpose: they let us detect a config that enables + // an unsupported mode (or several modes at once) and fail closed, rather than silently pooling + // with 'mean'/'cls' while ignoring a conflicting flag. + const enabled = [ + ['cls', config.pooling_mode_cls_token], + ['mean', config.pooling_mode_mean_tokens], + ['max', config.pooling_mode_max_tokens], + ['mean_sqrt_len', config.pooling_mode_mean_sqrt_len_tokens], + ['weightedmean', config.pooling_mode_weightedmean_tokens], + ['lasttoken', config.pooling_mode_lasttoken] + ].filter(([, value]) => value === true); + + if (config.pooling_mode !== undefined) { + if (!['mean', 'cls'].includes(config.pooling_mode)) { + throw new Error(`Unsupported or ambiguous Sentence Transformers pooling for '${repository}'`); + } + if (enabled.length > 0 && (enabled.length !== 1 || enabled[0][0] !== config.pooling_mode)) { + throw new Error(`Unsupported or ambiguous Sentence Transformers pooling for '${repository}'`); + } + return config.pooling_mode; + } + if (enabled.length !== 1 || !['mean', 'cls'].includes(enabled[0][0])) { + throw new Error(`Unsupported or ambiguous Sentence Transformers pooling for '${repository}'`); + } + return enabled[0][0]; +} + +function baseModelRepository(value) { + if (typeof value === 'string') return value; + if (Array.isArray(value) && value.length === 1 && typeof value[0] === 'string') return value[0]; + if (value && typeof value === 'object' && !Array.isArray(value) && typeof value.id === 'string') { + return value.id; + } + return undefined; +} + +async function discoverFileIntegrity(context, repository, revision, file, knownFile) { + const metadataChecksum = fileChecksum(file); + const metadataSize = positiveInteger(file?.size) ?? positiveInteger(file?.lfs?.size); + if (metadataChecksum && metadataSize) { + return { size: metadataSize, sha256: metadataChecksum }; + } + + const downloaded = knownFile ?? (await fetchFile(context, repository, revision, file.path)); + return { + size: downloaded.bytes.byteLength, + sha256: createHash('sha256').update(downloaded.bytes).digest('hex') + }; +} + +function fileChecksum(file) { + const candidate = file?.lfs?.sha256 ?? file?.lfs?.oid; + if (typeof candidate !== 'string') return undefined; + const checksum = candidate.replace(/^sha256:/, '').toLowerCase(); + return /^[a-f0-9]{64}$/.test(checksum) ? checksum : undefined; +} + +async function repositoryFiles(context, repository, revision) { + const key = `${repository}@${revision}`; + let files = context.fileLists.get(key); + if (!files) { + files = context.hubClient + .getFiles(repository, revision) + .then((value) => fileMap(value, repository)); + context.fileLists.set(key, files); + } + return files; +} + +async function fetchJsonFile(context, repository, revision, remotePath) { + const key = `${repository}@${revision}/${remotePath}`; + let file = context.jsonFiles.get(key); + if (!file) { + file = fetchFile(context, repository, revision, remotePath).then(({ bytes }) => { + try { + return { bytes, value: JSON.parse(bytes.toString('utf8')) }; + } catch (error) { + throw new Error( + `Invalid JSON returned for '${repository}/${remotePath}' at ${revision}: ${error.message}`, + { cause: error } + ); + } + }); + context.jsonFiles.set(key, file); + } + return file; +} + +async function fetchFile(context, repository, revision, remotePath) { + try { + const bytes = await context.hubClient.getFile(repository, revision, remotePath); + return { bytes: Buffer.from(bytes) }; + } catch (error) { + throw new Error( + `Cannot fetch Hugging Face file '${repository}/${remotePath}' at ${revision}: ${error.message}`, + { cause: error } + ); + } +} + +function positiveInteger(value) { + return Number.isSafeInteger(value) && value > 0 ? value : undefined; +} + +function uniquePositiveInteger(values, repository, configPath) { + const candidates = [ + ...new Set(values.map(positiveInteger).filter((value) => value !== undefined)) + ]; + if (candidates.length > 1) { + throw new Error( + `Conflicting embedding dimensions in '${repository}/${configPath}': ${candidates.join(', ')}` + ); + } + return candidates[0]; +} + +function minimumPositiveInteger(values) { + const candidates = values.map(positiveInteger).filter((value) => value !== undefined); + return candidates.length > 0 ? Math.min(...candidates) : undefined; +} + +const discoverModelDescriptor = discoverModel; + +export { checkModel, discoverModel, discoverModelDescriptor }; diff --git a/lib/vector_embedding/model-install.js b/lib/vector_embedding/model-install.js new file mode 100644 index 0000000..288cc31 --- /dev/null +++ b/lib/vector_embedding/model-install.js @@ -0,0 +1,69 @@ +import { setTimeout as delay } from 'node:timers/promises'; +import { discoverModel } from './model-discovery.js'; +import { + MODEL_PROVISIONING_IN_PROGRESS, + assertSafeRepository, + getModelDirectory, + getModelRoot, + provisionModel, + readModelLock +} from './model-utils.js'; + +const MODEL_PROVISION_TIMEOUT_MS = 15 * 60 * 1000; +const MODEL_PROVISION_RETRY_MS = 250; + +async function installModel(repository, options = {}) { + assertSafeRepository(repository); + const modelRoot = getModelRoot(options.directory, options.root, options.home); + const modelDir = getModelDirectory(modelRoot, repository); + const discover = options.discover ?? discoverModel; + + let model; + try { + model = await readModelLock(modelDir); + assertRepository(model, repository, modelDir); + } catch (error) { + if (!/Embedding model lock not found/.test(error.message)) throw error; + model = await discover(repository, { + fetchImpl: options.fetchImpl, + hubUrl: options.hubUrl + }); + assertRepository(model, repository, modelDir); + } + + const deadline = Date.now() + (options.timeoutMs ?? MODEL_PROVISION_TIMEOUT_MS); + const retryMs = options.retryMs ?? MODEL_PROVISION_RETRY_MS; + while (true) { + try { + // eslint-disable-next-line no-await-in-loop + await provisionModel(model, { + directory: modelDir, + fetchImpl: options.fetchImpl, + hubUrl: options.hubUrl, + validate: options.validate + }); + + return { model, modelDir, modelRoot }; + } catch (error) { + if (error.code !== MODEL_PROVISIONING_IN_PROGRESS) throw error; + const remaining = deadline - Date.now(); + if (remaining <= 0) { + throw new Error(`Timed out waiting for embedding model provisioning in ${modelDir}`, { + cause: error + }); + } + // eslint-disable-next-line no-await-in-loop + await delay(Math.min(retryMs, remaining)); + } + } +} + +function assertRepository(model, repository, modelDir) { + if (model.repository !== repository) { + throw new Error( + `Embedding model directory ${modelDir} contains ${model.repository}, not ${repository}. Choose another directory or remove it explicitly before installing the configured model.` + ); + } +} + +export { installModel }; diff --git a/lib/vector_embedding/model-utils.js b/lib/vector_embedding/model-utils.js new file mode 100644 index 0000000..4653a61 --- /dev/null +++ b/lib/vector_embedding/model-utils.js @@ -0,0 +1,831 @@ +import { createHash, randomUUID } from 'crypto'; +import { createReadStream } from 'fs'; +import fs from 'fs/promises'; +import os from 'os'; +import path from 'path'; + +const DOWNLOAD_TIMEOUT_MS = 5 * 60 * 1000; +const MODEL_LOCK_FILE = 'embedding.lock.json'; +const MODEL_INSTALL_LOCK_FILE = '.embedding.install.lock'; +const MODEL_LOCK_VERSION = 2; +const MODEL_PROVISIONING_IN_PROGRESS = 'ERR_EMBEDDING_MODEL_PROVISIONING_IN_PROGRESS'; +const INSTALL_LOCK_STALE_MS = 30 * 60 * 1000; +const PROVISIONED_DIRECTORY_MODE = 0o755; +const PROVISIONED_FILE_MODE = 0o644; +const REQUIRED_FILE_ROLES = ['model', 'tokenizer', 'tokenizerConfig']; +const ALLOWED_FILE_ROLES = new Set([...REQUIRED_FILE_ROLES, 'auxiliary']); +const RESERVED_ARTIFACT_PATHS = [MODEL_LOCK_FILE, MODEL_INSTALL_LOCK_FILE]; + +function validateModelDescriptor(model) { + if (!model || typeof model !== 'object' || Array.isArray(model)) { + throw new TypeError('The embedding model descriptor must be an object'); + } + + assertSafeRepository(model.repository); + if (typeof model.revision !== 'string' || !/^[a-fA-F0-9]{40,64}$/.test(model.revision)) { + throw new Error('embedding.revision must be an immutable 40-64 character commit hash'); + } + if (!Number.isSafeInteger(model.dimensions) || model.dimensions < 1) { + throw new Error('embedding.dimensions must be a positive integer'); + } + if (!Number.isSafeInteger(model.maxLength) || model.maxLength < 1) { + throw new Error('embedding.maxLength must be a positive integer'); + } + if (!Array.isArray(model.files) || model.files.length < REQUIRED_FILE_ROLES.length) { + throw new Error('embedding.files must include model, tokenizer, and tokenizerConfig files'); + } + + const names = new Set(); + const requiredRoles = new Set(); + for (const file of model.files) { + if (!file || typeof file !== 'object' || Array.isArray(file)) { + throw new TypeError('Each embedding file descriptor must be an object'); + } + if (!ALLOWED_FILE_ROLES.has(file.role)) { + throw new Error(`Unsupported embedding file role '${file.role}'`); + } + if (file.role !== 'auxiliary' && requiredRoles.has(file.role)) { + throw new Error(`Duplicate embedding file role '${file.role}'`); + } + requiredRoles.add(file.role); + assertSafeRelativePath(file.name, `embedding.files[${file.role}].name`); + assertSafeRelativePath(file.path, `embedding.files[${file.role}].path`); + if (names.has(file.name)) throw new Error(`Duplicate embedding file name '${file.name}'`); + for (const existingName of names) assertNoPathCollision(file.name, existingName); + for (const reservedPath of RESERVED_ARTIFACT_PATHS) { + assertNoPathCollision(file.name, reservedPath, 'provisioning metadata'); + } + names.add(file.name); + if (!Number.isSafeInteger(file.size) || file.size < 1) { + throw new Error(`Invalid size for embedding file '${file.name}'`); + } + if (typeof file.sha256 !== 'string' || !/^[a-f0-9]{64}$/.test(file.sha256)) { + throw new Error(`Invalid SHA-256 for embedding file '${file.name}'`); + } + } + for (const role of REQUIRED_FILE_ROLES) { + if (!requiredRoles.has(role)) throw new Error(`Missing embedding file role '${role}'`); + } + + const output = model.output; + if (!output || typeof output !== 'object' || Array.isArray(output)) { + throw new Error('embedding.output must describe the model output'); + } + if (typeof output.name !== 'string' || !/^[A-Za-z_][A-Za-z0-9_.-]*$/.test(output.name)) { + throw new Error('embedding.output.name must be a valid ONNX output name'); + } + if (!['mean', 'cls', 'none'].includes(output.pooling)) { + throw new Error("embedding.output.pooling must be 'mean', 'cls', or 'none'"); + } + if (typeof output.normalize !== 'boolean') { + throw new Error('embedding.output.normalize must be a boolean'); + } + if (typeof output.includePrompt !== 'boolean') { + throw new Error('embedding.output.includePrompt must be a boolean'); + } + + if (model.prompts !== undefined) validatePrompts(model.prompts); + if (model.prompts && !output.includePrompt) { + throw new Error('Embedding prompts require embedding.output.includePrompt to be true'); + } + + return model; +} + +function validatePrompts(prompts) { + if (!prompts || typeof prompts !== 'object' || Array.isArray(prompts)) { + throw new Error('embedding.prompts must be an object with query and/or document string values'); + } + const keys = Object.keys(prompts); + if (keys.length === 0) { + throw new Error('embedding.prompts must define at least one of query or document'); + } + for (const key of keys) { + if (key !== 'query' && key !== 'document') { + throw new Error(`Unsupported embedding.prompts key '${key}'`); + } + if (typeof prompts[key] !== 'string' || !prompts[key]) { + throw new Error(`embedding.prompts.${key} must be a non-empty string`); + } + } +} + +function assertNoPathCollision(left, right, description = 'another embedding file') { + const normalizedLeft = left.toLowerCase(); + const normalizedRight = right.toLowerCase(); + if ( + normalizedLeft === normalizedRight || + normalizedLeft.startsWith(`${normalizedRight}/`) || + normalizedRight.startsWith(`${normalizedLeft}/`) + ) { + throw new Error(`Embedding file '${left}' conflicts with ${description} '${right}'`); + } +} + +function assertSafeRepository(repository) { + if ( + typeof repository !== 'string' || + !/^[A-Za-z0-9][A-Za-z0-9._-]*(\/[A-Za-z0-9][A-Za-z0-9._-]*)?$/.test(repository) || + repository.split('/').some((part) => part === '.' || part === '..') + ) { + throw new Error('embedding.repository must be a safe Hugging Face repository ID'); + } +} + +function getModelRoot(directory, root = process.cwd(), home = os.homedir()) { + if (directory === undefined) return path.join(root, '.cds', 'models'); + if (directory === '~') return home; + if (/^~[\\/]/.test(directory)) return path.join(home, directory.slice(2)); + return path.resolve(root, directory); +} + +function getModelDirectory(root, repository) { + assertSafeRepository(repository); + return path.join(root, ...repository.split('/')); +} + +function assertSafeRelativePath(value, field) { + const parts = typeof value === 'string' ? value.split('/') : []; + if ( + typeof value !== 'string' || + value.length === 0 || + value.includes('\\') || + path.posix.isAbsolute(value) || + path.posix.normalize(value) !== value || + parts.some((part) => part === '' || part === '.' || part === '..') || + parts.some((part) => part.endsWith('.') || isWindowsDeviceName(part)) || + !/^[A-Za-z0-9._/-]+$/.test(value) + ) { + throw new Error(`${field} must be a safe relative path`); + } +} + +function isWindowsDeviceName(value) { + const basename = value.split('.')[0].toUpperCase(); + return /^(CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9])$/.test(basename); +} + +function fileForRole(model, role) { + return model.files.find((file) => file.role === role); +} + +function modelDescriptorDigest(model) { + validateModelDescriptor(model); + const files = model.files + .map(({ role, name, path: remotePath, size, sha256: checksum }) => ({ + role, + name, + path: remotePath, + size, + sha256: checksum + })) + .sort((left, right) => JSON.stringify(left).localeCompare(JSON.stringify(right))); + const canonical = JSON.stringify({ + repository: model.repository, + revision: model.revision.toLowerCase(), + dimensions: model.dimensions, + maxLength: model.maxLength, + files, + output: { + name: model.output.name, + pooling: model.output.pooling, + normalize: model.output.normalize, + includePrompt: model.output.includePrompt + }, + ...(model.prompts + ? { + prompts: { + ...(model.prompts.query !== undefined ? { query: model.prompts.query } : {}), + ...(model.prompts.document !== undefined ? { document: model.prompts.document } : {}) + } + } + : {}) + }); + return createHash('sha256').update(canonical).digest('hex'); +} + +async function sha256(filePath) { + const hash = createHash('sha256'); + for await (const chunk of createReadStream(filePath)) hash.update(chunk); + return hash.digest('hex'); +} + +async function isValidFile(filePath, file) { + try { + const stat = await fs.lstat(filePath); + return stat.isFile() && stat.size === file.size && (await sha256(filePath)) === file.sha256; + } catch (error) { + if (error.code === 'ENOENT') return false; + throw error; + } +} + +async function verifyModelDirectory(modelDir, model) { + validateModelDescriptor(model); + await assertModelDirectory(modelDir); + const validity = await Promise.all( + model.files.map(async (file) => { + await assertNoSymlinkComponents(modelDir, file.name); + return { + file, + valid: await isValidFile(path.join(modelDir, file.name), file) + }; + }) + ); + const invalid = validity.filter(({ valid }) => !valid).map(({ file }) => file.name); + if (invalid.length > 0) { + throw new Error( + `Embedding model is not provisioned or failed integrity checks in ${modelDir}. Missing or invalid files: ${invalid.join(', ')}` + ); + } + return modelDir; +} + +async function downloadFile(url, outputPath, file, options = {}) { + const { fetchImpl = globalThis.fetch, timeoutMs = DOWNLOAD_TIMEOUT_MS } = options; + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), timeoutMs); + const temporaryPath = `${outputPath}.${process.pid}.${randomUUID()}.tmp`; + let handle; + + try { + const response = await fetchImpl(url, { signal: controller.signal }); + if (!response.ok) { + throw new Error( + `Failed to download ${url}, status ${response.status} (${response.statusText})` + ); + } + if (!response.body) throw new Error(`Failed to download ${url}: response has no body`); + + const contentLength = Number(response.headers.get('content-length')); + if (Number.isFinite(contentLength) && contentLength > file.size) { + throw new Error(`Refusing ${url}: response exceeds the expected ${file.size} bytes`); + } + + await fs.mkdir(path.dirname(outputPath), { recursive: true }); + handle = await fs.open(temporaryPath, 'wx', PROVISIONED_FILE_MODE); + const hash = createHash('sha256'); + let bytesWritten = 0; + + for await (const value of response.body) { + const chunk = Buffer.from(value); + bytesWritten += chunk.byteLength; + if (bytesWritten > file.size) { + throw new Error(`Refusing ${url}: response exceeds the expected ${file.size} bytes`); + } + hash.update(chunk); + await handle.writeFile(chunk); + } + + await handle.sync(); + await handle.close(); + handle = undefined; + + if (bytesWritten !== file.size) { + throw new Error(`Invalid size for ${url}: expected ${file.size}, received ${bytesWritten}`); + } + const digest = hash.digest('hex'); + if (digest !== file.sha256) { + throw new Error(`Invalid SHA-256 for ${url}: expected ${file.sha256}, received ${digest}`); + } + + await fs.chmod(temporaryPath, PROVISIONED_FILE_MODE); + + try { + await fs.rename(temporaryPath, outputPath); + } catch (error) { + if (error.code !== 'EEXIST' && error.code !== 'EPERM') throw error; + if (await isValidFile(outputPath, file)) await fs.unlink(temporaryPath); + else { + await fs.unlink(outputPath).catch(() => {}); + await fs.rename(temporaryPath, outputPath); + } + } + await fs.chmod(outputPath, PROVISIONED_FILE_MODE); + } catch (error) { + if (error.name === 'AbortError') { + throw new Error(`Timed out after ${timeoutMs} ms while downloading ${url}`, { cause: error }); + } + throw error; + } finally { + clearTimeout(timeout); + await handle?.close().catch(() => {}); + await fs.unlink(temporaryPath).catch(() => {}); + } +} + +async function downloadModelIfNeeded(modelDir, model, options) { + validateModelDescriptor(model); + await ensureDirectory(modelDir); + const hubUrl = normalizeHubUrl(options.hubUrl); + + for (const file of model.files) { + const filePath = path.join(modelDir, file.name); + // eslint-disable-next-line no-await-in-loop + await prepareArtifactPath(modelDir, file.name); + // eslint-disable-next-line no-await-in-loop + if (await isValidFile(filePath, file)) { + // Keep build-time provisioning readable when the runtime uses another UID. + // eslint-disable-next-line no-await-in-loop + await fs.chmod(filePath, PROVISIONED_FILE_MODE); + continue; + } + + const url = `${hubUrl}/${model.repository}/resolve/${model.revision}/${file.path}`; + // Files are downloaded serially to avoid multiplying startup bandwidth and memory usage. + // eslint-disable-next-line no-await-in-loop + await downloadFile(url, filePath, file, options); + } +} + +function normalizeHubUrl(value = 'https://huggingface.co') { + if (typeof value !== 'string' || !value.trim()) { + throw new TypeError('The Hugging Face Hub URL must be a non-empty string'); + } + let url; + try { + url = new URL(value.trim()); + } catch (error) { + throw new TypeError('The Hugging Face Hub URL must be a valid HTTPS URL', { cause: error }); + } + if (url.protocol !== 'https:') { + throw new TypeError('The Hugging Face Hub URL must use HTTPS'); + } + if (url.username || url.password) { + throw new TypeError('The Hugging Face Hub URL must not include credentials'); + } + if (url.search || url.hash) { + throw new TypeError('The Hugging Face Hub URL must not include a query or fragment'); + } + return url.href.replace(/\/+$/, ''); +} + +async function prepareArtifactPath(modelDir, relativePath) { + await assertNoSymlinkComponents(modelDir, relativePath); + let current = modelDir; + for (const part of relativePath.split('/').slice(0, -1)) { + current = path.join(current, part); + // eslint-disable-next-line no-await-in-loop + await fs.mkdir(current, { mode: PROVISIONED_DIRECTORY_MODE }).catch((error) => { + if (error.code !== 'EEXIST') throw error; + }); + // eslint-disable-next-line no-await-in-loop + const stat = await fs.lstat(current); + if (stat.isSymbolicLink()) { + throw new Error(`Embedding artifact path must not contain symbolic links: ${current}`); + } + if (!stat.isDirectory()) { + throw new Error(`Embedding artifact parent is not a directory: ${current}`); + } + // eslint-disable-next-line no-await-in-loop + await fs.chmod(current, PROVISIONED_DIRECTORY_MODE); + } + await assertNoSymlinkComponents(modelDir, relativePath); +} + +async function assertNoSymlinkComponents(modelDir, relativePath) { + const exists = await assertModelDirectory(modelDir); + if (!exists) return; + let current = modelDir; + const parts = relativePath.split('/'); + for (let index = 0; index < parts.length; index++) { + current = path.join(current, parts[index]); + let stat; + try { + // eslint-disable-next-line no-await-in-loop + stat = await fs.lstat(current); + } catch (error) { + if (error.code === 'ENOENT') return; + throw error; + } + if (stat.isSymbolicLink()) { + throw new Error(`Embedding artifact path must not contain symbolic links: ${current}`); + } + if (index < parts.length - 1 && !stat.isDirectory()) { + throw new Error(`Embedding artifact parent is not a directory: ${current}`); + } + } +} + +async function assertModelDirectory(modelDir) { + try { + const stat = await fs.lstat(modelDir); + if (stat.isSymbolicLink()) { + throw new Error(`Embedding model directory must not be a symbolic link: ${modelDir}`); + } + if (!stat.isDirectory()) { + throw new Error(`Embedding model path is not a directory: ${modelDir}`); + } + return true; + } catch (error) { + if (error.code === 'ENOENT') return false; + throw error; + } +} + +async function ensureDirectory(directory) { + await ensureParentDirectories(directory); + await assertModelDirectory(directory); + await fs.chmod(directory, PROVISIONED_DIRECTORY_MODE); +} + +async function ensureParentDirectories(directory) { + const missing = []; + let current = directory; + // eslint-disable-next-line no-await-in-loop + while (!(await pathExists(current))) { + missing.push(current); + const parent = path.dirname(current); + if (parent === current) break; + current = parent; + } + + for (const missingDirectory of missing.reverse()) { + // eslint-disable-next-line no-await-in-loop + await fs.mkdir(missingDirectory, { mode: PROVISIONED_DIRECTORY_MODE }).catch((error) => { + if (error.code !== 'EEXIST') throw error; + }); + // Explicit chmod avoids umask making build-time model directories unreadable at runtime. + // eslint-disable-next-line no-await-in-loop + await fs.chmod(missingDirectory, PROVISIONED_DIRECTORY_MODE); + } +} + +async function pathExists(filePath) { + try { + await fs.lstat(filePath); + return true; + } catch (error) { + if (error.code === 'ENOENT') return false; + throw error; + } +} + +async function canonicalizeProvisioningDirectory(directory) { + const requestedDirectory = path.resolve(directory); + let current = requestedDirectory; + const missing = []; + + while (true) { + try { + // eslint-disable-next-line no-await-in-loop + const stat = await fs.lstat(current); + if (current === requestedDirectory && stat.isSymbolicLink()) { + throw new Error(`Embedding model directory must not be a symbolic link: ${directory}`); + } + // eslint-disable-next-line no-await-in-loop + const canonicalAncestor = await fs.realpath(current); + // eslint-disable-next-line no-await-in-loop + const canonicalStat = await fs.stat(canonicalAncestor); + if (!canonicalStat.isDirectory()) { + throw new Error(`Embedding model path is not a directory: ${current}`); + } + return path.join(canonicalAncestor, ...missing.reverse()); + } catch (error) { + if (error.code !== 'ENOENT') throw error; + missing.push(path.basename(current)); + const parent = path.dirname(current); + if (parent === current) throw error; + current = parent; + } + } +} + +async function readModelLock(modelDir) { + const lockPath = path.join(modelDir, MODEL_LOCK_FILE); + let lock; + try { + await assertModelDirectory(modelDir); + const stat = await fs.lstat(lockPath); + if (stat.isSymbolicLink()) { + throw new Error(`Embedding model lock must not be a symbolic link at ${lockPath}`); + } + lock = JSON.parse(await fs.readFile(lockPath, 'utf8')); + } catch (error) { + if (error.code === 'ENOENT') { + throw new Error(`Embedding model lock not found at ${lockPath}`, { cause: error }); + } + throw new Error(`Cannot read embedding model lock at ${lockPath}: ${error.message}`, { + cause: error + }); + } + if (lock.formatVersion !== MODEL_LOCK_VERSION) { + if (lock.formatVersion === 1) { + throw new Error( + `Embedding model lock version 1 at ${lockPath} predates prompt semantics; remove and reinstall the model` + ); + } + throw new Error( + `Unsupported embedding model lock version ${lock.formatVersion ?? 'missing'} at ${lockPath}` + ); + } + const { formatVersion, ...model } = lock; + void formatVersion; + return validateModelDescriptor(model); +} + +async function writeModelLock(modelDir, model) { + const lockPath = path.join(modelDir, MODEL_LOCK_FILE); + const temporaryPath = `${lockPath}.${process.pid}.${randomUUID()}.tmp`; + const contents = `${JSON.stringify({ ...model, formatVersion: MODEL_LOCK_VERSION }, null, 2)}\n`; + let handle; + try { + await ensureDirectory(modelDir); + handle = await fs.open(temporaryPath, 'wx', PROVISIONED_FILE_MODE); + await handle.writeFile(contents); + await handle.sync(); + await handle.close(); + handle = undefined; + await fs.chmod(temporaryPath, PROVISIONED_FILE_MODE); + await fs.rename(temporaryPath, lockPath); + await fs.chmod(lockPath, PROVISIONED_FILE_MODE); + } finally { + await handle?.close().catch(() => {}); + await fs.unlink(temporaryPath).catch(() => {}); + } +} + +async function provisionModel(model, options = {}) { + validateModelDescriptor(model); + if (typeof options.directory !== 'string' || !options.directory.trim()) { + throw new Error('A non-empty provisioning directory is required'); + } + const requestedDirectory = path.resolve(options.directory); + const directory = await canonicalizeProvisioningDirectory(requestedDirectory); + await ensureParentDirectories(path.dirname(directory)); + + return withInstallLock(directory, async () => { + const directoryExists = await assertModelDirectory(directory); + let lockedModel; + try { + lockedModel = await readModelLock(directory); + if (modelDescriptorDigest(lockedModel) !== modelDescriptorDigest(model)) { + throw new Error( + `Embedding model directory ${directory} is locked to a different model descriptor for ${lockedModel.repository}@${lockedModel.revision}. Choose another directory or remove it explicitly.` + ); + } + } catch (error) { + if (!/Embedding model lock not found/.test(error.message)) throw error; + } + + if (directoryExists) { + try { + await verifyModelDirectory(directory, model); + if (!lockedModel) await writeModelLock(directory, model); + await makeModelDirectoryReadable(directory, model); + await options.validate?.(directory, model); + return directory; + } catch (error) { + if (/symbolic link/.test(error.message)) throw error; + if (!lockedModel && (await directoryHasEntries(directory))) { + throw new Error( + `Embedding model directory ${directory} is not empty and has no valid lock. Choose an empty directory or remove its contents explicitly.`, + { cause: error } + ); + } + } + } + + const stagingDirectory = await createStagingDirectory(directory); + let published = false; + try { + await downloadModelIfNeeded(stagingDirectory, model, options); + await verifyModelDirectory(stagingDirectory, model); + await writeModelLock(stagingDirectory, model); + await options.validate?.(stagingDirectory, model); + await publishModelDirectory(stagingDirectory, directory); + published = true; + } finally { + if (!published) await fs.rm(stagingDirectory, { recursive: true, force: true }); + } + return directory; + }); +} + +async function withInstallLock(directory, callback) { + const lockPath = path.join( + path.dirname(directory), + `.${path.basename(directory)}${MODEL_INSTALL_LOCK_FILE}` + ); + const owner = { + formatVersion: 1, + pid: process.pid, + hostname: os.hostname(), + createdAt: new Date().toISOString(), + token: randomUUID() + }; + let handle; + let heartbeat; + try { + handle = await acquireInstallLock(lockPath, directory, owner); + heartbeat = setInterval( + () => { + const now = new Date(); + fs.utimes(lockPath, now, now).catch(() => {}); + }, + Math.min(INSTALL_LOCK_STALE_MS / 3, 60 * 1000) + ); + heartbeat.unref(); + return await callback(); + } finally { + clearInterval(heartbeat); + await handle?.close().catch(() => {}); + if (handle) await releaseInstallLock(lockPath, owner); + } +} + +async function acquireInstallLock(lockPath, directory, owner) { + try { + return await createInstallLock(lockPath, owner); + } catch (error) { + if (error.code !== 'EEXIST') throw error; + if (await recoverStaleInstallLock(lockPath)) { + return createInstallLock(lockPath, owner); + } + throw Object.assign( + new Error(`Embedding model directory ${directory} is already being provisioned`, { + cause: error + }), + { code: MODEL_PROVISIONING_IN_PROGRESS } + ); + } +} + +async function createInstallLock(lockPath, owner) { + let handle; + try { + handle = await fs.open(lockPath, 'wx', 0o600); + await handle.writeFile(`${JSON.stringify(owner)}\n`); + await handle.sync(); + return handle; + } catch (error) { + await handle?.close().catch(() => {}); + if (handle) await fs.unlink(lockPath).catch(() => {}); + throw error; + } +} + +async function recoverStaleInstallLock(lockPath) { + let contents; + let stat; + try { + [contents, stat] = await Promise.all([fs.readFile(lockPath, 'utf8'), fs.lstat(lockPath)]); + } catch (error) { + return error.code === 'ENOENT'; + } + + let owner; + try { + owner = JSON.parse(contents); + } catch { + if (Date.now() - stat.mtimeMs <= INSTALL_LOCK_STALE_MS) return false; + } + if (!isStaleInstallLock(owner, stat)) return false; + + const stalePath = `${lockPath}.${randomUUID()}.stale`; + try { + await fs.rename(lockPath, stalePath); + const movedContents = await fs.readFile(stalePath, 'utf8'); + if (movedContents !== contents) { + await fs.rename(stalePath, lockPath).catch(() => {}); + return false; + } + await fs.unlink(stalePath); + return true; + } catch (error) { + await fs.unlink(stalePath).catch(() => {}); + return error.code === 'ENOENT'; + } +} + +function isStaleInstallLock(owner, stat) { + if (!owner || typeof owner !== 'object') { + return Date.now() - stat.mtimeMs > INSTALL_LOCK_STALE_MS; + } + if (owner.hostname === os.hostname() && Number.isSafeInteger(owner.pid)) { + try { + process.kill(owner.pid, 0); + return false; + } catch (error) { + if (error.code === 'EPERM') return false; + if (error.code === 'ESRCH') return true; + return false; + } + } + return Date.now() - stat.mtimeMs > INSTALL_LOCK_STALE_MS; +} + +async function releaseInstallLock(lockPath, owner) { + try { + const currentOwner = JSON.parse(await fs.readFile(lockPath, 'utf8')); + if (currentOwner.token === owner.token) await fs.unlink(lockPath); + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } +} + +async function createStagingDirectory(directory) { + const parent = path.dirname(directory); + const stagingDirectory = await fs.mkdtemp( + path.join(parent, `.${path.basename(directory)}.staging-`) + ); + await fs.chmod(stagingDirectory, PROVISIONED_DIRECTORY_MODE); + return stagingDirectory; +} + +async function publishModelDirectory(stagingDirectory, directory) { + const targetExists = await assertModelDirectory(directory); + let backupDirectory; + if (targetExists) { + backupDirectory = path.join( + path.dirname(directory), + `.${path.basename(directory)}.${randomUUID()}.backup` + ); + await fs.rename(directory, backupDirectory); + } + + try { + await fs.rename(stagingDirectory, directory); + } catch (error) { + if (backupDirectory) await fs.rename(backupDirectory, directory).catch(() => {}); + throw error; + } + if (backupDirectory) await fs.rm(backupDirectory, { recursive: true, force: true }); +} + +async function directoryHasEntries(directory) { + return (await fs.readdir(directory)).length > 0; +} + +async function makeModelDirectoryReadable(directory, model) { + await fs.chmod(directory, PROVISIONED_DIRECTORY_MODE); + for (const file of model.files) { + let current = directory; + for (const part of file.name.split('/').slice(0, -1)) { + current = path.join(current, part); + // eslint-disable-next-line no-await-in-loop + await fs.chmod(current, PROVISIONED_DIRECTORY_MODE); + } + // eslint-disable-next-line no-await-in-loop + await fs.chmod(path.join(directory, file.name), PROVISIONED_FILE_MODE); + } + await fs.chmod(path.join(directory, MODEL_LOCK_FILE), PROVISIONED_FILE_MODE); +} + +async function loadModelAndTokenizer(modelDir, model) { + const [{ Tokenizer }, { SynchronousInferenceSession }] = await Promise.all([ + loadTokenizerPackage(), + import('./SynchronousInferenceSession.js') + ]); + const modelPath = path.join(modelDir, fileForRole(model, 'model').name); + const tokenizerPath = path.join(modelDir, fileForRole(model, 'tokenizer').name); + const tokenizerConfigPath = path.join(modelDir, fileForRole(model, 'tokenizerConfig').name); + const [tokenizerJson, tokenizerConfig] = await Promise.all([ + fs.readFile(tokenizerPath, 'utf8').then(JSON.parse), + fs.readFile(tokenizerConfigPath, 'utf8').then(JSON.parse) + ]); + const tokenizer = new Tokenizer(tokenizerJson, tokenizerConfig); + + return { + session: await SynchronousInferenceSession.create(modelPath), + tokenizer + }; +} + +async function loadTokenizerPackage(importModule = (specifier) => import(specifier)) { + try { + return await importModule('@huggingface/tokenizers'); + } catch (error) { + if ( + error?.code === 'ERR_MODULE_NOT_FOUND' && + /Cannot find package ['"]@huggingface\/tokenizers['"]/.test(error.message) + ) { + throw new Error( + "Using local SQLite embeddings requires @huggingface/tokenizers@0.1.3. Install it with 'npm add -D @huggingface/tokenizers@0.1.3'.", + { cause: error } + ); + } + throw error; + } +} + +export { + MODEL_LOCK_FILE, + MODEL_LOCK_VERSION, + MODEL_PROVISIONING_IN_PROGRESS, + assertSafeRepository, + downloadFile, + downloadModelIfNeeded, + fileForRole, + getModelDirectory, + getModelRoot, + isValidFile, + loadModelAndTokenizer, + loadTokenizerPackage, + modelDescriptorDigest, + normalizeHubUrl, + provisionModel, + readModelLock, + verifyModelDirectory, + validateModelDescriptor +}; diff --git a/package.json b/package.json index a16b543..aca09d5 100644 --- a/package.json +++ b/package.json @@ -8,24 +8,57 @@ "author": "SAP SE (https://www.sap.com)", "homepage": "https://cap.cloud.sap/", "main": "cds-plugin.js", + "bin": { + "cds-ai": "bin/cds-ai.js" + }, "scripts": { "lint": "npx -y eslint@10 .", - "test": "node --test tests/*.test.js", - "test:hybrid": "cds bind --exec -- node --test tests/*.test.js tests/integration/*.test.js", + "test:model:provision": "node tests/provision-model.js", + "test": "npm run test:model:provision && node --test tests/*.test.js", + "test:hybrid": "npm run test:model:provision && cds bind --exec -- node --test tests/*.test.js tests/integration/*.test.js", "format": "npx -y prettier@3 . --write && format-cds -f", "format:check": "npx -y prettier@3 --check . && format-cds --check" }, "files": [ + ".docs", "CHANGELOG.md", + "bin", "lib", "srv" ], "devDependencies": { "@cap-js/cds-test": "^1", - "@cap-js/cds-types": "^0.16.0" + "@cap-js/cds-types": "^0.16.0", + "@cap-js/sqlite": ">=2", + "@huggingface/hub": "^2.15.0", + "@huggingface/tokenizers": "0.1.3", + "onnxruntime-node": "1.20.1", + "oxigraph": "^0.5.9" }, "peerDependencies": { - "@sap/cds": ">=9" + "@cap-js/sqlite": ">=2", + "@huggingface/hub": "^2.15.0", + "@huggingface/tokenizers": "0.1.3", + "@sap/cds": ">=9", + "onnxruntime-node": "1.20.1", + "oxigraph": "^0.5.9" + }, + "peerDependenciesMeta": { + "@cap-js/sqlite": { + "optional": true + }, + "@huggingface/hub": { + "optional": true + }, + "@huggingface/tokenizers": { + "optional": true + }, + "onnxruntime-node": { + "optional": true + }, + "oxigraph": { + "optional": true + } }, "engines": { "node": ">=20.0.0" @@ -49,6 +82,12 @@ "vcap": { "label": "aicore" } + }, + "sqlite": { + "impl": "@cap-js/ai/lib/sqlite/AISQLiteService.js", + "embedding": { + "model": "sentence-transformers/all-MiniLM-L6-v2" + } } } } diff --git a/tests/bookshop/db/data/cap.ttl b/tests/bookshop/db/data/cap.ttl new file mode 100644 index 0000000..c58a8c0 --- /dev/null +++ b/tests/bookshop/db/data/cap.ttl @@ -0,0 +1,17 @@ +@prefix cap: . + +# CAP sample turtle file + +cap:Service cap:label "Service" . +cap:DatabaseService a cap:Service . +cap:DatabaseService cap:label "Database Service" . +cap:SQLiteService a cap:DatabaseService . +cap:SQLiteService cap:name "SQLite Service" . +cap:SQLiteService cap:label "SQLite Service" . +cap:HANAService cap:implementedBy cap:cap-js-sqlite . +cap:HANAService a cap:DatabaseService . +cap:HANAService cap:name "HANA Service" . +cap:HANAService cap:label "HANA Service" . +cap:HANAService cap:implementedBy cap:cap-js-hana . +cap:cap-js-hana cap:label "@cap-js/hana" . +cap:cap-js-sqlite cap:label "@cap-js/sqlite" . diff --git a/tests/bookshop/db/data/cap.ttl.gz b/tests/bookshop/db/data/cap.ttl.gz new file mode 100644 index 0000000..9595224 Binary files /dev/null and b/tests/bookshop/db/data/cap.ttl.gz differ diff --git a/tests/bookshop/package.json b/tests/bookshop/package.json index 646bd66..60d494f 100644 --- a/tests/bookshop/package.json +++ b/tests/bookshop/package.json @@ -22,7 +22,10 @@ }, "devDependencies": { "@cap-js/cds-types": "^0.16.0", - "@cap-js/sqlite": ">=2" + "@cap-js/sqlite": ">=2", + "@huggingface/hub": "^2.15.0", + "@huggingface/tokenizers": "0.1.3", + "onnxruntime-node": "1.20.1" }, "engines": { "node": "^22.11.0" @@ -43,6 +46,28 @@ "version": "https://sapui5nightly.int.sap.eu2.hana.ondemand.com" }, "requires": { + "[development]": { + "db": { + "kind": "sqlite", + "credentials": { + "url": ":memory:" + }, + "embedding": { + "directory": "../../.cds/models" + } + } + }, + "[test]": { + "db": { + "kind": "sqlite", + "credentials": { + "url": ":memory:" + }, + "embedding": { + "directory": "../../.cds/models" + } + } + }, "[production]": { "auth": "xsuaa", "db": { @@ -55,7 +80,10 @@ } }, "[hybrid]": { - "db": "hana" + "db": { + "kind": "hana", + "embedding": null + } }, "[with-mtx]": { "multitenancy": true diff --git a/tests/bookshop/srv/cat-service.cds b/tests/bookshop/srv/cat-service.cds index e7a86e5..00b6daf 100644 --- a/tests/bookshop/srv/cat-service.cds +++ b/tests/bookshop/srv/cat-service.cds @@ -75,7 +75,7 @@ service CatalogService { }; @requires: 'authenticated-user' - action submitOrder(book: Books:ID, quantity: Integer) returns { + action submitOrder(book: Books:ID, quantity: Integer) returns { stock : Integer }; @@ -86,5 +86,7 @@ service CatalogService { }; @requires: 'authenticated-user' - action callProcedure(); + action callProcedure(); + + function embedding(text: String) returns LargeString; } diff --git a/tests/bookshop/srv/cat-service.js b/tests/bookshop/srv/cat-service.js index 893d5f6..a0909ff 100644 --- a/tests/bookshop/srv/cat-service.js +++ b/tests/bookshop/srv/cat-service.js @@ -20,6 +20,16 @@ export default class CatalogService extends cds.ApplicationService { } else return req.error(409, `${quantity} exceeds stock for book #${book}`); }); + if (cds.env.requires.db.embedding?.model) { + this.on('embedding', async (req) => { + const [row] = await cds.db.run( + `SELECT VECTOR_EMBEDDING(?, 'DOCUMENT', 'local') AS embedding`, + [req.data.text] + ); + return row.embedding; + }); + } + this.before('UPDATE', Books.drafts, async (req) => { if (req.data.stock < 0) { req.warn({ diff --git a/tests/fixtures/sentence-transformers/all-MiniLM-L6-v2/embedding.lock.json b/tests/fixtures/sentence-transformers/all-MiniLM-L6-v2/embedding.lock.json new file mode 100644 index 0000000..c73bcd2 --- /dev/null +++ b/tests/fixtures/sentence-transformers/all-MiniLM-L6-v2/embedding.lock.json @@ -0,0 +1,43 @@ +{ + "repository": "sentence-transformers/all-MiniLM-L6-v2", + "revision": "1110a243fdf4706b3f48f1d95db1a4f5529b4d41", + "dimensions": 384, + "maxLength": 128, + "files": [ + { + "role": "model", + "name": "model.onnx", + "path": "onnx/model.onnx", + "size": 90405214, + "sha256": "6fd5d72fe4589f189f8ebc006442dbb529bb7ce38f8082112682524616046452" + }, + { + "role": "tokenizer", + "name": "tokenizer.json", + "path": "tokenizer.json", + "size": 466247, + "sha256": "be50c3628f2bf5bb5e3a7f17b1f74611b2561a3a27eeab05e5aa30f411572037" + }, + { + "role": "tokenizerConfig", + "name": "tokenizer_config.json", + "path": "tokenizer_config.json", + "size": 350, + "sha256": "acb92769e8195aabd29b7b2137a9e6d6e25c476a4f15aa4355c233426c61576b" + }, + { + "role": "auxiliary", + "name": "config.json", + "path": "config.json", + "size": 612, + "sha256": "953f9c0d463486b10a6871cc2fd59f223b2c70184f49815e7efbcab5d8908b41" + } + ], + "output": { + "name": "last_hidden_state", + "pooling": "mean", + "normalize": true, + "includePrompt": true + }, + "formatVersion": 2 +} diff --git a/tests/huggingface-hub.test.js b/tests/huggingface-hub.test.js new file mode 100644 index 0000000..010dbf1 --- /dev/null +++ b/tests/huggingface-hub.test.js @@ -0,0 +1,283 @@ +import assert from 'node:assert/strict'; +import { describe, test } from 'node:test'; + +import { + createHuggingFaceClient, + loadHuggingFaceHub +} from '../lib/vector_embedding/huggingface-hub.js'; + +describe('Hugging Face Hub adapter', () => { + test('pins all operations and forwards custom transport options', async () => { + const calls = []; + const fetchImpl = () => {}; + const hubApi = recordingHubApi(calls); + const client = createHuggingFaceClient({ + fetchImpl, + hubUrl: 'https://hub.example.test', + hubApi + }); + + assert.deepEqual(await client.getModelInfo('foo/bar'), { sha: '1'.repeat(40) }); + assert.deepEqual(await client.getFiles('foo/bar', '2'.repeat(40)), [ + { type: 'file', path: 'onnx/model.onnx', size: 42 } + ]); + assert.deepEqual( + await client.getFile('foo/bar', '2'.repeat(40), 'config.json'), + Buffer.from('contents') + ); + + for (const [, options] of calls) { + assert.equal(typeof options.fetch, 'function'); + assert.equal(options.hubUrl, 'https://hub.example.test'); + assert.equal('accessToken' in options, false); + } + assert.deepEqual(calls[1][1].repo, { type: 'model', name: 'foo/bar' }); + assert.equal(calls[1][1].revision, '2'.repeat(40)); + assert.equal(calls[2][1].xet, false); + }); + + test('rejects unsafe Hub URLs', () => { + const hubApi = downloadHubApi(); + assert.throws( + () => createHuggingFaceClient({ hubUrl: 'http://hub.example.test', hubApi }), + /must use HTTPS/ + ); + assert.throws( + () => createHuggingFaceClient({ hubUrl: 'https://user@hub.example.test', hubApi }), + /must not include credentials/ + ); + assert.throws( + () => createHuggingFaceClient({ hubUrl: 'https://hub.example.test?mirror=1', hubApi }), + /must not include a query or fragment/ + ); + }); + + test('matches the installed @huggingface/hub response contract', async () => { + const revision = '1'.repeat(40); + const contents = Buffer.from('{"hidden_size":384}'); + const requests = []; + const fetchImpl = async (input, init = {}) => { + const url = String(input); + const headers = new Headers(init.headers); + requests.push({ url, headers }); + + if (url.includes('/api/models/foo/bar/revision/HEAD?')) { + return Response.json({ + _id: 'model-id', + id: 'foo/bar', + private: false, + pipeline_tag: 'sentence-similarity', + downloads: 42, + gated: false, + likes: 7, + lastModified: '2026-01-01T00:00:00.000Z', + sha: revision, + siblings: [{ rfilename: 'config.json' }] + }); + } + if (url.includes(`/api/models/foo/bar/tree/${revision}?`)) { + return Response.json([ + { type: 'directory', path: 'onnx', size: 0 }, + { + type: 'file', + path: 'onnx/model.onnx', + size: contents.length, + lfs: { oid: '2'.repeat(64), size: contents.length, pointerSize: 128 } + } + ]); + } + if (url.endsWith(`/foo/bar/resolve/${revision}/config.json`)) { + if (headers.get('range')) { + return new Response(contents.subarray(0, 1), { + status: 206, + headers: { + 'content-range': `bytes 0-0/${contents.length}`, + 'content-type': 'application/json', + etag: '"config"' + } + }); + } + return new Response(contents, { headers: { 'content-type': 'application/json' } }); + } + throw new Error(`Unexpected request: ${url}`); + }; + const client = createHuggingFaceClient({ + fetchImpl, + hubUrl: 'https://hub.example.test', + requestRetries: 0 + }); + + const info = await client.getModelInfo('foo/bar'); + assert.equal(info.task, 'sentence-similarity'); + assert.equal(info.sha, revision); + assert.deepEqual(await client.getFiles('foo/bar', revision), [ + { + type: 'file', + path: 'onnx/model.onnx', + size: contents.length, + lfs: { oid: '2'.repeat(64), size: contents.length, pointerSize: 128 } + } + ]); + assert.deepEqual(await client.getFile('foo/bar', revision, 'config.json'), contents); + assert.ok(requests.every(({ headers }) => !headers.has('authorization'))); + }); + + test('retries transient Hub responses', async () => { + let attempts = 0; + const client = createHuggingFaceClient({ + fetchImpl: async () => { + attempts++; + return attempts === 1 ? new Response('unavailable', { status: 503 }) : Response.json({}); + }, + requestRetries: 1, + requestRetryMs: 0, + hubApi: fetchOnlyHubApi() + }); + + await client.getModelInfo('foo/bar'); + assert.equal(attempts, 2); + }); + + test('times out stalled Hub requests', async () => { + const client = createHuggingFaceClient({ + fetchImpl: (input, { signal }) => + new Promise((resolve, reject) => { + void input; + void resolve; + signal.addEventListener('abort', () => reject(signal.reason), { once: true }); + }), + requestRetries: 0, + requestTimeoutMs: 10, + hubApi: fetchOnlyHubApi() + }); + + await assert.rejects(client.getModelInfo('foo/bar'), /Timed out after 10 ms/); + }); + + test('does not retry non-transient Hub responses', async () => { + let attempts = 0; + const client = createHuggingFaceClient({ + fetchImpl: async () => { + attempts++; + return new Response('not found', { status: 404 }); + }, + requestRetries: 2, + requestRetryMs: 0, + hubApi: fetchOnlyHubApi() + }); + + await assert.rejects(client.getModelInfo('foo/bar'), /status 404/); + assert.equal(attempts, 1); + }); + + test('rejects oversized Hub files from declared response metadata', async () => { + const client = createHuggingFaceClient({ + fetchImpl: async () => + new Response('oversized', { + headers: { 'content-length': '9' } + }), + requestRetries: 0, + maxResponseBytes: 8, + hubApi: downloadHubApi() + }); + + await assert.rejects( + client.getFile('foo/bar', '1'.repeat(40), 'config.json'), + /exceeds 8 bytes/ + ); + }); + + test('rejects oversized Hub files from a range probe before downloading them', async () => { + const client = createHuggingFaceClient({ + fetchImpl: async () => + new Response('x', { + status: 206, + headers: { 'content-length': '1', 'content-range': 'bytes 0-0/9' } + }), + requestRetries: 0, + maxResponseBytes: 8, + hubApi: downloadHubApi() + }); + + await assert.rejects( + client.getFile('foo/bar', '1'.repeat(40), 'config.json'), + /exceeds 8 bytes/ + ); + }); + + test('rejects streamed Hub files that exceed the response limit', async () => { + const client = createHuggingFaceClient({ + fetchImpl: async () => + new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(Buffer.from('four')); + controller.enqueue(Buffer.from('more')); + controller.close(); + } + }) + ), + requestRetries: 0, + maxResponseBytes: 4, + hubApi: downloadHubApi() + }); + + await assert.rejects( + client.getFile('foo/bar', '1'.repeat(40), 'config.json'), + /exceeds 4 bytes/ + ); + }); + + test('explains how to install the missing optional discovery peer', async () => { + const missingHub = Object.assign( + new Error("Cannot find package '@huggingface/hub' imported from huggingface-hub.js"), + { code: 'ERR_MODULE_NOT_FOUND' } + ); + await assert.rejects( + loadHuggingFaceHub(async () => { + throw missingHub; + }), + /npm add -D @huggingface\/hub/ + ); + }); +}); + +function recordingHubApi(calls) { + return { + async modelInfo(options) { + calls.push(['modelInfo', options]); + return { sha: '1'.repeat(40) }; + }, + async *listFiles(options) { + calls.push(['listFiles', options]); + yield { type: 'directory', path: 'onnx', size: 0 }; + yield { type: 'file', path: 'onnx/model.onnx', size: 42 }; + }, + async downloadFile(options) { + calls.push(['downloadFile', options]); + return new Blob(['contents']); + } + }; +} + +function fetchOnlyHubApi() { + return { + async modelInfo({ fetch }) { + const response = await fetch('https://hub.example.test/model'); + if (!response.ok) throw new Error(`status ${response.status}`); + return {}; + }, + async *listFiles() {}, + async downloadFile() {} + }; +} + +function downloadHubApi() { + return { + async modelInfo() {}, + async *listFiles() {}, + async downloadFile({ fetch }) { + return (await fetch('https://hub.example.test/file')).blob(); + } + }; +} diff --git a/tests/knowledge-graph.test.js b/tests/knowledge-graph.test.js new file mode 100644 index 0000000..2b1680f --- /dev/null +++ b/tests/knowledge-graph.test.js @@ -0,0 +1,133 @@ +import { after, before, beforeEach, describe, test } from 'node:test'; +import assert from 'node:assert'; +import { symlink, unlink, writeFile } from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import cds from '@sap/cds'; + +describe('SQLite knowledge graph', () => { + let db; + + const data = fileURLToPath(new URL('./bookshop/db/data/cap.ttl', import.meta.url)); + const graph = 'https://cap.cloud.sap/example'; + + before(async () => { + db = await cds.connect.to('knowledge-graph-db', { + kind: 'sqlite', + credentials: { url: ':memory:' } + }); + }); + + beforeEach(async () => { + await db.disconnect(); + }); + + after(async () => { + await db?.disconnect(); + }); + + test('loads Turtle data', async () => { + assert.strictEqual(await load(data), undefined); + assert.strictEqual((await triples()).length, 13); + }); + + test('loads compressed Turtle data', async () => { + await load(`${data}.gz`); + assert.strictEqual((await triples()).length, 13); + }); + + test('rejects malformed SPARQL_EXECUTE calls', async () => { + for (const query of [ + `CALL SPARQL_EXECUTE('SELECT * WHERE { ?s ?p ?o }')`, + `CALL SPARQL_EXECUTE('SELECT * WHERE { ?s ?p ?o }','', ?)`, + `CALL SPARQL_EXECUTE('SELECT * WHERE { ?s ?p ?o }','', NULL, NULL)`, + `CALL SPARQL_EXECUTE(?, '', ?, ?)` + ]) { + // eslint-disable-next-line no-await-in-loop + await assert.rejects(db.run(query), /Unsupported SPARQL_EXECUTE syntax/); + } + }); + + test('returns query results through the RESPONSE output', async () => { + await load(data); + const result = await db.run( + `CALL SPARQL_EXECUTE('SELECT ?subject WHERE { ?subject ?predicate ?object }','accept:application/sparql-results+json', ?, ?)` + ); + + assert.deepStrictEqual(Object.keys(result), ['RESPONSE']); + const response = JSON.parse(result.RESPONSE); + assert.deepStrictEqual(response.head.vars, ['subject']); + assert.ok(response.results.bindings.length > 0); + }); + + test('rejects RDF files outside the project', async () => { + await assert.rejects( + db.run(`CALL SPARQL_EXECUTE('LOAD ','', ?, ?)`), + /outside the project/ + ); + }); + + test('rejects project-local symlinks pointing outside the project', async () => { + const link = path.join(cds.root, 'tests/bookshop/db/data/outside.ttl'); + await symlink('/etc/passwd', link); + try { + await assert.rejects( + db.run(`CALL SPARQL_EXECUTE('LOAD <${link}>','', ?, ?)`), + /outside the project/ + ); + } finally { + await unlink(link); + } + }); + + test('checks RDF format before trying to open the file', async () => { + await assert.rejects( + db.run(`CALL SPARQL_EXECUTE('LOAD <${data}.unsupported>','', ?, ?)`), + /Unsupported RDF file format: .unsupported/ + ); + }); + + test('supports SPARQL prologues and SELECT without WHERE', async () => { + await load(data); + const result = await db.run({ + SELECT: { + from: cds.ql.func( + 'sparql_table', + `BASE \nPREFIX cap: \nSELECT ?subject ?predicate { ?subject ?predicate ?object . }` + ) + } + }); + assert.ok(result.length > 0); + assert.deepStrictEqual(Object.keys(result[0]), ['subject', 'predicate']); + }); + + test('keeps a graph unchanged when a valid RDF file is malformed', async () => { + await load(data); + const malformed = path.join(cds.root, 'tests/bookshop/db/data/malformed.ttl'); + await writeFile( + malformed, + ' .\nnot turtle' + ); + try { + await assert.rejects(load(malformed)); + assert.strictEqual((await triples()).length, 13); + } finally { + await unlink(malformed); + } + }); + + async function load(file) { + return db.run(`CALL SPARQL_EXECUTE('LOAD <${file}> INTO GRAPH <${graph}>','', ?, ?)`); + } + + async function triples() { + return db.run({ + SELECT: { + from: cds.ql.func( + 'sparql_table', + 'SELECT ?subject ?predicate ?object WHERE { ?subject ?predicate ?object . }' + ) + } + }); + } +}); diff --git a/tests/model-discovery.test.js b/tests/model-discovery.test.js new file mode 100644 index 0000000..9754a72 --- /dev/null +++ b/tests/model-discovery.test.js @@ -0,0 +1,533 @@ +import assert from 'node:assert/strict'; +import { createHash } from 'node:crypto'; +import path from 'node:path'; +import { describe, test } from 'node:test'; + +import { checkModel, discoverModel } from '../lib/vector_embedding/model-discovery.js'; + +const REVISION = '1'.repeat(40); +const BASE_REVISION = '2'.repeat(40); +const REPOSITORY = 'example/embedding-model'; +const BASE_REPOSITORY = 'sentence-transformers/base-model'; + +describe('Hugging Face model discovery', () => { + test('allows a missing Hub task while checking metadata without downloading ONNX', async () => { + const hub = hubFor({ omitTask: true }); + + const result = await checkModel(REPOSITORY, { hubClient: hub.client }); + + assert.deepEqual( + { + repository: result.repository, + revision: result.revision, + task: result.task, + dimensions: result.dimensions, + maxLength: result.maxLength, + files: result.files.map(({ role, name, path }) => ({ role, name, path })), + output: result.output + }, + { + repository: REPOSITORY, + revision: REVISION, + task: undefined, + dimensions: 384, + maxLength: 96, + files: [ + { role: 'model', name: 'model.onnx', path: 'onnx/model.onnx' }, + { role: 'tokenizer', name: 'tokenizer.json', path: 'tokenizer.json' }, + { + role: 'tokenizerConfig', + name: 'tokenizer_config.json', + path: 'tokenizer_config.json' + }, + { role: 'auxiliary', name: 'config.json', path: 'config.json' } + ], + output: { + name: 'last_hidden_state', + pooling: 'mean', + normalize: true, + includePrompt: true + } + } + ); + assert.ok( + hub.fileRequests.every(([, , remotePath]) => !remotePath.toLowerCase().endsWith('.onnx')), + 'the metadata-only check must not fetch ONNX bytes' + ); + }); + + test('creates a descriptor from a conventional Sentence Transformers ONNX repository', async () => { + const hub = hubFor({ + tokenizer: { truncation: { max_length: 96 } }, + tokenizerConfig: { model_max_length: 128 }, + config: { hidden_size: 384, max_position_embeddings: 512 }, + sentenceConfig: { max_seq_length: 256 }, + normalize: true + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.repository, REPOSITORY); + assert.equal(descriptor.revision, REVISION); + assert.equal(descriptor.dimensions, 384); + assert.equal(descriptor.maxLength, 96); + assert.deepEqual(descriptor.output, { + name: 'last_hidden_state', + pooling: 'mean', + normalize: true, + includePrompt: true + }); + assert.deepEqual( + descriptor.files.map(({ role, name, path }) => ({ role, name, path })), + [ + { role: 'model', name: 'model.onnx', path: 'onnx/model.onnx' }, + { role: 'tokenizer', name: 'tokenizer.json', path: 'tokenizer.json' }, + { + role: 'tokenizerConfig', + name: 'tokenizer_config.json', + path: 'tokenizer_config.json' + }, + { role: 'auxiliary', name: 'config.json', path: 'config.json' } + ] + ); + const tokenizerFile = descriptor.files.find(({ role }) => role === 'tokenizer'); + assert.equal(tokenizerFile.sha256, digest(hub.files[REPOSITORY]['tokenizer.json'])); + assert.equal(tokenizerFile.size, hub.files[REPOSITORY]['tokenizer.json'].length); + assert.deepEqual(hub.fileLists, [[REPOSITORY, REVISION]]); + }); + + test('selects a unique nested export and adjacent tokenizer/configuration files', async () => { + const hub = hubFor({ + modelPath: 'exports/encoder/model.onnx', + assetDirectory: 'exports/encoder', + tokenizer: { truncation: null }, + tokenizerConfig: { model_max_length: 1e30 }, + config: { d_model: 768, n_positions: 256 }, + sentenceConfig: undefined + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.dimensions, 768); + assert.equal(descriptor.maxLength, 256); + assert.deepEqual( + descriptor.files.map(({ role, path }) => ({ role, path })), + [ + { role: 'model', path: 'exports/encoder/model.onnx' }, + { role: 'tokenizer', path: 'exports/encoder/tokenizer.json' }, + { role: 'tokenizerConfig', path: 'exports/encoder/tokenizer_config.json' }, + { role: 'auxiliary', path: 'exports/encoder/config.json' } + ] + ); + }); + + test('prefers metadata next to a nested ONNX export over repository-root files', async () => { + const hub = hubFor({ + modelPath: 'exports/encoder/model.onnx', + assetDirectory: 'exports/encoder', + tokenizer: { truncation: { max_length: 96 } }, + config: { hidden_size: 768, max_position_embeddings: 256 }, + rootAssets: { + tokenizer: { truncation: { max_length: 16 } }, + tokenizerConfig: { model_max_length: 16 }, + config: { hidden_size: 16, max_position_embeddings: 16 } + } + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.dimensions, 768); + assert.equal(descriptor.maxLength, 96); + assert.ok( + descriptor.files + .filter(({ role }) => role !== 'model') + .every(({ path }) => path.startsWith('exports/encoder/')) + ); + }); + + test('recognizes common Transformers dimension and sequence-limit aliases', async () => { + await Promise.all( + [ + [{ n_embd: 32, n_ctx: 1024 }, 32, 1024], + [{ d_model: 64, n_positions: 768 }, 64, 768], + [{ dim: 128, max_position_embeddings: 512 }, 128, 512] + ].map(async ([config, expectedDimensions, expectedLength]) => { + const hub = hubFor({ + tokenizer: { truncation: null }, + tokenizerConfig: { model_max_length: 1e30 }, + config, + sentenceConfig: undefined + }); + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + assert.equal(descriptor.dimensions, expectedDimensions); + assert.equal(descriptor.maxLength, expectedLength); + }) + ); + + const conflicting = hubFor({ + config: { hidden_size: 384, d_model: 768, max_position_embeddings: 512 } + }); + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: conflicting.client }), + /Conflicting embedding dimensions/ + ); + }); + + test('ignores generic tokenizer max_length when determining the model input window', async () => { + const hub = hubFor({ + tokenizer: { truncation: null }, + tokenizerConfig: { max_length: 32, model_max_length: 128 }, + config: { hidden_size: 384, max_position_embeddings: 512 }, + sentenceConfig: undefined + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.maxLength, 128); + }); + + test('follows a pinned base_model for Sentence Transformers semantics', async () => { + const hub = hubFor({ + tokenizer: { truncation: null }, + tokenizerConfig: { model_max_length: 128 }, + config: { hidden_size: 768, max_position_embeddings: 512 }, + modules: false, + baseModel: BASE_REPOSITORY + }); + hub.addBaseModel({ + sentenceConfig: { max_seq_length: 64 }, + pooling: 'cls', + normalize: false + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.maxLength, 64); + assert.equal(descriptor.output.pooling, 'cls'); + assert.equal(descriptor.output.normalize, false); + assert.ok(hub.modelInfos.includes(BASE_REPOSITORY)); + assert.ok( + hub.fileRequests.some( + ([repository, revision, remotePath]) => + repository === BASE_REPOSITORY && + revision === BASE_REVISION && + remotePath === 'modules.json' + ) + ); + }); + + test('includes external ONNX data files next to the selected model', async () => { + const hub = hubFor({ externalData: true }); + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.deepEqual( + descriptor.files.find(({ path }) => path === 'onnx/model.onnx_data'), + { + role: 'auxiliary', + name: 'model.onnx_data', + path: 'onnx/model.onnx_data', + size: hub.files[REPOSITORY]['onnx/model.onnx_data'].length, + sha256: digest(hub.files[REPOSITORY]['onnx/model.onnx_data']) + } + ); + }); + + test('rejects arbitrary sidecars when external ONNX data is declared', async () => { + const hub = hubFor({ externalData: 'weights.bin' }); + + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: hub.client }), + /declares external ONNX data but no data file exists/ + ); + }); + + test('rejects incompatible Hugging Face tasks before downloading artifacts', async () => { + await Promise.all( + [ + ['text-generation', 'openai-community/gpt2'], + ['fill-mask', 'FacebookAI/xlm-roberta-base'] + ].map(async ([task, repository]) => { + const hub = createHub({ + [repository]: { info: { sha: REVISION, task }, files: {} } + }); + await assert.rejects( + discoverModel(repository, { hubClient: hub.client }), + new RegExp(`declares task '${task}', not an embedding task`) + ); + assert.deepEqual(hub.fileLists, []); + }) + ); + }); + + test('rejects ambiguous ONNX exports and ambiguous Sentence Transformers semantics', async () => { + const ambiguousModel = hubFor({ + modelPath: 'exports/encoder.onnx', + assetDirectory: 'exports', + additionalOnnxPath: 'other/encoder.onnx' + }); + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: ambiguousModel.client }), + /ambiguous ONNX exports/ + ); + + const ambiguousPooling = hubFor({ pooling: ['mean', 'cls'] }); + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: ambiguousPooling.client }), + /Unsupported or ambiguous Sentence Transformers pooling/ + ); + + const invalidOrder = hubFor({ moduleOrder: ['Pooling', 'Transformer'] }); + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: invalidOrder.client }), + /unambiguous pooling pipeline/ + ); + + const unsupportedStage = hubFor({ moduleOrder: ['Transformer', 'Pooling', 'Dense'] }); + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: unsupportedStage.client }), + /Unsupported Sentence Transformers module 'sentence_transformers.models.Dense'/ + ); + }); + + test('downloads an artifact to derive integrity when file metadata has no checksum', async () => { + const hub = hubFor({ modelMetadata: false }); + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + const model = descriptor.files.find(({ role }) => role === 'model'); + const contents = hub.files[REPOSITORY]['onnx/model.onnx']; + assert.equal(model.size, contents.length); + assert.equal(model.sha256, digest(contents)); + }); + + test('should map sentence-transformers query and document prompts onto HANA text types', async () => { + const hub = hubFor({ + stConfig: { prompts: { query: 'query: ', document: 'document: ' }, default_prompt_name: null } + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.deepEqual(descriptor.prompts, { query: 'query: ', document: 'document: ' }); + assert.equal(descriptor.output.includePrompt, true); + }); + + test('should keep a query-only prompt without inventing a document prefix', async () => { + const hub = hubFor({ stConfig: { prompts: { query: 'query: ' } } }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.deepEqual(descriptor.prompts, { query: 'query: ' }); + }); + + test('should treat an empty document prompt as no prefix', async () => { + const hub = hubFor({ stConfig: { prompts: { query: 'query: ', document: '' } } }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.deepEqual(descriptor.prompts, { query: 'query: ' }); + }); + + test('should omit prompts when the model declares none', async () => { + const hub = hubFor({ stConfig: { max_seq_length: 256 } }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.prompts, undefined); + }); + + test('should reject models with prompts that exclude prompt tokens from pooling', async () => { + const hub = hubFor({ stConfig: { prompts: { query: 'query: ' } }, includePrompt: false }); + + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: hub.client }), + /include_prompt=false/ + ); + }); + + test('should preserve prompt-pooling behavior without discovered prompts', async () => { + const hub = hubFor({ includePrompt: false }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.equal(descriptor.output.includePrompt, false); + assert.equal(descriptor.prompts, undefined); + }); + + test('should reject invalid prompt-pooling metadata', async () => { + const hub = hubFor({ includePrompt: 'false' }); + + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: hub.client }), + /Invalid include_prompt/ + ); + }); + + test('should ignore unrelated prompt names and defaults', async () => { + const hub = hubFor({ + stConfig: { + prompts: { query: 'query: ', classification: 'Classify: ' }, + default_prompt_name: 'classification' + } + }); + + const descriptor = await discoverModel(REPOSITORY, { hubClient: hub.client }); + + assert.deepEqual(descriptor.prompts, { query: 'query: ' }); + }); + + test('should reject unimplemented modules even when their namespace suggests quantization', async () => { + await Promise.all( + ['sentence_transformers.quantization.Unknown', 'st_quantize.Unknown'].map(async (type) => { + const hub = hubFor({ moduleOrder: ['Transformer', 'Pooling', 'Normalize', type] }); + await assert.rejects( + discoverModel(REPOSITORY, { hubClient: hub.client }), + new RegExp(`Unsupported Sentence Transformers module '${type.replaceAll('.', '\\.')}'`) + ); + }) + ); + }); +}); + +function hubFor(options = {}) { + const modelPath = options.modelPath ?? 'onnx/model.onnx'; + const assetDirectory = options.assetDirectory ?? ''; + const asset = (name) => (assetDirectory ? `${assetDirectory}/${name}` : name); + const files = { + [modelPath]: Buffer.from('fake onnx model'), + [asset('tokenizer.json')]: json(options.tokenizer ?? { truncation: { max_length: 96 } }), + [asset('tokenizer_config.json')]: json(options.tokenizerConfig ?? { model_max_length: 128 }), + [asset('config.json')]: json( + options.config ?? { hidden_size: 384, max_position_embeddings: 512 } + ) + }; + if (options.rootAssets) { + files['tokenizer.json'] = json(options.rootAssets.tokenizer); + files['tokenizer_config.json'] = json(options.rootAssets.tokenizerConfig); + files['config.json'] = json(options.rootAssets.config); + } + if (options.externalData) { + const dataPath = + options.externalData === true + ? `${modelPath}_data` + : `${path.posix.dirname(modelPath)}/${options.externalData}`; + files[dataPath] = Buffer.from('external weights'); + const config = JSON.parse(files[asset('config.json')].toString()); + config['transformers.js_config'] = { use_external_data_format: { [modelPath]: 1 } }; + files[asset('config.json')] = json(config); + } + if (options.additionalOnnxPath) files[options.additionalOnnxPath] = Buffer.from('another model'); + + if (options.modules !== false) { + const moduleTypes = options.moduleOrder ?? [ + 'Transformer', + 'Pooling', + ...(options.normalize === false ? [] : ['Normalize']) + ]; + files['modules.json'] = json( + moduleTypes.map((type, index) => ({ + idx: index, + name: String(index), + path: type === 'Pooling' ? '1_Pooling' : '', + type: type.includes('.') ? type : `sentence_transformers.models.${type}` + })) + ); + files['1_Pooling/config.json'] = json( + poolingConfig(options.pooling ?? 'mean', options.includePrompt) + ); + if (options.sentenceConfig !== undefined) { + files['sentence_bert_config.json'] = json(options.sentenceConfig); + } else if (!Object.hasOwn(options, 'sentenceConfig')) { + files['sentence_bert_config.json'] = json({ max_seq_length: 256 }); + } + if (options.stConfig !== undefined) { + files['config_sentence_transformers.json'] = json(options.stConfig); + } + } + + const info = { + sha: REVISION, + ...(!options.omitTask ? { task: options.task ?? 'sentence-similarity' } : {}), + ...(options.baseModel ? { cardData: { base_model: options.baseModel } } : {}) + }; + const hub = createHub({ + [REPOSITORY]: { info, files, modelMetadata: options.modelMetadata } + }); + hub.addBaseModel = (baseOptions = {}) => { + const baseFiles = { + 'modules.json': json([ + { idx: 0, path: '', type: 'sentence_transformers.models.Transformer' }, + { idx: 1, path: '1_Pooling', type: 'sentence_transformers.models.Pooling' }, + ...(baseOptions.normalize + ? [{ idx: 2, path: '2_Normalize', type: 'sentence_transformers.models.Normalize' }] + : []) + ]), + '1_Pooling/config.json': json(poolingConfig(baseOptions.pooling ?? 'mean')), + 'sentence_bert_config.json': json(baseOptions.sentenceConfig ?? { max_seq_length: 128 }) + }; + hub.add(BASE_REPOSITORY, { info: { sha: BASE_REVISION }, files: baseFiles }); + }; + return hub; +} + +function createHub(repositories) { + const modelInfos = []; + const fileLists = []; + const fileRequests = []; + const client = { + async getModelInfo(repository) { + modelInfos.push(repository); + const entry = repositories[repository]; + if (!entry) throw new Error(`Unknown test repository ${repository}`); + return entry.info; + }, + async getFiles(repository, revision) { + fileLists.push([repository, revision]); + const entry = repositories[repository]; + if (!entry) throw new Error(`Unknown test repository ${repository}`); + return Object.entries(entry.files).map(([path, contents]) => ({ + path, + ...(path.endsWith('.onnx') && entry.modelMetadata !== false + ? { size: contents.length, lfs: { size: contents.length, sha256: digest(contents) } } + : {}) + })); + }, + async getFile(repository, revision, remotePath) { + fileRequests.push([repository, revision, remotePath]); + const contents = repositories[repository]?.files[remotePath]; + if (!contents) throw new Error(`Unknown test artifact ${repository}/${remotePath}`); + return contents; + } + }; + return { + client, + files: Object.fromEntries( + Object.entries(repositories).map(([name, entry]) => [name, entry.files]) + ), + modelInfos, + fileLists, + fileRequests, + add(repository, entry) { + repositories[repository] = entry; + this.files[repository] = entry.files; + } + }; +} + +function poolingConfig(pooling, includePrompt) { + const enabled = Array.isArray(pooling) ? pooling : [pooling]; + return { + pooling_mode_cls_token: enabled.includes('cls'), + pooling_mode_mean_tokens: enabled.includes('mean'), + pooling_mode_max_tokens: false, + pooling_mode_mean_sqrt_len_tokens: false, + pooling_mode_weightedmean_tokens: false, + pooling_mode_lasttoken: false, + ...(includePrompt !== undefined ? { include_prompt: includePrompt } : {}) + }; +} + +function json(value) { + return Buffer.from(JSON.stringify(value)); +} + +function digest(value) { + return createHash('sha256').update(value).digest('hex'); +} diff --git a/tests/model-provisioning.test.js b/tests/model-provisioning.test.js new file mode 100644 index 0000000..11ff3f1 --- /dev/null +++ b/tests/model-provisioning.test.js @@ -0,0 +1,793 @@ +import assert from 'node:assert/strict'; +import { createHash } from 'node:crypto'; +import fs from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import { afterEach, describe, test } from 'node:test'; +import { runModelCommand } from '../lib/vector_embedding/cli.js'; +import { + DEFAULT_EMBEDDING_MODEL, + resolveEmbeddingModel +} from '../lib/vector_embedding/embedding.js'; +import { + MODEL_LOCK_FILE, + MODEL_LOCK_VERSION, + getModelDirectory, + getModelRoot, + provisionModel, + readModelLock, + verifyModelDirectory +} from '../lib/vector_embedding/model-utils.js'; + +const temporaryDirectories = []; + +afterEach(async () => { + await Promise.all( + temporaryDirectories + .splice(0) + .map((directory) => fs.rm(directory, { recursive: true, force: true })) + ); +}); + +describe('runtime model configuration', () => { + test('uses the default model when no model is configured', async () => { + const content = Buffer.from('default configuration fixture'); + const model = fixtureModel(content, DEFAULT_EMBEDDING_MODEL); + const root = await createTemporaryDirectory(); + const modelDir = getModelDirectory(getModelRoot(undefined, root), model.repository); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + const resolved = await resolveEmbeddingModel(undefined, { root }); + + assert.equal(resolved.modelDir, modelDir); + assert.deepEqual(resolved.model, model); + }); + + test('uses the default model with a configured cache root', async () => { + const content = Buffer.from('default shared-cache fixture'); + const model = fixtureModel(content, DEFAULT_EMBEDDING_MODEL); + const directory = await createTemporaryDirectory(); + const modelDir = getModelDirectory(directory, model.repository); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + const resolved = await resolveEmbeddingModel({ directory }); + + assert.equal(resolved.modelDir, modelDir); + assert.deepEqual(resolved.model, model); + }); + + test('validates explicit model and directory while allowing additional properties', async () => { + const content = Buffer.from('configuration fixture'); + const model = fixtureModel(content); + await assert.rejects( + resolveEmbeddingModel({ model: '' }), + /cds\.env\.requires\.db\.embedding\.model must be a non-empty string/ + ); + await assert.rejects( + resolveEmbeddingModel({ model: 42 }), + /cds\.env\.requires\.db\.embedding\.model must be a non-empty string/ + ); + await assert.rejects( + resolveEmbeddingModel({ model: null }), + /cds\.env\.requires\.db\.embedding\.model must be a non-empty string/ + ); + await assert.rejects(resolveEmbeddingModel(model.repository), /embedding must be an object/); + await assert.rejects( + resolveEmbeddingModel({ model: model.repository, directory: '' }), + /embedding\.directory must be a non-empty string/ + ); + + const directory = await createTemporaryDirectory(); + const modelDir = getModelDirectory(directory, model.repository); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + const resolved = await resolveEmbeddingModel({ + model: model.repository, + directory, + revision: 'main', + extension: { enabled: true } + }); + + assert.equal(resolved.modelDir, modelDir); + assert.deepEqual(resolved.model, model); + }); + + test('merges configured prompts over discovered prompts per text type', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('prompt merge fixture'); + const model = { + ...fixtureModel(content), + prompts: { query: 'query: ', document: 'document: ' } + }; + const modelDir = getModelDirectory(directory, model.repository); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + const resolved = await resolveEmbeddingModel({ + model: model.repository, + directory, + prompts: { query: 'custom query: ' } + }); + + assert.deepEqual(resolved.model.prompts, { + query: 'custom query: ', + document: 'document: ' + }); + assert.deepEqual(await readModelLock(modelDir), model); + }); + + test('rejects configured prompts when prompt tokens are excluded from pooling', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('prompt exclusion fixture'); + const base = fixtureModel(content); + const model = { ...base, output: { ...base.output, includePrompt: false } }; + const modelDir = getModelDirectory(directory, model.repository); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + await assert.rejects( + resolveEmbeddingModel({ + model: model.repository, + directory, + prompts: { query: 'query: ' } + }), + /excludes prompt tokens from pooling/ + ); + }); +}); + +describe('explicit model provisioning', () => { + test('downloads, verifies, and locks a model idempotently', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('verified model fixture'); + const model = fixtureModel(content); + const requestedUrls = []; + const fetchImpl = createFetch(content, requestedUrls); + + await provisionModel(model, { directory, fetchImpl }); + await provisionModel(model, { directory, fetchImpl }); + + assert.deepEqual( + requestedUrls, + model.files.map( + (file) => `https://huggingface.co/example/model/resolve/${model.revision}/${file.path}` + ) + ); + assert.deepEqual(await readModelLock(directory), model); + assert.deepEqual(JSON.parse(await fs.readFile(path.join(directory, MODEL_LOCK_FILE), 'utf8')), { + ...model, + formatVersion: MODEL_LOCK_VERSION + }); + assert.deepEqual((await fs.readdir(directory)).sort(), [ + MODEL_LOCK_FILE, + 'model.onnx', + 'tokenizer.json', + 'tokenizer_config.json' + ]); + const modes = await Promise.all( + [MODEL_LOCK_FILE, ...model.files.map(({ name }) => name)].map(async (file) => + fs.stat(path.join(directory, file)).then(({ mode }) => mode & 0o777) + ) + ); + assert.deepEqual(modes, new Array(modes.length).fill(0o644)); + }); + + test('restores readable permissions on already valid artifacts', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('readable model fixture'); + const model = fixtureModel(content); + const modelPath = path.join(directory, model.files[0].name); + await provisionModel(model, { directory, fetchImpl: createFetch(content) }); + await fs.chmod(modelPath, 0o600); + + await provisionModel(model, { directory, fetchImpl: createFetch(content) }); + + assert.equal((await fs.stat(modelPath)).mode & 0o777, 0o644); + }); + + test('makes newly provisioned directories traversable across runtime users', async () => { + const parent = await createTemporaryDirectory(); + const modelsDirectory = path.join(parent, 'models'); + const directory = path.join(modelsDirectory, 'custom'); + const content = Buffer.from('directory mode fixture'); + const baseModel = fixtureModel(content); + const model = { + ...baseModel, + files: baseModel.files.map((file, index) => + index === 0 ? { ...file, name: 'onnx/model.onnx' } : file + ) + }; + const originalUmask = process.umask(0o077); + try { + await provisionModel(model, { directory, fetchImpl: createFetch(content) }); + } finally { + process.umask(originalUmask); + } + + assert.equal((await fs.stat(modelsDirectory)).mode & 0o777, 0o755); + assert.equal((await fs.stat(directory)).mode & 0o777, 0o755); + assert.equal((await fs.stat(path.join(directory, 'onnx'))).mode & 0o777, 0o755); + assert.equal((await fs.stat(path.join(directory, 'onnx/model.onnx'))).mode & 0o777, 0o644); + assert.equal((await fs.stat(path.join(directory, MODEL_LOCK_FILE))).mode & 0o777, 0o644); + }); + + test('does not repurpose a directory locked to a different descriptor', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('locked model fixture'); + const model = fixtureModel(content); + await provisionModel(model, { directory, fetchImpl: createFetch(content) }); + + await assert.rejects( + provisionModel({ ...model, repository: 'example/other-model' }, { directory }), + /locked to a different model descriptor/ + ); + await assert.rejects( + provisionModel({ ...model, dimensions: model.dimensions + 1 }, { directory }), + /locked to a different model descriptor/ + ); + await assert.rejects( + provisionModel( + { ...model, output: { ...model.output, includePrompt: false } }, + { directory } + ), + /locked to a different model descriptor/ + ); + await assert.rejects( + provisionModel({ ...model, prompts: { query: 'query: ' } }, { directory }), + /locked to a different model descriptor/ + ); + assert.deepEqual(await readModelLock(directory), model); + }); + + test('rejects legacy model locks that predate prompt semantics', async () => { + const directory = await createTemporaryDirectory(); + const model = fixtureModel(Buffer.from('legacy lock fixture')); + const { includePrompt, ...legacyOutput } = model.output; + void includePrompt; + await fs.writeFile( + path.join(directory, MODEL_LOCK_FILE), + JSON.stringify({ ...model, output: legacyOutput, formatVersion: 1 }) + ); + + await assert.rejects( + readModelLock(directory), + /version 1.*predates prompt semantics.*reinstall/ + ); + }); + + test('serializes provisioning attempts for the same directory', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('concurrent model fixture'); + const model = fixtureModel(content); + let releaseDownload; + let signalDownloadStarted; + const downloadStarted = new Promise((resolve) => { + signalDownloadStarted = resolve; + }); + const waitForRelease = new Promise((resolve) => { + releaseDownload = resolve; + }); + let firstRequest = true; + const fetchImpl = async () => { + if (firstRequest) { + firstRequest = false; + signalDownloadStarted(); + await waitForRelease; + } + return new Response(content); + }; + + const first = provisionModel(model, { directory, fetchImpl }); + await downloadStarted; + await assert.rejects( + provisionModel(model, { directory, fetchImpl }), + /already being provisioned/ + ); + releaseDownload(); + await first; + }); + + test('recovers a stale install lock owned by a terminated local process', async () => { + const parent = await createTemporaryDirectory(); + const directory = path.join(parent, 'model'); + const installLock = path.join(parent, '.model.embedding.install.lock'); + const content = Buffer.from('stale lock fixture'); + const model = fixtureModel(content); + await fs.writeFile( + installLock, + JSON.stringify({ + formatVersion: 1, + pid: 99999999, + hostname: os.hostname(), + createdAt: '2000-01-01T00:00:00.000Z', + token: 'stale-owner' + }) + ); + + await provisionModel(model, { directory, fetchImpl: createFetch(content) }); + + await assert.rejects(fs.access(installLock)); + assert.deepEqual(await readModelLock(directory), model); + }); + + test('recovers a stale install lock truncated by an interrupted write', async () => { + const parent = await createTemporaryDirectory(); + const directory = path.join(parent, 'model'); + const installLock = path.join(parent, '.model.embedding.install.lock'); + const content = Buffer.from('truncated lock fixture'); + const model = fixtureModel(content); + await fs.writeFile(installLock, '{'); + const staleTime = new Date('2000-01-01T00:00:00.000Z'); + await fs.utimes(installLock, staleTime, staleTime); + + await provisionModel(model, { directory, fetchImpl: createFetch(content) }); + + await assert.rejects(fs.access(installLock)); + assert.deepEqual(await readModelLock(directory), model); + }); + + test('rejects symlinked artifact path components', async () => { + const directory = await createTemporaryDirectory(); + const outside = await createTemporaryDirectory(); + const content = Buffer.from('symlink model fixture'); + const baseModel = fixtureModel(content); + const model = { + ...baseModel, + files: baseModel.files.map((file, index) => + index === 0 ? { ...file, name: 'nested/model.onnx' } : file + ) + }; + await fs.symlink(outside, path.join(directory, 'nested'), 'dir'); + + await assert.rejects( + provisionModel(model, { directory, fetchImpl: createFetch(content) }), + /must not contain symbolic links/ + ); + assert.deepEqual(await fs.readdir(outside), []); + }); + + test('rejects a symlinked or replaced model directory', async () => { + const parent = await createTemporaryDirectory(); + const directory = path.join(parent, 'model'); + const outside = await createTemporaryDirectory(); + const content = Buffer.from('root symlink fixture'); + const model = fixtureModel(content); + await fs.symlink(outside, directory, 'dir'); + + await assert.rejects( + provisionModel(model, { directory, fetchImpl: createFetch(content) }), + /model directory must not be a symbolic link/ + ); + assert.deepEqual(await fs.readdir(outside), []); + }); + + test('canonicalizes symlinked ancestor directories before provisioning', async () => { + const parent = await createTemporaryDirectory(); + const outside = await createTemporaryDirectory(); + const modelsDirectory = path.join(parent, 'models'); + const requestedDirectory = path.join(modelsDirectory, 'custom'); + const content = Buffer.from('ancestor symlink fixture'); + const model = fixtureModel(content); + await fs.symlink(outside, modelsDirectory, 'dir'); + + const directory = await provisionModel(model, { + directory: requestedDirectory, + fetchImpl: createFetch(content) + }); + + assert.equal(directory, path.join(await fs.realpath(outside), 'custom')); + assert.deepEqual(await readModelLock(directory), model); + }); + + test('publishes from staging and detects replacement of the target during download', async () => { + const parent = await createTemporaryDirectory(); + const directory = path.join(parent, 'model'); + const outside = await createTemporaryDirectory(); + const content = Buffer.from('target replacement fixture'); + const model = fixtureModel(content); + let replaced = false; + const fetchImpl = async () => { + if (!replaced) { + replaced = true; + await fs.symlink(outside, directory, 'dir'); + } + return new Response(content); + }; + + await assert.rejects( + provisionModel(model, { directory, fetchImpl }), + /model directory must not be a symbolic link/ + ); + assert.deepEqual(await fs.readdir(outside), []); + }); + + test('writes the lock only after every artifact passes verification', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('expected model fixture'); + const model = fixtureModel(content); + + await assert.rejects( + provisionModel(model, { + directory, + fetchImpl: async () => new Response(Buffer.from('invalid')) + }), + /Invalid size|Invalid SHA-256/ + ); + await assert.rejects(fs.access(path.join(directory, MODEL_LOCK_FILE))); + }); + + test('validates the runtime before publishing a newly installed model', async () => { + const parent = await createTemporaryDirectory(); + const directory = path.join(parent, 'model'); + const content = Buffer.from('runtime validation fixture'); + const model = fixtureModel(content); + let stagedDirectory; + + await assert.rejects( + provisionModel(model, { + directory, + fetchImpl: createFetch(content), + validate(candidate) { + stagedDirectory = candidate; + throw new Error('incompatible ONNX runtime'); + } + }), + /incompatible ONNX runtime/ + ); + + assert.notEqual(stagedDirectory, directory); + await assert.rejects(fs.access(directory)); + }); + + test('fails verification instead of downloading missing runtime files', async () => { + const directory = await createTemporaryDirectory(); + const model = fixtureModel(Buffer.from('fixture')); + + await assert.rejects( + verifyModelDirectory(directory, model), + new RegExp(`Embedding model is not provisioned.*${escapeRegExp(directory)}`, 's') + ); + assert.deepEqual(await fs.readdir(directory), []); + }); + + test('downloads a missing model into the project-local default directory and reuses it', async () => { + const root = await createTemporaryDirectory(); + const content = Buffer.from('lazy download fixture'); + const model = fixtureModel(content); + const requestedUrls = []; + const warnings = []; + let discoveries = 0; + const prompts = { query: 'query: ' }; + const options = { + root, + fetchImpl: createFetch(content, requestedUrls), + discover(name) { + discoveries++; + assert.equal(name, model.repository); + return model; + }, + validate: async () => {}, + warn: (message) => warnings.push(message) + }; + + const first = await resolveEmbeddingModel({ model: model.repository, prompts }, options); + const expectedDirectory = getModelDirectory(getModelRoot(undefined, root), model.repository); + + assert.deepEqual(first.model, { ...model, prompts }); + assert.equal(first.modelDir, expectedDirectory); + assert.equal(discoveries, 1); + assert.equal(warnings.length, 1); + assert.match(warnings[0], /Downloading it now; application startup may be delayed/); + assert.match(warnings[0], /repositories you trust/); + assert.equal(requestedUrls.length, model.files.length); + assert.deepEqual(await readModelLock(expectedDirectory), model); + + const second = await resolveEmbeddingModel({ model: model.repository, prompts }, options); + assert.deepEqual(second.model, { ...model, prompts }); + assert.equal(second.modelDir, expectedDirectory); + assert.equal(discoveries, 1); + assert.equal(warnings.length, 1); + assert.equal(requestedUrls.length, model.files.length); + }); + + test('waits for concurrent ad-hoc provisioning and reuses the completed download', async () => { + const root = await createTemporaryDirectory(); + const content = Buffer.from('concurrent lazy download fixture'); + const model = fixtureModel(content); + const requestedUrls = []; + const warnings = []; + let releaseDownload; + let signalDownloadStarted; + let firstRequest = true; + const downloadStarted = new Promise((resolve) => { + signalDownloadStarted = resolve; + }); + const waitForRelease = new Promise((resolve) => { + releaseDownload = resolve; + }); + const fetchImpl = async (url) => { + requestedUrls.push(url); + if (firstRequest) { + firstRequest = false; + signalDownloadStarted(); + await waitForRelease; + } + return new Response(content, { + headers: { 'content-length': String(content.length) } + }); + }; + const options = { + root, + fetchImpl, + discover: () => model, + validate: async () => {}, + warn: (message) => warnings.push(message), + provisionRetryMs: 5, + provisionTimeoutMs: 1000 + }; + + const first = resolveEmbeddingModel({ model: model.repository }, options); + await downloadStarted; + const second = resolveEmbeddingModel({ model: model.repository }, options); + await new Promise((resolve) => setTimeout(resolve, 20)); + releaseDownload(); + + const resolved = await Promise.all([first, second]); + assert.equal(resolved[0].modelDir, resolved[1].modelDir); + assert.equal(warnings.length, 2); + assert.equal(requestedUrls.length, model.files.length); + }); + + test('keeps explicitly configured directories offline', async () => { + const root = await createTemporaryDirectory(); + let fetched = false; + await assert.rejects( + resolveEmbeddingModel( + { + model: 'example/model', + directory: './models/minilm' + }, + { + root, + fetchImpl: () => { + fetched = true; + throw new Error('explicit directories must not fetch'); + } + } + ), + /@cap-js\/ai install-model example\/model --directory \.\/models/ + ); + assert.equal(fetched, false); + }); + + test('resolves relative directories from cds.root and preserves absolute directories', async () => { + const root = await createTemporaryDirectory(); + const content = Buffer.from('directory resolution fixture'); + const model = fixtureModel(content); + const modelRoot = path.join(root, 'models'); + const modelDir = getModelDirectory(modelRoot, model.repository); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + const relative = await resolveEmbeddingModel( + { model: model.repository, directory: './models' }, + { root } + ); + const absolute = await resolveEmbeddingModel( + { model: model.repository, directory: modelRoot }, + { root: await createTemporaryDirectory() } + ); + + assert.equal(relative.modelDir, modelDir); + assert.equal(absolute.modelDir, modelDir); + }); + + test('rejects a configured model name that does not match the provisioned lock', async () => { + const modelRoot = await createTemporaryDirectory(); + const content = Buffer.from('repository mismatch fixture'); + const model = fixtureModel(content); + const modelDir = getModelDirectory(modelRoot, 'example/other-model'); + await provisionModel(model, { directory: modelDir, fetchImpl: createFetch(content) }); + + await assert.rejects( + resolveEmbeddingModel({ model: 'example/other-model', directory: modelRoot }), + /contains example\/model, not example\/other-model/ + ); + }); + + test('gives models a model-name provisioning command', async () => { + const root = await createTemporaryDirectory(); + + await assert.rejects( + resolveEmbeddingModel({ model: 'example/custom', directory: './models/custom' }, { root }), + /@cap-js\/ai install-model example\/custom --directory \.\/models\/custom/ + ); + }); + + test('requires explicit lock recovery before reinstalling', async () => { + const modelRoot = await createTemporaryDirectory(); + const modelDir = getModelDirectory(modelRoot, 'example/model'); + await fs.mkdir(modelDir, { recursive: true }); + await fs.writeFile(path.join(modelDir, MODEL_LOCK_FILE), '{}'); + + await assert.rejects( + resolveEmbeddingModel({ model: 'example/model', directory: modelRoot }), + /Remove or replace the invalid lock explicitly, then run 'npx @cap-js\/ai install-model/ + ); + }); + + test('installs a model by name through the command API', async () => { + const root = await createTemporaryDirectory(); + const modelRoot = path.join(root, 'models'); + const content = Buffer.from('command fixture'); + const model = fixtureModel(content); + const output = []; + + await runModelCommand(['install-model', model.repository, '--directory', modelRoot], { + root, + discover: () => model, + fetchImpl: createFetch(content), + validate: async () => {}, + stdout: { write: (value) => output.push(value) } + }); + + const modelDir = getModelDirectory(modelRoot, model.repository); + assert.deepEqual(await readModelLock(modelDir), model); + assert.match(output.join(''), /Installed example\/model/); + assert.match(output.join(''), new RegExp(escapeRegExp(modelDir))); + }); + + test('installs into the project-local model cache when no directory is provided', async () => { + const root = await createTemporaryDirectory(); + const content = Buffer.from('default command fixture'); + const model = fixtureModel(content); + + await runModelCommand(['install-model', model.repository], { + root, + discover: () => model, + fetchImpl: createFetch(content), + validate: async () => {}, + stdout: { write() {} } + }); + + const modelDir = path.join(root, '.cds', 'models', 'example', 'model'); + assert.deepEqual(await readModelLock(modelDir), model); + }); + + test('resolves default and relative command directories from the CAP project root', async () => { + const root = await createTemporaryDirectory(); + const subdirectory = path.join(root, 'srv', 'nested'); + const content = Buffer.from('project root command fixture'); + const model = fixtureModel(content); + const options = { + cwd: subdirectory, + discover: () => model, + fetchImpl: createFetch(content), + validate: async () => {}, + stdout: { write() {} } + }; + await fs.mkdir(subdirectory, { recursive: true }); + await fs.writeFile( + path.join(root, 'package.json'), + JSON.stringify({ dependencies: { '@sap/cds': '^9' } }) + ); + + await runModelCommand(['install-model', model.repository], options); + assert.deepEqual( + await readModelLock(path.join(root, '.cds', 'models', 'example', 'model')), + model + ); + + await runModelCommand( + ['install-model', model.repository, '--directory', './shared-models'], + options + ); + assert.deepEqual( + await readModelLock(path.join(root, 'shared-models', 'example', 'model')), + model + ); + }); + + test('checks a model by name without provisioning it', async () => { + const root = await createTemporaryDirectory(); + const output = []; + const checked = { + repository: 'example/model', + revision: '1'.repeat(40), + task: 'sentence-similarity', + dimensions: 384, + maxLength: 128, + files: [ + { role: 'model', name: 'model.onnx', path: 'onnx/model.onnx' }, + { role: 'tokenizer', name: 'tokenizer.json', path: 'tokenizer.json' } + ], + output: { + name: 'last_hidden_state', + pooling: 'mean', + normalize: true, + includePrompt: true + }, + prompts: { query: 'query: ', document: 'document: ' } + }; + + await runModelCommand(['check-model', checked.repository], { + cwd: root, + check: async (repository) => { + assert.equal(repository, checked.repository); + return checked; + }, + stdout: { write: (value) => output.push(value) } + }); + + assert.match(output.join(''), /Likely compatible/i); + assert.match(output.join(''), /Prompt tokens in pooling: included/); + assert.match(output.join(''), /QUERY: "query: "/); + assert.match(output.join(''), /DOCUMENT: "document: "/); + assert.match(output.join(''), /install-model.*definitive/i); + await assert.rejects(fs.access(path.join(root, '.cds', 'models')), /ENOENT/); + }); + + test('requires a model name and accepts an optional cache root', async () => { + await assert.rejects( + runModelCommand(['install-model', '--directory', './models/custom']), + /Specify a model name/ + ); + await assert.rejects( + runModelCommand(['install-model', 'example/model', 'example/other']), + /Unexpected argument 'example\/other'/ + ); + await assert.rejects( + runModelCommand(['check-model', 'example/model', '--directory', './models']), + /Unknown option '--directory'/ + ); + }); +}); + +function createFetch(content, requestedUrls = []) { + return async (url) => { + requestedUrls.push(url); + return new Response(content, { + headers: { 'content-length': String(content.length) } + }); + }; +} + +async function createTemporaryDirectory() { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'cap-ai-provision-')); + temporaryDirectories.push(directory); + return directory; +} + +function fixtureModel(content, repository = 'example/model') { + const sha256 = createHash('sha256').update(content).digest('hex'); + return { + repository, + revision: 'deadbeefdeadbeefdeadbeefdeadbeefdeadbeef', + dimensions: 2, + maxLength: 8, + files: [ + { role: 'model', name: 'model.onnx', path: 'onnx/model.onnx', size: content.length, sha256 }, + { + role: 'tokenizer', + name: 'tokenizer.json', + path: 'tokenizer.json', + size: content.length, + sha256 + }, + { + role: 'tokenizerConfig', + name: 'tokenizer_config.json', + path: 'tokenizer_config.json', + size: content.length, + sha256 + } + ], + output: { + name: 'last_hidden_state', + pooling: 'mean', + normalize: true, + includePrompt: true + } + }; +} + +function escapeRegExp(value) { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} diff --git a/tests/provision-model.js b/tests/provision-model.js new file mode 100644 index 0000000..de60b9f --- /dev/null +++ b/tests/provision-model.js @@ -0,0 +1,22 @@ +import fs from 'node:fs/promises'; + +import { validateEmbeddingModel } from '../lib/vector_embedding/embedding.js'; +import { + MODEL_LOCK_VERSION, + getModelDirectory, + getModelRoot, + provisionModel +} from '../lib/vector_embedding/model-utils.js'; + +const lockUrl = new URL( + './fixtures/sentence-transformers/all-MiniLM-L6-v2/embedding.lock.json', + import.meta.url +); +const { formatVersion, ...model } = JSON.parse(await fs.readFile(lockUrl, 'utf8')); +if (formatVersion !== MODEL_LOCK_VERSION) { + throw new Error(`Unsupported test model lock version ${formatVersion}`); +} + +const modelDir = getModelDirectory(getModelRoot(undefined, process.cwd()), model.repository); +await provisionModel(model, { directory: modelDir, validate: validateEmbeddingModel }); +console.log(`Installed ${model.repository} in ${modelDir}`); diff --git a/tests/recommendations.test.js b/tests/recommendations.test.js index be142ed..7e52d58 100644 --- a/tests/recommendations.test.js +++ b/tests/recommendations.test.js @@ -233,3 +233,22 @@ describe('Row-level authorization', () => { ); }); }); + +describe('Local vector embeddings', () => { + test('Bookshop exposes an embedding preview', async (t) => { + if (cds.env.requires.db.impl !== '@cap-js/ai/lib/sqlite/AISQLiteService.js') { + t.skip('local SQLite embedding sample'); + return; + } + + const { status, data } = await GET( + "/odata/v4/catalog/embedding(text='A%20book%20about%20travel')" + ); + const embedding = JSON.parse(data.value); + + assert.strictEqual(status, 200); + assert.ok(Array.isArray(embedding)); + assert.ok(embedding.length > 0); + assert.ok(embedding.every((value) => typeof value === 'number')); + }); +}); diff --git a/tests/vector-unit.test.js b/tests/vector-unit.test.js new file mode 100644 index 0000000..9a584fe --- /dev/null +++ b/tests/vector-unit.test.js @@ -0,0 +1,451 @@ +import assert from 'node:assert/strict'; +import { createHash } from 'node:crypto'; +import fs from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import { afterEach, describe, test } from 'node:test'; +import { + createFeeds, + createTokenizerState, + poolOutput, + processEmbedding, + tokenizeToWindow, + validateSession +} from '../lib/vector_embedding/embedding.js'; +import { + downloadFile, + downloadModelIfNeeded, + getModelDirectory, + getModelRoot, + loadTokenizerPackage, + validateModelDescriptor +} from '../lib/vector_embedding/model-utils.js'; +import { loadOnnxRuntime } from '../lib/vector_embedding/load-onnx-runtime.js'; +import { loadSQLiteService } from '../lib/sqlite/load-sqlite.js'; + +const temporaryDirectories = []; + +test('explains how to install the optional tokenizer peer dependency', async () => { + const missing = Object.assign( + new Error("Cannot find package '@huggingface/tokenizers' imported from model-utils.js"), + { code: 'ERR_MODULE_NOT_FOUND' } + ); + + await assert.rejects( + loadTokenizerPackage(async () => { + throw missing; + }), + /npm add -D @huggingface\/tokenizers@0\.1\.3/ + ); +}); + +test('explains how to install the optional SQLite peer dependency', () => { + const missing = Object.assign( + new Error("Cannot find module '@cap-js/sqlite' required by load-sqlite.js"), + { code: 'MODULE_NOT_FOUND' } + ); + + assert.throws( + () => + loadSQLiteService(() => { + throw missing; + }), + /npm add -D @cap-js\/sqlite/ + ); +}); + +test('explains how to install the pinned ONNX Runtime peer dependency', () => { + const missing = Object.assign(new Error("Cannot find module 'onnxruntime-node/package.json'"), { + code: 'MODULE_NOT_FOUND' + }); + + assert.throws( + () => + loadOnnxRuntime(() => { + throw missing; + }), + /npm add -D onnxruntime-node@1\.20\.1/ + ); +}); + +test('does not mask unrelated optional-peer loading errors', () => { + const sqliteError = Object.assign(new Error('SQLite native binding failed'), { + code: 'ERR_DLOPEN_FAILED' + }); + const runtimeError = Object.assign(new Error('ONNX native binding failed'), { + code: 'ERR_DLOPEN_FAILED' + }); + + assert.throws( + () => + loadSQLiteService(() => { + throw sqliteError; + }), + sqliteError + ); + assert.throws( + () => + loadOnnxRuntime((specifier) => { + if (specifier.endsWith('package.json')) return { version: '1.20.1' }; + throw runtimeError; + }), + runtimeError + ); +}); + +afterEach(async () => { + await Promise.all( + temporaryDirectories + .splice(0) + .map((directory) => fs.rm(directory, { recursive: true, force: true })) + ); +}); + +describe('tokenizer input window', () => { + const tokenizer = { + encode(text, { add_special_tokens: addSpecialTokens }) { + const ids = text + .split(/\s+/u) + .filter(Boolean) + .map((_, index) => index + 10); + const attention_mask = ids.map((_, index) => index % 2); + const token_type_ids = ids.map((_, index) => index + 20); + return addSpecialTokens + ? { + ids: [101, ...ids, 102], + attention_mask: [1, ...attention_mask, 1], + token_type_ids: [9, ...token_type_ids, 10] + } + : { ids, attention_mask, token_type_ids }; + } + }; + + test('derives special-token boundaries and keeps one complete model window', () => { + const state = createTokenizerState(tokenizer, 5); + const input = tokenizeToWindow('one two three four five six seven', tokenizer, state); + + assert.deepEqual(input.ids, [101, 10, 11, 12, 102]); + assert.deepEqual(input.attention_mask, [1, 0, 1, 0, 1]); + assert.deepEqual(input.token_type_ids, [9, 20, 21, 22, 10]); + }); + + test('truncates content without relying on tokenizer-side truncation', () => { + const state = createTokenizerState(tokenizer, 4); + const input = tokenizeToWindow(new Array(9).fill('token').join(' '), tokenizer, state); + + assert.deepEqual(input.ids, [101, 10, 11, 102]); + }); +}); + +describe('model compatibility', () => { + test('filters standard int64 feeds by the model input names', () => { + const feeds = createFeeds({ + ids: [101, 200, 102], + attention_mask: [1, 0, 1], + token_type_ids: [0, 1, 1] + }); + + assert.deepEqual(Object.keys(feeds), ['input_ids', 'attention_mask', 'token_type_ids']); + assert.deepEqual(feeds.input_ids.dims, [1, 3]); + assert.deepEqual(Array.from(feeds.input_ids.data), [101n, 200n, 102n]); + assert.deepEqual(Array.from(feeds.attention_mask.data), [1n, 0n, 1n]); + assert.deepEqual(Array.from(feeds.token_type_ids.data), [0n, 1n, 1n]); + + const filtered = createFeeds( + { + ids: [101], + attention_mask: [1], + token_type_ids: [0] + }, + ['input_ids', 'attention_mask'] + ); + assert.deepEqual(Object.keys(filtered), ['input_ids', 'attention_mask']); + }); + + test('supports mean, CLS, and already-pooled outputs', () => { + const sequence = { + type: 'float32', + data: new Float32Array([1, 2, 3, 4]), + dims: [1, 2, 2] + }; + const pooled = { type: 'float64', data: new Float64Array([5, 6]), dims: [1, 2] }; + + assert.deepEqual(Array.from(poolOutput(sequence, 'mean')), [2, 3]); + assert.deepEqual(Array.from(poolOutput(sequence, 'cls')), [1, 2]); + assert.deepEqual(Array.from(poolOutput(pooled, 'none')), [5, 6]); + }); + + test('rejects non-floating-point model outputs', () => { + assert.throws( + () => poolOutput({ type: 'int64', data: new BigInt64Array([1n]), dims: [1] }, 'none'), + /must be float32 or float64/ + ); + }); + + test('rejects sessions without the required input', () => { + assert.throws( + () => + validateSession( + { inputNames: ['attention_mask'], outputNames: ['last_hidden_state'] }, + { output: { name: 'last_hidden_state' } } + ), + /must expose the standard int64 input 'input_ids'/ + ); + }); + + test('rejects unsupported model inputs', () => { + assert.throws( + () => + validateSession( + { inputNames: ['input_ids', 'position_ids'], outputNames: ['last_hidden_state'] }, + { output: { name: 'last_hidden_state' } } + ), + /unsupported inputs: position_ids/ + ); + }); + + test('rejects a missing configured output', () => { + assert.throws( + () => + validateSession( + { inputNames: ['input_ids'], outputNames: ['logits'] }, + { output: { name: 'last_hidden_state' } } + ), + /output 'last_hidden_state' not found\. Available outputs: logits/ + ); + }); + + test('rejects runtime output dimensions that differ from the descriptor', () => { + const session = { + inputNames: ['input_ids'], + run() { + return { + last_hidden_state: { + type: 'float32', + data: new Float32Array([1, 2]), + dims: [1, 1, 2] + } + }; + } + }; + const model = { + dimensions: 3, + output: { name: 'last_hidden_state', pooling: 'mean', normalize: false } + }; + + assert.throws( + () => + processEmbedding({ ids: [101], attention_mask: [1], token_type_ids: [0] }, session, model), + /produced 2 dimensions; configured 3/ + ); + }); + + test('requires immutable revisions, checksums, and traversal-safe paths', () => { + const model = fixtureModel(Buffer.from('fixture')); + assert.equal(validateModelDescriptor(model), model); + + assert.throws( + () => validateModelDescriptor({ ...model, revision: 'main' }), + /immutable 40-64 character commit hash/ + ); + assert.throws( + () => + validateModelDescriptor({ + ...model, + files: model.files.map((file, index) => + index === 0 ? { ...file, name: '../model.onnx' } : file + ) + }), + /safe relative path/ + ); + assert.throws( + () => + validateModelDescriptor({ + ...model, + files: model.files.map((file, index) => + index === 0 ? { ...file, name: 'embedding.lock.json' } : file + ) + }), + /conflicts with provisioning metadata/ + ); + assert.throws( + () => + validateModelDescriptor({ + ...model, + files: model.files.map((file, index) => + index === 0 ? { ...file, name: 'EMBEDDING.LOCK.JSON' } : file + ) + }), + /conflicts with provisioning metadata/ + ); + assert.throws( + () => + validateModelDescriptor({ + ...model, + files: model.files.map((file, index) => { + if (index === 0) return { ...file, name: 'nested' }; + if (index === 1) return { ...file, name: 'nested/tokenizer.json' }; + return file; + }) + }), + /conflicts with another embedding file/ + ); + assert.throws( + () => + validateModelDescriptor({ + ...model, + output: { ...model.output, includePrompt: false }, + prompts: { query: 'query: ' } + }), + /prompts require embedding\.output\.includePrompt to be true/ + ); + }); +}); + +describe('model directories', () => { + test('uses a project-local default and appends the model repository', () => { + const project = path.join(path.sep, 'project'); + const root = getModelRoot(undefined, project); + + assert.equal(root, path.join(project, '.cds', 'models')); + assert.equal(getModelDirectory(root, 'foo/bar'), path.join(root, 'foo', 'bar')); + }); + + test('resolves relative, absolute, and home-relative roots', () => { + const project = path.join(path.sep, 'project'); + const home = path.join(path.sep, 'home', 'user'); + + assert.equal(getModelRoot('./models', project, home), path.join(project, 'models')); + assert.equal( + getModelRoot(path.join(path.sep, 'shared', 'models'), project, home), + path.join(path.sep, 'shared', 'models') + ); + assert.equal(getModelRoot('~/.cds/models', project, home), path.join(home, '.cds', 'models')); + }); +}); + +describe('model download', () => { + test('uses a pinned revision and atomically caches verified files', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('verified model fixture'); + const model = fixtureModel(content); + const requestedUrls = []; + const fetchImpl = async (url) => { + requestedUrls.push(url); + const body = new ReadableStream({ + start(controller) { + controller.enqueue(content.subarray(0, 5)); + controller.enqueue(content.subarray(5)); + controller.close(); + } + }); + return new Response(body, { + headers: { 'content-length': String(content.length) } + }); + }; + + await downloadModelIfNeeded(directory, model, { fetchImpl }); + await downloadModelIfNeeded(directory, model, { fetchImpl }); + + assert.deepEqual( + requestedUrls, + model.files.map( + (file) => `https://huggingface.co/example/model/resolve/${model.revision}/${file.path}` + ) + ); + assert.deepEqual(await fs.readFile(path.join(directory, 'model.onnx')), content); + assert.deepEqual((await fs.readdir(directory)).sort(), [ + 'model.onnx', + 'tokenizer.json', + 'tokenizer_config.json' + ]); + }); + + test('honors a custom Hub URL for model downloads', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('custom Hub model fixture'); + const model = fixtureModel(content); + const requests = []; + const fetchImpl = async (url, options) => { + requests.push([url, options]); + return new Response(content); + }; + + await downloadModelIfNeeded(directory, model, { + fetchImpl, + hubUrl: 'https://hub.example.test///' + }); + + assert.ok( + requests.every(([url]) => url.startsWith('https://hub.example.test/example/model/resolve/')) + ); + assert.ok(requests.every(([, options]) => !('headers' in options))); + }); + + test('rejects oversized content without exposing a partial cache file', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('expected'); + const file = fixtureModel(content).files[0]; + const outputPath = path.join(directory, file.name); + const fetchImpl = async () => new Response(Buffer.concat([content, Buffer.from('extra')])); + + await assert.rejects( + downloadFile('https://example.test/model', outputPath, file, { fetchImpl }), + /exceeds the expected 8 bytes/ + ); + assert.deepEqual(await fs.readdir(directory), []); + }); + + test('rejects content that does not match the pinned checksum', async () => { + const directory = await createTemporaryDirectory(); + const content = Buffer.from('expected'); + const file = fixtureModel(content).files[0]; + const outputPath = path.join(directory, file.name); + const fetchImpl = async () => new Response(Buffer.from('tampered')); + + await assert.rejects( + downloadFile('https://example.test/model', outputPath, file, { fetchImpl }), + /Invalid SHA-256/ + ); + assert.deepEqual(await fs.readdir(directory), []); + }); +}); + +async function createTemporaryDirectory() { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'cap-ai-model-')); + temporaryDirectories.push(directory); + return directory; +} + +function fixtureModel(content) { + const sha256 = createHash('sha256').update(content).digest('hex'); + return { + repository: 'example/model', + revision: 'deadbeefdeadbeefdeadbeefdeadbeefdeadbeef', + dimensions: 2, + maxLength: 8, + files: [ + { role: 'model', name: 'model.onnx', path: 'onnx/model.onnx', size: content.length, sha256 }, + { + role: 'tokenizer', + name: 'tokenizer.json', + path: 'tokenizer.json', + size: content.length, + sha256 + }, + { + role: 'tokenizerConfig', + name: 'tokenizer_config.json', + path: 'tokenizer_config.json', + size: content.length, + sha256 + } + ], + output: { + name: 'last_hidden_state', + pooling: 'mean', + normalize: true, + includePrompt: true + } + }; +} diff --git a/tests/vector.test.js b/tests/vector.test.js new file mode 100644 index 0000000..5257716 --- /dev/null +++ b/tests/vector.test.js @@ -0,0 +1,293 @@ +import { after, before, describe, test } from 'node:test'; +import assert from 'node:assert'; +import cds from '@sap/cds'; +import { + DEFAULT_EMBEDDING_MODEL, + createEmbeddingRuntime, + createEmbeddingRuntimeFromModel, + resolveEmbeddingModel +} from '../lib/vector_embedding/embedding.js'; + +const MINILM_MODEL = 'sentence-transformers/all-MiniLM-L6-v2'; +const AI_SQLITE_IMPL = '@cap-js/ai/lib/sqlite/AISQLiteService.js'; + +let runtime; + +before(async () => { + runtime = await createEmbeddingRuntime(); +}); + +after(async () => { + await runtime?.dispose(); +}); + +describe('Vector embedding function (standalone)', () => { + describe('vector_embedding', () => { + test('computes embedding with ONNX model', async () => { + const result = runtime.vectorEmbedding('Hello world'); + + const embedding = JSON.parse(result); + assert.ok(Array.isArray(embedding), 'Embedding should be an array'); + + // Check that values are floats in reasonable range + embedding.forEach((val, idx) => { + assert.strictEqual(typeof val, 'number', `Value at index ${idx} should be a number`); + assert.ok(Math.abs(val) <= 1, `Value at index ${idx} should be normalized (-1 to 1)`); + }); + }); + + test('deterministic - same input produces same output', async () => { + const e1 = runtime.vectorEmbedding('test text'); + const e2 = runtime.vectorEmbedding('test text'); + + assert.strictEqual(e1, e2, 'Same input should produce identical embeddings'); + }); + + test('ignores text beyond the first model input window', () => { + const firstWindow = new Array(126).fill('token').join(' '); + const truncated = runtime.vectorEmbedding(firstWindow); + const withAdditionalText = runtime.vectorEmbedding( + `${firstWindow} this text must not affect the embedding` + ); + + assert.strictEqual(withAdditionalText, truncated); + }); + + test('different inputs produce different outputs', async () => { + const e1 = runtime.vectorEmbedding('hello world'); + const e2 = runtime.vectorEmbedding('goodbye world'); + + assert.notStrictEqual(e1, e2, 'Different inputs should produce different embeddings'); + }); + + test('semantically similar sentences produce similar vectors', async () => { + const e1 = runtime.vectorEmbedding('I love programming'); + const e2 = runtime.vectorEmbedding('I enjoy coding'); + + const v1 = JSON.parse(e1); + const v2 = JSON.parse(e2); + + const similarity = cosineSimilarity(v1, v2); + assert.ok( + similarity > 0.8, + `Semantically similar sentences should have high cosine similarity (got ${similarity.toFixed(3)})` + ); + }); + + test('semantically different sentences are far apart in vector space', async () => { + const e1 = runtime.vectorEmbedding('The cat sat on the mat'); + const e2 = runtime.vectorEmbedding('Quantum physics is fascinating'); + + const v1 = JSON.parse(e1); + const v2 = JSON.parse(e2); + + const similarity = cosineSimilarity(v1, v2); + assert.ok( + similarity < 0.1, + `Semantically different sentences should have low cosine similarity (got ${similarity.toFixed(3)})` + ); + }); + + test('handles empty text', async () => { + const result = runtime.vectorEmbedding(''); + + const embedding = JSON.parse(result); + assert.ok(Array.isArray(embedding), 'Empty text should return zero vector'); + assert.strictEqual(embedding.length, 384, 'Should have 384 dimensions'); + assert.ok( + embedding.every((v) => v === 0), + 'Empty text should return all zeros' + ); + }); + + test('handles null text', async () => { + const result = runtime.vectorEmbedding(null); + + const embedding = JSON.parse(result); + assert.ok(Array.isArray(embedding), 'Null text should return zero vector'); + assert.strictEqual(embedding.length, 384, 'Should have 384 dimensions'); + assert.ok( + embedding.every((v) => v === 0), + 'Null text should return all zeros' + ); + }); + + test('embeds text longer than the MiniLM token limit', () => { + const result = runtime.vectorEmbedding(new Array(300).fill('semantic').join(' ')); + + assert.strictEqual(JSON.parse(result).length, 384); + }); + + test('uses the configured dimensions for compatibility model identifiers', async () => { + const result1 = runtime.vectorEmbedding('test'); + const embedding1 = JSON.parse(result1); + assert.strictEqual(embedding1.length, 384, 'SAP_GXY.20250407 should have 384 dimensions'); + + const result2 = runtime.vectorEmbedding('test'); + const embedding2 = JSON.parse(result2); + assert.strictEqual(embedding2.length, 384, 'SAP_GXY.20240715 should have 384 dimensions'); + + const result3 = runtime.vectorEmbedding('test'); + const embedding3 = JSON.parse(result3); + assert.strictEqual( + embedding3.length, + 384, + 'Compatibility identifiers should use the configured model dimensions' + ); + }); + + test('disposes embedding runtimes safely', async () => { + const runtime = await createEmbeddingRuntime({ model: MINILM_MODEL }); + + await runtime.dispose(); + await runtime.dispose(); + + assert.throws(() => runtime.embedding('test'), /Inference session has been disposed/); + }); + }); +}); + +describe('text-type prompts via configured prompts', () => { + let promptRuntime; + + // MiniLM ships no Sentence-Transformers prompts, so there is nothing to discover. + // These prefixes come from `embedding.prompts.{query,document}`, the user-configured override that + // takes precedence over discovered prompts — the path a prompt-trained model without + // discoverable prompts relies on. + const PROMPTS = { query: 'query: ', document: 'document: ' }; + + before(async () => { + const { model, modelDir } = await resolveEmbeddingModel({ + model: MINILM_MODEL, + prompts: PROMPTS + }); + promptRuntime = await createEmbeddingRuntimeFromModel(modelDir, model); + }); + + after(async () => { + await promptRuntime?.dispose(); + }); + + test('should prepend the configured prompt for the forwarded text-type', () => { + assert.strictEqual( + promptRuntime.vectorEmbedding('a small cat', 'QUERY'), + runtime.vectorEmbedding('query: a small cat') + ); + }); + + test('should prepend the configured document prefix for the DOCUMENT text type', () => { + assert.strictEqual( + promptRuntime.vectorEmbedding('a small cat', 'DOCUMENT'), + runtime.vectorEmbedding('document: a small cat') + ); + }); + + test('should ignore the text type when no prefix is configured', () => { + assert.strictEqual( + runtime.vectorEmbedding('a small cat', 'QUERY'), + runtime.vectorEmbedding('a small cat', 'DOCUMENT') + ); + assert.strictEqual( + runtime.vectorEmbedding('a small cat', 'QUERY'), + runtime.vectorEmbedding('a small cat') + ); + }); + + test('should reject a non-string configured prompt before touching the model', async () => { + await assert.rejects( + resolveEmbeddingModel({ model: MINILM_MODEL, prompts: { query: 42 } }), + /embedding\.prompts\.query must be a non-empty string/ + ); + }); +}); + +describe('SQLite integration', () => { + let db; + + test('uses the default embedding model', () => { + assert.strictEqual(DEFAULT_EMBEDDING_MODEL, MINILM_MODEL); + const kind = cds.env.requires.kinds.sqlite; + assert.strictEqual(kind.impl, AI_SQLITE_IMPL); + assert.strictEqual(kind.embedding.model, DEFAULT_EMBEDDING_MODEL); + }); + + const memoryKind = cds.env.requires.kinds['sqlite:memory']; + test( + 'inherits the AI-enabled sqlite kind for sqlite:memory', + { + skip: memoryKind?.kind !== 'sqlite' + }, + () => { + assert.strictEqual(memoryKind.impl, AI_SQLITE_IMPL); + assert.strictEqual(memoryKind.embedding.model, DEFAULT_EMBEDDING_MODEL); + assert.strictEqual(memoryKind.credentials.url, ':memory:'); + assert.strictEqual(memoryKind.pool.evictionRunIntervalMillis, 0); + assert.strictEqual(memoryKind.pool.min, 1); + assert.strictEqual(memoryKind.pool.max, 1); + } + ); + + before(async () => { + db = await cds.connect.to( + 'vector-db', + memoryKind?.kind === 'sqlite' + ? { kind: 'sqlite:memory' } + : { kind: 'sqlite', credentials: { url: ':memory:' } } + ); + }); + + after(async () => { + await db?.disconnect(); + }); + + test('registers VECTOR_EMBEDDING for three and four arguments', async () => { + const [row] = await db.run(`SELECT + VECTOR_EMBEDDING('Hello world', 'DOCUMENT', 'SAP_GXY.20250407') AS local, + VECTOR_EMBEDDING('Hello world', 'DOCUMENT', 'SAP_GXY.20250407', 'remote') AS remote`); + + assert.strictEqual(JSON.parse(row.local).length, 384); + assert.strictEqual(row.remote, row.local); + }); + + test('preserves SQL null semantics', async () => { + const [row] = await db.run( + `SELECT VECTOR_EMBEDDING(NULL, 'DOCUMENT', 'SAP_GXY.20250407') AS embedding` + ); + + assert.strictEqual(row.embedding, null); + }); + + test('allows additional embedding properties', async () => { + const configuredDb = await cds.connect.to('extended-vector-db', { + kind: 'sqlite', + credentials: { url: ':memory:' }, + embedding: { revision: 'main', extension: { enabled: true } } + }); + + try { + const [row] = await configuredDb.run( + `SELECT VECTOR_EMBEDDING('Hello world', 'DOCUMENT', 'SAP_GXY.20250407') AS embedding` + ); + assert.strictEqual(JSON.parse(row.embedding).length, 384); + } finally { + await configuredDb.disconnect(); + } + }); +}); + +// Helper function to calculate cosine similarity between two vectors +function cosineSimilarity(a, b) { + if (a.length !== b.length) throw new Error('Vectors must have the same length'); + + let dotProduct = 0; + let normA = 0; + let normB = 0; + + for (let i = 0; i < a.length; i++) { + dotProduct += a[i] * b[i]; + normA += a[i] * a[i]; + normB += b[i] * b[i]; + } + + return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB)); +}