Add development Docker Compose deployment for NeMo Retriever

[charlesbluca]

Description

Adds a development-only Docker Compose deployment for running NeMo Retriever without a Kubernetes environment.

The default deployment builds the current checkout and runs the standalone Retriever service with a persistent LanceDB VectorDB and NVIDIA-hosted inference endpoints. Optional profiles and presets support self-hosted NIMs, local Hugging Face GPU models, and OpenTelemetry/Zipkin observability. The stack includes health checks, dependency ordering, configurable ports and GPU assignments, and persistent data and model-cache volumes.

Updates the development documentation with setup, profile composition, data-handling guidance, hardware-gated smoke checks, troubleshooting, and cleanup instructions. Production deployments and split service topology remain supported through the Helm chart.

Also updates the VectorDB launcher to resolve remote embedding credentials from NVIDIA_API_KEY or NGC_API_KEY, while preserving an explicit --embed-api-key override. This avoids placing the hosted API key in the Compose command line and is covered by a parametrized test.

Validation

• Rendered the deployment with Docker Compose using hosted defaults and self-hosted profile combinations.
• Started the default services with an existing local image and confirmed both containers became healthy.
• Confirmed the Retriever health endpoint returned {"status":"ok","mode":"standalone"}.
• Ran focused YAML and documentation contract assertions.
• Added coverage for environment-based and explicit VectorDB embedding API keys.
• Ran git diff --check.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[greptile-apps]

Greptile Summary

This PR introduces a development-only Docker Compose stack for running NeMo Retriever standalone (without Kubernetes), and fixes a security concern where the embedding API key was previously passed as a CLI argument to vectordb_app.

• Compose stack: service-mode.compose.yaml adds retriever + vectordb as always-on services, with optional profiles for self-hosted NIMs (page-elements, table-structure, OCR, embedding, reranker, parse, caption, answer, audio) and an OpenTelemetry/Zipkin observability stack; env-file presets and a local-models override compose file round out the configuration surface.
• API key fix: vectordb_app.main() now calls resolve_remote_api_key(args.embed_api_key) so the credential is consumed from NVIDIA_API_KEY/NGC_API_KEY environment variables rather than appearing on the process command line; the compose command drops --embed-api-key entirely and passes only the env var.
• Dockerfile: Both service and service-gpu targets pre-create /data/vectordb and chown it to the nemo non-root user, ensuring the LanceDB volume mount path is available at startup.

Confidence Score: 5/5

Safe to merge; the previously identified API key exposure in the vectordb command line is resolved, and no new correctness issues were found.

The core logic change — moving embed_api_key resolution from a CLI argument to resolve_remote_api_key() reading from the container environment — is correct and well-tested. Compose configuration, Dockerfile changes, and the inline service YAML are all internally consistent. The only open items are an observability nit (no healthcheck on otel-collector despite the extension being configured) and a missing NGC_API_KEY test case in the new parametrized test.

No files require special attention; the Compose file's nim-answer GPU defaults (7 & 8) remain a documentation note for developers on smaller machines, but this was flagged in a prior review pass.

Important Files Changed

Filename	Overview
nemo_retriever/dev/compose/service-mode.compose.yaml	New development-only Docker Compose stack; GPU defaults for nim-answer (7 & 8) still require a 9-GPU host but this was already flagged; otel-collector dependency correctly uses service_started + required: false; previously-flagged API key in command line is resolved in this PR
nemo_retriever/src/nemo_retriever/service/vectordb_app.py	Replaces direct --embed-api-key passthrough with resolve_remote_api_key(), resolving credential from NVIDIA_API_KEY/NGC_API_KEY env vars; logic is correct and backward-compatible
nemo_retriever/tests/test_service_vectordb_app.py	Adds parametrized test covering env-based and explicit embed API key resolution through main(); covers NVIDIA_API_KEY fallback and explicit override; NGC_API_KEY fallback not exercised here but resolve_remote_api_key is covered elsewhere
Dockerfile	Adds /data/vectordb directory creation and chown to nemo user in both service and service-gpu targets; correctly pre-creates the LanceDB mount point owned by the non-root runtime user
nemo_retriever/dev/compose/service-mode.local-models.compose.yaml	Structural override for in-container HF inference; replaces retriever and vectordb images with service-gpu target, sets per-container GPU assignments, and mounts shared HF model cache volume

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Dev as Developer
    participant Compose as Docker Compose
    participant Retriever as retriever:7670
    participant VectorDB as vectordb:7671
    participant NIM as NIM Services (optional)
    participant NVIDIA as NVIDIA-hosted APIs

    Dev->>Compose: docker compose up --build -d
    Compose->>VectorDB: start (NVIDIA_API_KEY in env)
    VectorDB->>VectorDB: resolve_remote_api_key reads NVIDIA_API_KEY
    VectorDB-->>Compose: healthy (lancedb ready)
    Compose->>NIM: start optional NIMs (profile-gated)
    NIM-->>Compose: healthy (NGC_API_KEY in env)
    Compose->>Retriever: start (depends_on: vectordb healthy)
    Retriever-->>Compose: healthy

    Dev->>Retriever: POST /v1/ingest
    Retriever->>NIM: extract (page elements / OCR / table) or NVIDIA-hosted
    alt NIM_EMBED_URL set to self-hosted
        Retriever->>NIM: embed
    else default hosted
        Retriever->>NVIDIA: embed (NVIDIA_API_KEY)
    end
    Retriever->>VectorDB: POST /v1/ingest (embeddings)
    VectorDB->>VectorDB: persist to LanceDB

    Dev->>Retriever: POST /v1/query
    Retriever->>VectorDB: POST /v1/query
    VectorDB->>VectorDB: embed query (env key or local model)
    VectorDB-->>Retriever: results
    Retriever-->>Dev: QueryResponse

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Dev as Developer
    participant Compose as Docker Compose
    participant Retriever as retriever:7670
    participant VectorDB as vectordb:7671
    participant NIM as NIM Services (optional)
    participant NVIDIA as NVIDIA-hosted APIs

    Dev->>Compose: docker compose up --build -d
    Compose->>VectorDB: start (NVIDIA_API_KEY in env)
    VectorDB->>VectorDB: resolve_remote_api_key reads NVIDIA_API_KEY
    VectorDB-->>Compose: healthy (lancedb ready)
    Compose->>NIM: start optional NIMs (profile-gated)
    NIM-->>Compose: healthy (NGC_API_KEY in env)
    Compose->>Retriever: start (depends_on: vectordb healthy)
    Retriever-->>Compose: healthy

    Dev->>Retriever: POST /v1/ingest
    Retriever->>NIM: extract (page elements / OCR / table) or NVIDIA-hosted
    alt NIM_EMBED_URL set to self-hosted
        Retriever->>NIM: embed
    else default hosted
        Retriever->>NVIDIA: embed (NVIDIA_API_KEY)
    end
    Retriever->>VectorDB: POST /v1/ingest (embeddings)
    VectorDB->>VectorDB: persist to LanceDB

    Dev->>Retriever: POST /v1/query
    Retriever->>VectorDB: POST /v1/query
    VectorDB->>VectorDB: embed query (env key or local model)
    VectorDB-->>Retriever: results
    Retriever-->>Dev: QueryResponse

Loading

_{Reviews (3): Last reviewed commit: "Merge branch 'main' into compose-service..." | Re-trigger Greptile}

[charlesbluca]

Following up on Greptile item 3: the answer NIM defaults to GPUs 7 and 8 intentionally. The documented combined-profile launch uses a collision-free allocation across the full stack: core NIMs use 0-3, reranker 4, parse 5, caption 6, answer 7-8, and audio 9. Defaulting answer to 0/1 would collide with the core profiles in that configuration. I am keeping the defaults and clarifying the README with the full allocation plus an answer-only example that overrides NIM_ANSWER_GPU_ID_0=0 and NIM_ANSWER_GPU_ID_1=1 for a two-GPU host.