diff --git a/source/examples/index.md b/source/examples/index.md
index 4a5843ad..36ebb259 100644
--- a/source/examples/index.md
+++ b/source/examples/index.md
@@ -27,4 +27,5 @@ rapids-morpheus-pipeline/notebook
 fraud-detection-mlops-pipeline/notebook
 lulc-classification-gpu/notebook
 cuml-ray-hpo/notebook
+rapids-topic-modeling-slurm/notebook
 ```
diff --git a/source/examples/rapids-topic-modeling-slurm/notebook.ipynb b/source/examples/rapids-topic-modeling-slurm/notebook.ipynb
new file mode 100644
index 00000000..d12f3d24
--- /dev/null
+++ b/source/examples/rapids-topic-modeling-slurm/notebook.ipynb
@@ -0,0 +1,70 @@
+{
+ "cells": [
+  {
+   "id": "intro",
+   "cell_type": "markdown",
+   "metadata": {
+    "tags": [
+     "platforms/hpc-slurm",
+     "library/cuml",
+     "library/dagster",
+     "library/gensim",
+     "data-format/parquet",
+     "workflow/topic-modeling"
+    ]
+   },
+   "source": "# GPU Topic Modeling on HPC with dagster-slurm, cuML UMAP, and HDBSCAN\n\n_July, 2026_\n\nIn this example workflow, we run a topic-modeling pipeline on a Slurm HPC\ncluster with [dagster-slurm](https://github.com/ascii-supply-networks/dagster-slurm), an open source integration that runs\n[Dagster](https://dagster.io/) assets as Slurm jobs directly from your laptop.\nCPU stages train [gensim](https://radimrehurek.com/gensim/) LDA models as a\npartitioned fan-out of `sbatch` jobs; GPU stages reduce and cluster the\nresulting topic vectors with [RAPIDS cuML](https://docs.rapids.ai/api/cuml/stable/)\nUMAP and HDBSCAN on a GPU node; the final asset streams a labeled meta-topic\nmap back into the Dagster UI.\n\nUnlike the cloud and Kubernetes examples in this gallery, this workflow\ntargets classic HPC infrastructure: a shared Slurm cluster with no container\nruntime, where Python environments arrive as\n[pixi-pack](https://github.com/Quantco/pixi-pack) archives over SSH. The same\nasset code runs in three settings without modification:\n\n- **local mode** on a laptop (no Slurm, no GPU),\n- a **docker Slurm cluster** for CI and development (CPU fallback for the\n  RAPIDS stages),\n- a **real HPC cluster**, where the UMAP and HDBSCAN stages run on cuML with\n  `gpus_per_node: 1`.\n\nThe example is a small-scale, public-data version of a production Common\nCrawl topic-modeling pipeline. It deliberately does not show GPU speedup\nbenchmarks: the corpus is small, and the point is the orchestration\nmechanics, which are identical at production scale.\n\n## Quickstart\n\n```bash\ngit clone https://github.com/ascii-supply-networks/dagster-slurm.git\ncd dagster-slurm && docker compose up -d     # local Slurm cluster\ncd examples && pixi run start-staging\n# open http://localhost:3000 and materialize the rapids_topics group\n```\n\nThat is the whole interface. The rest of this page explains what happens\nwhen you press the button, and how to point the same button at a real HPC\ncluster.\n\n### What you would normally do, and what happens here\n\nThe usual workflow for a pipeline like this is: SSH to the login node,\n`module load` a Python that is almost the right version, hand-maintain a\nvenv or conda environment on the cluster, write an `sbatch` script per\nstage, submit, poll `squeue`, and grep `slurm-*.out` files when something\nfails. Here, none of that is manual: dagster-slurm packs the exact\nenvironment from your repo's lockfile into a content-hashed archive, ships\nit and the payload script over SSH, generates and submits the `sbatch`\njobs, streams the logs back live, and records structured results per stage.\nThe cluster needs nothing beyond SSH and `sbatch`; your laptop needs\nnothing beyond pixi and docker.\n"
+  },
+  {
+   "id": "pipeline",
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## Overview\n\nThis workflow shows how to:\n\n- Fan out partitioned model training as independent `sbatch` jobs (one Slurm\n  job per partition) from a Dagster backfill\n- Target **different packed environments per asset**: CPU stages run in a\n  gensim environment, GPU stages in a self-contained RAPIDS environment\n- Request GPUs per asset (`gpus_per_node: 1`) only on deployments that have\n  them, with an automatic CPU fallback (umap-learn / hdbscan) elsewhere\n- Ship Python environments to clusters **without container runtimes** using\n  `pixi-pack`\n- Stream Slurm job logs, structured metadata, and inline plots back into the\n  Dagster UI over SSH\n- Override Slurm sizing (CPUs, memory, wall time, GPU count) per run from the\n  Dagster launchpad\n\n## The pipeline\n\nSix assets in the `rapids_topics` group:\n\n```{note}\nEvery stage in this example runs as an `sbatch` job by choice, to keep the\ndemo uniform. That is not a constraint: Dagster orchestrates before, during,\nand after the cluster, so ingest or publishing assets can run in the Dagster\nprocess (or anywhere else) while only the heavy middle targets Slurm, all in\none lineage graph. On sites where compute nodes have no internet access you\nwould want exactly that for `reuters_corpus`: run the download as a local or\nlogin-node asset instead of an `sbatch` job.\n```\n\n```\nreuters_corpus            CPU sbatch   workload-topic-modeling env\n    |                     download + SGML parse + shared gensim dictionary\n    v\nlda_models                CPU sbatch   one job per (month, seed) partition\n    |                     5 months x 3 seeds = 15 independent Slurm jobs\n    v\ntopic_term_matrix         CPU sbatch   stack topic-term vectors of all models\n    |\n    v\numap_embedding            GPU sbatch   packaged-cluster-rapids env\n    |                     cuML UMAP (umap-learn fallback on CPU)\n    v\nhdbscan_meta_topics       GPU sbatch   cuML HDBSCAN (hdbscan fallback on CPU)\n    |\n    v\ntopic_map                 CPU sbatch   labeled meta-topic scatter + JSON summary\n```\n\n![The rapids_topics asset group in the Dagster lineage view, fully materialized against a real Slurm cluster](../../images/dagster-slurm-topics-lineage-overview.png)\n\n### Why cluster topic-term vectors?\n\nDoc-topic vectors from independently trained LDA models are not comparable:\ntopic 7 in the February model has nothing to do with topic 7 in the March\nmodel. The chain therefore clusters **topic-term vectors** (rows of\n`get_topics()` over a dictionary shared by all models) into meta-topics.\nEvery topic from every `(month, seed)` model becomes one point in vocabulary\nspace; UMAP reduces those points to 2D and HDBSCAN groups recurring themes\nacross months and seeds into meta-topics.\n\n### Dataset\n\n[Reuters-21578](https://kdd.ics.uci.edu/databases/reuters21578/reuters21578.html),\nthe classic 1987 newswire research corpus (~21k documents, ~18k with usable\nbody text). It is small, quick to download, and dated, which preserves the\ntemporal partitioning story of the production pipeline: the corpus asset\nbuckets documents by month, and LDA training fans out over `(month, seed)`.\n\n### Code layout\n\ndagster-slurm separates orchestration from computation. The *assets* only\npick a payload script, an environment, and Slurm resources; the *payloads*\nare plain Python scripts that talk to Dagster through\n[Dagster Pipes](https://docs.dagster.io/guides/build/external-pipelines) and\nalso run standalone.\n\n| Piece | Path in the [dagster-slurm repo](https://github.com/ascii-supply-networks/dagster-slurm) |\n|---|---|\n| Asset definitions | [`examples/.../defs/rapids_topics/topic_assets.py`](https://github.com/ascii-supply-networks/dagster-slurm/blob/main/examples/projects/dagster-slurm-example/dagster_slurm_example/defs/rapids_topics/topic_assets.py) |\n| Payload scripts | [`examples/.../dagster_slurm_example_hpc_workload/rapids_topics/`](https://github.com/ascii-supply-networks/dagster-slurm/tree/main/examples/projects/dagster-slurm-example-hpc-workload/dagster_slurm_example_hpc_workload/rapids_topics) |\n| Environments | [`examples/pyproject.toml`](https://github.com/ascii-supply-networks/dagster-slurm/blob/main/examples/pyproject.toml) (`workload-topic-modeling`, `packaged-cluster-rapids`) |\n"
+  },
+  {
+   "id": "environments",
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## One packed environment per stage\n\nThe pipeline uses two pixi environments, declared per asset via\n`slurm_pack_cmd` metadata:\n\n- **`workload-topic-modeling`**: the standard cluster stack plus gensim. Used\n  by the CPU stages (corpus, LDA, aggregation).\n- **`packaged-cluster-rapids`**: a self-contained environment (Python 3.12,\n  `numpy<2.3`) with `cuml` from the `rapidsai` conda channel on linux-64, and\n  the CPU fallback libraries (umap-learn, hdbscan, matplotlib) co-installed.\n  Used by the UMAP, HDBSCAN, and report stages on every deployment, CPU\n  fallback included.\n\n```toml\n# examples/pyproject.toml (excerpt)\n[tool.pixi.feature.cluster-rapids]\nchannels = [{ channel = \"rapidsai\", priority = 1 }]\n\n[tool.pixi.feature.cluster-rapids.dependencies]\npython = \"3.12.*\"\nnumpy = \">=2.0,<2.3\"\numap-learn = \">=0.5,<1\"\nhdbscan = \">=0.8,<1\"\nmatplotlib = \">=3.9,<4\"\n\n[tool.pixi.feature.cluster-rapids.target.linux-64.dependencies]\ncuml = \">=25.10,<26\"\n```\n\nWhy a separate solve-group instead of adding cuML to the main cluster\nenvironment? RAPIDS pins numba, and numba pins numpy below what the rest of\nthe cluster stack wants. Co-locating umap-learn with the main stack makes the\nresolver backtrack into unbuildable sdists. Giving RAPIDS its own solve-group\nkeeps both environments installable, and dagster-slurm makes running each\nasset in its own environment a one-line metadata declaration:\n\n```python\n_RAPIDS_PACK_METADATA = {\n    \"slurm_pack_cmd\": [\n        \"pixi\", \"run\", \"-e\", \"opstooling\", \"--frozen\",\n        \"python\", \"scripts/pack_environment.py\",\n        \"--env\", \"packaged-cluster-rapids\", \"--build-missing\",\n    ],\n}\n\n@dg.asset(group_name=\"rapids_topics\", metadata=_RAPIDS_PACK_METADATA, ...)\ndef umap_embedding(...): ...\n```\n\nThe environment is packed with `pixi-pack` into a single self-extracting\narchive, shipped to the cluster over SSH, and cached by content hash. No\ncontainer runtime is needed on the cluster, which is the common case on HPC\nsites.\n\n## GPU on the cluster, CPU everywhere else\n\nThe payloads select their backend at import time. If cuML imports, the GPU\nimplementation is used; otherwise the CPU library. One payload serves both\nthe docker Slurm cluster and a GPU node:\n\n```python\n# umap_reduce.py (excerpt)\ntry:\n    import cuml\n    cuml.set_global_output_type(\"numpy\")\n    _HAS_CUML = True\nexcept ImportError:\n    _HAS_CUML = False\n\n\ndef make_umap(*, n_components, n_neighbors, min_dist, metric,\n              random_state, build_algo):\n    if _HAS_CUML:\n        from cuml.manifold import UMAP as _UMAP\n        return _UMAP(\n            n_components=n_components, n_neighbors=n_neighbors,\n            min_dist=min_dist, metric=metric,\n            random_state=random_state, build_algo=build_algo,\n            verbose=True,\n        )\n    from umap import UMAP as _UMAP\n    return _UMAP(\n        n_components=n_components, n_neighbors=n_neighbors,\n        min_dist=min_dist, metric=metric,\n        random_state=random_state, low_memory=True, verbose=True,\n    )\n```\n\nThe matching asset requests a GPU only when the deployment is a real\nsupercomputer:\n\n```python\ndef _gpu_slurm_opts() -> dict:\n    if _is_supercomputer():\n        return {\"nodes\": 1, \"cpus_per_task\": 8, \"mem\": \"32G\",\n                \"gpus_per_node\": 1}\n    return {\"nodes\": 1, \"cpus_per_task\": 2, \"mem\": \"4G\",\n            \"gpus_per_node\": 0}\n```\n\nEach materialization reports which backend actually ran\n(`backend: cuml (GPU)` or `backend: umap-learn (CPU)`) in its asset\nmetadata, so a misconfigured deployment is visible in the UI rather than\nsilent.\n\n### Practical notes on cuML on HPC\n\nHard-won details baked into the example, worth knowing before you adapt it:\n\n- **`cuml.set_global_output_type(\"numpy\")`** keeps the rest of the payload\n  backend-agnostic: downstream code sees numpy arrays whether cuML or the\n  CPU library produced them.\n- **UMAP `build_algo` stays on `\"auto\"`.** cuML picks brute-force kNN for\n  small inputs and GPU nn-descent at scale. Forcing `nn_descent` on a small\n  input (fewer than ~150 rows) crashes cuML with a CUDA invalid-argument\n  error. Only set it explicitly for genuinely large corpora.\n- **cuML HDBSCAN caps `min_samples` at 1023.** The payload clamps the value\n  on the GPU branch only.\n- **cuML HDBSCAN labels more points as noise** than the CPU library at\n  identical settings. Tune `min_cluster_size` / `min_samples` against your\n  real data, not against the CPU fallback.\n- **cuML is linux-64 only** in this setup, declared under\n  `[tool.pixi.feature.cluster-rapids.target.linux-64.dependencies]`, so the\n  same pixi environment still solves on a macOS laptop (CPU libraries only).\n"
+  },
+  {
+   "id": "running",
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## Running it\n\n### Prerequisites\n\n- [pixi](https://pixi.sh/) installed locally\n- A clone of the [dagster-slurm repository](https://github.com/ascii-supply-networks/dagster-slurm)\n- For the docker mode: docker compose\n- For the HPC mode: SSH access to a Slurm cluster (a login node you can\n  `sbatch` from); GPUs optional but required for the cuML path\n\n```bash\ngit clone https://github.com/ascii-supply-networks/dagster-slurm.git\ncd dagster-slurm/examples\n```\n\n### 1. Local mode (laptop, no Slurm)\n\n```bash\npixi run start\n# open http://localhost:3000, materialize assets in the rapids_topics group\n```\n\n```{note}\nLocal mode materializes **3 of the 6 assets**: `reuters_corpus`,\n`lda_models`, and `topic_term_matrix` run directly on your machine.\n`umap_embedding`, `hdbscan_meta_topics`, and `topic_map` need a Slurm\ndeployment with the rapids environment and will fail locally: the dev\nenvironment deliberately excludes umap-learn/hdbscan/matplotlib because of\nthe numba/numpy pin conflict described above. Use the docker Slurm cluster\nbelow for the full chain.\n```\n\n### 2. Docker Slurm cluster (full chain, CPU fallback)\n\nStart the dockerized Slurm cluster that ships with the repo and run in\nstaging mode, where environments are packed and deployed on demand:\n\n```bash\ndocker compose up -d          # repo root: slurmctld + compute nodes\ncd examples\npixi run start-staging\n```\n\nMaterialize the whole `rapids_topics` group. Every asset becomes an `sbatch`\njob inside the docker cluster; the UMAP/HDBSCAN payloads log\n`backend: umap-learn (CPU)` and produce the same artifact shapes as the GPU\npath. This is also what CI exercises.\n\nWhat to expect: the Reuters-21578 download is about 8 MB, and with cached\nenvironments the full chain completes in a few minutes (the individual jobs\ntake seconds to ~1 minute each). The first run additionally packs the two\nenvironments, which dominates wall-clock: the RAPIDS environment in\nparticular is large, so expect the initial pack to take on the order of tens\nof minutes depending on your machine and network.\n\n### 3. Real HPC cluster with GPUs\n\nPoint dagster-slurm at your cluster and start in supercomputer mode:\n\n```bash\nexport SLURM_EDGE_NODE_HOST=login.your-cluster.example\nexport SLURM_EDGE_NODE_USER=your-user\nexport SLURM_EDGE_NODE_KEY_PATH=~/.ssh/id_ed25519\n\ncd examples\npixi run start-staging-supercomputer      # pack + deploy envs on demand\n# or, with pre-deployed environments:\npixi run start-production-supercomputer\n```\n\nIn supercomputer deployments the GPU assets submit with `gpus_per_node: 1`\nand the payloads log `backend: cuml (GPU)`.\n\nTwo things an experienced Slurm user will ask:\n\n- **\"A 15-job fan-out is 15 queue waits, can I run those inside one\n  allocation?\"** dagster-slurm has session and heterogeneous-job modes for\n  exactly this, running multiple assets inside a single Slurm allocation to\n  amortize queueing. They are experimental at the time of writing, which is\n  why this example sticks to one `sbatch` per asset; see\n  [execution modes](https://dagster-slurm.geoheil.com/docs/how-to/execution-modes) for status.\n- **\"Can I target more than one cluster?\"** Deployments are configuration,\n  not code: each deployment names its own edge node, so the same asset graph\n  can run against your institute cluster in one deployment and a national\n  system in another (the dagster-slurm docs ship site notes for several\n  European HPC systems).\n\nOptional environment variables:\n\n| Variable | Effect |\n|---|---|\n| `RAPIDS_TOPICS_BASE` | Base output directory on the cluster (default `$HOME/rapids_topics`) |\n| `RAPIDS_TOPICS_CPU_ENV` | Path to an already-extracted CPU env on the cluster; skips packing |\n| `RAPIDS_TOPICS_GPU_ENV` | Same, for the rapids environment |\n\n```{note}\nPacking the rapids environment takes a while the first time (cuML is large).\nFor iterating on a real cluster, extract it once and set\n`RAPIDS_TOPICS_GPU_ENV`, or use the launchpad override below.\n```\n\n### Per-run overrides from the launchpad\n\nAll six assets share a config schema whose fields default to \"use the\ndeployment-aware defaults\". From the Dagster launchpad you can override\n`cpus_per_task`, `mem`, `time_limit`, `gpus_per_node`, and\n`pre_deployed_env_path` for a single run without touching code, plus the\nmodeling knobs (topic count, UMAP neighbors, HDBSCAN cluster sizes) each\nstage exposes.\n"
+  },
+  {
+   "id": "ui-walkthrough",
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## What you see in the UI\n\nThe screenshots below are from a run against a real Slurm cluster (staging\nsupercomputer mode, one A100 for the GPU stages).\n\n**Partitioned fan-out.** `lda_models` is partitioned by `(month, seed)`; a\nbackfill dispatches each partition as its own `sbatch` job. The lineage view\nshows the fan-out filling up while downstream assets wait:\n\n![lda_models mid-backfill: partitions filling while topic_term_matrix and umap_embedding wait](../../images/dagster-slurm-topics-backfill-fanout.png)\n\n**One Slurm job per run.** The backfill's run list tags every run with its\nSlurm job id (`dagster_slurm/job_id`), so you can correlate Dagster runs\nwith `sacct`/`squeue` output directly:\n\n![Backfill run list: each lda_models partition is a separate run with its own Slurm job id](../../images/dagster-slurm-topics-backfill-runs.png)\n\n**Live event log, including environment packing.** The event log shows the\nfull lifecycle: cache miss on the environment hash, the reproducible\n`pixi-pack` command, job submission, and live log streaming over SSH:\n\n![Event log of a downstream run: environment packing, submission, and state transitions](../../images/dagster-slurm-topics-run-event-log.png)\n\n**Raw Slurm stdout, streamed.** The `stdout` tab shows exactly what ran on\nthe compute node: working directory, payload path, which Python the packed\nenvironment resolved to:\n\n![Slurm job stdout streamed into the Dagster UI: environment activation and payload launch](../../images/dagster-slurm-topics-slurm-stdout.png)\n\n**Structured results per stage.** Every payload reports metadata through\nPipes: row counts, output paths, the Slurm job id, plus scheduler-derived\nefficiency numbers (`node_hours`, `cpu_efficiency_pct`, `max_memory_mb`):\n\n![STEP_OUTPUT of topic_term_matrix: model count, output path, Slurm job id, efficiency metrics](../../images/dagster-slurm-topics-step-output-metadata.png)\n\nThe corpus asset does the same at the head of the pipeline (document counts\nper month, dictionary size, output directory):\n\n![reuters_corpus materialization metadata: 17893 documents, 5 months, shared dictionary](../../images/dagster-slurm-topics-corpus-asset-metadata.png)\n\n**Metrics over time.** Because efficiency numbers are numeric metadata,\nDagster plots them across materializations for free. Cost regressions in a\npipeline stage show up as a line going the wrong way:\n\n![Metadata plots: cpu_efficiency_pct and elapsed_seconds across materializations](../../images/dagster-slurm-topics-metadata-plots.png)\n\n**The result.** The terminal `topic_map` asset reports everything a reader\nof the pipeline needs as materialization metadata: cluster counts\n(`n_meta_topics: 7`, `n_noise_topics: 2`), the plot and summary paths on\nthe cluster filesystem, the labeled cluster summary as JSON, and a markdown\npreview of the plot itself:\n\n![topic_map run view: all downstream stages green, materialization metadata with cluster counts, artifact paths, and Slurm job id](../../images/dagster-slurm-topics-topic-map-run-success.png)\n\nThat preview means the topic map renders directly inside the Dagster UI, no\n`scp` of PNGs off the cluster required:\n\n![topic_map_preview rendered inline in the Dagster UI: the meta-topic scatter inside the run view](../../images/dagster-slurm-topics-topic-map-inline-preview.png)\n\nThe map itself, as written to the cluster filesystem: 45 topic-term vectors\nfrom the LDA models, UMAP-reduced and HDBSCAN-clustered into 7 meta-topics,\neach labeled with its top shared terms. At this toy scale some clusters\ncollapse onto newswire boilerplate and function words (\"vs, mln, loss\" is\nthe earnings-report cluster; \"that, will, be\" is not a topic anyone would\npublish), which is exactly the honest output of 1987 newswire at 15 topics\nper month; the production-scale version of this chain uses far larger\ncorpora and vocabulary filtering:\n\n![Labeled meta-topic map: UMAP scatter of topic-term vectors, colored by HDBSCAN cluster](../../images/dagster-slurm-topics-topic-map.png)\n\n**The whole chain.** A complete backfill of the group on the real cluster,\nthree LDA partitions plus the four surrounding stages, finished in 19m28s\nend to end: the corpus stage (download + parse) took just under 12 minutes,\neach LDA partition under a minute, and the downstream\naggregation-UMAP-HDBSCAN-report run just under five. The backfill overview\nis the at-a-glance version:\n\n![Backfill overview: all six assets at 100 percent, every stage succeeded](../../images/dagster-slurm-topics-backfill-complete.png)\n"
+  },
+  {
+   "id": "metaxy",
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## Refined example: incremental reprocessing with metaxy\n\nThe basic pipeline recomputes everything downstream of a change: re-train\none month's LDA models and the aggregation, UMAP, and HDBSCAN stages all run\nagain over the full topic set. At toy scale that is fine. In production,\nwhere the fan-out is hundreds of partitions and model retrains arrive\ncontinuously, you want to know *which topic vectors actually changed* and\nskip the rest.\n\n[metaxy](https://github.com/anam-org/metaxy) adds sample-level incremental\ntracking on top of the same pipeline. The dagster-slurm examples ship two\nworking metaxy integrations (the `metaxy_simple` and `metaxy_ray` asset\ngroups); the refinement below applies the identical pattern to the topic\nchain.\n\n### Feature specs\n\nEach topic-term vector is a tracked sample, keyed by a stable id. Meta-topic\nassignments depend on them:\n\n```python\nimport metaxy as mx\n\nclass TopicTermVectors(\n    mx.BaseFeature,\n    spec=mx.FeatureSpec(\n        key=\"rapids_topics/topic_term_vectors\",\n        id_columns=[\"topic_uid\"],          # e.g. \"1987-02/seed=1/topic=7\"\n        fields=[\"month\", \"seed\", \"topic_id\", \"vector\"],\n    ),\n):\n    topic_uid: str\n    month: str\n    seed: int\n    topic_id: int\n    vector: list[float]\n\n\nclass MetaTopics(\n    mx.BaseFeature,\n    spec=mx.FeatureSpec(\n        key=\"rapids_topics/meta_topics\",\n        id_columns=[\"topic_uid\"],\n        fields=[\"embedding\", \"cluster\"],\n        deps=[TopicTermVectors],\n    ),\n):\n    topic_uid: str\n    embedding: list[float]\n    cluster: int\n```\n\n### Asset side\n\nThe aggregation asset registers vectors in a metaxy store instead of only\nwriting a parquet file. The store is selected per deployment in\n`metaxy.toml` (DuckDB locally, a Delta table on the shared cluster\nfilesystem in production), and the config file ships to the cluster with the\npayload via `extra_files`:\n\n```python\nimport metaxy.ext.dagster as mxd\n\n@mxd.metaxify\n@dg.asset(\n    metadata={\"metaxy/feature\": \"rapids_topics/topic_term_vectors\",\n              **_CPU_PACK_METADATA},\n    group_name=\"rapids_topics_metaxy\",\n    deps=[lda_models],\n)\ndef topic_term_matrix(context, compute: ComputeResource, config: TopicSlurmConfig):\n    metaxy_config = dg.file_relative_path(__file__, \"../../../../../metaxy.toml\")\n    return compute.run(\n        context=context,\n        payload_path=_payload(\"aggregate_topics_metaxy.py\"),\n        config=config,\n        extra_files=[metaxy_config],\n        extra_env={\"METAXY_STORE\": _store_for_deployment(), **_base_env()},\n        extra_slurm_opts=_merged_slurm_opts(_cpu_slurm_opts(), config),\n    ).get_results()\n```\n\n### Payload side\n\nInside the Slurm job, the payload asks the store what changed and processes\nonly that increment:\n\n```python\n# aggregate_topics_metaxy.py (core of the payload)\nimport metaxy as mx\n\ncfg = mx.init()                     # reads the shipped metaxy.toml\nstore = cfg.get_store()\n\nwith store:\n    increment = store.resolve_update(\"rapids_topics/topic_term_vectors\",\n                                     samples=stacked_vectors)\n\nto_write = increment.new.to_polars()\nstale = increment.stale.to_polars()   # vectors whose upstream model changed\ncontext.log.info(f\"{len(to_write)} new + {len(stale)} stale topic vectors\")\n\nwith store.open(mode=\"w\"):\n    if len(to_write) > 0:\n        store.write(\"rapids_topics/topic_term_vectors\", to_write)\n    if len(stale) > 0:\n        store.write(\"rapids_topics/topic_term_vectors\", stale)\n```\n\nThe UMAP/HDBSCAN payload then resolves the increment for\n`rapids_topics/meta_topics`. If the increment is empty, it reports\n`status: up_to_date` and exits without touching the GPU, which on a busy\ncluster means the job releases its allocation in seconds.\n\n### What this buys you, honestly\n\nUMAP and HDBSCAN are global models: when the topic set does change, the\nreduction and clustering rerun over the full set, because a partial re-embed\nis not meaningful. The incremental win for the GPU stages is therefore\n**change detection** (skip the whole GPU job when nothing upstream changed,\nfor example after a partial backfill retry) and **provenance** (every\nmeta-topic assignment is traceable to the exact model version that produced\nits topic vector). For the fan-in stage the win is the classic one: only new\nor stale vectors are re-registered.\n\nRe-materializing a single month's `lda_models` partitions now results in:\n\n1. `topic_term_matrix` registers only that month's ~45 vectors as stale;\n   everything else is untouched.\n2. `umap_embedding` sees a non-empty increment and reruns (global model).\n3. A second materialization with no upstream changes reports\n   `status: up_to_date` at every stage and submits no compute-heavy work.\n\nFor a complete, runnable reference of the store setup, `@metaxify` wiring,\nand `MetaxyDatasource`/`MetaxyDatasink` inside distributed payloads, see the\n`metaxy_simple` and `metaxy_ray` groups in the\n[dagster-slurm examples](https://github.com/ascii-supply-networks/dagster-slurm/tree/main/examples/projects/dagster-slurm-example/dagster_slurm_example/defs).\n\n## Conclusion\n\nThe pattern shown here, per-asset Slurm sizing and per-asset packed\nenvironments around a CPU fan-out plus GPU reduction, is the shape of a\nlarge class of scientific workloads: embarrassingly parallel training or\nextraction, followed by accelerated aggregation. dagster-slurm contributes\nthe orchestration ergonomics (lineage, backfills, live logs, structured\nmetadata) without asking the HPC site for anything beyond SSH and `sbatch`,\nand RAPIDS contributes drop-in GPU acceleration for the reduction stages\nwith a clean CPU fallback for development and CI.\n\nThe workflow argument underneath all of it is the iteration loop. Because\nthe same asset code runs in local mode, on the docker cluster, and on the\nreal cluster, you (or a coding agent, or CI) iterate at seconds-scale on\nyour laptop, promote to the dockerized Slurm cluster to check scheduling\nbehavior, and only then spend queue time and GPU hours, without rewriting\nanything between settings. Cluster time becomes something you spend on\npurpose, not something you burn debugging environment drift.\n\n- **dagster-slurm**: [repository](https://github.com/ascii-supply-networks/dagster-slurm), [documentation](https://dagster-slurm.geoheil.com)\n- **This example in the dagster-slurm repo**:\n  [assets](https://github.com/ascii-supply-networks/dagster-slurm/tree/main/examples/projects/dagster-slurm-example/dagster_slurm_example/defs/rapids_topics)\n  and\n  [payloads](https://github.com/ascii-supply-networks/dagster-slurm/tree/main/examples/projects/dagster-slurm-example-hpc-workload/dagster_slurm_example_hpc_workload/rapids_topics)\n- **Environment packaging details**:\n  [Packaging dependencies](https://dagster-slurm.geoheil.com/docs/how-to/environment-packaging)\n- **Upstream tracking issue**:\n  [rapidsai/deployment#715](https://github.com/rapidsai/deployment/issues/715)\n"
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/source/images/dagster-slurm-topics-backfill-complete.png b/source/images/dagster-slurm-topics-backfill-complete.png
new file mode 100644
index 00000000..9c3436fb
Binary files /dev/null and b/source/images/dagster-slurm-topics-backfill-complete.png differ
diff --git a/source/images/dagster-slurm-topics-backfill-fanout.png b/source/images/dagster-slurm-topics-backfill-fanout.png
new file mode 100644
index 00000000..cc19ede9
Binary files /dev/null and b/source/images/dagster-slurm-topics-backfill-fanout.png differ
diff --git a/source/images/dagster-slurm-topics-backfill-runs.png b/source/images/dagster-slurm-topics-backfill-runs.png
new file mode 100644
index 00000000..f1053d01
Binary files /dev/null and b/source/images/dagster-slurm-topics-backfill-runs.png differ
diff --git a/source/images/dagster-slurm-topics-corpus-asset-metadata.png b/source/images/dagster-slurm-topics-corpus-asset-metadata.png
new file mode 100644
index 00000000..d5d682c1
Binary files /dev/null and b/source/images/dagster-slurm-topics-corpus-asset-metadata.png differ
diff --git a/source/images/dagster-slurm-topics-lineage-overview.png b/source/images/dagster-slurm-topics-lineage-overview.png
new file mode 100644
index 00000000..0829ea11
Binary files /dev/null and b/source/images/dagster-slurm-topics-lineage-overview.png differ
diff --git a/source/images/dagster-slurm-topics-metadata-plots.png b/source/images/dagster-slurm-topics-metadata-plots.png
new file mode 100644
index 00000000..3b9bb233
Binary files /dev/null and b/source/images/dagster-slurm-topics-metadata-plots.png differ
diff --git a/source/images/dagster-slurm-topics-run-event-log.png b/source/images/dagster-slurm-topics-run-event-log.png
new file mode 100644
index 00000000..5dbbeaf6
Binary files /dev/null and b/source/images/dagster-slurm-topics-run-event-log.png differ
diff --git a/source/images/dagster-slurm-topics-slurm-stdout.png b/source/images/dagster-slurm-topics-slurm-stdout.png
new file mode 100644
index 00000000..f476d799
Binary files /dev/null and b/source/images/dagster-slurm-topics-slurm-stdout.png differ
diff --git a/source/images/dagster-slurm-topics-step-output-metadata.png b/source/images/dagster-slurm-topics-step-output-metadata.png
new file mode 100644
index 00000000..1f70125b
Binary files /dev/null and b/source/images/dagster-slurm-topics-step-output-metadata.png differ
diff --git a/source/images/dagster-slurm-topics-topic-map-inline-preview.png b/source/images/dagster-slurm-topics-topic-map-inline-preview.png
new file mode 100644
index 00000000..cbdc7bf2
Binary files /dev/null and b/source/images/dagster-slurm-topics-topic-map-inline-preview.png differ
diff --git a/source/images/dagster-slurm-topics-topic-map-run-success.png b/source/images/dagster-slurm-topics-topic-map-run-success.png
new file mode 100644
index 00000000..4abea8a7
Binary files /dev/null and b/source/images/dagster-slurm-topics-topic-map-run-success.png differ
diff --git a/source/images/dagster-slurm-topics-topic-map.png b/source/images/dagster-slurm-topics-topic-map.png
new file mode 100644
index 00000000..07713f3a
Binary files /dev/null and b/source/images/dagster-slurm-topics-topic-map.png differ