Skip to content

Update the cached dataset path shown in the quickstart - #15

Merged
mvinyard merged 2 commits into
mainfrom
fix/quickstart-cache-path
Aug 16, 2026
Merged

Update the cached dataset path shown in the quickstart#15
mvinyard merged 2 commits into
mainfrom
fix/quickstart-cache-path

Conversation

@mvinyard

Copy link
Copy Markdown
Member

Follow-up to scDiffEq/scDiffEq#113.

What changed upstream

scDiffEq now caches the raw download and the preprocessed result as separate files, so sdq.datasets.larry() loads from larry.processed.h5ad rather than larry.h5ad. Previously both shared one filename, which meant a dataset already on disk was returned unpreprocessed with no X_pca.

Why it matters here

tutorials/quickstart.ipynb renders on scdiffeq.com, and its stored output still showed the old path:

scDiffEq [INFO]: Loading data from /Users/mvinyard/data/scdiffeq_data/larry/larry.h5ad

A reader following that would go looking for a file the loader no longer writes. (Existing caches are auto-migrated, so the old name disappears on first use.)

Note on how this was fixed

This edits the recorded output rather than re-executing the notebook: that cell downloads 5.3 GB and the notebook goes on to train a model, so a full re-run was not practical.

The single replaced line is exactly what the loader emits today on a cached run, and nothing else changed — the AnnData repr (130,887 × 2,492) still matches the default variant, and the scaler.pkl / pca.pkl paths further down remain correct, since the default variant deliberately keeps those bare filenames.

Diff is one line; nbformat.validate passes.

Not touched

manuscript/tmp_revision/figure_r1/notebooks/FigureS9.ipynb shows the same old path. Left alone deliberately — manuscript notebooks record what was actually executed for the paper, and that directory is not published to the docs site (the workflow copies manuscript/figure_*/notebooks/, not manuscript/*/figure_*/).

🤖 Generated with Claude Code

Michael Vinyard and others added 2 commits August 15, 2026 11:53
scDiffEq now caches the raw download and the preprocessed result as separate
files, so `sdq.datasets.larry()` loads from `larry.processed.h5ad` rather than
`larry.h5ad`. The stored output in this notebook still showed the old path, and
since it renders on scdiffeq.com a reader following it would go looking for a
file the loader no longer writes.

This edits the recorded output rather than re-executing: the cell downloads
5.3 GB and the notebook goes on to train a model, so a full re-run was not
practical here. The single line replaced is exactly what the loader emits today
on a cached run, and nothing else in the cell changed -- the AnnData repr
(130,887 x 2,492) still matches the default variant.

See scDiffEq/scDiffEq#113.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This is a verbatim copy of the scDiffEq repository's docs workflow -- its first
step is even named "Checkout scDiffEq" -- but running here it checks out this
repository, which has no docs/ directory. It then clones this repository again,
copies notebooks into a docs/source/ it just created, and fails at
`pip install -r docs/requirements.txt` because that file does not exist here.

It cannot succeed in this repository, and has been failing on every push to main
independently of any pull request. If it did somehow build, it would deploy to
this repository's GitHub Pages rather than the site at scdiffeq.com.

The documentation site is built from the scDiffEq repository, whose own docs
workflow already clones this one to pull in manuscript/figure_*/notebooks/ and
tutorials/. Nothing is lost by removing this copy.

Note that notebook changes here therefore do not rebuild the site on their own;
that happens on the next scDiffEq build. Closing that gap would mean triggering
the scDiffEq workflow from here via repository_dispatch, which needs a token with
workflow scope on that repository -- deliberately not added as part of this
change.
@mvinyard

mvinyard commented Aug 15, 2026

Copy link
Copy Markdown
Member Author

Added a second commit removing .github/workflows/docs.yml, which is what was failing on this PR.

It turned out to be pre-existing, not caused by the notebook change, the same workflow has been failing on every push to main:

failure  pull_request  fix/quickstart-cache-path  Build and Deploy Docs
failure  push          main                       Build and Deploy Docs
failure  push          main                       Build and Deploy Docs

The file is a verbatim copy of the scDiffEq repository's docs workflow — its first step is even named Checkout scDiffEq, but running here it checks out this repository, which has no docs/ directory. It then clones this repository again, copies notebooks into a docs/source/ it just created, and dies at:

ERROR: Could not open requirements file: 'docs/requirements.txt'

It cannot succeed here, and if it did build it would deploy to this repository's GitHub Pages rather than scdiffeq.com. The site is built from the scDiffEq repository, whose own workflow already clones this one for manuscript/figure_*/notebooks/ and tutorials/.

One consequence worth knowing: notebook changes here don't rebuild the site on their own: that happens on the next scDiffEq build. Closing that gap properly means triggering the scDiffEq workflow from here via repository_dispatch, which needs a token with workflow scope on that repository. Deliberately not added here, since it's a credential decision.

@mvinyard
mvinyard merged commit a8018d4 into main Aug 16, 2026
@mvinyard
mvinyard deleted the fix/quickstart-cache-path branch August 16, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant