ci: keep the jupyter execution cache in the build-cache artefact - #235
Conversation
cache.yml uploads _build as the build-cache artefact, but without include-hidden-files: true. Since upload-artifact v4.4.0 (August 2024) hidden files and directories are excluded by default, so _build/.jupyter_cache has been dropped from every build-cache artefact. With execute_notebooks: cache, ci.yml and publish.yml have therefore been executing every notebook from an empty cache. lecture-python.myst and lecture-python-intro already set this on the same step. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
The change is minimal, correctly scoped to the artifact upload step, and aligns with the stated goal of preserving _build/.jupyter_cache in the build-cache artifact.
Pull request overview
Ensures the GitHub Actions “build-cache” artifact produced by cache.yml includes Jupyter’s execution cache directory under _build (a hidden dot-directory), so downstream workflows that download this artifact can actually reuse notebook execution results.
Changes:
- Set
include-hidden-files: truefor the_buildartifact upload step incache.yml.
File summaries
| File | Description |
|---|---|
.github/workflows/cache.yml |
Updates the artifact upload step to include hidden directories like _build/.jupyter_cache in the build-cache artifact. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
✅ Deploy Preview for sparkly-moxie-de1f0e ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for adorable-cucurucho-69162b ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
cache.ymluploads_buildas thebuild-cacheartefact thatci.ymlandpublish.ymldownload. It does not setinclude-hidden-files: true. Since upload-artifact v4.4.0 (August 2024), hidden files and directories are left out by default. So_build/.jupyter_cache, which is a dot-directory, has been missing from every build-cache artefact. This PR adds the one input that keeps it.The repository builds with
execute_notebooks: cache, so the missing directory matters. Two checks confirm it:build-cacheartefact (id 10331930533, from the scheduled run on 2026-09-14) contains onlyhtml/(112 entries) andjupyter_execute/(25 entries). There is no.jupyter_cachein it.ci.ymlrun (33453492567, 2026-09-01), myst-nb reportsUsing jupyter-cache at: .../_build/.jupyter_cache, then executes all eight notebook lectures from scratch, with cache IDs counting up fromCached executed notebook: ID=1. The cache it downloaded was empty.lecture-python.myst and lecture-python-intro set
include-hidden-files: trueon the same step (Upload "_build" folder (cache),actions/upload-artifact@v7,path: _build). This PR brings continuous_time_mcs into line with them.actions/upload-artifact@v7(cache.yml,Upload "_build" folder (cache))include-hidden-filesinputfalse)trueNo
uses:line changes. The diff also adds the newline that was missing at the end of the file, because the new line is appended afterpath: _build.Behaviour change. Once a main-branch artefact carries the cache, PR builds reuse the weekly outputs for lectures whose code cells have not changed, instead of executing them again. A lecture re-executes when its code changes. Package updates alone do not re-execute a cached lecture, but the weekly
cache.ymlrun still builds from scratch, because it does not download the artefact. The other lecture repositories already work this way. The input also brings in_build/.doctrees(Sphinx'senvironment.pickleand one doctree per page, 12 files), the other hidden directory under_build. lecture-python.myst and lecture-python-intro carry it too, because their upload step is identical. A fresh checkout gives every source file a newer mtime than the saved environment, so Sphinx still re-reads every page.What CI proves. This PR's own CI runs
ci.yml, which is unchanged and still downloads main's artefact, so it does not exercise the change. I dispatchedcache.ymlon this branch to prove it. Run 35319345874 passed. Its log showsinclude-hidden-files: trueand 160 files uploaded, and its artefact (id 10536696795, 5.16 MB, up from 4.18 MB) now contains.jupyter_cache/global.dband eight executed notebooks under.jupyter_cache/executed/. That artefact cannot reachci.ymlorpublish.yml, which both download withbranch: main. After merge, the next scheduledcache.ymlrun on main (Mondays 03:00 UTC) produces the first artefact with the cache in it, and theci.ymlrun after that should reuse cached notebooks rather than execute them.The
dawidd6/action-download-artifactv23 to v24 bump is in Dependabot's #233 and is not repeated here. This came out of the read-only sweep for the Node 24 upgrade across the lecture fleet, approved on 2026-09-18. It predates that upgrade and is unrelated to Node.🤖 Generated with Claude Code