Skip to content

lecture-python.myst: CPU build time sits at the lecture-validation ceiling #196

Description

@quantecon-services

To discuss: lecture-python.myst's CPU execution time, which is the second of the two causes behind the red lecture validation in #159 (diagnosis). The first cause, bayes_nonconj under numpyro 0.22.0, has its own lecture fix.

What we know

  • The limit bites often. In test-containers-lectures.yml, the lecture's HTML build takes 72–118 min on CPU, against a job limit of 120. Legs were cancelled at 120 min on 08-03, 08-10, 08-17, 08-24, 08-31, 09-07, 09-18 and 09-21. The 09-14 green run had 12–15 min to spare.
  • Three lectures dominate:
    • imp_sample takes 1,900–2,993 s, which is 35–46% of the HTML build. Its numba prange estimator was rewritten to JAX vmap with jax.random.beta in caf3ddc (2026-08-02). Locally, the old numba version runs the whole notebook in about 182 s. The JAX version was still inside one cell after 24 min.
    • phillips_drifts_volatilities takes 402–1,125 s.
    • sargent_surico, added 2026-08-01, takes 265–615 s.
  • Speed depends on the runner. Identical inputs took 71.9 min on one runner and 116.5 min on another. The runner CPU model is not yet logged.
  • Stopgap in this repo: the validation job gives lecture-python.myst a 180-min limit and records each runner's CPU (PR to follow, referencing this issue). This keeps the signal usable. It does not make the build any faster.

Questions

  1. Should imp_sample be fast on CPU again? Options: bring back the numba estimator, keep JAX but use smaller sample sizes when no GPU is present, or accept the cost.
  2. Is the lecture's own GPU build affected? The validation job runs on CPU only, so it may not show what readers' builds see.
  3. Once the lecture is faster, should the validation job's 180-min limit go back down to 120?

Related: #141 (the ceiling, first reported), #159 (the alert).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    discussOpen-ended team deliberation or a decision to be made

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions