The test job gets twelve minutes, not seven, as the suite nears 3700 tests - #534
Conversation
…ears 3700 tests A normal run takes about three minutes, but the time spreads unevenly across runners: PR #527 (run 35973409359) and PR #531 (run 35987830425) were cancelled at 7m15s with the suite at 98% and no failure, and the `test` check read as failed. Twelve minutes leaves headroom without hiding a hang, which the per-test bounds still catch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
| # The ceiling is on the job. Without one it inherits GitHub's 360-minute default, which | ||
| # is how a hung run stayed "pending" for six hours instead of turning red (ENG-913). | ||
| timeout-minutes: 7 | ||
| timeout-minutes: 12 |
There was a problem hiding this comment.
tests/test_ci_gates.py:151 pins this value — ("test.yml", "test"): 7 — and the case at :166 asserts timeout == minutes. That case carries no mark, so it runs in this job's own selection, not migration and not fresh_interpreter: the test check turns red on the one-line change that exists to stop it turning red. Bash here would not run pytest for me, so that is read from the file rather than executed. The pin has to move to 12 with the workflow.
| @@ -25,7 +25,7 @@ jobs: | |||
| # 2m32-3m06 (five runs: 35653484356, 35653965653, 35654431223, 35655564859, 35655963815). | |||
There was a problem hiding this comment.
Two prose records of seven survive the change. This comment block opens on line 23 with "Seven, over a Run tests step measured at 2m04-2m31…" — the measurements are still true, as the PR body says, but the first word is the ceiling itself, and it now reads Seven five lines above 12. docs/ci.md:20 carries the other: "under a 7-minute ceiling, against a step measured at 2m04-2m31 on the runner since ENG-980". Either edit both, or drop the leading number here and let the line under it be the value.
There was a problem hiding this comment.
The pin moved in 2b8618ae; these did not. At head .github/workflows/test.yml:23 still opens "Seven, over a Run tests step measured at 2m04-2m31…" five lines above timeout-minutes: 12, and docs/ci.md:20 still reads "under a 7-minute ceiling, against a step measured at 2m04-2m31 on the runner since ENG-980". grep -rn '7-minute|timeout-minutes: 7' docs/ .github/ tests/ returns those two and migrations.yml:45, which is its own job and not in scope. The same commit also added a third record that disagrees with both: tests/test_ci_gates.py:149 now says the step "measured 6m21s on #534". Same either/or as before — edit both, or drop the leading number here and let the line under it be the value.
There was a problem hiding this comment.
Reviewed the one-line ceiling change and everything in the tree that records that ceiling. Two findings, and they share a cause: the number lives in four places and this moves one of them. The pinned one is the one I would least want found by CI — tests/test_ci_gates.py asserts 7, runs in this very job's selection, and turns the test check red on the change written to stop it turning red. Not covering the alternatives the PR body sets aside, sharding and --durations. That is the complete list: two, nothing else.
The test that keeps every job under a timeout still pinned the test job at seven minutes, so raising the workflow alone turned that test red on the PR's own run. The pin follows the workflow, with the measurement that justified it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
Pass 2 — re-checked the two findings from pass 1 against 2b8618ae, nothing else. The gate pin moved with the workflow. The prose records did not: test.yml:23 still opens on Seven and docs/ci.md:20 still says a 7-minute ceiling, and the commit that fixed the pin added a third measurement that disagrees with both. Replied on that thread. One of two still open, no new findings, and the set from pass 1 is those two with nothing else in it. Not covering the other jobs' ceilings in that table — only the test row moved here.
…ured it Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
Reviewed the three changed files at 911c10a7, against the two points from the first pass. Both are closed: tests/test_ci_gates.py:153 pins 12, and grep -rn '7-minute|seven-minute|timeout-minutes: 7' docs/ .github/ tests/ now returns only migrations.yml:45, that job's own ceiling and not in scope, and the new comment's narration of the old value. Nothing new from me — everything added since 2b8618ae is comment and documentation prose. That is the whole set; there is nothing else outstanding here.
The
testjob gets twelve minutes instead of seven.What happens today
.github/workflows/test.ymlcaps thetestjob at 7 minutes. A normal run takes about three, but the time spreads unevenly across runners: PR #527 (run 35973409359) and PR #531 (run 35987830425) were cancelled at 7m15s with the suite at 98% and no failure, and thetestcheck read as failed. The suite is close to 3700 tests.What changes
timeout-minutes: 7→12. The ceiling stays on the job (ENG-913's reason: a hung run must turn red, not sit pending for six hours), with headroom for the suite's growth. The comment above it is unchanged: its measurements are still true and the reason for a ceiling is the same.Not done here: sharding the suite or hunting the slowest files (
--durations); the ticket lists them as alternatives, and the one-line ceiling is enough today.Invisible; CI only.
🤖 Generated with Claude Code