ci: promote BaseballHelm authed smoke to a required PR gate (#372) - #849
Conversation
|
PR title or description contains an excluded keyword. |
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing |
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (3)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Adds a baseball-auth-smoke job to ci.yml that runs the already-built e2e/baseball-smoke.spec.ts + e2e/baseball-onboarding-smoke.spec.ts (with their existing fail-loud auth setup) as a required PR gate, folded into the `all` aggregate — the same promotion mechanism used for Supabase RLS tests (#517) and the import-cycle ratchet (#808). Previously this authenticated coach/player smoke only ran post-merge via playwright.yml's push/manual-only `e2e` job, which stayed untouched. Skips (not fails) on fork/Dependabot PRs, which get no repo secrets; same-repo pushes/PRs must have the required secrets or the job fails loudly. Docs updated to reflect the new hard-gate status and its added CI-minutes cost. Also unmasks the `|| echo "Playwright suite has failures..."` exit-code shim in playwright.yml's advisory "Playwright (chromium)" job so a real failure there turns the job red instead of silently passing (reviewer- flagged on #812 as blocking trust in the e2e gate). That job is advisory, not a required check, so this cannot newly block a merge — it only makes the status honest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dependabot PRs are opened against this repo (not a fork), so the existing head.repo.full_name == github.repository check alone did not exclude them. The job's own docs already claimed a Dependabot skip, but without secrets it would hard-fail on the pull_request branch, blocking the required `all` aggregate on every Dependabot PR. Add an explicit github.actor check and sync the "skips on fork/Dependabot PRs" prose in the job comment, branch-protection.md, and CI_RUNBOOK.md to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
8bc2bab to
cbb01fd
Compare
|
Review fix applied (commit cbb01fd)
Rebased onto `origin/batch/bbh-finish-0714` (clean, no conflicts) and force-pushed. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#372 — BaseballHelm authed CI smoke as hard gate
Status before this PR: everything except the actual gate-wiring already existed.
playwright/baseball-auth.setup.tsfails loudly (throws) when required BaseballHelm auth secrets are missing in CI,e2e/baseball-smoke.spec.ts(non-skippable,@coach/@playertagged) covers Command Center, Calendar, Roster, Stats Center, Performance/Lift, Settings, plus a player→coach denied-capability redirect, ande2e/baseball-onboarding-smoke.spec.tscovers the anonymous onboarding-render slice. All of this ran — but only onmainpushes / manualworkflow_dispatch, viaplaywright.yml'se2ejob, which is advisory. A real authenticated regression could land onmainbefore this ever ran.What this PR does: adds a
baseball-auth-smokejob directly to.github/workflows/ci.yml(notplaywright.yml— jobs can onlyneeds:other jobs in the same workflow file, which is why import-cycles/lint-ratchet were added directly toci.ymltoo) that:playwright.yml).npm run build.playwright.ymlis untouched and keeps running the fuller suite including mobile-viewport regression).npm run seed:baseball:ci.e2e/baseball-onboarding-smoke.spec.ts+e2e/baseball-smoke.spec.tsunder--project=chromium --project=baseball-coach --project=baseball-player, with no|| echosoft-fail.Folded into
ci.yml'sallaggregateneeds:— the same promotion mechanism already used for Supabase RLS tests (#517) and the import-cycle ratchet (#808). No branch-protection/admin change needed;CI / allis already a required context.Fork/Dependabot handling
Public repo → fork PRs get no repo secrets from GitHub. The new job's
if:only runs onpushor onpull_requestwherehead.repo.full_name == github.repository(i.e. not a fork). On a fork PR the job is skipped, not failed —all's aggregate step only treatsfailure/cancelledas blocking, so a skip never blocks an external contribution. Same-repo pushes/PRs must have the secrets configured or the job fails loudly, same as the existing push-triggered version always has.Cost — please read before merging
This is a real, ongoing CI-minutes/wall-clock tradeoff, called out explicitly in the original ticket research as "should be Nick's call, not silently absorbed": it adds a second full
npm run build+ Playwright-chromium install to every same-repo PR, on top of the existingNext buildandSmoke checksbuilds. The job itself is scoped tightly (2 spec files, 3 projects) and budgeted at 30 min, but it's still meaningfully more PR wall-clock time than before. Flagging so this can be revisited if PR turnaround becomes a problem — happy to scope it down (e.g. behind a path filter, or only on PRs touchingsrc/app/baseball/**) as a fast follow if that's preferred.Also: unmask
|| echoexit-code shim in playwright.yml (deferred review item)playwright.yml's advisory "Playwright (chromium)" job hadrun: timeout 45m npx playwright test --project=chromium || echo "Playwright suite has failures or timed out — see report artifact"— masking the exit code for the entire broad chromium suite. Reviewer-flagged on #812 as blocking trust in the e2e gate (a real failure looked identical to a clean pass in the Actions UI). Removed the|| echo.Blast radius: this job is advisory, not a required check (
docs/CI_RUNBOOK.md/.github/branch-protection.md), so unmasking it cannot newly block a merge — it only makes the job's own GitHub Actions status honestly reflect pass/fail. Report/results-json artifact upload steps still run regardless (if: always()). However: the first red run after this merges needs a human watching — there may be specs in the broader suite that have been silently failing under the mask for a while, and this will be the first time that surfaces. I did not attempt to identify or fix any pre-existing failures; that's follow-up work once real signal exists.I deliberately left the other two similar
|| echosoft-fail shims in this file (thepicker-screenshotsjob'scourse-library.spec.tsrun, and the advisorybaseball-smokejob'sbaseball-stats-smoke.spec.tsrun) untouched — out of scope for tonight, noting them as a follow-up if wanted.Docs updated
docs/CI_RUNBOOK.md—all(CI) row now lists the new BaseballHelm authenticated smoke; added a note on the unmasked Playwright status; added the new job's wait-window + skip-is-not-a-failure notes..github/branch-protection.md—CI / allhard-aggregate bullet list now includesBaseballHelm authenticated smoke; added a promotion note mirroring the existing fix(baseball): dedup baseball_event_acknowledgements RLS policies (unblocks Supabase CI on main) #517 Supabase-RLS note.Gate evidence
actionlint .github/workflows/ci.yml .github/workflows/playwright.yml— clean, no findings.python3 -c "yaml.safe_load(...)"on both touched workflow files — parses clean.Deferred / not in this PR
baseball-auth-smokejob's cost down (path-filter, etc.) if the added PR wall-clock time turns out to be a problem.Do not merge — for review.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com