Skip to content

ci: promote BaseballHelm authed smoke to a required PR gate (#372) - #849

Merged
njrini99-code merged 2 commits into
batch/bbh-finish-0714from
task/ci-gates-372-373
Jul 15, 2026
Merged

ci: promote BaseballHelm authed smoke to a required PR gate (#372)#849
njrini99-code merged 2 commits into
batch/bbh-finish-0714from
task/ci-gates-372-373

Conversation

@njrini99-code

Copy link
Copy Markdown
Owner

#372 — BaseballHelm authed CI smoke as hard gate

Status before this PR: everything except the actual gate-wiring already existed. playwright/baseball-auth.setup.ts fails loudly (throws) when required BaseballHelm auth secrets are missing in CI, e2e/baseball-smoke.spec.ts (non-skippable, @coach/@player tagged) covers Command Center, Calendar, Roster, Stats Center, Performance/Lift, Settings, plus a player→coach denied-capability redirect, and e2e/baseball-onboarding-smoke.spec.ts covers the anonymous onboarding-render slice. All of this ran — but only on main pushes / manual workflow_dispatch, via playwright.yml's e2e job, which is advisory. A real authenticated regression could land on main before this ever ran.

What this PR does: adds a baseball-auth-smoke job directly to .github/workflows/ci.yml (not playwright.yml — jobs can only needs: other jobs in the same workflow file, which is why import-cycles/lint-ratchet were added directly to ci.yml too) that:

  • Installs Playwright chromium (same cache pattern as playwright.yml).
  • Runs npm run build.
  • Reuses the existing "Compute BaseballHelm auth readiness" + "Fail missing BaseballHelm secrets" steps verbatim (copied, not moved — the push/manual job in playwright.yml is untouched and keeps running the fuller suite including mobile-viewport regression).
  • Runs npm run seed:baseball:ci.
  • Runs only e2e/baseball-onboarding-smoke.spec.ts + e2e/baseball-smoke.spec.ts under --project=chromium --project=baseball-coach --project=baseball-player, with no || echo soft-fail.

Folded into ci.yml's all aggregate needs: — the same promotion mechanism already used for Supabase RLS tests (#517) and the import-cycle ratchet (#808). No branch-protection/admin change needed; CI / all is already a required context.

Fork/Dependabot handling

Public repo → fork PRs get no repo secrets from GitHub. The new job's if: only runs on push or on pull_request where head.repo.full_name == github.repository (i.e. not a fork). On a fork PR the job is skipped, not failed — all's aggregate step only treats failure/cancelled as blocking, so a skip never blocks an external contribution. Same-repo pushes/PRs must have the secrets configured or the job fails loudly, same as the existing push-triggered version always has.

Cost — please read before merging

This is a real, ongoing CI-minutes/wall-clock tradeoff, called out explicitly in the original ticket research as "should be Nick's call, not silently absorbed": it adds a second full npm run build + Playwright-chromium install to every same-repo PR, on top of the existing Next build and Smoke checks builds. The job itself is scoped tightly (2 spec files, 3 projects) and budgeted at 30 min, but it's still meaningfully more PR wall-clock time than before. Flagging so this can be revisited if PR turnaround becomes a problem — happy to scope it down (e.g. behind a path filter, or only on PRs touching src/app/baseball/**) as a fast follow if that's preferred.

Also: unmask || echo exit-code shim in playwright.yml (deferred review item)

playwright.yml's advisory "Playwright (chromium)" job had run: timeout 45m npx playwright test --project=chromium || echo "Playwright suite has failures or timed out — see report artifact" — masking the exit code for the entire broad chromium suite. Reviewer-flagged on #812 as blocking trust in the e2e gate (a real failure looked identical to a clean pass in the Actions UI). Removed the || echo.

Blast radius: this job is advisory, not a required check (docs/CI_RUNBOOK.md / .github/branch-protection.md), so unmasking it cannot newly block a merge — it only makes the job's own GitHub Actions status honestly reflect pass/fail. Report/results-json artifact upload steps still run regardless (if: always()). However: the first red run after this merges needs a human watching — there may be specs in the broader suite that have been silently failing under the mask for a while, and this will be the first time that surfaces. I did not attempt to identify or fix any pre-existing failures; that's follow-up work once real signal exists.

I deliberately left the other two similar || echo soft-fail shims in this file (the picker-screenshots job's course-library.spec.ts run, and the advisory baseball-smoke job's baseball-stats-smoke.spec.ts run) untouched — out of scope for tonight, noting them as a follow-up if wanted.

Docs updated

Gate evidence

  • actionlint .github/workflows/ci.yml .github/workflows/playwright.yml — clean, no findings.
  • python3 -c "yaml.safe_load(...)" on both touched workflow files — parses clean.
  • No TypeScript/test-helper files touched in this PR, so typecheck/eslint/vitest gates are not applicable per the task rails.

Deferred / not in this PR

Do not merge — for review.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

@greptile-apps

greptile-apps Bot commented Jul 15, 2026

Copy link
Copy Markdown

PR title or description contains an excluded keyword.

@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@vercel

vercel Bot commented Jul 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
helmv3 Ignored Ignored Preview Jul 15, 2026 9:27am

Request Review

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)
  • main
  • develop
  • release/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0f22ea3c-fc78-4a50-a798-3b28bbfc6b0f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch task/ci-gates-372-373
  • 🛠️ helm safety pass
  • 🛠️ dashboard ux pass
  • 🛠️ rls test pass

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@supabase

supabase Bot commented Jul 15, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project qmnssrrolpinvwjjnufo because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

Fable Integrator and others added 2 commits July 15, 2026 05:22
Adds a baseball-auth-smoke job to ci.yml that runs the already-built
e2e/baseball-smoke.spec.ts + e2e/baseball-onboarding-smoke.spec.ts (with
their existing fail-loud auth setup) as a required PR gate, folded into
the `all` aggregate — the same promotion mechanism used for Supabase RLS
tests (#517) and the import-cycle ratchet (#808). Previously this
authenticated coach/player smoke only ran post-merge via playwright.yml's
push/manual-only `e2e` job, which stayed untouched.

Skips (not fails) on fork/Dependabot PRs, which get no repo secrets;
same-repo pushes/PRs must have the required secrets or the job fails
loudly. Docs updated to reflect the new hard-gate status and its added
CI-minutes cost.

Also unmasks the `|| echo "Playwright suite has failures..."` exit-code
shim in playwright.yml's advisory "Playwright (chromium)" job so a real
failure there turns the job red instead of silently passing (reviewer-
flagged on #812 as blocking trust in the e2e gate). That job is advisory,
not a required check, so this cannot newly block a merge — it only makes
the status honest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dependabot PRs are opened against this repo (not a fork), so the existing
head.repo.full_name == github.repository check alone did not exclude them.
The job's own docs already claimed a Dependabot skip, but without secrets
it would hard-fail on the pull_request branch, blocking the required `all`
aggregate on every Dependabot PR. Add an explicit github.actor check and
sync the "skips on fork/Dependabot PRs" prose in the job comment,
branch-protection.md, and CI_RUNBOOK.md to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@njrini99-code
njrini99-code force-pushed the task/ci-gates-372-373 branch from 8bc2bab to cbb01fd Compare July 15, 2026 09:27
@njrini99-code

Copy link
Copy Markdown
Owner Author

Review fix applied (commit cbb01fd)

  • mustFix: `.github/workflows/ci.yml` job `baseball-auth-smoke` (~line 339-341) `if:` did not exclude Dependabot PRs — Dependabot opens PRs against this repo (not a fork), so the existing `head.repo.full_name == github.repository` check alone let them through to hard-fail on missing secrets, blocking the required `all` aggregate on every Dependabot PR despite the job's own comment already claiming a Dependabot skip.
    • Fix: added `&& github.actor != 'dependabot[bot]'` to the `pull_request` branch of the condition.
    • Also synced the "skips on fork/Dependabot PRs" prose so it matches the corrected behavior in: the job's own comment (ci.yml lines ~327-334), `.github/branch-protection.md`'s `baseball-auth-smoke` bullet, and `docs/CI_RUNBOOK.md`'s corresponding notes.
    • Verified with `actionlint` (clean, both the single file and full `.github/workflows/` tree) and `npm run typecheck` (clean — no TS/JS files touched, docs/workflow-only change).

Rebased onto `origin/batch/bbh-finish-0714` (clean, no conflicts) and force-pushed.

@njrini99-code
njrini99-code merged commit 6a22f79 into batch/bbh-finish-0714 Jul 15, 2026
28 of 32 checks passed
@njrini99-code
njrini99-code deleted the task/ci-gates-372-373 branch July 15, 2026 10:02
njrini99-code pushed a commit that referenced this pull request Jul 15, 2026
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant