Skip to content

fix(ci): run E2E matrix when CI workflow definitions change - #474

Open
Simone319 wants to merge 1 commit into
mainfrom
fix/ci-detect-changes-workflow-triggers
Open

fix(ci): run E2E matrix when CI workflow definitions change#474
Simone319 wants to merge 1 commit into
mainfrom
fix/ci-detect-changes-workflow-triggers

Conversation

@Simone319

Copy link
Copy Markdown
Contributor

The gap

pr-checks.yml's Detect Changes job gates every expensive E2E job (E2E Templates, E2E Sandbox, E2E Sandbox (VPC Smoke), E2E Production, E2E Supabase, E2E Hosting (...) matrix, Telemetry E2E) on needs.detect-changes.outputs.source-changed == 'true'. That output came from:

git diff --name-only origin/main...HEAD | grep -qE '^(packages|test-apps)/'

So a PR that only edits .github/workflows/** produced source-changed=false and the entire E2E matrix was skipped (and the Build and Test gate treats skipped E2E jobs as passing).

That is how PRs #465/#466 (commits 327f061 / 7401d30) changed CI behaviour — turning stale-stack reuse into fresh CREATEs in cleanup-stacks.yml — without a single E2E job ever running against the new behaviour. The resulting hosting regression (aws-cdk#15891 class) only surfaced later on #391/#460/#459.

The fix

Treat CI-definition changes that actually affect the E2E jobs as source changes:

  • .github/workflows/pr-checks.yml — the workflow defining the E2E jobs themselves
  • .github/workflows/cleanup-stacks.yml — the stale-stack cleanup that determines whether E2E deploys reuse or freshly CREATE stacks
  • .github/actions/ — composite actions the E2E jobs uses: (e.g. seed-telemetry-id)

Alternative considered

A blanket .github/workflows/** rule. Rejected: it would run the full, expensive E2E matrix (6-way hosting matrix + sandbox/production/supabase/telemetry AWS deploys) on every unrelated workflow edit — issue triage, publishing, docs bots, stale-issue lockers. The explicit list is the minimal set that actually influences E2E outcomes.

Notes

  • Diff touches only .github/workflows/pr-checks.yml — nothing under packages/, so no changeset is required (changeset-guard only tracks ^packages/([^/]+)/).
  • YAML parses cleanly; regex verified against sample paths (matches packages/, test-apps/, the two workflows, .github/actions/; does not match e.g. lock.yml or README.md).
  • This PR itself edits pr-checks.yml, so it should now self-trigger the E2E matrix — a live demonstration of the fix.

@Simone319
Simone319 requested a review from a team as a code owner September 1, 2026 20:46
@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: ce498a7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant