ci(e2e): keep Playwright traces and outputs from bazel-remote-gates as a run artifact - #269
Jesssullivan wants to merge 2 commits into
Conversation
An e2e failure in bazel-remote-gates left nothing behind: trace mode was off and the job uploaded no artifact, so today's 1/330 timeout could only be read from ephemeral stdout. Now PLAYWRIGHT_TRACE_MODE=retain-on-failure reaches the test action (.bazelrc --test_env), the Bazel config writes Playwright's output directory into TEST_UNDECLARED_OUTPUTS_DIR so Bazel keeps it in bazel-testlogs/playwright_chromium_e2e/test.outputs/outputs.zip, and an always-run step copies that directory (dereferenced, zip expanded) into a run artifact retained for seven days. No new permissions, no secrets, no change to what is tested.
|
Scope note against TIN-3130 / R336 (GF is retiring mandatory GitHub-Actions-artifact evidence latches because retention quota can block publishers): this upload is evidence, not a latch — no job, gate, or publisher consumes it, |
Run 33203042020 kept the Playwright trace for the failing spec but Bazel then refused the test: 'output tree artifact playwright_chromium_e2e/ test.outputs was not created'. Bazel 8 materializes undeclared outputs as an unzipped tree artifact by default and the spec directory names are non-ASCII. --zip_undeclared_test_outputs makes it one outputs.zip, which the evidence step already expands.
|
Run 33203042020 proved the plumbing (the failing spec's Note for the record: this is the second 30 s |
First bounded PR from the QA LOOK-loop design (2026-08-28): the blog's e2e already runs remotely on the overlay dind pool inside
bazel-remote-gates, but a failure leaves no evidence —PLAYWRIGHT_TRACE_MODEdefaulted tooffand the job had zeroupload-artifactsteps (the one inci.ymlbelongs to Lighthouse inbuild-and-test). Today's 1/330 e2e timeout on #263 could only be read from stdout..bazelrc:test --test_env=PLAYWRIGHT_TRACE_MODE(alongside the existing Chromium env passthrough).playwright.bazel.config.ts:outputDir=$TEST_UNDECLARED_OUTPUTS_DIR/playwrightunderbazel test(falls back totest-results), so traces/screenshots/error-contexts survive the sandbox inbazel-testlogs/playwright_chromium_e2e/test.outputs/outputs.zip.ci.ymlgates job:PLAYWRIGHT_TRACE_MODE: retain-on-failure; anif: always()step copiesbazel-testlogs/playwright_chromium_e2e(dereferenced, zip expanded) intoartifacts/playwright-e2e, thenactions/upload-artifact@v7(SHA-pinned) uploads it, 7-day retention,if-no-files-found: warn.No new permissions or secrets; nothing about what is tested changes. Local:
node scripts/test-workflow-authority.mjsgreen (file reads only). Hosted CI is the proof — the artifact should appear on this PR's own gates run.