Every E2E suite that creates server-side state uses a hardcoded name, and setup_file deletes those names before running to clear leftovers from previous runs. So two runs against the same server delete each other's fixtures mid-test.
This bites the Enterprise job, which targets one shared remote server and runs on every PR. Nothing serialised it, so any two PRs with overlapping CI were corrupting each other — presenting as unexplained E2E failures on unrelated PRs. #114 added a concurrency group as a stopgap, so overlapping runs queue rather than interleave. That has its own cost: a queued run can be superseded and report cancelled.
The OSS matrix is unaffected, since each leg downloads and runs its own private server.
Fix
Give every created fixture a run-scoped suffix from a single variable, defaulted so local runs need no setup.
Most of this is small — six of the seven affected suites already keep their names in variables:
| suite |
work |
workflow, task, secret, search, rerun |
suffix the vars, plus a couple of stray literals each |
api_gateway |
same, plus the hardcoded --path '/api/e2e' |
schedule |
real refactor — no variables, 55 inline literals |
Three parts aren't just renaming:
cli_e2e_test_workflow, cli_e2e_test_workflow_2 and cli_e2e_test_task live in on-disk JSON under test/e2e/, so they need templating per run — write a copy to a scratch dir rather than mutating the tracked fixtures, or a failed run leaves the repo dirty.api_gateway collides on the gateway path as well as the service name. Worth auditing for other non-name uniqueness.- Namespaced fixtures stop being cleaned by the next run's blanket delete, so
teardown_file needs to remove its own namespace and something needs to sweep orphans — otherwise the shared server accumulates junk indefinitely.
For schedule.bats, extract the variables in one behaviour-preserving commit and apply the suffix in another. A semantic change buried in a 55-site rename is hard to review, and the failure mode is a test passing because it no longer asserts on anything it created.
Verifying it
Start one local OSS server and run two full suites against it simultaneously; both should pass. Both fail today, so that is the regression test. Also kill a run mid-flight and check the orphan sweep reclaims its fixtures.
Done when
- Two concurrent runs against one server both pass
- No suite deletes a resource it did not create
- An aborted run leaves nothing that breaks a later run or grows without bound
- Local runs still work with no environment variable set
- The
concurrency block on e2e-enterprise is removed, since it exists only to work around this
Every E2E suite that creates server-side state uses a hardcoded name, and
setup_filedeletes those names before running to clear leftovers from previous runs. So two runs against the same server delete each other's fixtures mid-test.This bites the Enterprise job, which targets one shared remote server and runs on every PR. Nothing serialised it, so any two PRs with overlapping CI were corrupting each other — presenting as unexplained E2E failures on unrelated PRs. #114 added a
concurrencygroup as a stopgap, so overlapping runs queue rather than interleave. That has its own cost: a queued run can be superseded and report cancelled.The OSS matrix is unaffected, since each leg downloads and runs its own private server.
Fix
Give every created fixture a run-scoped suffix from a single variable, defaulted so local runs need no setup.
Most of this is small — six of the seven affected suites already keep their names in variables:
workflow,task,secret,search,rerunapi_gateway--path '/api/e2e'scheduleThree parts aren't just renaming:
cli_e2e_test_workflow,cli_e2e_test_workflow_2andcli_e2e_test_tasklive in on-disk JSON undertest/e2e/, so they need templating per run — write a copy to a scratch dir rather than mutating the tracked fixtures, or a failed run leaves the repo dirty.api_gatewaycollides on the gateway path as well as the service name. Worth auditing for other non-name uniqueness.teardown_fileneeds to remove its own namespace and something needs to sweep orphans — otherwise the shared server accumulates junk indefinitely.For
schedule.bats, extract the variables in one behaviour-preserving commit and apply the suffix in another. A semantic change buried in a 55-site rename is hard to review, and the failure mode is a test passing because it no longer asserts on anything it created.Verifying it
Start one local OSS server and run two full suites against it simultaneously; both should pass. Both fail today, so that is the regression test. Also kill a run mid-flight and check the orphan sweep reclaims its fixtures.
Done when
concurrencyblock one2e-enterpriseis removed, since it exists only to work around this