chore(workflow): migrate shared, computer, android and ios to rstest - #2958
Open
fi3ework wants to merge 4 commits into
Open
chore(workflow): migrate shared, computer, android and ios to rstest#2958fi3ework wants to merge 4 commits into
fi3ework wants to merge 4 commits into
Conversation
Decouple harmony's production TypeScript config from the test-only import attribute syntax: tsconfig.build.json pins module=ES2020 again so the build output semantics stay put, while tsconfig.json keeps inheriting ESNext for the tests. Drop the two @ts-ignore comments that described the old arrangement. Type the shared less-stub helper's rspack callback as Rspack.Configuration. The satisfies clause alone gives no protection here because tools.rspack is a ConfigChain union, so an explicit annotation is what actually catches typos.
The photon external and the __VERSION__ define are repeated verbatim in every
node-target test config. With five consumers after this batch, hoist them into
scripts/rstest-shared.ts. defineVersion also settles the encoding, which had
drifted between `'${version}'` and JSON.stringify(version) across packages.
Mechanical for 116 of 121 test files: the vitest specifier becomes @rstest/core
and the vi binding becomes rs. The per-package configs move test.* to the
top level, fileParallelism: false to pool.maxWorkers: 1, and
dangerouslyIgnoreUnhandledErrors to errors.unhandled.
Five files needed hand work, all for the same reason: rstest mock factories are
synchronous, so vitest's async (importOriginal) has no counterpart. Four of them
switch to an import attribute (import * as x from 'y' with { rstest:
'importActual' }) plus a synchronous spread, the recipe already used in
harmony.
The fifth is ios agent.test.ts. agentFromWebDriverAgent loads the device class
override through await import(overrideModule) where the specifier is a
user-supplied runtime value, which rstest's build-time mock transform cannot
reach (web-infra-dev/rstest#1454). vi.doMock(..., { virtual: true }) has no
rstest equivalent either, so the two override cases now point at real on-disk
fixture modules and observe them through a global counter. The suite also gains
an assertion that the default device class stays unused, which the mock-based
version could not express.
Test discovery is unchanged from vitest, package for package:
shared 34/441, android 18/357, ios 13/155, computer 14/91.
… see
Three problems that only surface under tsc: neither biome nor nx test type-checks
the AI suites, and the import attribute error needs a cleared tsbuildinfo to
reproduce.
android merge-reports.test.ts read ctx.task.result.state and .startTime. Rstest's
TestResult names the field status and does not carry a start timestamp, so the
duration now derives from the performance.now() stamp the suite already takes in
beforeEach. Same fix already applied to harmony.
computer chrome-extension-bridge.test.ts passed { timeout, retry } as the third
argument. Rstest takes test options in the second position instead.
android's tsconfig.json pinned module=ES2020, which rejects the import attribute
syntax the page and scrcpy-adapter suites now need. Move the pin to
tsconfig.build.json so the production build keeps its module semantics while the
tests inherit ESNext.
fi3ework
force-pushed
the
rstest-migration/03-mid-tier
branch
from
August 12, 2026 09:46
cd359bf to
5c21c84
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third batch of the incremental vitest 3.0.5 -> rstest 0.11.5 migration, after #2921 (four micro-packages) and #2945 (six cold packages). This one covers
shared,computer,androidandios— 117 test files, 1047 unit tests.Remaining after this:
cli+web-integration, thencore, thenstudioplus the vitest teardown.Performance
Roughly break-even overall, with the win concentrated in
shared. Short version: rstest's advantage is compilation, and three of these four packages barely compile — they mostly sleep.iosspends 1.3% of its run building and the rest waiting on real timers inside the device code under test, so there is nothing for a faster bundler to reclaim.sharedis 441 fast pure-function tests where compilation is the dominant cost, and it doubles. Breakdown below the table.Median of 3 runs per cell, runners alternated each round so machine drift hits both equally. Cold clears the bundler caches first.
pnpm -s testequivalent, Apple M3 Max, Node 26.5.0. Both runners exit 0 on every sample.The spread is not noise, and it is the same rule the
chrome-extensionresult in #2945 followed: the gain tracks the compile-to-execute ratio, not the file count. rstest's advantage is compilation, so a suite that barely compiles and mostly waits has nothing to win. rstest reports the split itself:sharedis 441 fast pure-function tests, so compilation is the largest movable cost and it doubles warm.iosspends 1.3% of the run compiling, which caps any bundler-side gain at about 1.3% — inside the measurement noise.Where
iosactually spends its 17s, measured file by file:device.test.tsalone takes 17.38s wall while the whole package takes 17.4s, so it is the critical path and the other 12 files finish alongside it. Inside that file, the 48 cases account for only 2.26s; the remaining ~15s isafterEachcallingawait device.destroy()48 times, about 315ms of real waiting per test, becausesrc/device.tssleeps for real (sleep(100),sleep(500),sleep(800),sleep(2000)) with no fake timers in play.computerandandroidare the same shape.These four are here for uniformity, not speed; the batches that moved the needle were the cold ones in #2945. Note the numbers above are unit tests only —
tests/ai/**is excluded from the default include, so no model or network calls are involved.Test discovery is identical, case by case
Not just the totals — I diffed the full
file > test namelist produced byrstest listagainstvitest liston the pre-migration tree, for every include mode each package has:AITEST=1AI_TEST_TYPE=computerAI_TEST_TYPE=computer-rdpAI_TEST_TYPE=androidAI_TEST_TYPE=iOSThe AI suites need real devices and models so they are not executed here, but
rstest listcompiles them, which is what caught two of the three type problems below.One formatting difference, not a discovery difference: for
test.eachnames, vitest quotes interpolated string values (retries 'a missing dump file') and rstest does not. Normalising the quotes makes all eight modes byte-identical.What is not a mechanical rename
112 of the 117 test files are a pure
vitest->@rstest/corespecifier swap plusvi.->rs.. The rest:1. Async mock factories have no rstest equivalent (android, 3 sites).
rstest'sMockFactoryis synchronous, sovi.mock('x', async (importOriginal) => ...)is a hard error. These switch to the import-attribute recipe already used byharmonyin #2945:page.test.tsx2,scrcpy-adapter.test.tsx1.2.
ios/tests/unit-test/agent.test.tsno longer mocks the device-class override. Two blockers stack here.agentFromWebDriverAgentresolves the override throughawait import(overrideModule)where the specifier is a user-supplied runtime value, which rstest's build-time mock transform cannot reach (web-infra-dev/rstest#1454). And rstest has no virtual-module mocking at all — itsMockModuleOptionsis only{ spy: true } | { mock: true }.The two override cases now point at real
.mjsfixtures undertests/unit-test/fixtures/and observe them through a global counter, which is what crosses the module-realm split between rstest's registry and the native loader. No test is skipped, and the suite gains an assertion that the default device class stays unused — something the mock-based version could not express.Worth noting for reviewers: the old code cast
vi.doMockto a signature vitest does not have ({ virtual: true }is a Jest option; vitest'sMockOptionsis{ spy?: boolean }). Removing that argument on the pre-migration tree leaves all 13 cases passing, so it was dead weight. It also meant the two success-path cases never exercised real module resolution while the failure-path case did.3. Three type errors that neither lint nor
nx testsees. The AI suites are compiled but not type-checked by either, so these only surface undertsc:android/tests/ai/merge-reports.test.tsreadctx.task.result.stateand.startTime. rstest'sTestResultnames the fieldstatusand carries no start timestamp, so the duration now derives from theperformance.now()stamp the suite already takes inbeforeEach. Same fix asharmonyin chore(workflow): migrate six packages to rstest #2945.computer/tests/ai/chrome-extension-bridge.test.tspassed{ timeout: 20 * 60 * 1000, retry: 0 }as the third argument. rstest takes options in the second position; in the third position the object is silently ignored, so the suite would have quietly fallen back to the config's 3-minutetestTimeout. Verified with a standalone repro: third position leavesretryCount: 0, second position honours it.android/tsconfig.jsonpinnedmodule: ES2020, which rejects the import-attribute syntax item 1 needs. The pin moves totsconfig.build.jsonso the production build keeps its module semantics while the tests inheritESNext. A staletsconfig.tsbuildinfomasks this error, which is worth knowing if you reproduce locally.Config translation
Per package:
test.*moves to the top level,ssr.externalbecomesoutput.externals,definebecomessource.define,fileParallelism: falsebecomespool: { maxWorkers: 1 }(computer, android), anddangerouslyIgnoreUnhandledErrorsbecomeserrors: { unhandled: false }(android, ios).computer's explicitenvironment: 'node'is preserved astestEnvironment: 'node'rather than dropped as redundant.scripts/rstest-shared.tsis new: the photon external and the__VERSION__define were repeated verbatim in every node-target config. With five consumers it is now defined once, andharmonymoves over too. This was deliberately deferred from #2945, whereharmonywas the only consumer.Also in this PR, the follow-ups from the #2945 review:
harmony/tsconfig.build.jsonpinsmodule: ES2020again so the production build is decoupled from the test-only import-attribute syntax, the two stale@ts-ignorecomments are gone, andscripts/rstest-style-stub.tstypes its rspack callback asRspack.Configuration(the suggestedsatisfies Pick<RstestConfig, 'tools'>alone catches nothing, becausetools.rspackis aConfigChainunion).Validation
All green. Discovery parity was checked with
rstest list/vitest listagainst a worktree of the pre-migration tree, as tabled above.Not run: the AI suites themselves, which need real devices and model credentials.