feat(macos): Add the UI test suite - #1010
Merged
Merged
Conversation
JeanMertz
force-pushed
the
pr/uitests
branch
2 times, most recently
from
August 21, 2026 06:31
f35d7c2 to
c065674
Compare
The app's unit tests run inside the app process, which puts three things out of reach: whether a menu item is enabled, what landed on the pasteboard, and what survives a terminate and relaunch. Those are exactly the behaviours that break without anyone noticing, and `QA.md` carried them as a manual checklist. This is the half of that checklist a machine can run. A UI test drives the app through its accessibility tree from a separate process, so `@testable import JP` is unavailable and must not be reached for. Anything checkable in-process belongs in `JPTests`, where it costs milliseconds instead of an app launch. Three things make the suite fast enough to be worth having. A suite shares one launched app, because launching costs about four seconds and the work under test costs milliseconds — so a test leaves the app as it found it, or the suite is ordered so that what one test leaves is what the next expects. Waits are on conditions rather than the clock, and not through `waitForExistence`, which reports an element about a second after it appears whatever the element; asking again finds it in under 100ms, and the timeout becomes the price of a failure rather than of a pass. And every animation goes through one lever the tests turn off, because XCUITest waits for the app to stop moving before each action it synthesizes. Anything the tests need the app to do differently goes through `DebugState`, gated on `#if DEBUG` and off unless an environment variable says otherwise, so a release build has no way to reach it. The pasteboard works this way and is enforced: no test may touch the system pasteboard, a debug build copies wherever `JP_DEBUG_PASTEBOARD` names, and `ClipboardPolicyTests` fails on any spelling of the general one. `just test-app-ui` is the CI job and runs everything even after a failure. While writing a test, run it by name instead. `fmt-app` and `lint-app` arrive here rather than with the app because they name every Swift directory in the repository, and this is the last one to exist. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The app's unit tests run inside the app process, which puts three things
out of reach: whether a menu item is enabled, what landed on the
pasteboard, and what survives a terminate and relaunch. Those are
exactly the behaviours that break without anyone noticing, and
QA.mdcarried them as a manual checklist. This is the half of that checklist a
machine can run.
A UI test drives the app through its accessibility tree from a separate
process, so
@testable import JPis unavailable and must not be reachedfor. Anything checkable in-process belongs in
JPTests, where it costsmilliseconds instead of an app launch.
Three things make the suite fast enough to be worth having. A suite
shares one launched app, because launching costs about four seconds and
the work under test costs milliseconds — so a test leaves the app as it
found it, or the suite is ordered so that what one test leaves is what
the next expects. Waits are on conditions rather than the clock, and not
through
waitForExistence, which reports an element about a secondafter it appears whatever the element; asking again finds it in under
100ms, and the timeout becomes the price of a failure rather than of a
pass. And every animation goes through one lever the tests turn off,
because XCUITest waits for the app to stop moving before each action it
synthesizes.
Anything the tests need the app to do differently goes through
DebugState, gated on#if DEBUGand off unless an environmentvariable says otherwise, so a release build has no way to reach it. The
pasteboard works this way and is enforced: no test may touch the system
pasteboard, a debug build copies wherever
JP_DEBUG_PASTEBOARDnames,and
ClipboardPolicyTestsfails on any spelling of the general one.just test-app-uiis the CI job and runs everything even after afailure. While writing a test, run it by name instead.
fmt-appandlint-apparrive here rather than with the app because they name everySwift directory in the repository, and this is the last one to exist.
Signed-off-by: Jean Mertz git@jeanmertz.com