feat(tools): Add tools for driving and profiling the macOS app - #1012
Open
JeanMertz wants to merge 14 commits into
Open
feat(tools): Add tools for driving and profiling the macOS app#1012JeanMertz wants to merge 14 commits into
JeanMertz wants to merge 14 commits into
Conversation
`jp_cli::load_workspace` assembled a workspace from disk by hand: find the root, load the ID, build the filesystem backend, wire the user-local silo, then store the ID back. That is workspace knowledge living in the CLI, and a second consumer has to repeat it. Repeating it wrong is silent: wiring the filesystem backend without user-local storage still compiles and still returns conversations, just fewer of them. `Workspace::open` owns that sequence now, with `DEFAULT_STORAGE_DIR` moved alongside it. A new `Error::WorkspaceNotFound` carries the directory that was searched, so the CLI keeps printing its `jp init` hint while other callers receive a typed error rather than a formatted string. The two in-memory constructors become `in_memory` and `in_memory_with_id`, so each constructor says which side of the disk it sits on. Two accessors come with them: `fs_storage` returns the filesystem backend an opened workspace holds, and `sessions` returns the session backend, which the `--no-persist` path wraps in `ReadOnlySessionBackend` instead of reaching for the filesystem backend that happened to be serving that role. Behavior is unchanged. A test covers the wiring that is otherwise invisible when wrong: a conversation created with `--local` lives only in the user-local silo, and a workspace opened from disk must list it. Signed-off-by: Jean Mertz <git@jeanmertz.com>
A reader outside this workspace — the FFI boundary, a plugin, a web view — needs two things the crate did not offer. It needs timestamps in a format its platform can parse, and it needs to know what an entry in the stream is without decoding the storage encoding itself. `rfc3339` formats a timestamp the one way every platform's date parser accepts. Storage keeps timestamps in `time`'s human-readable spelling (`2024-09-01 10:00:00.0`), which nothing outside Rust reads, and each reader otherwise reinvents the conversion. Sub-second precision is kept when the value has any. `rfc3339_str` does the same for a caller holding raw JSON rather than a typed event, returning `None` for a value that parses as neither spelling so the caller can leave it as it found it. `EventKind::type_tag` returns the tag serde writes, which is what a reader switches on. `as_str` returns the Rust name and is for messages addressed to somebody reading this code; the two were the same string by coincidence and nothing held them that way. Two tests do now: one checks every variant's tag against what serde actually serializes, the other against `TYPE_TAGS`, the list the deserializer uses to decide whether an entry is a known event. A tag missing from that list makes its variant unreachable, and the stream keeps every such event as raw JSON instead. `StreamEntry` and `ConversationStream::iter_entries` give a borrowed view of every entry in order, including the config deltas, compaction overlays and unrecognized entries that `iter` and `iter_events_by_turn` skip. Those are part of what happened, and a reader presenting the stream to somebody wants them. Nothing is copied or re-encoded on the way out. Signed-off-by: Jean Mertz <git@jeanmertz.com>
A plugin listing conversations received the title, the last activation time and the event count, but nothing about pinning. A pinned conversation was indistinguishable from any other, so a plugin could neither mark it nor sort by it without opening every conversation to find out. `ConversationSummary` carries the `pinned_at` timestamp the conversation metadata already holds. The field is skipped when serializing an unpinned conversation and defaults to absent when reading, so a plugin built against the older shape keeps working and one built against the newer shape reads an older host's messages. Signed-off-by: Jean Mertz <git@jeanmertz.com>
A native reader cannot call `jp_workspace` without a boundary, and the options for building one are all worse than a direct link. Shelling out to `jp --format json` binds to shapes `jp_cli::output` derives from table rows, which are not stable and would be frozen by Hyrum's Law the moment anything depended on them, and pays a full workspace load per read. Reading `.jp/` from the other language reimplements the storage format twice over and makes the on-disk layout a public contract. A sidecar process buys failure isolation that a read-only viewer does not need, at the cost of supervision and a protocol. `jp_ffi` compiles as a static library beside an rlib and exposes six entry points: open a workspace, list its conversations, read one conversation's turns, close the handle, free a returned string, and collect the last error. The rlib is what lets those entry points be unit-tested in-process rather than only through a linked app. Four rules hold the boundary together. Every entry point catches panics, because unwinding into the calling language is undefined behavior; a caught panic becomes a null return and a message. Failures return null and leave that message in a thread-local slot for `jp_last_error`. Only owned data crosses: a read copies out and drops its lock guard before returning, so no guard, reference or borrow escapes. And Rust frees what Rust allocates, which is what `jp_string_free` is for. Reads also measure their own phases and report them through an optional out-parameter. The timings ride back on the call that produced them rather than on a call of their own, so they cannot be attributed to the wrong read when two overlap, and they are durations rather than timestamps because the caller already has a clock and two that nearly agree are worse than one. Measuring happens here; writing does not. `display` decides what a reader shows. Which events carry prose, and where turn boundaries fall, are judgements about the conversation model rather than about any one reader: events before the first `TurnStart` form an implicit leading turn, and a `TurnStart` opens a new one only when the turn before it holds something. Neither is recoverable from the event shape, so both belong on this side rather than being re-derived by every reader. It reads the typed stream instead of a serialized copy, which would base64-encode the fields storage encodes only to decode them again, reparse every timestamp, and allocate a second copy of the conversation to read four fields off it. The crate depends on `jp_workspace`, `jp_conversation` and `jp_plugin`. Not `jp_config`, and not `jp_cli`: a reader has no per-tool style to apply, no reasoning display mode and no hidden-tool filtering, so keeping `jp_config` out keeps the layered load pipeline out with it. `just build-ffi` builds the library, stages it with a generated header under `apps/macos/.build/<profile>`, and is what Xcode invokes from a build phase so there is one build entry point rather than two competing ones. It asks cargo where it writes rather than assuming `./target`, because the target directory is redirectable and sibling worktrees here share one outside the checkout. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Conversations are readable from the terminal and, through the plugin system, from a browser. Neither suits browsing: scrolling back through weeks of turns, skimming several conversations side by side, or reading on the machine where the work happened. This is a real Mac app for that, with native windows, tabs, menus and state restoration rather than a web view in a window. It is a viewer. It opens a workspace and reads conversations, never writes them, and has no compose field. A window shows the conversation list beside the transcript, split by a divider whose position is restored across launches, and opening any directory inside a workspace opens that workspace, matching how `jp` itself walks up to find `.jp`. The scene is a plain `WindowGroup`. Keying it by workspace path made a window's identity its workspace, so ⌘N on a workspace already on screen brought that window forward instead of opening one, and ⌘T had nothing to duplicate. Each window decides which workspace it shows and keeps that choice in its own scene storage. Timestamps arrive as text and stay that way. The Rust side emits a fractional-seconds part whenever the stored value has one, and `JSONDecoder`'s `.iso8601` strategy rejects those, so a `Date` on the Swift struct would decode the whole-second case and fail on every real workspace. `ConversationDate` parses at the point of display instead. `ConversationSummary` and the event payloads are hand-maintained mirrors of Rust types with no compiler checking that they agree, so each one has a test pinning the exact JSON it decodes. The timings payload is pinned on both sides — `WorkspaceReaderTests` and `jp_ffi`'s `timing_tests` hold the same literal — and if one is edited alone the other is what says so. `project.yml` is the source of truth for the Xcode project, which is generated by `just gen-app` rather than committed, so targets, build settings and the Rust build phase stay reviewable. Swift 6 language mode with complete concurrency checking is on and warnings are errors, holding the app to the bar the Rust side is held to. Debug builds carry `dwarf-with-dsym` and disable Xcode's debug dylib split, because a profiler pointed at the bundle's executable otherwise reads a launcher stub whose UUID matches nothing in the trace. `just run-app` launches it in the foreground with output attached, and `just open-app` goes through LaunchServices for the AppKit behavior that only a normally registered launch produces. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Debugging the macOS app from a conversation needs a way to read what is actually on screen and act on it. Screenshots answer "what does it look like" but not "what is the button called, is it enabled, does the menu item exist" — and none of that is reachable from the app's own test bundle, which runs inside the process and cannot see menu enablement, the pasteboard, or a relaunch. `jpdrive` reads and drives another application through its accessibility tree, and writes one JSON document to stdout either way: a result, or an error with a non-zero exit status. It is what the `debug_app_*` tools shell out to. A standalone SwiftPM package rather than a target in the app's Xcode project, so the binary lands at a predictable path and nothing has to search derived data for it. The logic lives in a `DriveKit` library with a one-line executable on top, because SwiftPM cannot cleanly test an executable target and the tree traversal is where the bugs are. The tests run against a fake accessibility tree, so they need no running app and no accessibility grant. The package mirrors the app's strictness: Swift 6 language mode, `ExistentialAny`, and warnings as errors. SwiftPM 6.0 has no first-class setting for the last of those, so it goes through `unsafeFlags`, which is rejected only for a package consumed as a dependency — this one never is. Accessibility is gated by TCC, and a grant given to a terminal does not obviously reach a tool that terminal started. `just drive-doctor` reports whether the calling process may read another app's tree, so that question is answered by running it under each host rather than guessed at. Signed-off-by: Jean Mertz <git@jeanmertz.com>
The app's unit tests run inside the app process, which puts three things out of reach: whether a menu item is enabled, what landed on the pasteboard, and what survives a terminate and relaunch. Those are exactly the behaviours that break without anyone noticing, and `QA.md` carried them as a manual checklist. This is the half of that checklist a machine can run. A UI test drives the app through its accessibility tree from a separate process, so `@testable import JP` is unavailable and must not be reached for. Anything checkable in-process belongs in `JPTests`, where it costs milliseconds instead of an app launch. Three things make the suite fast enough to be worth having. A suite shares one launched app, because launching costs about four seconds and the work under test costs milliseconds — so a test leaves the app as it found it, or the suite is ordered so that what one test leaves is what the next expects. Waits are on conditions rather than the clock, and not through `waitForExistence`, which reports an element about a second after it appears whatever the element; asking again finds it in under 100ms, and the timeout becomes the price of a failure rather than of a pass. And every animation goes through one lever the tests turn off, because XCUITest waits for the app to stop moving before each action it synthesizes. Anything the tests need the app to do differently goes through `DebugState`, gated on `#if DEBUG` and off unless an environment variable says otherwise, so a release build has no way to reach it. The pasteboard works this way and is enforced: no test may touch the system pasteboard, a debug build copies wherever `JP_DEBUG_PASTEBOARD` names, and `ClipboardPolicyTests` fails on any spelling of the general one. `just test-app-ui` is the CI job and runs everything even after a failure. While writing a test, run it by name instead. `fmt-app` and `lint-app` arrive here rather than with the app because they name every Swift directory in the repository, and this is the last one to exist. Signed-off-by: Jean Mertz <git@jeanmertz.com>
The Rust side of this repository can be built, linted, formatted and tested from inside a conversation. The Swift side could not, which made the app the one place where a change had to be handed back to a human to find out whether it even compiled. These close that gap: each shells out to a toolchain binary from the repository root and reports diagnostics rather than raw build logs, the way the `cargo_*` tools do. Every tool that builds brings its own inputs up to date first — the `jp_ffi` static library, its generated header, and the Xcode project — so there is no setup step to forget and no failure mode where the tools disagree with what `just` would have produced. `xcodebuild` repeats the failing command line in full for every error, so a broken build's tail is almost entirely noise; diagnostics are capped and the head is what survives. `swift_check` builds the app, `swift_format` formats or reports without rewriting, and `swift_test` runs the unit tests and the `jpdrive` package. `swift_test_ui` is separate and requires test names. A UI test takes over the screen for as long as it runs, so the whole bundle is CI's job rather than something a conversation should trigger by accident; asking for the suite by name is the guard. It stops at the first failure and closes the app the run left behind, unless `CI=1` says to finish and report everything. Stopping early needs the process interrupted rather than killed, which `run_until` on the shared runner now supports. `SIGINT` gives `xcodebuild` the chance to tear down its test session, and tearing down that session is what stops the app the test was driving; killed outright, it leaves that app sitting on the screen. Five seconds to unwind, then a kill. A failure copies the screenshots the run recorded into `tmp/uitests/`, because a UI test that failed on something visual is unreadable from its assertion alone. Signed-off-by: Jean Mertz <git@jeanmertz.com>
A profile of the macOS app is only useful if its stacks carry symbols, and `xctrace export` does not hand you those. A Time Profiler row gives the leaf symbol, a count of the frames it withheld, and the rest of the stack as ASLR-slid integers fragmented across rows that reference each other by id. Recovering frames means reassembling those fragments, recovering dyld load addresses from `kdebug` `DBG_DYLD_UUID_MAP_A`/`_B` tracepoint pairs cross-referenced against the `kdebug-strings` table, and only then symbolicating against a dSYM. `xct2cli` does all of it in-process through `addr2line`, `gimli` and `object` rather than by shelling out to `atos`. It is vendored from <https://github.com/landaire/xct2cli> at `9ebb2e0` under MIT OR Apache-2.0, with both licence files kept alongside. Vendored rather than depended on because upstream is one author and a handful of commits, and the `.trace` format is undocumented and Apple's to change, so the maintenance is ours either way. The source is kept close to upstream so it can be re-synced: style lints upstream does not satisfy are allowed by name in its manifest rather than fixed, which keeps a diff against a fresh checkout readable. Four things differ. Swift demangling is added, because upstream carries `rustc-demangle` and `cpp_demangle` only and every Swift symbol otherwise reports as `$s2JP17Conversation…`; `symbol::swift` `dlopen`s `libswiftDemangle.dylib` from the active toolchain and degrades to the mangled name when it is missing. `quick-xml` is 0.41 rather than 0.39, which threads an XML version through `xml::XML_VERSION`. `redact::strip_environment` runs over everything `xctrace` produces. And the binary, terminal renderer, disassembly annotator and hardware counter paths are dropped, taking `capstone`, `annotate-snippets` and `owo-colors` with them; the counter paths needed kperf, which needs root. The redaction is the part worth knowing about. A `.trace` bundle embeds the full environment of the process it recorded, so recording against a shell-launched process captures every API key that shell exported, and `xctrace export --toc` prints them. Stripping `<environment>` at the one point where `xctrace` output is returned means using this library cannot disclose them. It does not sanitise the bundle: the values are still on disk inside it, and a recorded bundle should be treated as credential material and never committed. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Reading a GUI from a conversation is a different problem from reading a CLI. A CLI hands back its output; an app puts pixels on a screen nobody in the conversation can see. Guessing at what the app is doing, or handing every question back to a human, is not a development loop. These tools launch the app, drive it through its accessibility tree, read back what is on screen, capture pixels, take screenshots, record traces, and profile it. Together they answer "what is the app actually doing" without a human in the middle. Profiling goes through `xct2cli`, which turns an Instruments `.trace` bundle into symbolicated hotspots and a callgraph. Driving goes through `jpdrive`, so the accessibility work stays in Swift where the API lives and this side only shells out to it. Two pieces move out of `debug_jp` into `util` because both halves need them. The trace parser becomes `util::trace`: the same `tracing` output is read whether the process under inspection is `jp` or the app, and one parser reading one format is the point. Path shortening becomes `util::paths`, and `debug_jp` switches from relativizing each value before it goes into a report to shortening the finished report in one pass. These reports quote a subprocess's stderr verbatim, render dhat frames carrying source locations, and print trace fields naming whatever was being read — there is no enumerating where an absolute path can turn up, so enumerating was the wrong shape. One pass over the text also covers the artifact paths in the footer, which is why nothing upstream relativizes anything any more. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Squash into the xct2cli commit. Two separate breakages, both from the Swift demangler being loaded at runtime out of an Xcode toolchain. Windows could not compile the crate at all: `RTLD_LAZY`, `RTLD_LOCAL`, `dlopen` and `dlsym` are unix-only in `libc`, and `symbol::swift` imported them unconditionally. The loader and its path search are now `#[cfg(unix)]`, with a `not(unix)` `load` that returns `None` — the same answer a unix host with no toolchain gives. `libc` moves to a `cfg(unix)` dependency, so the platform boundary is visible to cargo rather than only to rustc. `demangle` and `is_mangled` stay platform-independent; they are string work and their tests should run everywhere. Linux compiled but failed an assertion: `a_name_is_demangled_by_whichever_scheme_claims_it` checked a Swift symbol alongside three needing no toolchain, so on a runner without Xcode it read the documented degradation as a failure. The Swift case moves to its own `#[cfg(target_os = "macos")]` test, matching `swift_tests.rs`. What remains needs no toolchain by construction: `is_mangled` rejects a Rust symbol, `main` and the empty string before the loader is reached. Both arms were compiled and tested by temporarily inverting the `unix` gates on macOS. The `not(unix)` arm builds warning-free, its 25 tests pass with the loader disabled, and the two toolchain tests fail — so neither is passing vacuously. Signed-off-by: Jean Mertz <git@jeanmertz.com>
…ndles Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carries a duplicate of the xct2cli commit so this branch builds before pr/xct2cli merges. Drop it on the rebase after that lands.