diff --git a/docs/testing/p02-virtualhid-output-proof.md b/docs/testing/p02-virtualhid-output-proof.md index 00d06ff30..a0eb873d0 100644 --- a/docs/testing/p02-virtualhid-output-proof.md +++ b/docs/testing/p02-virtualhid-output-proof.md @@ -2,40 +2,233 @@ ## Current result -P02 is not yet proven. A disposable managed macOS 15.7.7 clone now proves the -complete healthy runtime, exact q-to-w configuration, and independently -observed target-app output. The target received `q`, not `w`, because Tart's -VNC input did not appear as a guest HID device: Kanata reported -`InputGrab active=true devices=0`. This is an input-source limitation in the -lab, not a confirmed KeyPath remapping defect. - -The next proof must inject `q` through a guest-visible physical HID device or -an Apple-authorized virtual HID device and retain TextEdit as the independent -output oracle. A locally signed `IOHIDUserDevice` helper is not sufficient: -macOS requires the restricted -`com.apple.developer.hid.virtual.device` entitlement and AMFI rejects an ad-hoc -signature that merely declares it. Do not weaken the proof to accept KeyPath's -simulation result; simulation correctly reported q-to-w in these runs, while -the real non-HID events bypassed Kanata and produced `q`. - -The legacy PPPC payload draws the launcher and engine Accessibility switches as -managed and enabled, but macOS 26.2 and later no longer honor an Accessibility -grant from that payload. KeyPath's independent oracle correctly finds no -Accessibility TCC record and refuses to call the runtime operational. +**P02 is proven.** On September 15, 2026 a disposable Parallels macOS 26.5.2 +guest received real USB HID keystrokes from the ESP32 fixture and TextEdit +observed the remapped output. This closes the gap every earlier attempt hit: +Tart's VNC input never appeared as a guest HID device, so Kanata reported +`InputGrab active=true devices=0` and the target received the unmapped key. + +Parallels assigns a physical USB device to the guest that owns the session, so +the fixture enumerates inside the guest as a genuine keyboard. The guest's +`IOHIDDevice` tree contains `KeyPath Physical HID Fixture` (`CAFE:4010`) with +`PrimaryUsage 6` and `Transport USB`, and KeyPath's own device monitor logged +`Keyboard connected: KeyPath Physical HID Fixture (CAFE:4010)`. + +### The evidence + +One TextEdit document accumulated every fire, so the transitions are visible in +a single artifact. The document ended at `Qwwazvwbzqbz`: + +| fire | rules live | fixture sent | document gained | +| --- | --- | --- | --- | +| 1 | none | `q` | `q` (TextEdit autocapitalized it to `Q`) | +| 2 | `q -> w` | `q` | `w` | +| 3 | `q -> w` | `qaz` | `waz` | +| 4 | `q -> w`, `a -> b` | `qaz` | `wbz` | +| 5 | `a -> b` only | `qaz` | `qbz` | + +The `v` between fires 3 and 4 is a stray character from a host-side +`prlctl send-key-event` probe, not fixture output. + +Read the fires as one controlled experiment. Fire 1 against fire 2 isolates the +rule: identical device, key, and application, with only the rule changing. +Fires 3 and 4 add selectivity — unmapped keys pass through untouched, and a +second independent rule takes effect the same way. Fire 5 closes the loop: +removing only the `q` rule restored `q` while the untouched `a -> b` rule kept +working, which rules out a confound in the device, the app, or the runtime. + +Kanata logged every physical press and release (`[KeyInput] sent key=q +action=Press`), so the events reached the engine as real input rather than as +synthesized events. + +### Extended matrix on the same lease + +With the fixture attached, the following ran against the same guest and a +TextEdit oracle. All keystrokes were physical USB HID input. + +| test | sent | expected | result | +| --- | --- | --- | --- | +| shift and passthrough | `Q1!q` with `q -> w` | `W1!w` | pass | +| tap-hold, tap | `s` held 40 ms, rule tap `s` / hold `d`, timeout 200 ms | `s` | pass | +| tap-hold, hold | `s` held 400 ms | `d` | pass | +| throughput | 40 characters at 10 ms intervals | all 40, remap applied | pass, `the wuick brown fox jumps over lazy dogs` | +| remap after recovery | `qaz` after a service teardown and repair | `waz` | pass | +| key repeat through a remap | `q` held 1000 ms | repeated `w` | pass, seven `w` | +| key repeat, unmapped control | `z` held 1000 ms | repeated `z` | pass, seven `z` | +| shifted alternate output | rule `e -> r` with shifted `z`, sent `eE` | `rZ` | pass | +| rule disable then enable | `q` after disable, then after enable | `q` then `w` | pass | +| soak | 180 characters at 15 ms | `wz` repeated cleanly | pass, 360 reports, 0 late | +| live reload while typing | rule changed 7 s into a 10 s run | output switches mid-stream | pass, `wy` to `wb`, no loss at the boundary | +| overlapping keys | `q` down, `z` down while held, both released | `wb` | pass | +| home row mod, hold | `f` held 250 ms as Shift, then `a` | `A` | pass | +| home row mod, tap | `f` tapped 40 ms | `f` | pass | +| emergency stop | Ctrl+Space+Esc as a real chord, then `q` | `q`, remapping off | pass | +| layer activation | space leader held 400 ms | enters `nav`, returns to `base` | pass | + +The shift case matters: the remap applies to the physical key and the shift +modifier survives it, so `shift`+`q` yields `W`. Tap-hold is the case +simulation cannot settle, because the verdict depends on real hold duration +crossing the 200 ms timeout. The throughput run reported 82 of 82 USB reports +delivered with zero late reports and 41 microseconds maximum lateness, and +every character reached the app. + +Key repeat survives the remap and is not perturbed by it: a one-second hold +produced seven characters through the remap and seven through an unmapped key, +the same count either way. + +The rule lifecycle works in both directions from the CLI. `rule disable` +restored passthrough and `rule enable` reinstated the mapping, each taking +effect on the next keystroke, which is a different code path from the +add-and-remove cycle above. `--shifted` resolves against the real physical +shift: `e` produced `r` and shift-`e` produced `Z`. + +Holding a remapped key with macOS press-and-hold enabled opens the accent +picker for the **remapped** character (`ŵ`), which is further evidence the +remap is a real HID-level key rather than injected text. Disable +`ApplePressAndHoldEnabled` before measuring repeat. + +KeyPath's keyboard overlay independently rendered `W` on the physical Q key +while the rule was live. + +The live-reload case is the most demanding one here and the most +representative: a rule was applied while the fixture was still typing at about +33 characters per second. The emitted text switched from `wy` to `wb` partway +through with no garbled, dropped, or duplicated characters at the boundary, so +a config reload does not disturb input already in flight. + +Overlapping presses are handled correctly. The fixture's `run-text` emits one +key at a time, but its script format is a raw HID report stream +(` `), so a hand-built script can +hold one key while pressing another. With `q` still down when `z` went down, +the output was `w` then `b` — each key remapped, in the right order. + +That route then proved a home row mod, the feature this lab could never reach +before. With `f` bound to tap `f` / hold `lsft`, holding it 250 milliseconds and +pressing `a` produced `A`, while a 40 millisecond tap produced `f`. Both the +timing threshold and the modifier chord behaved correctly against real hardware. + +**A custom rule can be silently outranked by an enabled collection.** The first +home row mod attempt produced lowercase `a`. `rule add f --tap f --hold lsft +--on-conflict replace` reported `applied: true`, but the generated config still +bound `f` to `layer_home-arrows_f` from the enabled Home Row Arrows collection, +so the custom rule never took effect. Disabling that collection made the rule +active on the next reload and the test passed. Worth checking whether +`--on-conflict replace` should detect a collection-owned key rather than +reporting success. **Always confirm the generated config binds the key you +expect before concluding a rule does not work.** + +The emergency stop was exercised the same way and behaves as documented. A real +Ctrl+Space+Esc chord from the fixture stopped Kanata: `q` produced `w` +immediately before the chord and `q` immediately after, with the process gone +and port 37001 closed. That is the safety guarantee working against physical +hardware rather than a synthesized event. + +**Recovering from the emergency stop took two attempts.** With KeyPath's window +focused, the first activation produced "KeyPath Runtime, powered by Kanata +Engine, failed to start. Click Fix to retry."; the second restored a running +Kanata, an open port 37001, and a launchd service reporting `running`. Worth +understanding why the first attempt fails, since a user hitting the emergency +stop will meet this. Note also that the body text says "Click Fix" while the +button reads "Restart", the same copy mismatch seen on the earlier repair step. + +Layer activation works from physical input too. Holding the space leader key +for 400 milliseconds made Kanata log `Entered layer: (deflayer nav` and then +`(deflayer base` on release, so the leader-key layer resolves on real hold +timing rather than only in simulation. + +Still unproven with physical input: the hyper key and tap-dance. The chord +script is the route to both. + +**Trap: TextEdit autocorrect silently rewrites remapped output.** The first +throughput run appeared to show the remap failing at speed — the document read +`quick`, not `wuick`. Autocorrect had repaired the "misspelling" the remap +produced. Any future text-oracle test must disable +`NSAutomaticSpellingCorrectionEnabled` and `NSAutomaticCapitalizationEnabled` +first, or it will report false results in both directions. + +### Finding: a failed service restart leaves the daemon unregistered + +`keypath-cli service restart`, invoked as the console user from outside a GUI +session, stopped Kanata and then failed with "Could not restart Kanata +service". It did not merely fail to start: `com.keypath.kanata` was gone from +the launchd system domain entirely, so `launchctl kickstart` could not find it. +`keypath-cli system repair` then failed at +`install-required-runtime-services` with `userActionRequired: true`. + +The reporting was correct — the command failed closed and said so — but the +system was left worse off than before the call, and no command-line path +recovered it. + +Recovery came from the app. KeyPath's own UI had already detected the state and +offered a repair step reading "KeyPath runtime is not running. Click Fix to +start it." Activating it restored a running Kanata and TCP readiness, and the +`q -> w` rule was still in force afterwards. + +Two caveats before treating this as a product bug. The invocation was +`sudo -u keypathmdm` from a root guest-control context, which is not how a user +runs the CLI, and re-registering a service legitimately needs user approval. +The part worth investigating is the unregistration side effect on a failed +restart, not the authorization requirement. + +Minor: that repair step's body text says "Click Fix" while its button is +labelled "Start". + +### Lease and artifacts + +Lease `cbx_f4fdf8bd64de`, KeyPath commit +`5fce69baa1c967e05379f7ee8a9a5507aeeca67b`, signed installer SHA-256 +`88fbb04fd987e05dc9d5dfbc9d2967469d7f27cefc63fe67e54c8c12214fe130`, macOS +26.5.2 (`25F84`). Helper `1.3.2` fresh and working, Kanata running, the +VirtualHID driver installed with a healthy daemon and device. Artifacts: + +`/Volumes/KeyPath Lab/CrabBox/KeyPathInstallerLab/artifacts/cbx_f4fdf8bd64de/20260915T135147Z` + +### Deviations to keep in mind + +The artifact was Developer ID signed but **not notarized**, so Gatekeeper +assessed it as rejected. A notarized build has not yet been proven on this path. + +Three steps needed a human at the console and are not yet automated: macOS +Setup Assistant, the KeyPath permission grants (each raises an authenticated +password prompt that this lane cannot drive), and confirming the Parallels USB +attach dialogs. The remap itself, rule application, reload, and evidence +collection were all automated. + +The console session ran as `keypathmdm`, the lab's fallback administrator, +because `keypathqa` in this base carries a SecureToken whose authentication +fails under Parallels guest control. That detail matters when reading results: +`keypath-cli` run through the lab's SSH channel acts as `keypathqa` and reads a +**different** config file from the one Kanata loads for the console user. The +first fire produced `q` precisely because the rule had been written to +`keypathqa`'s config while Kanata was running the console user's. Apply rules +as the console user when validating runtime behavior. + +## Managed-policy limitation, still open + +This is separate from the output proof above and remains true. The legacy PPPC +payload draws the launcher and engine Accessibility switches as managed and +enabled, but macOS 26.2 and later no longer honor an Accessibility grant from +that payload. KeyPath's independent oracle correctly finds no Accessibility TCC +record and refuses to call the runtime operational. [Apple documents the replacement](https://developer.apple.com/documentation/devicemanagement/privacypreferencespolicycontrol/services-data.dictionary) as the declarative `com.apple.configuration.app-settings` configuration. -The smaller alternative is now available: `keypath-macos-15-managed` is a -stopped Tart base running macOS 15.7.7. It is user-approved MDM enrolled and -has the exact three installer-derived device profiles installed. System-level -lane admission passed before the staging image was renamed to the final base. -Disposable-clone admission and exact policy rehydration are now proven. +`keypath-macos-15-managed` is a stopped Tart base running macOS 15.7.7. It is +user-approved MDM enrolled and has the exact three installer-derived device +profiles installed. System-level lane admission passed before the staging image +was renamed to the final base. Disposable-clone admission and exact policy +rehydration are proven. -This is an approval-lane limitation, not a confirmed KeyPath remapping defect. -Do not turn it into a product bug or bypass it by modifying system-extension or +This is an approval-lane limitation, not a KeyPath remapping defect. Do not +turn it into a product bug or bypass it by modifying system-extension or privacy databases. -## Managed macOS 26 evidence on July 23, 2026 +## Earlier attempts + +The runs below predate the proof above and are retained for their lane and +policy findings. + +### Managed macOS 26 evidence on July 23, 2026 Lease `cbx_30fc557d2c01` used macOS 26.5.2 (`25F84`), KeyPath commit `03b3858dd200c9645265f6e9bf519359c834d2e4`, and signed installer SHA-256 @@ -62,7 +255,7 @@ failed closed with Kanata not running or TCP responsive. The lab also extended its Parallels RFB pointer probe to macOS 26; this clone rejected RFB authentication, so no native input assertion was claimed. -## Managed macOS 15 base evidence on July 23, 2026 +### Managed macOS 15 base evidence on July 23, 2026 The `keypath-macos-15-managed` Tart base was built from the clean `ghcr.io/cirruslabs/macos-sequoia-base:latest` source with a new virtual serial @@ -81,7 +274,7 @@ Controller evidence is retained at: `/Volumes/KeyPath Lab/CrabBox/KeyPathInstallerLab/artifacts/base-keypath-macos-15-managed/20260724T013334Z/managed-policy` -## Disposable macOS 15 proof attempt on July 23, 2026 +### Disposable macOS 15 proof attempt on July 23, 2026 Lease `cbx_629d00243876` used KeyPath commit `e13836bae9b0f1a15c7b47cfc8783abad1f9d8a0` and the signed installer with @@ -102,7 +295,7 @@ The exact rule was installed with `keypath-cli rule ensure q w --apply`. The same run's runtime log reported no captured input device. This cleanly separates the working output runtime from the unsuitable Tart VNC input source. -## Tart keyboard-path follow-up on July 23, 2026 +### Tart keyboard-path follow-up on July 23, 2026 Lease `cbx_f7009ff1b833` used KeyPath commit `229ec8014fd708fa4931b63d1c915879bd830f6d` and the same signed installer @@ -133,7 +326,7 @@ An ad-hoc-signed `IOHIDUserDevice` prototype was also rejected before launch. AMFI reported that its restricted entitlements were not validated. The dead prototype was removed rather than retained as a misleading resume path. -## Evidence captured on July 12, 2026 +### Evidence captured on July 12, 2026 The disposable unmanaged proof used lease `cbx_1b376f03fbb6`, macOS 15.7.7 (`24G720`), KeyPath commit @@ -164,7 +357,7 @@ postcondition remained: org.pqrs.Karabiner-DriverKit-VirtualHIDDevice [activated waiting for user] ``` -## Follow-up confirmation +### Follow-up confirmation A second clean unmanaged macOS 15.7.7 lease, `cbx_1a98a85674e7`, repeated the result with the signed KeyPath candidate from commit `7ca790ab`. Its artifacts @@ -189,39 +382,40 @@ working KeyPath-helper approval flow from the unresolved DriverKit activation: do not retry helper/background-item approval when the extension is already in this state. -## Required proof shape +### Required proof shape -P02 passes only when all of the following are true in one disposable lease: +P02 passes only when all of the following are true in one disposable lease. +The September 15, 2026 run above satisfied every point. -1. The approved, normal KeyPath runtime is healthy: Driver Extension enabled, - VirtualHID daemon running, Kanata running, and TCP readiness responding. -2. A deterministic configuration maps physical `q` to virtual `w`. +1. The KeyPath runtime is healthy: the VirtualHID driver installed with a + healthy daemon and device, Kanata running, and TCP readiness responding. +2. A deterministic configuration maps physical `q` to virtual `w`, applied to + the config the running Kanata actually loaded. 3. The harness focuses an independent target app with an observable text value. -4. `keypath-lab desktop-type LEASE --text q` reports the native `vnc-key` - delivery method. -5. The target app's accessibility value changes to `w`, not `q`. +4. The keystroke originates from a device the guest enumerates as a real USB + HID keyboard, confirmed in the guest's `IOHIDDevice` tree. +5. The target app's value changes to `w`, not `q`. Step 5 is the functional assertion. Driver metadata, a successful click, and a KeyPath-local input monitor are useful preparation evidence but are not output -proof. - -## Resume path +proof. Simulation output is never a substitute: it reported q-to-w correctly in +runs where the real key still produced `q`. -Continue with one explicit path: +### What remains -1. Attach or pass through a real USB keyboard to a disposable managed clone, - or obtain an Apple-authorized build carrying - `com.apple.developer.hid.virtual.device`, then repeat the q-to-w TextEdit - proof. -2. Add NanoMDM Declarative Device Management support and publish an +1. Prove the same path with a **notarized** artifact; the September 15 run used + a Developer ID build that Gatekeeper rejects. +2. Automate the three human steps: Setup Assistant, the permission grants that + raise an authenticated password prompt, and the Parallels USB attach + confirmation. Until then this proof needs someone at the console. +3. Add NanoMDM Declarative Device Management support and publish a `com.apple.configuration.app-settings` Accessibility configuration for - macOS 26.2 and later. + macOS 26.2 and later. This is the durable route for the managed lane and is + independent of the output proof. -The macOS 15 base remains the smaller route to P02 only when a real HID input -source is available. The declarative route is the durable macOS 26 and 27 -investment. Do not retry Tart VNC, guest synthesized events, an ad-hoc -restricted entitlement, or the USB-only Tart configuration. Do not continue -treating successful installation of the legacy macOS 26 PPPC payload as an +Do not retry Tart VNC, guest synthesized events, an ad-hoc restricted +entitlement, or the USB-only Tart configuration. Do not continue treating +successful installation of the legacy macOS 26 PPPC payload as an Accessibility grant. Keep the unmanaged lane for a small number of real approval-flow tests. If its