Skip to content

feat(provider)!: isolate usbmuxd in a sidecar - #46

Open
Dinip wants to merge 5 commits into
mainfrom
feat/usbmuxd-sidecar
Open

feat(provider)!: isolate usbmuxd in a sidecar#46
Dinip wants to merge 5 commits into
mainfrom
feat/usbmuxd-sidecar

Conversation

@Dinip

@Dinip Dinip commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • run usbmuxd in a dedicated Linux sidecar that receives host udev hotplug events
  • share the socket directory with the provider through the usbmuxd-socket volume, so socket replacement does not leave a stale bind mount
  • keep the provider on the Compose network for Caddy while only the sidecar uses host networking
  • keep adb in the provider image and remove the in-container usbmuxd watchdog
  • disable the sidecar in the macOS overlay and continue using the host bridge there
  • add a manual multi-platform usbmuxd image workflow; normal PR, edge, and release builds still cover only the three application images

The manual workflow accepts comma-separated tags such as 1.0.0,1.0,latest and defaults to latest. It builds the sidecar once and publishes every requested tag.

Host setup

Linux hosts must mask their usbmuxd service so it does not compete with the sidecar:

sudo systemctl mask --now usbmuxd.service

After this PR merges, run usbmuxd-image.yml once and make the GHCR package public before deploying the provider profile.

Breaking changes

The provider environment variables lose the YARD_ prefix:

  • YARD_PROVIDER_TOKEN becomes PROVIDER_TOKEN
  • YARD_PROVIDER_LOG_LEVEL becomes PROVIDER_LOG_LEVEL
  • YARD_PROVIDER_CONFIG becomes PROVIDER_CONFIG

Checks

  • bun run check
  • bun run typecheck
  • bun run protocol:check
  • protocol and web tests
  • web production build
  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • Linux, macOS, and local-build Compose configuration validation
  • coordinator integration tests, which need PostgreSQL and will run in CI
  • Linux iPhone unplug and replug validation

Dinip and others added 5 commits August 25, 2026 20:56
libusbmuxd documents the unix form as `UNIX:/path`, which contains a colon —
so it was read as host:port, failed to resolve, and fell back to the default
socket, silently ignoring the one setting that had been made. Only a bare path
worked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bind-mounting the host's /var/run/usbmuxd socket is a trap in both directions.
Docker binds the socket's inode, so the mount goes stale the moment usbmuxd
exits at the last unplugged device — and if the socket is missing when the
container starts, Docker creates a *directory* at that path, after which the
host's udev-triggered usbmuxd can never bind it again. Unplugging one iPhone
took iOS down until someone cleaned up by hand.

So the container runs its own, the way it already runs its own adb server, and
the host masks its copy. usbmuxd is supervised in a restart loop rather than
fire-and-forget, since exiting at the last device is its normal behaviour.

Hotplug inside a container needs a nudge: usbmuxd polls the bus each second
only until it registers for libusb hotplug events, and that registration
succeeds in a container while delivering nothing — libusb on Debian watches
udevd's netlink broadcast, which reaches only udevd's own netns. It runs as
`usbmuxd -f -z` with a SIGUSR2 every two seconds, which means "rescan the bus":
the same device-list walk the poll it replaces was doing. `-n` is not the
answer, since disabling hotplug disables the poll along with it.

/var/lib/lockdown is a volume for the same reason /root/.android is. It holds
the SystemBUID/HostID the host pairs under and one pair record per device, so
an empty one is a new computer to every iPhone in the rack — each asking to
Trust This Computer again, by hand, at the device.

PROVIDER_START_ADB and PROVIDER_START_USBMUXD gate the two daemons, defaulting
to `auto`: start it if /dev/bus/usb is mounted, so the macOS shape still points
at the host's daemons untouched. Setting one to `no` splits a host into a
provider per platform, which does not contend for the bus — a claim is per USB
interface, and adb and usbmuxd open disjoint devices.

BREAKING CHANGE: the provider's env vars lose the YARD_ prefix, since the
entrypoint's own knobs are not the binary's config. YARD_PROVIDER_TOKEN is now
PROVIDER_TOKEN, YARD_LOG is PROVIDER_LOG_LEVEL and YARD_CONFIG is
PROVIDER_CONFIG. A provider host's .env needs renaming before the next pull;
an unset PROVIDER_TOKEN fails at startup rather than at the first request.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Unplugging an iPhone and plugging it back in took it out until the container was
restarted: usbmuxd sat retrying the address the device used to be at, logging
LIBUSB_ERROR_IO with errno=19 every rescan, and never noticed the one it came
back as.

The SIGUSR2 rescan cannot fix that, because the list it walks is libusb's, and
libusb's is frozen. It learns about a plug or an unplug from udevd's netlink
broadcast, which reaches only udevd's own network namespace — so inside the
container the list is whatever the bus held when usbmuxd started, forever. Only
a fresh process sees the bus as it now is; hotplug appeared to work before only
because usbmuxd ran with -z and exited at the last unplug, which the restart
loop then papered over.

So the watchdog watches sysfs, which is the host's and always current: every
/sys/bus/usb/devices entry with Apple's vendor ID plus its devnum, since
re-enumeration is what a replug looks like. A change restarts usbmuxd. Because
that costs every other iPhone its connection, a change has to survive one more
tick before it counts — a re-enumerating device passes through intermediate
states that would otherwise each be their own restart.

-z goes with the nudge: supervised, usbmuxd should stay up with no devices so
the socket the backend polls is always there. PROVIDER_USBMUXD_RESCAN_SECONDS is
now PROVIDER_USBMUXD_WATCH_SECONDS, which is what it now paces.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant