Skip to content

Add the MLX backend to the mv3 Apple demo - #260

Open
shoumikhin wants to merge 2 commits into
mainfrom
add-mlx-backend
Open

Add the MLX backend to the mv3 Apple demo#260
shoumikhin wants to merge 2 commits into
mainfrom
add-mlx-backend

Conversation

@shoumikhin

@shoumikhin shoumikhin commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Add the MLX backend to the mv3 Apple demo

Follow-up to #259, which removed the deprecated MPS backend. This adds MLX,
ExecuTorch's Apple GPU backend, as a third option in the mv3 image-classification
demo, next to Core ML and XNNPACK.

What changed:

  • Both demo Xcode projects are repinned from the old swiftpm-1.1.0 SwiftPM branch to
    the nightly swiftpm-1.5.0.20260829. No earlier published branch carries the product:
    every swiftpm-1.5.0.2026082x branch through the 28th declares zero backend_mlx,
    because the runtime halves (Add the MLX backend to the Apple frameworks and SwiftPM package pytorch/executorch#22203 and #22261) only landed after
    those nightlies ran. The 29th is the first that will have it. This will be repinned
    to the 1.5 release branch once it is cut.
  • mv3/python/export.py lowers MobileNet v3 to MLX with MLXPartitioner and writes
    mv3_mlx.pte, the same shape as the existing Core ML and XNNPACK exports. MLX
    lowering is ahead-of-time and does not import the mlx runtime package, so it runs on
    the same Linux export runner as the others.
  • The model selector gains an MLX case that loads mv3_mlx.pte.
  • The mv3 Xcode project links executorch::backend_mlx from the package and bundles
    mv3_mlx.pte, mirroring how backend_coreml and mv3_coreml_all.pte are already wired.
  • The export workflow's artifact list includes mv3_mlx.pte.

Dependency, important: the iOS build (build-demo-ios, run from ExecuTorch's
test_ios_ci.sh) stages the demo's model files with its own export scripts, which were
portable, Core ML and XNNPACK only. That gap is now filled by
pytorch/executorch#22297, which lowers the model with MLXPartitioner inline and writes
mv3_mlx.pte. Lowering needs no mlx runtime package, so it runs on the existing runner.

So this PR now waits on exactly two things: #22297 landing, and the 20260829 nightly
publishing backend_mlx. Nothing further is needed on the app side, because the metallib
resource bundle is a dependency of the backend_mlx product in the template and SwiftPM
attaches it automatically.

Test Plan:

Verified the MLX path end to end on macOS arm64 in a fresh venv, using the macOS
wheel built at the tip of the merged runtime stack:

  • Exported MobileNet v3 with MLXPartitioner. The partitioner delegated every op, and
    the resulting program carries the MLX delegate. Ran it through the pybindings
    runtime and matched eager to a max error of 1.2e-07.
  • Built a small C++ app with find_package(executorch) that links
    executorch::backend_mlx, and ran the same delegated program from C++, exit 0. otool
    confirmed it links the shipped libexecutorch_backend_mlx.dylib and libexecutorch.dylib.

Static checks on this change:

  • export.py parses. The export workflow is valid YAML and lists all four mv3 outputs.
  • ClassificationController.swift parses with swiftc and its Mode switch is exhaustive
    over the three cases.
  • Both Xcode projects were checked with a real plist parser (plutil -convert json).
    The mv3 project has 116 objects with zero ids referenced-but-undefined and zero
    defined-but-unreferenced. backend_mlx is wired into all eight places backend_coreml
    is, across the app and test targets, and mv3_mlx.pte into all six places
    mv3_coreml_all.pte is. The etLLM project change is only the one-line branch repin.
    plutil -lint passes on both.

Follow-up to removing the MPS backend. The mv3 image-classification demo now offers
MLX, ExecuTorch's Apple GPU backend, as a third option next to Core ML and XNNPACK.

Both demo Xcode projects are repinned from the old swiftpm-1.1.0 SwiftPM branch to
the nightly swiftpm-1.5.0.20260827, which is the first published package that carries
the new backend_mlx product. (This will be repinned to the 1.5 release branch once
that is cut.)

In the mv3 demo:
- export.py lowers MobileNet v3 to MLX with MLXPartitioner and writes mv3_mlx.pte,
  the same shape as the existing Core ML and XNNPACK exports. MLX lowering is
  ahead-of-time and does not import the mlx runtime package, so it runs on the same
  Linux export runner as the others.
- the model selector gains an MLX case that loads mv3_mlx.pte.
- the Xcode project links executorch::backend_mlx from the package and bundles
  mv3_mlx.pte, mirroring exactly how backend_coreml and mv3_coreml_all.pte are wired.
- the export workflow's artifact list includes mv3_mlx.pte.

Test Plan:
Verified the wheel side of MLX end to end on macOS arm64 in a fresh venv: exported
MobileNet v3 with MLXPartitioner (all ops delegated), ran it through the pybindings
runtime, and matched eager to 1.2e-07. Also built a C++ app with
find_package(executorch) that links executorch::backend_mlx and ran the same
delegated program, exit 0.

export.py parses. The export workflow is valid YAML and lists all four mv3 outputs.
ClassificationController.swift parses with swiftc and its Mode switch is exhaustive
over the three cases.

Both Xcode projects were checked with a real plist parser (plutil -convert json).
The mv3 project has 116 objects with zero ids referenced-but-undefined and zero
defined-but-unreferenced. backend_mlx is wired into all eight places backend_coreml
is (build files, framework phase, product dependency, product block for both the app
and test targets) and mv3_mlx.pte into all six places mv3_coreml_all.pte is. The
etLLM project change is only the one-line branch repin. plutil -lint passes on both.

Known dependency: build-demo-ios in executorch stages the demo's .pte files with its
own export scripts (portable, Core ML, XNNPACK). That job needs an MLX export step
added before mv3_mlx.pte is produced in CI; until then the MLX button has no model in
that build. Filed separately.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 27, 2026
The pin named swiftpm-1.5.0.20260827 and the description claimed it was the first
published package with the MLX product. It is not: no published branch through
20260828 declares backend_mlx, because the runtime halves only landed afterwards.

The MLX backend (#22203) and its SwiftPM template half (#22261) are now on main
and on the swiftpm template, so the next nightly is the first that will carry the
product. Repin both demos to it.

Nothing else is needed on the app side. The metallib resource bundle is a
dependency of the backend_mlx product in the template, so SwiftPM attaches it
automatically rather than the project having to bundle it. The demo already targets
iOS 17, which matches the package floor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants