diff --git a/README.md b/README.md index 014c8501a..83d89a1b4 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,18 @@ # React Native WebGPU -WebGPU for React Native, powered by [Dawn](https://dawn.googlesource.com/dawn). React Native ≥ 0.81, New Architecture only. +WebGPU for React Native, powered by [Dawn](https://dawn.googlesource.com/dawn). -**Documentation:** [wcandillon.github.io/react-native-webgpu](https://wcandillon.github.io/react-native-webgpu/) +[wcandillon.github.io/react-native-webgpu](https://wcandillon.github.io/react-native-webgpu/) -```sh -npm install react-native-webgpu -``` +## Getting Started + +[Installation instructions](https://wcandillon.github.io/react-native-webgpu/docs/getting-started/installation) + +## Contributing + +For detailed information on library development, building, testing, and contributing guidelines, please see [CONTRIBUTING.md](packages/webgpu/CONTRIBUTING.md). -Expo: `npx create-expo-app@latest -e with-webgpu` -Example app: [`apps/example/`](apps/example/) ## Contributing diff --git a/packages/webgpu/CODE_OF_CONDUCT.md b/packages/webgpu/CODE_OF_CONDUCT.md deleted file mode 100644 index 45d257b29..000000000 --- a/packages/webgpu/CODE_OF_CONDUCT.md +++ /dev/null @@ -1,133 +0,0 @@ - -# Contributor Covenant Code of Conduct - -## Our Pledge - -We as members, contributors, and leaders pledge to make participation in our -community a harassment-free experience for everyone, regardless of age, body -size, visible or invisible disability, ethnicity, sex characteristics, gender -identity and expression, level of experience, education, socio-economic status, -nationality, personal appearance, race, caste, color, religion, or sexual -identity and orientation. - -We pledge to act and interact in ways that contribute to an open, welcoming, -diverse, inclusive, and healthy community. - -## Our Standards - -Examples of behavior that contributes to a positive environment for our -community include: - -* Demonstrating empathy and kindness toward other people -* Being respectful of differing opinions, viewpoints, and experiences -* Giving and gracefully accepting constructive feedback -* Accepting responsibility and apologizing to those affected by our mistakes, - and learning from the experience -* Focusing on what is best not just for us as individuals, but for the overall - community - -Examples of unacceptable behavior include: - -* The use of sexualized language or imagery, and sexual attention or advances of - any kind -* Trolling, insulting or derogatory comments, and personal or political attacks -* Public or private harassment -* Publishing others' private information, such as a physical or email address, - without their explicit permission -* Other conduct which could reasonably be considered inappropriate in a - professional setting - -## Enforcement Responsibilities - -Community leaders are responsible for clarifying and enforcing our standards of -acceptable behavior and will take appropriate and fair corrective action in -response to any behavior that they deem inappropriate, threatening, offensive, -or harmful. - -Community leaders have the right and responsibility to remove, edit, or reject -comments, commits, code, wiki edits, issues, and other contributions that are -not aligned to this Code of Conduct, and will communicate reasons for moderation -decisions when appropriate. - -## Scope - -This Code of Conduct applies within all community spaces, and also applies when -an individual is officially representing the community in public spaces. -Examples of representing our community include using an official e-mail address, -posting via an official social media account, or acting as an appointed -representative at an online or offline event. - -## Enforcement - -Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported to the community leaders responsible for enforcement at -[INSERT CONTACT METHOD]. -All complaints will be reviewed and investigated promptly and fairly. - -All community leaders are obligated to respect the privacy and security of the -reporter of any incident. - -## Enforcement Guidelines - -Community leaders will follow these Community Impact Guidelines in determining -the consequences for any action they deem in violation of this Code of Conduct: - -### 1. Correction - -**Community Impact**: Use of inappropriate language or other behavior deemed -unprofessional or unwelcome in the community. - -**Consequence**: A private, written warning from community leaders, providing -clarity around the nature of the violation and an explanation of why the -behavior was inappropriate. A public apology may be requested. - -### 2. Warning - -**Community Impact**: A violation through a single incident or series of -actions. - -**Consequence**: A warning with consequences for continued behavior. No -interaction with the people involved, including unsolicited interaction with -those enforcing the Code of Conduct, for a specified period of time. This -includes avoiding interactions in community spaces as well as external channels -like social media. Violating these terms may lead to a temporary or permanent -ban. - -### 3. Temporary Ban - -**Community Impact**: A serious violation of community standards, including -sustained inappropriate behavior. - -**Consequence**: A temporary ban from any sort of interaction or public -communication with the community for a specified period of time. No public or -private interaction with the people involved, including unsolicited interaction -with those enforcing the Code of Conduct, is allowed during this period. -Violating these terms may lead to a permanent ban. - -### 4. Permanent Ban - -**Community Impact**: Demonstrating a pattern of violation of community -standards, including sustained inappropriate behavior, harassment of an -individual, or aggression toward or disparagement of classes of individuals. - -**Consequence**: A permanent ban from any sort of public interaction within the -community. - -## Attribution - -This Code of Conduct is adapted from the [Contributor Covenant][homepage], -version 2.1, available at -[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]. - -Community Impact Guidelines were inspired by -[Mozilla's code of conduct enforcement ladder][Mozilla CoC]. - -For answers to common questions about this code of conduct, see the FAQ at -[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at -[https://www.contributor-covenant.org/translations][translations]. - -[homepage]: https://www.contributor-covenant.org -[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html -[Mozilla CoC]: https://github.com/mozilla/diversity -[FAQ]: https://www.contributor-covenant.org/faq -[translations]: https://www.contributor-covenant.org/translations diff --git a/packages/webgpu/CONTRIBUTING.md b/packages/webgpu/CONTRIBUTING.md index df5fa242d..c65930821 100644 --- a/packages/webgpu/CONTRIBUTING.md +++ b/packages/webgpu/CONTRIBUTING.md @@ -1,153 +1,52 @@ # Contributing -Contributions are always welcome, no matter how large or small! - -We want this community to be friendly and respectful to each other. Please follow it in all your interactions with the project. Before contributing, please read the [code of conduct](./CODE_OF_CONDUCT.md). - ## Development workflow -This project is a monorepo managed using [Yarn workspaces](https://yarnpkg.com/features/workspaces). It contains the following packages: - -- The library package in the root directory. -- An example app in the `example/` directory. - -To get started with the project, run `yarn` in the root directory to install the required dependencies for each package: - ```sh +git submodule update --init yarn +cd packages/webgpu && yarn install-dawn # prebuilt Dawn binaries +# or: yarn build-dawn # build Dawn from source ``` -> Since the project relies on Yarn workspaces, you cannot use [`npm`](https://github.com/npm/cli) for development. - -The [example app](/example/) demonstrates usage of the library. You need to run it to test any changes you make. - -It is configured to use the local version of the library, so any changes you make to the library's source code will be reflected in the example app. Changes to the library's JavaScript code will be reflected in the example app without a rebuild, but native code changes will require a rebuild of the example app. +Example app (from `apps/example/`): `yarn start` · `yarn ios` · `yarn android` -If you want to use Android Studio or XCode to edit the native code, you can open the `example/android` or `example/ios` directories respectively in those editors. To edit the Objective-C or Swift files, open `example/ios/WgpuExample.xcworkspace` in XCode and find the source files at `Pods > Development Pods > react-native-webgpu`. +From the repo root: `yarn lint` · `yarn tsc` · `yarn build:docs` -To edit the Java or Kotlin files, open `example/android` in Android studio and find the source files at `react-native-webgpu` under `Android`. +Tests (from `packages/webgpu/`): `yarn test:ref` (Chrome reference) · `yarn test` (E2E — open the example app on the E2E screen) -You can use various commands from the root directory to work with the project. +Other `packages/webgpu` scripts: `yarn codegen` · `yarn clean-dawn` · `yarn build-dawn` -To start the packager: +## Upgrading Dawn -```sh -yarn example start -``` +The Dawn version is pinned in two places that must stay in sync: -To run the example app on Android: +- `.gitmodules` → `submodule.externals/dawn.branch` (e.g. `chromium/7849`) +- `packages/webgpu/package.json` → the `"dawn"` field (same value, e.g. `chromium/7849`) -```sh -yarn example android -``` - -To run the example app on iOS: - -```sh -yarn example ios -``` +`yarn install-dawn` downloads **prebuilt** binaries from a GitHub release tagged `dawn-` (e.g. `dawn-chromium-7849`); the release host is configured at the top of `scripts/install-dawn.ts`. `yarn build-dawn` builds the same binaries from the submodule source instead. -By default, the example is configured to build with the old architecture. To run the example with the new architecture, you can do the following: +Steps to bump to a new Dawn version (`chromium/`): -1. For Android, run: +1. **Point the submodule at the new branch.** Update both `.gitmodules` and the `"dawn"` field in `package.json` to `chromium/`, then move the submodule to the new tip: ```sh - ORG_GRADLE_PROJECT_newArchEnabled=true yarn example android + git submodule set-branch --branch chromium/ externals/dawn + git submodule update --remote externals/dawn ``` -2. For iOS, run: +2. **Regenerate bindings.** Dawn's `src/dawn/dawn.json` drives our codegen, so re-run it after every bump: ```sh - cd example/ios - RCT_NEW_ARCH_ENABLED=1 pod install - cd - - yarn example ios + cd packages/webgpu && yarn codegen ``` -If you are building for a different architecture than your previous build, make sure to remove the build folders first. You can run the following command to cleanup all build folders: - -```sh -yarn clean -``` - -To confirm that the app is running with the new architecture, you can check the Metro logs for a message like this: - -```sh -Running "WgpuExample" with {"fabric":true,"initialProps":{"concurrentRoot":true},"rootTag":1} -``` - -Note the `"fabric":true` and `"concurrentRoot":true` properties. +3. **Publish prebuilt binaries.** Trigger the **Build Dawn** workflow (`.github/workflows/build-dawn.yml`, `workflow_dispatch`). It reads the branch from `.gitmodules`, builds Android + Apple, and creates the `dawn-chromium-` release with the headers, the Android `.so`s, and the Apple `.xcframework`. (To build locally instead, run `yarn build-dawn`; this requires the Android NDK and Xcode toolchains.) -Make sure your code passes TypeScript and ESLint. Run the following to verify: +4. **Pull the new binaries** once the release exists: -```sh -yarn typecheck -yarn lint -``` - -To fix formatting errors, run the following: - -```sh -yarn lint --fix -``` - -Remember to add tests for your change if possible. Run the unit tests by: - -```sh -yarn test -``` - -### Commit message convention - -We follow the [conventional commits specification](https://www.conventionalcommits.org/en) for our commit messages: - -- `fix`: bug fixes, e.g. fix crash due to deprecated method. -- `feat`: new features, e.g. add new method to the module. -- `refactor`: code refactor, e.g. migrate from class components to hooks. -- `docs`: changes into documentation, e.g. add usage example for the module.. -- `test`: adding or updating tests, e.g. add integration tests using detox. -- `chore`: tooling changes, e.g. change CI config. - -Our pre-commit hooks verify that your commit message matches this format when committing. - -### Linting and tests - -[ESLint](https://eslint.org/), [Prettier](https://prettier.io/), [TypeScript](https://www.typescriptlang.org/) - -We use [TypeScript](https://www.typescriptlang.org/) for type checking, [ESLint](https://eslint.org/) with [Prettier](https://prettier.io/) for linting and formatting the code, and [Jest](https://jestjs.io/) for testing. - -Our pre-commit hooks verify that the linter and tests pass when committing. - -### Publishing to npm - -We use [release-it](https://github.com/release-it/release-it) to make it easier to publish new versions. It handles common tasks like bumping version based on semver, creating tags and releases etc. - -To publish new versions, run the following: - -```sh -yarn release -``` - -### Scripts - -The `package.json` file contains various scripts for common tasks: - -- `yarn`: setup project by installing dependencies. -- `yarn typecheck`: type-check files with TypeScript. -- `yarn lint`: lint files with ESLint. -- `yarn test`: run unit tests with Jest. -- `yarn example start`: start the Metro server for the example app. -- `yarn example android`: run the example app on Android. -- `yarn example ios`: run the example app on iOS. - -### Sending a pull request - -> **Working on your first pull request?** You can learn how from this _free_ series: [How to Contribute to an Open Source Project on GitHub](https://app.egghead.io/playlists/how-to-contribute-to-an-open-source-project-on-github). - -When you're sending a pull request: + ```sh + cd packages/webgpu && yarn install-dawn + ``` -- Prefer small pull requests focused on one change. -- Verify that linters and tests are passing. -- Review the documentation to make sure it looks good. -- Follow the pull request template when opening a pull request. -- For pull requests that change the API or implementation, discuss with maintainers first by opening an issue. +5. **Verify and commit.** Build and run the example app, then commit the submodule bump together with the updated `.gitmodules`, `package.json`, and any regenerated code under `src/` and `cpp/`. \ No newline at end of file diff --git a/packages/webgpu/README.md b/packages/webgpu/README.md index 323865194..83d89a1b4 100644 --- a/packages/webgpu/README.md +++ b/packages/webgpu/README.md @@ -1,416 +1,36 @@ # React Native WebGPU -React Native implementation of WebGPU using [Dawn](https://dawn.googlesource.com/dawn). +WebGPU for React Native, powered by [Dawn](https://dawn.googlesource.com/dawn). -React Native WebGPU requires React Native 0.81 or newer. It doesn't support the legacy architecture. +[wcandillon.github.io/react-native-webgpu](https://wcandillon.github.io/react-native-webgpu/) -## Installation +## Getting Started -``` -npm install react-native-webgpu -``` - -> The package was previously published as `react-native-wgpu`. A shim with that name is still available and simply re-exports `react-native-webgpu`. - -## With Expo - -Expo provides a React Native WebGPU template that works with React Three Fiber. -This works on iOS, Android, and Web. - -``` -npx create-expo-app@latest -e with-webgpu -``` - -https://github.com/user-attachments/assets/efbd05f8-4ce0-46c2-919c-03e1095bc8ac - -Below are some examples from the [example app](/apps/example/). - -https://github.com/user-attachments/assets/116a41b2-2cf8-49f1-9f16-a5c83637c198 - -Starting from `r168`, Three.js runs out of the box with React Native WebGPU. -You need to have a slight modification of [the metro config](/apps/example/metro.config.js) to resolve Three.js to the WebGPU build. -We also support [react-three-fiber](/apps/example/src/ThreeJS/Fiber.tsx); to make it work, patch `node_modules/@react-three/fiber/package.json` (for instance via `patch-package`) so that it resolves to the WebGPU entry point instead of the React Native bundle: - -```diff -diff --git a/node_modules/@react-three/fiber/package.json b/node_modules/@react-three/fiber/package.json -@@ -- "react-native": "native/dist/react-three-fiber-native.cjs.js", -+ "react-native": "dist/react-three-fiber.cjs.js", -``` - -For model loading, we also need [the following polyfill](/apps/example/src/App.tsx#29). - -https://github.com/user-attachments/assets/5b49ef63-0a3c-4679-aeb5-e4b4dddfcc1d - -We also provide prebuilt binaries for visionOS and macOS. - -https://github.com/user-attachments/assets/2d5c618e-5b15-4cef-8558-d4ddf8c70667 - -## Usage - -Usage is identical to Web. - -```tsx -import React from "react"; -import { StyleSheet, View, PixelRatio } from "react-native"; -import { Canvas, CanvasRef } from "react-native-webgpu"; - -import { redFragWGSL, triangleVertWGSL } from "./triangle"; - -export function HelloTriangle() { - const ref = useRef(null); - useEffect(() => { - const helloTriangle = async () => { - const adapter = await navigator.gpu.requestAdapter(); - if (!adapter) { - throw new Error("No adapter"); - } - const device = await adapter.requestDevice(); - const presentationFormat = navigator.gpu.getPreferredCanvasFormat(); - - const context = ref.current!.getContext("webgpu")!; - const canvas = context.canvas as HTMLCanvasElement; - canvas.width = canvas.clientWidth * PixelRatio.get(); - canvas.height = canvas.clientHeight * PixelRatio.get(); - - if (!context) { - throw new Error("No context"); - } - - context.configure({ - device, - format: presentationFormat, - alphaMode: "opaque", - }); - - const pipeline = device.createRenderPipeline({ - layout: "auto", - vertex: { - module: device.createShaderModule({ - code: triangleVertWGSL, - }), - entryPoint: "main", - }, - fragment: { - module: device.createShaderModule({ - code: redFragWGSL, - }), - entryPoint: "main", - targets: [ - { - format: presentationFormat, - }, - ], - }, - primitive: { - topology: "triangle-list", - }, - }); - - const commandEncoder = device.createCommandEncoder(); - - const textureView = context.getCurrentTexture().createView(); - - const renderPassDescriptor: GPURenderPassDescriptor = { - colorAttachments: [ - { - view: textureView, - clearValue: [0, 0, 0, 1], - loadOp: "clear", - storeOp: "store", - }, - ], - }; - - const passEncoder = commandEncoder.beginRenderPass(renderPassDescriptor); - passEncoder.setPipeline(pipeline); - passEncoder.draw(3); - passEncoder.end(); - - device.queue.submit([commandEncoder.finish()]); - - context.present(); - }; - helloTriangle(); - }, [ref]); - - return ( - - - - ); -} - -const style = StyleSheet.create({ - container: { - flex: 1, - }, - webgpu: { - flex: 1, - }, -}); -``` - -## Example App - -To run the example app you first need to [install Dawn](#installing-dawn). - -From there you will be able to run the example app properly. - -## Similarities and Differences with the Web - -The API has been designed to be completely symmetric with the Web. -For instance, you can access the WebGPU context synchronously, as well as the canvas size. -Pixel density and canvas resizing are handled exactly like on the Web as well. - -```tsx -// The default canvas size is not scaled to the device pixel ratio -// When resizing the canvas, the clientWidth and clientHeight are updated automatically -// This behaviour is symmetric to the Web -const ctx = canvas.current.getContext("webgpu")!; -ctx.canvas.width = ctx.canvas.clientWidth * PixelRatio.get(); -ctx.canvas.height = ctx.canvas.clientHeight * PixelRatio.get(); -``` - -### Frame Scheduling - -In React Native, frame presentation is a manual operation: when you are ready to present a frame, call `present()` on the context after submitting your commands to the queue. This works the same on every runtime: the main JS runtime, the Reanimated UI runtime, and dedicated worklet runtimes (`createWorkletRuntime` / `runOnRuntime`, or a Vision Camera frame processor). `present()` runs synchronously on the calling thread, so the frame is presented from whichever thread did the rendering. - -```tsx -// draw -// submit to the queue -device.queue.submit([commandEncoder.finish()]); -// This method is React Native only -context.present(); -``` - -### Threading model - -react-native-webgpu can drive WebGPU from more than one JavaScript runtime: the main JS runtime, the Reanimated UI runtime, and dedicated worklet runtimes (`createWorkletRuntime` / `runOnRuntime`, or a Vision Camera frame processor). -This module also works well with [Bundle Mode](https://docs.swmansion.com/react-native-worklets/docs/bundleMode/) and lets you run complex Three.js scenes on the UI thread or dedicated worklet threads. - -There is a caveat with `device.lost` and `uncapturederror`: they are only delivered on the main JS runtime. This is usually fine because the GPU device is typically created on the main JS thread and then sent to the UI or a dedicated worklet thread. However, if for some reason you create the device outside the main JS thread, beware that `device.lost` and `uncapturederror` won't fire. - -### Canvas Transparency - -On Android, the `alphaMode` property is ignored when configuring the canvas. -To have a transparent canvas by default, use the `transparent` property. - -```tsx -return ( - - - -); -``` - -### External Textures - -This module provides a `createImageBitmap` function that you can use in `copyExternalImageToTexture`. - -```tsx -const url = Image.resolveAssetSource(require("./assets/image.png")).uri; -const response = await fetch(url); -const imageBitmap = await createImageBitmap(await response.blob()); +[Installation instructions](https://wcandillon.github.io/react-native-webgpu/docs/getting-started/installation) -const texture = device.createTexture({ - size: [imageBitmap.width, imageBitmap.height, 1], - format: "rgba8unorm", - usage: - GPUTextureUsage.TEXTURE_BINDING | - GPUTextureUsage.COPY_DST | - GPUTextureUsage.RENDER_ATTACHMENT, -}); -device.queue.copyExternalImageToTexture( - { source: imageBitmap }, - { texture }, - [imageBitmap.width, imageBitmap.height], -); -``` - -### Shared Texture Memory - -React Native WebGPU exposes Dawn's `SharedTextureMemory` so you can import a native pixel surface (an `IOSurface`-backed `CVPixelBuffer` on iOS, an `AHardwareBuffer` on Android) as a sampleable `GPUTexture` without copying pixels through the CPU. This is the path you want for camera frames, video frames, or anything coming out of a hardware producer. - -Like `importExternalTexture` on the web, this is **enabled by default**, there is nothing to request at device creation. The only thing to check is that the device supports it before importing. It always does on iOS/macOS; it can be missing on some Android drivers and emulators. - -```tsx -import type { NativeVideoFrame } from "react-native-wgpu"; - -const adapter = await navigator.gpu.requestAdapter(); -const device = await adapter!.requestDevice(); - -// On by default when supported; this is the only check you need. -if (!device.features.has("rnwebgpu/native-texture" as GPUFeatureName)) { - return; // rare: some Android drivers/emulators can't import native surfaces -} - -// `frame` here is a NativeVideoFrame whose .handle is the native surface -// (IOSurfaceRef / AHardwareBuffer*). NativeVideoFrames are produced by helpers -// like RNWebGPU.createVideoPlayer or RNWebGPU.createTestVideoFrame, or by -// any third-party module that hands you a compatible native pointer. -const memory = device.importSharedTextureMemory({ - handle: frame.handle, - label: "video-frame", -}); -const texture = memory.createTexture(); - -memory.beginAccess(texture, /* initialized */ true); -// ... bind `texture` into a sampler and render normally ... -memory.endAccess(texture); - -texture.destroy(); -frame.release(); -``` - -`beginAccess`/`endAccess` bracket the GPU's read window on the shared surface. Pass `initialized: true` when the producer has already written meaningful pixels (the typical video/camera case) and `false` when the next pass will fully overwrite the texture. - -### Importing External Textures - -`GPUDevice.importExternalTexture` is the higher-level path for sampling a native surface. You hand it a `NativeVideoFrame` and get back a `GPUExternalTexture` that you bind as a `texture_external` and read with `textureSampleBaseClampToEdge`. It does two things for you on top of `SharedTextureMemory`: - -- **Color conversion.** Camera and video surfaces are usually biplanar YUV (NV12), not RGB. An external texture carries the YUV→RGB matrix and the source/destination color-space transfer functions, so on the supported paths the sampler returns ready-to-use RGB in hardware. With raw `SharedTextureMemory` you would sample the luma/chroma planes and do that conversion by hand in WGSL. This is the main reason to prefer it for camera and video frames. -- **Lifecycle.** It owns the `SharedTextureMemory` + `createTexture` + `beginAccess`/`endAccess` sequence internally, so you just import the frame and `destroy()` the result. +## Contributing -It builds on the same default-on capability as Shared Texture Memory above, so feature-detect the device the same way before importing. +For detailed information on library development, building, testing, and contributing guidelines, please see [CONTRIBUTING.md](packages/webgpu/CONTRIBUTING.md). -> **Android note:** the hardware YUV→RGB conversion is fully automatic on iOS (NV12 `IOSurface`). On Android, camera frames arrive as an _opaque_ YCbCr `AHardwareBuffer`, and Dawn's Vulkan path forces an identity (`RGB_IDENTITY`) sampler conversion, so the external sample comes back as raw `[Y, Cb, Cr]`. You still get the zero-copy import and the rotation/mirror transform, but you need to apply the YUV→RGB matrix yourself in the shader. See the `CAMERA_PRELUDE` in the [VisionCamera example](/apps/example/src/VisionCamera/shaders.ts) for a ready-made BT.709 decode. -```tsx -const adapter = await navigator.gpu.requestAdapter(); -const device = await adapter!.requestDevice(); -// Feature-detect as shown above before importing on unsupported hardware. -const render = () => { - // A GPUExternalTexture expires once the queue work that used it is submitted, - // so re-import one every frame. - const externalTexture = device.importExternalTexture({ - source: frame, // a NativeVideoFrame - label: "video-frame", - }); +## Contributing - const bindGroup = device.createBindGroup({ - layout: pipeline.getBindGroupLayout(0), - entries: [ - { binding: 0, resource: externalTexture }, - { binding: 1, resource: sampler }, - ], - }); - - // ... encode a pass that samples `externalTexture`, then: - device.queue.submit([encoder.finish()]); - context.present(); - - // Release the surface's access window right after the submit that sampled it. - externalTexture.destroy(); -}; -``` - -Camera frames arrive in the sensor's native orientation, so `importExternalTexture` also accepts non-spec `rotation` (`0` | `90` | `180` | `270`, in degrees) and `mirrored` (horizontal flip) options. Dawn bakes them into the sampling transform, so the shader sees an upright frame. They map directly onto VisionCamera's `frame.orientation` / `frame.isMirrored`. - -#### Calling `destroy()` - -A `GPUExternalTexture` keeps an open access window on the underlying native surface until the wrapper is destroyed. On the Web `importExternalTexture` is core and the lifetime is handled for you; here the window is tied to the JavaScript object's lifetime. Call `externalTexture.destroy()` right after the `queue.submit()` that sampled it (never before) to release the surface back to its producer immediately. `destroy()` is idempotent, and the surface is also released when the object is garbage-collected, but relying on GC can starve a producer's buffer pool (e.g. an `AVPlayer`'s recycled `IOSurface`s) and pile up GPU resources, so prefer the explicit call in a render loop. - -### Reanimated Integration - -React Native WebGPU supports running WebGPU rendering on the UI thread using [React Native Reanimated](https://docs.swmansion.com/react-native-reanimated/) and [React Native Worklets](https://github.com/margelo/react-native-worklets). - -First, install the optional peer dependencies: +Yarn workspaces monorepo — use `yarn`, not `npm`. ```sh -npm install react-native-reanimated react-native-worklets -``` - -WebGPU objects are automatically registered for Worklets serialization when the module loads. You can pass WebGPU objects like `GPUDevice` and `GPUCanvasContext` directly to worklets. -Call `installWebGPU()` once at the top of the worklet to install flag constants like `GPUBufferUsage`, `GPUTextureUsage`, and so on. - -```tsx -import { Canvas, installWebGPU } from "react-native-webgpu"; -import { runOnUI } from "react-native-reanimated"; - -const renderFrame = (device: GPUDevice, context: GPUCanvasContext) => { - "worklet"; - installWebGPU(); - // WebGPU constants are now available on this worklet thread - const buffer = device.createBuffer({ - size, - usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ, - }); - // WebGPU rendering code runs on the UI thread - const commandEncoder = device.createCommandEncoder(); - // ... render ... - device.queue.submit([commandEncoder.finish()]); - context.present(); -}; - -// Initialize WebGPU on main thread, then run on UI thread -const device = await adapter.requestDevice(); -const context = canvasRef.current.getContext("webgpu"); -runOnUI(renderFrame)(device, context); -``` - -## Troubleshooting - -### iOS - -To run the React Native WebGPU project on the iOS simulator, you need to disable the Metal validation API. -In "Edit Scheme," uncheck "Metal Validation". Learn more [here](https://developer.apple.com/documentation/xcode/validating-your-apps-metal-api-usage/). - -Uncheck 'Metal Validation' - -## Library Development - -Make sure to check out the submodules: - -``` git submodule update --init -``` - -Make sure you have all the tools required for building the Skia libraries (Android Studio, XCode, Ninja, CMake, Android NDK/build tools). - -### Installing Dawn - -There is an alternative way which is to install the prebuilt binaries from GitHub. - -```sh -$ yarn -$ cd packages/webgpu -$ yarn install-dawn -``` - -### Building Dawn - -Alternatively, you can build Dawn locally. - -```sh yarn -cd packages/webgpu -yarn build-dawn +cd packages/webgpu && yarn install-dawn # prebuilt Dawn binaries +# or: yarn build-dawn # build Dawn from source ``` -### Upgrading - -1. `git submodule update --remote` -2. `yarn clean-dawn` -3. `yarn build-dawn` +Example app (from `apps/example/`): `yarn start` · `yarn ios` · `yarn android` -### Codegen +From the repo root: `yarn lint` · `yarn tsc` · `yarn build:docs` -* `cd packages/webgpu && yarn codegen` +Tests (from `packages/webgpu/`): `yarn test:ref` (Chrome reference) · `yarn test` (E2E — open the example app on the E2E screen) -### Testing +Other `packages/webgpu` scripts: `yarn codegen` · `yarn clean-dawn` · `yarn build-dawn` -In the `package` folder, to run the test against Chrome for reference: - -``` -yarn test:ref -``` - -To run the e2e test, open the example app on the e2e screen. -By default, it will try to connect to a localhost test server. -If you want to run the test suite on a physical device, you can modify the address [here](/apps/example/src/useClient.ts#L4). - -``` -yarn test -``` +See also [`packages/webgpu/CONTRIBUTING.md`](packages/webgpu/CONTRIBUTING.md). diff --git a/packages/webgpu/jest.config.js b/packages/webgpu/jest.config.js index 9429add35..0255acf67 100644 --- a/packages/webgpu/jest.config.js +++ b/packages/webgpu/jest.config.js @@ -1,6 +1,7 @@ module.exports = { preset: 'react-native', modulePathIgnorePatterns: [ + '/lib/', '(meshes)|(setup)|(setup.(ts|tsx))', 'components', 'config.ts',