diff --git a/development/serverless/overview.mdx b/development/serverless/overview.mdx new file mode 100644 index 000000000..86954350c --- /dev/null +++ b/development/serverless/overview.mdx @@ -0,0 +1,211 @@ +--- +title: "Serverless API" +description: "Build a versioned ComfyUI environment, deploy it as a managed endpoint, and run workflows through the API." +icon: "cloud-arrow-up" +--- + +Serverless API gives a ComfyUI workflow a managed URL and on-demand GPU capacity. The new Build and Deploy CLI keeps the build definition in your project, creates releases from that definition, and deploys a release when it is ready to serve traffic. + + + + Create a local build specification from your ComfyUI install. + + + Cut an immutable Linux/NVIDIA release from the Build. + + + Give the release a URL and managed GPU capacity. + + + Submit an API-format workflow to the active deployment. + + + +## Quick start + +Use these commands when the local install and API-format workflow are ready: + +Use the compute output to choose a valid region and GPU. Replace `` and `l4` if needed; `deploy up` records the deployment ID for the final command. + +```bash +comfy build init --name "my-comfy-build" --models-dir ./models --custom-nodes-dir ./custom_nodes + +comfy build push --release --target linux/nvidia +comfy deploy refs compute # get available regions and GPU classes +comfy deploy up --gpu l4 --region --min 1 --max 4 --watch +comfy deploy ls --workspace --status ready # get the deployment ID for the run command +comfy deploy run --workflow workflow_api.json --deployment --output-dir ./results +``` + + + + ```bash + comfy build init --name "my-comfy-build" --models-dir ./models --custom-nodes-dir ./custom_nodes + ``` + + + ```bash + comfy build push --release --target linux/nvidia + ``` + This syncs the Build and creates a release for the target. + + + If you do not already know where the selected GPU is available, check first: + + ```bash + comfy deploy refs compute --region + ``` + + Then create or reconcile the deployment: + + ```bash + comfy deploy up --gpu --region --min 1 --max 4 --watch + ``` + `deploy up` prints the new deployment ID. If you need to retrieve it later, list ready deployments: + + ```bash + comfy deploy ls --workspace --status ready + ``` + + Use the returned `dep_...` value with `--deployment`. + + + ```bash + comfy deploy run \\ + --workflow workflow_api.json \\ + --deployment \\ + --output-dir ./results + ``` + The CLI submits the API-format workflow and downloads its outputs into `./results`. + + + +## The build file + +`comfy-build.yaml` is the local source of truth for a Build. It stores the build definition and the last known remote state, so the CLI can choose the right Build automatically and warn before local changes overwrite a newer remote definition. + +Keep this file with the project. It describes the Build; it does not contain the model bytes themselves. + +## 1. Initialize a Build + +Start from a local ComfyUI install. This scans the models and custom nodes, then writes `comfy-build.yaml`. + +```bash +comfy build init --name "my-comfy-build" --models-dir ./models --custom-nodes-dir ./custom_nodes +``` + +Before pushing, check how the local spec compares with the install and the remote Build: + +```bash +comfy build status +``` + +## 2. Update and release + +After changing the local ComfyUI install, refresh the local build definition: + +```bash +comfy build update --yes +``` + +For the quick path, push the definition and release it for a target in one command: + +```bash +comfy build push --release --target linux/nvidia +``` + +When you need to create another release from an existing Build, inspect the supported targets and cut one explicitly: + +```bash +comfy build refs build-targets +comfy build release create --target linux/nvidia --watch +``` + +Follow one release's build log with: + +```bash +comfy build release logs rel_123456 --target linux/nvidia --follow +``` + +## Regions and GPU availability + +Region capacity changes, so do not copy a static list into a script. Query the platform catalog when you choose a deployment target: + +```bash +comfy deploy refs compute +``` + +Use `--region ` to filter the results. Copy a returned `region` and `gpu` pair into `comfy deploy up`: + +```bash +comfy deploy refs compute --region +``` + +The catalog is the source of truth for which GPU classes are available in each region at deployment time. + +## 3. Deploy a release + +Discover the available compute in a region, then create or reconcile a deployment for the selected release: + +```bash +comfy deploy refs compute --region US-MO-2 + +comfy deploy up \ + --gpu l4 \ + --region US-MO-2 \ + --min 1 \ + --max 4 \ + --watch +``` + +`--min` and `--max` set the worker bounds. Use `comfy deploy status --watch` to follow deployment health, release freshness, and serving activity. + +## 4. Run a workflow + +Submit an [API-format workflow](/development/api-development/workflow-api-format) to a ready deployment: + +```bash +comfy deploy run \ + --workflow workflow_api.json \ + --deployment dep_123456 \ + --output-dir ./results +``` + +The endpoint can also be called from the [Comfy SDKs](/development/api-development/sdks) by setting `COMFY_BASE_URL` to the deployment URL. + +## Operate a deployment + +```bash +# Change the worker bounds +comfy deploy scale --deployment dep_123456 --min 2 --max 5 + +# Pause or resume while retaining the deployment record +comfy deploy stop --deployment dep_123456 +comfy deploy start --deployment dep_123456 +``` + +## Inspect and clean up + +```bash +# Build state +comfy build ls +comfy build show --id bld_123456 +comfy build release ls +comfy build release show rel_123456 + +# Deployment state +comfy deploy ls --workspace --status ready +comfy deploy logs --deployment dep_123456 +comfy deploy events --deployment dep_123456 +``` + + + Deleting a deployment and deleting a Build are separate irreversible operations. Confirm the target before using `comfy deploy delete --yes` or `comfy build delete --id bld_123456 --yes`. + + +## Next steps + +- [Comfy SDKs](/development/api-development/sdks) +- [Comfy API v2 Overview](/api-reference/v2/overview) +- [Workflow API Format](/development/api-development/workflow-api-format) +- [Comfy CLI Reference](/comfy-cli/reference)